|
| 1 | +# Community Dashboard |
| 2 | + |
| 3 | +GitHub metrics dashboards for the `strands-agents` organization. Collects data from GitHub, PyPI, and npm into a SQLite database, then visualizes it through pre-built Grafana dashboards. |
| 4 | + |
| 5 | +Deployable locally via Docker Compose or to AWS via CDK (Fargate + EFS + CloudFront). |
| 6 | + |
| 7 | +## Directory Structure |
| 8 | + |
| 9 | +``` |
| 10 | +community-dashboard/ |
| 11 | +├── Cargo.toml # Rust CLI project |
| 12 | +├── src/ # strands-metrics CLI source |
| 13 | +│ ├── main.rs # CLI entry point (sync, sweep, query, etc.) |
| 14 | +│ ├── client.rs # GitHub API client (octocrab) |
| 15 | +│ ├── db.rs # SQLite schema & initialization |
| 16 | +│ ├── downloads.rs # PyPI/npm download tracking |
| 17 | +│ ├── goals.rs # Goal thresholds & team management |
| 18 | +│ └── aggregates.rs # Daily metric computation |
| 19 | +├── goals.yaml # Configurable goal thresholds |
| 20 | +├── team.yaml # Team members for performance tracking |
| 21 | +├── packages.yaml # Package-to-registry mappings |
| 22 | +├── docker-compose.yaml # Quick local Grafana (read-only) |
| 23 | +├── docker/ |
| 24 | +│ ├── Dockerfile # Unified Grafana + metrics-sync image |
| 25 | +│ ├── docker-compose.local.yaml # Local dev with auto-sync |
| 26 | +│ └── entrypoint.sh # Container startup script |
| 27 | +├── provisioning/ |
| 28 | +│ ├── datasources/ |
| 29 | +│ │ └── automatic.yaml # SQLite datasource config |
| 30 | +│ └── dashboards/ |
| 31 | +│ ├── dashboards.yaml # Dashboard folder provider config |
| 32 | +│ ├── general/ # Top-level dashboards |
| 33 | +│ │ ├── executive.json # Executive Summary |
| 34 | +│ │ └── health.json # Org Health |
| 35 | +│ ├── sdks/ # SDK-specific dashboards |
| 36 | +│ │ ├── evals.json # Evaluations |
| 37 | +│ │ ├── python-sdk.json # Python SDK |
| 38 | +│ │ └── typescript-sdk.json # TypeScript SDK |
| 39 | +│ └── operations/ # Operations dashboards |
| 40 | +│ ├── team.json # Team Performance |
| 41 | +│ └── triage.json # Triage |
| 42 | +└── cdk/ # AWS CDK deployment stack |
| 43 | + ├── bin/app.ts |
| 44 | + ├── lib/community-dashboard-stack.ts |
| 45 | + ├── package.json |
| 46 | + └── cdk.json |
| 47 | +``` |
| 48 | + |
| 49 | +## Quick Start (Local) |
| 50 | + |
| 51 | +### Option A: Docker Compose with existing database |
| 52 | + |
| 53 | +If you already have a `metrics.db`, place it in this directory and run: |
| 54 | + |
| 55 | +```bash |
| 56 | +docker compose up |
| 57 | +``` |
| 58 | + |
| 59 | +Open [http://localhost:3000](http://localhost:3000). Grafana starts in anonymous viewer mode with all dashboards pre-loaded. |
| 60 | + |
| 61 | +### Option B: Docker with auto-sync |
| 62 | + |
| 63 | +Build the unified image that syncs GitHub data automatically: |
| 64 | + |
| 65 | +```bash |
| 66 | +GITHUB_TOKEN=ghp_xxx docker compose -f docker/docker-compose.local.yaml up --build |
| 67 | +``` |
| 68 | + |
| 69 | +This builds the Rust CLI, runs an initial sync on startup, and schedules daily updates at 06:00 UTC via supercronic. |
| 70 | + |
| 71 | +### Option C: Standalone CLI |
| 72 | + |
| 73 | +Build and run `strands-metrics` directly: |
| 74 | + |
| 75 | +```bash |
| 76 | +# Build |
| 77 | +cargo build --release |
| 78 | + |
| 79 | +# Sync GitHub data (PRs, issues, stars, commits, CI runs, reviews) |
| 80 | +GITHUB_TOKEN=ghp_xxx cargo run --release -- sync |
| 81 | + |
| 82 | +# Garbage collection (reconcile stale open items) |
| 83 | +GITHUB_TOKEN=ghp_xxx cargo run --release -- sweep |
| 84 | + |
| 85 | +# Sync PyPI/npm download stats |
| 86 | +cargo run --release -- sync-downloads |
| 87 | + |
| 88 | +# Load goal thresholds into the database |
| 89 | +cargo run --release -- load-goals goals.yaml |
| 90 | + |
| 91 | +# Load team members for the Team dashboard |
| 92 | +cargo run --release -- load-team team.yaml |
| 93 | + |
| 94 | +# Backfill historical downloads (PyPI: ~180 days, npm: ~365 days) |
| 95 | +cargo run --release -- backfill-downloads |
| 96 | + |
| 97 | +# Run arbitrary SQL queries |
| 98 | +cargo run --release -- query "SELECT repo, SUM(prs_merged) FROM daily_metrics GROUP BY repo" |
| 99 | +``` |
| 100 | + |
| 101 | +Then start Grafana to visualize: |
| 102 | + |
| 103 | +```bash |
| 104 | +docker compose up |
| 105 | +``` |
| 106 | + |
| 107 | +## CLI Commands |
| 108 | + |
| 109 | +| Command | Description | |
| 110 | +|---------|-------------| |
| 111 | +| `sync` | Incremental sync of GitHub data (PRs, issues, stars, commits, CI, reviews, comments) | |
| 112 | +| `sweep` | Garbage collection -- checks open items against GitHub and marks missing ones as deleted | |
| 113 | +| `query <sql>` | Run raw SQL against the metrics database | |
| 114 | +| `load-goals [path]` | Load goal thresholds from YAML into the database | |
| 115 | +| `list-goals` | Display all configured goal thresholds | |
| 116 | +| `load-team [path]` | Load team members from YAML (or `--members alice,bob`) | |
| 117 | +| `sync-downloads` | Sync recent package downloads from PyPI and npm (default: 30 days) | |
| 118 | +| `backfill-downloads` | Backfill historical download data (PyPI: ~180 days, npm: ~365 days) | |
| 119 | + |
| 120 | +### Global flags |
| 121 | + |
| 122 | +| Flag | Default | Description | |
| 123 | +|------|---------|-------------| |
| 124 | +| `--db-path` / `-d` | `metrics.db` | Path to the SQLite database file | |
| 125 | + |
| 126 | +## Dashboards |
| 127 | + |
| 128 | +### General |
| 129 | + |
| 130 | +- **Executive Summary** -- High-level org overview: total stars, open PRs/issues, stale PR count, contributor trends |
| 131 | +- **Health** -- Org health metrics with goal lines: merge time, cycle time, CI failure rate, community PR %, contributor retention, response times |
| 132 | + |
| 133 | +### SDKs |
| 134 | + |
| 135 | +- **Python SDK** -- Python SDK-specific metrics: PRs, issues, stars, downloads from PyPI |
| 136 | +- **TypeScript SDK** -- TypeScript SDK metrics with npm download tracking |
| 137 | +- **Evaluations** -- Evals framework metrics |
| 138 | + |
| 139 | +### Operations |
| 140 | + |
| 141 | +- **Team Performance** -- Per-member activity tracking: PRs opened/merged, reviews given, issues closed |
| 142 | +- **Triage** -- Open issues and PRs requiring attention, sorted by staleness |
| 143 | + |
| 144 | +## Configuration |
| 145 | + |
| 146 | +### goals.yaml |
| 147 | + |
| 148 | +Defines target thresholds that appear as goal lines on Health dashboard panels: |
| 149 | + |
| 150 | +```yaml |
| 151 | +goals: |
| 152 | + avg_merge_time_hours: |
| 153 | + value: 24 |
| 154 | + label: "Goal (24h)" |
| 155 | + direction: lower_is_better # green below, red above |
| 156 | + # warning_ratio: 0.75 # optional, default varies by direction |
| 157 | +``` |
| 158 | + |
| 159 | +Each goal requires: |
| 160 | +- `value` -- The target threshold |
| 161 | +- `label` -- Display label for the goal line |
| 162 | +- `direction` -- `lower_is_better` or `higher_is_better` |
| 163 | +- `warning_ratio` -- (optional) Multiplier for warning threshold (default: 0.75 for lower, 0.70 for higher) |
| 164 | + |
| 165 | +### team.yaml |
| 166 | + |
| 167 | +Lists team members tracked in the Team Performance dashboard: |
| 168 | + |
| 169 | +```yaml |
| 170 | +members: |
| 171 | + - username: alice |
| 172 | + - username: bob |
| 173 | +``` |
| 174 | +
|
| 175 | +### packages.yaml |
| 176 | +
|
| 177 | +Maps GitHub repos to their published packages for download tracking: |
| 178 | +
|
| 179 | +```yaml |
| 180 | +repo_mappings: |
| 181 | + sdk-python: |
| 182 | + - package: strands-agents |
| 183 | + registry: pypi |
| 184 | + sdk-typescript: |
| 185 | + - package: "@strands-agents/sdk" |
| 186 | + registry: npm |
| 187 | +``` |
| 188 | +
|
| 189 | +## AWS Deployment (CDK) |
| 190 | +
|
| 191 | +### Prerequisites |
| 192 | +
|
| 193 | +1. AWS CLI configured with appropriate credentials |
| 194 | +2. Node.js 18+ |
| 195 | +3. A GitHub PAT stored in AWS Secrets Manager: |
| 196 | + ```bash |
| 197 | + aws secretsmanager create-secret \ |
| 198 | + --name strands-grafana/github-token \ |
| 199 | + --secret-string "ghp_xxx" \ |
| 200 | + --region us-west-2 |
| 201 | + ``` |
| 202 | + |
| 203 | +### Deploy |
| 204 | + |
| 205 | +```bash |
| 206 | +cd cdk |
| 207 | +cp .env.example .env |
| 208 | +# Edit .env with your GITHUB_SECRET_ARN |
| 209 | + |
| 210 | +npm install |
| 211 | +npx cdk deploy |
| 212 | +``` |
| 213 | + |
| 214 | +### Architecture |
| 215 | + |
| 216 | +``` |
| 217 | +CloudFront (HTTPS) -> ALB (HTTP:80) -> ECS Fargate -> Grafana + strands-metrics -> EFS (metrics.db) |
| 218 | +``` |
| 219 | + |
| 220 | +- **CloudFront** for HTTPS without needing ACM + custom domain |
| 221 | +- **EFS with RETAIN policy** so metrics survive redeployments |
| 222 | +- **Fargate** (0.5 vCPU / 1 GB) with daily cron via supercronic |
| 223 | +- **Anonymous viewer-only** Grafana with `ALLOW_EMBEDDING=true` for iframes |
| 224 | + |
| 225 | +## GitHub Actions Workflow |
| 226 | + |
| 227 | +The included workflow (`.github/workflows/community-dashboard.yaml`) runs daily at 06:00 UTC: |
| 228 | + |
| 229 | +1. Syncs GitHub data (PRs, issues, stars, commits, CI, reviews) |
| 230 | +2. Runs garbage collection (sweep) |
| 231 | +3. Syncs PyPI/npm download stats |
| 232 | +4. Loads goals and team configuration |
| 233 | +5. Commits the updated `metrics.db` back to the repository |
| 234 | + |
| 235 | +Required secret: `METRICS_PAT` -- a GitHub PAT with read access to the `strands-agents` org. |
| 236 | + |
| 237 | +## Data Flow |
| 238 | + |
| 239 | +``` |
| 240 | +GitHub API (octocrab) PyPI Stats API npm Registry API |
| 241 | + | | | |
| 242 | + v v v |
| 243 | + strands-metrics CLI (Rust) |
| 244 | + | |
| 245 | + v |
| 246 | + metrics.db (SQLite) |
| 247 | + | |
| 248 | + v |
| 249 | + Grafana (SQLite datasource plugin) |
| 250 | + | |
| 251 | + v |
| 252 | + 7 pre-built dashboards in 3 folders |
| 253 | +``` |
0 commit comments