feat(cursor): add team usage and cost plugin

backend/plugins/cursor/README.md

@@ -0,0 +1,188 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements.  See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+# Cursor Plugin (Usage & Cost)
+
+This plugin ingests **Cursor team usage, billing, and adoption metrics** from the [Cursor Admin API](https://cursor.com/docs/account/teams/admin-api) and stores them in DevLake tool-layer tables for Grafana dashboards and SQL analysis.
+
+It follows the same structure and patterns as other DevLake AI usage plugins (notably `backend/plugins/gh-copilot`).
+
+## What it collects
+
+**Cursor Admin API endpoints:**
+
+| Endpoint | Method | Data |
+|----------|--------|------|
+| `/teams/members` | GET | Team roster |
+| `/teams/spend` | POST | Per-user billing cycle spend |
+| `/teams/filtered-usage-events` | POST | Event-level usage and charges |
+| `/teams/daily-usage-data` | POST | Per-user per-day adoption metrics |
+
+**Stored data (tool layer):**
+
+| Table | Description |
+|-------|-------------|
+| `_tool_cursor_members` | Team member roster (email, name, role) |
+| `_tool_cursor_usage_events` | Billable usage events with model, tokens, and charged amounts |
+| `_tool_cursor_user_spend` | Per-user spend for the current billing cycle (on-demand and included) |
+| `_tool_cursor_daily_usage` | Daily adoption metrics: completions, requests by feature, tab acceptance, line edits |
+
+Data is collected in the **Raw → Tool** layers only. There is no domain-layer converter in this plugin; Grafana dashboards query `_tool_cursor_*` tables directly.
+
+## Data flow
+
+```mermaid
+flowchart LR
+  API[Cursor Admin API]
+  RAW[(Raw tables\n_raw_cursor_*)]
+  TOOL[(Tool tables\n_tool_cursor_*)]
+  GRAF[Grafana Dashboards]
+
+  API --> RAW --> TOOL --> GRAF
+```
+
+**Pipeline subtasks (in order):**
+
+1. `collectMembers` → `extractMembers`
+2. `collectUsageEvents` → `extractUsageEvents`
+3. `collectUserSpend` → `extractUserSpend`
+4. `collectDailyUsage` → `extractDailyUsage`
+
+## Repository layout
+
+- `api/` — REST layer for connections, scopes, and scope configs
+- `impl/` — plugin meta, blueprint v200, connection helpers
+- `models/` — tool-layer models and migration scripts
+- `tasks/` — collectors, extractors, and pipeline registration
+- `service/` — connection test logic (Admin API permission probes)
+- `e2e/` — E2E fixtures and golden CSV assertions
+
+## Setup
+
+### Prerequisites
+
+- A Cursor **Team or Business** plan with Admin API access
+- A **Team Admin API key** from the Cursor dashboard (Dashboard → API Keys)
+- Do **not** use a User API key from Settings → Integrations — user keys cannot access `/teams/*` endpoints
+
+Authentication uses HTTP Basic auth: the API key as username with an empty password.
+
+### 1) Create a connection
+
+1. DevLake UI → **Connections → Add Connection → Cursor**
+2. Fill in:
+   - **Name**: e.g. `Cursor Production Team`
+   - **Endpoint**: defaults to `https://api.cursor.com`
+   - **Token**: Team Admin API key
+   - **Rate Limit**: defaults to 1,200 requests/hour (Cursor documents 20 requests/minute)
+3. Click **Test Connection**. DevLake probes `/teams/members`, `/teams/spend`, and `/teams/filtered-usage-events` and reports which endpoints the key can access.
+4. Save the connection.
+
+When updating an existing connection, omit the token field to keep the encrypted value already stored in DevLake.
+
+### 2) Add a scope
+
+Cursor data is team-level. Add a **Team** scope for the connection. The default scope ID is `team`.
+
+### 3) Create a blueprint
+
+Use a blueprint plan like:
+
+```json
+[
+  [
+    {
+      "plugin": "cursor",
+      "options": {
+        "connectionId": 1,
+        "scopeId": "team"
+      }
+    }
+  ]
+]
+```
+
+Run the blueprint on a daily schedule to keep usage and cost data current.
+
+### Collection behavior
+
+- **Initial backfill**: usage events and daily usage collect up to **90 days** of history on the first run.
+- **Incremental runs**: subsequent runs use the pipeline sync policy (`TimeAfter`) to collect only new data.
+- **Date range chunking**: both `/teams/daily-usage-data` and `/teams/filtered-usage-events` requests are split into **30-day** chunks (API limit for daily usage; applied to usage events for resilience).
+- **Extract**: `extractUsageEvents` and `extractDailyUsage` use **StatefulApiExtractor** (incremental by default). A config version bump triggers a one-time full re-extract after upgrade.
+- **Rate limiting**: collectors honor `Retry-After` response headers and respect the configured `rateLimitPerHour`.
+
+## Dashboards
+
+Grafana dashboard JSON lives under `grafana/dashboards/mysql/`:
+
+| Dashboard | File | UID |
+|-----------|------|-----|
+| Cursor Usage & Cost | `cursor-usage.json` | `cursor_usage` |
+| AI Cost Efficiency (Cursor panels) | `ai-cost-efficiency.json` | — |
+| Multi-AI Comparison (Cursor panels) | `multi-ai-comparison.json` | — |
+
+## Error handling
+
+| Symptom | Likely cause |
+|---------|--------------|
+| **401 Unauthorized** on test connection | Invalid API key, or a User API key instead of a Team Admin key |
+| **403 Forbidden** on spend or usage events | Key lacks permission for that Admin API endpoint |
+| **429 Too Many Requests** | Rate limit exceeded — lower `rateLimitPerHour` or wait for `Retry-After` |
+| Empty usage events after successful run | Selected time range has no billable events, or sync policy excludes the date range |
+| Reconciliation delta on dashboard | Expected when comparing event-level charges to billing-cycle spend snapshots; see dashboard notes |
+
+Tokens are sanitized before persisting. Connection test results include a `permissions` object showing which Admin API endpoints succeeded.
+
+## Not collected (Enterprise AI Code Tracking)
+
+The following endpoints from the [AI Code Tracking API](https://cursor.com/docs/account/teams/ai-code-tracking-api) are **Enterprise plan only** and are **not implemented** in this plugin (no collector, extractor, or `_tool_cursor_*` table):
+
+| Endpoint | Purpose |
+|----------|---------|
+| `GET /analytics/ai-code/commits` | Per-commit AI line attribution (TAB vs Composer vs non-AI) |
+| `GET /analytics/ai-code/changes` | Granular accepted AI changes |
+| `GET /analytics/ai-code/commits.csv` / `changes.csv` | Bulk CSV exports of the above |
+
+Team/Business Admin API keys typically receive **401** on these routes. Connection test optionally probes `GET /analytics/team/dau` (`probeEnterpriseAnalytics`) to detect Enterprise access; it does **not** ingest analytics data.
+
+**Metrics this would unlock** (shown on the Cursor native dashboard but absent from DevLake today):
+
+- AI share of **committed** code (not editor line acceptance)
+- Per-commit `commitSource`: IDE, CLI, or cloud
+- Repo name and primary-branch filters
+- TAB vs Composer line attribution on commits
+
+The **Cursor Usage** Grafana dashboard (`grafana/dashboards/mysql/cursor-usage.json`) uses Admin API proxies from `_tool_cursor_daily_usage` and `_tool_cursor_usage_events` instead; panel descriptions note where metrics are approximate.
+
+## Limitations
+
+- **Team/Business Admin API only** — Enterprise-only Analytics API endpoints (`/analytics/*`) are not collected in this plugin (see [Not collected (Enterprise AI Code Tracking)](#not-collected-enterprise-ai-code-tracking) above).
+- **Tool layer only** — no domain-layer tables; cross-plugin joins (Jira, GitHub PRs, etc.) are done in Grafana SQL or separate tooling.
+- **Team-level scope** — one scope per connection represents the whole team; per-team multi-tenant collection is not supported.
+- **Beta** — the plugin is marked beta in Config UI while the Admin API surface continues to evolve.
+
+## Testing
+
+```sh
+# Unit tests
+cd backend && go test ./plugins/cursor/...
+
+# E2E (requires E2E_DB_URL)
+make e2e-test
+```
+
+E2E fixtures live in `backend/plugins/cursor/e2e/raw_tables/` and `e2e/snapshot_tables/`.

backend/plugins/cursor/api/blueprint_v200.go

@@ -0,0 +1,81 @@
+/*
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements.  See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package api
+
+import (
+	"github.com/apache/incubator-devlake/core/errors"
+	coreModels "github.com/apache/incubator-devlake/core/models"
+	"github.com/apache/incubator-devlake/core/plugin"
+	helper "github.com/apache/incubator-devlake/helpers/pluginhelper/api"
+	"github.com/apache/incubator-devlake/helpers/srvhelper"
+	"github.com/apache/incubator-devlake/plugins/cursor/models"
+	"github.com/apache/incubator-devlake/plugins/cursor/tasks"
+)
+
+// MakeDataSourcePipelinePlanV200 generates the pipeline plan for blueprint v2.0.0.
+func MakeDataSourcePipelinePlanV200(
+	subtaskMetas []plugin.SubTaskMeta,
+	connectionId uint64,
+	bpScopes []*coreModels.BlueprintScope,
+) (coreModels.PipelinePlan, []plugin.Scope, errors.Error) {
+	_, err := dsHelper.ConnSrv.FindByPk(connectionId)
+	if err != nil {
+		return nil, nil, err
+	}
+	scopeDetails, err := dsHelper.ScopeSrv.MapScopeDetails(connectionId, bpScopes)
+	if err != nil {
+		return nil, nil, err
+	}
+
+	plan, err := makeDataSourcePipelinePlanV200(subtaskMetas, scopeDetails)
+	if err != nil {
+		return nil, nil, err
+	}
+
+	return plan, nil, nil
+}
+
+func makeDataSourcePipelinePlanV200(
+	subtaskMetas []plugin.SubTaskMeta,
+	scopeDetails []*srvhelper.ScopeDetail[models.CursorScope, models.CursorScopeConfig],
+) (coreModels.PipelinePlan, errors.Error) {
+	plan := make(coreModels.PipelinePlan, len(scopeDetails))
+	for i, scopeDetail := range scopeDetails {
+		stage := plan[i]
+		if stage == nil {
+			stage = coreModels.PipelineStage{}
+		}
+
+		scope := scopeDetail.Scope
+		task, err := helper.MakePipelinePlanTask(
+			"cursor",
+			subtaskMetas,
+			nil,
+			tasks.CursorOptions{
+				ConnectionId: scope.ConnectionId,
+				ScopeId:      scope.Id,
+			},
+		)
+		if err != nil {
+			return nil, err
+		}
+		stage = append(stage, task)
+		plan[i] = stage
+	}
+	return plan, nil
+}

backend/plugins/cursor/api/connection.go

@@ -0,0 +1,105 @@
+/*
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements.  See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package api
+
+import (
+	"strings"
+
+	"github.com/apache/incubator-devlake/core/errors"
+	"github.com/apache/incubator-devlake/core/plugin"
+	helper "github.com/apache/incubator-devlake/helpers/pluginhelper/api"
+	"github.com/apache/incubator-devlake/plugins/cursor/models"
+)
+
+// PostConnections creates a new Cursor connection.
+func PostConnections(input *plugin.ApiResourceInput) (*plugin.ApiResourceOutput, errors.Error) {
+	connection := &models.CursorConnection{}
+	if err := helper.Decode(input.Body, connection, vld); err != nil {
+		return nil, err
+	}
+
+	connection.Normalize()
+	if err := validateConnection(connection); err != nil {
+		return nil, err
+	}
+
+	if err := connectionHelper.Create(connection, input); err != nil {
+		return nil, err
+	}
+	return &plugin.ApiResourceOutput{Body: connection.Sanitize()}, nil
+}
+
+func PatchConnection(input *plugin.ApiResourceInput) (*plugin.ApiResourceOutput, errors.Error) {
+	connection := &models.CursorConnection{}
+	if err := connectionHelper.First(connection, input.Params); err != nil {
+		return nil, err
+	}
+	if err := (&models.CursorConnection{}).MergeFromRequest(connection, input.Body); err != nil {
+		return nil, errors.Convert(err)
+	}
+	connection.Normalize()
+	if err := validateConnection(connection); err != nil {
+		return nil, err
+	}
+	if err := connectionHelper.SaveWithCreateOrUpdate(connection); err != nil {
+		return nil, err
+	}
+	return &plugin.ApiResourceOutput{Body: connection.Sanitize()}, nil
+}
+
+func DeleteConnection(input *plugin.ApiResourceInput) (*plugin.ApiResourceOutput, errors.Error) {
+	conn := &models.CursorConnection{}
+	output, err := connectionHelper.Delete(conn, input)
+	if err != nil {
+		return output, err
+	}
+	output.Body = conn.Sanitize()
+	return output, nil
+}
+
+func ListConnections(input *plugin.ApiResourceInput) (*plugin.ApiResourceOutput, errors.Error) {
+	var connections []models.CursorConnection
+	if err := connectionHelper.List(&connections); err != nil {
+		return nil, err
+	}
+	for i := range connections {
+		connections[i] = connections[i].Sanitize()
+	}
+	return &plugin.ApiResourceOutput{Body: connections}, nil
+}
+
+func GetConnection(input *plugin.ApiResourceInput) (*plugin.ApiResourceOutput, errors.Error) {
+	connection := &models.CursorConnection{}
+	if err := connectionHelper.First(connection, input.Params); err != nil {
+		return nil, err
+	}
+	return &plugin.ApiResourceOutput{Body: connection.Sanitize()}, nil
+}
+
+func validateConnection(connection *models.CursorConnection) errors.Error {
+	if connection == nil {
+		return errors.BadInput.New("connection is required")
+	}
+	if strings.TrimSpace(connection.Token) == "" {
+		return errors.BadInput.New("token is required")
+	}
+	if connection.RateLimitPerHour < 0 {
+		return errors.BadInput.New("rateLimitPerHour must be non-negative")
+	}
+	return nil
+}