Skip to content

Add telemetry for databricks aitools install#5862

Open
parthban-db wants to merge 1 commit into
mainfrom
parthban-db/stack/telemetry
Open

Add telemetry for databricks aitools install#5862
parthban-db wants to merge 1 commit into
mainfrom
parthban-db/stack/telemetry

Conversation

@parthban-db

@parthban-db parthban-db commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

🥞 Stacked PR

Use this link to review incremental changes.


Changes

Emit a structured AitoolsInstallEvent telemetry event on every run of databricks aitools install (and its deprecated experimental aitools skills install alias, which delegates to the same command). The event records three closed-set, non-PII dimensions: the agents targeted (agents, from the fixed agent registry), the install scope (scope, project or global), and whether experimental skills were requested (experimental).

aitools install is otherwise a fully local command (skills are fetched from GitHub), so it never resolves auth on its own. tryConfigureAuth resolves auth config best-effort into the context so telemetry.Upload at command exit has somewhere to send. A user with no configured profile still installs offline and simply emits nothing.

Why

aitools install had no usage signal. This lets us measure adoption of Databricks skills across coding agents — how many installs, for which agents, at which scope, and how often the experimental skills are pulled in.

Tests

Unit tests.

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Approval status: pending

/cmd/aitools/ - needs approval

5 files changed
Suggested: @simonfaltum
Also eligible: @fjakobs, @Shridhad, @atilafassina, @keugenek, @igrekun, @pkosiec, @MarioCadenas, @pffigueiredo, @ditadi, @calvarjorge, @renaudhartert-db, @hectorcast-db, @tanmay-db, @Divyansh-db, @tejaskochar-db, @mihaimitrea-db, @chrisst, @rauchy, @lennartkats-db

/libs/telemetry/ - needs approval

Files: libs/telemetry/protos/aitools_install.go, libs/telemetry/protos/frontend_log.go
Suggested: @simonfaltum
Also eligible: @renaudhartert-db, @hectorcast-db, @tanmay-db, @Divyansh-db, @tejaskochar-db, @mihaimitrea-db, @chrisst, @rauchy

Any maintainer (@andrewnester, @anton-107, @denik, @pietern, @shreyas-goenka, @simonfaltum, @renaudhartert-db, @janniklasrose, @lennartkats-db) can approve all areas.
See OWNERS for ownership rules.

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 733659b

Run: 29250115548

Env 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
💚​ aws linux 4 4 230 1081 4:54
💚​ aws windows 4 4 232 1079 6:59
💚​ aws-ucws linux 4 4 316 998 6:41
💚​ aws-ucws windows 4 4 318 996 7:30
💚​ azure linux 4 4 230 1080 4:58
💚​ azure windows 4 4 232 1078 6:54
💚​ azure-ucws linux 4 4 318 995 6:42
💚​ azure-ucws windows 4 4 320 993 7:27
💚​ gcp linux 4 4 229 1082 4:59
💚​ gcp windows 4 4 231 1080 6:53
8 interesting tests: 4 RECOVERED, 4 SKIP
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🙈​ TestAccept/bundle/invariant/no_drift 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/ssh/connection 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestFetchRepositoryInfoAPI_FromRepo 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestFetchRepositoryInfoAPI_FromRepo/root 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestFetchRepositoryInfoAPI_FromRepo/subdir 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 10 slowest tests (at least 2 minutes):
duration env testname
6:18 azure-ucws windows TestAccept
6:10 aws-ucws windows TestAccept
6:06 aws windows TestAccept
6:02 gcp windows TestAccept
6:00 azure windows TestAccept
2:55 aws linux TestAccept
2:54 azure linux TestAccept
2:52 gcp linux TestAccept
2:51 aws-ucws linux TestAccept
2:46 azure-ucws linux TestAccept

@parthban-db parthban-db force-pushed the parthban-db/stack/telemetry branch from cd71e38 to f2bf35b Compare July 8, 2026 14:08
@parthban-db parthban-db temporarily deployed to test-trigger-is July 8, 2026 14:08 — with GitHub Actions Inactive
@parthban-db parthban-db temporarily deployed to test-trigger-is July 8, 2026 14:08 — with GitHub Actions Inactive
@parthban-db parthban-db force-pushed the parthban-db/stack/telemetry branch from f2bf35b to b477ac6 Compare July 8, 2026 14:11
@parthban-db parthban-db temporarily deployed to test-trigger-is July 8, 2026 14:11 — with GitHub Actions Inactive
@parthban-db parthban-db temporarily deployed to test-trigger-is July 8, 2026 14:11 — with GitHub Actions Inactive
@parthban-db parthban-db changed the title Add Aitools install telemetry Add telemetry for databricks aitools install Jul 8, 2026
@@ -13,7 +14,7 @@ func TestLegacySkillsInstallDelegatesToInstall(t *testing.T) {
setupTestAgents(t)
calls := setupInstallMock(t)

ctx := cmdio.MockDiscard(t.Context())
ctx := telemetry.WithNewLogger(cmdio.MockDiscard(t.Context()))

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need the logger for the telemetry because otherwise, it will panic. A logger is always present in the production path through the root command.

@parthban-db parthban-db force-pushed the parthban-db/stack/telemetry branch from b477ac6 to 9b38779 Compare July 13, 2026 11:40
@parthban-db parthban-db force-pushed the parthban-db/stack/telemetry branch from 9b38779 to 733659b Compare July 13, 2026 12:28
@parthban-db parthban-db requested review from mihaimitrea-db and renaudhartert-db and removed request for renaudhartert-db July 13, 2026 12:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants