Skip to content

Add turn-start MCP and connector availability analytics - #31474

Draft
mzeng-openai wants to merge 1 commit into
mainfrom
dev/mzeng/turn-start-mcp-availability-analytics
Draft

Add turn-start MCP and connector availability analytics#31474
mzeng-openai wants to merge 1 commit into
mainfrom
dev/mzeng/turn-start-mcp-availability-analytics

Conversation

@mzeng-openai

Copy link
Copy Markdown
Contributor

Summary

  • extend codex_turn_event with the generic MCP server names and connector identities exposed at turn start
  • derive the snapshot from the first frozen, post-policy direct/deferred MCP exposure
  • sort, deduplicate, and cap both fields while excluding schemas, descriptions, URLs, and full tool metadata
  • add analytics serialization/reducer coverage plus focused core and app-server turn tests

Why

The connector-runtime RFC calls for distinguishing inventory availability from actual usage. Turn analytics currently reports MCP tool-call counts, so a zero cannot tell us whether no MCP/connector tools were available or whether available tools simply were not called.

RFC: https://app.notion.com/p/3818e50b62b081d28e88f9c3342d4d10

Payload

codex_turn_event now includes:

  • turn_start_available_mcp_server_names
  • turn_start_available_mcp_server_names_truncated
  • turn_start_available_connectors as { connector_id, connector_name }
  • turn_start_available_connectors_truncated

Each identity list is stably ordered, deduplicated, capped at 100 items and 16 KiB of serialized JSON, and accompanied by a truncation flag. The codex_apps MCP server is represented through connector identities rather than as a generic server name.

The snapshot represents post-policy exposure: model visibility, connector accessibility, app policy, and direct/deferred search gating have been applied. It intentionally precedes final router schema validation, tool-name collision resolution, and code-mode exclusions.

Validation

  • just test -p codex-analytics — 86 passed
  • just test -p codex-core defers_apps_and_non_app_mcp_tools
  • just test -p codex-app-server --test all turn_start_tracks_thread_originator_in_analytics
  • just fix -p codex-analytics
  • just fix -p codex-core
  • just fix -p codex-app-server
  • just fmt
  • release app-server test-target compilation

Draft status

Draft pending final cleanup around cumulative byte-budget test coverage and a few small reducer/test simplifications identified in the final review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant