Draft
Conversation
… stats bugs Group A of SchemaAnalyzer refactor: - Fix A1: array element stats overwrite bug (isNewTypeEntry) - Fix A2: probability >100% for array-embedded objects (x-documentsInspected) - Rename folder: src/utils/json/mongo/ → src/utils/json/data-api/ - Rename enum: MongoBSONTypes → BSONTypes - Rename file: MongoValueFormatters → ValueFormatters - Add 9 new tests for array stats and probability
Group B of SchemaAnalyzer refactor: - B1: SchemaAnalyzer class with addDocument(), getSchema(), reset(), getDocumentCount() - B2: clone() method using structuredClone for schema branching - B3: addDocuments() batch convenience method - B4: static fromDocument()/fromDocuments() factories (replaces getSchemaFromDocument) - B5: Migrate ClusterSession to use SchemaAnalyzer instance - B6-B7: Remove old free functions (updateSchemaWithDocument, getSchemaFromDocument) - Keep getPropertyNamesAtLevel, getSchemaAtPath, buildFullPaths as standalone exports
…x properties type Group C of SchemaAnalyzer refactor: - C1: Add typed x-minValue, x-maxValue, x-minLength, x-maxLength, x-minDate, x-maxDate, x-trueCount, x-falseCount, x-minItems, x-maxItems, x-minProperties, x-maxProperties to JSONSchema interface - C2: Fix properties type: properties?: JSONSchema → properties?: JSONSchemaMap - C3: Fix downstream type errors in SchemaAnalyzer.test.ts (JSONSchemaRef casts)
…temBsonType Group D of SchemaAnalyzer refactor: - D1: Add bsonType to FieldEntry (dominant BSON type from x-bsonType) - D2: Add bsonTypes[] for polymorphic fields (2+ distinct types) - D3: Add isOptional flag (x-occurrence < parent x-documentsInspected) - D4: Add arrayItemBsonType for array fields (dominant element BSON type) - D5: Sort results: _id first, then alphabetical by path - D6: Verified generateMongoFindJsonSchema still works (additive changes) - G4: Add 7 getKnownFields tests covering all new fields
… toFieldCompletionItems) Group E of SchemaAnalyzer refactor: - E1: generateDescriptions() — post-processor adding human-readable description strings with type info, occurrence percentage, and min/max stats - E2: toTypeScriptDefinition() — generates TypeScript interface strings from JSONSchema for shell addExtraLib() integration - E3: toFieldCompletionItems() — converts FieldEntry[] to CompletionItemProvider- ready FieldCompletionData[] with insert text escaping and $ references Also: - Rename isOptional → isSparse in FieldEntry and FieldCompletionData (all fields are implicitly optional in MongoDB API / DocumentDB API; isSparse is a statistical observation, not a constraint) - Fix lint errors (inline type specifiers) - 18 new tests for transformers + updated existing tests
- Add 5 tests for clone(), reset(), fromDocument(), fromDocuments(), addDocuments() - Mark all checklist items A-G as complete, F1-F2 as deferred - Add Manual Test Plan section (§14) with 5 end-to-end test scenarios - Document clone() limitation with BSON Binary types (structuredClone)
- Add monotonic version counter to SchemaAnalyzer (incremented on mutations) - Cache getKnownFields() with version-based staleness check - Add ClusterSession.getKnownFields() accessor (delegates to cached analyzer) - Wire collectionViewRouter to use session.getKnownFields() instead of standalone function - Add ext.outputChannel.trace for schema accumulation and reset events
Co-authored-by: tnaum-ms <171359267+tnaum-ms@users.noreply.github.com>
…ypeScript definitions and completion items
Move SchemaAnalyzer, JSONSchema types, BSONTypes, ValueFormatters, and getKnownFields into packages/schema-analyzer as @vscode-documentdb/schema-analyzer. - Set up npm workspaces (packages/*) and TS project references - Update all extension-side imports to use the new package - Configure Jest multi-project for both extension and package tests - Remove @vscode/l10n dependency from core (replaced with plain Error) - Fix strict-mode type issues (localeCompare bug, index signatures) - Update .gitignore to include root packages/ directory - Add packages/ to prettier glob
…itions The bsonToTypeScriptMap emits non-built-in type names (ObjectId, Binary, Timestamp, etc.) without corresponding import statements or declare stubs. Currently harmless since the output is for display/hover only, but should be addressed if the TS definition is ever consumed by a real TS language service. Addresses PR #506 review comment from copilot.
…ion names - Prefix with _ when PascalCase result starts with a digit (e.g. '123abc' → '_123abcDocument') - Fall back to 'CollectionDocument' when name is empty or only separators - Filter empty segments from split result - Add tests for edge cases Addresses PR #506 review comment from copilot.
Add comment explaining why the cast to JSONSchema is safe: our SchemaAnalyzer never produces boolean schema refs. Notes that a typeof guard should be added if the function is ever reused with externally-sourced schemas. Addresses PR #506 review comment from copilot.
…lashes - Replace SPECIAL_CHARS_PATTERN with JS_IDENTIFIER_PATTERN for proper identifier validity check (catches dashes, brackets, digits, quotes, etc.) - Escape embedded double quotes and backslashes when quoting insertText - Add tests for all edge cases (dashes, brackets, digits, quotes, backslashes) - Mark future-work item #1 as resolved; item #2 (referenceText/$getField) remains open for aggregation completion provider phase Addresses PR #506 review comment from copilot.
…PI alongside MongoDB API
… and completions - Introduced new test files for operator reference verification, parsing, and structural invariants. - Implemented tests for `getFilteredCompletions` and `docLinks` to ensure correct URL generation. - Enhanced Jest configuration to limit workers and updated test match patterns. - Added `parseOperatorReference` utility to structure operator data from markdown dumps. - Verified that all operators in the implementation match the documented reference and that no unsupported operators are present. - Ensured all entries have required fields, valid meta tags, and non-empty descriptions.
- Replaced dynamic documentation links with direct URLs for operators in accumulators, expression operators, query operators, update operators, window operators, and stages. - Updated metadata references in docLinks to reflect accurate categories for boolean and comparison expressions. - Enhanced test coverage for operator reference verification, ensuring merged dump and overrides are consistent with implementation. - Removed unused getDocLink imports from system variables.
- Updated the paths for scraped operator data and overrides in the generate-from-reference script. - Introduced snippet templates for operators, enhancing the generation process. - Adjusted the parsing logic to accommodate new snippet structures. - Modified the output files to reflect the new resource paths. - Updated tests and documentation to align with the new file structure.
…stants - Reordered import statements for consistency across files in `stages.ts`, `systemVariables.ts`, `updateOperators.ts`, and `windowOperators.ts`. - Consolidated multiline descriptions into single lines for clarity and consistency. - Updated registration of operators to use spread syntax for improved readability. - Ensured all links are marked as inferred from another category for better documentation.
…istency - Updated descriptions in query operators to ensure consistent formatting and clarity. - Reorganized imports in query operators, stages, system variables, update operators, and window operators for better structure. - Enhanced documentation links for various operators to provide clearer guidance. - Removed unnecessary whitespace and ensured consistent snippet formatting across all operator files.
…hatch
MetaTag is now (typeof ALL_META_TAGS)[number] | (string & {}), giving
IntelliSense for known tags while preserving extensibility for runtime-
injected tags like field:identifier.
…root Aligns with Approach A (monorepo delegation): dev tooling (jest, ts-jest, prettier, ts-node, @types/jest) is provided by the root package.json. Eliminates the Jest 29 vs 30 version mismatch. Added root-install note to README.
- Root prettier check now covers packages/ (was missing, only prettier-fix had it) - schema-analyzer jest.config.js: add maxWorkers '50%' to match documentdb-constants - schema-analyzer README: add monorepo root-install note (matching documentdb-constants) - No .nvmrc or prettier configs needed per-package (inherited from root)
…icit loadOperators() factory
…dir mappings in getDocLink
…dir mappings, update generator template
…lbacks in applyOverrides
…ert generator inferred-link override (scraper ground truth takes precedence)
… to prevent registry mutation
…eld with planned values and usage context
…4s are tolerated, connectivity errors are not
…returns a safe copy via Array.filter
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Resolves CodeQL 'Incomplete string escaping or encoding' alert by escaping backslash characters before escaping pipe characters.
tnaum-ms
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Shell Integration — DocumentDB Query Language & Autocomplete
Umbrella PR for the shell integration feature: a custom
documentdb-queryMonaco language with intelligent autocomplete, hover docs, and validation across all query editor surfaces (filter, project, sort, aggregation, shell).Work is organized as incremental steps, each delivered via a dedicated sub-PR merged into
feature/shell-integration.Progress
SchemaAnalyzer(JSON Schema output, incremental merge, 24 BSON types)@vscode-documentdb/schema-analyzerpackage, enrichedFieldEntrywith BSON types, added schema transformers, introduced monorepo structuredocumentdb-constantsPackage · feat: add documentdb-constants package — operator metadata for autocomplete #513 — 308 operator entries (DocumentDB API query operators, update operators, stages, accumulators, BSON constructors, system variables) as static metadata for autocompletedocumentdb-querycustom language with JS Monarch tokenizer (no TS worker), validated via POC across 8 test criteriaCompletionItemProvider· In progress · [WIP] documentdb-query language #518 —documentdb-querylanguage registration, per-editor model URIs, completion data store,CompletionItemProvider(filter/project/sort),HoverProvider,acornvalidation,$-prefix fix, remove legacy JSON Schema pipelineCompletionItemProviderCompletionItemProviderKey Architecture Decisions
documentdb-querycustom language — JS Monarch tokenizer, no TS worker (~400-600 KB saved)CompletionItemProvider+ URI routing (documentdb://{editorType}/{sessionId})documentdb-constantsbundled at build time; field data pushed via tRPC subscriptionacorn.parseExpressionAt()for syntax errors;acorn-walk+documentdb-constantsfor identifier validationlanguage="json"with JSON Schema validationlanguage="javascript"with full TS service +.d.tsviaaddExtraLib()Sub-PRs