Repository navigation
docs: advertise Hybrid LSP support for Ruby - #1713
Open
benrfairless wants to merge 25 commits into
Open
benrfairless wants to merge 25 commits into
benrfairless wants to merge 25 commits into
Conversation
In-process, type-aware call resolution for single Ruby files, mirroring the perl_lsp.c / php_lsp.c shape and dispatched from cbm_extract_file: - PASS 1 collects classes and modules with lexical nesting, superclasses, and include / prepend / extend mixins, plus method tables with instance and singleton methods classified from the AST. Singleton methods key on "<class_qn>.self", which cannot collide with a constant path. - PASS 1.5 infers instance-variable types from `@x = Const.new`; conflicting assignments latch to no type rather than guessing. - PASS 2 resolves constructors (`Const.new` targets the class node, the way Python's lsp_constructor does), implicit and explicit self dispatch, singleton methods, superclass chains, mixin lookup in Ruby's method-resolution order, super, and ivar / local / chained receivers. The stdlib seed covers Ruby core plus a curated Rails surface, including ActiveRecord query typing: `User.find(1)` is a User, and relation chains keep the model type. Dynamic dispatch (send, method_missing, define_method) and unresolved receivers emit no edge, and only project-defined targets are emitted. Cross-file resolution is a separate tier and is not part of this change; the 18 rows here drive the resolver through cbm_extract_file exactly as the extraction pipeline calls it. PROVENANCE: this resolver was authored clean-room from the tree-sitter-ruby grammar and Ruby language semantics. No Ruby language server source was consulted. scripts/check-lsp-originality.sh gains Shopify's ruby-lsp (MIT) as the Ruby reference so the boundary is auditable alongside the generated data it guards; `check-lsp-originality.sh --lang ruby` reports CLEAN — no verbatim string or comment overlap and no structural clones. Refs DeusData#1701 Assisted-by: OpenCode:anthropic/claude-opus-5 Signed-off-by: Ben Fairless <ben@oaf.org.au>
Adds the cross-file tier on top of the per-file resolver. cbm_run_ruby_lsp_cross builds its class table from the project-wide CBMLSPDef[], deriving each constant path by stripping the def's module QN prefix, and registers methods on both the instance and singleton receiver keys because CBMLSPDef does not carry the `def self.m` split. Ruby is registered on the fallback cbm_pxc_run_one path, like Java and Kotlin — there is no prebuilt tier-2 registry yet. Ruby also joins Rust in bypassing the own+imports module-def filter, for a different reason: Ruby constants live in one global namespace and Rails/Zeitwerk autoloads them with no require statement, so a filter keyed on requires would starve essentially all cross-file resolution in a real Ruby app. That exemption is the one deliberate design decision here. The two new rows drive cbm_run_ruby_lsp_cross with a hand-built CBMLSPDef[] in the shape pass_lsp_cross.c passes it, the second mirroring the Rails-shaped e2e fixture (User < ApplicationRecord < ActiveRecord::Base, a service object, and a caller in a third file). Refs DeusData#1701 Assisted-by: OpenCode:anthropic/claude-opus-5 Signed-off-by: Ben Fairless <ben@oaf.org.au>
Ruby joins the Hybrid LSP cohort in the README: the badge moves from 10 to 11 language families (C and C++ count as one, as they already did), Ruby is added to both prose language lists, and the capability table gains a Ruby row. Every capability the row names is backed by a row in the Ruby suite that landed in the preceding two commits — lexical nesting and `A::B` resolution, superclass chains, include / prepend / extend in Ruby's method-lookup order, the `def self.m` singleton split, `Const.new` constructor typing, self dispatch, super, instance-variable typing, ActiveRecord query typing, and the zero-edge guarantee for the dynamic-dispatch family and unresolved receivers. This lands last on purpose: the cross-file claims are only true once the cross-file tier is wired, so the documentation follows the slice that establishes the advertised level rather than preceding it. docs/index.html is deliberately untouched. Its structured data carries a pre-existing error (nine language families, Perl omitted) that predates this work; correcting it belongs in its own change rather than being folded into a feature PR. Resolves DeusData#1701 Assisted-by: OpenCode:anthropic/claude-opus-5 Signed-off-by: Ben Fairless <ben@oaf.org.au>
|
Thanks for opening this — it has been seen, and it is queued. This note is automated, but it is not a brush-off: it exists so you know where your PR stands instead of having to guess from silence. Current review status: working through a backlog. What that means for this PR, concretely:
Things that will genuinely speed it up whenever review does happen:
If this fixes a bug, a reproduction we can run is worth more than a description of the symptom. Thanks for contributing, and sorry in advance for the wait. |
Owner
|
Approved. Landing the docs last, with the slice that actually establishes the advertised support level, was the right call and the row reads honestly. I checked the count rather than trusting it. Eleven families with C and C++ as one, matching the prose list exactly: Python, TS/JS/JSX/TSX, PHP, C#, Go, C+C++, Java, Kotlin, Rust, Ruby, Perl. Ten before, eleven after. Correct in the badge and in both prose lists. And I checked your capability audit. You said you found Two things to pass onYour More importantly: nothing gates this number. I grepped That is not a change for this PR, and I am not asking you to make it. But it is worth recording alongside the Status: Thanks for the whole series. Splitting an umbrella PR into five ordered slices, measuring which ones are genuinely independent, and correcting two of your own earlier claims along the way is a lot more work than shipping the original, and the review was far better for it. |
DeusData
added
documentation
4 tasks done
The structural extractor names a class by appending the declaration's raw name text to the enclosing class QN, so `class Admin::User` yields `<module_qn>.Admin::User` while `module Admin; class User` yields `<module_qn>.Admin.User`. The resolver normalised both to the dotted form, and because the pipeline joins resolved rows to defs by exact caller QN, every row from or to a compact-declared class was silently dropped — the most common Rails style. All three passes now enter a class through one shared frame helper that computes the extractor-shaped QN from the raw name and keeps the dotted path only for constant lookup. The class table dedupes on QN. Nesting is now one raw frame per declaration joined by '.', and the constant probe strips one frame at a time: inside `class A::B` only `A.B.x` and `x` are tried, never `A.x`, matching Ruby's lexical scoping (`class A::B` does not open A). Tests pin exact QN equality on both ends of the row for compact, nested-compact, and the outer-scope-not-opened case. Assisted by OpenCode using anthropic/claude-fable-5-1. Signed-off-by: Ben Fairless <ben@oaf.org.au>
…ignment A `def` body inherited the enclosing scope, so a typed local from the surrounding body (or a preceding sibling method) typed a same-named parameter or local inside it. Parameters and block parameters were never bound, so they could not shadow an outer local either. And an assignment whose right-hand side had no proven type was ignored, so `a = Foo.new; a = x; a.bar` still resolved to Foo#bar. Ruby scoping is now modeled explicitly: `def`, `class` and `module` bodies start from a fresh, unparented scope; blocks and lambdas get a chained child scope; method, block and lambda parameters (including destructured groups and block-local names) bind to UNKNOWN; and any write the resolver cannot type — plain reassignment, multiple assignment, `||=` on an already-typed name, other compound operators, or an assignment inside a block — rebinds the nearest existing binding to UNKNOWN. `a ||= Foo.new` on a fresh name still types as Foo. Assisted by OpenCode using anthropic/claude-fable-5-1. Signed-off-by: Ben Fairless <ben@oaf.org.au>
PASS 1.5 only recorded `@x = Const.new` and ignored every other write, so `@x = compute` (or a parameter, a compound operator, a multiple assignment, or an unresolved constant's .new) left the ivar typed by whichever constructor it saw elsewhere. Any such write now latches the conflict, exactly as two different constructors already did; `nil` is neutral. `@x ||= Const.new` — the memoised-constructor idiom — counts as a constructor assignment. Assisted by OpenCode using anthropic/claude-fable-5-1. Signed-off-by: Ben Fairless <ben@oaf.org.au>
instance_eval/instance_exec, class_eval/class_exec, module_eval/ module_exec, define_method and Class.new/Module.new run their block with a different `self`, but bare and `self.` calls inside them were dispatched on the enclosing class. The eval call itself was already typed unknown; the walk into its block was not. The block of such a call is now walked with self_unknown set: bare calls, `self.` calls and `super` emit nothing there, `self` has no type, and PASS 1.5 does not attribute ivar writes inside it to the enclosing class. Receiver and argument expressions of the same call still resolve, and the suppression ends with the block. The *_exec forms join the dynamic-dispatch list. The existing negative test passed only because Meta had no `hidden`; it now has one, and the calls are written with `()` because tree-sitter-ruby parses a lone identifier as a variable reference, which the extractor never emits as a call. Assisted by OpenCode using anthropic/claude-fable-5-1. Signed-off-by: Ben Fairless <ben@oaf.org.au>
The lookup walks skipped a mixin whose module did not resolve and carried on to the class's own methods and superclass chain, and a bare call whose class ancestry did not resolve fell through to a same-named top-level def. An unseen ancestor may define or shadow the method, so none of those answers is trustworthy. Every lookup now carries a `blocked` flag set the moment the walk meets an unresolved prepend, include, extend or superclass reference, in Ruby's lookup order: an unresolved prepend blocks even an own-method hit; an unresolved include (or superclass) stops the walk before the superclass chain; an unresolved extend blocks the singleton walk. When blocked, nothing is emitted — no Object typing fallback, no top-level fallthrough, no `super` edge. Hits found before the unseen ancestor are still returned (an own method ahead of an unresolved include; a resolved include listed after an unresolved one). Assisted by OpenCode using anthropic/claude-fable-5-1. Signed-off-by: Ben Fairless <ben@oaf.org.au>
PASS 1 recursed through blocks and registered any `def` it met on the enclosing class or module, so ActiveSupport's `class_methods do … end` and `included do … end` bodies became instance methods of the concern and were then found through `include Concern` — edges to defs the structural extractor does not even emit. `include X` written inside such a block was likewise recorded on the concern instead of the includer. PASS 1 now tracks the block depth since the nearest class/module body and, inside a block, registers no methods and collects no mixins. In PASS 2 a class-body block def is skipped outright (no def exists for it to join), and a top-level block def (an RSpec group) is walked with self unknown: typed-receiver calls in its body may still resolve, bare and self. calls emit nothing, and other top-level code cannot reach it. Assisted by OpenCode using anthropic/claude-fable-5-1. Signed-off-by: Ben Fairless <ben@oaf.org.au>
The nested-modules row matched caller and callee by substring, which would have let a wrongly-spelled class path pass; it now pins the full extractor QNs on both ends with strcmp, as the compact-class rows already do. Refresh the file headers to describe the scope gates, fail-closed ancestry and self-rebinding rules the resolver now follows. Assisted by OpenCode using anthropic/claude-fable-5-1. Signed-off-by: Ben Fairless <ben@oaf.org.au>
Bring the seven review fixes from DeusData#1711 into the cross-file slice. The only conflict was tests/test_ruby_lsp.c, where both sides added tests; both are kept. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ben Fairless <ben@oaf.org.au>
main rewrote the README since this branch was cut: the Hybrid LSP badge no longer carries a language count, so only the Ruby additions are re-applied (both prose lists and the capability-table row). Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ben Fairless <ben@oaf.org.au>
The class table was a per-file array scanned linearly. Add a read-only project-wide table the resolver consults after the file's own, hashed by QN and by dotted constant path, and let method lookup search every reopening of a class declared in more than one file. The table is only populated by the builder in the next commit, so this changes nothing on its own. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ben Fairless <ben@oaf.org.au>
Every Ruby file rebuilt a class table over the whole project's definitions, so cross-file resolution cost grew with files x definitions. On a replicated fixture the complexity suite's per-file registry work rose by 3.96x when the corpus doubled, against a linear ~2. Build one sealed, read-only registry (a finalized method registry plus a hashed class table keyed by QN and by dotted constant path) before the resolve workers start, and resolve each file against it with O(1) lookups, as the Go, Python and TypeScript cross registries do. The resolver reads the shared table after the file's own declarations, so a class declared in the file still wins. Two cross-file gaps close with it: - a compact `class A::B` in another file is now found: its constant path is the dotted `A.B`, where the raw `A::B` spelling never matched; - a class reopened in another file groups with its other declarations, so a method defined in any of them resolves, and a superclass named in only one carries over. The constructor edge lands on the first. The per-file fallback entry point is kept, now a thin wrapper that builds a registry for one call. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ben Fairless <ben@oaf.org.au>
Add the Ruby complexity fixture the maintainer asked for. The suite replicates it k and 2k times and gates on the ratio of per-file registry work: ~2 is linear, ~4 is files x corpus. It measured 3.96 before the shared registry and 2.00 after. Add unit rows for a class reopened across files, a compact A::B class in another file, and one registry serving several files unchanged, and a pipeline-level row that indexes three Ruby files and checks the cross-file and reopened-class edges come from the type-aware pass. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ben Fairless <ben@oaf.org.au>
Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ben Fairless <ben@oaf.org.au>
Assisted-by: Claude Code:claude-sonnet-5-5
main reworded the README introduction again; only the Ruby additions are re-applied (the prose language list and the capability-table row). Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ben Fairless <ben@oaf.org.au>
|
You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool. What Enabling Code Scanning Means:
For more information about GitHub Code Scanning, check out the documentation. |
Owner
|
Thanks for keeping the Ruby documentation as the final slice after #1712. That dependency order still applies, and no independent rebase is requested now. Sorry for the delay on the implementation review—I’ll follow up soon. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Slice 5 of 5 splitting #1706. Stacked on #1712 — see the review note below.
Ruby joins the Hybrid LSP cohort in the README: the badge moves from 10 to 11 language families (C and C++ count as one, as they already did), Ruby is added to both prose language lists, and the capability table gains a Ruby row.
This lands last on purpose. The review asked that documentation ship with the slice that actually establishes the advertised support level, and the row's cross-file claims — ActiveRecord model typing across files, superclass chains that span files — are only true once #1712 is in.
Every claim in the row is backed by a test
While writing this I audited the row against the suite and found three capabilities named but untested:
prepend,method_missinganddefine_method. All three are genuinely implemented, but advertising untested behaviour is exactly what this split is meant to prevent, so rather than weaken the wording I added coverage to #1711:rubylsp_prepend_mixin— a prepended module shadows the class's own method, and the shadowed method is not also emitted. Both methods exist in the fixture, so a resolver that walked the class first would still emit an edge, just the wrong one.rubylsp_dynamic_dispatch_family_no_edge—public_send,__send__,define_methodandinstance_evaleach emit no edge (sendalready had its own row).What this PR deliberately does not touch
docs/index.htmlis left alone. Its structured data carries a pre-existing error — it says nine language families and omits Perl, both wrong before this work started. The Ruby edits would land on those exact strings, so adding Ruby without correcting them would mean authoring a sentence I know to be wrong, and correcting them here would mix an unrelated fix into a feature PR. It wants its own change; happy to file it.Reviewing just this slice
A PR from a fork can only target a branch in this repo, so the base is
mainand GitHub shows #1711 and #1712's commits too. Onlyaf9e6200is this slice — a 4-line README change. Merge the parents and the diff collapses to it.Where this sits in the series
CI baseline
mainat 4d7d9f1 is red for reasons unrelated to this change (#1181 × #1359 composition; fixed by #1704). Until that merges,ci-okwill be red here for that reason.Resolves #1701
Checklist
git commit -s)scripts/test.sh) — see fix(extraction): keep Ruby superclass names bare #1709 for the full-leg numbers; all failures are pre-existing on unmodifiedorigin/main.make -f Makefile.cbm lint-ci)Assisted by OpenCode using anthropic/claude-opus-5.