fix: make TextMarshaler detection independent of the toolchain environment#60
Conversation
…nment
IsTextMarshaler resolved encoding.TextMarshaler by importing the real
"encoding" package through go/importer's default importer, which locates
stdlib export data by running "go list -export" against the GOROOT the
binary was built against. When GOTOOLCHAIN selects a different toolchain
at runtime -- or the binary runs on a machine whose Go installation lives
at a different path than the build machine's -- that invocation fails with
"cannot find main module", the error is silently swallowed, and every
TextMarshaler check returns false. Types that should render as type:
string via a MarshalText method (for example UUID types such as
github.com/gofrs/uuid.UUID) then degrade to a $ref of their underlying
[16]byte array, which Swagger 2.0 forbids in path parameters, so the
generated spec fails validation.
Synthesize the interface with go/types instead: types.Implements compares
method sets structurally, so a synthesized
interface{ MarshalText() ([]byte, error) } is equivalent to the imported
one and removes the runtime dependency on a working "go list" entirely.
Signed-off-by: KT-Doan <kevin.doan@ory.sh>
c41fd4e to
e4307a9
Compare
|
Interesting issue. And a difficult one to reproduce… @KT-Doan wouldn’t the same issue occur when detecting type time.Time? If the detected behavior is correct (i.e locations of stdlib at build time and scan time differ - which I need to ascertain first) any check against the stdlib could go wrong the same way. I am wondering thus if there isn’t a more general and safer way than emulating a synthetic type by constructing it from tokens. |
|
@fredbi Yeah it probably would affect when detecting time.Time as well. It was a pain point for us causing a crazy amount of flakes when generating our SDKs in CI. The fix is in our own fork already just wanted to contribute it upstream. |
|
yeah I am trusting your judgement. I am trying to reproduce locally right now, to better understand the ramifications of the reported behavior. I can merge this week and push a patch release. Meanwhile I am trying to acquire a better understanding of how come the compiler would resolve the import at build time. Also looking for a simpler solution than building synthetic types (perhaps importing at runtime some fixed source code importing the local stdlib dependencies just for resolving the types and excluding this "fake" dep from the spec generation. Not sure yet. At any rate, I really appreciate your feedback and i am always curious about how the community is using the tool. |
|
@KT-Doan ok behavior confirmed and reproduced locally. Good catch.
|
fredbi
left a comment
There was a problem hiding this comment.
Needs a more maintainable way of producing the synthetic interface.
Needs a test to reproduce the issue in CI.
I am taking care of these follow-ups.
Good catch. Good diagnosis. Thank you for contributing this.
…face Replace the go/types builder tower (NewInterfaceType/NewFunc/NewSignatureType) introduced by #60 with mustIfaceFromSource, which type-checks a one-line `type T interface{ MarshalText() ([]byte, error) }` snippet under a nil importer. The snippet imports nothing (its result types []byte and error are Universe types), so the type-checker never consults an importer: same zero-runtime- resolution guarantee as the synthesized interface, but readable and extensible to other stdlib interfaces (json.Marshaler, fmt.Stringer) by editing a string. Guard the scope lookup so a snippet that type-checks but declares no interface T panics wrapping ErrInternal instead of nil-dereferencing, matching the sibling Must* assertions. Cover the helper contract with TestMustIfaceFromSource. Also drop a stray empty doc-comment line before `package godoclink` (gofmt). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Frederic BIDON <fredbi@yahoo.com>
Add a bespoke build-once / run-in-a-foreign-environment CI matrix (.github/workflows/toolchain-independence.yml), separate from the shared go-test workflow, that locks the property #60 fixed: IsTextMarshaler must not depend on a working build-time toolchain/GOROOT at runtime. - job A builds the resolvers + integration test binaries on `stable`. - job B (pinpoint) runs the resolvers binary under `env -i` — no Go present at all — proving the interface resolution resolves nothing at runtime. - job C (full workflow) runs the integration binary on `oldstable` with the build-time GOROOT removed and GOROOT unset, so the old go/importer path would fail while packages.Load still self-locates on PATH. TestToolchainIndependence_FullScan backs job C: a self-check that the baked runtime.GOROOT() is absent (fails loudly on masking) plus a full-scan symptom assertion that TextMarshaler fields still render as strings. It skips unless CODESCAN_TC_JOBC=1, so a normal `go test ./...` is unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Frederic BIDON <fredbi@yahoo.com>
What
Make
IsTextMarshalerindependent of the toolchain environment by synthesizing theencoding.TextMarshalerinterface withgo/types, instead of importing the realencodingpackage through
go/importer's default importer.Why
IsTextMarshalerresolvesencoding.TextMarshalerviaimporter.Default().Import("encoding").The default importer locates stdlib export data by running
go list -exportagainst the GOROOTthe binary was built against. Whenever
GOTOOLCHAINselects a different toolchain at runtime — orthe binary simply runs on a machine whose Go installation lives at a different path than the build
machine's — that invocation fails with
go: cannot find main module, the error is silentlyswallowed, and every
IsTextMarshalercheck returnsfalse.The result is a silently wrong spec. A type that should render as
type: stringvia itsMarshalTextmethod (for example UUID types such asgithub.com/gofrs/uuid.UUID) instead degradesto a
$refof its underlying[16]bytearray. Swagger 2.0 forbids$reffor path parameters, sothe generated spec fails validation. Both call sites are affected — the schema TextMarshaler
shortcut and JSON map-key handling (
IsJSONMapKey).This reproduces with any
GOTOOLCHAINvalue that differs from the binary's baked GOROOT; it is nottied to a specific Go release.
How
types.Implementsmatches method sets structurally, so a synthesizedinterface{ MarshalText() (text []byte, err error) }is equivalent to the imported one. Building itonce with
go/typesremoves the undocumented runtime dependency on a workinggo listentirely.The change is behavior-preserving wherever the old code succeeded.
Tests
Adds
TestIsTextMarshalerininternal/builders/resolvers, covering a type implementingMarshalText() ([]byte, error)(and a pointer to it), a type without the method, a wrong-signatureMarshalText(noerrorresult), and a builtin — verifying the synthesized interface matchesmethod sets precisely. The full suite, including the integration golden comparisons that exercise
TextMarshaler-backed types, passes unchanged, confirming the fix is a no-op wherever
importer.Default()previously succeeded.