Skip to content

feat(chat): route by named AWS resource before keywords - #261

Merged
Atom-oh merged 4 commits into
devfrom
feat/chat-resource-routing
Nov 14, 2026
Merged

Atom-oh merged 4 commits into
devfrom
feat/chat-resource-routing

Conversation

@Atom-oh

@Atom-oh Atom-oh commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Follow-up to #260 (merged). Split out of #260 because it changes the route of every chat request.

  • A prompt that names an AWS resource ID (eni-, sg-, vpc-, vgw-, i-, …) or ARN (iam, eks/ecs, rds, elasticloadbalancing/cloudfront, cloudformation) routes to the owning section after a pin (method resource).
  • Before keyword matching, IDs are blanked and an ARN is reduced to its service name (stripResources), so sg-… is never read as the sg keyword while arn:aws:dynamodb:… still matches the data rule.
  • The resource decides only when no IAM principal is named (IAM, role, 역할, principal; unless it is an IAM ARN), at most one area's keywords match, and that match is absent, the resource's own section, a lone list/count phrasing, or security only through generic words (보안, public, 노출) — plus the object's own policy words for a network resource. Otherwise keywords, the hybrid classifier and ADR-003 fan-out decide as before.

Known limits: when several resources are named, the first-named one decides; the routing heuristics took several review rounds on #260 and should be judged against a set of real prompts rather than more patching.

Test plan

  • cd web && npm test: 381 files / 5886 tests passed; npx tsc --noEmit -p . exit 0
  • Evaluate against a golden set of real chat prompts with IDs/ARNs before merge

🤖 Generated with Claude Code

@Atom-oh
Atom-oh force-pushed the feat/chat-resource-routing branch 3 times, most recently from 2540155 to 6746865 Compare November 14, 2026 14:06
@Atom-oh
Atom-oh changed the base branch from fix/route-korean-security-group to dev November 14, 2026 14:06
@Atom-oh
Atom-oh marked this pull request as ready for review November 14, 2026 14:23
A prompt that names a resource ID or ARN routes to the section that owns
it, unless an IAM principal, a strong security word or another area's
subject is also named, or the keywords span several areas. IDs are blanked
and ARNs reduced to their service name before keyword matching. Split out
of the section-handoff change so it can be evaluated on its own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Atom-oh
Atom-oh force-pushed the feat/chat-resource-routing branch from 6746865 to 60aa7f6 Compare November 14, 2026 14:37
@github-actions

github-actions Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

🤖 AI Code Review (two independent reviewers)

Reviewer responses: codex/ALL claude/ALL

Status: PASSED — No blocking issues found

Verification is complete — both spot checks confirm the panels' claims (custom-agent/product-help precedence sits above the computed gateway in chat/route.ts:488-513, and the network gateway's describe_network covers SG/NACL/RT/Subnet/VPC only, supporting the L4 note about vpce/nat/igw/pcx IDs). All surviving findings are MINOR; here is the synthesized chair review.


Chair Review — PR #261: feat(chat): route by named AWS resource before keywords

Summary

This PR adds a deterministic resource-first routing stage (web/lib/route-resources.ts) that routes prompts naming an AWS resource ID or ARN to the owning section before keyword matching, with carefully bounded exceptions (IAM-principal intent, multi-area keywords, the ops-is-weak-evidence classifier arbitration), plus thorough colocated tests, a reference-doc section and in-place CHANGELOG amendments. Both panel reports and my own verification against BASE plus the patch found the core machinery sound — stateless regexes, correct alternation ordering, no ReDoS, no new AWS calls or mutation surface — with only MINOR issues, mostly behavioral edges and doc freshness. Nothing rises to the merge gate.

Issues per lens

L2 — Code correctness

No CRITICAL/MAJOR. Codex found no defects; claude flagged five MINORs, which I verified against the resulting code (BASE route.ts reconciled with the patch):

  • MINOR (claude only; verified) — The generic 보안 carve-out (INSTANCE_WITH_GENERIC_SECURITY) covers only instance IDs. For "sg-0123… 보안 점검" or a VPC-family ID, the stripped text matches both network and security, resourceRoute bails at matched.length > 1, and securityOnlyFromNetworkObjects is false (bare 보안 is not a policy word), so on the hybrid path the classifier may pick the IAM-only security gateway. I confirmed this traces through base securityOnlyFromNetworkObjects/SECURITY_WORDS; it is pre-existing behavior for the bare word "sg" too (not a regression), and pickGateway still answers network by rule order. Extending the exception to the network tokens would close it.
  • MINOR (claude only; verified) — Resource routing now outranks previous-section continuity: a keyword-less follow-up that names the resource (e.g. "i-0123… 는?" in a cost thread) short-circuits to the resource's section instead of staying with the previous section. Defensible as "a named resource re-routes like a keyword," but it is an untested behavior change to the documented continuity feature.
  • MINOR (claude only) — resourceRoute filters aws-data out of matched, so resource-named list/count questions never reach the aws-data receiver. Impact is nil today (steampipeAvailable() is hard-disabled per ADR-001/010 and the key is rewritten to ops anyway); noted only as coupling.
  • MINOR (claude only) — stripResources deliberately erases keywords carried by user-chosen resource names (e.g. …:topic/alarm-notify no longer hits monitoring). Correct-by-design per the PR's stated intent, but neither direction is pinned by a test.
  • MINOR (claude only; verified in diff) — NETWORK_OBJECTS hardcodes the network-resource literal instead of interpolating the exported NETWORK_RESOURCE_TOKEN; a token rename would silently decouple them (the existing "every network ID prefix" test would catch it, so low risk).

L3 — Security / AWS mutation safety

No findings — both models agree. The diff is a pure string → section key function plus text preprocessing: no AWS SDK calls, no Terraform/IAM changes, no new tool grants, no secrets, no logging of prompts. ADR-005's frozen mutation boundary is untouched — routing selects only among gateways whose tool catalogs are already read-only, so a misroute is an answer-quality issue, not a privilege boundary. I independently verified custom-agent precedence is preserved (pickCustomAgent matches the raw prompt and customPick/productHelpIntent sit above the computed gateway in web/app/api/chat/route.ts:488-513), so resource routing cannot bypass custom-agent policy or an unavailable-policy denial. The prompt is never compiled into a regex (no injection surface).

L4 — Observability / data-integration correctness

No Steampipe query, CIS check or diagnosis logic is modified (both models agree). One MINOR:

  • MINOR (claude only; verified) — NETWORK_PREFIXES includes nat|igw|pcx|vpce|cgw|vgw|tgw-attach, but the network gateway's describe_network tool covers SG/NACL/RT/Subnet/VPC only (confirmed in scripts/v2/agentcore/catalog.py:144), and query_inventory's type list omits the synced nat_gateway/internet_gateway/vpc_endpoint/vpc_peering_connection types. These IDs now short-circuit deterministically to a gateway with no tool for the object, also skipping hybrid classifier arbitration. Not a regression (BASE fell through to the equally toolless ops catch-all), but worth treating like ops-owned resources (weak evidence) or extending query_inventory.

L5 — Docs / ADR consistency

No CRITICAL/MAJOR. CHANGELOG convention is followed correctly: the existing "AI chat grounding" bullet is amended in place, in both English and Korean 1:1, with no PR/round numbers — matching the repo rule exactly.

  • MINOR (claude only; verified) — docs/architecture.md:66 (and the Korean mirror) still summarizes ADR-003 routing as "regex fast-path + Haiku classifier + cross-domain auto-synthesis"; a deterministic resource stage now precedes the regex fast-path and the summary row should name it.
  • MINOR (claude only; plausible) — The new 05-agentcore.md paragraph documents pin and custom-agent precedence but omits productHelpIntent (which also outranks resource routing on the hybrid path) and its position above previous-section continuity.
  • MINOR (codex only; partially disputed) — Codex reads 05-agentcore.md's "When several resources are named, the first-named one decides" as claiming the first resource always wins, whereas resourceRoute defers to keywords/classifier when, e.g., an RDS ARN is followed by an IAM ARN (the injected role word trips PRINCIPAL_INTENT). Claude's review found the doc accurate claim-for-claim — the sentence follows the fully stated "decides alone only when…" conditions, which already cover this case, so the first-named resource is the candidate, not the unconditional winner. I adopt this only as a wording-clarity MINOR: moving the sentence next to the conditions (or adding "subject to the conditions above") would remove the ambiguity.

Panel agreement note: both models independently cleared L2/L3/L4 of gate-level issues; every adopted finding was verified against the diff and the BASE sources rather than taken on agreement alone.

Suggestions

  1. Extend INSTANCE_WITH_GENERIC_SECURITY to the network object words (sg, eni, NETWORK_RESOURCE_TOKEN) so "sg-… 보안 점검" is deterministic on the hybrid path too.
  2. Interpolate NETWORK_RESOURCE_TOKEN into NETWORK_OBJECTS instead of duplicating the literal.
  3. Treat vpce/nat/igw/pcx/cgw/vgw/tgw-attach IDs as weak evidence (classifier-arbitrated like ops resources) or add the synced types to query_inventory, so these IDs don't deterministically land on a gateway without a matching tool.
  4. Add a test pinning the resource-vs-previous-continuity precedence, and update docs/architecture.md's ADR-003 row plus the 05-agentcore.md precedence sentence (productHelpIntent, continuity, first-named-resource qualification).

Verdict

No CRITICAL or MAJOR findings from either panel survived or were raised by chair verification; all adopted issues are MINOR polish/doc items. The image manifest lists no images, no deletions and no unavailable/omitted entries, and both panel cells declared coverage not required.

IMAGE_COVERAGE: NOT_REQUIRED


Triggered by commit cc38bd54b29a48eb1f2f02bd00e9ec831570cb50 · workflow: .github/workflows/pr-review.yml

@Atom-oh
Atom-oh merged commit 8a6d911 into dev Nov 14, 2026
6 checks passed

This branch was previously deployed

1 inactive deployment
ci-review-auto — cc38bd54 Deployed Nov 14, 2026 by Atom-oh via AI Code Review #1033
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant