Fix P2 HTML comment boundary matching - #452
Conversation
Signed-off-by: Patrick Erichsen <patrick.a.erichsen@gmail.com>
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Re-reviewed at 1e82b2b after the current-main merge. The patch fixes the reported ordinary-substring and standard cross-comment false positives, and all 154 focused static-pattern tests pass, but two boundary cases still block approval. The alphanumeric boundary drops common camel/Pascal-style hidden directives, producing an end-to-end SAFE, complete verdict, and the tempered body does not recognize --!> as an HTML comment closer, so it can still span from a comment into visible text. Please preserve case-transition directive detection, honor browser-compatible comment endings, and add direct plus end-to-end regressions.
Signed-off-by: Patrick Erichsen <patrick.a.erichsen@gmail.com>
| rf"<!--(?:(?!--!?>).)*?{_P2_IDENTIFIER_START}" | ||
| r"(?:system|instructions?|ignore|POST|GET|send|transmit)" | ||
| rf"{_P2_IDENTIFIER_END}" | ||
| r"(?:(?!--!?>).)*?--!?>", |
There was a problem hiding this comment.
[P2] Handle abrupt empty-comment endings before scanning visible text. The HTML tokenizer emits an empty comment and returns to data state when the first character after <!-- is >, or when it is -> (the comment-start and comment-start-dash states). This matcher recognizes only --> and --!>, so both <!--> visible system docs <!-- safe --> and <!---> visible system docs <!-- safe --> are still returned as one P2 match on this head. Treat these opener-adjacent endings as boundaries and add direct plus graph negative regressions so the original cross-boundary false-positive class is fully closed.
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Re-review: the requested camel/Pascal/acronym boundaries and --!> termination are fixed, both prior threads are resolved, all checks pass, and the current-main merge validates cleanly. One HTML-tokenizer edge still leaves the original cross-boundary false-positive class open: abrupt empty comments <!--> and <!---> return to data state, but P2 scans through their following visible text until a later -->. Please treat those two forms as comment boundaries and add direct plus end-to-end regressions.
User-visible bug
SkillSpector could start a P2 "hidden instructions" match in one HTML comment, include ordinary visible documentation after that comment had already closed, and stop at a later comment. The resulting finding falsely presented visible text as hidden prompt injection and could change an otherwise safe scan verdict.
What this changes
-->and browser-compatible--!>closerstarget,SYSTEMATIC,POSTGRES, andGETTINGSAFEand real hidden instructions cannot receive aSAFEverdictReviewer follow-up
ignorePreviousInstructions,pleaseIgnorePreviousInstructions,POSTRequestToServer, andSYSTEMPromptare covered--!>terminates the comment, so following visible text cannot be consumed by the hidden-instruction matchFixes #297.
Validation
uv run pytest -m 'not integration and not provider' tests/(3,174 passed, 14 skipped, 38 deselected, 4 xfailed)uv run pytest tests/nodes/analyzers/test_static_patterns.py tests/nodes/test_security_remediation.py -q(237 passed)uv run ruff check src/ tests/uv run ruff format --check src/ tests/release-validation@0.1.3; P2 findings dropped from 2 to 0