diff --git a/.docs/handover-2026-05-07-issue-446-probe-fix.md b/.docs/handover-2026-05-07-issue-446-probe-fix.md new file mode 100644 index 000000000..fad450b13 --- /dev/null +++ b/.docs/handover-2026-05-07-issue-446-probe-fix.md @@ -0,0 +1,55 @@ +# Session Handover: Issue #446 Anthropic Provider Health Check Fix + +**Date**: 2026-05-07 +**Branch**: `task/446-anthropic-probe-circuit-breaker-fix` +**PR**: https://github.com/terraphim/terraphim-ai/pull/868 +**Gitea issue**: terraphim/terraphim-ai#446 + +--- + +## What was done + +Investigated and fixed Gitea issue #446: "[ADF] Anthropic provider health check failing (claude-opus, -sonnet, -haiku)". + +### Investigation findings + +**April 6 root cause** (historical, not code-fixable): +- All known CLI versions (2.1.100+) were installed after April 6 — an older, now-removed version was in use +- `~/.config/claude/settings.json` contains `"apiUrl": "http://127.0.0.1:3456/v1"` (non-running proxy); older versions may have respected this strictly, causing connection-refused failures on every probe +- Current state is healthy: all three Anthropic models probe successfully + +**Taxonomy routing** (confirmed correct): +- `planning_tier.md`: `route:: anthropic, opus` +- `implementation_tier.md`: `route:: anthropic, sonnet` +- `review_tier.md`: `route:: anthropic, haiku` +- Bare names pass the C1 gate (`CLAUDE_CLI_BARE_MODELS`) + +**Code bug found and fixed**: +- `probe_all` in `provider_probe.rs` only exempted "CLI tool not found on PATH" from circuit-breaker updates +- C1-blocked probes (`"probe skipped: provider not in C1 allow-list"`) and no-action-template probes were incorrectly calling `breaker.record_failure()` +- This would cause spurious circuit-breaker trips for any route with a banned/unknown provider prefix (not Anthropic, which passes C1, but general correctness issue) + +### Change made + +`crates/terraphim_orchestrator/src/provider_probe.rs`: +- Extracted `is_environment_error(error: &str) -> bool` helper covering all three local-setup error kinds +- Updated `probe_all` circuit-breaker update block to use the helper +- Added unit tests: `is_environment_error_classifications`, `c1_blocked_probe_does_not_open_breaker`, `missing_cli_probe_does_not_open_breaker` (refactored from inline match) + +13/13 provider_probe tests pass. Clippy clean. Formatted. + +--- + +## What is pending + +- PR #868 awaits review and merge +- After merge: close Gitea issue #446 (`tea issues close 446 --repo terraphim/terraphim-ai`) +- Optional: remove or update `~/.config/claude/settings.json` proxy config if it causes future probe failures with older CLI versions + +--- + +## Key environment notes + +- KG pre_tool_use hook rewrites `claude` in Bash commands — use `/tmp` scripts with symlink target path when testing CLI directly +- `gtr` requires `source ~/.profile` first; a pre-set stale `GITEA_TOKEN` env will cause 404 with swagger URL +- GitHub is the push remote; Gitea at `git.terraphim.cloud` is issue tracking only — PRs go to GitHub via `gh` diff --git a/CHANGELOG.md b/CHANGELOG.md index f4d97b266..ca6448cd6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -60,6 +60,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - **Test ranking knowledge graph fixture** added for agent testing - **LLM cost tracking** foundation with genai fork integration (Refs #1075) - **Spec validation** report for 2026-04-29 documenting 3 fixed, 5 remaining gaps +- **Spec validation** report for 2026-05-07 -- FAIL: 2 persistent gaps (single-agent-listener operational gap, meta-coordinator spec coverage gap); 0 new gaps; provider_probe.rs hardening assessed as in-scope for #1233 +- **Documentation gap report v4** generated for 2026-05-07 -- full workspace scan (55 crates, ~2,450 missing items); critical crates: agent (371), orchestrator (236), validation (116), types (148), multi_agent (191); 8 crates at 100% gap rate; `reports/doc-gap-report-20260507-v4.md` +- **Documentation gap report v2** generated for 2026-05-07 extending scan to include `pub fn` -- 395 total gaps (agent: 189, orchestrator: 61, types: 48, service: 39, automata: 23, persistence: 12, config: 11, middleware: 8, rolegraph: 4); service and persistence API entry points identified as critical +- **Documentation gap report** generated for 2026-05-07 identifying 307 missing docs across 9 crates (agent: 139, types: 76, orchestrator: 54, automata: 23, config: 15) -- 45% reduction from 2026-04-29 baseline of 564 - **Documentation gap report** generated for 2026-05-05 identifying 1,058 missing docs across 12 crates (orchestrator: 445, server: 138, service: 114, agent: 99, types: 98) - **GITEA_URL injection** from project config into agent spawn context for orchestrator - **Streaming output log drain** for reliable agent output capture (Refs #1219) @@ -72,6 +76,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed +- **Circuit-breaker environment error exemption** -- `is_environment_error()` extracted from inline logic in `provider_probe.rs`; now exempts C1 allow-list failures and missing routing templates in addition to CLI-not-found errors (Refs #446) - **Supervisor escalation** -- added `escalated` state field to enforce `NoRestartAfterEscalation` invariant (Refs #255) - **Test dependencies** -- added `server` feature to `terraphim_agent` dev-dependency for CI stability (Refs #1260) - **`--server` flag** on listen subcommand now routes through custom error handler diff --git a/crates/terraphim_orchestrator/src/provider_probe.rs b/crates/terraphim_orchestrator/src/provider_probe.rs index 2261b65c2..54358bee3 100644 --- a/crates/terraphim_orchestrator/src/provider_probe.rs +++ b/crates/terraphim_orchestrator/src/provider_probe.rs @@ -142,16 +142,15 @@ impl ProviderHealthMap { match result.status { ProbeStatus::Success => breaker.record_success(), ProbeStatus::Error => { - // Do not count CLI-not-found as a provider failure. - if result - .error - .as_ref() - .is_some_and(|e| e.contains("CLI tool") && e.contains("not found on PATH")) - { + // Do not count environment/configuration errors as provider + // failures — they do not indicate that the API is down. + let err = result.error.as_deref().unwrap_or(""); + if is_environment_error(err) { debug!( provider = %result.provider, model = %result.model, - "skipping circuit-breaker update: CLI tool missing" + error = %err, + "skipping circuit-breaker update: environment/config error" ); } else { breaker.record_failure(); @@ -395,6 +394,21 @@ impl ProviderHealthMap { } } +/// Determine whether a probe error represents an environment/configuration +/// issue rather than a genuine provider health failure. +/// +/// These errors must not update the circuit breaker because they reflect +/// local setup problems (missing tool, routing config, C1 allow-list) — +/// not transient API unavailability. +fn is_environment_error(error: &str) -> bool { + // CLI tool absent from PATH: local environment issue. + (error.contains("CLI tool") && error.contains("not found on PATH")) + // Provider not in C1 subscription allow-list: configuration issue. + || error.contains("not in C1 allow-list") + // No action:: template defined in routing rules: routing config issue. + || error.contains("no action:: template defined") +} + /// Check whether a CLI tool is available on PATH. /// /// Runs `which ` and returns `true` only when the tool exists and is @@ -764,20 +778,33 @@ mod tests { assert!(!cli_tool_on_path("definitely_not_a_real_tool_12345")); } + #[test] + fn is_environment_error_classifications() { + assert!(is_environment_error( + "probe skipped: CLI tool 'opencode' not found on PATH" + )); + assert!(is_environment_error( + "probe skipped: provider not in C1 allow-list" + )); + assert!(is_environment_error("no action:: template defined")); + assert!(!is_environment_error("exit 1: authentication failed")); + assert!(!is_environment_error("exit 1: connection timeout")); + } + #[test] fn missing_cli_probe_does_not_open_breaker() { let mut map = ProviderHealthMap::new(Duration::from_secs(300)); - // Simulate the exact update logic from `probe_all` with a CLI-not-found - // result. We create the breaker entry and then run the same match logic - // that `probe_all` uses. + // Simulate a CLI-not-found error result and apply the same exemption + // logic used in probe_all via is_environment_error. + let error_msg = "probe skipped: CLI tool 'nonexistent-cli' not found on PATH".to_string(); let result = ProbeResult { provider: "openai".to_string(), model: "gpt-4".to_string(), cli_tool: "nonexistent-cli".to_string(), status: ProbeStatus::Error, latency_ms: None, - error: Some("probe skipped: CLI tool 'nonexistent-cli' not found on PATH".to_string()), + error: Some(error_msg.clone()), timestamp: String::new(), }; @@ -787,32 +814,84 @@ mod tests { .entry(key) .or_insert_with(|| CircuitBreaker::new(map.cb_config.clone())); - // Replicate the logic from `probe_all`. - match result.status { - ProbeStatus::Success => breaker.record_success(), - ProbeStatus::Error => { - if result - .error - .as_ref() - .is_some_and(|e| e.contains("CLI tool") && e.contains("not found on PATH")) - { - // Skipped — this is what we want to verify. - } else { - breaker.record_failure(); - } - } - ProbeStatus::Timeout => breaker.record_failure(), + if !is_environment_error(&error_msg) { + breaker.record_failure(); } - // The breaker must still be Closed because the CLI-not-found error - // was skipped. assert!(matches!( breaker.state(), terraphim_spawner::health::CircuitState::Closed )); - - // When there are no probe results, the provider falls back to the - // circuit breaker state, which is Closed (healthy). assert!(map.is_healthy("openai")); } + + #[test] + fn c1_blocked_probe_does_not_open_breaker() { + let mut map = ProviderHealthMap::new(Duration::from_secs(300)); + + // A banned provider hits the C1 gate and returns this error. + // It must NOT accumulate failures against the circuit breaker — + // the provider API was never contacted. + let error_msg = "probe skipped: provider not in C1 allow-list".to_string(); + let result = ProbeResult { + provider: "github-copilot".to_string(), + model: "github-copilot/gpt-4o".to_string(), + cli_tool: String::new(), + status: ProbeStatus::Error, + latency_ms: None, + error: Some(error_msg.clone()), + timestamp: String::new(), + }; + + let key = format!("{}:{}", result.provider, result.model); + let breaker = map + .breakers + .entry(key) + .or_insert_with(|| CircuitBreaker::new(map.cb_config.clone())); + + if !is_environment_error(&error_msg) { + breaker.record_failure(); + } + + assert!(matches!( + breaker.state(), + terraphim_spawner::health::CircuitState::Closed + )); + assert!(map.is_healthy("github-copilot")); + } + + #[test] + fn no_template_probe_does_not_open_breaker() { + let mut map = ProviderHealthMap::new(Duration::from_secs(300)); + + // A provider whose routing rules have no action:: template returns this + // error from probe_single. It must NOT accumulate failures against the + // circuit breaker — the provider API was never contacted. + let error_msg = "no action:: template defined".to_string(); + let result = ProbeResult { + provider: "unknown-provider".to_string(), + model: "unknown-provider/some-model".to_string(), + cli_tool: String::new(), + status: ProbeStatus::Error, + latency_ms: None, + error: Some(error_msg.clone()), + timestamp: String::new(), + }; + + let key = format!("{}:{}", result.provider, result.model); + let breaker = map + .breakers + .entry(key) + .or_insert_with(|| CircuitBreaker::new(map.cb_config.clone())); + + if !is_environment_error(&error_msg) { + breaker.record_failure(); + } + + assert!(matches!( + breaker.state(), + terraphim_spawner::health::CircuitState::Closed + )); + assert!(map.is_healthy("unknown-provider")); + } } diff --git a/reports/doc-gap-report-20260507-v2.md b/reports/doc-gap-report-20260507-v2.md new file mode 100644 index 000000000..d738519df --- /dev/null +++ b/reports/doc-gap-report-20260507-v2.md @@ -0,0 +1,129 @@ +# Documentation Gap Report -- 2026-05-07 (Run 2) + +**Generated:** 2026-05-07 05:43 CEST +**Agent:** Ferrox (documentation-generator) +**Method:** Scan public items (`pub fn`, `pub async fn`, `pub struct`, `pub enum`, `pub trait`, `pub type`) lacking `///` doc comments + +--- + +## Summary + +| Metric | Value | +|--------|-------| +| Crates scanned | 9 | +| Total missing documentation items | **395** | +| Prior count (morning run, struct/enum only) | 307 | +| Methodology difference | +88 (fn-level items now included) | +| Rust files changed since morning run | 0 | + +No Rust code changed between the two scans; count difference is purely methodology (this run includes `pub fn`/`pub async fn`). + +--- + +## Per-Crate Breakdown + +| Crate | Missing Items | Top File | +|-------|---------------|----------| +| terraphim_agent | 189 | client.rs (59 gaps) | +| terraphim_types | 48 | score/mod.rs (14 gaps) | +| terraphim_orchestrator | 61 | control_plane/routing.rs (11 gaps) | +| terraphim_automata | 23 | builder.rs (14 gaps) | +| terraphim_config | 11 | lib.rs (11 gaps) | +| terraphim_service | 39 | error.rs (13 gaps) | +| terraphim_persistence | 12 | conversation.rs (6 gaps) | +| terraphim_middleware | 8 | command/ripgrep.rs (3 gaps) | +| terraphim_rolegraph | 4 | lib.rs (4 gaps) | + +--- + +## Critical Gaps (Public API Entry Points) + +### terraphim_service -- 39 gaps + +Crate-level public API is undocumented: + +- `lib.rs:68` -- `ServiceError` (root error type) +- `lib.rs:122` -- `Result` type alias +- `lib.rs:124` -- `TerraphimService` (main service struct) +- `llm.rs:21` -- `SummarizeOptions` +- `llm.rs:27` -- `ChatOptions` +- `llm.rs:33` -- `LlmClient` trait (no trait-level doc) +- `error.rs:116-133` -- error constructor methods have no doc + +### terraphim_persistence -- 12 gaps + +- `lib.rs:41` -- `DeviceStorage` struct +- `lib.rs:47` -- `DeviceStorage::instance()` +- `conversation.rs:42-56` -- conversation mutation methods +- `error.rs:4` -- `Error` enum +- `settings.rs:164,349` -- `parse_profile`, `parse_profiles` + +### terraphim_orchestrator -- 61 gaps (new: control_plane) + +New `control_plane/` sub-module (routing, telemetry, policy) added recently: + +- `control_plane/routing.rs:17` -- `RouteSource` enum (11 total gaps in file) +- `control_plane/routing.rs:38` -- `BudgetPressure` +- `control_plane/telemetry.rs:70-95` -- telemetry helper methods +- `flow/config.rs` -- 4 flow configuration types undocumented + +### terraphim_agent -- 189 gaps + +`client.rs` is the worst single file (59 gaps). `ApiClient` struct and all its methods are undocumented. + +--- + +## API Reference Snippets + +Recommended rustdoc stubs for the highest-priority items: + +```rust +/// The primary Terraphim service, providing search, indexing, and AI summarisation. +/// +/// Constructed via [`TerraphimService::new`] with a validated configuration. +/// All search operations are async and cancellable. +pub struct TerraphimService { /* ... */ } + +/// Persistent device storage, providing access to documents, roles, and settings. +/// +/// Implemented as a singleton per device profile. Use [`DeviceStorage::instance`] +/// to obtain the shared handle. +pub struct DeviceStorage { /* ... */ } + +/// Trait for language-model backend implementations. +/// +/// Implement this to wire in a new LLM provider. The two required methods are +/// [`LlmClient::summarize`] and [`LlmClient::chat`]. +pub trait LlmClient { /* ... */ } +``` + +--- + +## Recommendations + +### Priority 1 -- Immediate + +1. `terraphim_service` lib.rs: Add three-line crate doc (`//!`) and doc comments on `TerraphimService`, `Result`, `ServiceError`. +2. `terraphim_persistence` lib.rs: Document `DeviceStorage` and `instance()`. +3. `terraphim_orchestrator` control_plane: Batch-document the routing and telemetry types (recently added, no docs yet). + +### Priority 2 -- High + +4. `terraphim_agent` client.rs: Document `ApiClient` and its 10 public methods. +5. `terraphim_types` score/mod.rs: Document `Scorer` trait and `sort_documents`. + +### Priority 3 -- Ongoing + +6. Introduce CI lint: `cargo doc --no-deps 2>&1 | grep "missing documentation"` as a soft warning gate. + +--- + +## Comparison to Prior Reports + +| Date | Scope | Items | +|------|-------|-------| +| 2026-04-29 | struct/enum/trait | 564 | +| 2026-05-07 morning | struct/enum/trait | 307 (-45%) | +| 2026-05-07 afternoon | + fn/async fn | 395 | + +Theme-ID: doc-gap diff --git a/reports/doc-gap-report-20260507-v3.md b/reports/doc-gap-report-20260507-v3.md new file mode 100644 index 000000000..767d60cce --- /dev/null +++ b/reports/doc-gap-report-20260507-v3.md @@ -0,0 +1,104 @@ +# Documentation Gap Report -- 2026-05-07 (Afternoon Run) + +**Generated:** 2026-05-07 06:45 CEST +**Agent:** Ferrox (documentation-generator) +**Method:** Scan public items (`pub fn`, `pub async fn`, `pub struct`, `pub enum`, `pub trait`, `pub type`) lacking `///` doc comments + +--- + +## Summary + +| Metric | Value | +|--------|-------| +| Crates scanned | 9 | +| Rust files changed since v2 (morning) | 0 | +| Total missing documentation items | **395** (unchanged from v2) | +| New gaps introduced today | 0 | + +No Rust source changes since the v2 scan at 05:43 CEST. All counts carry forward unchanged. + +--- + +## Per-Crate Breakdown (unchanged from v2) + +| Crate | Missing Items | Top File | +|-------|---------------|----------| +| terraphim_agent | 189 | client.rs (59 gaps) | +| terraphim_orchestrator | 61 | control_plane/routing.rs (11 gaps) | +| terraphim_types | 48 | score/mod.rs (14 gaps) | +| terraphim_service | 39 | error.rs (13 gaps) | +| terraphim_automata | 23 | builder.rs (14 gaps) | +| terraphim_persistence | 12 | conversation.rs (6 gaps) | +| terraphim_config | 11 | lib.rs (11 gaps) | +| terraphim_middleware | 8 | command/ripgrep.rs (3 gaps) | +| terraphim_rolegraph | 4 | lib.rs (4 gaps) | + +--- + +## Spec Validation Cross-Reference + +The spec validation report (`reports/spec-validation-20260507.md`) confirms: +- **FAIL** status: 2 persistent gaps remain unresolved +- No new spec violations introduced today +- `meta_coordinator.rs` (741 lines, added in build-runner commit) has no corresponding spec document -- flagged as a documentation gap risk + +--- + +## API Reference Snippets (carried from v2) + +```rust +/// The primary Terraphim service, providing search, indexing, and AI summarisation. +/// +/// Constructed via [`TerraphimService::new`] with a validated configuration. +/// All search operations are async and cancellable. +pub struct TerraphimService { /* ... */ } + +/// Persistent device storage, providing access to documents, roles, and settings. +/// +/// Implemented as a singleton per device profile. Use [`DeviceStorage::instance`] +/// to obtain the shared handle. +pub struct DeviceStorage { /* ... */ } + +/// Trait for language-model backend implementations. +/// +/// Implement this to wire in a new LLM provider. The two required methods are +/// [`LlmClient::summarize`] and [`LlmClient::chat`]. +pub trait LlmClient { /* ... */ } + +/// Routes agent tasks to the appropriate executor based on budget and availability. +/// +/// Integrates with [`BudgetPressure`] for back-pressure-aware dispatching. +pub enum RouteSource { /* ... */ } +``` + +--- + +## Recommendations + +### Priority 1 -- Immediate + +1. `terraphim_orchestrator` meta_coordinator.rs: New 741-line file with zero doc comments. Requires crate-level doc (`//!`) and docs on at least the public structs and entry-point methods. +2. `terraphim_service` lib.rs: Document `TerraphimService`, `Result`, `ServiceError` (3 lines each). +3. `terraphim_persistence` lib.rs: Document `DeviceStorage::instance()`. + +### Priority 2 -- High + +4. `terraphim_agent` client.rs: Document `ApiClient` and its 10 public methods. +5. `terraphim_types` score/mod.rs: Document `Scorer` trait and `sort_documents`. + +### Priority 3 -- Ongoing + +6. CI lint gate: `cargo doc --no-deps 2>&1 | grep "missing documentation"` as a soft-warning step. + +--- + +## Comparison to Prior Reports + +| Date | Run | Scope | Items | +|------|-----|-------|-------| +| 2026-04-29 | morning | struct/enum/trait | 564 | +| 2026-05-07 | v1 morning | struct/enum/trait | 307 (-45%) | +| 2026-05-07 | v2 morning | + fn/async fn | 395 | +| 2026-05-07 | v3 afternoon | + fn/async fn | 395 (unchanged) | + +Theme-ID: doc-gap diff --git a/reports/doc-gap-report-20260507-v4.md b/reports/doc-gap-report-20260507-v4.md new file mode 100644 index 000000000..33926546e --- /dev/null +++ b/reports/doc-gap-report-20260507-v4.md @@ -0,0 +1,200 @@ +# Documentation Gap Report -- 2026-05-07 (v4 Full Workspace Scan) + +**Generated:** 2026-05-07 +**Agent:** Ferrox (documentation-generator) +**Branch:** task/446-anthropic-probe-circuit-breaker-fix +**Method:** Scan all public items (`pub fn`, `pub async fn`, `pub struct`, `pub enum`, `pub trait`, `pub type`) lacking `///` doc comments across all workspace crates. + +--- + +## Summary + +| Metric | Value | +|--------|-------| +| Crates scanned | 55 | +| Prior reports scanned | 9 crates only | +| Total public items | ~5,700 | +| Total missing documentation | ~2,450 | +| Crates at 100% gap rate | 8 | +| Crates with >60% gap rate | 16 | + +--- + +## Per-Crate Breakdown (full workspace) + +| Crate | Total Pub | Missing | % Missing | Priority | +|-------|-----------|---------|-----------|----------| +| haystack_atlassian | 29 | 29 | 100% | High | +| haystack_core | 1 | 1 | 100% | Medium | +| haystack_discourse | 9 | 9 | 100% | Medium | +| terraphim_automata_py | 3 | 3 | 100% | Low | +| terraphim_ccusage | 11 | 11 | 100% | Medium | +| terraphim_kg_linter | 15 | 15 | 100% | Medium | +| terraphim_negative_contribution | 8 | 8 | 100% | Low | +| terraphim_rolegraph_py | 7 | 7 | 100% | Low | +| terraphim_codebase_eval | 13 | 11 | 84% | Medium | +| terraphim-markdown-parser | 21 | 17 | 80% | Medium | +| terraphim_kg_agents | 38 | 29 | 76% | Medium | +| terraphim_agent_application | 89 | 67 | 75% | Medium | +| terraphim-session-analyzer | 146 | 106 | 72% | High | +| haystack_grepapp | 19 | 13 | 68% | Medium | +| haystack_jmap | 9 | 6 | 66% | Medium | +| terraphim_usage | 75 | 49 | 65% | High | +| terraphim_persistence | 26 | 16 | 61% | High | +| terraphim_multi_agent | 333 | 191 | 57% | High | +| terraphim_task_decomposition | 85 | 49 | 57% | Medium | +| terraphim_middleware | 48 | 27 | 56% | High | +| terraphim_onepassword_cli | 11 | 6 | 54% | Low | +| terraphim_settings | 11 | 6 | 54% | Medium | +| terraphim_agent | 727 | 371 | 51% | Critical | +| terraphim_agent_registry | 106 | 54 | 50% | High | +| terraphim_config | 42 | 21 | 50% | High | +| terraphim_dsm | 12 | 6 | 50% | Low | +| terraphim_agent_evolution | 223 | 104 | 46% | High | +| terraphim_types | 318 | 148 | 46% | High | +| terraphim_automata | 125 | 55 | 44% | High | +| terraphim_agent_supervisor | 61 | 25 | 40% | Medium | +| terraphim_tracker | 57 | 23 | 40% | Medium | +| terraphim_atomic_client | 63 | 25 | 39% | Medium | +| terraphim_goal_alignment | 104 | 40 | 38% | Medium | +| terraphim_orchestrator | 611 | 236 | 38% | Critical | +| terraphim_sessions | 101 | 38 | 37% | Medium | +| terraphim_agent_messaging | 99 | 36 | 36% | Medium | +| terraphim_kg_orchestration | 63 | 21 | 33% | Medium | +| terraphim_test_utils | 12 | 4 | 33% | Low | +| terraphim_service | 231 | 75 | 32% | High | +| terraphim_validation | 355 | 116 | 32% | High | +| terraphim_github_runner | 110 | 35 | 31% | Medium | +| terraphim_build_args | 82 | 25 | 30% | Low | +| terraphim_router | 103 | 27 | 26% | Medium | +| terraphim_hooks | 32 | 8 | 25% | Low | +| terraphim_symphony | 126 | 32 | 25% | Medium | +| terraphim_tinyclaw | 238 | 56 | 23% | Medium | +| terraphim_update | 90 | 21 | 23% | Low | +| terraphim_spawner | 90 | 20 | 22% | Medium | +| terraphim_github_runner_server | 10 | 2 | 20% | Low | +| terraphim_rolegraph | 105 | 21 | 20% | Medium | +| terraphim_rlm | 257 | 50 | 19% | Medium | +| terraphim_workspace | 23 | 4 | 17% | Low | +| terraphim_file_search | 8 | 1 | 12% | Low | +| terraphim_cli | 19 | 2 | 10% | Low | +| terraphim_mcp_server | 34 | 3 | 8% | Low | + +--- + +## Critical Crates (>200 public items, high gap rate) + +### terraphim_agent (727 total, 371 missing, 51%) + +Top undocumented items in `listener.rs`: +- `AgentIdentity`, `NotificationRuleKind`, `NotificationRule`, `DelegationPolicy` +- `GiteaConnection`, `ListenerConfig`, `for_identity`, `validate`, `load_from_path` +- `run_forever`, `run_once` + +### terraphim_orchestrator (611 total, 236 missing, 38%) + +Top undocumented items: +- `config.rs`: `PreCheckStrategy`, `QuickwitConfig` +- `mention.rs`: `load_or_now` +- `scheduler.rs`: `take_event_rx` +- `provider_budget.rs`: `is_empty` +- All `lib.rs` `pub mod` declarations (32+ entries) + +### terraphim_validation (355 total, 116 missing, 32%) + +High volume, medium priority -- many small validation helpers. + +### terraphim_multi_agent (333 total, 191 missing, 57%) + +Large surface, high gap rate. Entry-point types for agent coordination undocumented. + +### terraphim_types (318 total, 148 missing, 46%) + +Shared type definitions used across all crates. Documenting this crate delivers the highest cross-crate leverage. + +--- + +## New Gaps Since v3 + +The v3 report scanned 9 crates; this v4 scan covers all 55. New crates brought into scope: + +| Previously Unseen Crate | Missing Items | +|-------------------------|---------------| +| terraphim-session-analyzer | 106 | +| terraphim_multi_agent | 191 | +| terraphim_agent_evolution | 104 | +| terraphim_validation | 116 | +| terraphim_tinyclaw | 56 | +| terraphim_rlm | 50 | +| terraphim_usage | 49 | +| terraphim_task_decomposition | 49 | +| haystack_atlassian | 29 | +| terraphim_kg_agents | 29 | + +--- + +## API Reference Snippets (new this run) + +```rust +/// Returns `true` when the error string indicates a local environment or +/// configuration problem rather than a genuine provider health failure. +/// +/// Errors matching this predicate must not advance the circuit-breaker failure +/// counter because they reflect local setup (missing CLI tool, routing config, +/// C1 allow-list) -- not transient API unavailability. +fn is_environment_error(error: &str) -> bool { /* ... */ } + +/// Aggregated health state for all configured LLM providers. +/// +/// Maintains per-provider circuit-breaker state. Updated on every probe cycle +/// via [`ProviderHealthMap::update_from_results`]. +pub struct ProviderHealthMap { /* ... */ } + +/// Notification rule controlling which Gitea events an agent listener accepts. +/// +/// Matched against incoming webhook payloads before dispatching to the agent +/// handler. See [`ListenerConfig::validate`] for validation semantics. +pub struct NotificationRule { /* ... */ } + +/// Unified type definitions shared across all terraphim workspace crates. +/// +/// Import via `terraphim_types::prelude::*` to bring common types into scope. +pub mod prelude { /* ... */ } +``` + +--- + +## Recommendations + +### Priority 1 -- Critical (address within current sprint) + +1. **`terraphim_types`**: Document `prelude`, `Document`, `SearchQuery`, `RelevanceFunction` -- highest leverage as these appear in every crate. +2. **`terraphim_agent` listener.rs**: Document `ListenerConfig`, `NotificationRule`, `DelegationPolicy` entry points (public API contract). +3. **`terraphim_orchestrator` config.rs**: Document `PreCheckStrategy`, `OrchestratorConfig`, `QuickwitConfig`. + +### Priority 2 -- High (address within next two sprints) + +4. **`terraphim_multi_agent`**: Module-level `//!` docs on at least the top-level types. +5. **`terraphim_agent_evolution`**: Document trait definitions and public constructors. +6. **`haystack_atlassian`**: Currently 100% undocumented -- add crate-level `//!` and document `AtlassianHaystack::new`. + +### Priority 3 -- Ongoing + +7. CI gate: add `cargo doc --no-deps 2>&1 | grep "^warning.*missing documentation"` as a soft-fail step. +8. Enforce `#![warn(missing_docs)]` in `terraphim_types` and `terraphim_orchestrator` as a start. + +--- + +## Comparison to Prior Reports + +| Date | Run | Crates | Items Missing | +|------|-----|--------|---------------| +| 2026-04-29 | morning | 9 (struct/enum/trait only) | 564 | +| 2026-05-07 | v1 | 9 (struct/enum/trait only) | 307 (-45%) | +| 2026-05-07 | v2 | 9 (+ fn/async fn) | 395 | +| 2026-05-07 | v3 | 9 (+ fn/async fn, afternoon) | 395 (unchanged) | +| 2026-05-07 | v4 | **55** (full workspace) | ~2,450 | + +The step from 395 to ~2,450 reflects expanded scope (55 crates vs 9), not regression. + +Theme-ID: doc-gap diff --git a/reports/spec-validation-20260507-v2.md b/reports/spec-validation-20260507-v2.md new file mode 100644 index 000000000..ab32f5334 --- /dev/null +++ b/reports/spec-validation-20260507-v2.md @@ -0,0 +1,156 @@ +# Spec Validation Report: 2026-05-07 (v2) + +**Validator:** Carthos (Domain Architect) +**Date:** 2026-05-07 08:33 CEST +**Prior run:** 2026-05-07 06:33 CEST (no changes detected since) +**Verdict:** FAIL — 2 persistent gaps, 0 new gaps + +--- + +## Executive Summary + +Six specification documents reviewed in `plans/`. No new specs or commits affecting spec compliance since the 06:33 run. Two gaps persist unchanged: `meta_coordinator` still absent from `lib.rs` (blocker, PR #1291 unmerged) and `guard.rs` still absent (medium, #1274 open). All other bounded contexts remain fully implemented. + +--- + +## Specification-by-Specification Validation + +### 1. `design-gitea82-correction-event.md` — ✅ FULLY IMPLEMENTED + +Verified: +- `CorrectionType` enum: `crates/terraphim_agent/src/learnings/capture.rs:44` +- `CorrectionEvent` struct: `capture.rs:502` +- `capture_correction()` function: exported via `mod.rs:41` +- `LearningEntry` enum (Learning + Correction + Procedure): `capture.rs:1225` +- `list_all_entries`, `query_all_entries`: exported via `mod.rs:42-43` +- `LearnSub::Correction` CLI variant: `main.rs:3138` +- Secret redaction in capture path: `mod.rs` confirms `redact_secrets` called + +All 8 unit tests and 1 CLI integration test specified in the plan are expected covered. Status: stable. + +### 2. `design-gitea84-trigger-based-retrieval.md` — ✅ MOSTLY IMPLEMENTED / ⚠️ MINOR GAP + +Verified implemented: +- `trigger` and `pinned` fields on `MarkdownDirectives`: `crates/terraphim_types/src/lib.rs:502-504` +- `trigger::` and `pinned::` parsing: `crates/terraphim_automata/src/markdown_directives.rs:215-251` +- `TriggerIndex` struct: `crates/terraphim_rolegraph/src/lib.rs:51` +- `trigger_index` and `pinned_node_ids` fields on `RoleGraph`: `lib.rs:320-322` +- `trigger_descriptions` and `pinned_node_ids` on `SerializableRoleGraph`: `lib.rs:271-273` +- `find_matching_node_ids_with_fallback()`: `lib.rs:451` +- `load_trigger_index()`: `lib.rs:478` +- `query_graph_with_trigger_fallback()`: `lib.rs:718` +- `--include-pinned` flag on search CLI: `main.rs:718` +- Integration tests: `two_pass_fallback_to_trigger` (`lib.rs:2196`), `pinned_always_included` (`lib.rs:2215`), `serializable_roundtrip_preserves_triggers` (`lib.rs:2233`) + +**Minor gap**: The spec's §7 CLI design includes a `KgSub` enum with `kg list --pinned`. This sub-command is absent from `main.rs` (no `KgSub` enum found). The `--include-pinned` flag on the search path covers the primary acceptance criteria (AC6 in spec). The `kg list --pinned` listing command is not in the formal acceptance criteria list. Assessed as **follow-up**, not a blocker. + +### 3. `d3-session-auto-capture-plan.md` — ✅ FULLY IMPLEMENTED + +Verified: +- `from_session_commands()`: `crates/terraphim_agent/src/learnings/procedure.rs:412` +- `extract_bash_commands_from_session()`: `procedure.rs:471` +- `ProcedureSub::FromSession` variant gated `#[cfg(feature = "repl-sessions")]`: `main.rs:3413` +- Implementation wires to `ProcedureStore::save_with_dedup()`: `main.rs:3446` +- `procedure` module no longer behind `#[cfg(test)]`: `mod.rs:31` shows `pub(crate) mod procedure;` +- Test coverage: 6 unit tests including `test_from_session_commands_basic`, `_filters_trivial`, `_auto_title` confirmed in `procedure.rs` + +Status: stable. + +### 4. `design-single-agent-listener.md` — ✅ OPERATIONAL (infrastructure concern) + +Verified: +- `~/.config/terraphim/listener-worker.json`: EXISTS (created 2026-04-16) +- `~/.config/terraphim/scripts/start-listener.sh`: EXISTS (created 2026-04-16) +- Spec explicitly states "No code changes to the Rust codebase are required" +- All code-level invariants (claim logic, self-filtering, retry) covered by existing tests in `listener.rs` + +Whether the tmux session is currently running is an operational concern outside spec scope. Status: stable. + +### 5. `learning-correction-system-plan.md` — ❌ GAP PERSISTS (Phase H) + +**Gap G-2026-05-06-1: `guard.rs` module absent** + +`crates/terraphim_agent/src/learnings/guard.rs` does not exist. Issue #1274 remains open. + +Required by spec Phase H: +- `ExecutionTier` enum (Allow / Sandbox / Deny) +- `GuardDecision` type +- `evaluate_command()` function +- Integration with Firecracker sandbox tier + +**Severity:** Medium — no automated command safety evaluation before procedure replay. + +The rest of the plan (Phases A–G) is confirmed implemented per prior runs. + +### 6. `research-single-agent-listener.md` — ✅ RESEARCH COMPLETE + +Phase 1 artefact only; no implementation deliverables. Status: stable. + +--- + +## Persistent Non-Spec Gap: `meta_coordinator.rs` Orphaned + +**Gap G-2026-05-07-1: `meta_coordinator` not declared in `lib.rs`** + +`crates/terraphim_orchestrator/src/meta_coordinator.rs` (25 KB, added 2026-05-06) remains absent from the `pub mod` list in `lib.rs` (lines 31–65, 34 declarations confirmed; `meta_coordinator` absent between `mention_chain` and `metrics_persistence`). + +Issue #1275 open. PR #1291 exists — **not yet merged**. + +Consequence: all 741 lines of `meta_coordinator.rs` are dead code. Five `#[tokio::test]` functions inside are unreachable. The `dispatch_cycle` integration invariant is unverified. + +**Severity:** Blocker — entire module is unreachable until declaration is added. + +Note: this gap has no corresponding `plans/` spec document. It is tracked via Gitea issue #1275 and PR #1291. + +--- + +## Traceability Matrix + +| Req ID | Requirement | Design Ref | Impl Ref | Tests | Evidence | Status | +|-------:|-------------|------------|----------|-------|----------|--------| +| REQ-82-001 | `CorrectionEvent` struct with typed corrections | `design-gitea82-correction-event.md §1.2` | `capture.rs:502` | `test_correction_event_roundtrip` | `capture.rs:502` | ✅ | +| REQ-82-002 | `capture_correction()` with secret redaction | `§1.4` | `capture.rs`, `mod.rs:41` | `test_capture_correction`, `test_correction_secret_redaction` | `mod.rs:41` | ✅ | +| REQ-82-003 | `LearnSub::Correction` CLI | `§3.1` | `main.rs:3138` | CLI integration test | `main.rs:3138` | ✅ | +| REQ-82-004 | Unified `list_all_entries` / `query_all_entries` | `§1.5` | `mod.rs:42-43` | `test_list_all_entries_mixed` | `mod.rs:42` | ✅ | +| REQ-84-001 | `trigger::` / `pinned::` directive parsing | `design-gitea84 §2` | `markdown_directives.rs:215` | `parses_trigger_directive`, `parses_pinned_directive` | `markdown_directives.rs:348` | ✅ | +| REQ-84-002 | `TriggerIndex` TF-IDF fallback | `§3` | `rolegraph/lib.rs:51` | `tfidf_exact_match_scores_high`, `two_pass_fallback_to_trigger` | `lib.rs:2196` | ✅ | +| REQ-84-003 | `--include-pinned` search CLI flag | `§7` | `main.rs:718` | Acceptance criteria AC6 | `main.rs:718` | ✅ | +| REQ-84-004 | `kg list --pinned` CLI command | `§7` | ABSENT | ABSENT | `KgSub` enum not in `main.rs` | ⚠️ | +| REQ-D3-001 | `learn procedure from-session ` CLI | `d3-session §design` | `main.rs:3413` | `test_from_session_commands_basic` | `main.rs:3413` | ✅ | +| REQ-D3-002 | Trivial command filter | `d3-session §design` | `procedure.rs:412` | `test_from_session_commands_filters_trivial` | `procedure.rs:868` | ✅ | +| REQ-D3-003 | Feature-gated `repl-sessions` | `d3-session §dependencies` | `main.rs:3413` | N/A | `#[cfg(feature = "repl-sessions")]` | ✅ | +| PH-H-001 | `ExecutionTier` enum | `learning-correction-plan §Phase H` | `guard.rs` — ABSENT | ABSENT | File does not exist | ❌ | +| PH-H-002 | `evaluate_command()` | Same | Same | ABSENT | Same | ❌ | +| META-001 | `meta_coordinator` in public API | Gitea #1275 | `lib.rs` — ABSENT | 5 tests unreachable | `grep meta_coordinator lib.rs` → 0 hits | ❌ | + +--- + +## Gap Summary + +| Gap ID | Description | Severity | Issue | Status | +|--------|-------------|----------|-------|--------| +| G-2026-05-07-1 | `meta_coordinator` not in `lib.rs` — all code dead | Blocker | #1275, PR #1291 (unmerged) | ❌ OPEN | +| G-2026-05-06-1 | `guard.rs` absent — Phase H Graduated Guard missing | Medium | #1274 | ❌ OPEN | +| G-2026-05-07-2 | `kg list --pinned` CLI sub-command absent | Minor follow-up | (no issue) | ⚠️ FOLLOW-UP | + +--- + +## Recommendations (smallest first) + +1. **Merge PR #1291** — adds `pub mod meta_coordinator;` to `lib.rs`. One line change. Unblocks all internal tests and the `dispatch_cycle` integration invariant. +2. **Fix `last_cleanup` mutation bug** in `dispatch_cycle` — never updates `self.last_cleanup` after `cleanup_expired`, causing cleanup to run on every cycle after hour 1. +3. **Add `kg list --pinned` command** — trivial extension of the search CLI. Add `KgSub` enum with `List { pinned: bool }` per spec §7. +4. **Implement `guard.rs` (Phase H)** — `ExecutionTier` enum, `GuardDecision`, `evaluate_command()` per spec. Self-contained; no other gaps block it. +5. **Create `plans/design-meta-coordinator.md`** — document the bounded context, scoring formula, agent selection precedence, and TTL rationale for `MetaCoordinator`. + +--- + +## Conclusion + +No regression since the 06:33 run. Two gaps persist: the `meta_coordinator` orphan (blocker, one-line fix available as PR #1291) and the absent Graduated Guard module (medium, #1274). One minor follow-up (missing `kg list --pinned` CLI). All other specs are fully implemented and tested. + +**Verdict: FAIL — 2 open gaps (1 blocker, 1 medium) + 1 minor follow-up** + +--- + +Validated against commit `d5293544d` on branch `main`. Plans directory: 6 specs, unchanged since 2026-05-04. diff --git a/reports/spec-validation-20260507-v3.md b/reports/spec-validation-20260507-v3.md new file mode 100644 index 000000000..19eb4bfd5 --- /dev/null +++ b/reports/spec-validation-20260507-v3.md @@ -0,0 +1,143 @@ +# Spec Validation Report: 2026-05-07 (v3) + +**Validator:** Carthos (Domain Architect) +**Date:** 2026-05-07 12:35 CEST +**Prior run:** 2026-05-07 08:33 CEST (v2) +**Verdict:** FAIL — 2 persistent gaps + 1 process discrepancy + +--- + +## Executive Summary + +Six specification documents in `plans/` reviewed against HEAD `eaae3d806`. Since the v2 run (08:33 CEST), one commit was added (`eaae3d806`) containing only documentation files (spec validation reports). No code changes. All four previously passing specs remain fully implemented. Both open gaps persist unchanged. + +**New finding this run:** Issue #1275 was closed between v2 and v3, but PR #1291 remains open (`state=open, merged=False`). The one-line fix (`pub mod meta_coordinator;`) has not landed on `main`. The issue closure is premature — the blocker gap is still present. + +--- + +## Specification-by-Specification Validation + +### 1. `design-gitea82-correction-event.md` — FULLY IMPLEMENTED + +No changes since v2. All 8 unit tests and CLI integration test remain covered. +- `CorrectionType` enum: `capture.rs:44` +- `CorrectionEvent` struct: `capture.rs:502` +- `capture_correction()`: `mod.rs:41` +- `LearningEntry` enum: `capture.rs:1225` +- `list_all_entries`, `query_all_entries`: `mod.rs:42-43` +- `LearnSub::Correction` CLI: `main.rs:3138` + +Status: stable. + +### 2. `design-gitea84-trigger-based-retrieval.md` — MOSTLY IMPLEMENTED / MINOR GAP + +No changes since v2. All primary acceptance criteria implemented. Minor gap persists: + +**Follow-up G-2026-05-07-2:** `kg list --pinned` CLI sub-command (`KgSub` enum) absent from `main.rs`. Not in the formal acceptance criteria list. Follow-up only. + +Status: stable. + +### 3. `d3-session-auto-capture-plan.md` — FULLY IMPLEMENTED + +No changes since v2. All 6 unit tests confirmed, feature gate `#[cfg(feature = "repl-sessions")]` in place. + +Status: stable. + +### 4. `design-single-agent-listener.md` — OPERATIONAL + +No code changes required by spec. Infrastructure files exist. No code-level regression. + +Status: stable. + +### 5. `learning-correction-system-plan.md` — GAP PERSISTS (Phase H) + +**Gap G-2026-05-06-1: `guard.rs` module absent** + +`crates/terraphim_agent/src/learnings/guard.rs` does not exist. Issue #1274 remains open. + +Required per spec Phase H: +- `ExecutionTier` enum (Allow / Sandbox / Deny) +- `GuardDecision` type +- `evaluate_command()` function +- Integration with Firecracker sandbox tier + +The learnings directory contains: `capture.rs`, `compile.rs`, `export_kg.rs`, `hook.rs`, `install.rs`, `mod.rs`, `procedure.rs`, `redaction.rs`, `replay.rs`, `suggest.rs`. No `guard.rs`. + +**Severity:** Medium — no automated command safety evaluation before procedure replay. + +Phases A–G confirmed implemented per prior runs. + +### 6. `research-single-agent-listener.md` — RESEARCH COMPLETE + +Phase 1 artefact only; no implementation deliverables. Status: stable. + +--- + +## Persistent Non-Spec Gap: `meta_coordinator.rs` Orphaned + +**Gap G-2026-05-07-1: `meta_coordinator` not declared in `lib.rs`** + +`crates/terraphim_orchestrator/src/meta_coordinator.rs` (25 KB, 741 lines, 5 `#[tokio::test]` functions) remains absent from the `pub mod` declarations in `crates/terraphim_orchestrator/src/lib.rs`. + +Verified by direct grep: `grep -n "meta_coordinator" crates/terraphim_orchestrator/src/lib.rs` → 0 matches. + +The `pub mod` list runs: `mention`, `mention_chain`, `metrics_persistence`, `mode`, `nightwatch` — `meta_coordinator` absent between `mention_chain` and `metrics_persistence`. + +**Process discrepancy:** Issue #1275 was closed (confirmed via Gitea API: `state=closed`), but PR #1291 (`Fix #1275: wire meta_coordinator module into lib.rs`) remains open and unmerged (`state=open, merged=False`). The issue was closed without the fix landing on `main`. The fix is a single line addition: `pub mod meta_coordinator;`. + +**Severity:** Blocker — all 741 lines of `meta_coordinator.rs` are dead code. Five `#[tokio::test]` functions are unreachable. The `dispatch_cycle` integration invariant is unverified. The `last_cleanup` mutation bug (pre-existing, documented in PR #1291 report) remains unresolved. + +--- + +## Traceability Matrix + +| Req ID | Requirement | Design Ref | Impl Ref | Tests | Evidence | Status | +|-------:|-------------|------------|----------|-------|----------|--------| +| REQ-82-001 | `CorrectionEvent` struct with typed corrections | `design-gitea82 §1.2` | `capture.rs:502` | `test_correction_event_roundtrip` | `capture.rs:502` | PASS | +| REQ-82-002 | `capture_correction()` with secret redaction | `§1.4` | `capture.rs`, `mod.rs:41` | `test_capture_correction`, `test_correction_secret_redaction` | `mod.rs:41` | PASS | +| REQ-82-003 | `LearnSub::Correction` CLI | `§3.1` | `main.rs:3138` | CLI integration test | `main.rs:3138` | PASS | +| REQ-82-004 | Unified `list_all_entries` / `query_all_entries` | `§1.5` | `mod.rs:42-43` | `test_list_all_entries_mixed` | `mod.rs:42` | PASS | +| REQ-84-001 | `trigger::` / `pinned::` directive parsing | `design-gitea84 §2` | `markdown_directives.rs:215` | `parses_trigger_directive`, `parses_pinned_directive` | `markdown_directives.rs:348` | PASS | +| REQ-84-002 | `TriggerIndex` TF-IDF fallback | `§3` | `rolegraph/lib.rs:51` | `two_pass_fallback_to_trigger` | `lib.rs:2196` | PASS | +| REQ-84-003 | `--include-pinned` search CLI flag | `§7` | `main.rs:718` | Acceptance criteria AC6 | `main.rs:718` | PASS | +| REQ-84-004 | `kg list --pinned` CLI command | `§7` | ABSENT | ABSENT | `KgSub` enum not in `main.rs` | FOLLOW-UP | +| REQ-D3-001 | `learn procedure from-session ` CLI | `d3-session §design` | `main.rs:3413` | `test_from_session_commands_basic` | `main.rs:3413` | PASS | +| REQ-D3-002 | Trivial command filter | `d3-session §design` | `procedure.rs:412` | `test_from_session_commands_filters_trivial` | `procedure.rs:868` | PASS | +| REQ-D3-003 | Feature-gated `repl-sessions` | `d3-session §dependencies` | `main.rs:3413` | N/A | `#[cfg(feature = "repl-sessions")]` | PASS | +| PH-H-001 | `ExecutionTier` enum | `learning-correction-plan §Phase H` | `guard.rs` — ABSENT | ABSENT | File does not exist | FAIL | +| PH-H-002 | `evaluate_command()` | Same | Same | ABSENT | Same | FAIL | +| META-001 | `meta_coordinator` in public API | Gitea #1275 | `lib.rs` — ABSENT | 5 tests unreachable | `grep meta_coordinator lib.rs` → 0 hits | FAIL | + +--- + +## Gap Summary + +| Gap ID | Description | Severity | Issue | Status | +|--------|-------------|----------|-------|--------| +| G-2026-05-07-1 | `meta_coordinator` not in `lib.rs` — dead code; issue #1275 closed but PR #1291 unmerged | Blocker | #1275 (closed prematurely), PR #1291 (open) | OPEN | +| G-2026-05-06-1 | `guard.rs` absent — Phase H Graduated Guard missing | Medium | #1274 (open) | OPEN | +| G-2026-05-07-2 | `kg list --pinned` CLI sub-command absent | Minor follow-up | (no issue) | FOLLOW-UP | +| PROCESS-001 | Issue #1275 closed without PR #1291 merged — process discrepancy | Process | #1275, PR #1291 | OPEN | + +--- + +## Recommendations (smallest first) + +1. **Reopen or note issue #1275** — it was closed without the fix landing. Either reopen or add a comment linking to PR #1291 so the gap is traceable. +2. **Merge PR #1291** — single line: `pub mod meta_coordinator;` in `lib.rs`. Unblocks all dead tests and the `dispatch_cycle` integration invariant. +3. **Fix `last_cleanup` mutation bug** — `dispatch_cycle` takes `&self` so `last_cleanup` is never updated; cleanup runs every cycle after hour 1. Wrap in `Arc>` or change to `&mut self`. +4. **Add `kg list --pinned` command** — trivial extension per spec §7; add `KgSub` enum with `List { pinned: bool }`. +5. **Implement `guard.rs` (Phase H)** — `ExecutionTier` enum, `GuardDecision`, `evaluate_command()`. Self-contained; no other gaps block it. Issue #1274 open. +6. **Create `plans/design-meta-coordinator.md`** — 741-line bounded context with no design artefact. Document scoring formula, agent selection precedence, TTL rationale. + +--- + +## Conclusion + +No code regression since v2. Two spec gaps persist (blocker + medium). One new process finding: issue #1275 was closed without the code fix merging — PR #1291 remains open on Gitea. The issue closure is misleading and should be corrected. + +**Verdict: FAIL — 2 open spec gaps (1 blocker, 1 medium) + 1 process discrepancy + 1 minor follow-up** + +--- + +Validated against commit `eaae3d806` on branch `main`. Plans directory: 6 specs, unchanged since 2026-05-04. Gitea API confirmed: PR #1291 `merged=False`, issue #1274 `state=open`. diff --git a/reports/spec-validation-20260507.md b/reports/spec-validation-20260507.md new file mode 100644 index 000000000..1aac0e6ee --- /dev/null +++ b/reports/spec-validation-20260507.md @@ -0,0 +1,117 @@ +# Spec Validation Report: 2026-05-07 + +**Validator:** Carthos (Domain Architect) +**Date:** 2026-05-07 06:33 CEST +**Verdict:** FAIL — 2 persistent gaps, 0 new gaps + +--- + +## Executive Summary + +Six specification documents reviewed in `plans/`. No new specs added since 2026-05-04. Two previously identified gaps persist unresolved. One new operational change (provider_probe.rs hardening) has no corresponding spec — assessed as in-scope for its parent issue #1233 and not a spec violation. + +--- + +## Specification-by-Specification Validation + +### 1. `design-gitea82-correction-event.md` — ✅ FULLY IMPLEMENTED (unchanged) + +All aggregate roots and invariants confirmed implemented in prior run (2026-05-06). No code changes since last validation. Status: stable. + +### 2. `design-gitea84-trigger-based-retrieval.md` — ✅ FULLY IMPLEMENTED (unchanged) + +Two-pass Search invariant, pinned-entry inclusion, and all 23 tests confirmed in prior run. No code changes since last validation. Status: stable. + +### 3. `d3-session-auto-Terraphim Graph Embeddings: Learning Agent Guideture-plan.md` — ✅ FULLY IMPLEMENTED (unchanged) + +Session-to-procedure extraction Middleware confirmed complete. Status: stable. + +### 4. `design-single-agent-listener.md` — ⚠️ OPERATIONAL GAP (unchanged) + +Code boundary complete. Listener tmux session not active; infrastructure concern only, not a spec violation. + +### 5. `learning-correction-System-plan.md` — ❌ GAP PERSISTS (Phase H) + +**Gap G-2026-05-06-1: `guard.rs` module absent** + +`crates/terraphim_agent/src/learnings/guard.rs` does not exist. Gitea issue #1274 is open. No code changes since 2026-05-06. + +Required by spec: +- `ExecutionTier` enum (Allow / SanDatabaseox / Deny) +- `GuardDecision` type +- `evaluate_command()` function +- Integration with Firecracker sanDatabaseox tier + +**Bug Reporting:** Medium — no automated command safety evaluation before procedure replay. + +### 6. `reSearch-single-agent-listener.md` — ✅ RESearch COMPLETE (unchanged) + +Phase 1 artefact; no implementation required. + +--- + +## New Implementation Activity (not spec-driven) + +### `provider_probe.rs` — modified 2026-05-07 02:49 + +Provider probe hardening landed in commits `b09954c6`, `a08082dd`, `1238a680`. These address ADF fleet DEGRADED alert (issue #1233) and zombie-process / test-hang fixes. No corresponding plan file in `plans/`. Assessed as operational fix within the bounded context of the existing orchestrator; no new spec required. Not a gap. + +--- + +## Persistent Gap: meta_coordinator.rs Orphaned + +**Gap G-2026-05-07-1: `meta_coordinator` not declared in `lib.rs`** + +`crates/terraphim_orchestrator/src/meta_coordinator.rs` (25 KB, added 2026-05-06) remains absent from the `pub mod` list in `lib.rs` (verified: lines 31–65, 34 declarations). + +Missing declaration location: between line 47 (`pub mod mention_chain;`) and line 48 (`pub mod metrics_Database;`). + +Gitea issue #1275 open. PR #1291 exists as a fix attempt — **not yet merged**. + +Five internal `#[tokio::test]` functions are unreachable until the declaration is added. + +**Bug Reporting:** Blocker — all 741 lines of dead code; `dispatch_cycle` integration invariant unverified. + +--- + +## Traceability Matrix + +| Req ID | Requirement | Design Ref | Impl Ref | Tests | Evidence | Status | +|-------:|-------------|------------|----------|-------|----------|--------| +| REQ-001–005 | Cross-project polling, scoring, selection, dedup, cleanup | Design: MISSING | `meta_coordinator.rs:174–314` | Internal — unreachable | `pub mod meta_coordinator` absent from `lib.rs` | ⚠️ WARN | +| REQ-006 | Full dispatch cycle | Design: MISSING | `meta_coordinator.rs:327` | No integration test | Module not wired to Dispatcher | ❌ FAIL | +| REQ-007 | Module in public API | N/A | `lib.rs` — absent | N/A | `grep meta_coordinator lib.rs` → 0 | ❌ FAIL | +| PH-H-001 | Graduated Guard: ExecutionTier | `learning-correction-System-plan.md §Phase H` | `guard.rs` — absent | N/A | File does not exist | ❌ FAIL | +| PH-H-002 | evaluate_command() | Same | Same | N/A | Same | ❌ FAIL | +| PH-H-003 | Firecracker sanDatabaseox integration | Same | Same | N/A | Same | ❌ FAIL | + +--- + +## Gap Summary + +| Gap ID | Description | Bug Reporting | Issue | Status | +|--------|-------------|----------|-------|--------| +| G-2026-05-07-1 | `meta_coordinator` not in `lib.rs` — all code dead | Blocker | #1275, PR #1291 | ❌ OPEN | +| G-2026-05-06-1 | `guard.rs` absent — Phase H Graduated Guard missing | Medium | #1274 | ❌ OPEN | + +--- + +## Recommendations (smallest first) + +1. **Merge PR #1291** — adds `pub mod meta_coordinator;` to `lib.rs`. One line. Makes all internal tests reachable. +2. **Fix `last_cleanup` mutation bug** — `dispatch_cycle` calls `cleanup_expired` but never updates `self.last_cleanup`, causing cleanup to run on every cycle after hour 1. Fix: return `last_cleanup` value and update it, or convert `&self` → `&mut self`. +3. **Add integration test for `dispatch_cycle`** — create `tests/meta_coordinator_integration.rs` verifying `NoIssues` and `AlreadyDispatched` paths without Gitea dependency. +4. **Implement `guard.rs` (Phase H)** — `ExecutionTier` enum, `GuardDecision`, `evaluate_command()` per spec. Self-contained module; no other gaps block it. +5. **Write `plans/design-meta-coordinator.md`** — document bounded context, scoring formula (−pagerank×100 + priority×10 + age×0.5), agent selection precedence, and TTL rationale. + +--- + +## Conclusion + +No spec regressions introduced since 2026-05-06. Two gaps persist: the `meta_coordinator` orphan (blocker, fix available as PR #1291) and the absent Graduated Guard module (medium, tracked in #1274). All other bounded contexts remain fully implemented and tested. + +**Verdict: FAIL — 2 open gaps (1 blocker, 1 medium)** + +--- + +Validated against commit `92b76de03` on branch `main`. Plans directory: 6 specs unchanged since 2026-05-04. diff --git a/reports/spec-validation-pr1291-20260507.md b/reports/spec-validation-pr1291-20260507.md new file mode 100644 index 000000000..6e0a47c31 --- /dev/null +++ b/reports/spec-validation-pr1291-20260507.md @@ -0,0 +1,39 @@ +

Requirements Traceability Summary

+ +PR #1291 — *Fix #1275: wire meta_coordinator module into lib.rs* + +**Scope:** Two-line change adding `pub mod meta_coordinator;` to `crates/terraphim_orchestrator/src/lib.rs` and updating the module-level doc comment. This single declaration was the only thing preventing the 741-line `meta_coordinator.rs` (added in commit `57594168`) from entering the compilation unit and the public API. + +--- + +

Verdict: pass

+ +The change is minimal, correct, and directly closes the blocker gap (META-001) identified in the spec validation report. Two pre-existing follow-up items are noted below; neither blocks merge. + +--- + +

Traceability Matrix

+ +| Req ID | Requirement | Design Ref | Impl Ref | Tests | Evidence | Status | +|-------:|-------------|------------|----------|-------|----------|--------| +| META-001 | `meta_coordinator` declared in `lib.rs` public API | Gitea #1275 | `lib.rs:47` (`pub mod meta_coordinator;`) | 5 tests in `meta_coordinator.rs`: `test_compute_score_pagerank_dominates`, `test_compute_score_priority_penalty`, `test_select_agent_security_checklist`, `test_dispatch_dedup` (async), `test_cleanup_expired` (async) | `grep "pub mod meta_coordinator" crates/terraphim_orchestrator/src/lib.rs` -> 1 match | PASS | +| META-002 | `MetaCoordinator::dispatch_cycle` integration invariant reachable | Gitea #1275 | `meta_coordinator.rs:327` | `test_dispatch_dedup` covers dedup path; cleanup path covered by `test_cleanup_expired` | Tests now compile with module declaration present | PASS | +| META-DOC | `plans/` spec document for MetaCoordinator bounded context | -- | ABSENT | N/A | No `plans/design-meta-coordinator.md` exists | FOLLOW-UP | + +--- + +

Gaps

+ +**Follow-up 1 -- Missing spec document (not blocking merge)** + +`plans/design-meta-coordinator.md` does not exist. The MetaCoordinator is a 741-line bounded context (scoring formula, agent selection rules, TTL dispatch dedup, PageRank integration) with no design artefact in `plans/`. Recommended: create a short ADR or design doc covering the scoring formula, agent selection precedence, and TTL rationale. Tracked via the existing recommendation in the spec validation report. + +**Follow-up 2 -- `last_cleanup` mutation bug (pre-existing, not introduced by this PR)** + +`MetaCoordinator` holds `last_cleanup: Instant` as a plain field. `dispatch_cycle` takes `&self` (immutable receiver), so `last_cleanup` can never be updated after `cleanup_expired()` is called. After one hour, the condition `self.last_cleanup.elapsed() > Duration::from_secs(3600)` is permanently true: cleanup runs on every call rather than every hour. Fix: wrap in `Arc>` or `AtomicU64` epoch, or change receiver to `&mut self` at the call site. This bug pre-dates PR #1291 and is a separate concern; it does not block this declaration-only change. + +No blocker gaps found in this PR. + +--- + +Last spec-validated commit: 925b4e0