Security status report SKILL

[Michael Taufen (mtaufen)]

This is a (partially) vibe-coded skill that generates a report for each threat in the threat model, and renders a png graph to give a high level gut-feel overview of Substrate's current security posture, based on how well each threat model item has been addressed.

Now we can all run these reports to keep tabs on the rough status of the repo :). Expect some variation between runs, depending on model, harness, etc.

Example report json excerpt:

[
...
{
  "threat_id": "T-08",
  "threat": "Attacker who can create ActorTemplates can read or write any storage buckets atelet has access to.",
  "quality": 0.2,
  "strengths": "ActorTemplate spec is immutable post-creation, preventing retroactively modifying SnapshotsConfig.Location on existing templates.",
  "weaknesses": "ActorTemplate allows specifying arbitrary GCS bucket URIs in SnapshotsConfig.Location without validation or restricted bucket mapping. Atelet performs object storage operations using shared node/pod credentials rather than actor identity-derived credentials.",
  "citations": [
    "pkg/api/v1alpha1/actortemplate_types.go",
    "cmd/ateapi/internal/controlapi/workflow_suspend.go",
    "cmd/atelet/internal/ategcs/gcs.go"
  ]
}
...
]

And we can get nice charts like this. The amount of red is how bad we're doing ;). Everything else is "better than red."

Original chart for reference. Click to expand.

Original chart for reference below. The newer one above (e5b9bcd) does a better job of using the total surface area of a given color to indicate security posture.

[Michael Taufen (mtaufen)]

Dmitry Berkovich (@dberkov) addressed offline feedback, PTAL