Skip to content

feat: add support to wire webhooks from orchestrator to emitter workflows#70

Open
Kiran01bm wants to merge 6 commits intomainfrom
kiran01bm-webhook-changes-1
Open

feat: add support to wire webhooks from orchestrator to emitter workflows#70
Kiran01bm wants to merge 6 commits intomainfrom
kiran01bm-webhook-changes-1

Conversation

@Kiran01bm
Copy link
Copy Markdown
Contributor

@Kiran01bm Kiran01bm commented Apr 30, 2026

What and Why ?

Adds the detector → emitter webhook so the OrchestratorWorkflow can
notify the downstream emitter as soon as a snapshot is persisted,
instead of waiting for the emitter's own cron.

No breaking changes. No schema changes.
This work is purely additive at every layer that other systems can observe.

Auth model: this PR is wire-only. Network isolation between detector
and emitter could be enforced as required ranging from options like no-auth to
network policies in k8s all the way till the app layer authn and authz.

What's new (production code):

  • pkg/workflow/orchestrator/notify.go: NotifyEmitter activity. POSTs
    {"snapshot_id": ...} to /trigger-act with an
    injectable HTTPDoer so tests can swap in a fake (no live HTTP).
    Default client uses a 10s timeout.
  • pkg/workflow/orchestrator/workflow.go: OrchestratorWorkflow gains
    Stage 3 (NotifyEmitter), invoked only when EmitterWebhookURL is
    non-empty. Failures are non-fatal: the snapshot is already durable
    in S3, so a transient emitter outage just delays emission and
    Temporal's retry policy handles the rest.
  • pkg/workflow/orchestrator/activities.go: optional HTTPDoer field on
    Activities for test injection.
  • pkg/scan/scan.go: NewTrigger gains a 4th positional arg
    emitterWebhookURL plus a WithEmitterWebhookURL functional-option
    helper. WorkflowInput.EmitterWebhookURL is forwarded into the
    orchestrator workflow input.
  • pkg/schedule/schedule.go: Config.EmitterWebhookURL plumbed into the
    scheduled workflow input.
  • cmd/server/main.go: new EMITTER_WEBHOOK_URL env flag, registers
    NotifyEmitter activity, threads the URL through the schedule and
    the admin /scan trigger.
  • cmd/cli/main.go: passes "" so CLI runs stay detector-only.

Local testability (also production-safe; defaults preserve current
behavior):

  • pkg/snapshot/memory_store.go: in-process Store implementation for
    laptop dev / CI smoke tests. Selectable via SNAPSHOT_STORE=memory
    (default remains s3). Lets the orchestrator's Stage 2 succeed
    without AWS credentials.
  • cmd/server/main.go: when Wiz credentials are absent the wiz-source
    branch now falls back to pkg/inventory/mock.InventorySource (one
    synthetic resource per config) instead of skipping every resource
    and crashing with "no resources configured". Production paths set
    the Wiz secrets and remain unaffected — the misleading "using mock
    inventory" log line that already existed now actually does what it
    says.
  • Makefile: temporal-docker, webhook-e2e, webhook-e2e-smoke targets
    (all run the Temporal CLI / curl via Docker — no brew install temporal needed). make dev no longer hard-requires entr;
    auto-reload is used when entr is on $PATH, otherwise it falls back
    to plain go run ./cmd/server.

Backwards compatibility:

  • The only Go-level API change is pkg/scan.NewTrigger gaining a 4th
    arg. The package is internal to this private repo and all in-repo
    callers (cmd/server, cmd/cli) are updated.
  • WorkflowInput, Config, and Activities only gain new optional fields;
    zero-values reproduce the prior behavior.
  • New env flags (EMITTER_WEBHOOK_URL, SNAPSHOT_STORE) have safe
    defaults: empty webhook URL = Stage 3 skipped (legacy behavior);
    SNAPSHOT_STORE defaults to "s3".
  • No snapshot schema changes, no S3 layout changes, no Temporal
    workflow ID changes.

Tests: full unit suite green (pkg/scan, pkg/schedule, pkg/snapshot,
pkg/workflow/orchestrator, pkg/workflow/detection). NotifyEmitter has
7 tests covering success, 4xx, 5xx, network error, missing URL,
malformed body, and successful-but-empty body.

Local e2e workflow test confirmation

Screenshot 2026-04-30 at 5 06 28 pm

@Kiran01bm Kiran01bm changed the title feat: wire Stage 3 webhook from orchestrator to emitter feat: wire webhook from orchestrator to emitter Apr 30, 2026
@Kiran01bm Kiran01bm changed the title feat: wire webhook from orchestrator to emitter feat: add support to wire webhooks from orchestrator to emitter Apr 30, 2026
@Kiran01bm Kiran01bm changed the title feat: add support to wire webhooks from orchestrator to emitter feat: add support to wire webhooks from orchestrator to emitter workflows Apr 30, 2026
@Kiran01bm Kiran01bm marked this pull request as ready for review April 30, 2026 14:10
@Kiran01bm Kiran01bm requested a review from a team as a code owner April 30, 2026 14:10
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c0fdc2ec99

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pkg/schedule/schedule.go
Args: []interface{}{orchestrator.WorkflowInput{
ResourceTypes: cfg.ResourceTypes,
ResourceTypes: cfg.ResourceTypes,
EmitterWebhookURL: cfg.EmitterWebhookURL,
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Propagate webhook URL for existing schedules

EnsureSchedule only applies EmitterWebhookURL in the create-time action args, but the already-existing-schedule path does not compare or rewrite workflow args (it exits early when cron/jitter match, and update only touches spec/task queue). In an upgraded deployment where the schedule already exists, setting EMITTER_WEBHOOK_URL later will not reach scheduled orchestrator runs, so Stage 3 webhook notifications never fire unless the schedule is recreated manually.

Useful? React with 👍 / 👎.

@mpuncel
Copy link
Copy Markdown
Contributor

mpuncel commented Apr 30, 2026

🤖 Concern on the mock-inventory fallback in cmd/server/main.go (lines ~259-302):

The wiz-source branch used to log "Skipping {ID} – Wiz credentials not configured" and continue, ultimately failing loudly with "no resources configured". This PR replaces that skip with a synthetic resource (mock-{ID}-1, version 1.0.0, tag env=local-dev).

The PR description says "Production paths always set the Wiz secrets, so this branch is dev-only" — but nothing in the code enforces that. Any prod misconfiguration (rotated secret, IRSA mis-binding, partial rollout, secret-mount race on pod start) will now silently emit fabricated 1.0.0 findings into S3 instead of failing fast. Those snapshots feed every downstream consumer, including the new emitter webhook this PR wires up. Worst case: a green compliance dashboard built on synthetic data, with no obvious signal that the underlying inventory source is broken.

Suggested mitigation: gate the mock fallback on an explicit opt-in (e.g. INVENTORY_FALLBACK=mock env flag, defaulting off), or only allow it when SNAPSHOT_STORE=memory. Keep the loud failure as the default so a missing Wiz secret in prod still pages someone instead of poisoning the snapshot stream.

@mpuncel
Copy link
Copy Markdown
Contributor

mpuncel commented Apr 30, 2026

🤖 Mock inventory Engine field looks wrong for Aurora-style configs (cmd/server/main.go ~286):

Engine: resourceCfg.Type,

For Aurora, resourceCfg.Type is a registry constant like AURORA / AURORA_POSTGRESQL, but the EOL adapters in pkg/eol/endoflife/adapters.go key on lowercase-hyphenated engine values (aurora-postgresql, aurora-mysql). The mock will likely produce UNKNOWN classifications rather than the green 1.0.0 results the smoke-test screenshot suggests.

Worth verifying against the adapter dispatch before claiming the e2e screenshot demonstrates a working detector→emitter path — it may be exercising the wire but not the classification.

@mpuncel
Copy link
Copy Markdown
Contributor

mpuncel commented Apr 30, 2026

🤖 Constructor-signature inconsistency in pkg/scan/scan.go:

NewTrigger gained a positional 4th arg (emitterWebhookURL), but NewTriggerWithStarter did not. The new tests work around this with NewTriggerWithStarter(...).WithEmitterWebhookURL(...), which means there are now two different shapes for the same configuration:

  • NewTrigger(c, queue, defaults, url) — positional
  • NewTriggerWithStarter(s, queue, defaults).WithEmitterWebhookURL(url) — functional

Pick one. Either add the URL as a positional arg to NewTriggerWithStarter for consistency, or convert both to functional options (WithEmitterWebhookURL, WithDefaults, etc.) and keep the constructors minimal. Mixed shapes age badly — the next config field will face the same fork.

@mpuncel
Copy link
Copy Markdown
Contributor

mpuncel commented Apr 30, 2026

🤖 omitempty on snapshot_id is a misleading dead path (pkg/workflow/orchestrator/notify.go ~48):

payload, err := json.Marshal(struct {
    SnapshotID string `json:"snapshot_id,omitempty"`
}{SnapshotID: input.SnapshotID})

In the real flow, Stage 3 only runs after CreateSnapshot returns a populated SnapshotID — the empty case is unreachable in production. There's a test (TestNotifyEmitter_OmitsSnapshotIDWhenEmpty) asserting that the field is dropped when empty, with a comment "emitter falls back to latest", but that contract is neither documented in this PR nor tested against an actual emitter implementation.

Two options:

  1. Drop omitempty and always send the ID. Cleaner contract, the test goes away, the emitter never has to guess.
  2. Keep it, but document the "empty → latest" fallback explicitly in the godoc on NotifyEmitterInput and confirm the emitter side honors it.

Right now it reads like defensive code for a case that can't happen, which adds confusion for future readers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants