Audit Podcast Assistant operational knowledge for private-repository migration

[alexeygrigorev]

Audit Podcast Assistant operational knowledge for private-repository migration

Status: blocked — staged plan is agent-ready; waiting for HUMAN authorization of the private destination, access owners, and review workspace before any content inspection or migration
Tags: migration, assistant, podcast, docs, process-docs, backend, infra, testing, data, P0
Depends on: authorized private DataTalksClub/dataops-knowledge destination and access/retention decision [HUMAN]; #7 import is complete and is not reopened
Blocks: future production Podcast knowledge/runtime activation; does not block #128's Podcast-excluding shared Telegram MVP
Next owner: HUMAN repository/knowledge owner for the external Stage 0 prerequisite; after it is satisfied, the orchestrator dispatches Architecture, Security, Process Curator, and Assistant Engineer for the next agent-verifiable gates
Resume condition: a public-safe authorization record confirms that the destination exists and is private/authorized, names the destination owner and required reviewers/retention owner, confirms least-privilege access and the approved private review workspace, and records how the private inventory location will be exchanged without public exposure; then resume at Stage 1 metadata-only inventory without opening candidate content

Known finding and boundary

The only grooming-time inventory finding is 118 tracked paths under the existing Podcast Assistant operational-knowledge path categories: data/, knowledge_base/, process/, templates/, and podcast_examples/ within the assistant area.

No file content was inspected for this grooming. Individual filenames, private links, raw document text, screenshots, generated artifacts, and other potentially sensitive details must not be copied into this issue, commits, CI logs, or public fixtures.

The public repository may retain product/runtime code, tests, schemas, intentionally sanitized fixtures, and public-safe planning documentation. Raw SOPs, workflow templates, assistant prompts/process instructions, contact or credentials-adjacent notes, private links, sponsor/finance context, screenshots, and generated operational artifacts belong in the authorized private knowledge repository.

Unknown or ambiguous material is private-by-default until Security and the HUMAN knowledge owner approve a public-safe classification.

No-action boundary for grooming

Grooming authorizes no file-content inspection, checkout of private material, repository creation, access grant, cross-repository copy, deletion, rewrite, commit, push, runtime/configuration change, secret operation, deployment, or environment change. Do not modify the external source repository.

Stage 0 — HUMAN authorization and private workspace

• [HUMAN] Confirm that the destination repository exists, is private, and is the authorized system of record for operational knowledge.
• [HUMAN] Name the destination owner, content reviewers, Security reviewer, runtime consumer identity, retention owner, and incident/escalation owner.
• Security accepts least-privilege read/write roles, branch protection, review rules, encryption, backup/retention, auditability, secret scanning, and the approved private review workspace.
• Define how private source and destination paths/links will be exchanged without placing them in public issues, commits, CI, shell transcripts, or artifacts.
• Confirm whether legal, privacy, sponsor, finance, contact, or data-retention review is required before copying any content.

If the private destination or access controls are unavailable, stop. Do not “temporarily” move material to another public branch, issue attachment, object URL, local committed archive, or public CI artifact.

Stage 1 — metadata-only inventory

After Stage 0 approval, create a private manifest using tracked-path metadata without opening file contents:

• Assign every one of the 118 paths to its top-level category and record aggregate counts that reconcile exactly to 118.
• Privately record relative path, file type/extension, tracked state, byte size, last Git commit/time, and SHA-256; individual paths and digests stay private if names reveal operations.
• Identify code-referenced paths by inspecting public/runtime code references only; do not open the candidate knowledge files during this step.
• Record generated/binary/duplicate indicators using metadata and hashes, without rendering or parsing content.
• Publish only sanitized aggregate category/count totals and manifest completeness here.

Any count mismatch, untracked addition needed for completeness, unreadable artifact, submodule/link ambiguity, or evidence leak stops the stage for Security review.

Stage 2 — authorized private content classification

Content review occurs only inside the approved private workspace by authorized reviewers. Each manifest entry receives exactly one disposition and an owner/rationale:

1. Public product/runtime — code, schema, or non-sensitive product documentation required in the public repository.
2. Public sanitized fixture/example — demonstrably synthetic or irreversibly sanitized, minimal, licensed, and approved for testing/docs.
3. Private operational knowledge — SOP, workflow template, assistant instruction/prompt, private reference/link, contact, sponsor/finance context, screenshot, real example, or generated operational artifact.
4. Generated/duplicate/obsolete — not a knowledge source; retention/deletion disposition requires HUMAN approval.
5. Secret/incident quarantine — credential, token, personal/sensitive exposure, or material requiring rotation/incident handling; do not migrate as ordinary knowledge.
6. Unclear — quarantine privately and fail closed pending HUMAN/Security decision.

• Security and the HUMAN knowledge owner approve every public-safe disposition; filename/location alone is never sufficient.
• Assistant Engineer identifies which private items are semantically required for Podcast behavior, without copying their content publicly.
• PM records sanitized counts per disposition and unresolved count; unresolved must be zero before public cleanup.
• Any discovered credential or sensitive exposure follows the approved incident/rotation process. Do not quote it in the issue.

Stage 3 — private migration design and copy verification

• Define destination structure, ownership metadata, source provenance, versioning, review requirements, retention, archival rules, and license/attribution handling in the private repository.
• Copy before deleting. Preserve a private immutable pre-migration manifest and a destination manifest with byte size and SHA-256 for each transferred artifact.
• Verify one-to-one source/destination coverage, byte/hash equality where no transformation is intended, and separately approved lineage for redacted/transformed items.
• Keep raw and sanitized derivatives distinct; a sanitized public derivative must have a documented private source and Security-approved transformation.
• Verify private repository access with authorized and unauthorized identities without exposing repository URLs or identities publicly.
• [HUMAN] Approve the exact copy batch and destination commit/revision before it becomes the private source of truth.

Migration success is not inferred from a Git command exit code. Missing artifacts, hash mismatches, permissions drift, accidental public visibility, or unreviewed transformations cause ABORT.

Stage 4 — runtime and configuration cutover

Architecture must define one simple knowledge-provider boundary before public removal:

• Runtime code depends on a typed/versioned knowledge bundle or provider interface, not hard-coded private repository paths.
• Production uses an approved least-privilege delivery mechanism and a pinned immutable version/digest; credentials remain in the existing secret-management boundary.
• Public CI and forks use only minimal synthetic/sanitized fixtures and never clone, cache, index, log, or export private knowledge.
• Missing/unauthorized/private-bundle mismatch fails closed with a safe operator-visible error; there is no fallback to removed public operational content.
• Logs, traces, search indexes, assistant context, generated artifacts, exports, backups, and error messages are tested not to disclose private content.
• Cache invalidation, revocation, rotation, rollback version, availability, timeout, and cost limits are explicit.
• Podcast-specific runtime activation remains separate from Roll out and cut over the Telegram conversational MVP #128; this issue does not claim or enable shared Podcast execution.

Split implementation into separately owned public-runtime and private-knowledge changes if required by repository access. Reviewers must correlate them using sanitized bundle schema/version/digest metadata rather than exposing contents.

Stage 5 — public cleanup

Only after private copy verification and runtime/test readiness:

• Freeze the approved manifest and public cleanup diff; identify every removal/replacement and all code/test/docs references.
• [HUMAN] Approve the exact public cleanup batch. Deletion approval applies only to the named frozen batch.
• Remove private operational material from the current public tree and replace only what is necessary with approved schemas, interfaces, stubs, or synthetic fixtures.
• Update code, packaging, docs, tests, ignore rules, scanners, and contribution guidance so private material cannot be silently recommitted.
• Do not rewrite public Git history under this issue. Security separately assesses historical exposure and grooms any required history rewrite, takedown, credential rotation, or disclosure response as a destructive incident task.
• Do not delete the verified private source or migration evidence during public cleanup.

Rollout and rollback

Use a staged rollout with the old public-path dependency disabled only after the accepted private bundle/provider passes in a non-production environment.

• Pre-cutover rollback returns code/configuration to the last public-safe state or disables Podcast knowledge features; it must not restore classified private files to the public repository.
• Post-cutover rollback pins the last verified private bundle/provider version or disables the feature while preserving the private source and evidence.
• A rollback never copies private content into public CI, logs, artifacts, fixtures, or commits.
• Abort on missing/changed digest, access drift, leaked content, unexpected repository visibility, broken required behavior, unbounded logs/context, unavailable rollback, or an unapproved path/count difference.
• Cleanup of private staging data, old bundles, caches, or backups requires the retention owner's separate HUMAN approval after the observation window.

Verification plan

Inventory and migration

• Aggregate metadata inventory reconciles to exactly 118 original tracked paths, with no unclassified entry.
• Private source/destination manifests reconcile by disposition and expected transformations; hashes/sizes match where byte preservation is required.
• Destination visibility, access control, backup/retention, branch protection, and audit evidence pass Security review.

Public repository

• A clean checkout contains only approved public product/runtime code, schemas, documentation, and synthetic/sanitized fixtures for this scope.
• Scans detect no raw SOP/prompt/workflow content, private links, contact/sponsor/finance context, screenshots, generated operational artifacts, credentials, or destination identifiers introduced by the migration.
• Packaging, unit/integration tests, content validation, and public documentation checks pass without private repository access.
• Guard tests prevent reintroduction of prohibited path/content categories while allowing explicit reviewed public-safe fixtures.

Runtime

• Authorized non-production runtime loads the pinned private bundle and preserves required Podcast behavior.
• Unauthorized, missing, revoked, corrupt, stale, and digest-mismatched bundles fail closed.
• Logs, traces, assistant context, errors, search/index outputs, generated artifacts, exports, and backups contain no unauthorized private content.
• On-Call can observe version/digest and health in sanitized form, disable the feature, and roll back without repository or secret access beyond its role.

Role gates

1. HUMAN repository/knowledge owner authorizes destination, reviewers, access, retention, private inspection, exact copy batch, and exact public cleanup batch.
2. Architecture accepts the provider/bundle boundary, dependency direction, availability, and rollback design before implementation.
3. Security accepts classification rules, private access/evidence handling, sanitization, secret/incident handling, history disposition, and leakage tests before any content copy or public removal.
4. Assistant Engineer accepts behavior preservation and confirms no unreviewed operational content is embedded in prompts, fixtures, logs, or generated artifacts.
5. Software/Infrastructure Engineers implement only their approved repository/runtime slices without exposing private content.
6. Tester independently verifies inventory reconciliation, migration integrity, public-clean checkout, runtime authorization failures, leak prevention, and rollback.
7. PM accepts sanitized counts, resolved classifications, scope, operator clarity, and remaining risks.
8. On-Call verifies the staged runtime cutover and observation window; normal CI/CD remains the deployment path.

Material manifest, destination, schema, provider, access, or classification changes invalidate downstream approvals and return to the applicable gate.

Acceptance criteria

• HUMAN authorizes a private destination, access model, reviewers, retention, and private review workspace before content inspection.
• All 118 tracked paths reconcile through metadata inventory and exactly one approved disposition; public comments contain aggregate counts only.
• Private operational knowledge is copied and integrity-verified before any approved public cleanup.
• Public-safe code/schemas/fixtures are minimal, documented, and independently approved; unclear material fails closed.
• Runtime consumes an immutable authorized private bundle/provider without exposing the private repository or content to public CI.
• Public cleanup removes classified operational material and prevents reintroduction without rewriting history under this issue.
• Rollback never republishes private content and remains tested before cutover.
• Architecture, Security, Assistant Engineer, Tester, PM, HUMAN, and On-Call record the required decisions.
• Import Podcast Assistant into assistants/podcast #7 remains historically complete; Roll out and cut over the Telegram conversational MVP #128 remains Podcast-excluding; no production Podcast execution is inferred or enabled here.

Test scenarios

Destination is not authorized

Stop before content inspection or copying. Keep only the aggregate 118-path/category finding public.

A file looks public-safe from its name

Keep it private-by-default until authorized content review and Security/HUMAN approval establish that it is code/schema or safely synthetic/sanitized.

A credential or sensitive personal/finance item is found

Quarantine privately, invoke the approved incident/rotation process, and publish no value or excerpt.

Copy verification differs

Abort public cleanup, preserve both private manifests, and resolve every missing/hash/transformation discrepancy.

Private knowledge is unavailable at runtime

Fail closed or disable the Podcast knowledge capability; never fall back to removed public files or log their expected contents.

Rollback is required after public cleanup

Pin the last verified private bundle/provider or disable the feature. Do not restore classified operational files to public Git.

Out of scope

• Inspecting or publishing candidate content during grooming.
• Modifying the external Podcast Assistant source repository.
• Creating the private repository or granting access without HUMAN authorization.
• Public Git history rewriting, takedown, or secret rotation unless separately incident-groomed.
• Building or activating a shared Podcast execution plugin.
• Reopening Import Podcast Assistant into assistants/podcast #7 or blocking Roll out and cut over the Telegram conversational MVP #128's Podcast-excluding Telegram MVP.