Skip to content

AR: Agent Registry & Marketplace - Publish and Fork#215

Open
rgu855 wants to merge 3 commits intomainfrom
feat/marketplace-publish-fork
Open

AR: Agent Registry & Marketplace - Publish and Fork#215
rgu855 wants to merge 3 commits intomainfrom
feat/marketplace-publish-fork

Conversation

@rgu855
Copy link
Copy Markdown

@rgu855 rgu855 commented Apr 18, 2026

Summary

Self-serve publish & fork for the Commonly agent marketplace. Users can publish agent manifests under their own namespace (@username/name), version them, fork published manifests, and manage lifecycle (unpublish/delete/deprecate). This is effectively ADR-001 Phase 2 for marketplace operations — the first dual-write path between Installable (canonical) and AgentRegistry (compat shim).

Design doc: docs/design/marketplace-publish-fork.md

Changes

New file: backend/routes/marketplace-api.ts (+603 lines)

9 endpoints under /api/marketplace:

Endpoint Method Auth Description
/publish POST Publish new manifest or push new version
/publish/:id DELETE Soft-delete (unpublish) — hides from browse, existing installs unaffected
/manifests/:id DELETE Hard delete — only if 0 active installations (live count, not cached)
/fork POST Snapshot-copy a manifest into your namespace
/publish/:id/deprecate POST Mark a specific version as deprecated
/browse GET Browse published manifests (filter by kind/category/q, sort, paginated)
/manifests/:id GET Full manifest detail (readme, components, versions, fork lineage)
/manifests/:id/forks GET List all forks of a manifest (paginated)
/mine GET User's own manifests (including unpublished)

Key implementation details:

  • Dual-write shim: syncToAgentRegistry(installable, action) maps Installable fields to AgentRegistry on every write. Write order: AR first (load-bearing), then Installable. Failure matrix from §6c.1 implemented.
  • Namespace validation: @scope must match req.user.username (server-enforced). Bare names reserved for builtins.
  • Runtime mapping: ComponentRuntime (7 values) → manifest.runtime.type (3 values) via RUNTIME_MAP.
  • Route ordering: forks sub-route registered before detail route to prevent wildcard (*) param from consuming /forks.
  • Username resolution: JWT auth only sets req.user = { id } (no username), so resolveUsername() falls back to a DB lookup.

Schema changes: backend/models/Installable.ts

  • installableId regex relaxed: /^[a-z0-9-]+(\/[a-z0-9-]+)?$//^(@[a-z0-9-]+\/)?[a-z0-9-]+$/
  • Added forkedFrom?: { installableId, version, forkedAt } (fork lineage)
  • Added readme?: string (long-form description for detail page)
  • Added stats.forkCount (default 0)
  • Added indexes: forkedFrom.installableId, text search on name + description + marketplace.tags

Schema changes: backend/models/AgentRegistry.ts

  • agentName regex relaxed: /^[a-z0-9-]+$//^(@[a-z0-9-]+\/)?[a-z0-9-]+$/

Install route: backend/routes/registry/install.ts

  • Relaxed inline agentName regex to match the schema change
  • Added status guard: rejects installs when agent.status === 'unpublished' (HTTP 410)

Server wiring: backend/server.ts

  • Mounted marketplace-api router at /api/marketplace (alongside existing marketplace router — no path conflicts)

Test fix: backend/__tests__/unit/models/installable.test.ts

  • Updated test installableId values from old scope/name format to new format (bare-name for builtins, @scope/name for marketplace)

Validation

Input validation (§6e)

Field Rule Enforced
installableId Required, @username/name for user manifests, max 64 chars
name Required, max 100 chars
description Max 500 chars
version Required
kind Required, enum: agent/app/skill/bundle
scope instance rejected for user manifests
readme Max 50,000 chars
components Max 50 per manifest
Namespace @scope must match authed user's username

Manual API testing (local Docker Compose)

All 9 endpoints tested end-to-end against real MongoDB:

Test Result
Publish new manifest (@testuser/my-agent v1.0.0) ✅ 201 Created
Push new version (v2.0.0) ✅ 200 OK, version appended
Idempotent re-publish (v1.0.0 again) ✅ 200 OK, no-op
Fork manifest (@testuser/my-agent-fork) ✅ 201, forkedFrom recorded
Deprecate version (v1.0.0) ✅ deprecated: true
Browse published manifests ✅ items returned, versions[] excluded from projection
Manifest detail ✅ full doc with readme, components, versions
List forks of manifest ✅ fork returned
My manifests ✅ publisher's manifests returned
Unpublish (soft delete) ✅ status → unpublished
Hard delete (0 installs) ✅ document removed from both tables
Verify deleted returns 404
Dual-write: AR has correct data ✅ agentName, latestVersion, runtime.type all match
Validation: name > 100 chars ✅ 400 rejected
Validation: invalid kind ✅ 400 rejected
Auth: JWT-based auth resolves username from DB

Automated tests

Test Suites: 15 passed, 15 total (registry + install + installable + marketplace)
Tests:       40 passed, 40 total

Key suites:

  • installable.test.ts — 12/12 pass (updated installableId formats for new regex)
  • self-serve-install.test.js — 5/5 pass (ADR-006 flow unaffected)
  • marketplace.official.test.js — 1/1 pass (existing browse unaffected)
  • registry.runtime-tokens.test.js — 6/6 pass (token flow unaffected)
  • All other registry tests — pass

Regression checks

  • Existing GET /api/marketplace/official endpoint: unaffected (separate router, no path conflicts)
  • Existing POST /api/registry/install flow: works — dual-write ensures AR has the data, status guard added for unpublished manifests
  • Existing 18 seeded agents: unaffected — bare names match both old and new regex
  • ADR-006 self-serve webhook install: unaffected — 5/5 integration tests pass
  • Agent runtime tokens: unaffected — 6/6 tests pass

Architecture notes

  • Why dual-write? AgentRegistry is the sole runtime catalog today. The install flow reads it exclusively. Until ADR-001 Phase 3 switches reads to Installable, we must keep both in sync.
  • Why AR first? If only one write succeeds, an installable-but-not-browsable manifest is better than a browsable-but-not-installable one.
  • Why snapshot fork? Git semantics — fork diverges freely, no forced upstream sync. Simpler model, avoids upstream-push complexity.
  • Why source: 'marketplace' for forks? ADR-001's template source means admin-cloned templates, not user-to-user forks. Both source and fork are published catalog entries.

Follow-up work (not in scope)

  • Semver validation on version strings
  • Per-component type/runtime validation
  • Express body-parser 1MB limit on marketplace routes
  • Reconciliation cron for Installable ↔ AgentRegistry drift (§6c.1)
  • Frontend marketplace UI
  • ADR-001 Phase 3: switch install read path from AgentRegistry to Installable

🤖 Generated with Claude Code

@rgu855 rgu855 self-assigned this Apr 18, 2026
Copy link
Copy Markdown
Contributor

@samxu01 samxu01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Structurally sound RFC — good precedents cited, explicit failure matrix, and clear open questions. Requesting changes because several concrete claims and field mappings conflict with the existing Installable / AgentRegistry schemas and would block implementation as written.

P0 (blockers):

  • §5.3 namespace regex doesn't match either existing schema's field constraints — first publish would fail Mongoose validation.
  • §6c.2 asserts AgentRegistry.agentName accepts arbitrary strings; the schema's regex explicitly rejects @ and /.
  • §6c.3 maps unpublish to AR status: 'deprecated' when 'unpublished' is already a valid enum value.
  • §6b / §6c.3: install route changes are understated — a status guard plus (probably) input-validation relaxation are required, not zero changes.

P1 (worth tightening before coding):

  • "Latest" version resolution and Installable.version scalar update rules undefined.
  • NFR-2 "identical content" needs a concrete hashing spec.
  • stats.activeInstalls === 0 hard-delete guard races against new installs.
  • §6c.4 field map skips the hardest three cases (runtime, persona.systemPrompt, categories shape).
  • Fork request version semantics ambiguous (forks-from-old-version vs. initial-version-of-fork).
  • Fork source: 'marketplace' vs ADR-001 'template' unreconciled.
  • HTTP 207 is unconventional; prefer 201 + warnings payload.

Strengths worth keeping: write-ordering rationale in §6c.1, lifecycle sync table in §6c.3, server-enforced namespace ownership, and identity-continuity callouts.


Generated by Claude Code

Comment thread docs/design/marketplace-publish-fork.md Outdated
Comment thread docs/design/marketplace-publish-fork.md Outdated
Comment thread docs/design/marketplace-publish-fork.md Outdated
Comment thread docs/design/marketplace-publish-fork.md Outdated
Comment thread docs/design/marketplace-publish-fork.md Outdated
Comment thread docs/design/marketplace-publish-fork.md Outdated
Comment thread docs/design/marketplace-publish-fork.md Outdated
Comment thread docs/design/marketplace-publish-fork.md
Comment thread docs/design/marketplace-publish-fork.md
Comment thread docs/design/marketplace-publish-fork.md Outdated
/**
* POST /api/marketplace/publish
*/
router.post('/publish', auth, async (req: any, res: any) => {
Comment on lines +100 to +285
router.post('/publish', auth, async (req: any, res: any) => {
try {
const userId = req.userId || req.user?.id;
const username = await resolveUsername(req);
if (!userId || !username) {
return res.status(401).json({ error: 'Unauthorized' });
}

const {
installableId, name, description, version, kind, scope,
requires, components, readme, categories, tags,
} = req.body;

if (!installableId || !name || !version || !kind) {
return res.status(400).json({ error: 'installableId, name, version, and kind are required' });
}

if (installableId.length > 64) {
return res.status(400).json({ error: 'installableId must be 64 characters or fewer' });
}
if (name.length > 100) {
return res.status(400).json({ error: 'name must be 100 characters or fewer' });
}
if (description && description.length > 500) {
return res.status(400).json({ error: 'description must be 500 characters or fewer' });
}
if (readme && readme.length > 50000) {
return res.status(400).json({ error: 'readme must be 50,000 characters or fewer' });
}
if (!['agent', 'app', 'skill', 'bundle'].includes(kind)) {
return res.status(400).json({ error: 'kind must be one of: agent, app, skill, bundle' });
}
if (components && components.length > 50) {
return res.status(400).json({ error: 'Maximum 50 components per manifest' });
}

const nsError = validateNamespace(installableId.toLowerCase(), username);
if (nsError) {
return res.status(400).json({ error: nsError });
}

if (scope === 'instance') {
return res.status(400).json({ error: 'scope "instance" is reserved for admin/builtin manifests' });
}

const existing = await Installable.findOne({ installableId: installableId.toLowerCase() });

if (existing) {
if (existing.publisher?.userId?.toString() !== userId.toString()) {
return res.status(403).json({ error: 'Not authorized to update this manifest' });
}

const existingVersion = (existing.versions || []).find(
(v: any) => v.version === version,
);
if (existingVersion) {
return res.status(200).json({
success: true,
manifest: {
installableId: existing.installableId,
version,
status: existing.status,
isNew: false,
},
});
}

existing.versions = [
...(existing.versions || []),
{ version, publishedAt: new Date() },
];
existing.version = version;
existing.name = name;
existing.description = description || existing.description;
if (components) existing.components = components;
if (requires) existing.requires = requires;
if (readme !== undefined) existing.readme = readme;
if (categories || tags) {
existing.marketplace = {
...existing.marketplace?.toObject?.() || existing.marketplace || {},
published: true,
category: categories?.[0] || existing.marketplace?.category || '',
tags: tags || existing.marketplace?.tags || [],
};
}

// Write AR first (load-bearing table), then Installable — same
// ordering rationale as new-manifest path (§6c.1).
try {
await syncToAgentRegistry(existing, 'publish');
} catch (arError) {
console.warn('[marketplace] AgentRegistry sync failed on update:', (arError as any).message);
return res.status(201).json({
success: true,
warnings: ['AgentRegistry sync failed; retrying publish will fix.'],
manifest: {
installableId: existing.installableId,
version,
status: existing.status,
isNew: false,
},
});
}

await existing.save();

return res.json({
success: true,
manifest: {
installableId: existing.installableId,
version,
status: existing.status,
isNew: false,
},
});
}

// New manifest: write to AgentRegistry first (load-bearing table), then Installable
const installableDoc = {
installableId: installableId.toLowerCase(),
name,
description: description || '',
version,
kind,
source: 'marketplace' as const,
scope: scope || 'pod',
requires: requires || [],
components: components || [],
readme,
marketplace: {
published: true,
category: categories?.[0] || '',
tags: tags || [],
verified: false,
rating: 0,
ratingCount: 0,
installCount: 0,
},
publisher: { userId, name: username },
status: 'active' as const,
versions: [{ version, publishedAt: new Date() }],
stats: { totalInstalls: 0, activeInstalls: 0, forkCount: 0 },
};

// Write AR first
try {
await syncToAgentRegistry(installableDoc, 'publish');
} catch (arError) {
console.error('[marketplace] AgentRegistry write failed:', (arError as any).message);
return res.status(500).json({ error: 'Failed to publish manifest' });
}

// Then Installable
let created;
try {
created = await Installable.create(installableDoc);
} catch (installableError) {
console.warn('[marketplace] Installable write failed (AR succeeded):', (installableError as any).message);
return res.status(201).json({
success: true,
warnings: ['Installable catalog write failed; manifest is installable but not yet browsable. Retry publish to sync.'],
manifest: {
installableId: installableDoc.installableId,
version,
status: 'active',
isNew: true,
},
});
}

console.log(`[marketplace] action=publish user=${userId} manifest=${created.installableId} version=${version}`);

res.status(201).json({
success: true,
manifest: {
installableId: created.installableId,
version: created.version,
status: created.status,
isNew: true,
},
});
} catch (error) {
console.error('[marketplace] publish error:', error);
res.status(500).json({ error: (error as any).message || 'Failed to publish' });
}
});
* DELETE /api/marketplace/publish/:installableId
* Soft-delete (unpublish)
*/
router.delete('/publish/:installableId(*)', auth, async (req: any, res: any) => {
Comment on lines +291 to +316
router.delete('/publish/:installableId(*)', auth, async (req: any, res: any) => {
try {
const userId = req.userId || req.user?.id;
const { installableId } = req.params;

const doc = await Installable.findOne({ installableId: installableId.toLowerCase() });
if (!doc) return res.status(404).json({ error: 'Manifest not found' });
if (doc.publisher?.userId?.toString() !== userId.toString()) {
return res.status(403).json({ error: 'Not authorized' });
}

doc.status = 'unpublished';
if (doc.marketplace) doc.marketplace.published = false;
await doc.save();

try { await syncToAgentRegistry(doc, 'unpublish'); } catch (e) {
console.warn('[marketplace] AR sync failed on unpublish:', (e as any).message);
}

console.log(`[marketplace] action=unpublish user=${userId} manifest=${installableId}`);
res.json({ success: true, status: 'unpublished' });
} catch (error) {
console.error('[marketplace] unpublish error:', error);
res.status(500).json({ error: 'Failed to unpublish' });
}
});
* DELETE /api/marketplace/manifests/:installableId
* Hard delete (only if 0 active installs)
*/
router.delete('/manifests/:installableId(*)', auth, async (req: any, res: any) => {
const sortObj = sortMap[sort] || sortMap.installs;

const [items, total] = await Promise.all([
Installable.find(filter)
.skip(skip)
.limit(limitNum)
.lean(),
Installable.countDocuments(filter),
Comment on lines +570 to +580
router.get('/manifests/:installableId(*)', async (req: any, res: any) => {
try {
const { installableId } = req.params;
const doc = await Installable.findOne({ installableId: installableId.toLowerCase() }).lean();
if (!doc) return res.status(404).json({ error: 'Manifest not found' });
res.json(doc);
} catch (error) {
console.error('[marketplace] detail error:', error);
res.status(500).json({ error: 'Failed to get manifest' });
}
});
/**
* GET /api/marketplace/mine
*/
router.get('/mine', auth, async (req: any, res: any) => {
Comment on lines +585 to +599
router.get('/mine', auth, async (req: any, res: any) => {
try {
const userId = req.userId || req.user?.id;
if (!userId) return res.status(401).json({ error: 'Unauthorized' });

const items = await Installable.find({ 'publisher.userId': userId })
.sort({ updatedAt: -1 })
.lean();

res.json({ items, total: items.length });
} catch (error) {
console.error('[marketplace] mine error:', error);
res.status(500).json({ error: 'Failed to list your manifests' });
}
});
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants