Skip to content

Give scenario factsheets their own URLs - #2444

Open
jh-RLI wants to merge 3 commits into
developfrom
feature-oekg-api-scenario-subresources
Open

Give scenario factsheets their own URLs#2444
jh-RLI wants to merge 3 commits into
developfrom
feature-oekg-api-scenario-subresources

Conversation

@jh-RLI

@jh-RLI jh-RLI commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Summary of the discussion

Sixth of the twelve OEKG REST API slices, and the last linear one — after it the work
fans into three independent strands. It gives scenario factsheets their own URLs and
lets a bundle be created with its scenarios in one call.

Why a scenario gets a URL and a framework does not is decided by the shape rather
than per class: a list of objects carrying its own oeo:OEO_00390095 (has uuid)
becomes a sub-resource; a set of IRIs with labels stays a field. ex:ScenarioShape
requires that uuid, ex:FrameworkShape does not.

Two asymmetries carry the slice.

A bundle POST builds its scenarios with it while a bundle PATCH cannot reach one.
That is what lets a pipeline create a whole bundle in one call without giving any call
the power to drop its parts by omitting them — in a graph an omitted field is
indistinguishable from a deleted one, which is today's data-loss mechanism. The
difference is structural, a serializer subclass for create, not a flag.

And the bundle owns the version, the ownership and the entity tag, because a scenario has
no independent existence to guard. A sub-resource read returns the bundle's ETag:
anything else would guarantee a 412 on the client's very next call. The accepted cost,
following from one version per bundle, is that two clients editing two different
scenarios of one bundle conflict — conflict granularity follows the aggregate root.

The two things underneath that had to change

The read went one hop deep. A scenario is one hop away, but its study region is two
and that region's reference is three. A one-hop read would have returned no regions and
hidden them from the validator
— so the slice that introduces regions would have shipped
with a post-state check blind to exactly the nodes it adds. It is bounded at three now,
the longest chain the shape permits. Deliberately not a transitive closure: that would be
an unbounded walk on a public endpoint, reachable by anyone who can put a triple in the
graph. A test pins the read and the post-state pruner to the same depth, which is the
diff's subtlest claim.

The field machinery was wired to the bundle as its subject. A scenario is the same
problem one level down — a closed set of fields, some literal, some picked from the
shape's own lists, some minted nodes — so field_triples, linked_field_triples,
resource_payload and resource_delta now work against a subject and a table. Without
that a scenario would have needed a second copy of the same logic, and the two would have
drifted. Two tables today; at four the machinery should move to a module of its own,
which the docstring says.

Three modules come out of it, because every sub-resource write does the same things in the
same order and gets them wrong the same ways: writes.py holds the sequence once,
reads.py the bounded read and the label lookup, preconditions.py the If-Match rules.
The bundle PATCH from #2438 rides the same spine rather than keeping a second copy.

Two defects the review found, both real

Both were reverted and watched failing before the fix was kept.

  • The rename refusal did not see regions. It was driven by a hardcoded list of bundle
    field names, so a payload could cite an existing region with a different label and be
    accepted — and the response then carried the stored label rather than the one sent, so
    the client was quietly told something other than what it asked for. It is driven by the
    field tables now: a list stops covering a table the moment that table gains a node
    field, which is exactly what happened, while the table cannot.
  • An unknown scenario answered 428/412 instead of 404, because the precondition
    was checked before the part's existence — sending a client to look for a version it
    could not use, for something that is not there. open_write is split into open_bundle
    and require_write so a sub-resource view can put its own existence check between them.

Also from the review: BundleWrite no longer describes a write that did not happen
(post_state is assigned only after the guard is confirmed, and a second apply()
refuses rather than recomputing from a stale pre-state); Refused became a DRF exception,
so five views stopped repeating a clause; and the scenario collection is paginated.

What this slice deliberately does not do

No DELETE. The spec's endpoint table lists one, but the slice ticket says
POST, GET, PATCH — and removing a part safely is a typed containment walk with a
guard clause whose allowlist "was already wrong once while it was being drafted". It
belongs to its own slice, built once rather than twice.

No dataset links. ex:ScenarioShape validates has information input/output; those
are the next slice's resource. The conformance test names them as deferred rather than
passing over them, so that slice deletes a list instead of discovering a gap.

One consequence worth reading before merging

Validating the post-state now reaches three hops, so a bundle carrying a pre-existing
shape violation two or three hops out will refuse an otherwise valid PATCH
, naming
violations the client did not cause. The live graph is known to carry such violations —
the user interface never types organisations, funders or contact persons.

This follows from WF-05's "validate the post-state, not the diff" and is not this slice's
to overturn. It does change what a deploy feels like: a bundle written only through this
API always conforms and is always patchable; one written through the browser may not be.
Three ways out are written down in the map and none is decided — that choice belongs
with whoever owns the live-data cleanup.

Type of change (CHANGELOG.md)

Features

  • Scenario factsheets are now addressable through the REST API: POST, GET
    and PATCH under /api/v0/scenario-bundles/<uid>/scenarios/. A bundle can
    also be created with its scenarios in a single call, which is what a modelling
    pipeline needs; a bundle PATCH still cannot reach into one, so no call can
    drop a scenario by leaving it out. Writes use the containing bundle's version
    and ownership, and a read of a scenario carries the bundle's ETag

Changes

  • GET /api/v0/scenario-bundles/<uid>/ now returns its scenarios nested under a
    scenarios key. Additive for a reader, and it is what lets a client send back what it
    read — the property the replace endpoint will depend on.

Bugs

None as a separate entry. Both defects fixed here were introduced and fixed inside this
PR, so recording them would tell a reader of the release notes that something they were
running had failed.

Removed

None.

Documentation updates

None, for the third slice running, and the gap is now the map's actual frontier. WF-13
(where the API description comes from) has been the next unblocked ticket since
2026-09-08 and blocks WF-14 and WF-15, while each slice adds endpoints that will have to
be described afterwards and drf-spectacular sits on an unmerged branch drifting further
from develop. Worth taking before slice 7.

Workflow checklist

Automation

Closes #2443

PR-Assignee

  • 🐙 Follow the workflow in
    CONTRIBUTING.md
  • 📝 Update the
    CHANGELOG.md
  • 📙 Update the documentation on
    mkdocs — deliberately not done,
    see Documentation updates above

Reviewer

  • 🐙 Follow the
    Reviewer Guidelines
  • 🐙 Provided feedback and show sufficient appreciation for the work done

Deployment

No migration, no new dependency, no configuration change. Unlike #2441, which needed
factsheet.0012, this slice adds endpoints only. The standing requirement from #2428 is
unchanged: python manage.py fetch_oekg_shapes must have run, or the bundle endpoints
answer 503.

Verifying the deploy — reads are public, so this needs no token:

curl -s https://openenergyplatform.org/api/v0/scenario-bundles/<uid>/scenarios/

A paginated body with count, next and results means it is live. An empty results
on a bundle the browser wrote is correct rather than a fault.

Reviewer's guide to the diff

Two commits: the slice, then the review's yield.

File What to look at
oekg/bundles.py The generalisation onto subject + table, and SCENARIO_FIELDS beside BUNDLE_FIELDS. BUNDLE_DEPTH and why it is 3.
oekg/writes.py The sequence every mutating endpoint shares: exist, own, precondition, validate the whole bundle, guard, read back, record. refuse_renames is table-driven for the reason above.
oekg/reads.py The bounded read. _query builds one UNION per depth; a test asserts it agrees with the pruner.
oekg/scenario_views.py The order in patch: bundle exists, then the scenario exists, then the precondition.
oekg/serializers.py ScenarioBundleCreateSerializer is where the create/patch asymmetry lives — a subclass, not a flag.
oekg/tests/test_scenario_api.py SharedRegionTest and ScenarioNotFoundTest are the two defects. ReadDepthTest is the depth claim.

Tests: 900 green (850 before this branch), against a real Fuseki and the fetched shape
artifacts. Without a store or artifacts the graph tests skip with a stated reason.

jh-RLI and others added 3 commits September 11, 2026 12:55
A scenario is a sub-resource because the shape says so: it carries its own
has-uuid, and that rule is checkable against the shape rather than negotiated
per class. So it gets a collection and an item URL supporting POST, GET and
PATCH -- and stays part of its bundle for everything else.

Two asymmetries carry the slice. A bundle POST builds its scenarios with it
while a bundle PATCH cannot reach one, which is what lets a pipeline create a
whole bundle in one call without giving any call the power to drop its parts by
omission; the difference is structural, a serializer subclass for create, not a
flag. And the bundle owns the version, the ownership and the entity tag,
because a scenario has no independent existence to guard -- every constraint in
the shape is bundle-local, so a scenario alone is not a unit the shape could
judge. A sub-resource read therefore returns the bundle's ETag: handing out
anything else would guarantee a 412 on the client's very next call.

The slice forced two changes underneath it.

The field machinery was wired to the bundle as its subject. A scenario is the
same problem one level down, so field_triples, linked_field_triples,
resource_payload and resource_delta now work against a subject and a table.
Two tables today; at four the machinery should move to a module of its own,
which the docstring says.

The read went one hop deep. A scenario is one hop away but its study region is
two and that region's reference is three, so a one-hop read would have returned
no regions AND hidden them from the validator -- the post-state check would
have been blind to exactly the nodes this slice adds. It is bounded at three
now, the longest chain the shape allows. Deliberately not a transitive closure:
that would be an unbounded walk on a public endpoint, reachable by anyone who
can put a triple in the graph.

Three modules come out of it, because every sub-resource write does the same
things in the same order and gets them wrong the same ways: writes.py holds the
sequence once (exist, own, precondition, validate the whole bundle, guard the
write, read back, record), reads.py the bounded read and the label lookup,
preconditions.py the If-Match rules. The bundle PATCH from the previous slice
now rides the same spine rather than keeping a second copy of it, and refusals
travel as an exception so no call site can forget to test for one.

A bundle GET nests its scenarios. That looks inconsistent beside a PATCH that
refuses them until you ask which write each is for: a read sent back to POST
copies the bundle whole, and the replace endpoint will want exactly this shape.
A PATCH is partial by nature and nobody sends a whole read to one.

The conformance test names the two predicates this slice does not build -- the
dataset links that are the next slice's resource -- rather than passing over
them, so that slice deletes a list instead of discovering a gap.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two real defects from reviewing the slice, both now held down by tests that
were watched failing against the reverted fix.

The rename refusal did not see regions. It was driven by a hardcoded list of
bundle field names, so a payload could cite an existing region with a different
label and be accepted -- and the response then carried the stored label rather
than the one sent, so the client was quietly told something other than what it
had asked for. It is driven by the field tables now: a list stops covering a
table the moment that table gains a node field, which is exactly what happened
here, while the table cannot. The rule moves next to the other write-path rules
so both write paths make it rather than one.

An unknown scenario answered 428 or 412 instead of 404, because the precondition
was checked before the part's existence. My own stated order is existence
first, and one level down I did not keep it -- which sends a client looking for
a version it could not use for something that is not there. `open_write` is
split into `open_bundle` and `require_write` so a sub-resource view can put its
own existence check between them; that also retires a boolean flag that meant
"actually, do not write".

`BundleWrite` no longer describes a write that did not happen: `post_state` is
assigned only after the guard is confirmed, and a second `apply()` refuses
rather than recomputing from a stale pre-state and silently undoing the first.
No caller does either today; both were one edit away.

`Refused` becomes a DRF exception, so five views stop repeating a clause to
catch it. The scenario collection is paginated -- a bundle bounds its scenarios,
but "no unbounded mode" is the rule for every public collection, and the nested
form in a bundle read is the deliberate exception because a client sends that
one back. Smaller: the table lookup is a cache rather than a list to maintain,
`referenced_node_iris` stops reaching for a key only the bundle table has, and a
flag that was never passed is gone.

One test added for the diff's subtlest claim, which had none: the read and the
post-state pruner walk to the same depth. Shorter and the shape would judge a
bundle the client never sees; deeper and a write could be refused over a node no
read returns.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Scenario factsheets as addressable sub-resources

1 participant