Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,45 @@ All notable changes to the [Nucleus Python Client](https://github.com/scaleapi/n
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.22.2](https://github.com/scaleapi/nucleus-python-client/releases/tag/v0.22.2) - 2026-09-01

### Added
- **`Model.model_runs()`.** Lists the ids of every model run for a model — the model-scoped counterpart to `Dataset.model_runs()`, which only lists a single dataset's runs. Pass `include_versions=True` to union runs across the model's version lineage (its version root and all descendants). Results are scoped server-side to runs on datasets you can read.

```python
run_ids = model.model_runs()
```

> **Server dependency:** requires the `GET /nucleus/model/:modelId/modelRun` route in scaleapi. Unit tests pass regardless; live calls 404 until that deploys.

## [0.22.1](https://github.com/scaleapi/nucleus-python-client/releases/tag/v0.22.1) - 2026-08-31

### Deprecated
- **`allowed_label_matches` on Evaluation V2.** `create_evaluation_v2_preset()`, `update_evaluation_v2_preset()`, `create_benchmark_evaluation_v2()`, and `Benchmark.create_evaluation_v2()` still accept `allowed_label_matches` / `allowed_label_matches_id` for backwards compatibility, but they now emit a `DeprecationWarning`. Use `rollup_groups` instead. `AllowedLabelMatch` and the corresponding fields on `EvaluationV2` / `EvaluationV2Preset` are likewise marked deprecated.

## [0.22.0](https://github.com/scaleapi/nucleus-python-client/releases/tag/v0.22.0) - 2026-08-26

### Added
- **Run-free ("model v2") predictions.** Predictions can now be uploaded and read directly against a `Model`, with no `ModelRun` or `Dataset` involved — the concept is `(model, dataset_item) -> prediction`. New methods on `Model`:
- `Model.upload_predictions(predictions, update=False, batch_size=5000, ...)` — upserts predictions onto the model (`box` / `polygon` / `cuboid` only), targeting `model/{id}/predictions`, and reusing the existing `PredictionUploader` batching machinery. Synchronous only for now: `asynchronous=True` raises `NotImplementedError`.
- `Model.predictions_loc(dataset_item_id)`, `Model.predictions_refloc(reference_id)`, `Model.predictions_iloc(i)` — model-scoped reads returning the same shape as their `Dataset` equivalents.
- `Model.copy_predictions_from_run(model_run_id)` — synchronously backfills the run-free store from an existing model run, returning a dict `{model_id, model_run_ids, predictions_copied, predictions_skipped_unsupported}`.
- **Model-anchored benchmark evaluations.** `NucleusClient.create_benchmark_evaluation_v2()` accepts a `model_id` (a `prj_*` id or a `Model`) as an alternative to `model_run_id`; the model-anchored flow evaluates the model's run-free predictions and ignores model runs. Provide exactly one of the two. `EvaluationV2` now exposes an optional `model_id` field alongside `model_run_id`.
- **`list_evaluations_v2` accepts a model.** `NucleusClient.list_evaluations_v2()` takes exactly one of `model_run_id` (`run_*`) or `model_id` (`prj_*` or a `Model`). The model-anchored path hits `GET model/{id}/evaluationsV2` and returns that model's run-free evaluations.

### Changed
- The existing run-based prediction paths (`Dataset.upload_predictions`, `ModelRun.add_predictions`, `create_benchmark_evaluation_v2(model_run_id=...)`) are unchanged and continue to work; the model-centric methods are purely additive.

### Deprecated
- **Model-run-anchored Evaluation V2 is deprecated** in favor of the run-free (`model_id`) path. `Benchmark.create_evaluation_v2()` gains a `model_id` argument (run-free anchor) to match `create_benchmark_evaluation_v2()`. Passing `model_run_id` to `create_benchmark_evaluation_v2()`, `Benchmark.create_evaluation_v2()`, or `list_evaluations_v2()` now emits a `DeprecationWarning`; all keep working. `EvaluationV2.model_run_id` is documented as deprecated (it is `None` on run-free evaluations). On the leaderboard, `LeaderboardRankingEntry` / `LeaderboardF1CurveEntry` `model_run_id` and `model_run_name` are deprecated and now `Optional` (they are `None` for run-free evaluations — previously `model_run_id` was a required field and would fail to parse), and `collapse="allRuns"` on `leaderboard_ranking()` is discouraged. Prefer anchoring on and identifying evaluations by `model_id`.

### Removed
- **`allowed_label_matches` removed from the EvaluationV2 surface** (breaking). The run-free (model-source) eval path — the one this SDK now steers toward — rejects `allowedLabelMatches` server-side (400, "use rollupGroups"); it only survives as a legacy fallback on the deprecated model-run path, where `rollupGroups` wins anyway. Removed the `AllowedLabelMatch` class (and its top-level export), the `allowed_label_matches` / `allowed_label_matches_id` arguments from `create_benchmark_evaluation_v2()`, `Benchmark.create_evaluation_v2()`, `create_evaluation_v2_preset()`, and `update_evaluation_v2_preset()`, and the `allowed_label_matches*` fields from `EvaluationV2` and `EvaluationV2Preset`. Use `rollup_groups` (:class:`RollupGroup`) exclusively.
- **`dataset_id` dropped from the EvaluationV2 surface** (breaking). An evaluation is no longer anchored on a single dataset — a model run now carries a *set* of datasets and a benchmark's items may span several — so the backend no longer returns a denormalized dataset on evaluations or leaderboards. Removed `EvaluationV2.dataset_id`, and `dataset_id` / `dataset_name` from `LeaderboardRankingEntry` and `LeaderboardF1CurveEntry`, matching the current backend responses. Without this, `EvaluationV2.from_json` raised `KeyError: 'dataset_id'` on every model-anchored (run-free) benchmark evaluation, since those payloads never carry a `dataset_id`.

### Fixed
- `Model.predictions_loc` / `predictions_refloc` / `predictions_iloc` now actually parse their responses. The run-free read endpoints return a flat `{"predictions": [...]}` list (each element carrying its own `"type"`), but `format_prediction_response` only understood the legacy type-keyed `{"annotations": {"box": [...]}}` shape, so these methods returned the raw payload unparsed instead of the documented `{"box": [...], "polygon": [...], "cuboid": [...]}` dict.

## [0.21.2](https://github.com/scaleapi/nucleus-python-client/releases/tag/v0.21.2) - 2026-08-17

### Added
Expand Down
Loading