Conversation
Keep the historical cache layout for the default service while assigning every custom registry a stable endpoint namespace. Apply the same namespace to downloads, offline lookup, verification, scanning, clearing, and download locks. Add regression coverage for cross-registry artifact isolation, legacy-cache handling, cache management, and verification.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Problem
HubApisupports selecting a registry withendpoint, but the physical cache identity currently contains only:The endpoint that supplied the bytes is not represented. If two compatible registries expose the same logical coordinates, the second request can return the first registry's cached artifact without making an artifact request to the current registry.
Minimal sequence:
cache_dir, repo id, revision, and file path with both clients.Before this change, the B request returns A's bytes and B receives no artifact GET. The behavior also reproduces in the reverse order. This allows content from one registry to remain effective after the caller switches to another registry using the same cache.
The regression test exercises the public
HubApi.download_file()path with two registry identities and endpoint-specific payloads.Implementation
The default service retains the historical path so existing ModelScope caches remain usable:
Custom registries use a SHA-256 namespace derived from their normalized endpoint:
Unscoped legacy caches are only reused by the default endpoint. Cache inspection, verification and deletion operate on the current client's endpoint namespace as well.
Users of custom endpoints will download artifacts once into the new namespace. Explicit
local_dirbehavior is unchanged because that path is caller-managed rather than an SDK cache.Verification
The issue was reproduced against both:
v0.4.5(c8f3505530a9a7eaa1b0a9eb8ac3c797f6c92965);main(0b2a3bacef4cbeccfd9a64545149d56cb1bb98a1).Three fresh loopback-registry processes per target reproduced both A-to-B and B-to-A reuse. In every baseline run, the second registry received zero artifact GETs. Three fresh verification processes with this patch returned the current registry's canary in both directions and issued one artifact GET to that registry.
Local checks: