remove_content() silently fails to update RepositoryContent when content queryset transitively depends on content_ids (>= 65,535 items)

[git-hyagi]

Version

pulpcore 3.113.0 (also affects any version with the content_ids memoization and the >= 65,535 DB subquery path in get_content)

Describe the bug

RepositoryVersion.remove_content() re-evaluates the content queryset parameter multiple times. When the queryset transitively depends on new_version.content (which reads content_ids from the database via a subquery for versions with >= 65,535 content items), the self.save() on line 1242 writes the updated content_ids to the database between evaluations. Subsequent evaluations of the same queryset resolve to empty because the removed items are no longer in content_ids, so the RepositoryContent delete/update queries on lines 1246-1255 become no-ops.

This leaves content_ids correctly updated but RepositoryContent entries orphaned (still version_removed=NULL), causing the _compute_counts() assertion to fail:

assert len(self.content_ids) == self._content_relationships().count()

The bug is triggered when a plugin's finalize_new_version builds a queryset from new_version.content and passes it to remove_content — e.g. pulp_maven's _generate_metadata removing stale metadata:

stale = MavenMetadata.objects.filter(
    pk__in=new_version.content,  # ← transitively reads content_ids from DB
    ...
)
new_version.remove_content(stale)

To Reproduce

1. Have a Maven repository with >= 65,535 content items
2. Have a second Maven repository with content (so finalize_new_version / _generate_metadata has stale metadata to remove)
3. Use modify with base_version pointing to version 1 of the first repository:

   POST /api/pulp/<domain>/api/v3/repositories/maven/maven/<target-repo-uuid>/modify/
   {"base_version": "/api/pulp/<domain>/api/v3/repositories/maven/maven/<source-repo-uuid>/versions/1/"}
   

4. Task fails with:

   File ".../pulpcore/app/models/repository.py", line 1442, in _compute_counts
       assert len(self.content_ids) == self._content_relationships().count()
   AssertionError
   

Expected behavior

remove_content() should update both content_ids and RepositoryContent consistently regardless of how the content queryset was constructed or how many content items are in the version.

Root cause

In remove_content(), the content parameter is evaluated three times:

• Line 1238 (to_remove = set(content.values_list(...))): before self.save() — correct
• Line 1248 (content_id__in=content): after self.save() — re-evaluates against updated DB state, resolves to empty
• Line 1254 (content_id__in=content): same problem

The issue only manifests when get_content() takes the >= 65535 code path (line 1013-1019), which uses a DB subquery (SELECT unnest(content_ids) FROM core_repositoryversion WHERE ...) instead of inlining the Python list. The DB subquery re-reads content_ids from the database on each evaluation, while the inline list path captures values at queryset creation time.

Suggested fix

Use the already-materialized to_remove set for the RepositoryContent queries instead of re-evaluating content:

content_ids = set(self._get_content_ids())
to_remove = set(content.values_list("pk", flat=True))
with transaction.atomic():
    if to_remove:
        self.content_ids = list(content_ids - to_remove)
        self.save()

    RepositoryContent.objects.filter(
        repository=self.repository,
        content_id__in=to_remove,    # ← use materialized set instead of content
        version_added=self,
        version_removed=None,
    ).delete()

    q_set = RepositoryContent.objects.filter(
        repository=self.repository,
        content_id__in=to_remove,    # ← use materialized set instead of content
        version_removed=None
    )
    q_set.update(version_removed=self)

Additional context

• pulp_maven's _generate_metadata (models.py lines 347 and 377) triggers this by passing a queryset built from new_version.content to remove_content. A plugin-side workaround is to materialize the queryset before passing it, but the core issue is in remove_content itself.
• Any plugin that follows the same pattern in finalize_new_version would be affected.
• Related: Data repair API #7272 (data repair / memoization hardening)