Setup CI to publish a python package to PyPi

[machshev]

DVSim should be installable as any other Python package using pip install. While it is possible to reference a git repository directly, in general it's more convenient to publish packages to PyPi.

• Determine the semver package version from commit messages: Utilization of python-semantic-release for versioning and publishing releases #16
• Update CI to build and upload a new package if the version has changed

[machshev]

The issue we have at the moment is #16 doesn't work on master due to the branch protection rules. It seems adding a commit to master to update the version in pyproject.toml is not easy. The other option would be to move to dynamic version in the pyproject.toml and establish the package version from the release tags (which is the source of truth for python-semantic-release anyway).

Proposal: drop the requirement for the commit to master and just use tags as the single source of truth for package version.

[project]
dynamic = ["version"]

https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#static-vs-dynamic-metadata

[rswarbrick]

My proposal:

Stick something in the "merge my changes" process (I don't know how GitHub names this) so that:

• It works through the commits since master (a bit like git rebase -i master..HEAD) and calls python-semantic-release version --no-commit and then git commit --amend each time.
• Once that has run, we'll have fudged things so that the version in pyproject.toml always matches the logical version (not just on PR heads)
• At this point, the repository (which I assume is the thing that gets merged to the "upstream repo") is a fast-forward from origin/master and has been updated correctly.

Wouldn't that work?

[rswarbrick]

About making releases on GitHub, you could then presumably get it to automatically create a GitHub release for the version that we end up with at the head of each PR

[rswarbrick]

@KinzaQamar and I have just spent some more time (and she has taught me a bit about GitHub workflows!)

New plan:

• Add a pre-commit hook script somewhere in the repository that basically runs semantic-release --no-commit; git add pyproject.toml
• Add some documentation that tells contributors to symlink .git/hooks/pre_commit to it.
• Add a check in CI to make sure the contributor remembered to run it! This could use git rebase -i -exec ??? origin/master where ??? is something like semantic-release --no-commit; git status --porcelain | (check this is empty).

I think this will work nicely! (Probably...)

[machshev]

@rswarbrick It's a nice idea, but not sure it will work? That only checks that the PR is correct at the time of submission. If there are several PRs going in parallel they will all have the same version (which is calculated relative to the version tags in the repo).

At the point the first PR is merged then the version may be correct? (probably not if any other PR has been merged after it's creation). But all the following PRs will now be incorrect, even if the CI has passed.

At best this means versions are missed, at worst it means versions could actually go backwards.
In either case, the publish to pypi stage would fail as it will not accept pushing a package with the same version as an existing package.

This is a fundamental problem with trying to calculate versions in a PR CI workflow, there will always be that race condition. As far as I can tell the only reliable way of doing this would be to do it on a merge to master, in a guaranteed sequential operation.

[rswarbrick]

Gah, good point. I don't think I can see a way for the merging operation to futz with the thing that's being merged though. Maybe the right thing to do is to duplicate the third point (the "check in CI") so that it runs just as the serialization happens?

I'm wondering about the merge_group action ?

[rswarbrick]

If we made that change then you couldn't merge two concurrent PRs that both bumped the version. Somewhat irritating, but just a case of rebasing the later guy and re-pushing (and it's not like we're imaging a CI run to take hours).

[rswarbrick]

Also, it's not too bad. If there are PRs A and B, the author can choose to do the serialization manually by making B include A and adding a note in the PR to merge A first.