Add output analogous to exit code

[rethab]

Hi,

I'm using this action to validate whether the version in the openapi spec was updated correctly. So for example, I'd have a PR which does not bump the version inside the spec file, but it makes a change, which according to this action, is flagged as major. I then let the workflow fail. However, if the same breaking change was made alongside a version bump, then I'm fine with it.

I do this in my workflow by calling this action with continue-on-error: true and then a subsequent step check if the version was changed and fail the workflow if it did. See below for an excerpt of my workflow.

Now the problem is that this action fails both if it cannot parse the spec as well as if there is a breaking change. While it distinguishes between the two cases with exit codes (1 = breaking, >=2 = fail to parse), there's no way for me to detect which one it was. I want to make sure that if the parsing fails, the entire workflow should fail.

My idea is to add a new output (e.g. status: OK | BREAKING | ERROR) which I can then use to distinguish between the two cases.

      - name: Detect breaking change
        id: breaking
        if: github.event_name == 'pull_request' && steps.base.outputs.has_base == 'true'
        continue-on-error: true
        uses: oasdiff/oasdiff-action/breaking@v0.0.51
        with:
          base: base-spec.yaml
          revision: ${{ env.SPEC_FILE }}
          fail-on: ERR

      - name: Enforce version bump
        if: github.event_name == 'pull_request' && steps.base.outputs.has_base == 'true'
        env:
          BASE_V: ${{ steps.base.outputs.base_version }}
          HEAD_V: ${{ steps.base.outputs.head_version }}
          CHANGED: ${{ steps.diff.outcome == 'failure' }}
          BREAKING: ${{ steps.breaking.outcome == 'failure' }}
        run: |
          if [ "$CHANGED" != "true" ]; then
            echo "No meaningful spec change (cosmetic only, if any); no version bump required."
            exit 0
          fi
          if [ "$HEAD_V" = "$BASE_V" ]; then
            echo "::error::Spec $SPEC_FILE changed but info.version was not bumped (still $BASE_V). Bump it."
            exit 1
          fi
          if [ "$BREAKING" = "true" ] && [ "${HEAD_V%%.*}" -le "${BASE_V%%.*}" ]; then
            echo "::error::Breaking change in $SPEC_FILE; requires a MAJOR version bump ($BASE_V -> $HEAD_V)."
            exit 1
          fi
          echo "Version bump OK for $SPEC_FILE: $BASE_V -> $HEAD_V (breaking=$BREAKING)."

[reuvenharrison]

Thanks for the detailed writeup @rethab, and for pasting your workflow. It makes the goal really clear, and I suspect a lot of people hit the same thing.

Let's take your idea one step further. The real goal here is to enforce that info.version was bumped to match the severity of the change (a breaking change needs a major bump, any change needs some bump). Today that means assembling it yourself from the step outcome plus some shell version math. It works, but it's more moving parts than it should be, and everyone solving this is rebuilding the same glue.

oasdiff is in a good spot to do this directly, since it already knows both halves: the change severity, and info.version on each side. So rather than a status output that just helps people hand-roll the check, I'd like to put the check in oasdiff itself: an opt-in rule that flags when the version wasn't bumped appropriately, for example "breaking change but the major didn't increase," or "the spec changed but info.version is unchanged." It would flow through the normal --fail-on and reporting path, so the enforcement step collapses into enabling one check, from the CLI or any of the actions.

I'd like to build this for everyone who versions an API, so before locking the design I'd love input from anyone following this issue:

• Do you set info.version with semver (major.minor.patch), or dates / other schemes? I want to handle non-semver gracefully rather than error on it.
• Should the major rule follow strict semver, including the pre-1.0 case where a breaking change bumps the minor while still on 0.x?
• Beyond "breaking needs a major" and "any change needs a bump," is "an additive change needs at least a minor" something you'd want? That mapping is fuzzier, so I'd probably start with the first two rules and refine from there.
• Anything else you enforce around API versioning in CI today that this should cover?

@rethab in the meantime your current approach is fine. If telling a parse failure apart from a breaking result would help before this lands, a small status output (OK / BREAKING / ERROR) is cheap and I can add it as an interim. But the version check is the real fix, and I'd rather get it right with input from folks here.

I've opened oasdiff/oasdiff#1007 to track it. Design input welcome there too.