Add versioned API for rack commissioning (RFD 710)

[andrewjstone]

This was a claude oneshot to copy over the existing wicketd APIs that are used in rkdeploy and then version them. It seems to have done this correctly.

Note that it uses types from wicket_common that are unversioned. If these change in the commissioned API, we will have to copy them to V1 and update the users. Claude knows how to do this. Doing it now would be overkill and would likely lead to a lot of From impls. Since we are just starting off, let's keep things simple for now and not change the internal types yet.

[iliana]

Can we wait to merge this until after #10755 lands? (Or remove /artifacts-and-event-reports from the list of APIs?)

[JustinAzoff]

We need artifacts-and-event-reports or a simpler api that returns just the fields we need for automating updates.

[andrewjstone]

Can we wait to merge this until after #10755 lands? (Or remove /artifacts-and-event-reports from the list of APIs?)

I don't think so. We need this API so that we can stabilize fridge, which is now required for racklette deployments, since rackletteadm is broken. Then we will have to add a new version to that API for #10755. Otherwise we'll end up with a flag day. I have had #10518 ready to merge for a week but have held off so that we could fix the versioning issue. Out of fairness, I think #10755 will have to get in line behind #10518.

This comment describes the situation a bit more.

And RFD 710 describes why we need this interface in the first place.

However, after doing this work, I"m not sure that we shouldn't just version wicketd itself. My gut is that were going to have a lot of duplicate interfaces.

I'd like to discuss this all in multirack huddle today.

[andrewjstone]

fields we need for automating updates.

Ignore my last (deleted) reply. I realize you were responding to iliana now.

[iliana]

I mean if we land this before #10755 it means we are immediately revving the API again, and we now have to keep around a bunch of types that are no longer relevant. It greatly complicates the Tufaceous v2 integration because these types refer to the old name/version/kind set that is no longer possible in the new repo format.

[iliana]

returns just the fields we need for automating updates

Can we land a change that removes the artifact types from this API then? What are the fields you need?

[andrewjstone]

I mean if we land this before #10755 it means we are immediately revving the API again, and we now have to keep around a bunch of types that are no longer relevant. It greatly complicates the Tufaceous v2 integration.

Yeah, I did plan on working on the rev 2 today building on top with the new BaseboardId changes required for #10518. However, I do realize that the TUF changes may be more difficult.

The problem is that rackletteadm/rkdeploy will literally not works after either of these PRs goes in. Maybe that's fine. Maybe they should just be updated to work with the latest stuff and then we do the versioning. This means those need to be fixed to deal with the breakage and then all outstanding PRs need to merge in latest main to get the updates from both of our PRs so the fixed tools will work.

I'm fine holding off in that case. Hell, I'd prefer it. It would give us more time to discuss the versioned API and I'd finally get to merge my change in. I'm a touch irritated I've already waited a week in this case, but whatever.

[JustinAzoff]

For artifact bits the main thing I need is the top level system_version to know what repo wicket has, if any.

As for the full list of artifacts, I don't think we really need any of that and we can parse it locally from the tuf repo if we do need it. The main consumer of this data currently is the fancy mupdate dashboard, which mostly replicates the wicket TUI across multiple racks. That isn't part of any of the automation and if it prevents things from moving forward we can remove it. The main thing we need to be able to automate is this:

given a TUF repo and a rack, determine if updating the rack to this TUF repo would result in any of the components being changed, and we need to be able to do this without uploading the TUF repo to the rack

[sunshowers]

I definitely want to avoid making the full API versioned -- I think that is exactly the issue we'd like to prevent via this interface. My hope is that we can provide a simple summary that rkdeploy can use. (Manufacturing works fine with that kind of summary.)

[sunshowers]

As for the full list of artifacts, I don't think we really need any of that and we can parse it locally from the tuf repo if we do need it.

If we don't need the full list of artifacts, let's drop it from the commissioning interface. Otherwise I agree with iliana that #10755 should land first and we should just deal with the breakage.

[JustinAzoff]

Note for the full artifacts-and-event-reports we need the event reports to monitor for the progress/completion of a mupdate. I outlined what I think a good simpler api is in https://github.com/oxidecomputer/rkdeploy/issues/99#issuecomment-4898848743

• Use case A: TUF repo version check - this is just the system_version field
• Use case B: Update progress monitoring (event reports) - complicated, need a simpler response type
• Use case C: Component version comparison - possibly not needed if we can fetch all this from the local repo.

[andrewjstone]

Note for the full artifacts-and-event-reports we need the event reports to monitor for the progress/completion of a mupdate. I outlined what I think a good simpler api is in oxidecomputer/rkdeploy#99 (comment)

* Use case A: TUF repo version check - this is just the `system_version` field

* Use case B: Update progress monitoring (event reports) - complicated, need a simpler response type

* Use case C: Component version comparison - possibly not needed if we can fetch all this from the local repo.

@JustinAzoff I think you are going to get your chance to make your ideal API right away.

The plan seems to be:

• Close this PR (it's too broad)
• Open PRs to:
  • Create an actual minimal, versioned commissioning API with only what you need
  • Change rkdeploy to use the new minimal commissioning API
  • Change fridge to use latest rkdeploy
• Have a flag day where:
  • We merge the above 3 PRs as well as Clean up and remove stale code #10518 and [RFD 621 2/n] switch to Tufaceous v2 in Wicket and Installinator #10755
  • Rename the existing fridge binary on castle to fridge-v21 or something
  • Publish a new binary with the latest PRs all merged

At the end of all this we'll have a private wicketd interface that is no longer in use by rkdeploy, a new versioned commissioning interface that we can change freely along with rkdeploy, and 2 versions of fridge on castle to cross the flag day. Given the new versioned API we shouldn't have a need for another flag day.

@sunshowers has volunteered to work on this minimal API and coordinate with @JustinAzoff to ensure that rkdeploy has what it needs. Justin will then simultaneously build out the rkdeploy changes.

I will then start on the next phase of multirack, which is #10637, and builds on #10518 on a branch. I can help as needed on this minimal API also.