WIP: halui: pin-only GUI bridge surface (halui side of the #4613 split)

[grandixximo]

Status: WIP sketch, not for merge. Nothing here is agreed yet; looking for @c-morley's take on the direction and on how to cooperate before going further.

This is what the halui side of the split I proposed on #4613 could look like, working and live-tested. It gives an external GUI bridge component everything it could need through HAL pins, so halui itself carries no python, no ZMQ and no new dependencies:

• halui.gui.* IN request pins (cycle-start/pause, softkey.00-19, response.ok/cancel, reload-preview, shutdown, mdi-command.) for a bridge to poll and forward; mdi/macro names are read from the INI directly ([MDI_COMMAND_LIST], [MACROS]/[DISPLAY]), same naming the GUIs use today
• halui.axis.jog-speed-angular; rotary axes ([AXIS_x]TYPE=ANGULAR) jog at the angular rate
• two-way jog-rate sync: halui pulls gui_bridge.jog-rate(-angular) by name (hal_getref_p) and arbitrates against its own IN pins, most recent change wins; effective rates are published on halui.gui.jog-rate(-angular) OUT. No bridge loaded: behavior exactly as before. Bridge (re)start: no jump
• tool.number and axis.selected stay uint

171 lines in halui.cc instead of 655; the whole embedded-interpreter class of problems reviewed on #4613 would be gone by construction. The ZMQ message protocol the GUI sides speak could stay unchanged, so the qtdragon/gmoccapy/axis work in #4613 could carry over as-is; bridge.py could move out of halui into a normal userspace comp (same shape as the existing hal_bridge.py).

Loading and startup order: none needed in either direction. halui late-binds to the bridge pins (verified: clean idle until the bridge appears, re-bind after bridge restart), and the bridge side tolerates halui being absent. The comp could be auto-loaded by the run script next to the existing HALUI= block, gated on an INI entry; qtdragon configs already carry a commented HALBRIDGE= line as precedent for INI-loaded bridges. No user pin wiring anywhere.

Live test (headless sim + python stand-in for the bridge): pin inventory, INI mdi names, IN-pin behavior without bridge, no-jump on bridge appear, bridge writes win linear and angular, later IN edit wins, graceful bridge death, restart baseline. 9/9.

Still missing: the gui_bridge comp itself (thin: owns its IO pins, polls halui.gui.* and the halui state pins, speaks the existing ZMQ protocol), axis-selection/mpg-aux channels (same pattern as jog-rate), a real GUI wired to it, tests.

@c-morley if the direction looks right to you, two ways we could cooperate, your pick:

A. I open this as a PR against your fork's zmq-halui-gui branch. It replaces the halui.cc embedded-python parts of #4613; your GUI sides and the ZMQ half of bridge.py stay. #4613 becomes: pin-only halui + bridge comp + your GUI work.

B. This goes to master as a standalone PR (halui side only; with no bridge loaded behavior is unchanged), bridge comp and GUI sides follow, and #4613 either rebases onto it or closes.

Docs (halui.adoc + halui.1) included. Happy to adjust pin naming to match whatever the GUI sides already expect.

[grandixximo]

Mechanism question where I'd like maintainer input: @BsAtHome @rene-dev

The GUI-to-halui channel needs one bridge-owned value per synced setting (jog rate shown as the example). halui pulls it cross-component by name with hal_getref_p; nothing is netted, nothing is user-wired. Two candidate shapes for that value:

1. HAL_IO pin. Conventional for runtime values, observable, setp-able when unlinked. But it is linkable, so a user can net an internal channel that exists only so halui and the bridge can find each other by name. The mechanism still works if they do (getref follows the signal), but it invites wiring I would rather not support.

2. Param. Cannot be linked at all (HAL has no signals for params), which makes "internal use only" structural rather than documented. Still resolvable via hal_getref_p with the same late-bind and comp-death detection, still setp-able for testing. Cost: params are conventionally configuration, not runtime channels, and they are less visible (no show pin listing).

HAL offers no way to actually hide a pin or param, so the name will be visible either way; the question is only whether "users should not connect this" can be enforced. My lean is param, precisely because connection becomes impossible, but I am aware "runtime value as param" is not the usual pattern.

Which shape would you pick here? Is param-as-runtime-channel acceptable practice, or is IO pin plus "do not connect" documentation the lesser evil?

[BsAtHome]

Other places this is states as:

hal_query_t q = {};

No need to use memset i C++.

[BsAtHome]

You assume that target is HAL_REAL but do not verify the q.pp.type field.

Could be fine, but also bears risks.

[BsAtHome]

Small issues besides, I have a hard time understanding the layering of the system and everything we are patch-fixing and patch-adding makes it harder to understand.

Using HAL means that it only runs on the machine that runs LCNC's core. We normally use NML to communicate with the GUIs, which has some severe shortcomings. We also have HAL pins that convey information between GUI and ..., right, yet another communication channel. Now we introduce ZMQ too and layer on both HAL and NML, sometimes on top, sometimes on the side, sometimes parallel and it might get under it as well?

This is just increasing the mess instead of cleaning up. Most of us probably agree on that we need to replace NML. But, so far, I have not seen the list of requirements what we actually need and what we want to achieve with the replacement. Architectural changes need to be planned so we do not run into the same mess down the road. Every addition we make to the old base will probably break when we kill NML. Some real planning helps us to reduce the extent of the break, or, will allow us to have a better future path without sprouting wild patches for the foreseeable future.

[grandixximo]

Fair points, and #4571 is the right frame for them. Where this PR sits relative to each:

No new channel in core. This PR removes one instead: the embedded-python/ZMQ channel of #4613 leaves halui entirely. What remains on the halui side is plain HAL state that any present or future transport can serve. ZMQ would live only in a small userspace comp outside core, same shape as the already-merged hal_bridge.

NML-agnostic. The diff touches no NML code and extends no NML usage, so nothing here breaks when NML dies.

halui-in-task compatible. The whole change is pin exports, one arbitration function and INI reading; it ports into whatever component task becomes when halui moves in. The pin names and semantics are the durable piece users see, and they are transport-independent. If rene's branch lands first I will happily rebase onto it.

Future UI API compatible. When the flatbuffers/websocket API from #4571 lands, GUIs get a first-class channel and the bridge comp is simply deleted. The HAL surface stays because panels and user HAL need it regardless of what transport GUIs use.

On requirements: agreed the project needs the written list before the NML replacement gets designed, and #4571 is where that belongs. This PR does not try to answer that question; it tries to be a small step that survives every plausible answer to it.

Side note for the open remote-HAL question in #4571: pull-by-name with late binding, which this PR uses for the sync direction, is the same shape a remote-HAL bridge would need.

[BsAtHome]

It is exactly the interaction with HAL that I worry about.
HAL was originally designed to be the RT interface. Now it has become a go to place for all kinds of communication that requires a constant polling interface. That is generally bad design in a multi tasking system. You want to handle that using events (on file descriptors).

[grandixximo]

You are of course describing the correct design, and that is exactly how big rewrites start around here. Last time the interim-versus-proper question came up we got a whole new HAL API. Great work, but my rebase queue still remembers it, so let me be careful what I ask for.

On the substance: agreed, events on file descriptors is the right pattern for this kind of traffic, and HAL has no such API today. I would rather not grow one as a side effect of a jog rate. The pull rides halui's existing 50 Hz loop, costs half a microsecond per value, and disappears quietly the day the #4571 API makes it unnecessary. It is a stopgap that knows it is a stopgap.

And if HAL ever does grow subscription semantics, this is the kind of surface they would serve first: slow, event-shaped values that today get polled because there is no other option.

[c-morley]

Well this seems the best way to get the behavior I'm trying for.
I appreciate your enthusiasm and trying to help.

I would think hal_glib should poll the pins, IIRC there is now no need of a HAL component just to read pins.
This would eliminate the need for ZMQ. it was only used between hal_glib and HALUI.

in HALUI
-axis_select still needs to be sint as I need away to identify no axis selected and MPG selected.
-Not need for this work but should really consider changing toolnumber to sint for practical reasons (as we are breaking configs anyways)

[grandixximo]

Route C works for me, and it simplifies everything downstream. If hal_glib (and each GUI generally) polls halui's pins directly, the bridge comp and ZMQ both disappear: nothing to load, no ports, no protocol. The halui side in this PR does not change at all; it pulls names and exports pins, and does not care who owns the pulled pins.

Naming: the pull targets just need a stable convention. One process can own several HAL components (verified: python hal.component("gui") plus hal.component("gui_bridge") side by side, pins resolve fine), so a GUI can create a small conventional component named gui_bridge next to its own and export jog-rate there. halui then pulls the same names no matter which GUI is running. An INI-given prefix is the alternative, but the fixed convention keeps configs identical across GUIs, which seems worth more.

axis.selected as sint: accepted. -1 for none and 100 for MPG then need documenting in halui.adoc so the encoding becomes part of the pin contract.

tool.number as sint: what is the practical case for negative tool numbers? If it is "no tool", T0 already carries that. Changing uint to sint breaks every existing link to that pin at load time, so I would want the use case to be real before paying it.

@BsAtHome @rene-dev this is where the shape has landed, and I would like your explicit read on it as the interim:

• halui exports panel-facing pins and pulls a few conventional values by name. That is HAL doing its original job: the local panel interface. Nothing new in HAL, no API change, no comp, no ZMQ, no NML touched.
• GUIs poll and write pins they own, the existing hal_glib precedent (iocontrol/spindle pins today).
• The arbitration logic is 40 lines that port unchanged into task when halui moves in (NML replacement #4571), and the pin names stay the same so panels and configs never notice.
• When the NML replacement #4571 API exists, GUIs swap their pin poller for the real API and halui's surface goes back to panel-only duty. Nothing to unmerge.

Can you two live with that as the stopgap?