Skip to content

External HIF programs#612

Open
lzrd wants to merge 21 commits intomasterfrom
stoltz/hif-asm
Open

External HIF programs#612
lzrd wants to merge 21 commits intomasterfrom
stoltz/hif-asm

Conversation

@lzrd
Copy link
Copy Markdown
Contributor

@lzrd lzrd commented Apr 8, 2026

Add a HIF assembler crate and extend humility hiffy with --exec,
--verify, and --assemble commands for writing, assembling, and
executing HIF programs loaded from text (*.hif) or pre-assembled
(*.hifb) files.

This PR was prompted by a need to do high-frequency I2C stress testing
and the desire to incorporate those tests into HWCI workflows.

Previously, HIF programs were hand-coded within the humility
source code. Being able to create HIF programs outside of humility
enables a faster way to write ad hoc tests and distribute them outside of
a humility release. This is especially useful for HW-based CI testing
where a HIF test can be decoupled from a humility release.

humility-hif-assembler crate: parses a text language with
syntactic sugar for:

  • I2C operations,
  • Idol RPC calls,
  • generic HIF function calls,
  • loops,
  • sleep, and
  • constants.

Anything symbolic and reply sizes are resolved against the given Hubris
archive's DWARF information. I2C devices can be referenced by part
name (e.g. i2c_read mid sbtsi reg=0x01 2) instead of hex address.

A ProgramBuilder API exists for Rust-driven program generation and
assembly-time stats computation.

humility hiffy subcommand extensions:

  • --exec program.hif assembles and runs a program on a target,
    with result decoding (Idol results get struct-level decode
    via DWARF, same as humility hiffy -c)
  • --verify program.hif checks a program offline (no target needed),
    shows stats and annotated disassembly with hex byte encoding,
    resolved function names, and I2C/Idol symbolic context
  • --assemble program.hif --bundle-output program.hifb produces a
    bundle file for CI artifacts
  • --json output for scripting

RFD 659

In trying to stay out of the way of any work-in-progress for RFD 659,
this crate intentionally overlaps with parts of humility-hiffy:

  • DWARF function discovery,
  • result decoding,
  • I2C parameter resolution

The overlap exists in part, because humility-hiffy is coupled to a live
target connection and cannot be used for offline assembly or scripted
test generation. The assembler uses only public HubrisArchive APIs.

When RFD 659 work lands, or as part of that work, TargetConfig
and HifAssembler are candidates for replacing the ad-hoc archive
introspection and per-command Op construction used across humility's
subcommands.

Pre-existing HIF programs are left as is. Many of them are just
one-liners, but all are duplicated in the tests in order to ensure
compatibility.

Testing

Tested on hardware: grapefruit (probe) and sidecar (network via NetHiffy):

  • I2C temperature reads,
  • fan controller reads,
  • I2C multi-bus operations (mux switching),
  • SpRot Idol calls, and
  • sensor polling.

There are Fixture tests, unit tests, and duplication of existing HIF
programs.

Fixture tests use checked-in JSON snapshots of the archived-derived
information from gimlet-c, cosmo-b, sidecar-b, and grapefruit archives.

Verified that all existing humility hiffy modes (-l, -L, -c) still work
with the CommandKind change to Unattached. The affected cargo tests that
match program output are updated.

Future Hubris changes

Several Hubris-side changes would improve what HIF stress tests can do.
Stress tests run many Ops that are successful but where returning
results uses up the rstack which limits the total number of Ops that the HIF
program can execute. However, failures are very interesting and those results
probably need to be returned along with overall stats.

Suggestions for some helpful HIF engine features:

  • Failure-only RSTACK mode and/or mark and pop for rstack:
    • A mode to record only errors, not successes, so programs can run
      thousands of iterations instead of ~250. A result filter callback in
      hif::execute() would be ~5 lines.
    • Implement rstack-mark and rstack-pop-to-mark PseudoOps
      that allow conditional popping of rstack contents.
  • Conditional result capture: new ops (PushResult,
    PushResultByte, ResetResult) that let the program inspect
    a function's return value and decide whether to record it.
    This turns hiffy into a programmable instrument.
  • Timing functions: GetTicks and ReadCycleCount HIF
    functions for measuring per-operation timing on-target.
  • I2C address-only probe: support zero-length I2C reads as
    an addressability check (currently rejected with BadArg).

Assuming this PR gets merged, a Hubris issue will be written for the HIF
enhancements.

mkeeter and others added 21 commits March 30, 2026 07:47
Hubris I2C stress testing requires sending looping HIF programs to
targets via the hiffy task.  Today, each humility subcommand (i2c,
pmbus, etc.) manually constructs Op sequences in Rust, making it
difficult to script test programs from external tools.

This crate introduces a text-based language for HIF programs with
syntactic sugar for common operations and an assembler that resolves
symbolic names against a Hubris archive.  The assembler validates
programs against target buffer sizes and embeds the archive's image
ID in the output bundle so the runner can reject mismatched uploads.

Key pieces:
- Text parser with syntactic sugar for i2c_read/write, idol calls,
  repeat loops, sleep, bus scan, and raw op blocks
- Single-pass assembler lowering to hif::Op bytecode
- Idol call lowering with scalar argument encoding
- TargetConfig: serializable intermediate form capturing buses,
  devices, muxes, sensors, HIF functions (with typed args and error
  codes), Idol interfaces, and buffer sizes
- Archive loader extracting TargetConfig from DWARF debug info
- Result decoder for postcard-encoded HIFFY_RSTACK output
- Verify mode reporting program fit and resource estimates
- Pre-generated JSON fixtures for gimlet-c and cosmo-b
- 9 annotated example programs covering common test patterns
- 59 tests (47 unit, 12 fixture-based)
Programs can now be executed on SP devices via `humility hiffy --exec
program.hif`.  The assembler produces the HIF bytecode; humility's
HiffyContext handles execution and result retrieval.  Results are fully
decoded using DWARF type info — Idol calls show struct-level output
identical to `humility hiffy -c`.

Tested on grapefruit (probe) and sidecar (network via NetHiffy):
I2C temperature reads, fan controller reads, multi-bus operations,
SpRot Idol calls, and sensor polling — 230 operations across
5 test programs with zero errors.

Key changes:
- Generic `call` statement for invoking any HIF function by name,
  eliminating the need for per-function sugar
- Fixed repeat loop counter pattern to match humility cmd/i2c
  (increment toward limit, not decrement)
- Idol reply sizes computed from DWARF during archive extraction
- `humility hiffy --exec` with full result decoding, JSON output,
  and --save-bundle for CI artifacts
- Assembly-time program stats (transactions, bytes, mux switches)
- ProgramBuilder API for Rust-driven program generation
- Archive validation: checks for hiffy task and net feature
- Sidecar and grapefruit fixtures; gimlet/cosmo regenerated
The hiffy command now supports offline operations that don't
require a target connection.  --verify shows program stats and
disassembled ops.  --assemble produces a .hifb bundle file.
Existing online operations (--exec, --call, -L) detect the need
for a target and attach on demand.

Changed hiffy's CommandKind from Attached to Unattached so
offline operations bypass target attachment.  Online operations
give a clear error if no probe or IP is specified.

Added 10 equivalent .hif examples that reproduce humility's
hardcoded HIF programs (qspi, i2c, gpio, hash, idol, validate)
and 18 tests covering both text and ProgramBuilder assembly.
Fixed block comment style to match the project convention.
I2C operations now accept device names from the archive's topology
in place of hex addresses (e.g. `i2c_read mid sbtsi reg=0x01 2`).
Names are resolved case-insensitively against the TargetConfig.
Numeric addresses still work.

The --verify disassembly now emits raw assembler syntax with
postcard byte encoding in comments and resolved function names.
The output is valid `raw {}` block syntax.

Updated existing tests and one example to exercise both named
and numeric address forms.
The --verify disassembly now annotates ops with their role in
context: I2C parameters are labeled with bus and device names
resolved from the archive.  Idol Send calls show the task name
and op_code.  Loop patterns are identified (loop_start, counter,
limit).

Updated README and lib.rs docs with annotated examples.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants