feat(installer): MDB-first install with standalone DBC flash - #5
Open
teal-bauer wants to merge 58 commits into
Open
feat(installer): MDB-first install with standalone DBC flash#5teal-bauer wants to merge 58 commits into
teal-bauer wants to merge 58 commits into
Conversation
Reorder install phases into two stages: a Stage-1 dashboardPrep step that runs Bluetooth pairing and keycard enrollment in the foreground while the DBC image/tiles upload in the background, then a standalone DBC swap-and-flash step. The Begin button unlocks once BT and keycard are done-or-skipped and the upload has completed. - Replace dbcPrep/dbcFlash phases with dashboardPrep and dbcSwapAndFlash; group BT+keycard under the new mdbPrep major step; make reconnect a hidden failure-resume phase. - Extract BT/keycard builders into content methods reused by both the standalone phases and the dashboardPrep screen (no duplicated UI). - Drop the host-side DBC-flash MDB reconnect poll on the happy path; the user confirms via the dashboard-lit-up button, failures route to verify. - Persist resume checkpoints (mdb-booted, dbc-staged, bt-paired, keycard-enrolled, trampoline-armed) to /data/installer/state.json. - Add l10n keys for the new phase/major-step labels.
…nup paths The hazards tails, signal_error_stop and stop-error-signals.sh deactivated the four blinker PWM channels. The imx_pwm_led active flag gates output, so a deactivated channel stays dark even while vehicle-service drives it, until vehicle-service re-inits. stop-error-signals.sh runs on reconnect/resume with nothing to re-activate afterwards, so blinkers could stay dead until the finish reboot. Set duty 0 but leave the channels active everywhere. The success path here never deactivates the blinkers (the progress bar that did was dropped), so this is the error/resume-path counterpart of the main fade_progress_off / stop-error-signals fixes.
Derive the offered map regions from the published OSM and Valhalla tile assets (intersection of both, so a region only shows when display and routing tiles both exist) instead of a hardcoded list. This surfaces Belgium, Netherlands, Luxembourg, Ile-de-France and Italy (northwest), which already had tiles but were never selectable. Group the region dropdown by country, and switch IP preselection from ip-api.com (HTTP, non-commercial only) to ipwho.is (HTTPS) keyed on the ISO 3166-2 region_code. Auto-detect covers the German states plus the neighbour regions.
The resolveRelease/buildDownloadQueue tests still mocked the old direct GitHub /releases endpoint and the in-resolveRelease stable->testing fallback, both of which were replaced by the per-channel latest.json manifest. Rewrite them against the current behaviour (manifest Map keyed by channel, fallback moved to channel selection) and clear the shared on-disk manifest cache between tests for isolation.
Releases now ship librescoot-unu-mdb-minimal-* / librescoot-unu-dbc-minimal-* sdimg assets alongside the full images. The unu-mdb-/unu-dbc- substring match classified these as full firmware too, so the installer could queue two "MDB firmware" downloads and flash the bootstrap image onto a scooter.
The step-1 wait required /sys/class/udc/ci_hdrc.0/state to read 'not attached', but the i.MX6 OTG port has no VBUS sensing from userspace, so that state never appears on unplug: the chipidea suspend irq parks the state at 'suspended' (an idle port reads 'default'). The trampoline hung in step 1 forever and the DBC flash never started. Exit when the state leaves 'configured' instead, held for 3 consecutive 1s samples so an RNDIS re-enumeration blip can't fire it early. Also kill any still-running trampoline instance before arming a new one, so a retry after a failed round can't stack two instances racing over the USB role and DBC power.
…lone-dbc-flash # Conflicts: # assets/trampoline.sh.template
- parse latest.json before writing cache so a captive-portal 200 can't poison it - guard both cache reads so a corrupt cache falls through to the bundled fallback - close the download sink in a finally so a mid-transfer error can't leak it - verify OSM/Valhalla tiles against GitHub's per-asset sha256 digest
- detect a declined pkexec/sudo prompt so a cancelled elevation shows the elevation-required dialog instead of silently exiting the app - write the macOS elevation launcher to a unique temp dir, not a fixed world-writable /tmp path (symlink TOCTOU before an admin-privileged run) - delete the Windows driver temp dir recursively so rndis.cat no longer leaks it - drop dead _sanitizeOutput - localize the unhandled-error snackbar instead of hardcoding German
- start(): verify the trampoline actually launched (pgrep) and throw otherwise, so a silent launch failure surfaces instead of a stuck screen - hold usb0 up during the pairing window by masking librescoot-ums at upload start, so the OTG UDC isn't grabbed away and SSH survives to the flash - skip path: guard restore_gadget with gadget_rescue_start and stop the watchdog - kill the journalctl capture on the fail, skip, and V3-success exit paths - treat a failed umount in grow_and_populate_dbc as fatal so the USB role is never flipped with the DBC partition still mounted
- treat a CHECKSUM MISMATCH from the flasher as fatal instead of only logging it, so a corrupt write can never report success - add set -o pipefail to the dd fallback scripts (run the Linux one under bash) so a dd/gunzip failure mid-write aborts instead of passing a boot-sector-only verify on a truncated image - cross-check the macOS flash target against the 0525/A4A5 USB identity and refuse to write if it can't be confirmed, so a second external disk can't be picked as the target
- close the SSH session on the timeout/error path in _runCommandOnce and downloadFile so a stalled command no longer leaks a channel until the connection's channel limit is hit - ping every 5s (below dartssh2's 10s default) through the idle pairing window
- drop the orphaned dd-phase flash helpers, the unused _reconnectToMdb and _batteryRemovalStarted, and the unused LaunchArgs import - debugPrint instead of print in network_service; escape angle brackets in two doc comments
|
|
Signal battery-service that the seatbox is open (stop vehicle-service so it can't re-assert the latch, then HSET vehicle seatbox:lock open and PUBLISH vehicle seatbox:lock) so it opens the main pack's contactors. Wait for battery:0 to leave the active state, with a 30s fallback into the UMS reboot which deactivates the pack regardless. Drops the physical-removal step and its now-dead openSeatbox helper.
Since the main battery is now deactivated in place rather than removed, it can be live after the reflash, and reconnecting the CBB onto a powered HV bus is unsafe. Move the CBB reconnect ahead of the AUX reconnect (both in the boot step, with a warning) so the CBB is connected while the scooter is fully depowered. The CBB-reconnect screen becomes verification-only: confirm CBB and main battery are detected, drop the physical reconnect and insert-battery instructions and the now-stale insert-battery step.
pkill -f matches the full command line of every process. The [t] bracket keeps the pkill token itself from matching, but the nohup clause on the same line contained the bare /data/installer/trampoline.sh path, so pkill killed its own launching shell before nohup ran. The trampoline never started, no log was written, and state.json still said trampoline-armed. Split the kill and the launch into separate SSH commands, and clear a stale trampoline-status when arming so an old result can't be read as the new run's outcome.
- Resuming into dbcSwapAndFlash now checks whether the trampoline process is alive and re-arms it when it is gone (launch failure, MDB reboot, crash) instead of sending the user to the swap-cables screen with nothing listening for the cable swap. - Walk-away screen: the success signal is the keycard LED blinking green, not the dashboard turning on. The dashboard cycles several times mid-install, and the fallback path finishes with the DBC off. - The headline no longer asks to unplug a laptop that is already unplugged when the screen appears. - Finish screen: skip the cable-swap steps and the MDB-side settings reset and reboot when arriving from the walk-away path. There is no MDB connection on that path; the stale SSH transport stacked multi-minute TCP timeouts on every step. - Resume text no longer claims the install restarts from the beginning. - Point the unknown-status hint at /data/installer/trampoline.log.
ipwho.is places Berlin consumer IPs in Hessen; ip-api.com resolves the same IP correctly. Its region field is the ISO 3166-2 subdivision code, so the existing geo map works unchanged. The free tier is HTTP-only and non-commercial, which is fine here: worst case a spoofed answer preselects a dropdown the user can change.
The boot LED guard now doubles as a progress display: amber pulse groups, one pulse per started quarter, fed from step boundaries and the block-device sector counter during the long image write. Green and red stay reserved for the terminal states. The success exits that skip the MDB reboot (V3 UMS tile copy, DBC already at target) left the scooter degraded: keycard/ums/pm stopped or masked and the installer-only overrides (usb0-policy=always-on, auto-standby=0, alarm off) persisted, so a walk-away user ended up with a dead keycard reader and no alarm. All success paths now restore the services and wipe the override settings. Starting keycard is safe in the MDB-first flow: cards are enrolled before the DBC is staged, so a master exists and auto-master-learn stays off.
Resuming into a post-flash phase now starts the keycard service when state.json says cards are enrolled; the resolver carries the bt_paired/keycard_enrolled flags through the trampoline phases so the resume handler can tell. Previously a resume that landed on the finish screen left the reader dead until the next reboot. Text continuity fixes from a full flow audit: - disconnectCbbDesc claimed the battery must be removed; the previous step only switches it off in place (safety-instruction mismatch) - the DBC prep strip showed "DBC flash successful!" once the upload to the MDB finished, before the flash ever ran; new dbcPrepComplete key for the staged state - sidebar step said "Remove Battery" for the deactivate-in-place phase - the disabled Begin button hinted "waiting for downloads" during the upload stage - unlockScooterDesc assumed keycards and Bluetooth were set up; both are skippable - reconnectUsbToLaptop now says to unplug the DBC cable first (the MDB port is occupied at that point) - finish-screen cable steps are phrased state-safely for the resume path, which cannot know whether the swap already happened
Non-release builds skipped the MDB cleanup entirely, so state.json stayed at trampoline-armed and every reconnect after a successful finish resumed to the finish screen as an unfinished install. Keep the images and logs for postmortem but remove the resume triggers. Also reword the reboot-wait screen: the old text read like an instruction to unplug USB, but the app is just waiting for the MDB reboot to drop the link on its own.
The trampoline skips the DBC flash when the target version is already installed. That makes full end-to-end testing impossible on an up-to-date scooter; this flag threads through to FORCE_DBC_REFLASH in the trampoline template and flashes unconditionally. Carried across the self-elevation relaunch like --dry-run.
When version-service data in Redis (version:dbc, written the last time the DBC booted Librescoot) shows the DBC already at the exact target version, ask the user: flash again or skip. Skipping reuses the existing skip-DBC path straight to finish, avoiding the pointless cable swap the trampoline's silent at-target exit required. Re-flash answers force FORCE_DBC_REFLASH into the generated trampoline, same as --force-dbc-reflash. With no version:dbc key (stock DBC, or no DBC boot since the last MDB reboot) nothing changes: the trampoline's own SSH check still handles the silent skip.
Success used to arm a persistent green blink that ran until someone power-cycled the scooter or the installer reconnected. With the success paths now restoring normal operation, the scooter is immediately usable, so signal completion by silence: the amber progress pulses stop and the LED goes dark. Red blink and hazards stay as the error signal.
…t unlock The DBC-flash walk-away screen now carries the full story: the install runs on its own for 10-20 minutes, progress reads as amber pulse groups on the keycard LED (one pulse after start, up to four near the end), done means the LED stops pulsing and stays off, and the getting-started/handbook block moves up so there is something to read in the meantime. The installer is done at this point; the primary button just continues to the finish screen. Bluetooth pairing no longer unlocks the vehicle. The nRF52 accepts bonding whenever it advertises without whitelist (its gate is the advertising mode, not the lock state) and the passkey reaches the installer via the ble hash, so pairing now pushes advertising-restart-no-whitelisting instead of scooter:state unlock. Unlocking also powered on the DBC, which at that point may still run the stock firmware. Done/skip restore whitelist advertising instead of re-locking.
Waiting users should land in the getting-started tips and handbook link right below, not be told to close the installer.
A scooter accidentally flashed with a minimal/bootstrap rootfs answers SSH but ships no redis, so the pre-flash lock (LPUSH scooter:state lock), the parked-state gate, and the health telemetry all die with "redis-cli: not found". The installer wedged on "interrupted install detected" with no way forward. Probe for redis-cli right after connect. When it is absent, mark the MDB as incomplete, skip the resume screen and unlock gate, and route to the health screen, which now shows a recovery banner and re-flashes the full firmware. The skip-MDB-reflash option is suppressed for this case (a minimal image reports as Librescoot but must be re-flashed). The pre-flash lock is also made best-effort so a stray exit 127 can never abort the connect flow.
AppImageKit is archived; appimagetool now downloads the AppImage runtime at package time (type2-runtime continuous) instead of embedding it. Its built-in fetch doesn't follow GitHub's 302 redirect to the asset CDN and fails intermittently, which is what killed the v1.0.31-beta.2 release build. Split the linuxdeploy deploy from the final packaging. linuxdeploy populates the AppDir, then we call appimagetool ourselves with a runtime pre-fetched via curl -L (which follows redirects) and handed over as --runtime-file, so appimagetool never downloads anything.
isSafeToFlash and verifyImage were both definition-only on this branch too. isSafeToFlash duplicated a weaker subset of validateDevice (no PID check, no path check) and verifyImage returned a digest nothing ever compared against, which couldn't have matched a sparse write anyway. Two functions that look like the safety net while validateDevice is the one actually doing the work is how the guard went unwired for months in the first place. Also fall back to the detector's currentDevice at the validateDevice call site: _device is set from the poll stream via setState, so it can lag the path we just resolved, and that lag would present as vendorId 0 and block a good flash. Both null still refuses, which is what we want. Add flash_safety_test.dart pinning the cases that must never pass.
Start-Process joins its -ArgumentList with spaces and adds no quoting of its own, so any argument holding a space arrived at the elevated process split in two. That silently breaks --mdb-image / --dbc-image paths picked out of a folder with a space in the name. Quote each argument per CommandLineToArgvW rules (doubling backslash runs in front of a quote, and a trailing run before the closing quote) before handing the array to PowerShell.
The log only lived in memory, so a user who hit a problem had nothing to send unless they thought to open the log dialog and copy it out first. Each run now appends to a timestamped file under Documents/Librescoot Installer (resolved via SHGetKnownFolderPath on Windows, so OneDrive known-folder redirection is handled), with per-platform fallbacks for the case where that folder cannot be written. Lines are written synchronously so the log survives a hang mid-flash, and the 20 most recent runs are kept. Elevation splits a run across two processes. The unelevated one picks the path and passes it down via --log-file, so one run stays one file, with every line tagged user or admin. It resolves the invoking user's home rather than root's, and hands ownership of anything the elevated process created back to the user so they can still read and delete it. The log dialog shows the path with a button to reveal it in the file manager.
The check shelled out to wmic, which is deprecated and absent by default on Windows 11 24H2, and every failure path returned "system disk". On those machines flashing was blocked outright with nothing the user could do about it. The parse was wrong even where wmic still runs: it searched the whole output for "Disk #N" and for "C:" independently rather than per row, so any disk carrying a lettered partition matched, which the gadget's FAT boot partition always is. Ask the storage stack instead: Get-Disk IsBoot/IsSystem, falling back to a CIM associator walk compared against $env:SystemDrive rather than an assumed C:. Disk 0 is still refused outright without probing. An inconclusive probe no longer blocks, since the vendor, product, size and path guards in FlashService all still have to agree before anything is written.
The build path starts the flash whenever it is neither running nor already started. The safety-check and no-device-path failures cleared exactly those two flags, so the next frame started the flash again, and again, several times a second, each pass stopping and restarting the USB detector. The retry button below was unreachable for the same reason: its condition could never hold during a build. Failures that a retry cannot clear on its own now block the phase and surface retry plus show log instead.
An inconclusive system-disk probe previously just proceeded on the strength of the vendor, product, size and disk-0 guards. Put it to the user instead, since they can tell their own drive from a 4 GB gadget at a glance. The verdict is now three-valued, so unknown is distinguishable from a confirmed no. On unknown, the flash stops and shows the detected disk with its model, size and physical drive path, alongside the other USB disks attached so there is something to compare against. Only the detected disk can be accepted, which keeps the vendor and product identity behind whatever gets written. Internal disks are never listed.
…lone-dbc-flash Picks up the macOS 26 USB detection fixes, which this branch needs as much as main does: without them the installer never resolves a flash target and sits on "waiting for device path" forever. Conflicts and how they went: lib/services/usb_detector.dart: took main's version whole. _findLibrescootUsbDiskNumber now asks ioreg first and only falls back to the system_profiler scrape, and parseIoregDiskNumber comes along with it. Nothing on this branch touched those functions. lib/main.dart: kept --force-dbc-reflash, which only exists here. lib/screens/installer_screen.dart: kept _buildDashboardPrep and the _resumeDecision field, which replaced main's _buildDbcPrep on this branch. The macOS 26 diagnostics snapshot merged in cleanly alongside them. assets/trampoline.sh.template: main replaced the hardcoded chipidea role path with a probed $ROLE_PATH, but step 5 has moved into do_dbc_flash_round() here, so that substitution was applied by hand at its new home. Both sides had separately added internalError and copyLog to the .arb files, in different places, so the merge silently kept both copies and the generated localizations declared each getter twice. Dropped the later pair and regenerated.
resolveTileAssets asked GitHub for releases/tags/latest. osm-tiles now publishes each build under its own tiles-<timestamp> tag so that asset URLs are immutable, and no longer updates 'latest', so that request returns a release frozen at the last build of the old scheme. Nothing fails: the assets are still there, so the installer would have kept flashing stale tiles indefinitely. Reads downloads.librescoot.org/releases/<repo>.json instead, which the manifest generator already keeps pointing at the current release and which this service already uses for firmware. Release resolution now lives in one place rather than being restated by each client, which is how these drifted apart to begin with. Tiles read the url field like firmware does; the manifest has no browser_download_url. A cache written by an older build holds a GitHub release object rather than a plain list, so both shapes are accepted and an upgrade does not discard tiles already downloaded. The existing asset helper emits both url and browser_download_url, so no current test could have caught this. The new one asserts the request goes to the manifest and that the URL names a specific build; it fails against the old code with 'manifest error ... 500'.
The tile source moved from the releases API to downloads.librescoot.org/releases/<repo>.json, and the two publish the sha256 under different names. The API gives "digest": "sha256:<hex>"; the manifest gives a plain "sha256" hex, the same field firmware already reads. _sha256FromAssetDigest only knew the first one, so every tile asset came back with a null checksum and downloadItem skipped verification entirely. Nothing fails, so tile integrity would just have stopped being checked. Renamed to _sha256FromAsset and taught it both shapes: the manifest field first, the digest form second so a disk cache written by an older build still verifies instead of silently going unchecked. The asset helper in the tests emitted browser_download_url alongside url, which is why no test noticed the URL field change either; the manifest has no such field, so the helper no longer invents one.
Every platform build failed while the test job passed: dartssh2-2.14.0/lib/src/sftp/sftp_client.dart:602:29: A value of type 'int?' can't be assigned to a variable of type 'num'. Context: Variable 'length' could not be promoted due to an 'await' Newer Flutter tightened type promotion and caught an old dependency. The analyzer and tests never see it because it only surfaces when the AOT compiler builds the package. dartssh2 2.22.5 fixes it but needs pointycastle 4.x, so both constraints move. The only direct pointycastle use is AES-CBC in ssh_service.dart (CBCBlockCipher, AESEngine, ParametersWithIV, KeyParameter), all unchanged in 4.x. Verified by reproducing the failing build locally: flutter build linux --release now succeeds, and the full test suite passes.
valhalla-tiles publishes both a plain tar and a zstd form, but the slug pattern is anchored to .tar so the compressed assets were invisible and the installer always fetched the largest file it downloads. For Berlin that is 202 MB against 62.7 MB. The tiles now go to the DBC compressed and are expanded there. Valhalla mmaps the tar so it cannot stay compressed, but decompressing on the DBC shrinks the upload over the vehicle's own network as well as the download, and avoids a native zstd dependency in a desktop app that ships for three platforms. zstd is in the DBC image and ums-service already installs routing tiles exactly this way. Regions are still derived from the plain .tar assets, so the catalogue is unchanged; only the queued download switches, and it falls back to the tar for any region with no compressed asset published. The remote filename follows whatever was downloaded, and the trampoline picks its path from the suffix. Traced through the generated script rather than assumed: the tile block is written into /data/onboot.sh by an unquoted heredoc, so trampoline variables expand at write time while onboot variables stay escaped. The generated onboot.sh keeps $DBC_IP and $((TILE_ERRORS + 1)) literal, bakes in the concrete tile path, selects the zst branch, and parses cleanly. Ported onto this branch's two-stage flow, which adds a second install path: grow_and_populate_dbc copies the tiles straight onto the DBC data partition over UMS. A plain cp there would have written the compressed file to tiles.tar and left valhalla with a tar it cannot mmap, so that copy now decompresses too. The MDB image carries no zstd, so it is guarded on command -v and returns non-zero when absent, which drops through to the onboot push that decompresses on the DBC. Checksums come from the manifest asset on this branch, and that applies to the zst asset unchanged, so no sidecar checksum name is reintroduced.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Reworks the installer into two stages so the host does all MDB-side work in one pass, then the scooter flashes the DBC standalone after a single cable swap, with no host reconnect on the happy path. Cuts the four cable swaps to two and takes the blind-RNDIS-reconnect path (the most field-broken part) off the happy flow.
dashboardPrepstep runs Bluetooth pairing + keycard enrollment (both MDB-side hardware) while the DBC image uploads in the background, then a singledbcSwapAndFlash.reconnectis demoted to a failure-only path./data/installer/state.jsonon the MDB drives precise, machine-independent resume; the scooter-writtentrampoline-statusstays the Stage-2 verdict.state); DBC identity read + idempotent skip when already at target; pre-write target sanity gate; retry-until-complete with a terminal short-circuit on the SDP/BootROM hard-brick.sh -nrender test guards the trampoline template against shell-syntax regressions.Reviewed commit-by-commit (spec + quality) and as a whole.
flutter analyzeclean (no new issues); all new unit tests pass. The 3 failingdownload_servicetests are pre-existing offline-network failures in untouched code.Test plan (on-hardware, cannot run in CI)
/datais full-size and holds tiles after flash; verify the data partition number (mmcblk3p4-> UMSsdX4) and that the MDB carriesparted/sfdisk/resize2fs. If grow-over-UMS is unreliable, confirm the onboot HTTP fallback still installs tiles.15a2:) condition short-circuits with the imx_usb_loader message.Notes / deferred
librescoot-ID only; on the stable channelVERSION_ID(1.0.1) will not equal the release tag, so it simply flashes (safe by default). Wiring the real DBC image version is a follow-up.