Skip to content

PVA reader qMonitor queue + ioc CLI command#113

Merged
pecomyint merged 9 commits into
mainfrom
dev
Jul 15, 2026
Merged

PVA reader qMonitor queue + ioc CLI command#113
pecomyint merged 9 commits into
mainfrom
dev

Conversation

@Osayi-ANL

@Osayi-ANL Osayi-ANL commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator

What this does

  • Faster, crash-proof PVA reader. Frames now land in a queue that a
    background thread processes, so receiving and decoding no longer fight over
    one thread. Under a flood, old frames are dropped instead of crashing the
    viewer. Replaces the old fixed-rate throttle.
  • New DashPVA ioc command. Starts the RSM-parameter IOC simulator.
    --prefix (default pvapy) sets the PV prefix; --no-gui runs it headless.

Also included

Two earlier dev fixes not yet on main: the "all constants live in settings.py"
change, and a guard so a cache-save ValueError no longer crashes the viewer.

Bump to version 1.0.3

Testing

  • Live test against the sim server on area detector: frames decode correctly, no
    drops at normal speed, threads shut down cleanly.
  • With a live detector move your detector angle fast - DashPVA should not crash
  • 175/175 unit tests pass; ruff clean.

The network thread enqueues frames into a bounded PvObjectQueue and a consumer thread drains them, dropping the newest frame under overload instead of overrunning the monitor thread. Adds PVA_MONITOR_QUEUE_SIZE/PVA_MONITOR_REQUEST.
--prefix defaults to pvapy (the module appends the trailing ':'); --gui/--no-gui toggles the Qt control GUI, with --no-gui running the IOC headless via the module's --ioc-mode.
The reader subscribed with a bare 'field()' request, leaving the
pvAccess monitor queue at the tiny default so high-rate frames were
dropped (uniqueId gaps) before reaching the client. Add a tunable
PVA_MONITOR_SERVER_QUEUE_SIZE composed into the request as
record[queueSize=N], separate from the client-side queue.

@pecomyint pecomyint left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work. The qMonitor + consumer-thread approach is the right call (matches Sinisa's suggestion). It's thread-safe (queued Qt signals, no cross-thread widget access), the start/stop lifecycle is clean, and it clearly drops fewer frames than main. Three things before I can merge:

  1. Blocker — client queue default is too large. PVA_MONITOR_QUEUE_SIZE = 15000 is bounded at depth × frame size, but 15k real detector frames is ~10-30 GB compressed (far more uncompressed). Please drop it to a burst-sized default (~256-512); dropping sooner under sustained overload is the intended behavior.
  2. Descope — pull the DashPVA ioc CLI command into its own PR. This is a safety-critical reader change and I'd like the diff tight and focused; the ioc command is unrelated (and ideally split the AGENTS.md policy edit too). The ValueError save-guard is fine to keep here.
  3. Can you run 1 kHz simulaiton for ~60 s (~60k frames) with a modest queue (512 frames) and confirm frames_missed stays ~0? If drops persist, that's a per-frame processing limit (not a queue-size one), and the default should reflect that. It would be nice to document that beefore and after.

@Osayi-ANL Osayi-ANL requested a review from pecomyint July 14, 2026 17:28
…n PR (kept on branch agents-md-const-policy)

This reverts commit 99820c7.
@Osayi-ANL

Copy link
Copy Markdown
Collaborator Author

Nice work. The qMonitor + consumer-thread approach is the right call (matches Sinisa's suggestion). It's thread-safe (queued Qt signals, no cross-thread widget access), the start/stop lifecycle is clean, and it clearly drops fewer frames than main. Three things before I can merge:

  1. Blocker — client queue default is too large. PVA_MONITOR_QUEUE_SIZE = 15000 is bounded at depth × frame size, but 15k real detector frames is ~10-30 GB compressed (far more uncompressed). Please drop it to a burst-sized default (~256-512); dropping sooner under sustained overload is the intended behavior.
  2. Descope — pull the DashPVA ioc CLI command into its own PR. This is a safety-critical reader change and I'd like the diff tight and focused; the ioc command is unrelated (and ideally split the AGENTS.md policy edit too). The ValueError save-guard is fine to keep here.
  3. Can you run 1 kHz simulaiton for ~60 s (~60k frames) with a modest queue (512 frames) and confirm frames_missed stays ~0? If drops persist, that's a per-frame processing limit (not a queue-size one), and the default should reflect that. It would be nice to document that beefore and after.
  1. Done
  2. Done
  3. With a queue size of 512 and 1kHZ you start getting missing frames at around 1.4 seconds. In about 60 seconds you miss about half of the frames received
Screenshot 2026-07-14 at 3 37 29 PM

@pecomyint pecomyint left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work and tests.

@pecomyint pecomyint merged commit 819ef70 into main Jul 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants