Add objective integration tests for all example_ws configs#534
Draft
Add objective integration tests for all example_ws configs#534
Conversation
JWhitleyWork
requested changes
Mar 23, 2026
Member
JWhitleyWork
left a comment
There was a problem hiding this comment.
Is a good start but needs some tweaking.
| moveit_license_key: ${{ secrets.STUDIO_CI_LICENSE_KEY }} | ||
|
|
||
| # Per-config objective integration tests (run in parallel via matrix) | ||
| objective-integration-test: |
Member
There was a problem hiding this comment.
The downside to doing this with a matrix is that it launches a separate Github Actions runner for each invocation of the matrix. If we do these all as a single colcon test --packages-select <list_of_packages>, many of them can still be run in parallel but we only spin up an instance, install dependencies, clone stuff, etc. once. Also, we should be doing this test for at least Humble and Jazzy (in a matrix because they are completely different environments).
| ament_lint_auto_find_test_dependencies() | ||
| ament_add_pytest_test( | ||
| objectives_integration_test test/objectives_integration_test.py | ||
| TIMEOUT 600 |
Member
There was a problem hiding this comment.
10 minutes? We usually give 10 minutes for all of colcon test. Does it actually take anywhere near this long?
Replicate the lab_sim objectives_integration_test.py pattern across all 11 remaining sim config packages: kitchen_sim, april_tag_sim, grinding_sim, dual_arm_sim, factory_sim, hangar_sim, kinova_sim, space_satellite_sim, space_satellite_sim_camera_cal, mock_sim, and multi_arm_sim. Each test auto-discovers objectives via SystemConfigParser and runs them, with per-config skip lists for objectives requiring user input or GPU/ML inference. New objectives are automatically tested — if one requires skipping, CI fails and the PR author adds it to the skip list. Also adds a CI matrix job to run each config's tests in parallel, and adds missing ament_cmake_pytest + moveit_pro_test_utils test deps (including for the existing lab_sim test). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
863b4ca to
a853397
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
objectives_integration_test.pyto 11 config packages (kitchen_sim, april_tag_sim, grinding_sim, dual_arm_sim, factory_sim, hangar_sim, kinova_sim, space_satellite_sim, space_satellite_sim_camera_cal, mock_sim, multi_arm_sim)SystemConfigParserand runs them with per-config skip lists for user-input/ML/GPU objectivesworkspace_integration_testreusable workflowament_cmake_pytest+moveit_pro_test_utilstest deps across all configs (including existing lab_sim)Known TODOs
integration-test-in-studio-containerjob still runs all tests including these, causing double-runs. Consider adding--packages-skipfor config packages to that jobTest plan
main_tree_to_executeXML attributes — all match🤖 Generated with Claude Code