Add --fast mode to reconstruction benchmark#4359
Add --fast mode to reconstruction benchmark#4359ahojnnes wants to merge 1 commit intouser/jsch/eval-gpu-schedulefrom
Conversation
Add a --fast flag that, when set, restricts evaluation to the --fast_num_scenes (default 1) smallest scenes per category across all selected datasets. Scene size is determined by the number of input images, recorded in SceneInfo.num_images during list_scenes() for each dataset.
There was a problem hiding this comment.
Code Review
This pull request introduces a "fast mode" for reconstruction evaluation, enabling users to run benchmarks on only the smallest scenes per category to save time. To support this, the SceneInfo class was updated to include a num_images field, and image counting logic was implemented for the BlendedMVS, ETH3D, and IMC datasets. Additionally, a new utility function filter_smallest_scenes_per_category was added along with corresponding unit tests. Review feedback suggests improving the robustness of the image counting logic in the ETH3D and IMC datasets by filtering for specific image file extensions (e.g., .jpg, .png) to avoid counting non-image metadata or hidden files.
| num_images = sum( | ||
| 1 for p in image_path.rglob("*") if p.is_file() | ||
| ) |
There was a problem hiding this comment.
The current implementation counts all files recursively, which might include non-image files (e.g., metadata, hidden files, or documentation) and lead to an incorrect scene size calculation. It is safer to filter by common image extensions to ensure the count reflects the actual number of images COLMAP will process.
| num_images = sum( | |
| 1 for p in image_path.rglob("*") if p.is_file() | |
| ) | |
| num_images = sum( | |
| 1 for p in image_path.rglob("*") | |
| if p.is_file() and p.suffix.lower() in {".jpg", ".jpeg", ".png"} | |
| ) |
| num_images = sum( | ||
| 1 for p in image_path.iterdir() if p.is_file() | ||
| ) |
There was a problem hiding this comment.
Similar to the ETH3D implementation, this counts all files in the directory. Filtering by image extensions ensures a more accurate count of input images, avoiding potential issues with hidden files or other non-image assets often found in benchmark datasets.
| num_images = sum( | |
| 1 for p in image_path.iterdir() if p.is_file() | |
| ) | |
| num_images = sum( | |
| 1 for p in image_path.iterdir() | |
| if p.is_file() and p.suffix.lower() in {".jpg", ".jpeg", ".png"} | |
| ) |
Add a --fast flag that, when set, restricts evaluation to the
--fast_num_scenes (default 1) smallest scenes per category across all
selected datasets. Scene size is determined by the number of input
images, recorded in SceneInfo.num_images during list_scenes() for each
dataset.