Skip to content

Add --fast mode to reconstruction benchmark#4359

Open
ahojnnes wants to merge 1 commit intouser/jsch/eval-gpu-schedulefrom
user/jsch/eval-fast-mode
Open

Add --fast mode to reconstruction benchmark#4359
ahojnnes wants to merge 1 commit intouser/jsch/eval-gpu-schedulefrom
user/jsch/eval-fast-mode

Conversation

@ahojnnes
Copy link
Copy Markdown
Contributor

Add a --fast flag that, when set, restricts evaluation to the
--fast_num_scenes (default 1) smallest scenes per category across all
selected datasets. Scene size is determined by the number of input
images, recorded in SceneInfo.num_images during list_scenes() for each
dataset.

Add a --fast flag that, when set, restricts evaluation to the
--fast_num_scenes (default 1) smallest scenes per category across all
selected datasets. Scene size is determined by the number of input
images, recorded in SceneInfo.num_images during list_scenes() for each
dataset.
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a "fast mode" for reconstruction evaluation, enabling users to run benchmarks on only the smallest scenes per category to save time. To support this, the SceneInfo class was updated to include a num_images field, and image counting logic was implemented for the BlendedMVS, ETH3D, and IMC datasets. Additionally, a new utility function filter_smallest_scenes_per_category was added along with corresponding unit tests. Review feedback suggests improving the robustness of the image counting logic in the ETH3D and IMC datasets by filtering for specific image file extensions (e.g., .jpg, .png) to avoid counting non-image metadata or hidden files.

Comment on lines +71 to +73
num_images = sum(
1 for p in image_path.rglob("*") if p.is_file()
)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The current implementation counts all files recursively, which might include non-image files (e.g., metadata, hidden files, or documentation) and lead to an incorrect scene size calculation. It is safer to filter by common image extensions to ensure the count reflects the actual number of images COLMAP will process.

Suggested change
num_images = sum(
1 for p in image_path.rglob("*") if p.is_file()
)
num_images = sum(
1 for p in image_path.rglob("*")
if p.is_file() and p.suffix.lower() in {".jpg", ".jpeg", ".png"}
)

Comment on lines +144 to +146
num_images = sum(
1 for p in image_path.iterdir() if p.is_file()
)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Similar to the ETH3D implementation, this counts all files in the directory. Filtering by image extensions ensures a more accurate count of input images, avoiding potential issues with hidden files or other non-image assets often found in benchmark datasets.

Suggested change
num_images = sum(
1 for p in image_path.iterdir() if p.is_file()
)
num_images = sum(
1 for p in image_path.iterdir()
if p.is_file() and p.suffix.lower() in {".jpg", ".jpeg", ".png"}
)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants