feat(task356, task3): implement MPI-IO parallel cube file write#7415
Open
xiaoxuPekingUniversity wants to merge 2 commits into
Open
feat(task356, task3): implement MPI-IO parallel cube file write#7415xiaoxuPekingUniversity wants to merge 2 commits into
xiaoxuPekingUniversity wants to merge 2 commits into
Conversation
Add write_cube_mpi() function for collective MPI-IO parallel writing of cube files. Each rank writes its z-slice range via strided MPI file views, eliminating the serial I/O bottleneck in rank-0-only writes. Key changes: - cube_io.h: declare write_cube_mpi() API under __MPI guard - write_cube.cpp: implement MPI-IO write with binary marker (CIPM) - write_cube_test.cpp: add MPI parallel write tests - CMakeLists.txt: register new test target with MPI Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
This PR adds MPI-parallel cube writing support and introduces new GoogleTest coverage (including roundtrip correctness and benchmark-style timing tests) for cube I/O.
Changes:
- Added
ModuleIO::write_cube_mpi()using MPI-IO with a binary payload after a text cube header. - Added a new
write_cube_test.cppcovering text roundtrip, header checks, layout assumptions, and MPI consistency. - Updated
source_io/testCMake to build/run the new test (including an mpirun-based CTest entry).
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 15 comments.
| File | Description |
|---|---|
| source/source_io/test/write_cube_test.cpp | New correctness + MPI + benchmark-style tests for cube read/write paths |
| source/source_io/test/CMakeLists.txt | Adds the new test target and an mpirun-based parallel test entry |
| source/source_io/module_output/write_cube.cpp | Implements MPI-IO parallel cube writer with marker + collective writes |
| source/source_io/module_output/cube_io.h | Declares write_cube_mpi() and conditionally includes MPI types |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+171
to
+175
| ModuleIO::read_cube(tmp_file_, cmt, natom, org, nx, ny, nz, | ||
| dx, dy, dz, atype, acharge, apos, rdata); | ||
|
|
||
| EXPECT_EQ(cmt[0], meta_.comment[0]); | ||
| EXPECT_EQ(cmt[1], meta_.comment[1]); |
Comment on lines
+213
to
+214
| ModuleIO::read_cube(tmp_file_, cmt, natom, org, nx_r, ny_r, nz_r, | ||
| dx_r, dy_r, dz_r, at, ac, ap, rdata); |
Comment on lines
+365
to
+368
| ModuleIO::write_cube(fn_txt, m.comment, m.natom, m.origin, | ||
| m.nx, m.ny, m.nz, m.dx, m.dy, m.dz, | ||
| m.atom_type, m.atom_charge, m.atom_pos, | ||
| m.data, 6); |
Comment on lines
+273
to
+275
| TEST_F(WriteCubeTest, Bench_WriteCube_SerialText_64) | ||
| { | ||
| auto m = make_cube_meta(64, 64, 64); |
Comment on lines
+295
to
+297
| TEST_F(WriteCubeTest, Bench_WriteCube_SerialText_256) | ||
| { | ||
| auto m = make_cube_meta(256, 256, 256); |
| header_bytes = header_str.size(); | ||
| } | ||
|
|
||
| MPI_Bcast(&header_bytes, 1, MPI_UNSIGNED_LONG, 0, comm); |
Comment on lines
+342
to
+344
| MPI_File_open(comm, file.c_str(), | ||
| MPI_MODE_WRONLY | MPI_MODE_CREATE, | ||
| MPI_INFO_NULL, &fh); |
Comment on lines
+296
to
+297
| int nz_local = nz / nprocs; | ||
| int remainder = nz % nprocs; |
| if (my_rank < remainder) | ||
| nz_local += 1; | ||
|
|
||
| const int my_count = nz_local * nxy; |
| // Create strided file view: for each ixy row, this rank writes nz_local | ||
| // values with a stride of nz (skipping data written by other ranks). | ||
| MPI_Datatype filetype; | ||
| MPI_Type_vector(nxy, nz_local, nz, MPI_DOUBLE, &filetype); |
The task3 branch removed binary/compression format detection from read_cube(), but write_cube_mpi() writes binary MPI cube files with a CIPM marker (0x4D504943). This caused tests 273 and 274 to fail because read_cube() could not parse the binary data — all values returned as 0. Add back detection of the MPI binary format marker so read_cube() can correctly read both text and MPI binary cube files. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add write_cube_mpi() function for collective MPI-IO parallel writing of cube files. Each rank writes its z-slice range via strided MPI file views, eliminating the serial I/O bottleneck in rank-0-only writes.
Key changes:
Reminder
Linked Issue
Fix #...
Unit Tests and/or Case Tests for my changes
What's changed?
Any changes of core modules? (ignore if not applicable)