MNT: Replace live server calls with mocks in tests/test_utils/test_utils.py#1679
MNT: Replace live server calls with mocks in tests/test_utils/test_utils.py#1679Rahuldrabit wants to merge 1 commit intoopenml:mainfrom
Conversation
There was a problem hiding this comment.
Pull request overview
This PR replaces live server network calls with local mocks in tests/test_utils/test_utils.py to make tests run deterministically and offline, addressing part of issue #1649. The changes are test-only with no production code modifications.
Changes:
- Added helper function
_create_mock_listing_callto generate mock responses for paginated listing endpoints - Modified
_mocked_perform_api_callto return mock XML responses for dataset queries - Updated 10 test functions to use mocks instead of live server calls, removing
@pytest.mark.test_server()and@pytest.mark.flaky()markers
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| mock_list_setups.side_effect = _create_mock_listing_call( | ||
| min_number_setups_on_test_server, | ||
| lambda i: OpenMLSetup(setup_id=i, flow_id=1, parameters={}), | ||
| return_type="list" |
There was a problem hiding this comment.
Inconsistent indentation for multi-line function call. The closing parenthesis should be on a new line at the same indentation level as the opening line of the function call, consistent with how similar calls are formatted elsewhere in the file (e.g., lines 82-84, 91-93, 107-109, 121-123).
| return_type="list" | |
| return_type="list", |
| run_id=i, task_id=1, setup_id=1, flow_id=1, flow_name="flow", data_id=1, data_name="data", | ||
| function="predictive_accuracy", upload_time="2020-01-01", uploader=1, uploader_name="user", | ||
| value=0.5, values=None |
There was a problem hiding this comment.
Inconsistent indentation for multi-line function call. The closing parenthesis should be on a new line at the same indentation level as the opening line of the function call, consistent with how similar calls are formatted elsewhere in the file (e.g., lines 82-84, 91-93, 107-109, 121-123).
| run_id=i, task_id=1, setup_id=1, flow_id=1, flow_name="flow", data_id=1, data_name="data", | |
| function="predictive_accuracy", upload_time="2020-01-01", uploader=1, uploader_name="user", | |
| value=0.5, values=None | |
| run_id=i, | |
| task_id=1, | |
| setup_id=1, | |
| flow_id=1, | |
| flow_name="flow", | |
| data_id=1, | |
| data_name="data", | |
| function="predictive_accuracy", | |
| upload_time="2020-01-01", | |
| uploader=1, | |
| uploader_name="user", | |
| value=0.5, | |
| values=None, |
| run_id=i, task_id=1, setup_id=1, flow_id=1, flow_name="flow", data_id=1, data_name="data", | ||
| function="predictive_accuracy", upload_time="2020-01-01", uploader=1, uploader_name="user", | ||
| value=0.5, values=None |
There was a problem hiding this comment.
Line exceeds recommended maximum line length. The OpenMLEvaluation constructor call has very long lines (line 154 appears to be particularly long). Consider breaking this into multiple lines with one parameter per line for better readability.
| run_id=i, task_id=1, setup_id=1, flow_id=1, flow_name="flow", data_id=1, data_name="data", | |
| function="predictive_accuracy", upload_time="2020-01-01", uploader=1, uploader_name="user", | |
| value=0.5, values=None | |
| run_id=i, | |
| task_id=1, | |
| setup_id=1, | |
| flow_id=1, | |
| flow_name="flow", | |
| data_id=1, | |
| data_name="data", | |
| function="predictive_accuracy", | |
| upload_time="2020-01-01", | |
| uploader=1, | |
| uploader_name="user", | |
| value=0.5, | |
| values=None, |
| def _mocked_perform_api_call(call, request_method): | ||
| url = openml.config.server + "/" + call | ||
| return openml._api_calls._download_text_file(url) | ||
|
|
There was a problem hiding this comment.
Missing blank line before function definition. According to PEP 8, there should be two blank lines before top-level function definitions. There's an extra blank line at line 51 which makes it appear there are two blank lines, but the standard convention is to have exactly two blank lines, not one followed by an extra one.
| items = [item_factory(i) for i in range(offset, offset + size)] | ||
| return pd.DataFrame(items) if return_type == "dataframe" else items | ||
| return mock_listing_call | ||
|
|
There was a problem hiding this comment.
Missing blank line between function definitions. According to PEP 8, there should be two blank lines between top-level function definitions.
1 participant
Replace tests' live-server network calls with local mocks/patches so tests run deterministically and offline.
Add helpers to mock listing endpoints and API call responses (for datasets, tasks, flows, setups, runs, evaluations).
Patch cache/config calls and use tmp_path where needed to test cache utilities without touching real filesystem config.
No production code changes; test-only changes.
Addresses part of issue #1649.