Silence the fork-safety warning from num_workers>0 pipeline tests
CI / Format (ruff format) (push) Successful in 32s
CI / Lint (ruff check) (push) Successful in 32s
CI / Sync project version with tag (push) Has been skipped
CI / Lint (ruff check) (pull_request) Successful in 33s
CI / Type check (ty) (push) Successful in 34s
CI / Format (ruff format) (pull_request) Successful in 30s
CI / Sync project version with tag (pull_request) Has been skipped
CI / Type check (ty) (pull_request) Successful in 30s
CI / Tests (push) Successful in 3m38s
CI / Tests (pull_request) Successful in 3m37s
CI / Format (ruff format) (push) Successful in 32s
CI / Lint (ruff check) (push) Successful in 32s
CI / Sync project version with tag (push) Has been skipped
CI / Lint (ruff check) (pull_request) Successful in 33s
CI / Type check (ty) (push) Successful in 34s
CI / Format (ruff format) (pull_request) Successful in 30s
CI / Sync project version with tag (pull_request) Has been skipped
CI / Type check (ty) (pull_request) Successful in 30s
CI / Tests (push) Successful in 3m38s
CI / Tests (pull_request) Successful in 3m37s
The two DataLoader-num_workers quota tests are the only ones in the file that leave num_workers>0, so they're the only ones that actually spawn forked worker subprocesses under pytest's multi-threaded process and hit Python's fork-safety DeprecationWarning. The thing under test is just the pre-flight quota-check message, emitted before the DataLoader is built. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
@@ -186,14 +186,19 @@ def test_run_train_job_no_topn_map_for_physical_target(tmp_path, data):
|
||||
assert loaded.topn_maps == {}
|
||||
|
||||
|
||||
@pytest.mark.filterwarnings("ignore::DeprecationWarning:multiprocessing.popen_fork")
|
||||
def test_run_train_job_warns_when_num_workers_exceeds_shared_quota(
|
||||
tmp_path, data, monkeypatch
|
||||
):
|
||||
# num_workers>0 makes DataLoader actually fork worker subprocesses
|
||||
# (unlike every other test here, which runs with num_workers=0) — pytest
|
||||
# itself is multi-threaded, hence Python's fork-safety warning below.
|
||||
monkeypatch.setattr("giant.pipeline.os.cpu_count", lambda: 8) # quota = 2
|
||||
echo = _run(data, tmp_path / "out", num_workers=3)
|
||||
assert any("num-workers=3" in m and "exceeds" in m for m in echo)
|
||||
|
||||
|
||||
@pytest.mark.filterwarnings("ignore::DeprecationWarning:multiprocessing.popen_fork")
|
||||
def test_run_train_job_no_warning_when_num_workers_within_shared_quota(
|
||||
tmp_path, data, monkeypatch
|
||||
):
|
||||
|
||||
Reference in New Issue
Block a user