Drop orphaned child tracks instead of nulling secondary targets

A listed child_track_id can fail to match any first-step row (e.g. a
secondary absorbed below the tracking threshold at birth). The
parent->child left join in _add_secondary_attributes left these as
nulls, which silently became NaN once the parquet round-tripped
through the loader's float32 padding — poisoning every later secondary
slot in that step via the cumulative "remaining budget" in
encode_secondaries, while e_sec quietly undercounted and n_sec (from
len(child_track_ids)) overcounted relative to the actual lists.

Drop orphans from both the per-secondary lists and child_track_ids
itself so downstream counts stay consistent, and thread the per-file
orphaned count back through convert_steps_to_parquet so both the
sequential and --jobs>1 batch paths in `dwarf convert` can report an
aggregate total instead of relying on grepping printed output.

Also floors encode_secondaries' slot-0 budget to _EPS (matching the
i>0 branch), fixing a harmless but noisy 0/0 divide warning on
zero-secondary steps.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-09 14:36:06 +02:00
parent a8fc019e32
commit 740ebdf6b6
5 changed files with 96 additions and 12 deletions
+11
View File
@@ -26,6 +26,12 @@ from pathlib import Path
GEN_RE = re.compile(r"^gen\d+$")
SCHEMA_RE = re.compile(r"^schema(\d+)$")
# Matches the per-file orphan-drop message printed by
# steps_to_parquet._add_secondary_attributes — each subprocess's count is
# parsed back out of its captured stdout since there's no in-process return
# value across the subprocess boundary.
_ORPHAN_RE = re.compile(r"dropping (\d+) orphaned child_track_id")
class DestinationError(ValueError):
pass
@@ -210,4 +216,9 @@ def run_parallel_job(
print(f" {root_file}", file=sys.stderr)
raise SystemExit(1)
total_orphaned = sum(
int(m.group(1)) for _, _, stdout, _ in results for m in _ORPHAN_RE.finditer(stdout)
)
if total_orphaned:
print(f"\n{total_orphaned} orphaned child track(s) dropped across {len(results)} file(s).")
print(f"\nAll {len(results)} conversion(s) completed.")