You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Flaky test: test_snapshot_pruning_removes_outdated_records fails with assert 3 == 2 #1734
test_snapshot_pruning_removes_outdated_records in tests/unit/_autoscaling/test_snapshotter.py is consistently failing across multiple CI runs on Ubuntu (Python 3.13 and 3.14), even when running in isolation (run_alone marker with numprocesses=1).
The test expects that after emitting 4 events with timestamps [now-5h, now-3h, now-2h, now] and a _SNAPSHOT_HISTORY of 2 hours, only 2 snapshots should remain (the ones within the 2-hour window). However, 3 snapshots are retained instead of 2.
Root cause hypothesis
The snapshot pruning logic likely has an off-by-one or boundary condition issue when determining which records fall outside the history window. The now-2h timestamp sits exactly at the boundary of the 2-hour history window, and depending on timing/precision of datetime.now() between the test setup and the pruning execution, it may or may not be pruned.
I think that the problem is probably in some race condition in sorting/pruning snapshots. Where both are modifying the list in place and most likely are running at the same time.
Activity
test: Skip flaky `test_snapshot_pruning_removes_outdated_records`
Pijukatel commentedon Feb 11, 2026
I think that the problem is probably in some race condition in sorting/pruning snapshots. Where both are modifying the list in place and most likely are running at the same time.
fix: Fix `Snapshotter` handling of out of order samples (#1735)