Repository navigation
fix(file-system): stop KVS pagination from skipping or repeating keys - #2431
Merged
vdusek merged 2 commits intoNov 10, 2026
Merged
Conversation
vdusek
reviewed
Nov 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The filesystem KVS client sorted records by their encoded filenames, but it returned and compared the key stored in each record's metadata. The two orders differ, so cursor pagination could skip or repeat keys:
itemanditem-1,item-1.__metadata__.jsonsorts first, so the cursoritem-1skipsitem.éclairora/b.jsonthe same way.INPUT.jsonhas the metadata keyINPUT, so it came back again after the cursorINPUT.iterate_keysnow reads all record metadata in one thread call. It then filters byexclusive_start_key, sorts, and applieslimit, all by the metadata key. A record with invalid metadata is skipped. Before this change, such a record made iteration yield the previous record a second time.Each call reads every metadata file, because the metadata key can differ from the filename. With 5,000 keys, full pagination at 1,000 keys per page takes 0.36 s (0.43 s on master), and one
limit=10page takes 0.06 s (0.03 s on master).Public signatures, log messages, and stored files are unchanged.
AI assistance: I used Codex to investigate, implement, and test this change.
Issues
No linked issue.
Testing
INPUTandINPUT.jsonboth end after cursorINPUT.Checklist