perf: Add MAXLEN to Redis streams to prevent unbounded memory growth #111
+10
−1
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
REDIS_STREAM_MAXLENenvironment variable (default: 10,000)xadd()withmaxlenandapproximate=Truefor O(1) trimmingProblem
Redis streams were created without any size limit, causing unbounded memory growth over time. A long-running task could accumulate unlimited events in its stream.
Solution
Added
maxlenparameter toxadd()calls with configurable limit via environment variable. Usesapproximate=Truefor better performance (O(1) vs O(N) trimming).Test Results
How to Verify
Files Changed
src/config/environment_variables.py- AddedREDIS_STREAM_MAXLENenv varsrc/adapters/streams/adapter_redis.py- Added maxlen to xadd()