You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
context.request.user_data mutations are silently dropped on retry #2061
Mutating context.request.user_data inside a handler doesn't persist across a retry. The change is visible within the same handler run, but silently gone on the next attempt.
Cause: context.request is an isolated deepcopy created for handler execution (RequestHandlerRunResult, src/crawlee/_types.py:277), while the retry reclaims the original request. So in-handler mutations to context.request (including user_data) are dropped on retry. This differs from Crawlee for JS, where request.userData mutations persist across retries.
Surfaced in discussion #2028, where the natural workaround (stash state on request.user_data, read it on the retry) silently fails.
js/python parity is the bigger deal here tbh — silent data loss on retry is brutal to debug when porting handlers from Crawlee JS. deepcopy isolation is fine for handler safety, but maybe an explicit request.persist_user_data() would cover retry-state without changing mutation semantics
Activity
Mantisus commentedon Jul 10, 2026
This conflicts with the PR - #1585
We need to decide what the final behavior should be.
Huaian666 commentedon Jul 13, 2026
js/python parity is the bigger deal here tbh — silent data loss on retry is brutal to debug when porting handlers from Crawlee JS. deepcopy isolation is fine for handler safety, but maybe an explicit
request.persist_user_data()would cover retry-state without changing mutation semanticsfix(crawlers): Keep `Request` mutations across handler runs and retri…