You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
failed_request_handler runs and logs but context.push_data(...) does not write to dataset (PlaywrightCrawler) #1532
When a request fails in PlaywrightCrawler, my @crawler.failed_request_handler runs (I can see context.log output) but await context.push_data(...) does not create any dataset rows. Logging works (I can see the context.log.info(...) lines), only the dataset push appears to be ignored.
Hi @loic-bellinger, thanks for reporting this! The issue is that we "buffer" any storage-related calls on the context object so that partial results of failing request handler runs are not stored. This is an unintended side effect of this feature. We'll do our best to fix this soon 🙂
Activity
janbuchar commentedon Nov 5, 2025
Hi @loic-bellinger, thanks for reporting this! The issue is that we "buffer" any storage-related calls on the context object so that partial results of failing request handler runs are not stored. This is an unintended side effect of this feature. We'll do our best to fix this soon 🙂
As a temporary workaround, you may try this:
It should bypass the "buffering" mechanism.
loic-bellinger commentedon Nov 5, 2025
Wouldn't calling
dataset = await Dataset.open(name="failed_request_handler_errors")on each error create some overhead?janbuchar commentedon Nov 5, 2025
There is a lot of caching, you should be fine. But let us know if anything happens!
fix: Make context helpers work in `FailedRequestHandler` and `ErrorHa…