You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Add support for request to use a specific session #1076
I'm scraping a single site with 2 different logins. I don't see a way to specify which session/cookies a request should use?
The documentation for this is lacking.
changed the title [-][Feature Request] Support requests using a specific session[/-][+]Add support for request to use a specific session[/+]on Mar 13, 2025
Hello! You may specify additional cookies via headers when constructing a Request object. If you don't do this explicitly, you can use the transform_request_function argument of context.enqueue_links. In PlaywrightCrawler, this should take precedence over headers from session. In http-based crawlers, this is unfortunately implementation-dependent, so you'd need to experiment with your setup.
As @vdusek said, specifying which session should be used for a particular request is not supported as of now.
Activity
vdusek commentedon Mar 13, 2025
Hi @dickermoshe, thanks for your interest in Crawlee! This feature isn't supported yet, but it's on the roadmap.
[-][Feature Request] Support requests using a specific session[/-][+]Add support for request to use a specific session[/+]janbuchar commentedon Mar 13, 2025
Hello! You may specify additional cookies via
headerswhen constructing aRequestobject. If you don't do this explicitly, you can use thetransform_request_functionargument ofcontext.enqueue_links. InPlaywrightCrawler, this should take precedence over headers from session. In http-based crawlers, this is unfortunately implementation-dependent, so you'd need to experiment with your setup.As @vdusek said, specifying which session should be used for a particular request is not supported as of now.
feat: add Session binding capability via `session_id` in `Request` (#…