Skip to content

Add support for request to use a specific session #1076

Closed

Description

@dickermoshe

I'm scraping a single site with 2 different logins. I don't see a way to specify which session/cookies a request should use?
The documentation for this is lacking.

Thanks

Activity

  1. added
    t-toolingIssues with this label are in the ownership of the tooling team.
    on Mar 12, 2025
  2. vdusek commented on Mar 13, 2025

    @vdusek
    Collaborator

    Hi @dickermoshe, thanks for your interest in Crawlee! This feature isn't supported yet, but it's on the roadmap.

  3. changed the title [-][Feature Request] Support requests using a specific session[/-] [+]Add support for request to use a specific session[/+] on Mar 13, 2025
  4. janbuchar commented on Mar 13, 2025

    @janbuchar
    Collaborator

    Hello! You may specify additional cookies via headers when constructing a Request object. If you don't do this explicitly, you can use the transform_request_function argument of context.enqueue_links. In PlaywrightCrawler, this should take precedence over headers from session. In http-based crawlers, this is unfortunately implementation-dependent, so you'd need to experiment with your setup.

    As @vdusek said, specifying which session should be used for a particular request is not supported as of now.

  5. added a commit that references this issue on Mar 31, 2025
    cda7b31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request.t-toolingIssues with this label are in the ownership of the tooling team.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions