Skip to content

Add Router.use() middleware support #1742

Closed

Description

@vdusek

Context

JS Crawlee Router supports .use() middleware — functions that run sequentially before every matched route handler. Python Router has no equivalent. Users cannot add cross-cutting concerns (logging, auth, rate limiting) at the router level.

JS implementation

packages/core/src/router.ts

router.use(middleware: (ctx: Context) => Awaitable<void>): void
  • Middlewares stored in a private array, executed in registration order
  • All run before the matched handler, sharing the same context object
  • No early-exit / next() pattern — just sequential execution, throwing stops the chain

Dispatch flow (router.ts L193–L202):

for (const middleware of router.middlewares) {
    await middleware(context);
}
return router.getHandler(label)(context);

Tests: test/core/router.test.ts

Current Python state

src/crawlee/router.py — simple label-based dispatcher with default_handler and handler(label) decorators, no middleware support. The ContextPipeline provides crawler-level middleware, but not at the router level.

What to implement

Add a use() method to Router that registers pre-handler middleware, matching JS semantics.

Activity

  1. added
    t-toolingIssues with this label are in the ownership of the tooling team.
    on Feb 12, 2026
  2. B4nan commented on Feb 17, 2026

    @B4nan
    Member

    We might want to use decorators in python.

  3. added this to the 2.0 milestone on Feb 17, 2026
  4. added a commit that references this issue on Apr 28, 2026
    23d7d6c
  5. removed this from the 2.0 milestone on Apr 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request.t-toolingIssues with this label are in the ownership of the tooling team.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions