Skip to content

chore: remove redundant desired concurrency defaults from docs - #2036

Merged
vdusek merged 3 commits into
apify:masterfrom
anxkhn:docs/fix-desired-concurrency-default
Jul 7, 2026
Merged

vdusek merged 3 commits into
apify:masterfrom
anxkhn:docs/fix-desired-concurrency-default

Conversation

@anxkhn

@anxkhn anxkhn commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

ConcurrencySettings.desired_concurrency defaults to 10
(src/crawlee/_types.py, desired_concurrency: int = 10), and that value is used
verbatim by the autoscaled pool. The docstring and the scaling guide, however,
still say it defaults to min_concurrency, which is stale and misleading for
anyone tuning concurrency.

This corrects both documentation sites to state the real default. Docs-only, no
behavior change.

  • src/crawlee/_types.py: docstring for desired_concurrency now reads "By
    default, it is 10." (was "By default, it is min_concurrency.")
  • docs/guides/scaling_crawlers.mdx: the "Desired concurrency" section now reads
    "It defaults to 10." (was "It defaults to the same value as
    min_concurrency.")

Background: the default was changed from None (resolved to min_concurrency) to
a hardcoded 10 in #1419, but the docstring (originally from #456) and the guide
were not updated at the time.

Issues

  • No related issue. Self-identified documentation drift; this repo has no
    good first issue label and issues are maintainer-assigned. Happy to open an
    issue first if preferred.

Testing

  • uv run poe lint passes clean (ruff format check + ruff check).
  • uv run poe type-check shows the same 30 pre-existing diagnostics as master
    (the optional pydantic_ai unresolved import); zero introduced by this change.
  • Runtime sanity: ConcurrencySettings().desired_concurrency == 10 while
    min_concurrency == 1, confirming the old wording was wrong.
  • No unit test added: this is a pure docs correction with no behavior change, and
    every existing test that builds ConcurrencySettings passes desired_concurrency
    explicitly, so nothing depends on the default. (Consistent with the docstring-fix
    precedent in docs: fix docstring of Configuration.available_memory_ratio #742.)

Checklist

  • CI passed

anxkhn and others added 3 commits July 7, 2026 13:14
The ConcurrencySettings docstring and the scaling crawlers guide both state
that desired_concurrency defaults to min_concurrency, but the signature
default is a literal 10 and the value is used verbatim by AutoscaledPool.
This drifted when the default changed from None (resolved to min_concurrency)
to 10 while the docs were left untouched. Update both to state 10.

Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
@vdusek vdusek changed the title docs: correct desired_concurrency default value chore: remove redundant desired concurrency defaults from docs Jul 7, 2026

@vdusek vdusek left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rather remove the defaults from the docs (they are redundant)

@vdusek
vdusek merged commit e7e23b3 into apify:master Jul 7, 2026
35 checks passed
Mantisus pushed a commit to Mantisus/crawlee-python that referenced this pull request Jul 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants