Repository navigation
chore: remove redundant desired concurrency defaults from docs - #2036
Merged
Merged
Conversation
The ConcurrencySettings docstring and the scaling crawlers guide both state that desired_concurrency defaults to min_concurrency, but the signature default is a literal 10 and the value is used verbatim by AutoscaledPool. This drifted when the default changed from None (resolved to min_concurrency) to 10 while the docs were left untouched. Update both to state 10. Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
vdusek
approved these changes
Jul 7, 2026
vdusek
left a comment
Collaborator
There was a problem hiding this comment.
rather remove the defaults from the docs (they are redundant)
Mantisus
pushed a commit
to Mantisus/crawlee-python
that referenced
this pull request
Jul 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ConcurrencySettings.desired_concurrencydefaults to10(
src/crawlee/_types.py,desired_concurrency: int = 10), and that value is usedverbatim by the autoscaled pool. The docstring and the scaling guide, however,
still say it defaults to
min_concurrency, which is stale and misleading foranyone tuning concurrency.
This corrects both documentation sites to state the real default. Docs-only, no
behavior change.
src/crawlee/_types.py: docstring fordesired_concurrencynow reads "Bydefault, it is
10." (was "By default, it ismin_concurrency.")docs/guides/scaling_crawlers.mdx: the "Desired concurrency" section now reads"It defaults to
10." (was "It defaults to the same value asmin_concurrency.")Background: the default was changed from
None(resolved tomin_concurrency) toa hardcoded
10in #1419, but the docstring (originally from #456) and the guidewere not updated at the time.
Issues
good first issuelabel and issues are maintainer-assigned. Happy to open anissue first if preferred.
Testing
uv run poe lintpasses clean (ruff format check + ruff check).uv run poe type-checkshows the same 30 pre-existing diagnostics asmaster(the optional
pydantic_aiunresolved import); zero introduced by this change.ConcurrencySettings().desired_concurrency == 10whilemin_concurrency == 1, confirming the old wording was wrong.every existing test that builds
ConcurrencySettingspassesdesired_concurrencyexplicitly, so nothing depends on the default. (Consistent with the docstring-fix
precedent in docs: fix docstring of
Configuration.available_memory_ratio#742.)Checklist