Merge master 20240320 #33

apatrushev · 2024-03-19T22:56:39Z

subj

## Summary Django's `mark_safe` can also be used as a decorator, so we should detect usages of `@mark_safe` for the purpose of the relevant Bandit rule. Closes astral-sh#9780.

…ethod calls (astral-sh#9865) Fixes astral-sh#9857. ## Summary Statements like `logging.info("Today it is: {day}")` will no longer be ignored by RUF027. As before, statements like `"Today it is: {day}".format(day="Tuesday")` will continue to be ignored. ## Test Plan The snapshot tests were expanded to include new cases. Additionally, the snapshot tests have been split in two to separate positive cases from negative cases.

## Summary On `main`, string lexing consists of walking through the string character-by-character to search for the closing quote (with some nuance: we also need to skip escaped characters, and error if we see newlines in non-triple-quoted strings). This PR rewrites `lex_string` to instead use `memchr` to search for the closing quote, which is significantly faster. On my machine, at least, the `globals.py` benchmark (which contains a lot of docstrings) gets 40% faster... ```text lexer/numpy/globals.py time: [3.6410 µs 3.6496 µs 3.6585 µs] thrpt: [806.53 MiB/s 808.49 MiB/s 810.41 MiB/s] change: time: [-40.413% -40.185% -39.984%] (p = 0.00 < 0.05) thrpt: [+66.623% +67.181% +67.822%] Performance has improved. Found 2 outliers among 100 measurements (2.00%) 2 (2.00%) high mild lexer/unicode/pypinyin.py time: [12.422 µs 12.445 µs 12.467 µs] thrpt: [337.03 MiB/s 337.65 MiB/s 338.27 MiB/s] change: time: [-9.4213% -9.1930% -8.9586%] (p = 0.00 < 0.05) thrpt: [+9.8401% +10.124% +10.401%] Performance has improved. Found 3 outliers among 100 measurements (3.00%) 1 (1.00%) high mild 2 (2.00%) high severe lexer/pydantic/types.py time: [107.45 µs 107.50 µs 107.56 µs] thrpt: [237.11 MiB/s 237.24 MiB/s 237.35 MiB/s] change: time: [-4.0108% -3.7005% -3.3787%] (p = 0.00 < 0.05) thrpt: [+3.4968% +3.8427% +4.1784%] Performance has improved. Found 7 outliers among 100 measurements (7.00%) 2 (2.00%) high mild 5 (5.00%) high severe lexer/numpy/ctypeslib.py time: [46.123 µs 46.165 µs 46.208 µs] thrpt: [360.36 MiB/s 360.69 MiB/s 361.01 MiB/s] change: time: [-19.313% -18.996% -18.710%] (p = 0.00 < 0.05) thrpt: [+23.016% +23.451% +23.935%] Performance has improved. Found 8 outliers among 100 measurements (8.00%) 3 (3.00%) low mild 1 (1.00%) high mild 4 (4.00%) high severe lexer/large/dataset.py time: [231.07 µs 231.19 µs 231.33 µs] thrpt: [175.87 MiB/s 175.97 MiB/s 176.06 MiB/s] change: time: [-2.0437% -1.7663% -1.4922%] (p = 0.00 < 0.05) thrpt: [+1.5148% +1.7981% +2.0864%] Performance has improved. Found 10 outliers among 100 measurements (10.00%) 5 (5.00%) high mild 5 (5.00%) high severe ```

## Summary Corrects mentions of `Path.is_link` to `Path.is_symlink` (the former doesn't exist). ## Test Plan ```sh python scripts/generate_mkdocs.py && mkdocs serve -f mkdocs.public.yml ```

…4`, `E305`, `E306`) (astral-sh#9266) Co-authored-by: Micha Reiser <[email protected]>

…f `String` (astral-sh#9885)

## Summary This PR reduces the size of `Expr` from 80 to 64 bytes, by reducing the sizes of... - `ExprCall` from 72 to 56 bytes, by using boxed slices for `Arguments`. - `ExprCompare` from 64 to 48 bytes, by using boxed slices for its various vectors. In testing, the parser gets a bit faster, and the linter benchmarks improve quite a bit.

…stral-sh#9894) ## Summary This review contains a fix for [D405](https://docs.astral.sh/ruff/rules/capitalize-section-name/) (capitalize-section-name) The problem is that Ruff considers the sub-section header as a normal section if it has the same name as some section name. For instance, a function/method has an argument named "parameters". This only applies if you use Numpy style docstring. See: [ISSUE](astral-sh#9806) The following will not raise D405 after the fix: ```python def some_function(parameters: list[str]): """A function with a parameters parameter Parameters ---------- parameters: A list of string parameters """ ... ``` ## Test Plan ```bash cargo test ``` --------- Co-authored-by: Mikko Leppänen <[email protected]> Co-authored-by: Charlie Marsh <[email protected]>

…h#9907)

## Summary If a generic appears multiple times on the right-hand side, we should only include it once on the left-hand side when rewriting. Closes astral-sh#9904.

## Summary Closes astral-sh#9898. ## Test Plan ```sh python scripts/generate_mkdocs.py && mkdocs serve -f mkdocs.public.yml ```

Closes astral-sh#9869.

@ThiefMaster

…e always on their own line (astral-sh#9793) ## Summary Currently these rules apply the heuristic that if the original sequence doesn't have a newline in between the final sequence item and the closing parenthesis, the autofix won't add one for you. The feedback from @ThiefMaster, however, was that this was producing slightly unusual formatting -- things like this: ```py __all__ = [ "b", "c", "a", "d"] ``` were being autofixed to this: ```py __all__ = [ "a", "b", "c", "d"] ``` When, if it was _going_ to be exploded anyway, they'd prefer something like this (with the closing parenthesis on its own line, and a trailing comma added): ```py __all__ = [ "a", "b", "c", "d", ] ``` I'm still pretty skeptical that we'll be able to please everybody here with the formatting choices we make; _but_, on the other hand, this _specific_ change is pretty easy to make. ## Test Plan `cargo test`. I also ran the autofixes for RUF022 and RUF023 on CPython to check how they looked; they looked fine to me.

Mostly removes dependencies.

…stral-sh#9599) Fixes astral-sh#8368 Fixes astral-sh#9186 ## Summary Arbitrary TOML strings can be provided via the command-line to override configuration options in `pyproject.toml` or `ruff.toml`. As an example: to run over typeshed and respect typeshed's `pyproject.toml`, but override a specific isort setting and enable an additional pep8-naming setting: ``` cargo run -- check ../typeshed --no-cache --config ../typeshed/pyproject.toml --config "lint.isort.combine-as-imports=false" --config "lint.extend-select=['N801']" ``` --------- Co-authored-by: Micha Reiser <[email protected]> Co-authored-by: Zanie Blue <[email protected]>

## Summary We've had success with `nextest` in other projects, so lets migrate Ruff. The Linux tests look a little bit faster (from 2m32s down to 2m8s), the Windows tests look a little bit slower but not dramatically so.

Can [mdformat-ruff](https://github.com/Freed-Wu/mdformat-ruff) be hosted in <https://github.com/astral-sh> like other integrations of ruff? TIA!

## Summary I noticed the example line in CONTRIBUTING.md: ```shell cargo install nextest ``` Didn't appear to install the intended package cargo-nextest. ![nextest](https://github.com/astral-sh/ruff/assets/12672027/7bbdd9c3-c35a-464a-b586-3e9f777f8373) So I checked what it [should be](https://nexte.st/book/installing-from-source.html) and replaced the line: ```shell cargo install cargo-nextest --locked ``` ## Test Plan Just checked the cargo install appeared to give sane looking results --------- Co-authored-by: Charlie Marsh <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…le=tabs` (astral-sh#9915)

## Summary astral-sh#10151 documented the deviations between Ruff and Black with the new 2024 style guide in the `ruff-python-formatter/README.md`. However, that's not the documentation shown on the website when navigating to [intentional deviations](https://docs.astral.sh/ruff/formatter/black/). This PR streamlines the `ruff-python-formatter/README.md` and links to the documentation on the website instead of repeating the same content. The PR also makes the 2024 style guide deviations available on the website documentation. ## Test Plan I built the documentation locally and verified that the 2024 style guide known deviations are now shown on the website.

Bumps [wasm-bindgen-test](https://github.com/rustwasm/wasm-bindgen) from 0.3.41 to 0.3.42. <details> <summary>Commits</summary> <ul> <li>See full diff in <a href="https://github.com/rustwasm/wasm-bindgen/commits">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=wasm-bindgen-test&package-manager=cargo&previous-version=0.3.41&new-version=0.3.42)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…sh#10412)

## Summary Added Nokia to the "Who's Using Ruff?" section in the README. ## Test Plan Multiple teams in my department alone moved to Ruff (either exclusively or partially) 🚀

…thout any root directories or workspaces (astral-sh#10398) ## Summary Fixes astral-sh#10324 This removes an overeager failure case where we would exit early if no root directory or workspace folders were provided on server initialization. We now fall-back to the current working directory as a workspace for that file. ## Test Plan N/A

…ral-sh#10452) ## Summary This PR removes the `Iterator::chain(...)` sequence in `RuleCodePrefix::iter()` with `Vec::expand` to avoid an overlong-recursive types. The existing `RuleCodePrefix::iter` method chains all rule group iterators together. This leads to very long recursive types `Chain<Map<Chain<Map<Chain<Map.....>>>>` (proportional to the number of rule groups). This PR rewrites the macro to use `Vec::extend` instead, which removes the long recursive type (at the cost of introducing a potential allocation). ## Alternatives An alternative would be to use a stack allocated array by unrolling the `Linter::iter` methods (generated by `EnumIter`). I don't think it's worth the extra complexity, considering that `RuleCodePrefix::iter` isn't a hot function. ## Test Plan `cargo test`

…#10314) Co-authored-by: Micha Reiser <[email protected]>

…ues-wrong-type` (`PT007`) (astral-sh#10461) ## Summary This adds automatic fixes for the `PT007` rule. I am currently reviewing and adding Ruff rules to Home Assistant. One rule is PT007, which has multiple hundred occurrences in the codebase, but no automatic fix, and this is not fun to do manually, especially because using Regexes are not really possible with this. My knowledge of the Ruff codebase and Rust in general is not good and this is my first PR here, so I hope it is not too bad. One thing where I need help is: How can I have the transformed code to be formatted automatically, instead of it being minimized as it does it now? ## Test Plan Using the existing fixtures and updated snapshots.

…ble cores (astral-sh#10399) ## Summary Fixes astral-sh#10369. ## Test Plan N/A

Closes astral-sh#10455.

## Summary Clarify `extend-select` documentation to avoid confusion regarding the default `select`. Also match the `select` documentation. Resolves astral-sh#10389. ## Test Plan

## Summary I used `cargo-shear` (see [tweet](https://twitter.com/boshen_c/status/1770106165923586395)) to remove some unused dependencies that `cargo udeps` wasn't reporting.  ## Test Plan `cargo test`

…0478) ## Summary We're seeing failures in astral-sh#10470 because `resolve_qualified_import_name` isn't guaranteed to return a specific import if a symbol is accessible in two ways (e.g., you have both `import logging` and `from logging import error` in scope, and you want `logging.error`). This PR breaks up the failing tests such that the imports aren't in the same scope. Closes astral-sh#10470. ## Test Plan I added a `bindings.reverse()` to `resolve_qualified_import_name` to ensure that the tests pass regardless of the binding order.

## Summary This PR fixes a panic in the linter for `W605`. Consider the following f-string: ```python f"{{}}ab" ``` The `FStringMiddle` token would contain `{}ab`. Notice that the escaped braces have _reduced_ the string. This means we cannot use the text value from the token to determine the location of the escape sequence but need to extract it from the source code. fixes: astral-sh#10434 ## Test Plan Add new test cases and update the snapshots.

Closes astral-sh#10465.

strmwalker · 2024-03-26T13:40:48Z

Did in #34

charliermarsh and others added 30 commits February 7, 2024 23:10

Detect mark_safe usages in decorators (astral-sh#9887)

f76a3e8

## Summary Django's `mark_safe` can also be used as a decorator, so we should detect usages of `@mark_safe` for the purpose of the relevant Bandit rule. Closes astral-sh#9780.

Corrected Path symlink method name (PTH114) (astral-sh#9896)

eb2784c

## Summary Corrects mentions of `Path.is_link` to `Path.is_symlink` (the former doesn't exist). ## Test Plan ```sh python scripts/generate_mkdocs.py && mkdocs serve -f mkdocs.public.yml ```

Use Rust 1.76 (astral-sh#9897)

688177f

[pycodestyle] Add blank line(s) rules (E301, E302, E303, `E30…

9027169

…4`, `E305`, `E306`) (astral-sh#9266) Co-authored-by: Micha Reiser <[email protected]>

Reduce Result<Tok, LexicalError> size by using Box<str> instead o…

fe7d965

…f `String` (astral-sh#9885)

Filter out test rules in RuleSelector JSON schema (astral-sh#9901)

49c5e71

Fix clippy unused variable warning (astral-sh#9902)

bd8123c

Fix E30X panics on blank lines with trailing white spaces (astral-s…

12a91f4

…h#9907)

Respect duplicates when rewriting type aliases (astral-sh#9905)

52ebfc9

## Summary If a generic appears multiple times on the right-hand side, we should only include it once on the left-hand side when rewriting. Closes astral-sh#9904.

Update pyproject-toml to 0.9 (astral-sh#9916)

00ef01d

Use usize instead of TextSize for indent_len (astral-sh#9903)

1ce07d6

Corrected PTH203–PTH205 rule descriptions (astral-sh#9914)

7ca515c

## Summary Closes astral-sh#9898. ## Test Plan ```sh python scripts/generate_mkdocs.py && mkdocs serve -f mkdocs.public.yml ```

Remove unnecessary string cloning from the parser (astral-sh#9884)

6f0e4ad

Closes astral-sh#9869.

Run cargo update (astral-sh#9917)

b21ba71

Mostly removes dependencies.

Migrate to nextest (astral-sh#9921)

af2cba7

## Summary We've had success with `nextest` in other projects, so lets migrate Ruff. The Linux tests look a little bit faster (from 2m32s down to 2m8s), the Windows tests look a little bit slower but not dramatically so.

Run doctests as part of CI pipeline (astral-sh#9939)

341c269

Bump chrono from 0.4.33 to 0.4.34 (astral-sh#9940)

37ff436

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bump thiserror from 1.0.56 to 1.0.57 (astral-sh#9941)

2e1160e

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bump indicatif from 0.17.7 to 0.17.8 (astral-sh#9942)

6dc1b21

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Stabilize quote-style preserve (astral-sh#9922)

4946a18

Docstring formatting: Preserve tab indentation when using `indent-sty…

8657a39

…le=tabs` (astral-sh#9915)

Bump the actions group with 1 update (astral-sh#9943)

90f8e4b

MichaReiser and others added 23 commits March 18, 2024 08:22

Bump clap from 4.5.2 to 4.5.3 (astral-sh#10446)

88adb96

Bump thiserror from 1.0.57 to 1.0.58 (astral-sh#10450)

c8912cf

Bump proc-macro2 from 1.0.78 to 1.0.79 (astral-sh#10449)

ac6a966

Bump toml from 0.8.10 to 0.8.11 (astral-sh#10447)

d6204f9

Apply NFKC normalization to unicode identifiers in the lexer (astral-…

92e6026

…sh#10412)

Add Nokia to "Who's Using Ruff?" (astral-sh#10460)

b98d8ae

## Summary Added Nokia to the "Who's Using Ruff?" section in the README. ## Test Plan Multiple teams in my department alone moved to Ruff (either exclusively or partially) 🚀

Track casing of r-string prefixes in the tokenizer and AST (astral-sh…

162d2eb

…#10314) Co-authored-by: Micha Reiser <[email protected]>

Spruce up docs for flake8-pyi rules (astral-sh#10422)

ae0ff9b

ruff server sets worker thread pool size based on the user's availa…

d9f1cdb

…ble cores (astral-sh#10399) ## Summary Fixes astral-sh#10369. ## Test Plan N/A

Avoid code comment detection in PEP 723 script tags (astral-sh#10464)

938118b

Closes astral-sh#10455.

Fix typo in string_token_flags.rs (astral-sh#10476)

ffd6e79

Allow SPDX license headers to exceed the line length (astral-sh#10481)

f7740a8

Closes astral-sh#10465.

chore: merge origin/main

653d9e2

apatrushev temporarily deployed to release March 19, 2024 22:56 — with GitHub Actions Inactive

fmt

4c37dd8

strmwalker temporarily deployed to release March 21, 2024 09:05 — with GitHub Actions Inactive

clippy

5a8c99a

strmwalker deployed to release March 21, 2024 10:19 — with GitHub Actions Active

strmwalker closed this Mar 26, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Merge master 20240320 #33

Merge master 20240320 #33

apatrushev commented Mar 19, 2024

strmwalker commented Mar 26, 2024

Merge master 20240320 #33

Merge master 20240320 #33

Conversation

apatrushev commented Mar 19, 2024

strmwalker commented Mar 26, 2024