Skip to content

Use package.include in all published crates#497

Merged
jeertmans merged 1 commit intomaciejhirsz:masterfrom
42triangles:master
Aug 22, 2025
Merged

Use package.include in all published crates#497
jeertmans merged 1 commit intomaciejhirsz:masterfrom
42triangles:master

Conversation

@42triangles
Copy link
Contributor

@42triangles 42triangles commented Aug 20, 2025

I came across the issue that the whole mdbook is included in the published crates during a dependency review, but logos.png is especially big and doesn't have to be included for everything to build.

This PR makes use of package.include to not package these files.
The excluded files based on the current version would be: RELEASE-PROCESS.md, book/**, examples/**, logos.{svg,png}, release.toml, tests/** and logos-{codegen,cli}/tests/**.

This reduces the size of almost all crates, but especially logos (note the change in units):

crate full without package.include compressed full with package.include compressed
logos-codegen 275.1KiB 54.8KiB 211.0KiB 49.3KiB
logos-derive 20.4KiB 8.0KiB 20.5KiB 8.0KiB
logos 1.8MiB 1.6MiB 59.0KiB 18.9KiB
logos-cli 48.5KiB 15.5KiB 41.4KiB 14.1KiB

Changes of data used (using the number of downloads of the previous version, as of 2025-8-20):

crate per download number of downloads per month bandwidth/month
logos-codegen -5.5KiB 149416 -802.5MiB
logos-derive ±0.0KiB 149368 ±0.0KiB
logos -1.6MiB 149522 -233.6GiB
logos-cli -1.4KiB 90 -126.0KiB

Though it should be noted that older versions also see a lot of downloads as well, so if the same holds for future versions, the actual bandwidth usage decrease would be much more pronounced over time.
Using all versions over the last 90 days, the logos crate would end up with around 2.0TiB/month less bandwidth usage.

@codecov
Copy link

codecov bot commented Aug 20, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 63.27%. Comparing base (331da82) to head (57ff85f).
⚠️ Report is 2 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #497      +/-   ##
==========================================
+ Coverage   63.23%   63.27%   +0.04%     
==========================================
  Files          36       36              
  Lines        2377     2377              
==========================================
+ Hits         1503     1504       +1     
+ Misses        874      873       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jeertmans jeertmans added the continuous integration GitHub workflows and others label Aug 20, 2025
@codspeed-hq
Copy link

codspeed-hq bot commented Aug 20, 2025

CodSpeed Performance Report

Merging #497 will not alter performance

Comparing 42triangles:master (57ff85f) with master (331da82)

Summary

✅ 6 untouched benchmarks

Copy link
Collaborator

@jeertmans jeertmans left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @42triangles, very nice PR! Thanks for your analysis :-)

See my small comment about source files inclusion.

Cargo.toml Outdated
repository = "https://github.com/maciejhirsz/logos"
rust-version = "1.74.0"
version = "0.15.1"
include = ["build.rs", "/src", "LICENSE-MIT", "LICENSE-APACHE"]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
include = ["build.rs", "/src", "LICENSE-MIT", "LICENSE-APACHE"]
include = ["build.rs", "src/**", "LICENSE-MIT", "LICENSE-APACHE"]

I think it should be src/**, per https://doc.rust-lang.org/cargo/reference/manifest.html#the-exclude-and-include-fields

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The example on https://doc.rust-lang.org/cargo/reference/manifest.html#the-exclude-and-include-fields is

include = ["/src", "COPYRIGHT", "/examples", "!/examples/big_example"]

which also does include the full directory (while src would be equivalent to **/src based on my reading)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's just that /src matches any file or directly named src. While src/ matches any directory named src. That's a slight difference that will likely never bother us.

Using stars (src/**) is probably useless if one wants to include everything anyway.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which do you prefer? (I personally prefer /src or /src/, but I'm happy to change it to something else too ^^)

@42triangles
Copy link
Contributor Author

Though the example from https://doc.rust-lang.org/cargo/reference/manifest.html#the-exclude-and-include-fields also does remind me: With these changes, cargo publish will throw warnings about ignoring examples & tests that aren't included in the final package, too, since they are still referenced in the Cargo.tomls, ie:

warning: ignoring example `brainfuck` as `examples/brainfuck.rs` is not included in the published package

but the resulting package does not need them to build, so that's probably fine - forgot to mention that earlier, my bad!

@jeertmans
Copy link
Collaborator

Though the example from https://doc.rust-lang.org/cargo/reference/manifest.html#the-exclude-and-include-fields also does remind me: With these changes, cargo publish will throw warnings about ignoring examples & tests that aren't included in the final package, too, since they are still referenced in the Cargo.tomls, ie:

warning: ignoring example `brainfuck` as `examples/brainfuck.rs` is not included in the published package

but the resulting package does not need them to build, so that's probably fine - forgot to mention that earlier, my bad!

I think it's still very common to include them in the package: I don't know why, but that might be worth it.

@42triangles
Copy link
Contributor Author

42triangles commented Aug 20, 2025

With examples & tests added ("/examples", "/tests"), it results in this (again, the main difference is logos.png):

crate full without package.include compressed full with package.include compressed
logos-codegen 275.1KiB 54.8KiB 211.0KiB ⇒ 275.2KiB 49.3KiB ⇒ 54.8KiB
logos-derive 20.4KiB 8.0KiB 20.5KiB ⇒ 20.6KiB 8.0KiB ⇒ 8.0KiB
logos 1.8MiB 1.6MiB 59.0KiB ⇒ 105.0KiB 18.9KiB ⇒ 28.1KiB
logos-cli 48.5KiB 15.5KiB 41.4KiB ⇒ 48.6KiB 14.1KiB ⇒ 15.5KiB
crate per download number of downloads per month bandwidth/month
logos-codegen ±0.0KiB 149416 ±0.0KiB
logos-derive ±0.0KiB 149368 ±0.0KiB
logos -1.6MiB 149522 -233.6GiB
logos-cli ±0.0KiB 90 ±0.0KiB

It would still be 1.5× the size for logos though.

(Also I am very confused why logos-derive gains an additional 0.1KiB with that, but that was the output of cargo publish --workspace --dry-run)

@jeertmans
Copy link
Collaborator

With examples & tests added ("/examples", "/tests"), it results in this (again, the main difference is logos.png):

crate full without package.include compressed full with package.include compressed
logos-codegen 275.1KiB 54.8KiB 211.0KiB ⇒ 275.2KiB 49.3KiB ⇒ 54.8KiB
logos-derive 20.4KiB 8.0KiB 20.5KiB ⇒ 20.6KiB 8.0KiB ⇒ 8.0KiB
logos 1.8MiB 1.6MiB 59.0KiB ⇒ 105.0KiB 18.9KiB ⇒ 28.1KiB
logos-cli 48.5KiB 15.5KiB 41.4KiB ⇒ 48.6KiB 14.1KiB ⇒ 15.5KiB
crate per download number of downloads per month bandwidth/month
logos-codegen ±0.0KiB 149416 ±0.0KiB
logos-derive ±0.0KiB 149368 ±0.0KiB
logos -1.6MiB 149522 -233.6GiB
logos-cli ±0.0KiB 90 ±0.0KiB
It would still be 1.5× the size for logos though.

(Also I am very confused why logos-derive gains an additional 0.1KiB with that, but that was the output of cargo publish --workspace --dry-run)

Did you check what files were included with cargo package --list?

@42triangles
Copy link
Contributor Author

Did you check what files were included with cargo package --list?

Yup, all tests & examples are included as expected (since it's always relative to the Cargo.toml it's referenced in, so ie logos-codegen/tests is included from logos-codegen/Cargo.toml; and tests in the root directory of the repository isn't because it's its own crate)

This reduces the size of the `logos` considerably, while others
stay the same regarding their compressed size.
Excluding `RELEASE-PROCESS.md`, `book/**`, `logos.svg`, `logos.png`
(this one is especially large) and `release.toml` reduces the compressed
size of the `logos` crate from 1.8MiB to 28.1KiB, resulting in a very
conservative estimate of a reduction of 233.6GiB monthly for downloading
the crate from crates.io (using the number of downloads of the previous
version, as of 2025-08-20).

It would be possible to also remove tests & examples for even further
reductions in size:
                without `package.include`   with `package.include`
                full        compressed      full        compressed
logos-codegen   275.1KiB       54.8KiB      211.0KiB       49.3KiB
logos-derive     20.4KiB        8.0KiB       20.5KiB        8.0KiB
logos             1.8MiB        1.6MiB       59.0KiB       18.9KiB
logos-cli        48.5KiB       15.5KiB       41.4KiB       14.1KiB

Changes of data used in that case:
                per dl  #dls/month  bandwidth/month
logos-codegen   -5.5KiB     149416  -802.5MiB
logos-derive    ±0.0KiB     149368    ±0.0KiB
logos           -1.6MiB     149522  -233.6GiB
logos-cli       -l.4KiB         90  -126.0KiB
@42triangles 42triangles changed the title Use package.exclude in all published crates Use package.include in all published crates Aug 22, 2025
@jeertmans
Copy link
Collaborator

Looks good to me, thanks @42triangles! Maybe we could further optimize this, but that's already big savings in terms of bandwidth :-)

@jeertmans jeertmans merged commit 82f6a11 into maciejhirsz:master Aug 22, 2025
58 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

continuous integration GitHub workflows and others

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants