Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Improve makefile build commands #19498

Merged
merged 4 commits into from
Oct 28, 2024
Merged

Conversation

orlp
Copy link
Collaborator

@orlp orlp commented Oct 28, 2024

This does a couple things:

  1. We now properly match the CPU flags to that of our release. Specify LTS_CPU=1 e.g. make build-release LTS_CPU=1 to build a lts-cpu variant.
  2. You can now pass along arguments to maturin through the makefile commands, e.g. make build ARGS="--no-default-features".
  3. I redefined the build profiles. We now have (ordered by compilation speed):
Name Optimization Debug assertions Debug symbols Compilation Speed
debug No Yes Yes Fastest
nodebug-release Yes No No Slow
release Yes No Minimal Slower
debug-release Yes No Yes Slower still
dist-release Extreme No No Slowest

Almost all the time you want debug, which is just make build. If you are investigating performance, you want make build-release.

The remaining profiles are more niche. Use dist-release to exactly match the optimization we use when distributing, which is not necessary most of the time. The nodebug/debug variants of release exist because I found it adds significant (30-60 sec) compile time between them (all fresh compiles):

Apple M2
make build 1:50
make nodebug-release 4:51
make release 5:33
make debug-release 6:39
make dist-release ~20 (couldn't be bothered to run it)

Use nodebug if you are iterating on something performance-sensitive but you've already done profiling so you know what to improve without needing further profiling. Use debug when hunting down a bug with a debugger that only occurs in release mode.

@orlp orlp changed the title Makefile build refactor: Improve makefile build commands Oct 28, 2024
@github-actions github-actions bot added internal An internal refactor or improvement python Related to Python Polars rust Related to Rust Polars and removed title needs formatting labels Oct 28, 2024
Copy link

codspeed-hq bot commented Oct 28, 2024

CodSpeed Performance Report

Merging #19498 will improve performances by 35.4%

Comparing orlp:makefile-build (64ad3f5) with main (6a7d140)

Summary

⚡ 1 improvements
✅ 40 untouched benchmarks

Benchmarks breakdown

Benchmark main orlp:makefile-build Change
test_to_numpy_series_with_nulls 435.7 µs 321.8 µs +35.4%

@ritchie46 ritchie46 merged commit 3cded29 into pola-rs:main Oct 28, 2024
20 of 21 checks passed
@c-peters c-peters added the accepted Ready for implementation label Nov 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
accepted Ready for implementation internal An internal refactor or improvement python Related to Python Polars rust Related to Rust Polars
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants