-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[MAINTENANCE] Reduce test duration of flakey test #10663
Conversation
✅ Deploy Preview for niobium-lead-7998 canceled.
|
Codecov ReportAll modified and coverable lines are covered by tests ✅
✅ All tests successful. No failed tests found. Additional details and impacted files@@ Coverage Diff @@
## develop #10663 +/- ##
===========================================
- Coverage 80.45% 80.45% -0.01%
===========================================
Files 462 462
Lines 40102 40102
===========================================
- Hits 32264 32263 -1
- Misses 7838 7839 +1 Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If this test continues to be a problem, we can just remove it. It's shoehorning a benchmark into a test, and really we could use a more holistic performance assessment setup in CI.
This test fails with some regularity in CI, but not in the way the test expects. See this example. It fails with
Failed: Timeout >2.0s
because we currently invoke pytest with--timeout=2
, so we would would never fail the assertion as written.We could also throw a
mark.pytest.slow
on this instead of cutting the time in half, but I'm not sure if that matters.The fix here:
large_number
by half so we can see the expected error if appropriate. I'm assuming complexity is O(n) withlarge_number
.large_number
by another 25% to avoid the flakiness.invoke lint
(usesruff format
+ruff check
)For more information about contributing, visit our community resources.
After you submit your PR, keep the page open and monitor the statuses of the various checks made by our continuous integration process at the bottom of the page. Please fix any issues that come up and reach out on Slack if you need help. Thanks for contributing!