Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Code Coverage] Codecov.io reports appear to be mostly broken #7015

Closed
unlikelyzero opened this issue Aug 29, 2023 · 4 comments · Fixed by #7582
Closed

[Code Coverage] Codecov.io reports appear to be mostly broken #7015

unlikelyzero opened this issue Aug 29, 2023 · 4 comments · Fixed by #7582
Labels
help_wanted Help the Open MCT project! needs:test instructions Missing testing notes type:maintenance tests, chores, or project maintenance verified Tested or intentionally closed
Milestone

Comments

@unlikelyzero
Copy link
Collaborator

unlikelyzero commented Aug 29, 2023

Summary

I noticed that we had a codecov.io report for release/3.0.0 which had the following calculations:

  • 48% total
  • 57% for the e2e-stable flag
  • 44% for e2e-full flag
  • 37% for unit flag

How is it possible that our total is 48% but the e2e-stable tests cover 57%?

I switched back to 'master' and noticed something our lowest coverage was for 'plot/plugins/mctPlot.vue'. We which shouldn't near-zero (0.27%) as this area is heavily exercised in our e2e suites.

Opening the plot/plugins/MctPlot.vue revealed that the two lines which were covered appeared to be randomly distributed
Screenshot 2023-08-29 at 12 19 06 PM

We'll likely need to do a deeper investigation into why the code coverage is mostly inaccurate.

Probable causes:

@unlikelyzero unlikelyzero added help_wanted Help the Open MCT project! type:maintenance tests, chores, or project maintenance labels Aug 29, 2023
@evenstensberg
Copy link
Contributor

evenstensberg commented Aug 31, 2023

@unlikelyzero Could you explain the process of generating a coverage report? Reason for asking is that this PR is probably better to be done with someone that has CI and codecov permissions.

@unlikelyzero unlikelyzero changed the title [Code Coverage] Code Cov.io reports appear to be mostly broken [Code Coverage] Codecov.io reports appear to be mostly broken Sep 1, 2023
@unlikelyzero
Copy link
Collaborator Author

unlikelyzero commented Sep 1, 2023

@evenstensberg I'm not ready to blame codecov.io quite yet. It's possible that we're generating the coverage incorreclty and codecov.io is just the bearer of bad news. Please see this documention update and provide feedback https://github.com/nasa/openmct/pull/7027/files

@msf-caesar
Copy link

Try with:

  1. Double-checking the sourcemaps configuration for test suites and code coverage tooling (karma istanbul).
  2. Verifying that the npm run start:coverage script is correctly set up.
  3. Investigating if the issue is related to the known root cause mentioned in the GitHub issue.
  4. Running additional tests and analyzing test results to identify any areas that are not being properly covered.
  5. Considering updating the code coverage tooling or configuration if necessary.

@evenstensberg
Copy link
Contributor

@unlikelyzero okay, I'll have a look at the build.

@ozyx ozyx self-assigned this Mar 12, 2024
@ozyx ozyx added this to the Target:4.0.0 milestone Mar 12, 2024
unlikelyzero added a commit that referenced this issue Mar 13, 2024
…#7582)

* fix(?): the robot says to do this...

* refactor: remove unused env var

---------

Co-authored-by: John Hill <[email protected]>
@unlikelyzero unlikelyzero added the needs:test instructions Missing testing notes label Mar 19, 2024
@unlikelyzero unlikelyzero added the verified Tested or intentionally closed label Mar 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help_wanted Help the Open MCT project! needs:test instructions Missing testing notes type:maintenance tests, chores, or project maintenance verified Tested or intentionally closed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants