Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(canvas snapshots): position mismatch in headless mode #33575

Merged
merged 8 commits into from
Nov 14, 2024

Conversation

Skn0tt
Copy link
Member

@Skn0tt Skn0tt commented Nov 13, 2024

The code for canvas rendering currently assumes that the screenshot it crops from has the same aspect ratio as the frame it's displaying in. That assumption is wrong. The fix is record the canvas position at snapshot time, so that it's relative to the screnshot aspect ratio.
Closes #33562.

This comment has been minimized.

@Skn0tt Skn0tt requested a review from dgozman November 13, 2024 14:28

This comment has been minimized.

This comment has been minimized.

@dgozman
Copy link
Contributor

dgozman commented Nov 13, 2024

The code for canvas rendering currently assumes that the screenshot it crops from has the same aspect ratio as the frame it's displaying in. That assumption is wrong. The fix is to only use the image and frame width when cropping, not the height.

Why width and not height? If anything, I'd assume that width might differ due to different scrollbars, but height would likely be correct. Do you have a real repro that shows the discrepancy in width and/or height?

To test this, I first tried comparing a screenshot of the snapshotted page with a screenshot of the rendered snapshot, but that didn't work because those screenshots had different dimensions. So I had to fall back to a visual regression test. Let me know if you have a better idea, or if it'd be better to not test it at all.

I'd skip the test in this case.

@Skn0tt
Copy link
Member Author

Skn0tt commented Nov 13, 2024

Why width and not height? If anything, I'd assume that width might differ due to different scrollbars, but height would likely be correct. Do you have a real repro that shows the discrepancy in width and/or height?

I was wondering the same. In the repro the customer shared, the height differed between headed and headless, but the width did not. If we know that the width will also differ because of scrollbars, then maybe this whole approach doesn't work and we need to capture dimensions together with the snapshot itself.

@Skn0tt
Copy link
Member Author

Skn0tt commented Nov 14, 2024

I've updated with the approach discussed yesterday, where we record the bounding rect at snapshot time and use that to identify the right img crop. Here's how it looks on the customer repro:

Screenshot 2024-11-14 at 09 11 53

You can see that the text isn't perfectly on top of one another, so the alignment also isn't perfect - but workable, i'd say. And more likely to be correct than what we had before.

This comment has been minimized.

Copy link
Contributor

@dgozman dgozman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that PR description is now wrong.

if (!boundingRectAttribute)
continue;

const boundingRect = JSON.parse(boundingRectAttribute) as { left: number, top: number, right: number, bottom: number };
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's try/catch parsing just in case.


const partiallyUncaptured = xEnd > 1 || yEnd > 1;
const fullyUncaptured = xStart > 1 || yStart > 1;
drawCheckerboard(context, canvas);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's keep not drawing the checkerboard for fullyUncaptured canvases, to avoid any performance penalties.

Copy link
Contributor

Test results for "tests 1"

1 flaky ⚠️ [playwright-test] › ui-mode-test-ct.spec.ts:117:5 › should run component tests after editing test and component @ubuntu-latest-node20-1

36885 passed, 650 skipped
✔️✔️✔️

Merge workflow run.

@Skn0tt Skn0tt merged commit 25c0394 into microsoft:main Nov 14, 2024
29 checks passed
Skn0tt added a commit that referenced this pull request Nov 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug]: Trace screenshots of canvas elements look incorrect
2 participants