Add test for `ShapeEnv` recording fallback. #109944

ysiraichi · 2023-09-23T17:55:29Z

Stack from ghstack (oldest at bottom):

-> Add test for ShapeEnv recording fallback. #109944

This PR adds a test for the previous PR in this stack: #109904. In summary, it calls
functions decorated with @record_shapeenv_event, that don't have an explicit ShapeEnv
parameter, with arguments that don't hold a ShapeEnv instance.

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng

This PR adds a test for the previous PR in this stack: #109904. In summary, it calls functions decorated with `@record_shapeenv_event`, that don't have an explicit `ShapeEnv` parameter, with arguments that don't hold a `ShapeEnv` instance. [ghstack-poisoned]

pytorch-bot · 2023-09-23T17:55:33Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/109944

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit a0bb2f0 with merge base ac3190c ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

This PR adds a test for the previous PR in this stack: #109904. In summary, it calls functions decorated with `record_shapeenv_event`, that don't have an explicit `ShapeEnv` parameter, with arguments that don't hold a `ShapeEnv` instance. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx chenyang78 aakhundov kadeng [ghstack-poisoned]

ezyang · 2023-09-25T17:13:03Z

@pytorchbot merge

pytorchmergebot · 2023-09-25T17:15:42Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

This PR adds a test for checking `ShapeEnv` state when it's built with `should_record_events=False`. Pull Request resolved: #109945 Approved by: https://github.com/ezyang ghstack dependencies: #109904, #109944

atalman · 2023-09-26T13:07:14Z

This is failing internally

=================================== FAILURES ===================================
_____________ MiscTests.test_shape_env_recorded_function_fallback ______________

self = <caffe2.test.dynamo.test_misc.MiscTests testMethod=test_shape_env_recorded_function_fallback>

    def test_shape_env_recorded_function_fallback(self):
        # Make sure the record/replay mechanism for ShapeEnv will fallback
        # if no ShapeEnv instance is found.
>       constrain_range(5, min=2, max=10)

../buck-out/v2/gen/fbcode/363cd7e240f5d021/caffe2/test/dynamo/__test_dynamo__/test_dynamo#link-tree/caffe2/test/dynamo/test_misc.py:7506: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../buck-out/v2/gen/fbcode/363cd7e240f5d021/caffe2/test/dynamo/__test_dynamo__/test_dynamo#link-tree/torch/fx/experimental/recording.py:232: in wrapper
    self = _extract_shape_env_and_assert_equal(args, kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = (5,), kwargs = {'max': 10, 'min': 2}


=================================== FAILURES ===================================
_ DynamicShapesMiscTests.test_shape_env_recorded_function_fallback_dynamic_shapes _

self = <torch._dynamo.testing.DynamicShapesMiscTests testMethod=test_shape_env_recorded_function_fallback_dynamic_shapes>

    def test_shape_env_recorded_function_fallback(self):
        # Make sure the record/replay mechanism for ShapeEnv will fallback
        # if no ShapeEnv instance is found.
>       constrain_range(5, min=2, max=10)

../buck-out/v2/gen/fbcode/363cd7e240f5d021/caffe2/test/dynamo/__test_dynamo__/test_dynamo#link-tree/caffe2/test/dynamo/test_misc.py:7506: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../buck-out/v2/gen/fbcode/363cd7e240f5d021/caffe2/test/dynamo/__test_dynamo__/test_dynamo#link-tree/torch/fx/experimental/recording.py:232: in wrapper
    self = _extract_shape_env_and_assert_equal(args, kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = (5,), kwargs = {'max': 10, 'min': 2}

atalman · 2023-09-26T13:09:36Z

@pytorchbot revert -m "New test failing internally" -c ghfirst

pytorchmergebot · 2023-09-26T13:11:14Z

@pytorchbot successfully started a revert job. Check the current status here.
Questions? Feedback? Please reach out to the PyTorch DevX Team

pytorchmergebot · 2023-09-26T13:11:27Z

@ysiraichi your PR has been successfully reverted.

This reverts commit a4dec8d. Reverted #109944 on behalf of https://github.com/atalman due to New test failing internally ([comment](#109944 (comment)))

ysiraichi · 2023-09-26T14:49:29Z

@ezyang @atalman Could you tell me what is the error raised there?

ezyang · 2023-09-26T20:25:01Z

atalman didn't post the full log. Here is the full log

============================= test session starts ==============================
platform linux -- Python 3.8.6, pytest-7.2.2, pluggy-0.13.1
rootdir: /data/sandcastle/boxes/eden-trunk-hg-fbcode-fbsource/fbcode
plugins: hypothesis-6.70.1
collected 337 items

. F

=================================== FAILURES ===================================
_____________ MiscTests.test_shape_env_recorded_function_fallback ______________

self = <caffe2.test.dynamo.test_misc.MiscTests testMethod=test_shape_env_recorded_function_fallback>

    def test_shape_env_recorded_function_fallback(self):
        # Make sure the record/replay mechanism for ShapeEnv will fallback
        # if no ShapeEnv instance is found.
>       constrain_range(5, min=2, max=10)

../buck-out/v2/gen/fbcode/363cd7e240f5d021/caffe2/test/dynamo/__test_dynamo__/test_dynamo#link-tree/caffe2/test/dynamo/test_misc.py:7506: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../buck-out/v2/gen/fbcode/363cd7e240f5d021/caffe2/test/dynamo/__test_dynamo__/test_dynamo#link-tree/torch/fx/experimental/recording.py:232: in wrapper
    self = _extract_shape_env_and_assert_equal(args, kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = (5,), kwargs = {'max': 10, 'min': 2}

    def _extract_shape_env_and_assert_equal(args, kwargs):
        from torch.fx.experimental.symbolic_shapes import ShapeEnv, SymTypes
    
        def assert_equal(old: Optional[ShapeEnv], new: ShapeEnv) -> ShapeEnv:
            if old is not None:
                assert old is new, "call with different ShapeEnv"
            return new
    
        shape_env = None
        for val in itertools.chain(args, kwargs.values()):
            if isinstance(val, ShapeEnv):
                shape_env = assert_equal(shape_env, val)
            if isinstance(val, SymTypes):
                shape_env = assert_equal(shape_env, val.node.shape_env)
    
>       assert shape_env is not None, "ShapeEnv not found"
E       AssertionError: ShapeEnv not found

../buck-out/v2/gen/fbcode/363cd7e240f5d021/caffe2/test/dynamo/__test_dynamo__/test_dynamo#link-tree/torch/fx/experimental/recording.py:188: AssertionError
=========================== short test summary info ============================
FAILED ::MiscTests::test_shape_env_recorded_function_fallback - AssertionErro...
============================== 1 failed in 8.48s ===============================


self = <caffe2.test.dynamo.test_misc.MiscTests testMethod=test_shape_env_recorded_function_fallback>

    def test_shape_env_recorded_function_fallback(self):
        # Make sure the record/replay mechanism for ShapeEnv will fallback
        # if no ShapeEnv instance is found.
>       constrain_range(5, min=2, max=10)

../buck-out/v2/gen/fbcode/363cd7e240f5d021/caffe2/test/dynamo/__test_dynamo__/test_dynamo#link-tree/caffe2/test/dynamo/test_misc.py:7506: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../buck-out/v2/gen/fbcode/363cd7e240f5d021/caffe2/test/dynamo/__test_dynamo__/test_dynamo#link-tree/torch/fx/experimental/recording.py:232: in wrapper
    self = _extract_shape_env_and_assert_equal(args, kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = (5,), kwargs = {'max': 10, 'min': 2}

    def _extract_shape_env_and_assert_equal(args, kwargs):
        from torch.fx.experimental.symbolic_shapes import ShapeEnv, SymTypes
    
        def assert_equal(old: Optional[ShapeEnv], new: ShapeEnv) -> ShapeEnv:
            if old is not None:
                assert old is new, "call with different ShapeEnv"
            return new
    
        shape_env = None
        for val in itertools.chain(args, kwargs.values()):
            if isinstance(val, ShapeEnv):
                shape_env = assert_equal(shape_env, val)
            if isinstance(val, SymTypes):
                shape_env = assert_equal(shape_env, val.node.shape_env)
    
>       assert shape_env is not None, "ShapeEnv not found"
E       AssertionError: ShapeEnv not found

../buck-out/v2/gen/fbcode/363cd7e240f5d021/caffe2/test/dynamo/__test_dynamo__/test_dynamo#link-tree/torch/fx/experimental/recording.py:188: AssertionError

ysiraichi · 2023-09-26T20:42:02Z

@ezyang thank you for the log. It seems that it was executed without the previous PR in this stack (#109904). Let me rebase it.

This PR adds a test for the previous PR in this stack: #109904. In summary, it calls functions decorated with `record_shapeenv_event`, that don't have an explicit `ShapeEnv` parameter, with arguments that don't hold a `ShapeEnv` instance. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx chenyang78 aakhundov kadeng [ghstack-poisoned]

This PR adds a test for the previous PR in this stack: #109904. In summary, it calls functions decorated with `record_shapeenv_event`, that don't have an explicit `ShapeEnv` parameter, with arguments that don't hold a `ShapeEnv` instance. ghstack-source-id: eaa3bafd7d13e3fac2e9cab4e2b2ea93b7a32a57 Pull Request resolved: #109944

ezyang · 2023-09-27T00:20:23Z

@pytorchbot merge

pytorchmergebot · 2023-09-27T00:27:34Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

pytorch-bot bot added the release notes: fx release notes category label Sep 23, 2023

This was referenced Sep 23, 2023

Don't record function call if ShapeEnv is not found. #109904

Closed

Add test for ShapeEnv state when not recording. #109945

Closed

github-actions bot added module: dynamo ciflow/inductor labels Sep 23, 2023

pytorchbot added the open source label Sep 23, 2023

ysiraichi requested a review from ezyang September 25, 2023 16:44

ezyang approved these changes Sep 25, 2023

View reviewed changes

pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Sep 25, 2023

pytorchmergebot added the merging label Sep 25, 2023

pytorchmergebot added Merged and removed merging labels Sep 25, 2023

pytorchmergebot closed this in a4dec8d Sep 25, 2023

pytorchmergebot added the Reverted label Sep 26, 2023

ezyang reopened this Sep 26, 2023

pytorchmergebot added the merging label Sep 27, 2023

pytorchmergebot removed the merging label Sep 27, 2023

pytorchmergebot closed this in 51a8c16 Sep 27, 2023

kit1980 removed the Reverted label Sep 29, 2023

facebook-github-bot deleted the gh/ysiraichi/92/head branch September 30, 2023 14:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add test for `ShapeEnv` recording fallback. #109944

Add test for `ShapeEnv` recording fallback. #109944

ysiraichi commented Sep 23, 2023 •

edited

Loading

pytorch-bot bot commented Sep 23, 2023 •

edited

Loading

ezyang commented Sep 25, 2023

pytorchmergebot commented Sep 25, 2023

atalman commented Sep 26, 2023 •

edited

Loading

atalman commented Sep 26, 2023

pytorchmergebot commented Sep 26, 2023

pytorchmergebot commented Sep 26, 2023

ysiraichi commented Sep 26, 2023

ezyang commented Sep 26, 2023

ysiraichi commented Sep 26, 2023

ezyang commented Sep 27, 2023

pytorchmergebot commented Sep 27, 2023

Add test for ShapeEnv recording fallback. #109944

Add test for ShapeEnv recording fallback. #109944

Conversation

ysiraichi commented Sep 23, 2023 • edited Loading

pytorch-bot bot commented Sep 23, 2023 • edited Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/109944

✅ No Failures

ezyang commented Sep 25, 2023

pytorchmergebot commented Sep 25, 2023

Merge started

atalman commented Sep 26, 2023 • edited Loading

atalman commented Sep 26, 2023

pytorchmergebot commented Sep 26, 2023

pytorchmergebot commented Sep 26, 2023

ysiraichi commented Sep 26, 2023

ezyang commented Sep 26, 2023

ysiraichi commented Sep 26, 2023

ezyang commented Sep 27, 2023

pytorchmergebot commented Sep 27, 2023

Merge started

Add test for `ShapeEnv` recording fallback. #109944

Add test for `ShapeEnv` recording fallback. #109944

ysiraichi commented Sep 23, 2023 •

edited

Loading

pytorch-bot bot commented Sep 23, 2023 •

edited

Loading

atalman commented Sep 26, 2023 •

edited

Loading