Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add pytest suite for airflow processor #1317

Merged
merged 7 commits into from
Mar 12, 2021
Merged

Conversation

akchinSTC
Copy link
Member

See original PR #490 for more details

Developer's Certificate of Origin 1.1

   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.

@elyra-bot
Copy link

elyra-bot bot commented Feb 20, 2021

Thanks for making a pull request to Elyra!

To try out this branch on binder, follow this link: Binder

@akchinSTC
Copy link
Member Author

depends and based on #1316

@akchinSTC akchinSTC added the status:Work in Progress Development in progress. A PR tagged with this label is not review ready unless stated otherwise. label Feb 20, 2021
Copy link
Member

@kevin-bates kevin-bates left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for putting this together. I just had a comment on the fixture names.

Also, there's a typo in the test filename: test_airlfow_processory.py

@akchinSTC akchinSTC force-pushed the af-tests branch 2 times, most recently from e8df8f7 to 6fbf5bf Compare February 23, 2021 22:57
@akchinSTC akchinSTC removed the status:Work in Progress Development in progress. A PR tagged with this label is not review ready unless stated otherwise. label Feb 23, 2021
Adds a basic test suite specific to the airflow processor.
Related elyra-ai#185
@ptitzler
Copy link
Member

ptitzler commented Mar 1, 2021

Will a github method test be added separately?

@akchinSTC
Copy link
Member Author

@ptitzler -yes, will add tests for the git utils, but can go separately in another PR

@akchinSTC akchinSTC added this to the 2.1.0 milestone Mar 8, 2021
@lresende
Copy link
Member

lresende commented Mar 9, 2021

@ptitzler and @kevin-bates It seems that you guys have been reviewing this, so I will wait for your final approvals before merging.

Copy link
Member

@ptitzler ptitzler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM



def test_create_file(monkeypatch, processor, parsed_pipeline, parsed_ordered_dict, sample_metadata):
correct_hash_value = '9a15eba3337ba3c90457d5b60495720333f944fd2cec8e4ce40c32238d2a4206'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add a comment indicating how this hash value is computed so folks making changes know what to do?

Frankly. I'd prefer we validate the export w/o this dependency, perhaps by looking for expected values in the output python file - or things like that. This hash stuff has proven to be quite fragile and was abandoned in kfp-notebook.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 abandoning the hash stuff. Ill refactor to verify via expected values

@akchinSTC
Copy link
Member Author

Im working through parsing the generated airflow DAG file. Will have an updated PR no later than EOD today. Attempted to use the ast lib to parse but that felt heavy.

Copy link
Member

@kevin-bates kevin-bates left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - took tests for a spin - all good. Thanks for removing the hash check - this will be much more sustainable.

@lresende lresende merged commit 9ba9cc8 into elyra-ai:master Mar 12, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants