Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make pipeline process and export asynchronous #695

Merged
merged 1 commit into from
Jun 29, 2020

Conversation

kevin-bates
Copy link
Member

@kevin-bates kevin-bates commented Jun 27, 2020

  • PipelineManager now invokes the process and export methods via a loop executor allowing asynchronous behavior via the REST API.
  • Converted all REST methods from @gen.coroutine/yield to async def/await.

Resolves #576

Developer's Certificate of Origin 1.1

   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.

@kevin-bates kevin-bates marked this pull request as draft June 27, 2020 15:13
@kevin-bates
Copy link
Member Author

The exception chaining work here should be delivered in a separate PR that can fully address #680, so I'm moving this to draft and will resubmit once that separation is complete.

PipelineManager now invokes the process and export methods via a loop executor
allowing asynchronous behavior via the REST API.

Also converted all REST method from `@gen.coroutine/yield` to `async def/await`.
@kevin-bates kevin-bates marked this pull request as ready for review June 27, 2020 15:23
@lresende
Copy link
Member

Thank you @kevin-bates , this makes submitting the COVID scenario pipeline 4x faster, and fix the issues related to hanging JupyterLab actions

@lresende lresende merged commit 845d5d3 into elyra-ai:master Jun 29, 2020
@kevin-bates kevin-bates deleted the async-pipeline-api branch July 7, 2020 19:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Submitting pipeline with large dependencies disable other JupyterLab actions
2 participants