Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add pipeline source annotations to container ops #1331

Merged
merged 10 commits into from
Feb 27, 2021
Merged

Add pipeline source annotations to container ops #1331

merged 10 commits into from
Feb 27, 2021

Conversation

ptitzler
Copy link
Member

@ptitzler ptitzler commented Feb 23, 2021

This PR adds the elyra/pipeline-source metadata to KFP pipelines that are

  • run from the pipeline editor,
  • exported to YAML,
  • exported to DSL Python, or
  • created implicitly when a notebook is submitted.

For the scenarios listed above the elyra/pipeline-source property is set as follows:

  • run from pipeline edior: the pipeline file name, e.g. mypipeline.pipeline
  • exported to YAML: the pipeline file name, e.g. mypipeline.pipeline
  • exported to DSL Python: the pipeline file name, e.g. mypipeline.pipeline
  • created implicitly during notebook submission: static string, e.g. Generated for thisnotebook.ipynb

Additional PRs are required to attach the metadata for pipelines that are

  • created implicitly when a Python script is submitted
  • run from the CLI.

The PR

To test:

$ git clone https://github.com/ptitzler/elyra.git
$ cd elyra
$ git checkout i1299
$ make clean install
$ cd ..
$ git clone https://github.com/ptitzler/kfp-notebook.git
$ cd kfp-notebook
$ git checkout add-pipelinesource-annotation
$ pip uninstall -y kfp-notebook
$ make clean install
$ cd ..
$ export ELYRA_GITHUB_BRANCH=add-pipelinesource-annotation
$ export ELYRA_GITHUB_ORG=ptitzler
$ jupyter lab 

Developer's Certificate of Origin 1.1

   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.

bourdakos1 and others added 6 commits February 19, 2021 10:08
This adds support for pipeline versioning and enables the pod labels
and annotations to help administrators identify which pods are
associated with Elyra pipelines (in general) as a well as enabling
correlation with the artifact associated with the pod.
@elyra-bot
Copy link

elyra-bot bot commented Feb 23, 2021

Thanks for making a pull request to Elyra!

To try out this branch on binder, follow this link: Binder

@ptitzler ptitzler changed the title I1299 Add pipeline source annotations to container ops Feb 23, 2021
Co-authored-by: Alan Chin <akchin@us.ibm.com>
@karlaspuldaro karlaspuldaro mentioned this pull request Feb 24, 2021
4 tasks
@ptitzler ptitzler added kind:enhancement New feature or request platform: pipeline-Kubeflow Related to usage of Kubeflow Pipelines as pipeline runtime labels Feb 27, 2021
@lresende lresende merged commit 7fd2bb6 into elyra-ai:master Feb 27, 2021
@ptitzler ptitzler deleted the i1299 branch March 1, 2021 14:37
@lresende lresende added this to the 2.1.0 milestone Mar 11, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind:enhancement New feature or request platform: pipeline-Kubeflow Related to usage of Kubeflow Pipelines as pipeline runtime
Projects
None yet
4 participants