Add support for running pipelines on Argo and Tekton

[lresende]

Red Hat ODH release 1.0 will be using Kubeflow Pipelines
with Tekton support, so we are enabling Elyra to compile
pipelines for both Argo (default) or Tekton based on
runtime configuration.

Fixes #1222

Developer's Certificate of Origin 1.1

   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.

[elyra-bot]

Thanks for making a pull request to Elyra!

To try out this branch on binder, follow this link:

[lresende]

KFP servers based on Argo and Tekton are now accepting and running the pipelines, but Tekton deployment is having issues displaying the results due to some metrics related issues (FYI @ptitzler)

[lresende]

On Argo:

On Tekton:

[ptitzler]

In a way this is perhaps not consistent with schema's default, which is Argo. If, for whatever reason, the runtime configuration does contain an invalid value XYZ, the logic would run the Tekton compiler.

[ptitzler]

This is also inconsistent with the logic in https://github.com/elyra-ai/elyra/blob/bde3b4f3d284407dc0f42a3c80f8983e1066b749/elyra/pipeline/processor_kfp.py#L157, where we test for Tekton.

[lresende]

other values should not really come in as the metadata is validated and invalid ones will not be shown in the user interface....

[ptitzler]

It appears that export does not yet take into account the engine type if Python DSL is generated. The Jinja template https://github.com/elyra-ai/elyra/blob/master/elyra/templates/kfp_template.jinja2#L43 always uses the Argo compiler (unless there is some other PR that contains additional changes.

[lresende]

The export has already been updated by @akchinSTC

[ptitzler]

Need a separate doc PR to reflect the changes in https://elyra.readthedocs.io/en/latest/user_guide/runtime-conf.html

[akchinSTC]

I am good with this up to the doc changes

[ptitzler]

The doc update looks good. I also ran a couple tests without issues.