add tags and descriptions to default runtime images

[ptitzler]

This PR

• adds tags and descriptions to all pre-defined runtime images
• updates the Makefile to address a parsing error that was triggered by above changes
• closes Default "runtime images" metadata should include description and tags #1022

Developer's Certificate of Origin 1.1

   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.

[elyra-bot]

Thanks for making a pull request to Elyra!

To try out this branch on binder, follow this link:

[ptitzler]

Looks like the Makefile doesn't handle the JSON file parsing properly
(https://github.com/ptitzler/elyra/blob/a01cd9b723f96ea020e9d81cb7faaa63a9317748/Makefile#L159), causing image validation to fail:

--> etc/config/metadata/runtime-images/tensorflow_py3.json
--> elyra/tensorflow:1.15.2-py3,
Checking elyra/tensorflow:1.15.2-py3, in etc/config/metadata/runtime-images/tensorflow_py3.json for curl...
Image elyra/tensorflow:1.15.2-py3, is not present, pulling...
ERROR: Image elyra/tensorflow:1.15.2-py3, did not meet criteria for command: curl
Checking elyra/tensorflow:1.15.2-py3, in etc/config/metadata/runtime-images/tensorflow_py3.json for python3...
Image elyra/tensorflow:1.15.2-py3, is not present, pulling...
ERROR: Image elyra/tensorflow:1.15.2-py3, did not meet criteria for command: python3
Removing image elyra/tensorflow:1.15.2-py3,...
Error response from daemon: invalid reference format
make: *** [validate-runtime-images] Error 1

Will take a look

[kevin-bates]

These changes look good Patrick - I like the choice of tags. I guess we should hold off on this merge until #985 is merged - correct? Or does the schema validation tolerate the additive properties?

Just had the one suggestion for the Makefile validation - thanks for catching that.

[kevin-bates]

Thanks Patrick.

[ptitzler]

I guess we should hold off on this merge until #985 is merged - correct? Or does the schema validation tolerate the additive properties?

I tested using the master branch, which didn't include the PR. Everything appeared to work as expected. Unless I am missing something this PR should be safe to merge.

[lresende]

Looks like the Makefile doesn't handle the JSON file parsing properly
(https://github.com/ptitzler/elyra/blob/a01cd9b723f96ea020e9d81cb7faaa63a9317748/Makefile#L159), causing image validation to fail:

--> etc/config/metadata/runtime-images/tensorflow_py3.json
--> elyra/tensorflow:1.15.2-py3,
Checking elyra/tensorflow:1.15.2-py3, in etc/config/metadata/runtime-images/tensorflow_py3.json for curl...
Image elyra/tensorflow:1.15.2-py3, is not present, pulling...
ERROR: Image elyra/tensorflow:1.15.2-py3, did not meet criteria for command: curl
Checking elyra/tensorflow:1.15.2-py3, in etc/config/metadata/runtime-images/tensorflow_py3.json for python3...
Image elyra/tensorflow:1.15.2-py3, is not present, pulling...
ERROR: Image elyra/tensorflow:1.15.2-py3, did not meet criteria for command: python3
Removing image elyra/tensorflow:1.15.2-py3,...
Error response from daemon: invalid reference format
make: *** [validate-runtime-images] Error 1

Have you changed the image tags? There are some images that don't have curl and/or python3 and that is a blocker for us.

[lresende]

Also, I noticed we are adding an image description and we already had display_name that was used to display the entry to the user. If we are going to make it duplicated, I would at least put them in sync or even remove display_name if that is not required.

[kevin-bates]

or even remove display_name if that is not required.

It is required. I think some of the descriptions Patrick chose to make different are fine - like using parenthesized values, etc.

What is the reason for making them the same?

[ptitzler]

Have you changed the image tags? There are some images that don't have curl and/or python3 and that is a blocker for us.

No. The problem is that the existing parsing logic didn't properly extract the image name from the json file, resulting in a trailing comma: elyra/tensorflow:1.15.2-py3,. As a result the image pull of elyra/tensorflow:1.15.2-py3, failed. Our existing error handling didn't detect the issue and therefore raises "bogus" errors.

[ptitzler]

Also, I noticed we are adding an image description and we already had display_name that was used to display the entry to the user.

The description was there all along but we never specified any values, henceI created the issue.

{
  "display_name": "Tensorflow 2.3.0 with GPU",
  "metadata": {
    "description": "TensorFlow 2.3 (with GPU support)",
    "image_name": "tensorflow/tensorflow:2.3.0-gpu",
    "tags": ["gpu", "tensorflow"]
  },
  "schema_name": "runtime-image"
}

But you are right in that the description is currently nowhere surfaced to the user outside of the editor. Why it isn't I don't know.

[ptitzler]

I do think having a description is important to convey information that enables the user to make a well-informed decision which image to pick. Granted there is currently no additional information in the descriptions I've added but that doesn't mean my draft proposals cannot be tweaked to do that, if neccessary.

[lresende]

I do think having a description is important to convey information that enables the user to make a well-informed decision which image to pick. Granted there is currently no additional information in the descriptions I've added but that doesn't mean my draft proposals cannot be tweaked to do that, if neccessary.

Agree, which in this case I would expect that the description would provide more info than what is on the display name, but in some cases, we are providing less info.

[ptitzler]

I can make the descriptions the same if that addresses your concern.