Importing annotations without deleting current ones

[domef]

My actions before raising this issue

• Read/searched the docs
• Searched past issues

When importing annotations into a task, CVAT deletes current annotations. Is there a way to keep them and have both sets of annotations in the same task?

[ArtMarciano]

Im having the same issue. Was it resolved?

[OmarHassan2334]

Can you describe the problem in more detail like what web browser you're using and what step you took for the issue to occur, also please share images to show the problem better.

[OmarHassan2334]

Hey @nmanovic, I was also able to replicate the error and would like to have a go at it.

[OmarHassan2334]

Hey @nmanovic, I was also able to replicate the error and would like to have a go at it.

[nmanovic]

@OmarHassan2334 , could you please send me an email with a screenshot of debugging of CVAT? I cannot locate it. If you read instructions, only after you confirm that you setup the development environment, I can assign an issue.

[EBayego]

Hi @nmanovic, I will take on this other issue to solve it

[zhiltsov-max]

Hi, I wouldn't call it's an "easy" problem to solve, nor would I call it a bug or an error. It's implemented this way on purpose - so that after importing annotations you had only what you just uploaded. Though I agree, it can be improved by allowing to keep existing annotations. This is handled by the server, not UI. Thus, the nice solution would be:

• to add a switch in UI annotation import dialog
• to add a server API flag (e.g. a query parameter) for keeping old annotations on uploading (should be enabled by default)
• forward the flag from the API all the way up to this place
• add corresponding tests

This should be done both for tasks and jobs as all of them allow annotation uploading.

[EBayego]

Hi, thank you for your help, and sorry for taking so long to answer, I've been sick this past week. I'm having trouble understanding the complete structure of the project, could you tell me where the server API flags are located, or provide an example in the code of their usage? Thank you very much. El jue, 4 abr 2024 a las 17:53, Maxim Zhiltsov ***@***.***>) escribió:

…

Hi, I wouldn't call it's an "easy" problem to solve, nor would I call it a bug or an error. It's implemented this way on purpose - so that after importing annotations you had only what you just uploaded. Though I agree, it can be improved by allowing to keep existing annotations. Thus, the nice solution would be: - to add a switch in UI annotation import dialog - to add a server API flag for keeping old annotations on uploading (should be enabled by default) - forward the flag from the API all the way up to this place <https://github.com/cvat-ai/cvat/blob/ea47d00103f9df8bd31102ea2ae364e32fa20f04/cvat/apps/dataset_manager/task.py#L799> - add corresponding tests This should be done both for tasks and jobs as all of them allow annotation uploading. — Reply to this email directly, view it on GitHub <#4747 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOCQJZOMNR22PL4TW62ACV3Y3VZOBAVCNFSM53NVNYHKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMBTG42TQMRWHE4A> . You are receiving this because you were assigned.Message ID: ***@***.***>

[zhiltsov-max]

@EBayego, I hope you feel better now. For API please check cvat/apps/*/views.py, mostly engine.views. An example of a parameter is conv_mask_to_poly.

[EBayego]

@zhiltsov-max
Yes, I'm feeling better now, thank you!
If I'm not mistaken, the parameter is retrieved from a query using request.query_params.get(), but I can't find where it's created. Could you give me a quick guide of how the flow works? I'm sorry for needing this much help. Thank you!

[zhiltsov-max]

@EBayego, it's parsed by Django automatically. As you can see in the example above, you can add an extra API documentation so that it was shown in OpenAPI schema or swagger (check http://localhost:8080/api/swagger/ and http://localhost:8080/api/docs/).

[EBayego]

Hi, I'm facing I suppose the last problem. I'm trying to maintain the current annotations by creating new shapes (and probably tags and tasks too) with the same data, but letting django create new ids for them in order to avoid errors. But even with that, I'm getting the following error:

I'm only creating new ids for those who already exists, by checking the list that contains current shapes. models.LabeledShape.objects.values_list('id', flat=True). Any idea on how to deal with this?

Thank you!

[zhiltsov-max]

Hi,

I'm facing I suppose the last problem. I'm trying to maintain the current annotations by creating new shapes (and probably tags and tasks too) with the same data, but letting django create new ids for them in order to avoid errors.

It's certainly doable this way too, but why did you decide to it like this? Did you try to comment the annotation removal line?

About the error - are you sure it's about annotation ids? It's hard to say for sure without the stack trace, please try to check it via docker logs cvat_worker_import.

[EBayego]

It's certainly doable this way too, but why did you decide to it like this? Did you try to comment the annotation removal line?

Yes, you are right, that's the option I tried at first, but I encountered an error which I attempted to resolve in this way. However, I am no longer able to reproduce it in any way, so it was likely my mistake in the import file format or in managing labels or not saving annotations, i don't really know.

Finally, regarding the tests, I simply modified the lines where it calls import_task_annotations by adding a final True corresponding to the new flag I created. Should I create any more tests? Thank you very much for your help and time, I appreciate it!

[zhiltsov-max]

@EBayego, we're trying to move away from using the django-based tests. Please add a test in the tests/python/rest_api tests.

[EBayego]

Okay, in that case, I believe that this existing test, if runs successfully, covers the development I have done, as I have only included a parameter in the internal query, the same case as conv_mask_to_poly. I will prepare the pull request following your documentation. If there are any changes or suggestions, let me know!

[zhiltsov-max]

I believe that this existing test, if runs successfully, covers the development I have done, as I have only included a parameter in the internal query, the same case as conv_mask_to_poly.

I'm not sure I got what you mean here. In this PR there is a new API option (actually 2 - for tasks and for jobs) to allow keeping the old annotations during the import. In needs to be tested. There is no way it could be covered by any existing test, as it didn't exist before this PR.

[EBayego]

I assumed that this was a parameter with the same functionality as the one you gave me as an example, conv_mask_to_poly. In the Swagger documentation, this parameter does not appear:

And in the code, therefore, it also does not appear as an OpenApiParameter. However, it is taken from the query parameters within the method here. I assumed that either the Swagger documentation had not been updated, or that it was simply an internal parameter relevant only for specific queries, and the parameter was just a part of 'options', which is a list sent along with other options, in this case only conv_mask_to_poly from here.

If it needs to be added to the Swagger documentation, then I imagine the test I need to perform is similar to this one, but adding the part of passing more parameters to the query and not just the file format and the file to import.

Since I don't see direct examples to follow and save time, it will take me more time to figure out how to write the appropriate tests. And given that today, if I'm not mistaken, is the last day to have a PR ready, I will submit the PR to document what I've done so far, and I will continue with the development in the coming days.

[zhiltsov-max]

In the Swagger documentation, this parameter does not appear

Most of the OpenAPI parameters must be added manually with the decorator, except regular filtering parameters and several others. conv_mask_to_poly was supposed to be a temporary solution, and it can be considered deprecated, so we don't want to advertise it in the API.