Using Generative AI (Copilot, Gemini, or ChatGPT) to get indexing terms for reports

[GrumpyMonk099]

Select Topic Area

Question

Body

Part of my job is to find indexing terms for reports to tag in a data base management software. The challenging thing is that I need to pick relevant terms for the report from a list of around 21000 terms. I was thinking about ways to accomplish this using Copilot. I tried by uploading the report in pdf format and list of terms in excel sheet and asking the copilot to find relevant terms. However, it fails to do so, probably because of the how big the list is. When I pick a single world and ask if it is relevant for the report, it was able to provide answer to the same. Can anyone suggest a workaround for this, or at lease let me know if it is possible or not. Thank you in advance.

[Charlesnorris509]

Given the challenge of finding relevant indexing terms from a list of 21,000, here are a few suggestions:

Chunking the List: Instead of processing the entire list at once, consider breaking it down into smaller chunks. This might help Copilot manage the data more efficiently.
Pre-processing the Report: Extract key phrases or concepts from the report using natural language processing (NLP) techniques before matching them against your term list.
External Tools: Utilize specialized text analysis tools or libraries (such as spaCy, NLTK, or TF-IDF) to identify important terms in the report and then match them with your list.
Iterative Approach: If Copilot can handle single terms effectively, consider iteratively querying terms in smaller batches to build a relevant index
@GrumpyMonk099

[GrumpyMonk099]

@Charlesnorris509 Thank you for the help.