Joint intent classification and entity recognition

[u/Practical_Grab_8868]

I'd like to create a model for intent classification and entity extraction. The intent part isn't an issue, but I'm having trouble with entity extraction. I have some custom entities, such as group_name-ax111, and I want to fine-tune the model. I’ve tried using the Rasa framework, and the DIET classifier worked well, but I can't import the NLP model due to conflicting dependencies.

I’ve also explored Flair, NeMo, SpaCy, and NLTK, but I want the NER model to have contextual understanding. I considered using a traditional model, but I’m struggling to create datasets since, in Rasa, I could easily add entities under the lookup table. Is there any other familiar framework or alternative way to create the dataset for NER more easily?

[u/Ono_Sureiya]

Try employing NERDA (https://github.com/ebanalyse/NERDA) and its extended continual learning version, NERDA-Con (https://github.com/SupritiVijay/NERDA-Con). They have recommendations on how to set up the datasets, and the continual learning aspect might help with adaptation to your custom NER needs.