I think they are using the LLM as few shot learner then using that to label the rest of the training data and finally using the now fully labeled data to train a more traditional supervised classifier like DistilBert.