Automated screenings have been used for quite a few years now. I never liked the practice, but an LLM can't do any worse than a basic algorithm that attempts to scrub text out of a PDF or Word document and filter by keywords.
> an LLM can't do any worse than a basic algorithm that attempts to scrub text out of a PDF or Word document and filter by keywords.
Keyword filtering looks at what you explicitly tell it to look at. Magic LLM filtering looks at who knows what, especially if you use is as more than just a fancier entity extractor.
But both work poorly in my experience. We'd need to define metrics and a lot more data to really know which one performs worse.
Keyword filtering is only as good as those who decide the keywords to filter on and how well the algorithm can match keywords and variations of keywords. I've worked with plenty of tech recruiters over the years, they rarely understood anything about the jobs they recruited for. I never trusted the keywords and parameters they would set on any automated screenings.
The magic black box can have ~unlimited downside, if you give it a question that's too open-ended and for some reason it decides to key off of something you're not allowed to look at.
Dumb keyword filtering may be useless, but it shouldn't have that extra risk.
The unlimited downside is really driven by how people use the tool and what they do with the results rather than how the tool itself works.
If any hiring manager uses a tool poorly and trusts bad information that comes out the other end, that's on them not the tool.
That said, I very much am concerned with how quickly and blindly we're trying to develop and use AI. LLMs being used for narrow tasks like filtering results are at least limited a bit in risk, but what isn't limited is all the dangerous things we can do as we learn more from how these algorithms do those tasks well or poorly.
