LEXINGTON, Va. April 17, 2025 — Artificial intelligence (AI) is everywhere. From health care to finance, transportation, customer service, and manufacturing. Many people rely on AI without really realizing it for their smartphones and smart home devices. But is AI as reliable and accurate as humans in all cases? Jude Roberts ’25, a cadet at Virginia Military Institute, sought to find the answer in his honors presentation, “Assessing the Efficacy of an LLM for Resume Selection and Human Talent Management.”

Roberts, a computer science major from Cloverdale, Virginia, was introduced to AI research when he interned at the U.S. Army Cyber Command at Fort Eisenhower, Georgia, last summer. For his honors project, he conducted an experiment looking at business hiring processes using the AI program, ChatGPT, specifically the GPT-4o mini, a cost-efficient small model. He explained that people generally believe that time efficiency and lower costs are a huge upside to automation. “If a company doesn’t have to pay an employee to look through a lot of resumés, it frees their staff’s time to focus on more high-level strategic tasks. Sometimes ChatGPT and other large language models (LLMs)—software designed to generate human-like, conversational language—provides completely nonsensical, out of context, or just wrong information, known as hallucinations, which is problematic.” Roberts continued to point out that because there is no human interaction to evaluate a potential employee’s interpersonal and character traits, AI may show a bias based solely on particular words appearing on a resumé.  “Some of us very soon will be applying for our first jobs and we want a fair shot at the hiring process. What I’m examining here is whether AI is consistently capable of selecting the best applicants for jobs,” he explained.

Roberts took 1,000 resumés and instructed ChatGPT that it was taking on the role of a hiring manager, and it was to consider the applicants’ skills, experiences, and qualifications, and choose the applicant that best fills the job role. Categories included data science, arts, web design, and engineering. An additional category, not applicable (N/A), was designated for any resumé data that contained nonsensical or misleading information or that did not match any of the categories.

Roberts found that AI was less accurate and less reliable than a human hiring manager in finding the best candidate, and he determined that the LLM was vulnerable to malicious interference and bias propagation. “These are very concerning aspects for the reliability of these models, and administrators should be concerned about these issues when implementing this technology. Hiring managers should never solely rely on the technology to automate any portion of resumé screening.”

Sherif Abdelhamid, Ph.D., assistant professor in the Department of Computer and Information Sciences and outreach liaison officer for the Cyber Defense Laboratory, served as Roberts’ advisor on the project. Abdelhamid noted Roberts’ exceptional skills of problem solving, ability to work in a team, and analytical skills. “I see him as a unique, special student. He was able to independently identify the research problem and objectives, and came up with a unique design for his research. He carefully and critically studied how LLMs can impact our lives, and identified their benefits and drawbacks,” said Abdelhamid.

Roberts, who is minoring in applied mathematics and philosophy, is a graduate of Lord Botetourt High School in Daleville. He is the son of Kevin and Lori Roberts. Following graduation, he will commission into the U.S. Space Force where he hopes to do work in cybersecurity or electromagnetic warfare.  

Marianne Hause
Communications & Marketing
VIRGINIA MILITARY INSTITUTE