Full Title:AI-Assisted Rubric-Based Essay Grading in Dental Hygiene Education: A retrospective observational study

Authors: Courtney D. Routh, RDH, MSDH, Richard Halpin, MEd, EdD, EMBA, J. Nathaniel Holland, PhD, Tulsi Patel, RDH, MHA, FADHA

JDH Issue: October 2026

Program Track: Research

Abstract: Purpose Assessment of reflective and professionalism-focused writing is a critical component of health professions education; competencies such as ethical reasoning, communication, and self-awareness are foundational to professional identity formation and clinical readiness. The purpose of this study was to evaluate the consistency of a large language model (LLM) in rubric-based grading of reflective essays in dental hygiene education, compare agreement patterns and systematic differences between artificial intelligence (AI) and faculty grading, and examine its potential to support faculty-driven assessment processes. Methods: Deidentified, archived reflective essays from first-year dental hygiene student cohorts (2022–2024; N = 90) were regraded using a secure, institutionally managed AI program (ChatGPT, Model 4o). Essays were originally assessed by faculty using a standardized rubric and re-evaluated by the LLM using identical criteria. Faculty and AI scores were compared overall and by domain using paired statistical tests. Grading time was recorded, and test–retest reliability was evaluated across three independent scoring runs. Results: Domain-level analyses required complete rubric data across all five domains; because of incomplete rubric data, 77 submissions (N = 77) were included in domain-level analyses. Faculty assigned significantly higher overall scores than the AI system (p < 0.001), with an average difference of approximately 1.9 points on a 10-point scale. Significant differences were found in four of the five rubric domains. Artificial intelligence grading was rapid, averaging approximately 39 seconds per essay, and demonstrated strong internal consistency across repeated evaluations (ICC = 0.87). The AI scores showed greater dispersion and more conservative scoring patterns, whereas faculty scores demonstrated ceiling effects. Conclusion: Artificial intelligence-based grading demonstrated strong reproducibility and rapid evaluation, with systematic differences in score calibration compared with faculty assessment. The use of AI may complement faculty-driven assessment by supporting consistent, scalable grading practices in dental hygiene education, potentially enhancing grading reliability while reducing faculty workload.

Learning Objectives:

• Compare faculty and artificial intelligence scoring patterns across the five reflective-writing rubric domains.
• Analyze the benefits and limitations of using artificial intelligence to support rubric-based assessment in dental hygiene education.
• Evaluate strategies for integrating artificial intelligence into grading workflows while maintaining faculty oversight and professional judgment.

CE Credit Hours: 2.0

The American Dental Hygienists' Association is designated as an approved provider by the American Academy of Dental Hygiene, Inc. #AADHADH (Jan. 1, 2026–Dec. 31, 2026). Approval does not imply acceptance by a state or provincial Board of Dentistry. Licensee should maintain this document in the event of an audit.

The American Dental Hygienists’ Association is an approved provider of dental continuing education as recognized by the Dental Board of California, Provider #5238.