Information Systems Auditing of Psychiatric Diagnoses Using Context-Enriched QA-Based Language Models

The growing use of Artificial Intelligence (AI)-driven decision support systems (DSS) in healthcare requires effective auditing frameworks to ensure diagnostic accuracy and transparency. This study introduces a novel information system (IS) approach for psychiatric diagnosis auditing, transforming traditional multi-label classification into a question-answering (QA)-based framework enriched with diagnostic descriptions and explanations. Evaluating traditional classifiers, transformer models, Large Language Model (LLM), and the proposed Psychiatric QA model, results show that task transformation and context enrichment improve accuracy, with the QA model achieving the highest sum-F1 score and lowest precision fail rate. This study contributes to AI-driven IS, algorithmic auditing, and healthcare decision support, with managerial implications for electronic health record integration, insurance claim validation, and clinician workflow optimization.