Assessment of Assistant Physicians' Competencies for Making Accurate Diagnosis with Agreement Coefficients: A Methodological Study

Abstract

Objective: The objective of this study is to introduce some statistical methods, which are thought to be used in evaluating the ability of resident physicians to make the correct diagnosis, with a sample application. Material and Methods: The most common way to show the accuracy of a decision maker's decisions relative to the decisions to be referenced is to construct an error matrix. In the ideal case, all off-diagonal elements of the error matrix would have a value of zero, indicating that no decision of the decision maker was wrong with respect to the reference. The calculations of the methods in the study are shown on a sample error matrix. The data in the error matrix are randomly derived data. In the evaluation of the error matrix, the agreement coefficients suggested by Bishop et al., Hellde? and Short were used. Results: When the decisions of the decision maker (assistant physician) and the reference (specialist physician) decisions are similar, the variance of the error matrix decreases, the confidence intervals narrow, and the value of the coefficients of agreement both in the categorical and in general increases. Conclusion: Objective assessment techniques rather than subjective assessment are important for the standardization and quality of education. According to the findings, it was concluded that the agreement coefficients for proficiency in specialization education are usable and the quality of education will increase if they are used together with other evaluation methods. © 2023 by Türkiye Klinikleri.