Authors: Kristin Scardamalia, Keisha L. Bentley-Edwards, and Kairys Grasty
Abstract: The federal definition of emotional disturbance (ED) has been heavily criticized as vaguely defined and poorly operationalized yet there has not been a formal analysis of the reliability of the ED criteria. This study examined the reliability of the federal criteria for a special education designation of ED. A total of 179 school psychologists reviewed a mock special education report, made an eligibility determination, and provided information about their eligibility decision. In all, 56 participants found the student met ED eligibility criteria using 16 different combinations of the five criteria. Krippendorff’s α, a statistic preferred for content analysis, was calculated as a measure of criterion reliability. Results indicated extremely poor reliability (α = 0.2011). These findings demonstrate the importance of redefining the existing criteria.
Key Findings
- Of the 179 school psychologists in the study, 56 found the candidate to reach the ED eligibility criteria.
- However, these participants reached this determination using a wide range (16 different combinations) of the criteria, which indicated extremely poor levels of reliability and consistency.
- In short, as this criteria is the foundation for decision making for special education eligibility, the existing criteria must be redefined to improve the consistency of its application.
