Exploring the Reliability of an Objective Severity Tool to Classify Severe Problem Behaviour

Abstract

The term ‘severe’ is a common descriptor for problem behaviour in research and practice. However, it is often applied inconsistently, and at times based on ill-defined or arbitrary criteria. Existing problem behaviour measurement tools often rely solely on caregiver recall (e.g., interviewing primary caregivers). This study explores the reliability of the first iteration of a severity tool employing direct measurement strategies (e.g., response rate, injury severity as evidenced by permanent product) to classify an individual’s problem behaviour severity. Nine Board Certified Behavior Analyst (BCBA) raters were recruited, five novice raters and four expert raters. They each experienced two conditions. In the first condition, raters classified the severity of 20 case scenarios without access to the tool. In the second condition, raters classified the severity of 20 novel scenarios after completing the tool for each case. All items of the tool (n=26) had good internal consistency (∝=.831). Intraclass correlations showed a meaningful increase in reliability for both groups when they had access to the tool (novice r=0.860, expert r=0.912) compared to when they did not have access to the tool to rate case severity (novice r=0.781, expert r=0.803). Most raters either strongly agreed or agreed that the severity tool had good applicability across research and clinical settings. This suggests that inconsistencies that may exist in the classification of severe problem behaviour could be mitigated with the proposed tool.