Investigating the effects of common analytical techniques on reaction time data

Abstract

The heavy right skew of reaction time data creates challenges for analyses. Common analytical techniques may require a set of assumptions that are not found in this type of data. Some of the effects are known while others are not. The current study uses Monte Carlo simulation to assess the effects of ignoring distributional assumptions, aggregation, transformation, and truncation on reaction time data. The effects of these current practices were compared to fitting a generalized linear model. Each analysis was simulated to obtain false alarm and hit rates. From these values, the discriminability and criterion values from signal detection theory were calculated. Parameter estimates were also obtained and compared to the theoretical values from the simulation to produce estimates of parameter bias and accuracy. While fitting a generalized linear model had the highest discriminability and unbiased criterion, it was not very different from ignoring distributional assumptions and aggregating the data. Transforming the data using a log transformation resulted in biased and inaccurate parameter estimates and had the lowest discriminability. Truncating the data inflated the error and resulted in poor signal detection and poor parameter estimation.