Review of Helmet and Helmet Law Effectiveness Studies
By Jonathan Goldstein
I. Helmet Effectiveness Studies based on Variants of Correlation Analysis Which Support Motorcycle Helmet Effectiveness
There are two types of studies in this category: (1) before-after helmet law repeal studies, and (2) accident victim studies. The former tests the difference between death rates, injury rates' location rates of injuries, and severity rates of particular types of injuries (for all accidents occurring in a particular geographic region) for a similar period of time before and after helmet law repeal (Dare et al., 1979; McSwain and Lummis, 1980). The latter test the difference between these same rates for helmeted and non-helmeted accident victims (for a sample of all accidents during a single time period) in a geographic locale that does not have a mandatory helmet use law (Chang, 1981: Dare et al., 1979; Heilman et al., 1982; Hurt et al. F 1981; Kraus et al., 1975; Luna et al., 1981; Scott 1983). In each case statistically significant differences are attributed to helmet use or non-use. Typical results associated with this literature are death and injury rates two to three times greater for non- helmeted accident victims and increases in occurrence rates (death and injury) in repeal years that range from 19% to 63%. Typical conclusions are that the non-use of a helmet are responsible for these differences: helmet use is effective in the reduction of death and injury.
The major limitation of these studies is the use of some variant of correlation analysis as a statistical methodology. In general' correlation analysis cannot be used to establish a causal mechanism between any two variables (i.e. helmets and injury). This limitation of correlation analysis is universally recognized and is not disputed. The fatal flaw in correlation analysis is its inability to control for all relevant factors that could be responsible for movements or changes in a targeted (dependent variable) variable such as death or injury. Failure to control for all such factors implies that a third (uncontrolled for) factor could be responsible for the movements in one or both of the correlated variables. Thus' the effects of a third factor could erroneously be assigned to one of the two correlated variables implying that the "causal" effect of that variable is either over or understated.
In accident studies, the uncontrolled factor is the difference in risk aversion between helmeted and non- helmeted riders. non-helmeted riders are inherently more risky implying that on average they are more likely to have higher speeds, crash speeds and alcohol consumption (see Goldstein 1985) for statistical evidence). Given that speed and alcohol are major determinants of death and injury in motorcycle accidents, the failure of correlation analysis to control for these differences implies that the death and injury inducing effect of higher speed and alcohol levels will be inappropriately assigned to the non- use of a helmet. Thus, helmet effectiveness is systematically overstated by these types of studies.
The before-after designs fail to control for time as a third factor. Dramatic time trends towards (1) lower median age and experience of motorcycle owners , ( 2) higher average annual miles traveled, (3) higher displacement machines, and (4) less strict enforcement of federally mandated speed limits are not controlled for. Given that most before-after studies are done during a period when these injury and death inducing trends are dramatically increasing, these studies erroneously assign these time related increases in death and injury to the repeal of helmet laws which occur concurrently with these other trends. Thus, before-after studies systematically overstate the effectiveness of mandatory helmet use laws.
The most commonly used statistical methodology that can potentially avoid the methodological flaws inherent in correlation analysis is regression analysis. Unfortunately regression analysis is not a panacea for the problems of correlation analysis. Regression equations must be carefully specified in order to avoid specification bias which could lead to the systematic over or understatement of helmet effectiveness. Regression studies are analyzed below in Section III.
In the reviews that follow references will be made to the concepts developed in this introduction.
1. Hurt, H. H., Jr., Ouellet, J. V., and Thom, D. R. (1981, January) Motorcycle Accident Cause Factors and Identification of Countermeasures, Volume I: Technical Report, Volume II: Appendix/ Supplemental Data, Contract DOT HS-5-01160. NHTSA, Washington, DC.
The "Hurt Study", a two-volume' 829 page report for the National Highway Traffic Safety Administration (NHTSA) based on a $501,814 grant, is the most widely referenced study on motorcycle accidents and motorcycle safety issues. This accident victim study is based on data from the on-scene, in-depth investigation of 900 motorcycle accidents in the Los Angeles area for which 1,045 pieces of information (variables) where collected for each accident. The most important contribution of this study is its data collection. This study provides the best and most comprehensive set of data on motorcycle accidents available in the world. The quality and detail of the data, compared to typical accident reports, was enhanced through the use of an on-scene multi-disciplinary research team that collected more complete and accurate accident reconstruction, engineering, medical, environmental, protection system, and operator and passenger information.
The main weakness of the study is its statistical methodology. Comparisons of statistical frequencies for helmeted and non helmeted riders and cross tabulations, both forms of correlation analysis (criticized in above introduction), are the statistical techniques used to establish "causal" relations. The Hurt study's findings on protective equipment are that "the only significant protective equipment is the qualified safety helmet, and it is capable of a spectacular reduction of head injury frequency and severity." In particular, helmeted riders (1) show significantly lower injury frequency in all types of lesions -- helmeted riders comprised 39.4% of the sample and experienced 22.8% of all head and neck injuries; (2) show significantly lower injury frequency at all levels of injury severity; (3) are more likely to have no head or neck injuries; and (4) are less likely to experience severe F critical and clearly fatal head and neck injuries. While the Hurt study recognizes the theoretical possibility for helmets to cause neck injuries (p. 292), the findings indicate that "there is no world- shaking advantage or disadvantage of motorcycle helmet use in relation to neck injury."
The study also recognizes (p. 281) that extreme accident conditions present a formidable problem for head protection by helmets. Thus, practical limitations exist. Other main findings of the study are that the main causes of accidents and injury are: (1) lack of caution and awareness of involved automobile drivers; (2) lack of motorcycle conspicuity; (3) lack of operator skill; and (4) crash speed. The best countermeasures proposed are: (1) motorcycle training; (2) better licensing procedures; (3) use of safety helmets; (4) use of eye protection; (5) use of headlamps during daytime; and (5) use of bright upper torso garments.
The statistical methodology employed in this study systematically overstates the effectiveness of helmets (see introduction). Thus, this study does not provide any scientifically valid evidence in favor of motorcycle helmet effectiveness. An alternative study (Goldstein 1986, 1988) that uses the Hurt data and controls for the uncontrolled factors in the Hurt study finds dramatically different results concerning helmet effectiveness.
2. McSwain, N.E., and Lummis, M. (1980) "Impact of Repeal of Motorcycle Helmet Law." Surgery, Gynecology, and Obstetrics 151: 215-24. This study was supported by the Federal Department of Transportation: Contract No. DOT-HS- 7-01563.
This paper conducts both a before-after study for the State of Kansas using data from 1975 and 1976 and an accident victim study Limited to three primary population centers in the state. The study finds: (1) a 19.4% increase in the crude accident rate (accidents/ 1,000 registrations) from 12.8 to 15.3; (2) a 63.3% increase in the crude fatality rate (fatalities per 1,000 accidents) from 15.0 to 24.5; (3) a 95% increase in fatalities per 100,000 registrations from 19.2 to 37.5; (4) a 333% greater fatality per 1,000 occupants for non helmeted riders; and (5) a 51% increase in head injuries per 1,000 accidents after helmet law repeal. The accident victim study found: (1) a 25% greater injuries per 1,000 accident victims; (2) a 35% higher general body injury rate; (3) a 106% greater head injury rate; and (4) a 67% increase in head injury severity for non helmeted riders. The study also finds that fewer neck injuries are associated with helmet use. The study concludes that significant differences in death and injury rates based on helmet use imply that mandatory helmet legislation significantly reduces death and disability.
The statistical methodology is based on the comparison of before-after and helmeted-non helmeted rates, a form of correlation analysis. The accident victim study is subject to the above discussed criticism. In addition, the inability to explain why non helmeted riders have 35% more general body injuries highlights that the more risky behavior of this group of riders has not been controlled for by the simple correlation analysis. The before-after comparisons fail to control for the time trends discussed above and the dramatic change in economic conditions from the 1975 recession to the 1976 recovery which affect riding patterns. In particular, the inability to explain a 19.5% increase in the accident rate highlights these limitations.
3. Dare, C. E., Owens, J.C., and Krane, S. (1979) "Effect of Motorcycle Safety Helmet Use on Injury Location and Severity: Before-and-After Helmet Law Repeal in Colorado".
This paper conducts both a before-after helmet law repeal study (1976-1977) and an accident victim study to determine the effect of helmet usage on the location, number , and severity of motorcycle injuries. The before- after study finds: (1) a 48% increase in total accidents; and (2) a 140% increase in the number of fatalities. The accident victim study finds: (1) an increase in the number of injuries at all levels of injury severities except the most minor types of injury; (2) a fatality rate 2.2 times greater; (3) a critical injury rate 3.3 times greater; and (4) a head injury rate that is 2.6 times greater for non helmeted riders. In addition the study shows that the rate of the most severe neck injury decreased slightly for non helmeted riders but, non helmeted riders had a neck injury rate (for all neck injuries) 1.3 times greater than helmeted riders. It is also found that non helmeted riders are over represented in accidents.
The statistical methodology employed is the same as other studies in this category. Failure to explain the 48% increase in total accidents suggests that all relevant trends in the post-repeal period have not been controlled for and failure to explain the over representation of non helmeted riders in accidents implies that the inherently more risky behavior of this group of riders has not been considered. Thus overstated estimates of helmet effectiveness result.
4. Kraus, J. F., Riggins, R. S., and Franti, C. E. (1975). "Some Epidemiologic Features of Motorcycle Collision Injuries." American Journal of Epidemiology, 102:99-109.
This accident victim study (Sacramento County, California) establishes the major factors associated with motorcycle injuries and the severity of such injuries. Findings include: (1) the severity of injury is positively related to age; (2) the frequency of serious head injuries was highest for non helmeted drivers -- 14.1% of helmeted riders compared to 22.8% of non helmeted riders had serious head injuries; and (3) the severity of injury is positively related to crash speed -- serious injuries increased when crash speeds exceeded 48/km/hr. This study is good for isolating potential factors (age, speed, helmet use) which should be included in an (regression) analysis that controls for all factors associated with motorcycle injuries. The study, because of its correlation methodology , is not capable of isolating the individual effect of helmets in reducing injuries. Thus, the author's conclusion that mandatory helmet laws are effective for reducing fatalities and injuries is not scientifically supported.
5. Heilman, D. R., Weisbuch, J. B., Blair, R. W., and Graf, L. L. (19821. "Motorcycle-Related Trauma and Helmet Usage in North Dakota." Annuals of Emergency Medicine, 11:659-664.
This accident victim study (North Dakota) uses comparative frequencies and cross tabulations to find that helmetless riders: (1) suffer head, neck and face injuries at a rate 2.3 greater; (2) sustain 3.19 times more fatalities and 2.83 more life threatening injuries; and (3) have more injuries and more severe injuries than helmeted riders. Quoting other studies from Maryland and Colorado, the study addresses the costs to society: the average medical costs without doctor's fees of a motorcycle accident is estimated to be $11,038 and 53.7% of those costs are paid by the taxpayer. The study also recognizes that non helmeted riders are over represented in accidents and that 50% of crash victims didn't have a valid license to operate a motorcycle. The Hurt study has similar findings on over representation and the unlicensed driver Problem. Policy recommendations include mandatory helmet use and stricter licensing requirements and enforcement. On the basis of its flawed statistical methodology, this study inappropriately concludes that helmets are effective and inappropriately infers that the costs to society could be reduced by the imposition of a helmet law.
6. Luna, G. K., Copass, M. K., Oreskovich, M. R., and Carrico, C. J. (1981). "The Role of Helmets in Reducing Head Injuries from Motorcycle Accidents: A Political or Medical Issue?" The Western Journal of Medicine, 135:89-92.
This accident victim study (Seattle) finds: (1) the death rate for helmeted and non helmeted was not significantly different; (2) helmeted and unhelmeted fatal accident victims had statistically identical total severity scores; (3) the chance of sustaining a severe head injury was significantly higher for non helmeted riders -- 82% of such injuries were sustained by unhelmeted riders; and (4) the incidence of long-term neurological damage from severe head injury was three times greater for non helmeted riders. While this study supports typical findings on head injury severity for non helmeted riders, it does contradict the findings of earlier studies on a three times -greater incidence of sustaining a fatal injury for non helmeted riders. The statistical methodology of this study -- frequency comparisons -is subject to the same criticisms discussed above.
© Copyright Jonathan P. Goldstein Ph.D. 1986. All Rights Reserved.