


Review of the 1990 CDC Study on Head Injury Deaths and Helmet Laws
Reviewed by What the Study Does In "Head Injury  Associated Deaths From Motorcycle Crashes: Relationship to HelmetUse Laws" by D. M. Sosin, et. al, (JAMA, Nov. 14,1990  Vol. 264, No. 18), the authors estimate the rate of motorcyclerelated deaths associated with head injury for states with comprehensive (full) helmetuse laws, states with partial (youth) helmetuse laws, and states with no laws. Two different rates are calculated for each state for the years between 1979 1986: a populationbased rate (motorcycle deaths associated with head injuries divided by state population) and a registration based rate (deaths associated with head injuries divided by the number of state motorcycle registrations). The end result is an average populationbased rate and an average registrationbased rate of motorcycle deaths associated with head injuries for each of the three helmet law groupings of states during the entire 19791986 period. The authors find that the populationbased rate in states with partial laws (10.2 fatalities per million residents) and no helmetuse laws (11.4) were almost twice those in states with comprehensive helmetuse laws (5.5). Their registrationbased rates show dramatically less contrast between the three groupings: partial law (3.7 per 10,000 registered motorcycles), no law (3.5), and comprehensive law (3.0). On the basis of comparisons of these rates, primarily the populationbased rates, across groups of states with different helmet law coverage, the authors conclude that comprehensive helmetuse laws are an extremely effective mechanism for reducing the severity of nonfatal head injuries and the rate of fatal injuries. Their policy prescription is the adoption of comprehensive motorcycle helmetuse legislation. The contribution of this study is the creation and use of an alternative data set on motorcycle fatalities. In particular, the authors are able to separate out motorcyclerelated head injury deaths from all motorcyclerelated deaths. Thus, isolating a fatality variable which can more directly be used to test propositions on motorcycle helmet effectiveness. Unfortunately, the statistical methodology employed  comparing rates of headinjury related deaths for states with different helmetuse law coverage in order to infer the effectiveness of such legislation  is fundamentally flawed. In summary, the authors should be praised for their data collection techniques/innovations, but criticized for their inappropriate statistical methods and thus invalid conclusions. The Major Weakness of the Study The main conclusion of the studyhelmetuse laws and thus helmets are extremely effective for reducing the fatality rate associated with head injuriesis derived from a statistical technique known as correlation analysis. Using this approach, the authors simply observe that states with comprehensive laws have lower fatality rates than states without comprehensive laws and thus conclude that the existence of comprehensive helmet laws are the sole cause of these differences in fatality rates. But as all beginning statistics students know, correlation does not imply causality. The method employed in the CDC study fails to control for differences (across comprehensive and noncomprehensive helmet law states) in other factors  speed, alcohol consumption, and registrations/motorcycle usage  that can dramatically impact the populationbased fatality rates used to arrive at the CDC study's conclusion. Thus the higher fatality rates in the CDC study observed in noncomprehensive law states could be explained by higher average driving speeds, more alcohol consumption, and more motorcycle usage per person, rather than by the lack of a comprehensive helmetuse law. The data in Table I clearly establishes that states without comprehensive helmet laws have these critical characteristics over the same period, 197986, used in the CDC study. Thus the CDC study fails to determine which of higher speeds, more alcohol consumption, more use of motorcycles per person, and the lack of a comprehensive helmet law are responsible for the higher fatality rates. By not controlling for the impact of these other factors, the CDC study erroneous assigns their impact to the one factor that is considered in the analysis  nonexistence of a comprehensive helmetuse law. Given that these other factors cause higher fatality rates, the CDC study clearly and dramatically overstates (distorts) the effectiveness of helmet use laws by assuming that higher fatality rates in noncomprehensive law states are determined by the absence of a law rather than by higher speed, alcohol and usage. The level of distortion could be large enough that if it is corrected, the end result would show that helmetuse laws do not have a statistically significant effect on headinjury related fatalities. Other studies that employ the correct statistical methods (Goldstein (1985,1986)) by controlling for the impact of speed, alcohol, etc. find that helmets and helmetuse laws have no statistically significant effect on the probability of fatality or fatality rates. Referring to Table I, the heart of the problem in the CDC study can be addressed. The number of motorcycle registrations per 1000 population is dramatically higher in states without a comprehensive lawin these states, people are more likely to own/register/ride a motorcycle. Thus, motorcycle usage per capita is higher as will be fatality rates. In particular, in states with comprehensive laws, there are 18.7 registrations per 1000 population during 197986, in states with no laws there are 35.9, in states with partial laws 33.9 and in states without a comprehensive law (states with no laws or partial laws) there are 34.6. This implies that part, if not all, of the differences in the populationbased rates of the CDC study could be explained by the existence of more motorcycles per capita rather than by the nonexistence of a helmet law.
Differences in Other Key Determinants of Fatality Rate,
**Pure Alcohol consumption per adult person in wine gallons complied from reports of the Wine Institute and Distilled Spirits Council of the U.S. +Source: "Quarterly Speed Summary," Federal Highway Administration, U.S. DOT.
Another disturbing aspect associated with the registration based rates is that the CDC study ignores the fact that partial law states have a higher fatality rate than states with no laws. Using the same flawed logic contained in the CDC study, one could conclude from this result that helmet laws do not work because they result in higher fatality rates. Yet the study just ignores this finding. Of course, the contradictory result supports more the notion that the CDC study has produced distorted estimates of helmet law effectiveness than it supports the notion that helmets cause fatalities. We have just seen that when the CDC study controls for registrations, the differences between comprehensive and noncomprehensive law states virtually disappear (in a statistical sense they may totally be eliminated). If the CDC study controlled not only for registration/usage differences across states but also for the speed and alcohol differences exhibited in Table I, the differences in fatality rates would be reduced further. The higher average driving speed and alcohol consumption found in the noncomprehensive law states certainly explains some, if not all, of the remaining differences in fatality rates between states with and without comprehensive laws., Other studies (Goldstein (1985, 1986)) have shown that speed and alcohol are the major determinants of deaths in motorcycle accidents. In addition, it is also a wellknown fact that excessive speed and alcohol consumption are a primary cause of accidents. Thus, once all relevant factors are controlled for, it is highly likely that no statistically significant impact of helmet laws on fatality rates will exist. In order to control for all relevant factors and thus sort out the unbiased (undistorted) effect of helmet laws on fatality rates requires the application of multiple regression analysis. The CDC study does not employ this statistical methodology. Studies that have used this technique (Goldstein (1985,1986)) generate an undistorted estimate of helmetuse law effectiveness which shows that these laws have no statistically significant effect. In conclusion, the CDC study dramatically overstates the effectiveness of helmetuse laws on fatality rates related to head injuries because it fails to control for all of the relevant factors that affect fatality rates (motorcycle usage, speed, alcohol). When all factors are controlled for, the differences between comprehensive helmetuse law states' fatality rates and all other states' fatality rates disappear implying that helmetuse laws are ineffective in reducing fatality rates. Other (More Technical) Problems in the Study The CDC study claims that it is not necessary to address sampling error and thus statistical tests of significance because the entire population of death certificates of motorcyclerelated deaths is analyzed. First, the entire population of motorcycle related deaths associated with head injury, the relevant variable in the study, is not analyzed. In the data, 23% of all motor vehicle deaths were not classified by the type of vehicle. This implies that 23% of motorcycle deaths were not classified. Given that in the classified deaths, motorcycle deaths associated with head injuries accounted for 53% of motorcycle deaths, this implies that 12% of motorcycle deaths associated with head injuries are not considered. This further implies that a sample is being used head injuryassociated deaths is a random variable. State fatality rates are subject to a sampling error. In addition, there exists a random measurement error  deaths are recorded by residence instead of location  implying that state fatality rates will be measured with an additional sampling error. Finally, the appropriate sample size when considering state fatality rates is the number of states, not the number or percent of the population of head injury related deaths. The existence of sampling error implies that statistical tests of significance are appropriate. Given that the appropriate sample size is the number of states in each group (a small sample), it is highly unlikely that the differences in registrationbased fatality rates reported in the CDC study are statistically significant implying that helmetuse laws have not be shown to be effective. The analysis in the CDC study that compares the before and after fatality rates in the 3 states that changed their laws between 19791986 is invalid. These comparisons are based on a sample size of 2 and 1 and cannot be used to reach any statistically valid conclusion. The data on fatalities in the CDC study includes deaths from motorized scooters, tricycles and mopeds. These types of fatalities are not representative of motorcycle deaths and should be excluded. They could distort the results. Of course, no data set is perfect and this limitation of the data is not a serious problem. Finally, many little facts in the CDC study point to the major problem in the study: (1) the higher registrationbased fatality rate for partial law states; (2) the significant drop in the differences between fatality rates when registrationbased rates are employed and (3) the decline in all fatality rates between 19791986. (1), (2), and (3) all suggest that important factors that influence fatality rates have not been controlled for. (2) points to the importance of motorcycle usage and (3) points to the importance of stricter speed enforcement and stricter OUI laws, factors not considered in the CDC's calculation of the causes of fatality rate differences. Conclusion In conclusion, the CDC Study attributes differences in populationbased fatality rates (associated with head injury) across states with different helmetuse law coverage solely to differences in a state's helmetuse law. The study concludes that lower fatality rates are the direct result of comprehensive helmet use laws. The study fails to control for important differences in registration/usage, average driving speed, and alcohol consumption in determining fatality rates. In particular, states with comprehensive laws have lower registration/usage rates, lower driving speeds, and lower alcohol consumption all of which lower the fatality rate. Failure to control for these factors leads to distorted estimates of helmet law effectiveness which systematically and dramatically overstate their effectiveness. Once these factors are appropriately controlled for, no statistically significant differences in fatality rates across states with different helmet law coverage can be found. Footnote ^{1}While the speed and alcohol figures in Table I are respectively for all vehicles and for the entire adult population and the differences between groups of states seem small, these differences are relevant Higher average speeds (on 55 mph highways) are associated with less strict enforcement of speed limits on all highways Given the higher relatives rates of acceleration of motorcycles when compared to other vehicles, a small speed difference for all vehicles can translate into higher speed differentials for motorcycles In addition, the alcohol differences and could be magnified in the motorcycling community References Goldstein, J. 1986. "The Effect of Motorcycle Helmet Use on the Probability of Fatality and the Severity of Head and Neck Injuries," Evaluation Review, 10 3, June. Goldstein, J. 1985. "The Effect of Motorcycle Helmet Use on Motorcycle Fatalities An Econometric Approach," Discussion Paper #85115, Public Affairs Research Center, Bowdoin College.  Goldstein Study  Other Reviews by Goldstein  © Copyright Jonathan P. Goldstein Ph.D. 1986. All Rights Reserved. 
