The California
Motorcyclist Safety Program
California Motorcyclis Safety Program Website

Program Effectiveness:
Accident Evaluation

EXECUTIVE SUMMARY

Written by:
John W. Billheimer

Prepared for the CHP by
Systan, Inc. Los Altos, California

August 22, 1996

The opinions, findings, and conclusions expressed in this publication are those of the authors and not necessarily those of the California Highway Patrol. This publication does not constitute a standard, specification, or regulation.



Table of Contents

1.1   Background
1.2 Evaluation Overview
1.3 Accident Trends
1.4 California vs. the Rest of the U.S.
1.5 Matched-Pair Analysis
1.6 Additional Issues
1.7 Evaluation Summary

Exhibits

1.1 Fatalities Per Thousand Registrations
1.2 Accidents Per Thousand Licensed Riders
1.3 Total Accidents for Under-18 Riders
1.4 Accident Per Hundred Thousand Miles Before and After Training Matched Pairs of Trained and Untrained Novice Riders






1.1 BACKGROUND

The California Motorcyclist Safety Program (CMSP) is a legislatively mandated, statewide Program that has trained over 95,000 motorcyclists in the nine years since its implementation in July 1987. The CMSP was formed by Assembly Bill (AB) 412, which charged the California Highway Patrol (CHP) with the responsibility for developing and implementing the Program. The CMSP is funded through a $2 per vehicle surcharge on motorcycle registration fees which currently generates approximately $1.3 million annually. Subsequent legislation (AB 3255, effective January 1, 1988) made CMSP training mandatory for all riders under 18 seeking a California motorcycle license. AB 55, which took effect on January 1,1991, raised the mandatory training age to include all riders under 21 years of age and mandated a formal evaluation of the impact of training on motorcycle accidents. AB 229, which took effect on January 1, 1994, allowed students successfully completing the CMSP course to waive the riding skills test required for licensing by the Department of Motor Vehicles. This report describes the evaluation activities undertaken to assess the program's impact on motorcycle accidents in California.




1.2 EVALUATION OVERVIEW

The ultimate measure of the effectiveness of any motorcycle training program on accident rates. The current evaluation takes a three-pronged approach to impact of CMSP training on motorcycle accidents in California:

(1) Motorcycle accident trends are traced before and after the format)' and the introduction of mandatory training laws;

(2) Motorcycle accident trends in California since the start of the Program are compared with trends in the remainder of the U.S.; and

(3) Matched samples of trained and untrained riders from the Los Angeles area are developed over a five year period, and the riding records of these pairs of riders are compared for three time periods: (1) Six months after training; (2) One year after training; and (3) Two years after training.




1.3 ACCIDENT TRENDS

Over the nine years the CMSP has been in operation, fatal motorcycle accidents in California have dropped by 69%, falling from 840 fatal accidents in 1986 to 263 fatal accidents in 1995. At the same time, total motorcycle accidents have fallen from 29,742 in 1986 to 9,710 in 1995, a drop of 67%.

1.3.1 Fatalities Per Registered Motorcycle

Exhibit 1.1 plots the number of registered motorcycles and the number of fatalities per registered motorcycle from 1960, the first year in which motorcycle accident statistics were reported separately, to 1995. As the graph shows, between the introduction of the CMSP in


1987 and the introduction of a mandatory helmet law in 1992, the fatality level dropped from 1.2 to 0.78 fatalities per registered motorcycle. At the time, this equaled the lowest level achieved since the state first began recording motorcycle fatalities separately. Since 1992, the level of fatalities per registered motorcycle has continued to drop. The current level is 37% below the lowest pre-CMSP level, recorded in 1975, and 59% below the level recorded in 1986, the year before the CMSP originated.

EXHIBIT 1.1

FATALITIES PER THOUSAND REGISTRATIONS

Exhibit 1.1

Editor's Note and Comments - What's not shown in this graph...


1.3.2 Accidents Per Licensed Rider

Exhibit 1.2 plots total motorcycle accidents per thousand licensed riders for the eighteen years between 1977 and 1995. For the total population of riders, the accident rates per thousand licensed riders ebb and flow with the total accident rate, generally rising before 1986 before beginning a continuous drop in 1987. For younger riders under the age of 25, however, the graph shows a remarkable increase in the ratio of accidents to licensed riders during the years just prior to the start of the CMSP. This ratio increased steadily between 1977 and 1986, reaching a peak of 146 accidents per thousand licensed riders in 1986. The ratio dropped slightly in 1987 and dropped still further in 1988, the first full year of CMSP operations. The ratio for younger riders stood at 72 accidents per thousand licensed riders in 1995, 50% below the peak achieved in the year prior to the opening of the CMSP. The ratio of total accidents to the entire population of licensed riders has dropped continuously since the formation of the CMSP, reaching a low of 12 accidents per 1000 riders in 1995, 70% below the pre-CMSP peak.

EXHIBIT 1.2

ACCIDENTS PER THOUSAND LICENSED RIDERS

Exhibit 1.2



1.3.3 Under-18 Accidents

The decline in accident rates has been even more dramatic for that portion of the population first required by law to take the basic course offered by California Motorcycle Training: Riders Under 18. The bar graph of Exhibit 1.3 traces historical accident rates among under-18 riders, and shows a significant drop since January 1988, when training became mandatory for riders in this age group. Since the implementation of AB 3255, accidents among under-18 riders have dropped by 88%, while accidents among riders 18 years of age and older have dropped 61%.

EXHIBIT 1.3

TOTAL ACCIDNETS FOR UNDER 18 RIDERS

Exhibit 1.3




1.4 CALIFORNIA VS. THE REST OF THE U.S.

1.4.1 Relative Accident Rates

Since the start of the CMSP, the decline in motorcycle accidents in California has outpaced the decline in the number of registered motorcycles and licensed riders, particularly among the under-25 age group. A number of factors besides the formation of the CMSP might explain the drop in accident rates. These include the aging of the baby-boom population and a general decline in motorcycle sales. In fact, motorcycle accidents have generally been dropping throughout the U.S. over the life of the CMSP. A comparison of motorcycle accident rates in California with those in the rest of the U.S. since the start of the CMSP shows that the decline of accidents and fatalities in California has far outstripped the decline in the remainder of the U.S. On the average, motorcycle fatalities per thousand registrations in California have dropped by 33% since the formation of the CMSP in 1987. Over the same period, the levels in the rest of the U.S. Only dropped 17%.


1.4.2 Projected Savings

Accidents and Fatalities Avoided. If post-1986 accident rates in California had paralleled those in the rest of the U.S. over the first eight years of CMSP operations, the State would have experienced 18,992 more motorcycle accidents (2,374 per year) and 942 more fatalities (117 per year). While there is no guarantee that the implementation of the CMSP is totally responsible for these imputed savings, the use of the rest of the U.S., as a comparison group helps to screen out some of those factors (i.e. demographic trends, changes in motorcycle riding habits) which tend to obscure the Program's impacts.

Dollar Value. Using conservative estimates of the costs of accidents and fatalities, a savings of 2,374 accidents and 117 fatalities per year represents an annual savings of $173 million, more than one hundred times the cost of the Program itself.




1.5 MATCHED-PAIR ANALYSIS

Both the before/after trend analysis and the comparison of California with the rest of the U.S. show significant reductions in motorcycle accidents since the start of the CMSP. The third element of the accident investigation is a matched-pair analysis designed to isolate the impact of CMSP training on specific riders and, in the words of AB 55, to measure "...accident rates of persons completing classes in contrast with persons not having completed classes."

1.5.1 Approach Overview

Sample Development. Under the matched-pair approach, a profile of the Los Angeles motorcycling population was obtained by sending on-site interviewers to places where motorcyclists congregate (dealerships, accessory shops, schools, malls, etc.). These profiles identified specific motorcyclists by such key factors as age, sex, years riding, miles ridden/year, and primary purpose of riding (commuting, recreation, etc.). Additional information was gathered (driver's license number, helmet use, etc.) but these five were the primary factors used to match the untrained riders with trained riders for comparison purposes. The five-year sampling process, which was initiated in late 1989, produced interviews with over 16,000 untrained motorcyclists, which in turn produced 2,351 matched pairings with trained riders: 1,139 with riders taking the basic 16-hour course, and 1,182 with riders taking the 8-hour experienced rider course (ERC).

Survey Follow-Up. As part of the overall evaluation of CMSP activities, regular telephone surveys were conducted with samples of trained and untrained riders, including many of those riders in the matched-pair analysis. Survey responses showed that recent CMSP trainees consistently reported higher usage of such protective equipment as helmets, boots, and jackets than the general population of untrained riders. Survey responses also suggested that although trained and untrained riders had been carefully matched with regard to riding habits prior to training, their post-training riding experience diverged in many cases. To explore these findings in more detail, mail-back surveys were sent to all riders included in the matched-pair analysis in June 1994. Responses from 37% of those samples showed that trained riders tended to ride more after training than their untrained counterparts. Basic course trainees reported riding an average of 5,500 miles during the first year after training, for example, as compared with an average of 4,300 miles reported by untrained riders over the same period.

There is no way of knowing whether the confidence instilled by training leads to added riding, or whether the decision to ride (or ride more often) leads riders to take a training course. In any event, since trained riders were clearly putting more miles on their motorcycles (and hence increasing their exposure to accidents) after training, it was necessary to estimate post-training mileage for all trained and untrained riders so that post-training accident data could be measured in accidents per mile. Wherever possible, survey responses were used as indicators of post training mileage. Where survey responses were unavailable, regression models were developed relating post-training mileage to prior riding experience.

1.5.2 Accident Comparisons

Novice Riders. Exhibit 1.4 plots the accident rates before and after training or

interviewing for trained and untrained riders with 500 miles or less of prior experience. Both groups experience extremely high accident rates during their initial months of riding prior to training. In the six months following training, the accident rate of trained novices drops significantly to 0.39 accidents per 100,000 miles, less than half of the rate of .85 accidents per 100,000 miles attributed to their untrained counterparts. As time elapses and both groups gain more experience, the accident rates of both trained and untrained riders converge. While the accident rates registered by untrained riders remain slightly higher than those of trained riders, the differences, which were significant at the six- month mark, were not found to be statistically significant at the one- and two-year mileposts.

EXHIBIT 1.4

ACCIDENTS PER HUNDRED THOUSAND MILES BEFORE AND AFTER TRAINING
MATCHED PAIRS OF TRAINED AND UNTRAINED NOVICE RIDERS

Exhibit 1.4



Thus the matched-pair analysis shows that basic CMSP training leads to lower accident rates among novice motorcyclists with little prior riding experience for a period of at least six months after training. This category of rider, the novice with little prior experience, is particularly important, since over half of all CMSP trainees arrive with no prior riding experience and statistics show that beginning riders are far more susceptible to accidents than more experienced motorcyclists.

Experienced Riders. Post-training differences in accident rates were not so pronounced for more experienced riders. No significant post-training differences were detected between trained and untrained riders with more than 500 miles of prior experience at any time after taking the basic course. Trained riders registered lower accident rates than their untrained counterparts six months after taking the ERC course, but these differences vanished over longer periods of time.

1.5.3 Violation Comparisons

Traffic violation rates for novice riders also drop lower than those of their untrained counterparts for the first six months after training. Whereas more experienced riders have lower accident rates than novice riders, they tend to have higher violation rates, suggesting that experience and training may lead to an understanding and acceptance of certain types of risks which are not perceived as potential accident threats. Other researchers have observed the same phenomenon. McKnight and Robinson (1990) noted that "...the inexperienced are somewhat intimidated by the highway traffic environment and therefore compliant with the law. Experience may lead to greater confidence which, in turn, can lead to different perceptions on the part of riders as to what speeds can be safely traveled, what gaps in traffic can be safely accepted, or what maneuvers can be safely executed."

Experienced riders electing to take the voluntary ERC had significantly lower violation rates than their untrained counterparts prior to training. Although these differences were not reflected in the accident rates of the two groups, they may confirm the speculation of several researchers that riders taking safety courses voluntarily are more inclined to safe practices than riders who elect not to take such courses. The violation rates of experienced riders rose after training, although they never exceeded those of the untrained group. At the same time, the accident rates of ERC graduates remained lower than those of their untrained counterparts, particularly during the six-month period following training.

1.5.4 Reconciliation with Past Studies

Past attempts to measure the effects of motorcycle training on accident and violation rates in other locations have shown decidedly mixed results. While one recent evaluation in British Columbia found that training lowered accident rates, earlier studies in New York, Pennsylvania, Wisconsin, and Ontario found training to have no impact on accident rates. One small-sample 1980 study found, counter intuitively, that trained riders had higher accident rates than untrained riders. While a few of these past evaluations have been plagued by such pitfalls as mismatched cohort groups, inadequate sample sizes, and potentially biasing sampling procedures, even the best of them have focused on groups of trained and untrained riders who had significant amounts of prior riding experience. The two studies dealing with the largest sample sizes, New York and Pennsylvania, drew their samples from riders applying for licenses. By the time these riders showed up at the DMV, however, they had typically logged several years of riding experience. As a result, these past studies included relatively few untrained riders with no prior riding experience. As the current study shows, however, it is precisely this crucial group of riders, true novices with minimal riding experience, which most benefits from training. As motorcyclists become more experienced, day-to-day riding will help untrained riders overcome the disadvantages of their lack of formal training. Since most past studies included few true novices in their untrained sample and focused on post-training periods of at least a year, it is not surprising that they failed to detect any differences in the post-training accident rates of trained and untrained riders.




1.6 ADDITIONAL ISSUES

1.6.1 Parallel Safety Campaigns

In addition to training motorcyclists, the CMSP has launched a number of public information campaigns aimed at improving motorcycle safety and reducing accidents within the state. These information campaigns reach a broader audience than the training population, or even the motorcycling population, and should be acknowledged in any review of the Program's impact on accident trends. Campaign targets have included:

(1 ) Motorist Awareness. The CMSP has worked with the California DMV to include motorcycle awareness materials in the driver's license tests and manual; produced and distributed bumper strips (see below) designed to remind motorists that someone's relatives can be found under the rider's leathers and gear; and posted statewide billboards fostering driver awareness of motorcyclists.

Bumper Sticker


(2) Unlicensed Riding. Unlicensed riding is a serious problem in California. The fatality rate for unlicensed riders is three times that of licensed riders, and for the past ten years, an average of 65% of all fatally injured motorcyclists have been operating without a valid motorcycle endorsement. To address this problem, the CMSP has prepared and distributed licensing information and participated in state and nationwide campaigns reminding motorcyclists that unlicensed riders can have their motorcycles impounded. In addition, AB 229 has encouraged both training and licensing by enabling basic course graduates to waive the DMV skills test required for licensing. In the wake of this activity, over the past two years the unlicensed fatality rate has dipped below 60% of all state motorcycle fatalities for the first time in the past decade.

(3) Impaired Riding. In an effort to reduce the incidence of impaired riding in California, the CMSP designed and produced a set of anti-DUI posters, information cards, and billboards featuring a downed motorcycle and the caption "One for the Road" which were widely distributed during the 1996 riding season.

The overall impact of the indicated public information campaigns on accident rates cannot be known. However, such campaigns should be an integral part of any comprehensive motorcycle safety program. By their nature, they reach a much larger audience than the riders who elect to take a training program. Whereas roughly 25% of the motorcyclists currently active in the state report that they have taken CMSP training, telephone surveys show that over half of California's general riding population recognize the CMSP bumper strip slogan "My (son, daughter, etc.) rides, please drive carefully."

1.6.2 The Program as an Effective Sieve

One often overlooked impact of formal motorcycle training is its effect in discouraging some individuals from becoming motorcyclists and keeping riders who should not be on the road from injuring themselves. Follow-up surveys indicate that 16% of those trainees who no longer rode a year after training said that the basic course was a major factor in convincing them not to ride. This represents 5% of all riders taking the basic course and includes a disproportionate number of those failing it. It follows that roughly 3,900 unpromising riders (5% of all basic course students) have been kept off the roads since the start of CMSP operations. If these riders had continued to ride and logged 5,000 miles per year (about average for basic course graduates) and experienced the average accident rate registered by graduates, they would have accounted for roughly 120 accidents per year. Conservatively estimating the cost of a motorcycle accident at $49,500, this represents a savings to society of $5.9 million per year. Thus it could be argued that the $1.3 million annual expenditure for the CMSP is more than justified solely on the basis of the unpromising riders it discourages.

1.6.3 Future Implications

In view of the fact that the primary measurable impacts of training are realized by rank novices with less than 500 miles of prior riding experience, future research efforts and future public information campaigns should target this accident-prone group of riders.

1.6.4 Limitations of Findings

This investigation has focused on a specific program, the California Motorcyclist Safety Program (CMSP), and the findings necessarily reflect and are limited to the administration, curriculum, and quality control procedures at the heart of this Program.




1.7 EVALUATION SUMMARY

Analyses of statewide accident trends show that total motorcycle accidents have dropped 67% since the introduction of the CMSP, with a drop of 88% among the under-18 riders for whom training has been mandatory since 1988. If accident trends in California had paralleled those in the rest of the U.S. over this period, the State would have experienced an additional 117 fatalities per year. A matched-pair analysis shows that the accident rates of untrained novice riders are more than double the rates of their trained counterparts for at least six months after training, when riding experience begins to have a leveling effect on the differences between the two groups. By any measure, the CMSP is a cost-effective Program that pays for itself many times over in saved lives and reduced accident rates.


California Motorcyclis Safety Program Website

| Home | Studies | States | Nation | World | Press | Archives | Backfire | Contact Us |