Safety Reports
Aviation Safety Data Accessibility Study Index:
Safety Data
In the broadest sense, safety data include reports of events, such as accidents and incidents, inspection results, reports of enforcement actions or other sanctions, and other data which characterize the activities of the air transportation system. It must be noted that only accidents (and some incidents) involve measurable harm to persons or property, and that many types of incidents are reported to FAA by the carriers themselves.

It should be noted that there is no consensus among researchers and participants in the aviation industry about what exactly constitutes "safety data." This lack of consensus was strongly expressed in the comments to the initial draft of this report. Although accidents are universally regarded as events that should be avoided and eliminated if possible, there is little or no statistical evidence for U.S. domestic commercial aviation that other forms of "safety data"-incidents, surveillance results, or enforcement actions-serve as predictors of future accidents or are correlated with accident rates for individual carriers. While the remainder of this section discusses types of data that are commonly regarded as relevant to safety concerns and to the fulfillment of FAA's safety responsibilities, the exact nature of the relationships between these data and the safety of airline operations remains the subject of research in the aviation community.

Accident and Incident Data

Aircraft accidents and incidents are events that involve direct or potentially direct effects on the safety of aircraft operations and of persons involved in those operations. Accidents result in death or serious injury to a person in, upon, or about the aircraft, or in substantial damage to the aircraft itself. Incidents are less serious events "that affect or could affect the safety of operations." (FAA, 1996b) Because accidents and incidents, once reported and investigated, are believed to represent a relatively unambiguous record of unfavorable safety events, they are the safety measures most commonly used by researchers for analyzing changes in aviation safety over time and differences among carriers and groups of carriers. However, the raw data on accidents and incidents must be converted to accident and incident rates before it can legitimately be used for making comparisons about safety over time, among groups of carriers, or among individual carriers. This type of conversion, which controls for exposure to risk, is called normalization and is discussed below in the section on exposure data.

Some observers have suggested that the classification scheme for aviation accidents used by reporting agencies is needlessly arcane, and the Federal Aviation Authorization Act of 1996 directs the NTSB, in conjunction with FAA, to develop a more comprehensible and refined classification of accidents involving fatalities, injuries, or substantial damage. (Congress. House, 1996) NTSB has recently responded with a proposed classification format that addresses these concerns. (NTSB, 1996)

Inspection and Surveillance Data

Some have argued that it is possible to identify or compile "safety indicators" that provide insights as to whether a carrier is more or less likely to undertake unsafe practices. Researchers, including GAO (1988), have focused on four broad aspects of airline operations that are believed to be important to safe operations.

• Pilot competence
• Maintenance quality
• Financial stability
• Management attitude

GAO (1988) concluded that there were no comparable and objective measures of relationships between airline safety and these four areas. The role of safety indicators was also discussed in a recent report on Australian aviation safety by the Australian Parliament (Parliament of the Commonwealth of Australia, 1995), which noted that "accident statistics are of limited use and there are no…safety indicators that can be used as effective alternatives of the statistics. In short, there is a scarcity of measurements of safety." Recently, however, GAO (1996) recommended that FAA reexamine the feasibility of developing objective measures relating airline safety to carrier performance in these areas.

Some believe that information on factors that could affect airline safety practices can be found in inspection and surveillance reports on airline operations. If these data provide useful information about current or future carrier safety practices, then public reporting of these data could provide a positive incentive for the level of effort carriers put into safety, over and above the obvious self interest of the carriers. Public reporting of inspection and surveillance data could allow consumers to make their own comparisons of carriers based on how well or poorly they have done when inspected by FAA. The degree of compliance with FAA regulations might be an indicator of an airline's diligence in the safety arena. It must be noted, however, that there may be no relationship between inspection results and the probability that a carrier will have an accident in the future, especially if carrier operations improve as a result of FAA findings.

Data from inspection and surveillance reports are not currently available to the public, although results from these oversight activities are sometimes reported publicly. There is a very large number of inspection and surveillance reports filed by FAA inspectors. However, there has been comparatively little systematic analysis of these data, especially in terms of its relationship (if any) to accidents and accident rates. Some analysts have questioned the quality, reliability, and management of inspection and surveillance data within FAA (GAO 1992, 1995), and consideration of releasing detailed inspection and surveillance data to the public could be made contingent on improvements in FAA's ability to manage and utilize these data adequately for its internal needs. There are also concerns related to the belief that some surveillance reports represent subjective evaluations of a carrier's operations. Additional analyses of these data should be conducted before FAA makes a decision to release inspection and surveillance data on individual carriers to the public on a routine basis.

Top of Page
Exposure Data

Computation of an accident or incident rate requires normalizing information about the level of exposure to risk. For comparative purposes, it is essential that accident and incident data be normalized in some way, since the system's (or a carrier's) exposure to risk changes over time. One carrier's exposure to risk in a particular time period will likely differ from that of another, because different carriers have different levels and types of activity. Measures of exposure to risk commonly used to normalize event data include number of flights, hours flown, passenger enplanements, and passenger miles flown. Villareal (1988) discusses advantages and disadvantages of the various exposure measures used for normalizing safety research data. Most researchers prefer to use the number of flights (measured as departures) for normalizing data, rather than hours or miles flown, because the risk of accident for an aircraft is greatest during takeoff and landing. For consumers, the most relevant measure is also likely to be a flight or a round trip.

Although a commercial aircraft spends only about six percent of its flight time in the takeoff, initial climb, final approach, and landing components of its flight, around 70 percent of "hull loss" accidents have occurred during these stages. (Weener and Wheeler, 1992) Because of this, using an hours flown-based measure or a mileage-based measure of risk can be misleading. This is especially true when comparisons are being made between segments of the industry that have different average flight lengths. Using a mileage-based measure will make a commuter type carrier with very short average flight lengths look more risk prone relative to a major jet carrier flying longer stage lengths on average. (This occurs because a carrier with shorter average flights will make more takeoffs and landings per mile flown, and a carrier is most exposed to the risk of an accident or incident during takeoff and landing.) Prior research has shown the importance of comparing like groups of carriers (termed "peer groups") when comparing safety performance. (GRA 1988)

Accident and incident rates commonly reported to the public by FAA, the NTSB, and intermediaries such as the media and consumer groups thus combine event data-accident counts and incident counts-with exposure data to provide a measure of the frequency with which events have occurred. Thus, if in a hypothetical time period there were two commercial aviation accidents and one million commercial departures, the accident rate for this time period could be reported as 0.2 accidents per 100,000 departures. This accident rate could also (perhaps more informatively for the average person) be reported as an average of 500,000 departures per accident.

In many cases, the usefulness of reported safety data for the public might come from such small and simple changes in the style of reporting. For example, as part of its proposed reclassification system for airline accidents, the NTSB would begin reporting passenger fatality rates in a "passenger miles per fatality" format rather than the customary "fatalities per million passenger miles" format. (NTSB, 1996) In its discussion of various accident and fatality statistics, NTSB (1996) notes that "none of the statistics, taken alone, can be considered an accurate measure of airline safety and can be misleading."

The accident, incident, and exposure data described above are already available to the public, although it may take some degree of computer and statistical expertise to convert these data into useful information. Since any new FAA system for safety communication is likely to include accident and incident data, the more important question is what additional data should be included. As noted above, some discussion has been given to releasing surveillance and inspection data.

Making reasonable comparisons between carriers with this data also requires some form of normalization, such as a "percentage of satisfactory inspections" format. Because such data occurs on a carrier specific basis, surveillance and inspection data should be examined to see if there are no persistent statistical differences among individual carriers as normalized accident and incident data are. With these new data in mind, it might be useful to distinguish between "safety performance," which would include negative outcomes (like accidents and incidents) and positive outcomes (like safe uneventful flights), and "safety effort," which would include the sorts of items examined in a surveillance or inspection report. The logic of this distinction is that "safety effort" by carriers seeks to ensure that most or all "safety performance" outcomes are positive. As is discussed below, it is well established that carriers cannot be distinguished by "safety performance," but additional research is needed to determine whether this is also true for "safety effort," and whether differences in "safety effort" are informative about "safety performance."

Click here to return to Aviation Safety Data Accessibility Study Index