Questionable Statistics
SUVs and Statistics

The New Yorker is legendary for the rigor of their fact-checking department. They leave nothing to chance:

A “Talk of the Town” piece once said that the musician Art Garfunkel had gesticulated nervously with his hands during an interview. Garfunkel later recalled getting a call from a New Yorker fact-checker asking if he still had both arms.

That’s obsessive.

So a recent Malcolm Gladwell article — entitled “Big and Bad: How the S.U.V. ran over automotive safety” — in the magazine is all the more puzzling. Gladwell has in the past shown himself to be a remarkably level-headed person; I’m sure that, like all of us, he has his biases, but in his writing he’s been very good about supporting his assertions with evidence. It appears, though, that his dislike of SUVs is greater than his dedication to honesty.

I have rather a lot to say about this, with charts and graphs and all, to the tune of about 30 KB.

(The New Yorker did not publish this article on their website. You just might find an 800K PDF version here, though.)

Gladwell’s main thesis appears to be: The best strategy for automotive safety is to rely on active safety, e.g. the superior maneuverability of a sports car. Reliance on passive safety, of the kind you get by surrounding yourself with thousands of pounds of SUV, is at best illusory. He implies that this is a particularly American delusion.

My guess is the Gladwell is correct, in a vacuum. His article does not prove this, though, and in fact at a number of points he contradicts his own conclusions.

Gladwell bases his argument on several points: his experiences at the Consumer’s Union testing ground, an analysis of vehicular death statistics, and the comments of a French semiotician (really). The biggest problem, and what I spend the most time on here, is with the statistics, but the rest of the article is pretty baffling, too.

The statistics appear to show (but don’t) that the larger a vehicle is, the more dangerous it is; Gladwell’s Consumer’s Union experience shows that SUVs don’t handle very well; and the French semiotician hints that SUV drivers might feel sexually inadequate.

At Consumer’s Union, Gladwell drove two vehicles: a Chevy Trailblazer and a Porsche Boxter. At the outset, it must be observed that he is comparing a mid-range SUV built to price with one of the world’s most capable (and expensive) sports cars.

It’s hardly surprising that the Boxster did much better in the handling tests than the Trailblazer. The Boxster was engineered from the ground up to be maneuverable; it seats two; and prices start at $42,600. The Trailblazer was engineered to be inexpensive and useful; it seats seven and starts at $27,420.

These are very different vehicles, and it doesn’t really make sense to compare them. No one buys a car based solely on safety; fuel economy, utility, and price are important, too. Gladwell is really only discussing safety here:

Most of us think that S.U.V.s are much safer than sports cars. If you asked the young parents of America whether they would rather strap thatir infant child in the back seat of the TrailBlazer or in the passenger seat of the Boxster, they would choose the TrailBlazer.

He ignores that other factors — like the ability to haul more than one adult and that child — might be valuable, but as far as safety goes, he’s right. Provided that you’re paying attention, which you should be doing anyway, you are more able to avoid collisions, and thus potentially safer, driving a Boxster — or any sports car — than you are driving a less-maneuverable car like an SUV.

So far, so good. Gladwell’s argument is weakened somewhat by the fact that he’s comparing extremes — a Taurus and an SUV would have been more convincing — but there aren’t any real problems yet.

Statistics

However, Gladwell then goes on to quote statistics. These appear to come from the study described here, and they appear to show that SUVs are, in fact, much more dangerous than other cars.

These statistics are so flawed, though, it’s hard to know quite where to begin. Gladwell’s article contains a large table of a number of the statistics from the study:

Make/Model	Type	Driver deaths	Other deaths	Total deaths
Toyota Avalon	large	40	20	60
Chrysler Town & Country	minivan	31	36	67
Volkswagen Jetta	subcompact	47	23	70
Toyota Camry	mid-size	41	29	70
Ford Windstar	minivan	37	35	72
Nissan Maxima	mid-size	53	26	79
Honda Accord	mid-size	53	27	80
Chevrolet Venture	minivan	51	34	85
Buick Century	minivan	70	23	93
Subaru Legacy/Outback	compact	74	24	98
Mazda 626	compact	70	29	99
Chevrolet Suburban	SUV	46	59	105
Jeep Grand Cherokee	SUV	61	44	105
Honda Civic	subcompact	84	25	109
Toyota Corolla	subcompact	81	29	110
Ford Expedition	SUV	55	57	112
GMC Jimmy	SUV	76	39	115
Ford Taurus	mid-size	78	39	117
Nissan Altima	compact	72	49	121
Mercury Marquis	large	80	43	123
Toyota 4Runner	SUV	94	43	137
Nissan Sentra	subcompact	95	43	138
Chevrolet Tahoe	SUV	68	74	142
Dodge Stratus	mid-size	103	40	143
Lincoln Town Car	large	100	47	147
Ford Explorer	SUV	88	60	148
Pontiac Grand Am	compact	118	39	157
Toyota Tacoma	pickup	111	59	170
Chevrolet Cavalier	subcompact	146	41	187
Dodge Neon	subcompact	161	39	200
Pontiac Sunfire	subcompact	158	44	202
Ford F-Series	pickup	110	128	238

This table as it stands does not really support the central argument that SUVs are particularly dangerous. The most-dangerous vehicle is the Ford F-Series truck, but the second-, third-, and fourth-most-dangerous vehicles are all quite small cars. No SUVs appear in the top eleven safest vehicles, but other than that they’re salted pretty well through the list.

(If you sort the list by rates of death to people not in the vehicle in question, the worst vehicles are all SUVs — but not by much. However, one of the other criticisms of SUVs in the article is that they are bought by people who are “vain, self-centered, and self-absorbed”. A self-centered and self-absorbed person is by definition not going to give a damn what risk his vehicle poses to others, to there’s no mystery about the SUV’s appeal there.)

One point of note is that SUV size doesn’t appear to correlate to hazard. The safest SUV on the list, the Chevrolet Suburban, is also the largest. The most dangerous, the Ford Explorer, is one of the smallest. The Chevrolet Suburban also proves to be significantly safer than the Chevrolet Tahoe — which is precisely the same vehicle, only shorter.

Keeping in mind that these statistics are being offered to support an argument that large vehicles are not necessarily safer than small vehicles, you might start to scratch your head about now. But wait, it gets worse.

You will note that these statistics are based on deaths per million vehicles, not deaths per million vehicle-miles, which is the usual metric for things like this. It’s not included in Gladwell’s chart, but the original study shows that the Ford Crown Victoria is far and away the most dangerous thing on the road, by any measure. This is because the vast majority of Crown Victorias these days are used as taxis or police cars, and as such spend an average of something like eighteen hours a day on the road.

Obviously, the more a vehicle is kept in the garage, the less dangerous it’s going to be. I’m sure it’s possible to be killed by a stationary car that’s not running, but it’s not the usual way this happens.

SUVs and pickup trucks are, on average, driven more than any other class of vehicle. The authors of the study point this out in their report:

new SUVs, minivans, and pickups tend to be driven 7% to 14% more miles per year than new cars.

In the very next sentence, they announce that this is a problem:

Using annual miles traveled rather than sales would tend to increase our estimate of the risk in cars relative to that in SUVs, vans, and pickups […]

Or, put another way, using miles traveled rather than vehicles sold would tend to make the estimates of relative risks more accurate, but they’re not really concerned about that. In a footnote elsewhere, they admit

Annual miles driven probably is an even better measure of the “exposure” of vehicles to fatal crashes; however, these data also are not readily available by vehicle model.

And, as we all remember from high-school science class, things that are difficult to measure are always insignificant, and can safely be ignored entirely. At least as long as ignoring those things tends to skew the results in such a way as to support our prejudices. Let’s all have a hand for science ladies and gentlemen! Hurrah!

(And never mind that counting by number of vehicles and not proportional representation in the fleet is also going to skew the figures for expensive (and thus rarer) vehicles as a few particularly dangerous or particularly safe drivers will have a greater effect on the deaths per vehicle than they would were they driving very common cars.)

Now, I’m not a scientist. I studied languages in college, for Christ’s sake. And even I notice that there’s a problem here. You can’t believe for a moment that the authors of this study, a professor of Physics at the University of Michigan and a researcher are Lawrence Berkeley National Laboratory, genuinely thought that these numbers would have any validity.

Ah, but wait. Tom Wenzel, of LBL, has a master’s degree in public policy — which I believe amounts to ‘government’ — from Berkeley. Not that there’s anything wrong with that, but it calls into question whether his research is aimed at finding the truth, or whether it’s aimed at justifying a particular policy.

And as for Marc Ross, of MIchigan, all we know about him is that he graduated from the University of Wisconsin in 1952. That he is (or was, actually; the university lists him as professor emeritus) a professor of Physics at the University of Michigan suggests that he should understand that these statistics are hardly scientific.

Anyway, as I mentioned, I’m not a scientist, and I don’t have government grants or any graduate students to do research for me, so I am at something of a disadvantage. It occurred to me, though, that it should be possible to estimate the number of miles the average owner of a given type of car drives it in an average year by looking at used car ads.

The online auction site eBay has a large section dedicated to cars, and it’s easy to search for specific makes and models there.

This is hardly scientific: cars sold on eBay might tend to have higher mileage than the average; a number of other used-car-sales channels, such as dealer used-car departments and companies like CarMax, seek out low-mileage vehicles. For the used car with higher-than-average mileage, there are fewer sales channels available.

Nevertheless, the comparison should be relatively accurate, since I used the same potentially-biased source for mileage figures for all the vehicles; while we might be missing out on low-mileage SUVs, we’re also missing out on low-mileage Toyota Avalons. I looked on eBay for 2000 model-year vehicles, and I noted the mileage on the top ten vehicles in the list — that is, the vehicles for the next ten auctions to end. I then computed an average four-year mileage for each type of vehicle.

Some vehicles the list Gladwell excerpts were not produced in 2000, and some models had fewer than ten cars for sale on eBay, so not all the cars are included in my calculations. A few clear patterns appear:

Most of the vehicles are between 60,000 and 67,000 miles. Four are conspicuously lower, and five — three SUVs and a pickup truck — are noticeably higher.

If we then divide the average mileage by the total number of deaths per million vehicles (as quoted by Gladwell) gives us a rather different ranking. This chart only includes the vehicles for which there were at least ten 2000-model-year versions for sale on eBay:

adj. rank	orig. rank	Make/Model	Avg. Miles	deaths/ million veh./ mile
1	1	Toyota Avalon	62198	0.000965
2	8	Ford Windstar	67037	0.001074
3	9	Chrysler Town & Country	61243	0.001094
4	2	Volkswagen Jetta	60333	0.001160
5	6	Honda Accord	65756	0.001217
6	14	Chevrolet Suburban	84869	0.001237
7	7	Toyota Camry	55107	0.001270
8	13	Ford Expedition	75927	0.001475
9	10	Ford Taurus	77904	0.001502
10	5	Nissan Maxima	50776	0.001556
11	12	Jeep Grand Cherokee	60236	0.001743
12	17	Chevrolet Tahoe	79225	0.001792
13	3	Subaru Legacy/Outback	54536	0.001797
14	4	Honda Civic	53766	0.002027
15	11	Toyota 4Runner	61852	0.002215
16	16	Ford Explorer	66807	0.002215
17	15	Toyota Tacoma	60935	0.002790
18	18	Ford F-Series	84468	0.002818

The standard measure of this sort of thing — the measure used by studies without an axe to grind — is incidents per million vehicle-miles, but as I am lazy I have just divided their number of deaths per million vehicles by the average number of miles driven for 2000 vehicles of the type. The number, described as “deaths per million vehicles per mile over four years” is itself pretty close to meaningless, but it’s a useful handicap of the deaths/million vehicles numbers supplied.

The Toyota Avalon is still the safest vehicle, and the Ford F-Series truck still the most dangerous, but there’s now much less of a difference (4:1 before, now less than 3:1) The Chevrolet Suburban, the largest vehicle on the list, drops from 14th-safest to 6th-safest; and its slightly smaller cousin, the Tahoe, drops from 17th to 13th. The Nissan Maxima, Subaru Legacy, and Honda Civic fall quite a bit, from fifth, third, and fourth places to eleventh, fourteenth, and fifteenth.

This chart shows the relationships in the rightmost column of the previous table. The yellow and pink columns are SUVs. You will note that

I have separated two of these SUVs out — the Suburban and the Tahoe — because they are the same vehicle. The only difference between the two is that the Tahoe is twenty inches shorter — and fifty-eight percent more dangerous. The SUVs are overrepresented on the high end of the chart — simple physics will demonstrate that a 5,000-lb. vehicle has more energy in it than a 3,000-lb. vehicle when moving at the same speed, and so more destructive force when it meets an Object At Rest, never mind anything about maneuverability. But there is clearly something at work here other than vehicle size.

It might be the raw hatred of SUVs that seems to be tout ce qu’il y a de chic these days.

Vitriol

Read these questions from the New Yorker’s Q & A — an interview with Gladwell about the article — on their website. Remember, these are questions they ask Gladwell, not answers:

You mention that people are more concerned with feeling safe than with being safe, and that this results in the popularity of limited-feedback, poorly performing vehicles like S.U.V.s. Are there other examples of this S.U.V. culture, like McMansions and so forth? […]

Market research shows that S.U.V.s tend to be bought by people who are “insecure, vain, self-centered, and self-absorbed, who are frequently nervous about their marriages, and who lack confidence in their driving skills.” […]

Does the relationship between passive safety and active safety change when the roads of the nation become lousy with S.U.V.s? […]

Gosh, now that’s some interview technique. The point of interviews is usually to find out what the other guy has to say. Leading questions are not necessarily always a bad idea, but this is just silly.

It’s nothing, though, compared to what you find in the part of the article about Detroit’s house French semiotician:

Over the past decade, a number of major automakers in America have relied on the services of a French-born cultural anthropologist, G. Clotaire Rapaille,

I swear, some part of me still believes that this whole article is satire. Could The Onion do any better? An assertion that a truck doesn’t handle as well as a Porsche, some statistics that directly contradict the conclusion, and now someone named G. Clotaire to tell us what’s wrong with Americans.

Over the past decade, a number of major automakers in America have relied on the services of a French-born cultural anthropologist, G. Clotaire Rapaille, whose speciality is getting beyond the rational — what he calls “cortex” — impressions of consumers and tapping into their deeper, “reptilian” responses. And what Rapaille concluded from countless, intensive sessions with car buyers was that when S.U.V. buyers thought about safety they were thinking about something that reached into their deepest unconscious. “The No. 1 feeling is that everything surrounding you should be round and soft, and should give,” Rapaille told me. “There should be air bags everywhere. Then there’s this notion that you need to be up high. That’s a contradiction, because the people who buy these S.U.V.s know at the cortex level that if you are high there is more chance of a rollover. But at the reptilian level they think that if I am bigger and taller I’m safer. You feel secure because you are higher and dominate and look down. That you can look down is psychologically a very powerful notion. And what was the key element of safety when you were a child? It was that your mother fed you, and there was warm liquid. That’s why cupholders are absolutely crucial for safety. If there is a car that has no cupholder, it is not safe. If I can put my coffee there, if I can have my food, if everything is round, if it’s soft, and if I’m high, then I feel safe. It’s amazing that intelligent, educated women will look at a car and the first thing they will look at is how many cupholders it has.”

Good brakes make me feel safer. For G. Clotaire, what does the trick is apparently cupholders. Chacun à son gout.

The problem with French semioticians is that they look too closely at everything, and they assume that everyone — except the French semiotician class — is deeply fucked up. A French Cultural Anthropologist will say that we have windows in our houses because we’re exhibitionists, and because we want strangers to see us naked like our mothers did while bathing us as infants: this is the (for)bidden.

Me, I have windows so I can see out, and so I can save on the lighting bill. People like sitting up high in a car because they can see further that way, and being able to see what’s going on makes them feel safer. People like drink holders because they hold their drinks, which allows them to keep both hands on the steering wheel, which makes them feel safer. Intelligent, educated women look at how many cupholders a car has because they want to know how many goddamned cupholders it has.

Sometimes a cigar, I have heard it said, is just a cigar. But not if you’re a French semiotician. Or, apparently, American journalist Malcolm Gladwell, who goes on to write:

As Keith Bradsher writes in “High and Mighty”-perhaps the most important book about Detroit since Ralph Nader’s “Unsafe at Any Speed” […]

That’s pretty dubious praise for the book, frankly; Unsafe at Any Speed was more about launching Nader’s career as a public scold than anything else. The book was about Detroit’s resistance to safety innovations in general, but it focused mainly on some design deficiencies of the Chevrolet Corvair. The book maintained that these design deficiencies were the result of a peculiarly American brand of greedy capitalism, despite the fact that Volkswagen had at the time been producing cars with the same design deficiencies — a front-mounted fuel tank and a swing-axle design that could cause the car to tend to tip over during particularly stupid maneuvering — for years. By the time the book was published, the Corvair design had already been changed by GM, but the bad publicity still sunk the car.

Bradsher brilliantly captures the mixture of bafflement and contempt that many auto executives feel toward the customers who buy their S.U.V.s.

According to this article, in 1999 the CEO of GM drove a Suburban — which might cast some doubt on how much ‘contempt’ ‘auto executives’ feel for SUV drivers. But never mind. Gladwell has more from Bradsher:

Fred J. Schaafsma, a top engineer for General Motors, says, “Sport-utility owners tend to be more like ‘I wonder how people view me,’ and are more willing to trade off flexibility or functionality to get that.” According to Bradsher, internal industry market research concluded that S.U.V.s tend to be bought by people who are insecure, vain, self-centered, and self-absorbed, who are frequently nervous about their marriages, and who lack confidence in their driving skills.

Maybe they’re gay, too. And probably ugly, and they have B.O. Where they hell are they getting this stuff (other than from French semioticians)?

What is it about cars that causes people to feel the need to psychoanalyze others based on the things they buy? Why do people so resist believing that people buy the cars they buy because they like them? People might buy a certain car to fit in (or to stand out), or to communicate something that they believe about themselves; cars are very much like clothing in that regard.

But while we judge people to an extent based on clothing, nobody would presume to say that someone who wears a certain kind of shirt does so because he’s insecure about his marriage. The New Yorker doesn’t run articles about the naiveté of people who buy high-profit-margin designer clothes; in fact, the New Yorker now has an annual ‘fashion issue’ where they celebrate the practice of paying a lot for clothing.

I am not going to spend a lot of time trying to figure out why a certain class of people — usually people who publicly proclaim that one’s possessions are unimportant — is so wrapped up in what kind of car other people choose to drive.

I will instead once again humbly suggest that the reason people drive SUVs is that calling something a ‘truck’ is the only way you can sell a type of vehicle that Americans appear to like to drive.

The United States is different from most of the rest of the world in its taste in automobiles, for a number of reasons. Among them is certainly the relatively low cost of fuel here, but it’s important not to discount the fact that Americans spend a lot more time in cars than do people in most other countries. Your requirements for a car you spend 30 hours a week in are quite different from those for a car you spend four hours a week in.

For whatever reason, Americans have long seemed to prefer much larger vehicles than people in other countries. The pleasure of driving a land-yacht or SUV isn’t the nuanced experience you get from a fine European sports car, but then there’s none of the noise and kidney-pounding, either.

I own a fine European sports car, and I like it very much, thank you. I polish it with special cloths, and I tinker with the thing endlessly. The handling, the sounds, the smells, the entire experience of the thing is wonderful. But if I’m going to drive any serious distance — or if I have to take more than one other person with me, or if I have to haul anything — it really just won’t work.

(Incidentally, my Fine European Sports Car gets about 15 miles to the gallon, significantly less than many SUVs; until they were met with a chorus of disinterest, the anti-SUVers refrain was that SUVs were inefficient.)

So a lot of Americans want or need larger vehicles. This is hardly a fad; American cars have long been legendarily large and powerful. In a country with cheap fuel and plenty of parking, there’s no particular reason not to have a large vehicle if it’s going to be more convenient.

Americans did not start driving SUVs as a result of some marketing campaign, they started driving them because the vehicles that had formerly offered what the SUV offers had disappeared. Remember James Brown’s Cherokee, and OJ Simpson’s Bronco? A generation ago, both of these guys would have driven Cadillacs. By the laste 80s and early 90s, though, Cadillacs were underpowered, overpriced front-wheel-drive cars that were hamstrung by the government’s CAFE laws.

The CAFE — Corporate Average Fuel Economy — standards require that the cars that a manufacturer sells attain a certain average fuel mileage. Obviously, this is a problem if you are Cadillac, and in the business of selling large and powerful cars to people who don’t particularly care about fuel economy.

However, if your vehicle is a truck, you have to meet a different, less stringent, standard. So we wind up with the SUV, which is basically a station wagon built on a ‘truck’ chassis. What’s a truck? The EPA says:

Light-Duty Truck (LDT) Any motor vehicle rated at 8,500 pounds GVWR or less which has a vehicle curb weight of 6,000 pounds or less and which has a basic vehicle frontal area of 45 square feet or less, which is:

1. Designed primarily for purposes of transportation of property or is a derivation of such a vehicle, or
2. Designed primarily for transportation of persons and has a capacity of more than 12 persons, or
3. Available with special features enabling off-street or off-highway operation and use. (40 CFR 86.1803-01)

So if you build a Cadillac Sedan de Ville, you have to meet car standards. If you build a Cadillac Escalade, which is a derivation of the Chevrolet Suburban, which is in turn a derivation of the basic Chevrolet pickup truck, you have to meet truck standards, which are less stringent.

This, and not any balderdash about sexual inadequacy, is why SUVs are so damned popular, and at the same time it’s why they’re less safe than passenger cars, to the extent that this is actually true.

If you want to sell a large, powerful vehicle in this country, it’s much easier and cheaper to do so if the thing is legally a truck. Since it’s easier and cheaper to build, it’s easier and cheaper to buy. The SUV boom is the market’s adaptation to the government’s diktat that cars shall achieve certain fuel economy standards.

One would think that the lesson to be taken from this is that the market is an incredibly powerful force, and that trying to overcome the will of the market will produce unintentional and possibly undesired results.

Or, if you spin the numbers right, you can decide that what’s really needed is vilification of people who are just trying to get what they want a way they’re allowed to have it — clearly a state of affairs that calls for more regulation.

Posted by tino at 17:40 20.01.04