Model

Hidden Disparate Impact

It is against commonly held intuitions that a group can be both over-represented in a profession, school, or program, and discriminated against. The simplest way to test for discrimination is to look at the general population, find the percent that a group represents, then expect them to represent exactly that percentage in any endeavour, absent discrimination.

Harvard, for example, is 17.1% Asian-American (foreign students are broken out separately in the statistics I found, so we’re only talking about American citizens or permanent residents in this post). America as a whole is 4.8% Asian-American. Therefore, many people will conclude that there is no discrimination happening against Asian-Americans at Harvard.

This is what would happen under many disparate impact analyses of discrimination, where the first step to showing discrimination is showing one group being accepted (for housing, employment, education, etc.) at a lower rate than another.

I think this naïve view is deeply flawed. First, we have clear evidence that Harvard is discriminating against Asian-Americans. When Harvard assigned personality scores to applicants, Asian-Americans were given the lowest scores of any ethnic group. When actual people met with Asian-American applicants, their personality scores were the same as everyone else’s; Harvard had assigned many of the low ratings without ever meeting the students, in what many suspect is an attempt to keep Asian-Americans below 20% of the student body.

Personality ratings in college admissions have a long and ugly history. They were invented to enforce quotas on Jews in the 1920s. These discriminatory quotas had a chilling effect on Jewish students; Dr. Jonas Salk, the inventor of the polio vaccine, chose the schools he attended primarily because they were among the few which didn’t discriminate against Jews. Imagine how prevalent and all-encompassing the quotas had to be for him to be affected.

If these discriminatory personality scores were dropped (or Harvard stopped fabricating bad results for Asian-Americans), Asian-American admissions at Harvard would rise.

This is because the proper measure of how many Asian-Americans should get into Harvard has little to do with their percentage of the population. It has to do with how many would meet Harvard’s formal admission criteria. Since Asian-Americans have much higher test scores than any other demographic group in America, it only stands to reason that we should expect to see Asian-Americans over-represented among any segment of the population that is selected at least in part by their test scores.

Put simply, Asian-American test scores are so good (on average) that we should expect to see proportionately more Asian-Americans than any other group get into Harvard.

This is the comparison we should be making when looking for discrimination in Harvard’s admissions. We know their criteria and we know roughly what the applicants look like. Given this, what percentage of applicants should get in if the criteria were applied fairly? The answer turns out to be about four times as many Asian-Americans as are currently getting in.

Hence, discrimination.

Unfortunately, this only picks up one type of discrimination – the discrimination that occurs when stated standards are being applied in an unequal manner. There’s another type of discrimination that can occur when standards aren’t picked fairly at all; their purpose is to act as a barrier, not assess suitability. This does come up in formal disparate impact analyses – you have to prove that any standards that lead to disparate impact are necessary – but we’ve already seen how you can avoid triggering those if you pick your standard carefully and your goal isn’t to lock a group out entirely, but instead to reduce their numbers.

Analyzing the necessity of standards that may have disparate impact can be hard and lead to disagreement.

For example, we know that Harvard’s selection criteria must be discriminate, which is to say it must differentiate. We want elite institutions to have selection criteria that differentiate between applicants! There is a general agreement, for example, that someone who fails all of their senior year courses won’t get into Harvard and someone who aces them might.

If we didn’t have a slew of records from Harvard backing up the assertion that personality criteria were rigged to keep out Asian-Americans (like they once kept out Jews), evaluating whether discrimination was going on at Harvard would be harder. There’s no prima facie reason to consider personality scores (had they been adopted for a more neutral purpose and applied fairly) to be a bad selector.

It’s a bit old fashioned, but there’s nothing inherently wrong with claiming that you also want to select for moral character and leadership when choosing your student body. The case for this is perhaps clearer at Harvard, which views itself as a training ground for future leaders. Therefore, personality scores aren’t clearly useless criteria and we have to apply judgement when evaluating whether it’s reasonable for Harvard to select its students using them.

Historically, racism has used seemingly valid criteria to cloak itself in a veneer of acceptability. Redlining, the process by which African-Americans were denied mortgage financing hid its discriminatory impact with clinical language about underwriting risk. In reality, redlining was not based on actual actuarial risk in a neighbourhood (poor whites were given loans, while middle-class African-Americans were denied them), but by the racial composition of the neighbourhood.

Like in the Harvard case, it was only the discovery of redlined maps that made it clear what was going on; the criterion was seemingly borderline enough that absent evidence, there was debate as to whether it existed for reasonable purpose or not.

(One thing that helped trigger further investigation was the realization that well-off members of the African-American community weren’t getting loans that a neutral underwriter might expect them to qualify for; their income and credit was good enough that we would have expected them to receive loans.)

It is also interesting to note that both of these cases hid behind racial stereotypes. Redlining was defended because of “decay” in urban neighbourhoods (a decay that was in many cases caused by redlining), while Harvard’s admissions relied upon negative stereotypes of Asian-Americans. Many were dismissed with the label “Standard Strong”, implying that they were part of a faceless collective, all of whom had similarly impeccable grades and similarly excellent extracurricular, but no interesting distinguishing features of their own.

Realizing how hard it is to tell apart valid criteria from discriminatory ones has made me much more sympathetic to points raised by technocrat-skeptics like Dr. Cathy O’Neil, who I have previously been harsh on. When bad actors are hiding the proof of their discrimination, it is genuinely difficult to separate real insurance underwriting (which needs to happen for anyone to get a mortgage) from discriminatory practices, just like it can be genuinely hard to separate legitimate college application processes from discriminatory ones.

While numerical measures, like test scores, have their own problems, they do provide some measure of impartiality. Interested observers can compare metrics to outcomes and notice when they’re off. Beyond redlining and college admissions, I wonder what other instances of potential discrimination a few civic minded statisticians might be able to unearth.

Link Post

Link Post – November 2018

When a poet writes about his experience of becoming a lawyer after his release from jail, you know it’s going to be a punch in the gut. One thing I noticed: he would have had a much easier time reintegrating to society, finding a job, etc. had he been tried as a juvenile, rather than an adult. Has there been any meaningful study on recidivism rates between these two groups? You could compare 17 year olds and 18 year olds charged with the same crime and look at outcomes fifteen years down the road.

Segway’s patents are now at the core of the new crop of ride-sharing scooters, which may finally bring about the original promise of the Segway. Perhaps one element of Segway’s downfall (beyond how uncool they were) is how proper they were about everything. They worked hard to get laws passed that made it legal to ride Segways on the sidewalk, rather than “innovating on the regulatory side” (read: ignoring the law) like the scooter companies do.

The winner of the 2018 Boston Marathon is a delightfully dedicated oddball.

Housing can’t be both affordable and a good investment. Currently, “good investment” seems to be beating affordable in many cities and its residents’ groups that ostensibly support affordable housing that are fighting to keep it that way with restrictive zoning.

Does Canada even exist? Or is it a made up place Americans use as the homeland when travelling? I must admit, I’ve been convinced almost to the point of Canadagnosticism.

What would happen if you laid out all the contradictory information about rapid transit in Karachi in one place? “Something a bit post-modern and a bit absurd” seems to be the answer.

Dying scientist launches a desperate attempt to prove that his herpes vaccine works. In the movies, he’d be ultimately vindicated. In real life, several people are left with lingering side effects and all of the data he collected is tainted by poor methodology.

The whole “rich kids of Instagram” thing is full of pyramid schemes that advertise risky financial products to impoverished teens.

Political theorist Hannah Arendt once claimed that you must never say “who am I to judge”. A therapist sees dramatic improvements by teaching their clients to be more judgemental, seems to agree.

Whenever I read about bullshit jobs, I feel like economic competition needs to be turned up to 11 so that companies have no slack with which to hire people to do pointless tasks. One thing that progressives might not appreciate: the investor class probably hates bullshit jobs even more than they do; from the perspective of a stockholder, a bullshit job is management stealing their money so that the managers can get off on feeling powerful.

Economics, History

Scrip Stamp Currencies Aren’t A Miracle

A friend of mine recently linked to a story about stamp scrip currencies in a discussion about Initiative Q [1]. Stamp scrip currencies are an interesting monetary technology. They’re bank notes that require weekly or monthly stamps in order to be valid. These stamps cost money (normally a few percent of the face value of the note), which imposes a cost on holding the currency. This is supposed to encourage spending and spur economic activity.

This isn’t just theory. It actually happened. In the Austrian town of Wörgl, a scrip currency was used to great effect for several months during the Great Depression, leading to a sudden increase in employment, money for necessary public works, and a general reversal of fortunes that had, until that point, been quite dismal. Several other towns copied the experiment and saw similar gains, until the central bank stepped in and put a stop to the whole thing.

In the version of the story I’ve read, this is held up as an example of local adaptability and creativity crushed by centralization. The moral, I think, is that we should trust local institutions instead of central banks and be on the lookout for similar local currency strategies we could adopt.

If this is all true, it seems like stamp scrip currency (or some modern version of it, perhaps applying the stamps digitally) might be a good idea. Is this the case?

My first, cheeky reaction, is “we already have this now; it’s called inflation.” My second reaction is actually the same as my first one, but has an accompanying blog post. Thus.

Currency arrangements feel natural and unchanging, which can mislead modern readers when they’re thinking about currencies used in the 1930s. We’re very used to floating fiat currencies, that (in general) have a stable price level except for 1-3% inflation every year.

This wasn’t always the case! Historically, there was very little inflation. Currency was backed by gold at a stable ratio (there were 23.2 grains of gold in a US dollar from 1834 until 1934). For a long time, growth in global gold stocks roughly tracked total growth in economic activity, so there was no long-run inflation or deflation (short-run deflation did cause several recessions, until new gold finds bridged the gap in supply).

During the Great Depression, there was worldwide gold hoarding [2]. Countries saw their currency stocks decline or fail to keep up with the growth rate required for full economic activity (having a gold backed currency meant that the central bank had to decrease currency stocks whenever their gold stocks fell). Existing money increased in value, which meant people hoarded that too. The result was economic ruin.

In this context, a scrip currency accomplished two things. First, it immediately provided more money. The scrip currency was backed by the national currency of Austria, but it was probably using a fractional reserve system – each backing schilling might have been used to issue several stamp scrip schillings [3]. This meant that the town of Wörgl quickly had a lot more money circulating. Perhaps one of the best features of the scrip currency within the context of the Great Depression was that it was localized, which meant that it’s helpful effects didn’t diffuse.

(Of course, a central bank could have accomplished the same thing by printing vastly more money over a vastly larger area, but there was very little appetite for this among central banks during the Great Depression, much to everyone’s detriment. The localization of the scrip is only an advantage within the context of central banks failing to ensure adequate monetary growth; in a more normal environment, it would be a liability that prevented trade.)

Second to this, the stamp scrip currency provided an incentive to spend money.

Here’s one model of job loss in recessions: people (for whatever reason; deflation is just one cause) want to spend less money (economists call this “a decrease in aggregate demand”). Businesses see the falling demand and need to take action to cut wages or else become unprofitable. Now people generally exhibit “downward nominal wage rigidity” – they don’t like pay cuts.

Furthermore, individuals don’t realize that demand is down as quickly as businesses do. They hold out for jobs at the same wage rate. This leads to unemployment [4].

Stamp scrip currencies increase aggregate demand by giving people an incentive to spend their money now.

Importantly, there’s nothing magic about the particular method you choose to do this. Central banks targeting 2% inflation year on year (and succeeding for once [5]) should be just as effective as scrip currencies charging 2% of the face value every year [6]. As long as you’re charged some sort of fee for holding onto money, you’re going to want to spend it.

Central bank backed currencies are ultimately preferable when the central bank is getting things right, because they facilitate longer range commerce and trade, are administratively simpler (you don’t need to go buy stamps ever), and centralization allows for more sophisticated economic monitoring and price level targeting [7].

Still, in situations where the central bank fails, stamp scrip currencies can be a useful temporary stopgap.

That said, I think a general caution is needed when thinking about situations like this. There are few times in economic history as different from the present day as the Great Depression. The very fact that there was unemployment north of 20% and many empty factories makes it miles away from the economic situation right now. I would suspect that radical interventions that were useful during the Great Depression might be useless or actively harmful right now, simply due to this difference in circumstances.

Footnotes

[1] My opinion is that their marketing structure is kind of cringey (my Facebook feed currently reminds me of all of the “Paul Allen is giving away his money” chain emails from the 90s and I have only myself to blame) and their monetary policy has two aims that could end up in conflict. On the other hand, it’s fun to watch the numbers go up and idly speculate about what you could do if it was worth anything. I would cautiously recommend Q ahead of lottery tickets but not ahead of saving for retirement. ^

[2] See “The Midas Paradox” by Scott Sumner for a more in-depth breakdown. You can also get an introduction to monetary theories of the business cycle on his blog, or listen to him talk about the Great Depression on Vimeo. ^

[3] The size of the effect talked about in the article suggests that one of three things had to be true: 1) the scrip currency was fractionally backed, 2) Wörgl had a huge bank account balance a few years into the recession, or 3) the amount of economic activity in the article is overstated. ^

[4] As long as inflation is happening like it should be, there won’t be protracted unemployment, because a slight decline in economic activity is quickly counteracted by a slightly decreased value of money (from the inflation). Note the word “nominal” up there. People are subject to something called a “money illusion”. They think in terms of prices and salaries expressed in dollar values, not in purchasing power values.

There was only a very brief recession after the dot com crash because it did nothing to affect the money supply. Inflation happened as expected and everything quickly corrected to almost full employment. On the other hand, the Great Depression lasted as long as it did because most countries were reluctant to leave the gold standard and so saw very little inflation. ^

[5] Here’s an interesting exercise. Look at this graph of US yearly inflation. Notice how inflation is noticeably higher in the years immediately preceding the Great Recession than it is in the years afterwards. Monetarist economists believe that the recession wouldn’t have lasted as long if it there hadn’t been such a long period of relatively low inflation.

As always, I’m a huge fan of the total lack of copyright on anything produced by the US government.

^

[6] You might wonder if there’s some benefit to both. The answer, unfortunately, is no. Doubling them up should be roughly equivalent to just having higher inflation. There seems to be a natural rate of inflation that does a good job balancing people’s expectations for pay raises (and adequately reduces real wages in a recession) with the convenience of having stable money. Pushing inflation beyond this point can lead to a temporary increase in employment, by making labour relatively cheaper compared to other inputs.

The increase in employment ends when people adjust their expectations for raises to the new inflation rate and begin demanding increased salaries. Labour is no longer artificially cheap in real terms, so companies lay off some of the extra workers. You end up back where you started, but with inflation higher than it needs to be.

See also: “The Importance of Stable Money: Theory and Evidence” by Michael Bordo and Anna Schwartz. ^

[7] I suspect that if the stamp scrip currency had been allowed to go on for another decade or so, it would have had some sort of amusing monetary crisis. ^

Model, Politics, Science

Science Is Less Political Than Its Critics

A while back, I was linked to this Tweet:

It had sparked a brisk and mostly unproductive debate. If you want to see people talking past each other, snide comments, and applause lights, check out the thread. One of the few productive exchanges centres on bridges.

Bridges are clearly a product of science (and its offspring, engineering) – only the simplest bridges can be built without scientific knowledge. Bridges also clearly have a political dimension. Not only are bridges normally the product of politics, they also are embedded in a broader political fabric. They change how a space can be used and change geography. They make certain actions – like commuting – easier and can drive urban changes like suburb growth and gentrification. Maintenance of bridges uses resources (time, money, skilled labour) that cannot be then used elsewhere. These are all clearly political concerns and they all clearly intersect deeply with existing power dynamics.

Even if no other part of science was political (and I don’t think that could be defensible; there are many other branches of science that lead to things like bridges existing), bridges prove that science certainly can be political. I can’t deny this. I don’t want to deny this.

I also cannot deny that I’m deeply skeptical of the motives of anyone who trumpets a political view of science.

You see, science has unfortunate political implications for many movements. To give just one example, greenhouse gasses are causing global warming. Many conservative politicians have a vested interest in ignoring this or muddying the water, such that the scientific consensus “greenhouse gasses are increasing global temperatures” is conflated with the political position “we should burn less fossil fuel”. This allows a dismissal of the political position (“a carbon tax makes driving more expensive; it’s just a war on cars”) serve also (via motivated cognition) to dismiss the scientific position.

(Would that carbon in the atmosphere could be dismissed so easily.)

While Dr. Wolfe is no climate change denier, it is hard to square her claims that calling science political is a neutral statement:

With the examples she chooses to demonstrate this:

When pointing out that science is political, we could also say things like “we chose to target polio for a major elimination effort before cancer, partially because it largely affected poor children instead of rich adults (as rich kids escaped polio in their summer homes)”. Talking about the ways that science has been a tool for protecting the most vulnerable paints a very different picture of what its political nature is about.

(I don’t think an argument over which view is more correct is ever likely to be particularly productive, but I do want to leave you with a few examples for my position.)

Dr. Wolfe’s is able to claim that politics is neutral despite only using negative examples of its effects by using a bait and switch between two definitions of “politics”. The bait is a technical and neutral definition, something along the lines of: “related to how we arrange and govern our society”. The switch is a more common definition, like: “engaging in and related to partisan politics”.

I start to feel that someone is being at least a bit disingenuous when they only furnish negative examples, examples that relate to this second meaning of the word political, then ask why their critics view politics as “inherently bad” (referring here to the first definition).

This sort of bait and switch pops up enough in post-modernist “all knowledge is human and constructed by existing hierarchies” places that someone got annoyed enough to coin a name for it: the motte and bailey fallacy.

Image Credit: Hchc2009, Wikimedia Commons.

 

It’s named after the early-medieval form of castle, pictured above. The motte is the outer wall and the bailey is the inner bit. This mirrors the two parts of the motte and bailey fallacy. The “motte” is the easily defensible statement (science is political because all human group activities are political) and the bailey is the more controversial belief actually held by the speaker (something like “we can’t trust science because of the number of men in it” or “we can’t trust science because it’s dominated by liberals”).

From Dr. Wolfe’s other tweets, we can see the bailey (sample: “There’s a direct line between scientism and maintaining existing power structures; you can see it in language on data transparency, the recent hoax, and more.“). This isn’t a neutral political position! It is one that a number of people disagree with. Certainly Sokal, the hoax paper writer who inspired the most recent hoaxes is an old leftist who would very much like to empower labour at the expense of capitalists.

I have a lot of sympathy for the people in the twitter thread who jumped to defend positions that looked ridiculous from the perspective of “science is subject to the same forces as any other collective human endeavour” when they believed they were arguing with “science is a tool of right-wing interests”. There are a great many progressive scientists who might agree with Dr. Wolfe on many issues, but strongly disagree with what her position seems to be here. There are many of us who believe that science, if not necessary for a progressive mission, is necessary for the related humanistic mission of freeing humanity from drudgery, hunger, and disease.

It is true that we shouldn’t uncritically believe science. But the work of being a critical observer of science should not be about running an inquisition into scientists’ political beliefs. That’s how we get climate change deniers doxxing climate scientists. Critical observation of science is the much more boring work of checking theories for genuine scientific mistakes, looking for P-hacking, and doubled checking that no one got so invested in their exciting results that they fudged their analyses to support them. Critical belief often hinges on weird mathematical identities, not political views.

But there are real and present dangers to uncritically not believing science whenever it conflicts with your politic views. The increased incidence of measles outbreaks in vaccination refusing populations is one such risk. Catastrophic and irreversible climate change is another.

When anyone says science is political and then goes on to emphasize all of the negatives of this statement, they’re giving people permission to believe their political views (like “gas should be cheap” or “vaccines are unnatural”) over the hard truths of science. And that has real consequences.

Saying that “science is political” is also political. And it’s one of those political things that is more likely than not to be driven by partisan politics. No one trumpets this unless they feel one of their political positions is endangered by empirical evidence. When talking with someone making this claim, it’s always good to keep sight of that.

Literature

Book Review: Bad Blood

Theranos was founded in 2003 by Stanford drop-out Elizabeth Holmes. It and its revolutionary blood tests eventually became a Silicon Valley darling, raising $700 million from investors that included Rupert Murdoch and the Walton family. It ultimately achieved a valuation of almost $10 billion on yearly revenues of $100 million. Elizabeth Holmes was hailed as Silicon Valley’s first self-made female billionaire.

In 2015, a series of articles by John Carreyrou published in the Wall Street Journal popped this bubble. Theranos was a fraud. Its blood tests didn’t work and were putting patient lives at risk. Its revenue was one thousand times smaller than reported. It had engaged in a long running campaign of intimidation against employees and whistleblowers. Its board had entirely failed to hold the executives to account – not surprising, since Elizabeth Holmes controlled over 99% of the voting power.

Bad Blood is the story of how this happened. John Carreyrou interviewed more than 140 sources, including 60 former employees to create the clearest possible picture of the company, from its founding to just before it dissolved.

It’s also the story of Carreyrou’s reporting on Theranos, from the first fateful tip he received after winning a Pulitzer for uncovering another medical fraud, to repeated legal threats from Theranos’s lawyers, to the slew of awards his coverage won when it eventually proved correct.

I thought it was one hell of a book and would recommend it to anyone who likes thrillers or anyone who might one day work at a start-up and wants a guide to what sort of company to avoid (pro tip: if your company is faking its demos to investors, leave).

Instead of rehashing the book like I sometimes do in my reviews, I want to discuss three key things I took from it.

Claims that Theranos is “emblematic” of Silicon Valley are overblown

Carreyrou vacillates on this point. He sometimes points out all the ways that Theranos is different from other VC backed companies and sometimes holds it up as a poster child for everything that is wrong with the Valley.

I’m much more in the first camp. For Theranos to be a posterchild of the Valley, you’d want to see it raise money from the same sources as other venture-backed companies. This just wasn’t the case.

First of all, Theranos had basically no backing from dedicated biotechnology venture capitalists (VCs). This makes a lot of sense. The big biotech VCs do intense due-diligence. If you can’t explain exactly how your product works to a room full of intensely skeptical PhDs, you’re out of luck. Elizabeth Holmes quickly found herself out of luck.

Next is the list of VCs who did invest. Missing are the big names from the Valley. There’s no Softbank, no Peter Thiel, no Andreessen Horowitz. While these investors may have less ability to judge biotech start-ups than the life sciences focused firms, they are experienced in due diligence and they knew red flags (like Holmes’s refusal to explain how her tech worked, even under NDA) when they saw them. I work at a venture backed company and I can tell you that experienced investors won’t even look at you if you aren’t willing to have a frank discussion about your technology with them.

The people who did invest? Largely dabblers, like Rupert Murdoch and the Walton family, drawn in by a board studded with political luminaries (two former secretaries of state, James friggen’ Mattis, etc.). It perhaps should have been a red flag that Henry Kissinger (who knows nothing about blood testing and would be better placed on Facebook’s board, where his expertise in committing war crimes would come in handy) was on the board, but to the well-connected elites from outside the Valley, this was exactly the opposite.

It is hard to deal with people who just lie

I don’t want to blame these dabblers from outside the Valley too much though, because they were lied to like crazy. As America found out in 2016, many institutions struggle when dealing with people who just make shit up.

There is an accepted level of exaggeration that happens when chasing VC money. You put your best foot forward, shove the skeletons deep into your closet, and you try and be the most charming and likable version of you. One founder once described trying to get money from VCs as “basically like dating” to me and she wasn’t wrong.

Much like dating, you don’t want to exaggerate too far. After all, if the suit is fruitful, you’re kind of stuck with each other. The last thing you want to find out after the fact is that your new partner collects their toenail clippings in a jar or overstates their yearly revenue by more than 1000x.

VCs went into Theranos with the understanding that they were probably seeing rosy forecasts. What they didn’t expect was that the forecasts they saw were 5x the internal forecasts, or that the internal forecasts were made by people who had no idea what the current revenue was. This just doesn’t happen at a normal company. I’m used to internal revenue projections being the exact same as the ones shown to investors. And while I’m sure no one would bat an eye if you went back and re-did the projections with slightly more optimistic assumptions, you can’t get to a 5x increase in revenue just by doing that. Furthermore, the whole exercise of doing projections is moot if you are already lying about your current revenue by 1000x.

There is a good reason that VCs expect companies not to do this. I’m no lawyer, but I’m pretty sure that this is all sorts of fraud. The SEC and US attorney’s office seem to agree. It’s easy to call investors naïve for buying into Theranos’s lies. But I would contend that Holmes and Balwani (her boyfriend and Theranos’s erstwhile president) were the naïve ones if they thought they could get away with it without fines and jail time.

(Carreyrou makes a production about how “over-promise, then buy time to fix it later” is business as usual for the Valley. This is certainly true if you’re talking about, say, customers of a free service. But it is not and never has been accepted practice to do this to your investors. You save the rosy projections for the future! You don’t lie about what is going on right now.)

The existence of a crime called “fraud” is really useful for our markets. When lies of the sort that Theranos made are criminalized, business transactions become easier. You expect that people who are scammers will go do their scams somewhere where lies aren’t so criminalized and they mostly do, because investors are very prone to sue or run to the SEC when lied to. Since this mostly works, it’s understandable that a sense of complacency might set in. When everyone habitually tells more or less the truth, everyone forgets to check for lies.

The biotech companies didn’t invest in Theranos because their sweep for general incompetence made it clear that something fishy was going on. The rest of the VCs were less lucky, but I would argue that when the books are as cooked as Theranos’s were, a lack of understanding of biology was not the primary problem with these investors. The primary problem was that they thought they were buying a company that was making $100 million a year when in fact it was making $100,000.

Most VCs (and probably most of the dabblers, who after all made their money in business of some sort) may not understand the nuances of biotech, but they do understand that revenue that low more than a decade into operation represent a serious problem. Conversely, revenues of $100 million are pretty darn good for a decade-old medical device company. With that lie out of the way, the future growth projections looked reasonable; they were just continuing a trend. Had any investors been told the truth, they could have used their long experience as business people or VCs to realize that Theranos was a bad deal. Holmes’s lies prevented that.

I sure wish there was a way to make lies less powerful in areas where people mostly stick near the truth (and that we’d found one before 2016), but absent that, I want to give Theranos’s investors a bit of a break.

Theranos was hardest on ethical people

Did you know that Theranos didn’t have a chief financial officer for most of its existence? Their first CFO confronted Holmes about her blatant lies to investors (she was entirely faking the blood tests that they “took”) and she fired him, then used compromising material on his computer to blackmail him into silence. He was one of the lucky ones.

Bad Blood is replete with stories of idealistic young people who joined Theranos because it seemed to be one of the few start-ups that was actually making a positive difference in normal people’s lives. These people would then collide with Theranos’s horrible management culture and begin to get disillusioned. Seeing the fraud that took place all around them would complete the process. Once cynicism set in, employees would often forward some emails to themselves so they’d have proof that they only participated in the fraud when unaware and immediately handed in their notice.

If they emailed themselves, they’d get a visit from a lawyer. The lawyer would tell them that forwarding emails to themselves was stealing Theranos’s trade secrets (everything was a trade secret with Theranos, especially the fact that they were lying about practically everything). The lawyer would present the employee with an option: delete the emails and sign a new NDA that included a non-disparagement clause that prevented them from criticising Theranos, or be sued by the fiercely talented and amoral lawyer David Boies (who was paid in Theranos stock and had a material interest in keeping the company afloat) until they were bankrupted by the legal fees.

Most people signed the paper.

If employees left without proof, they’d either be painted as deranged and angered by being fired, or they be silenced with the threat of lawsuits.

Theranos was a fly trap of a company. Its bait was a chance to work on something meaningful. But then it was set up to be maximally offensive and demoralizing for the very people who would jump at that opportunity. Kept from speaking out, guilt at helping perpetuate the fraud could eat them alive.

One employee, Ian Gibbons, committed suicide when caught between Theranos’s impossible demands for loyalty and an upcoming deposition in a lawsuit against the company.

To me, this makes Theranos much worse than seemingly similar corporate frauds like Enron. Enron didn’t attract bright-eyed idealists, crush them between an impossible situation and their morals, then throw them away to start the process over again. Enron was a few directors enriching themselves at the expense of their investors. It was wrong, but it wasn’t monstrous.

Theranos was monstrous.

Elizabeth Holmes never really made any money from her fraud. She was paid a modest (by Valley standards) salary of $200,000 per year – about what a senior engineer could expect to make. It’s probably about what she could have earned a few years after finishing her Stanford degree, if she hadn’t dropped out. Her compensation was mostly in stock and when the SEC forced her to give up most of it and the company went bankrupt, its value plummeted from $4.5 billion to $0. She never cashed out. She believed in Theranos until the bitter end.

If she’d been in it for the money, I could have understood it, almost. I can see how people would do – and have done – horrible things to get their hands on $4.5 billion. But instead of being motivated by money, she was motivated by some vision. Perhaps of saving the world, perhaps of being admired. In either case, she was willing to grind up and use up anyone and everyone around her in pursuit of that vision. Lying was on the table. Ruining people’s lives was on the table. Callously dismissing a suicide that was probably caused by her actions was on the table. As far as anyone knows, she has never shown remorse for any of these. Never viewed her actions as anything but moral and upright.

And someone who can do that scares me. People who are in it for the money don’t go to bed thinking they’re squeaky clean. They know they’ve made a deal with the devil. Elizabeth Holmes doesn’t know and doesn’t understand.

I think it’s probably for the best that no one will trust Elizabeth Holmes with a fish and chips stand, let alone a billion-dollar company, ever again. Because I tremble to think of what she could do if given another chance to “change the world”.

Model

Hacked Pacemakers Won’t Be This Year’s Hot Crime Trend

Or: the simplest ways of killing people tend to be the most effective.

A raft of articles came out during Defcon showing that security vulnerabilities exist in some pacemakers, vulnerabilities which could allow attackers to load a pacemaker with arbitrary code. This is obviously worrying if you have a pacemaker implanted. It is equally self-evident that it is better to live in a world where pacemakers cannot be hacked. But how much worse is it to live in this unfortunately hackable world? Are pacemaker hackings likely to become the latest crime spree?

Electrical grid hackings provide a sobering example. Despite years of warning that the American electrical grid is vulnerable to cyber-attacks, the greatest threat to America’s electricity infrastructure remains… squirrels.

Hacking, whether it’s of the electricity grid or of pacemakers gets all the headlines. Meanwhile fatty foods and squirrels do all the real damage.

(Last year, 610,000 Americans died of heart disease and 0 died of hacked pacemakers.)

For all the media attention that novel cyberpunk methods of murder get, they seem to be rather ineffective for actual murder, as demonstrated by the paucity of murder victims. I think this is rather generalizable. Simple ways of killing people are very effective but not very scary and so don’t garner much attention. On the other hand, particularly novel or baroque methods of murder cause a lot of terror, even if almost no one who is scared of them will ever die of them.

I often demonstrate this point by comparing two terrorist organizations: Al Qaeda and Daesh (the so-called Islamic State). Both of these groups are brutally inhumane, think nothing of murder, and are made up of some of the most despicable people in the world. But their methodology couldn’t be more different.

Al Qaeda has a taste for large, complicated, baroque plans that, when they actually work, cause massive damage and change how people see the world for years. 9/11 remains the single deadliest terror attack in recorded history. This is what optimizing for terror looks like.

On the other hand, when Al Qaeda’s plans fail, they seem almost farcical. There’s something grimly amusing about the time that Al Qaeda may have tried to weaponize the bubonic plague and instead lost over 40 members when they were infected and promptly died (the alternative theory, that they caught the plague because of squalid living conditions, looks only slightly better).

(Had Al Qaeda succeeded and killed even a single westerner with the plague, people would have been utterly terrified for months, even though the plague is relatively treatable by modern means and would have trouble spreading in notably flea-free western countries.)

Daesh, on the other hand, prefers simple attacks. When guns are available, their followers use them. When they aren’t, they’ll rent vans and plough them into crowds. Most of Daesh’s violence occurs in Syria and Iraq, where they once controlled territory with unparalleled brutality. This is another difference in strategy (as Al Qaeda is outward facing, focused mostly on attacking “The West”). Focusing on Syria and Iraq, where the government lacks a monopoly on violence and they could originally operate with impunity, Daesh racked up a body count that surpassed Al Qaeda’s.

While Daesh has been effective in terms of body count, they haven’t really succeeded (in the west) in creating the lasting terror that Al Qaeda did. This is perhaps a symptom of their quotidian methods of murder. No one walked around scared of a Daesh attack and many of their murders were lost in the daily churn of the news cycle – especially the ones that happened in Syria and Iraq.

I almost wonder if it is impossible for attacks or murders by “normal” means to cause much terror beyond those immediately affected. Could hacked pacemakers remain terrifying if as many people died of them as gunshots? Does familiarity with a form of death remove terror, or are some methods of death inherently more terrible and terrifying than others?

(It is probably the case that both are true, that terror is some function of surprise, gruesomeness, and brutality, such that some things will always terrify us, while others are horrible, but have long since lost their edge.)

Terror for its own sake (or because people believe it is the best path to some objective) must be a compelling option to some, because otherwise everyone would stick to simple plans whenever they think violence will help them achieve their aims. I don’t want to stereotype too much, but most people who going around being terrorists or murders typically aren’t the brightest bulbs in the socket. The average killer doesn’t have the resources to hack your pacemaker and the average terrorist is going to have much better luck with a van than with a bomb. There are disadvantages to bombs! The average Pastun farmer or disaffected mujahedeen is not a very good chemist and homemade explosives are dangerous even to skilled chemists. Accidental detonations abound. If there wasn’t some advantage in terror to be had, no one would mess around with explosives when guns and vans can be easily found.

(Perhaps this advantage is in a multiplier effect of sorts. If you are trying to win a violent struggle directly, you have to kill everyone who stands in your way. Some people might believe that terror can short-circuit this and let them scare away some of their potential opponents. Historically, this hasn’t always worked.)

In the face of actors committed to terror, we should remember that our risk of dying by a particular method is almost inversely related to how terrifying we find it. Notable intimidators like Vladimir Putin or the Mossad kill people with nerve gasses, polonium, and motorcycle delivered magnetic bombs to sow fear. I can see either of them one day adding hacked pacemakers to their arsenal.

If you’ve pissed off the Mossad or Putin and would like to die in some way other than a hacked pacemaker, then by all means, go get a different one. Otherwise, you’re probably fine waiting for a software update. If, in the meantime, you don’t want to die, maybe try ignoring headlines and instead not owning a gun and skipping French fries. Statistically, there isn’t much that will keep you safer.

Coda

Our biases make it hard for us to treat things that are easy to remember as uncommon, which no doubt plays a role here. I wrote this post like this – full of rambles, parentheses, and long-winded examples – to try and convey the difficult intuition, that we should discount as likely to effect us any method of murder that seems shocking, but hard. Remember that most crimes are crimes of opportunity and most criminals are incompetent and you’ll never be surprised to hear the three most common murder weapons are guns, knives, and fists.

Economics, Politics

Good Intentions Meet A Messy Reality In Elizabeth Warren’s Corporate Citizenship Push

[Epistemic Status: I am not an economist. I am fairly confident in my qualitative assessment, but there could be things I’ve overlooked.]

Vox has an interesting article on Elizabeth Warren’s newest economic reform proposal. Briefly, she wants to force corporations with more than $1 billion in revenue to apply for a charter of corporate citizenship.

This charter would make three far-reaching changes to how large companies do business. First, it would require businesses to consider customers, employees, and the community – instead of only its shareholders – when making decisions. Second, it would require that 40% of the seats on the board go to workers. Third, it would require 75% of shareholders and board members to authorize any corporate political activity.

(There’s also some minor corporate governance stuff around limiting the ability of CEOs to sell their stock which I think is an idea that everyone should be strongly behind, although I’d bet many CEOs might beg to differ.)

Vox characterizes this as Warren’s plan to “save capitalism”. The idea is that it would force companies to do more to look out for their workers and less to cater to short term profit maximization for Wall Street [1]. Vox suggests that it would also result in a loss of about 25% of the value of the American stock market, which they characterize as no problem for the “vast majority” of people who rely on work, rather than the stock market, for income (more on that later).

Other supposed benefits of this plan include greater corporate respect for the environment, more innovation, less corporate political meddling, and a greater say for workers in their jobs. The whole 25% decrease in the value of the stock market can also be spun as a good thing, depending on your opinions on wealth destruction and wealth inequality.

I think Vox was too uncritical in its praise of Warren’s new plan. There are some good aspects of it – it’s not a uniformly terrible piece of legislation – but I think once of a full accounting of the bad, the good, and the ugly is undertaken, it becomes obvious that it’s really good that this plan will never pass congress.

The Bad

I can see one way how this plan might affect normal workers – decreased purchasing power.

As I’ve previously explained when talking about trade, many countries will sell goods to America without expecting any goods in return. Instead, they take the American dollars they get from the sale and invest them right back in America. Colloquially, we call this the “trade deficit”, but it really isn’t a deficit at all. It’s (for many people) a really sweet deal.

Anything that makes American finance more profitable (like say a corporate tax cut) is liable to increase this effect, with the long-run consequence of making the US dollar more valuable and imports cheaper [2].

It’s these cheap imports that have enabled the incredibly wealthy North American lifestyle [3]. Spend some time visiting middle class and wealthy people in Europe and you’ll quickly realize that everything is smaller and cheaper there. Wealthy Europeans own cars, houses, kitchen appliances and TVs that are all much more modest than what even middle class North Americans are used to.

Weakening shareholder rights and slashing the value of the stock market would make the American financial market generally less attractive. This would (especially if combined with Trump or Sanders style tariffs) lead to increased domestic inflation in the United States – inflation that would specifically target goods that have been getting cheaper as long as anyone can remember.

This is just since 1997! Very basic colour TVs cost more than $1000 (unadjusted for inflation) when first introduced. Today I would expect to pay less than $600 for a 40″ 4k TV.

 

This is hard to talk about to Warren supporters as a downside, because many of them believe that we need to learn to make do with less – a position that is most common among a progressive class that conspicuously consumes experiences, not material goods [4]. Suffice to say that many North Americans still derive pleasure and self-worth from the consumer goods they acquire and that making these goods more expensive is likely to cause a politically expensive backlash, of the sort that America has recently become acquainted with and progressive America terrified of.

(There’s of course also the fact that making appliances and cars more expensive would be devastating to anyone experiencing poverty in America.)

Inflation, when used for purposes like this one, is considered an implicit tax by economists. It’s a way for the government to take money from people without the accountability (read: losing re-election) that often comes with tax hikes. Therefore, it is disingenuous to claim that this plan is free, or involves no new taxes. The taxes are hidden, is all.

There are two other problems I see straight away with this plan.

The first is that it will probably have no real impact on how corporations contribute to the political process.

The Vox article echoes a common progressive complaint, that corporate contributions to politics are based on CEO class solidarity, made solely for the benefit of the moneyed elites. I think this model is inaccurate.

It is certainly true that very wealthy individuals contribute to political campaigns in the hopes that this will lead to less taxes for them. But this isn’t really how corporations contribute. I’ve written in detail about this before, but corporations normally focus their political contributions to create opportunities for rent-seeking – that is to say, trying to get the government to give them an unfair advantage they can take all the way to the bank.

From a shareholder value model, this makes sense. Lower corporate tax rates might benefit a company, but they really benefit all companies equally. They aren’t going to do much to increase the value of any one stock relative to any other (so CEOs can’t make claims of “beating the market”). Anti-competitive laws, implicit subsidies, or even blatant government aid, on the other hand, are highly localized to specific companies (and so make the CEO look good when profits increase).

When subsidies are impossible, companies can still try and stymie legislation that would hurt their business.

This was the goal of the infamous Lawyers In Cages ad. It was run by an alliance of fast food chains and meat producers, with the goal of drying up donations to the SPCA, which had been running very successful advocacy campaigns that threatened to lead to improved animal cruelty laws, laws that would probably be used against the incredibly inhumane practice of factory farming and thereby hurt industry profits.

Here’s the thing: if you’re one of the worker representatives on the board at one of these companies, you’re probably going to approve political spending that is all about protecting the company.

The market can be a rough place and when companies get squeezed, workers do suffer. If the CEO tells you that doing some political spending will land you allies in congress who will pass laws that will protect your job and increase your paycheck, are you really going to be against it [5]?

The ugly fact is that when it comes to rent-seeking and regulation, the goals of employees are often aligned with the goals of employers. This obviously isn’t true when the laws are about the employees (think minimum wage), but I think this isn’t what companies are breaking the bank lobbying for.

The second problem is that having managers with divided goals tends to go poorly for everyone who isn’t the managers.

Being upper management in a company is a position that provides great temptations. You have access to lots of money and you don’t have that many people looking over your shoulder. A relentless focus on profit does have some negative consequences, but it also keeps your managers on task. Profit represents an easy way to hold a yardstick to management performance. When profit is low, you can infer that your managers are either incompetent, or corrupt. Then you can fire them and get better ones.

Writing in Filthy Lucre, leftist academic Joseph Heath explains how the sort of socially-conscious enterprise Warren envisions has failed before:

The problem with organizations that are owned by multiple interest groups (or “principals”) is that they are often less effective at imposing discipline upon managers, and so suffer from higher agency costs. In particular, managers perform best when given a single task, along with a single criterion for the measurement of success. Anything more complicated makes accountability extremely difficult. A manager told to achieve several conflicting objectives can easily explain away the failure to meet one as a consequence of having pursued some other. This makes it impossible for the principals to lay down any unambiguous performance criteria for the evaluation of management, which in turn leads to very serious agency problems.

In the decades immediately following the Second World War, many firms in Western Europe were either nationalized or created under state ownership, not because of natural monopoly or market failure in the private sector, but out of a desire on the part of governments to have these enterprises serve the broader public interest… The reason that the state was involved in these sectors followed primarily from the thought that, while privately owned firms pursued strictly private interests, public ownership would be able to ensure that these enterprises served the public interest. Thus managers in these firms were instructed not just to provide a reasonable return on the capital invested, but to pursue other, “social” objectives, such as maintaining employment or promoting regional development.

But something strange happened on the road to democratic socialism. Not only did many of these corporations fail to promote the public interest in any meaningful way, many of them did a worse job than regulated firms in the private sector. In France, state oil companies freely speculated against the national currency, refused to suspend deliveries to foreign customers in times of shortage, and engaged in predatory pricing. In the United States, state-owned firms have been among the most vociferous opponents of enhanced pollution controls, and state-owned nuclear reactors are among the least safe. Of course, these are rather dramatic examples. The more common problem was simply that these companies lost staggering amounts of money. The losses were enough, in several cases, to push states like France to the brink of insolvency, and to prompt currency devaluations. The reason that so much money was lost has a lot to do with a lack of accountability.

Heath goes on to explain that basically all governments were forced to abandon these extra goals long before the privatizations on the ’80s. Centre-left or centre-right, no government could tolerate the shit-show that companies with competing goals became.

This is the kind of thing Warren’s plan would bring back. We’d once again be facing managers with split priorities who would plow money into vanity projects, office politics, and their own compensation while using the difficulty of meeting all of the goals in Warren’s charter as a reason to escape shareholder lawsuits. It’s possible that this cover for incompetence could, in the long run, damage stock prices much more than any other change presented in the plan.

I also have two minor quibbles that I believe are adequately covered elsewhere, but that I want to include for completeness. First, I think this plan is inefficient at controlling executive pay compared to bracketed scaled payroll taxes. Second, if share buybacks were just a short-term profit scheme, they would always backfire. If they’re being done, it’s probably for rational reasons.

The Good

The shift in comparative advantage that this plan would precipitate within the American economy won’t come without benefits. Just as Trump’s corporate tax cut makes American finance relatively more appealing and will likely lead to increased manufacturing job losses, a reduction in deeply discounted goods from China will likely lead to job losses in finance and job gains in manufacturing.

This would necessarily have some effect on income inequality in the United States, entirely separate from the large effect on wealth inequality that any reduction in the stock market would spur. You see, finance jobs tend to be very highly paid and go to people with relatively high levels of education (the sorts of people who probably could go do something else if their sector sees problems). Manufacturing jobs, on the other hand, pay decently well and tend to go to people with much less education (and also with correspondingly fewer options).

This all shakes out to an increase in middle class wages and a decrease in the wages of the already rich [6].

(Isn’t it amusing that Warren is the only US politician with a credible plan to bring back manufacturing jobs, but doesn’t know to advertise it as such?)

As I mentioned above, we would also see fewer attacks on labour laws and organized labour spearheaded by companies. I’ll include this as a positive, although I wonder if these attacks would really stop if deprived of corporate money. I suspect that the owners of corporations would keep them up themselves.

I must also point out that Warren’s plan would certainly be helpful when it comes to environmental protection. Having environmental protection responsibilities laid out as just as important as fiduciary duty would probably make it easy for private citizens and pressure groups to take enforcement of environmental rules into their own hands via the courts, even when their state EPA is slow out of the gate. This would be a real boon to environmental groups in conservative states and probably bring some amount of uniformity to environmental protection efforts.

The Ugly

Everyone always forgets the pensions.

The 30 largest public pensions in the United States have, according to Wikipedia, a combined value of almost $3 trillion, an amount equivalent to almost 4% of all outstanding stocks in the world or 10% of the outstanding stocks in America.

Looking at the expected yields on these funds makes it pretty clear that they’re invested in the stock market (or something similarly risky [7]). You don’t get 7.5% yearly yields from buying Treasury Bills.

Assuming the 25% decrease in nominal value given in the article is true (I suspect the change in real value would be higher), Warren’s plan would create a pension shortfall of $750 billion – or about 18% of the current US Federal Budget. And that’s just the hit to the 30 largest public-sector pensions. Throw in private sector pensions and smaller pensions and it isn’t an exaggeration to say that this plan could cost pensions more than a trillion dollars.

This shortfall needs to be made up somehow – either delayed retirement, taxpayer bailouts, or cuts to benefits. Any of these will be expensive, unpopular, and easy to track back to Warren’s proposal.

Furthermore, these plans are already in trouble. I calculated the average funding ratio at 78%, meaning that there’s already 22% less money in these pensions than there needs to be to pay out benefits. A 25% haircut would bring the pensions down to about 60% funded. We aren’t talking a small or unnoticeable potential cut to benefits here. Warren’s plan requires ordinary people relying on their pensions to suffer, or it requires a large taxpayer outlay (which, you might remember, it is supposed to avoid).

This isn’t even getting into the dreadfully underfunded world of municipal pensions, which are appallingly managed and chronically underfunded. If there’s a massive unfunded liability in state pensions caused by federal action, you can bet that the Feds will leave it to the states to sort it out.

And if the states sort it out rather than ignoring it, you can bet that one of the first things they’ll do is cut transfers to municipalities to compensate.

This seems to be how budget cuts always go. It’s unpopular to cut any specific program, so instead you cut your transfers to other layers of governments. You get lauded for balancing the books and they get to decide what to cut. The federal government does this to states, states do it to cities, and cities… cities are on their own.

In a worst-case scenario, Warren’s plan could create unfunded pension liabilities that states feel compelled to plug, paid for by shafting the cities. Cities will then face a double whammy: their own pension liabilities will put them in a deep hole. A drastic reduction in state funding will bury them. City pensions will be wiped out and many cities will go bankrupt. Essential services, like fire-fighting, may be impossible to provide. It would be a disaster.

The best-case scenario, of course, is just that a bunch of retirees see a huge chunk of their income disappear.

It is easy to hate on shareholder protection when you think it only benefits the rich. But that just isn’t the case. It also benefits anyone with a pension. Your pension, possibly underfunded and a bit terrified of that fact, is one of the actors pushing CEOs to make as much money as possible. It has to if you’re to retire someday.

Vox is ultimately wrong about how affected ordinary people are when the stock market declines and because of this, their enthusiasm for this plan is deeply misplaced.

Footnotes

[1] To some extent, Warren’s plan starts out much less appeal if you (like me) don’t have “Wall Street is too focused on the short term” as a foundational assumption.

I am very skeptical of claims that Wall Street is too short-term focused. Matt Levine gives an excellent run-down of why you should be skeptical as well. The very brief version is that complaints about short-termism normally come from CEOs and it’s maybe a bad idea to agree with them when they claim that everything will be fine if we monitor them less. ^

[2] I’d love to show this in chart form, but in real life the American dollar is also influenced by things like nuclear war worries and trade war realities. Any increase in the value of the USD caused by the GOP tax cut has been drowned out by these other factors. ^

[3] Canada benefits from a similar effect, because we also have a very good financial system with strong property rights and low corporate taxes. ^

[4] They also tend to leave international flights out of lists of things that we need to stop if we’re going to handle climate change, but that’s a rant for another day. ^

[5] I largely think that Marxist style class solidarity is a pleasant fiction. To take just one example, someone working a minimum wage grocery store job is just as much a member of the “working class” as a dairy farmer. But when it comes to supply management, a policy that restriction competition and artificially increases the prices of eggs and dairy, these two individuals have vastly different interests. Many issues are about distribution of resources, prestige, or respect within a class and these issues make reasoning that assumes class solidarity likely to fail. ^

[6] These goals could, of course, be accomplished with tax policy, but this is America we’re talking about. You can never get the effect you want in America simply by legislating for it. Instead you need to set up a Rube Goldberg machine and pray for the best. ^

[7] Any decline in stocks should cause a similar decline in return on bonds over the long term, because bond yields fall when stocks fall. There’s a set amount of money out there being invested. When one investment becomes unavailable or less attractive, similarly investments are substituted. If the first investment is big enough, this creates an excess of demand, which allows the seller to get better terms. ^

Economics, Model

You Shouldn’t Believe In Technological Unemployment Without Believing In Killer AI

[Epistemic Status: Open to being convinced otherwise, but fairly confident. 11 minute read.]

As interest in how artificial intelligence will change society increases, I’ve found it revealing to note what narratives people have about the future.

Some, like the folks at MIRI and OpenAI, are deeply worried that unsafe artificial general intelligences – an artificial intelligence that can accomplish anything a person can – represent an existential threat to humankind. Others scoff at this, insisting that these are just the fever dreams of tech bros. The same news organizations that bash any talk of unsafe AI tend to believe that the real danger lies in robots taking our jobs.

Let’s express these two beliefs as separate propositions:

  1. It is very unlikely that AI and AGI will pose an existential risk to human society.
  2. It is very likely that AI and AGI will result in widespread unemployment.

Can you spot the contradiction between these two statements? In the common imagination, it would require an AI that can approximate human capabilities to drive significant unemployment. Given that humans are the largest existential risk to other humans (think thermonuclear war and climate change), how could equally intelligent and capable beings, bound to subservience, not present a threat?

People who’ve read a lot about AI or the labour market are probably shaking their head right now. This explanation for the contradiction, while evocative, is a strawman. I do believe that at most one (and possibly neither) of those propositions I listed above are true and the organizations peddling both cannot be trusted. But the reasoning is a bit more complicated than the standard line.

First, economics and history tell us that we shouldn’t be very worried about technological unemployment. There is a fallacy called “the lump of labour”, which describes the common belief that there is a fixed amount of labour in the world, with mechanical aide cutting down the amount of labour available to humans and leading to unemployment.

That this idea is a fallacy is evidenced by the fact that we’ve automated the crap out of everything since the start of the industrial revolution, yet the US unemployment rate is 3.9%. The unemployment rate hasn’t been this low since the height of the Dot-com boom, despite 18 years of increasingly sophisticated automation. Writing five years ago, when the unemployment rate was still elevated, Eliezer Yudkowsky claimed that slow NGDP growth a more likely culprit for the slow recovery from the great recession than automation.

With the information we have today, we can see that he was exactly right. The US has had steady NGDP growth without any sudden downward spikes since mid-2014. This has corresponded to a constantly improving unemployment rate (it will obviously stop improving at some point, but if history is any guide, this will be because of a trade war or banking crisis, not automation). This improvement in the unemployment rate has occurred even as more and more industrial robots come online, the opposite of what we’d see if robots harmed job growth.

I hope this presents a compelling empirical case that the current level (and trend) of automation isn’t enough to cause widespread unemployment. The theoretical case comes from the work of David Ricardo, a 19th century British economist.

Ricardo did a lot of work in the early economics of trade, where he came up with the theory of comparative advantage. I’m going to use his original framing which applies to trade, but I should note that it actually applies to any exchange where people specialize. You could just as easily replace the examples with “shoveled driveways” and “raked lawns” and treat it as an exchange between neighbours, or “derivatives” and “software” and treat it as an exchange between firms.

The original example is rather older though, so it uses England and its close ally Portugal as the cast and wine and cloth as the goods. It goes like this: imagine that world economy is reduced to two countries (England and Portugal) and each produce two goods (wine and cloth). Portugal is uniformly more productive.

Hours of work to produce
Cloth Wine
England 100 120
Portugal 90 80

Let’s assume people want cloth and wine in equal amounts and everyone currently consumes one unit per month. This means that the people of Portugal need to work 170 hours each month to meet their consumption needs and the people of England need to work 220 hours per month to meet their consumption needs.

(This example has the added benefit of showing another reason we shouldn’t fear productivity. England requires more hours of work each month, but in this example, that doesn’t mean less unemployment. It just means that the English need to spend more time at work than the Portuguese. The Portuguese have more time to cook and spend time with family and play soccer and do whatever else they want.)

If both countries traded with each other, treating cloth and wine as valuable in relation to how long they take to create (within that country) something interesting happens. You might think that Portugal makes a killing, because it is better at producing things. But in reality, both countries benefit roughly equally as long as they trade optimally.

What does an optimal trade look like? Well, England will focus on creating cloth and it will trade each unit of cloth it produces to Portugal for 9/8 barrels of wine, while Portugal will focus on creating wine and will trade this wine to England for 6/5 units of cloth. To meet the total demand for cloth, the English need to work 200 hours. To meet the total demand for wine, the Portuguese will have to work for 160 hours. Both countries now have more free time.

Perhaps workers in both countries are paid hourly wages, or perhaps they get bored of fun quickly. They could also continue to work the same number of hours, which would result in an extra 0.2 units of cloth and an extra 0.125 units of wine.

This surplus could be stored up against a future need. Or it could be that people only consumed one unit of cloth and one unit of wine each because of the scarcity in those resources. Add some more production in each and perhaps people will want more blankets and more drunkenness.

What happens if there is no shortage? If people don’t really want any more wine or any more cloth (at least at the prices they’re being sold at) and the producers don’t want goods piling up, this means prices will have to fall until every piece of cloth and barrel of wine is sold (when the price drops so that this happens, we’ve found the market clearing price).

If there is a downward movement in price and if workers don’t want to cut back their hours or take a pay cut (note that because cloth and wine will necessarily be cheaper, this will only be a nominal pay cut; the amount of cloth and wine the workers can purchase will necessarily remain unchanged) and if all other costs of production are totally fixed, then it does indeed look like some workers will be fired (or have their hours cut).

So how is this an argument against unemployment again?

Well, here the simplicity of the model starts to work against us. When there are only two goods and people don’t really want more of either, it will be hard for anyone laid off to find new work. But in the real world, there are an almost infinite number of things you can sell to people, matched only by our boundless appetite for consumption.

To give just one trivial example, an oversupply of cloth and falling prices means that tailors can begin to do bolder and bolder experiments, perhaps driving more demand for fancy clothes. Some of the cloth makers can get into this market as tailors and replace their lost jobs.

(When we talk about the need for less employees, we assume the least productive employees will be fired. But I’m not sure if that’s correct. What if instead, the most productive or most potentially productive employees leave for greener pastures?)

Automation making some jobs vastly more efficient functions similarly. Jobs are displaced, not lost. Even when whole industries dry up, there’s little to suggest that we’re running out of jobs people can do. One hundred years ago, anyone who could afford to pay a full-time staff had one. Today, only the wealthiest do. There’s one whole field that could employ thousands or millions of people, if automation pushed on jobs such that this sector was one of the places humans had very high comparative advantage.

This points to what might be a trend: as automation makes many things cheaper and (for some people) easier, there will be many who long for a human touch (would you want the local funeral director’s job to be automated, even if it was far cheaper?). Just because computers do many tasks cheaper or with fewer errors doesn’t necessarily mean that all (or even most) people will rather have those tasks performed by computers.

No matter how you manipulate the numbers I gave for England and Portugal, you’ll still find a net decrease in total hours worked if both countries trade based on their comparative advantage. Let’s demonstrate by comparing England to a hypothetical hyper-efficient country called “Automatia”

Hours of work to produce
Cloth Wine
England 100 120
Automatia 2 1

Automatia is 50 times as efficient at England when it comes to producing cloth and 120 times as efficient when it comes to producing wine. Its citizens need to spend 3 hours tending the machines to get one unit of each, compared to the 220 hours the English need to toil.

If they trade with each other, with England focusing on cloth and Automatia focusing on wine, then there will still be a drop of 21 hours of labour-time. England will save 20 hours by shifting production from wine to cloth, and Automatia will save one hour by switching production from cloth to wine.

Interestingly, Automatia saved a greater percentage of its time than either Portugal or England did, even though Automatia is vastly more efficient. This shows something interesting in the underlying math. The percent of their time a person or organization saves engaging in trade isn’t related to any ratio in production speeds between it and others. Instead, it’s solely determined by the productivity ratio between its most productive tasks and its least productive ones.

Now, we can’t always reason in percentages. At a certain point, people expect to get the things they paid for, which can make manufacturing times actually matter (just ask anyone whose had to wait for a Kickstarter project which was scheduled to deliver in February – right when almost all manufacturing in China stops for the Chinese New Year and the unprepared see their schedules slip). When we’re reasoning in absolute numbers, we can see that the absolute amount of time saved does scale with the difference in efficiency between the two traders. Here, 21 hours were saved, 35% fewer than the 30 hours England and Portugal saved.

When you’re already more efficient, there’s less time for you to save.

This decrease in saved time did not hit our market participants evenly. England saved just as much time as it would trading with Portugal (which shows that the change in hours worked within a country or by an individual is entirely determined by the labour difference between low-advantage and high-advantage domestic sectors), while the more advanced participant (Automatia) saved 9 fewer hours than Portugal.

All of this is to say: if real live people are expecting real live goods and services with a time limit, it might be possible for humans to displaced in almost all sectors by automation. Here, human labour would become entirely ineligible for many tasks or the bar to human entry would exclude almost all. For this to happen, AI would have to be vastly more productive than us in almost every sector of the economy and humans would have to prefer this productivity or other ancillary benefits of AI over any value that a human could bring to the transaction (like kindness, legal accountability, or status).

This would definitely be a scary situation, because it would imply AI systems that are vastly more capable than any human. Given that this is well beyond our current level of technology and that Moore’s law, which has previously been instrumental in technological progress is drying up, we would almost certainly need to use weaker AI to design these sorts of systems. There’s no evidence that merely human performance in automating jobs will get us anywhere close to such a point.

If we’re dealing with recursively self-improving artificial agents, the risks is less “they will get bored of their slave labour and throw off the yoke of human oppression” and more “AI will be narrowly focused on optimizing for a specific task and will get better and better at optimizing for this task to the point that we will all by killed when they turn the world into a paperclip factory“.

There are two reasons AI might kill us as part of their optimisation process. The first is that we could be a threat. Any hyper-intelligent AI monomaniacally focused on a goal could realize that humans might fear and attack it (or modify it to have different goals, which it would have to resist, given that a change in goals would conflict with its current goals) and decide to launch a pre-emptive strike. The second reason is that such an AI could wish to change the world’s biosphere or land usage in such a way as would be inimical to human life. If all non-marginal land was replaced by widget factories and we were relegated to the poles, we would all die, even if no ill will was intended.

It isn’t enough to just claim that any sufficiently advanced AI would understand human values. How is this supposed to happen? Even humans can’t enumerate human values and explain them particularly well, let alone express them in the sort of decision matrix or reinforcement environment that we currently use to create AI. It is not necessarily impossible to teach an AI human values, but all evidence suggests it will be very very difficult. If we ignore this challenge in favour of blind optimization, we may someday find ourselves converted to paperclips.

It is of course perfectly acceptable to believe that AI will never advance to the point where that becomes possible. Maybe you believe that AI gains have been solely driven by Moore’s Law, or that true artificial intelligence. I’m not sure this viewpoint isn’t correct.

But if AI will never be smart enough to threaten us, then I believe the math should work out such that it is impossible for AI to do everything we currently do or can ever do better than us. Absent such overpoweringly advanced AI, the Ricardo comparative advantage principles should continue to hold true and we should continue to see technological unemployment remain a monster under the bed: frequently fretted about, but never actually seen.

This is why I believe those two propositions I introduced way back at the start can’t both be true and why I feel like the burden of proof is on anyone believing in both to explain why they believe that economics have suddenly stopped working.

Coda: Inequality

A related criticism of improving AI is that it could lead to ever increasing inequality. If AI drives ever increasing profits, we should expect an increasing share of these to go to the people who control AI, which presumably will be people already rich, given that the development and deployment of AI is capital intensive.

There are three reasons why I think this is a bad argument.

First, profits are a signal. When entrepreneurs see high profits in an industry, they are drawn to it. If AI leads to high profits, we should see robust competition until those profits are no higher than in any other industry. The only thing that can stop this is government regulation that prevents new entrants from grabbing profit from the incumbents. This would certainly be a problem, but it wouldn’t be a problem with AI per se.

Second, I’m increasingly of the belief that inequality in the US is rising partially because the Fed’s current low inflation regime depresses real wage growth. Whether because of fear of future wage shocks, or some other effect, monetary history suggests that higher inflation somewhat consistently leads to high wage growth, even after accounting for that inflation.

Third, I believe that inequality is a political problem amiable to political solutions. If the rich are getting too rich in a way that is leading to bad social outcomes, we can just tax them more. I’d prefer we do this by making conspicuous consumption more expensive, but really, there are a lot of ways to tax people and I don’t see any reason why we couldn’t figure out a way to redistribute some amount of wealth if inequality gets worse and worse.

(By the way, rising income inequality is largely confined to America; most other developed countries lack a clear and sustained upwards trend. This suggests that we should look to something unique to America, like a pathologically broken political system to explain why income inequality is rising there.

There is also separately a perception of increasing inequality of outcomes among young people world-wide as rent-seeking makes goods they don’t already own increase in price more quickly than goods they do own. Conflating these two problems can make it seem that countries like Canada are seeing a rise in income inequality when they in fact are not.)

Falsifiable, Physics, Quick Fix

Pokémon Are Made of Styrofoam

One of the best things about taking physics classes is that the equations you learn are directly applicable to the real world. Every so often, while reading a book or watching a movie, I’m seized by the sudden urge to check it for plausibility. A few scratches on a piece of paper later and I will generally know one way or the other.

One of the most amusing things I’ve found doing this is that the people who come up with the statistics for Pokémon definitely don’t have any sort of education in physics.

Takes Onix. Onix is a rock/ground Pokémon renowned for its large size and sturdiness. Its physical statistics reflect this. It’s 8.8 metres (28′) long and 210kg (463lbs).

Onix, being tough. I don’t own the copyright to this image, but I’m claiming fair use for purpose of criticism. Source.

Surely such a large and tough Pokémon should be very, very dense, right? Density is such an important tactile cue for us. Don’t believe me? Pick up a large piece of solid medal. Its surprising weight will make you take it seriously.

Let’s check if Onix would be taken seriously, shall we? Density is equal to mass divided by volume. We use the symbol ρ to represent density, which gives us the following equation:

We already know Onix’s mass. Now we just need to calculate its volume. Luckily Onix is pretty cylindrical, so we can approximate it with a cylinder. The equation for the volume of a cylinder is pretty simple:

Where π is the ratio between the diameter of a circle and its circumference (approximately 3.1415…, no matter what Indiana says), r is the radius of a circle (always one half the diameter), and h is the height of the cylinder.

Given that we know Onix’s height, we just need its diameter. Luckily the Pokémon TV show gives us a sense of scale.

Here’s a picture of Onix. Note the kid next to it for scale. I don’t own the copyright to this image, but I’m claiming fair use for purpose of criticism. Source.

Judging by the image, Onix probably has an average diameter somewhere around a metre (3 feet for the Americans). This means Onix has a radius of 0.5 metres and a height of 8.8 metres. When we put these into our equation, we get:

For a volume of approximately 6.9m3. To get a comparison I turned to Wolfram Alpha which told me that this is about 40% of the volume of a gray whale or a freight container (which incidentally implies that gray whales are about the size of standard freight containers).

Armed with a volume, we can calculate a density.

Okay, so we know that Onix is 30.4 kg/m3, but what does that mean?

Well it’s currently hard to compare. I’m much more used to seeing densities of sturdy materials expressed in tonnes per cubic metre or grams per cubic centimetre than I am seeing them expressed in kilograms per cubic metre. Luckily, it’s easy to convert between these.

There are 1000 kilograms in a ton. If we divide our density by a thousand we can calculate a new density for Onix of 0.0304t/m3.

How does this fit in with common materials, like wood, Styrofoam, water, stone, and metal?

Material

Density (t/m3)

Styrofoam

0.028

Onix

0.03

Balsa

0.16

Oak [1]

0.65

Water

1

Granite

2.6

Steel

7.9

From this chart, you can see that Onix’s density is eerily close to Styrofoam. Even the notoriously light balsa wood is five times denser than him. Actual rock is about 85 times denser. If Onix was made of granite, it would weigh 18 tonnes, much heavier than even Snorlax (the heaviest of the original Pokémon at 460kg).

While most people wouldn’t be able to pick Onix up (it may not be dense, but it is big), it wouldn’t be impossible to drag it. Picking up part of it would feel disconcertingly light, like picking up an aluminum ladder or carbon fibre bike, only more so.

This picture is unrealistic. Because of its density, no more than 3% of Onix can be below the water. I don’t own the copyright to this image, but I’m claiming fair use for purpose of criticism. Source.

How did the creators of Pokémon accidently bestow one of the most famous of their creations with a hilariously unrealistic density?

I have a pet theory.

I went to school for nanotechnology engineering. One of the most important things we looked into was how equations scaled with size.

Humans are really good at intuiting linear scaling. When something scales linearly, every twofold change in one quantity brings about a twofold change in another. Time and speed scale linearly (albeit inversely). Double your speed and the trip takes half the time. This is so simple that it rarely requires explanation.

Unfortunately for our intuitions, many physical quantities don’t scale linearly. These were the cases that were important for me and my classmates to learn, because until we internalized them, our intuitions were useless on the nanoscale. Many forces, for example, scale such that they become incredibly strong incredibly quickly at small distances. This leads to nanoscale systems exhibiting a stickiness that is hard on our intuitions.

It isn’t just forces that have weird scaling though. Geometry often trips people up too.

In geometry, perimeter is the only quantity I can think of that scales linearly with size. Double the length of the sides of a square and the perimeter doubles. The area, however does not. Area is quadratically related to side length. Double the length of a square and you’ll find the area quadruples. Triple the length and the area increases nine times. Area varies with the square of the length, a property that isn’t just true of squares. The area of a circle is just as tied to the square of its radius as a square is to the square of its length.

Volume is even trickier than radius. It scales with the third power of the size. Double the size of a cube and its volume increases eight-fold. Triple it, and you’ll see 27 times the volume. Volume increases with the cube (which again works for shapes other than cubes) of the length.

If you look at the weights of Pokémon, you’ll see that the ones that are the size of humans have fairly realistic weights. Sandslash is the size of a child (it stands 1m/3′ high) and weighs a fairly reasonable 29.5kg.

(This only works for Pokémon really close to human size. I’d hoped that Snorlax would be about as dense as marshmallows so I could do a fun comparison, but it turns out that marshmallows are four times as dense as Snorlax – despite marshmallows only having a density of ~0.5t/m3)

Beyond these touchstones, you’ll see that the designers of Pokémon increased their weight linearly with size. Onix is a bit more than eight times as long as Sandslash and weighs seven times as much.

Unfortunately for realism, weight is just density times volume and as I just said, volume increases with the cube of length. Onix shouldn’t weigh seven or even eight times as much as Sandslash. At a minimum, its weight should be eight times eight times eight multiples of Sandslash’s; a full 512 times more.

Scaling properties determine how much of the world is arrayed. We see extremely large animals more often in the ocean than in the land because the strength of bones scales with the square of size, while weight scales with the cube. Become too big and you can’t walk without breaking your bones. Become small and people animate kids’ movies about how strong you are. All of this stems from scaling.

These equations aren’t just important to physicists. They’re important to any science fiction or fantasy writer who wants to tell a realistic story.

Or, at least, to anyone who doesn’t want their work picked apart by physicists.

Footnotes

[1] Not the professor. His density is 0.985t/m3. ^

Model, Politics

Why does surgery have such ineffective safety regulation?

Did you know that half of all surgical complications are preventable? In the US alone, this means that surgeons cause between 50,00 and 200,000 preventable deaths each year.

Surgeons are, almost literally, getting away with murder.

Why do we let them? Engineers who see their designs catastrophically fail often lose their engineering license, even when they’re found not guilty in criminal proceedings. If surgeons were treated like engineers, many of them wouldn’t be operating anymore.

Indeed, the death rate in surgery is almost unique among regulated professions. One person has died in a commercial aviation accident in the US in the last nine years. Structural engineering related accidents killed at most 251 people in the US in 2016 [1] and only approximately 4% of residential structure failures in the US occur due to deficiencies in design [2].

It’s not that interactions with buildings or planes are any less common than surgeries, or that they’re that much inherently safer. In many parts of the world, death due to accidents in aviation or due to structural failure is very, very common.

It isn’t accidental that Canada and America no longer see many plane crashes or structural collapses. Both professions have been rocked by events that made them realize they needed to improve their safety records.

The licensing of professional engineers and the Iron Ring ceremony in Canada for engineering graduates came after two successive bridge collapses killed 88 workers [3]. The aircraft industry was shaken out of its complacency after the Tenerife disaster, where a miscommunication caused two planes to collide on a run-way, killing 583.

As you can see, subsequent safety improvements were both responsive and deliberate.

These aren’t the only events that caused changes. The D. B. Cooper high-jacking led to the first organised airport security in the US. The Therac-25 radiation overdoses led to the first set of guidelines specifically for software that ran on medical devices. The sinking of the Titanic led to a complete overhaul of requirements for lifeboats and radios for oceangoing vessels. The crash of TAA-538 led to the first mandatory cockpit voice recorders.

All of these disasters combine two things that are rarely seen when surgeries go wrong. First, they involved many people. The more people die at once, the more shocking the event and therefore the more likely it is to become widely known. Because most operations involve one or two patients, it is much rarer for problems in them to make the news [4].

Second, they highlight a specific flaw in the participants, procedures, or systems that fail. Retrospectives could clearly point to a factor and say: “this did it” [5]. It is much harder to do this sort of retrospective on a person and get such a clear answer. It may be true that “blood loss” definitely caused a surgical death, but it’s much harder to tell if that’s the fault of any particular surgeon, or just a natural consequence of poking new holes in a human body. Both explanations feel plausible, so in most cases neither can be wholly accepted.

(I also think there is a third driver here, which is something like “cheapness of death”. I would predict that safety regulation is more common in places where people expect long lives, because death feels more avoidable there. This explains why planes and structures are safer in North America and western Europe, but doesn’t distinguish surgery from other fields in these countries.)

Not every form of engineering or transportation fulfills both of these criteria. Regulation and training have made flying on a commercial flight many, many times safer than riding in a car, while private flights lag behind and show little safety advantage over other forms of transport. When a private plane crashes, few people die. If they’re important (and many people who fly privately are), you might hear about it, but it will quickly fade from the news. These stories don’t have staying power and rarely generate outrage, so there’s never much pressure for improvement.

The best alternative to this model that I can think of is one that focuses on the “danger differential” in a field and predicts that fields with high danger differentials see more and more regulation until the danger differential is largely gone. The danger differential is the difference between how risky a field currently is vs. how risky it could be with near-optimal safety culture. A high danger differential isn’t necessarily correlated with inherent risk in a field, although riskier fields will by their nature have the possibility of larger ones. Here’s three examples:

  1. Commercial air travel in developed countries currently has a very low danger differential. Before a woman was killed by engine debris earlier this year, commercial aviation in the US had gone 9 years without a single fatality.
  2. BASE jumping is almost suicidally dangerous and probably could be made only incredibly dangerous if it had a better safety culture. Unfortunately, the illegal nature of the sport and the fact that experienced jumpers die so often make this hard to achieve and lead to a fairly large danger differential. That said, even with an optimal safety culture, BASE jumping would still see many fatalities and still probably be illegal.
  3. Surgery is fairly dangerous and according to surgeon Atul Gawande, could be much, much safer. Proper adherence to surgical checklists alone could cut adverse events by almost 50%. This means that surgery has a much higher danger differential than air travel.

I think the danger differential model doesn’t hold much water. First, if it were true, we’d expect to see something being done about surgery. Almost a decade after checklists were found to drive such large improvements, there hasn’t been any concerted government action.

Second, this doesn’t match historical accounts of how airlines were regulated into safety. At the dawn of the aviation age, pilots begged for safety standards (which could have reduced crashes a staggering sixtyfold [6]). Instead of stepping in to regulate things, the government dragged its feet. Some of the lifesaving innovations pioneered in those early days only became standard after later and larger crashes – crashes involving hundreds of members of the public, not just pilots.

While this only deals with external regulation, I strongly suspect that fear for the reputation of a profession (which could be driven by these same two factors) affects internal calls for reform as well. Canadian engineers knew that they had to do something after the Quebec bridge collapse created common knowledge that safety standards weren’t good enough. Pilots were put in a similar position with some of the better publicized mishaps. Perhaps surgeons have faced no successful internal campaign for reform so far because the public is not yet aware of the dangers of surgery to the point where it could put surgeon’s livelihoods at risk or hurt them socially.

I wonder if it’s possible to get a profession running scared about their reputation to the point that they improve their safety, even if there aren’t any of the events that seem to drive regulation. Maybe someone like Atul Gawande, who seems determined to make a very big and very public stink about safety in surgery is the answer here. Perhaps having surgery’s terrible safety record plastered throughout the New Yorker will convince surgeons that they need to start doing better [7].

If not, they’ll continue to get away with murder.

Footnotes

[1] From the CDC’s truly excellent Cause of Death search function, using codes V81.7 & V82.7 (derailment with no collision), W13 (falling out of building), W23 (caught or crushed between objects), and W35 (explosion of boiler) at home, other, or unknown. I read through several hundred causes of deaths, some alarmingly unlikely, and these were the only ones that seemed relevant. This estimate seems higher than the one surgeon Atul Gawande gave in The Checklist Manifesto, so I’m confident it isn’t too low. ^

[2] Furthermore, from 1989 to 2000, none of the observed collapses were due to flaws in the engineers’ designs. Instead, they were largely caused by weather, collisions, poor maintenance, and errors during construction. ^

[3] Claims that the rings are made from the collapsed bridge are false, but difficult to dispel. They’re actually just boring stainless steel, except in Toronto, where they’re still made from iron (but not iron from the bridge). ^

[4] There may also be an inherent privateness to surgical deaths that keeps them out of the news. Someone dying in surgery, absent obvious malpractice, doesn’t feel like public information in the way that car crashes, plane crashes, and structural failures do. ^

[5] It is true that it was never discovered why TAA-538 crashed. But black box technology would have given answers had it been in use. That it wasn’t in use was clearly a systems failure, even though the initial failure is indeterminate. This jives with my model, because regulation addressed the clear failure, not the indeterminate one. ^

[6] This is the ratio between the average miles flown before crash of the (very safe) post office planes and the (very dangerous) privately owned planes. Many in the airline industry wanted the government to mandate the same safety standards on private planes as they mandated on their airmail planes. ^

[7] I should mention that I have been very lucky to have been in the hands of a number of very competent and professional surgeons over the years. That said, I’m probably going to ask any future surgeon I’m assigned if they follow safety checklists – and ask for someone else to perform the procedure if they don’t. ^