blog - steve t. piantadosi

A few days ago when I was riding home on the T with John Kraemer, we ran into his friend, Brian Olson, a guy from google who has some neat political-computational ideas. Pretty quickly on the ride home, he sold me on the idea of fair redistricting, which is meant to provide an impartial way to solve problems of gerrymandering (Hilariously, wikipedia tells me that this is a portmanteau of “Gerry” and “Salamander,” the former being the governor of MA and the latter being the shape of the district chosen to help him).

The idea of fair redistricting is really clever: define an objective measure for the best districting scheme, and use that to define the districts, rather than allowing politicians to haggle over them. You can picture having some metric for how “good” a districting scheme is, and then setting them by law to be the best according to goodness. This is clever because it leaves no room to wiggle. I find it hard to argue with.

Brian Olson’s scheme is to pick a districting such that the average distance of people in the district to the center of the district is minimized. Of course, there are others, but that happens to be relatively straightforward to compute (though finding the best is hard). On his webpage, he provides a program (note: requires protobuf) for computing the best districting scheme for each state according to this metric. This scheme produces regions that look quite reasonable, for instance converting the gerrymandered FL districting on the left to the objectively “better” one on the right.

And not only is it objectively better, but it bears no hallmarks–weird shaped regions–of a gross political system.

There are of course a few additional things you want. You want there to be a unique solution (for some reason, I find this plausible when there are so many constraints.. more on that later maybe). You also want the solution to be findable–or at least able to be approximated. Here that doesn’t mean finding a district whose score is close to the best score. It means finding a map whose districting is close to the best one, which is a slightly different problem. A practical way around this search problem might be to have each party submit their preferred districting, and then choose the one with the highest score according to the objective function.

Anyway this looks like the right direction to move in: formalize our political intuitions of goodness, and go by them without letting the grimy politicians get in the way! Thanks Brian!

I was standing in airport security a while ago, wondering about how much all of this security cost society. Unfortunately, this is a hard thing to quantify: how much is our time worth? How much does it cost us to stand a few minutes in an airport security line while you fly? I could imagine quantifying it in terms of lost wages or the cost to pay TSA workers, but what is best?

One way that’s good is to figure out how much time is lost per year–how many hours does it take away from American’s lives every year? The TSA reports that there were 708,400,522 people screened in 2006, with an average wait time of 3.79 minutes. This comes out to 5108 people-years of lost time due to airport security screenings. Or, if you assume that people live to be 80 years old, airport security costs an equivalent of 63 person-lifetimes every year. 63 lifetimes. American lost 63 lives worth of time due to airport security. How many lives worth of time lost due to hijackings? None.

It’s worth thinking about what an optimal airport security policy would be. It should be clear that if security were too lax, we would be losing more people to terrorism than airport security wait lines. Conversely, if security were too tight, we would be losing more people due to security wait times. Picture a system in which you were screened for a few weeks before flying. This might totally eliminate terrorism on aircraft (but probably not), yet it would cost us a substantial amount of time. The way to determine if it is too much time is to see if the medicine costs more American lifetimes than the disease. Which, at 63-0, it seems to.

Said another way, what matters is the total number of lives lost–lives lost to security policies plus lives lost to terrorism. If no lives are lost to terrorism, then we could reduce the total number of lives lost by decreasing the security until the number of lives worth of lost time due to security is equal to the number lost due to terrorism. This increases the lives lost to terrorism, but decreases the total lives lost overall.

This means it is not rational to have a “zero tolerance” policy for airline terrorism. The best policy allows as many deaths, on average, from airline terrorism as it does from waiting in security lines. Unfortunately, it’s not clear how much to adjust it–maybe a small change would lead to a huge number of terrorist deaths.

Of course, people may have a different definition of “best” than the one here. After all, waiting in a security line is a guaranteed loss of 3.7 minutes, but potentially dying from terrorism is a small probability of a much bigger loss of minutes. Prospect theory tells us that people are risk averse for small probabilities of losses. So, people might rather pay the 3.7 minutes than have a very small chance of being killed by a terrorist. Even if that’s what people want, though, anything other than an equal number of deaths due to terrorism and security wait times costs society more human lifetimes.

What’s the point? The point is that security is not free. It is not free in terms of our rights and civil liberties. And it is demonstratably nonoptimal in terms of the number of American lifetimes worth of time it costs. The only thing worse than terrorism costing Americans part of their lives is the government doing it more so.

In 1991, Anderson and Schooler published a paper, “Reflections of the environment in memory” in which they attempted to provide a rational explanation for the laws of human memory. Human memory shows characteristic power law decays in a number of ways; for example, Ebbinghaus’ data shows a power law relationship between performance (as measured in percent savings when relearning a list) and time elapsed since learning.

Anderson and Schooler attempted to provide a rational explanation for why memory decays like a power law: many things in the environment also decay like power laws, and so human memory may have evolved its “forgetting” function in order to match the probaiblities with which things occur in the environment. Memory may representing what information is needed when. They showed that many naturally occuring phenomena scale like power laws, including the probability of a word appearing in a headline, the probability of words appearing in parental speech, and the probability of receiving an email message from someone.

I became curious about power laws and media reporting. For example, what is the decay law for media news stories: is there a power law relationship between the amount of time that’s passed since an event and the number of news stories that mention it? Does human memory match this pattern, and, maybe more interestingly, does the pattern exist because of how human memory works, or does is it the other way around?

I whipped up a quick perl script to count the number of stories google news finds for “September 11th” restricted to each of the months following September 11th 2001. These show a very clear power law decay.

Here, the red dots are Septembers and the black dots are any other month. The x-axis is number of days since September 11′th 2001, and the Y axis is the number of newspaper stories mentioning September 11′th. The power law decay is clear for Septembers and other months, and the slopes of each line are remarkably similar: -0.5628 for Septembers and -0.5773 for other months. This means that newpaper stories mentioning September 11′th are dying out at about the same rate for September, and other months. The intercept for Septembers is 13.28 and 12.12 for other months.

So media reportage of September 11th shows a power law decay. It would be fun to test other big events too (and small ones) but google banned my script since it requested webpages in quick succession, so I may leave it to someone else to write a program that can scan google news and look like a human.

By the way, using this law we can compute when a month should pass with only one newpaper story referencing September 11′th: 48,467,636 years for a September to pass only mentioning it once, and 3,530,904 years for a non-September month.