Friday, March 20, 2009

Little Watchmen

Little Brother is the novel by Cory Doctorow about individuals using technology to turn the tables on Big Brother. It was an enjoyable read, but I think the enjoyment will have much to do with your political views. If you believe in putting your trust in government (like the mode was for at least 6 of the 8 years of the Bush Presidency) you probably won't like it. If you prefer free speech and freedom from government control, and big brother style spying by government you probably will enjoy it. It also has a dose of techno-geek culture.

Corey Doctorow understand the new possibilities technology allows. The content of the book is also available for free online

Giving away ebooks gives me artistic, moral and commercial satisfaction. The commercial question is the one that comes up most often: how can you give away free ebooks and still make money?

For me -- for pretty much every writer -- the big problem isn't piracy, it's obscurity (thanks to Tim O'Reilly for this great aphorism). Of all the people who failed to buy this book today, the majority did so because they never heard of it, not because someone gave them a free copy.

A particularly nice little lesson on the paradox of false positives from the book:

If you ever decide to do something as stupid as build an automatic terrorism detector, here's a math lesson you need to learn first. It's called "the paradox of the false positive," and it's a doozy.

Say you have a new disease, called Super-AIDS. Only one in a million people gets Super-AIDS. You develop a test for Super-AIDS that's 99 percent accurate. I mean, 99 percent of the time, it gives the correct result -- true if the subject is infected, and false if the subject is healthy. You give the test to a million people.

One in a million people have Super-AIDS. One in a hundred people that you test will generate a "false positive" -- the test will say he has Super-AIDS even though he doesn't. That's what "99 percent accurate" means: one percent wrong.

What's one percent of one million?

1,000,000/100 = 10,000

One in a million people has Super-AIDS. If you test a million random people, you'll probably only find one case of real Super-AIDS. But your test won't identify *one* person as having Super-AIDS. It will identify *10,000* people as having it.

Your 99 percent accurate test will perform with 99.99 percent *inaccuracy*.

That's the paradox of the false positive. When you try to find something really rare, your test's accuracy has to match the rarity of the thing you're looking for.
This is the paradox of the false positive, and here's how it applies to terrorism:

Terrorists are really rare. In a city of twenty million like New York, there might be one or two terrorists. Maybe ten of them at the outside. 10/20,000,000 = 0.00005 percent. One twenty-thousandth of a percent.

That's pretty rare all right. Now, say you've got some software that can sift through all the bank-records, or toll-pass records, or public transit records, or phone-call records in the city and catch terrorists 99 percent of the time.

In a pool of twenty million people, a 99 percent accurate test will identify two hundred thousand people as being terrorists. But only ten of them are terrorists. To catch ten bad guys, you have to haul in and investigate two hundred thousand innocent people.

Guess what? Terrorism tests aren't anywhere *close* to 99 percent accurate. More like 60 percent accurate. Even 40 percent accurate, sometimes.

What this all meant was that the Department of Homeland Security had set itself up to fail badly. They were trying to spot incredibly rare events -- a person is a terrorist -- with inaccurate systems

"I'd recommend Little Brother over pretty much any book I've read this year." - Neil Gaiman

I wouldn't agree with that but it is a fun read, though with quite a strong worldview that some will find overwhelms the story. I myself and tired of tired of incompetent government harassment, so that wasn't a problem for me.

Related: Watching the Watchmen - Freedom Increasingly at Risk - Systemic Failure of SWAT Raids - Photographers are not a Threat

No comments: