Resources

Links and references to extra readings will be posted here.

One

Here is a link to the Harvard Business Review issue on decision making (University of Chicago access required).

For more on decision analysis, see Robert Clemen (1997), Making Hard Decisions, perhaps the best decision analysis text book around.

For a review of decision analysis software (for the hardcore!), click here.

There are lots of websites with images of various visual illusions, including the one I presented in class. Here is one, but the best one is probably Roger Shepard's unbelievable table turning illusion.

There are many websites devoted to the Monty Hall problem. Here is the discussion from the Car Talk Guys. You can also check out the self-proclaimed, "Official Let's Make a Deal Website" here. Wikipedia also has an entry on the Monty Hall problem. Some of the comments sent to Parade are found here. If you want to know more about Marilyn vos Savant, click here.

For an excellent book on medical decision making, see Harold Bursztajn, Richard Feinbloom, Robert M. Hamm, Archie Brodsky (1990), Medical Choices, Medical Chances (Routledge). For an excellent (and readable) book on medical decision making, see Harold Bursztajn, Richard Feinbloom, Robert M. Hamm, Archie Brodsky (1990), Medical Choices, Medical Chances (Routledge). I also recommend that you read the book: Atul Gawande (2002), Complications (Metropolitan Books). The last chapter, "The Case of the Red Leg", is especially good.

My neurosurgeon friend is Jim Reese, Google employee number 19. Here is an excerpt from an article from The New Yorker about Google about his interesting career change

I had lunch one day with a few of the company's researchers, including Jim Reese, who told me that he was employee No. 19. His business card describes him as chief operations engineer and head neurosurgeon. That's because, before coming to Google, he was a neurosurgeon, at Stanford. Reese spends a lot of his time at Google's "server farms," warehouses filled with computers that have the fastest connections to the Internet. One of Google's facilities is run by a company called Exodus, in nearby Santa Clara, and Google stores some of its network of nearly four thousand Linux computers there, each with eighty gigabytes of hard-drive space, on which it keeps constant downloads of the Web. (Many other companies, including Hotmail and eBay, use Exodus as their electronic storage vault.)

Two

For more detail on judgmental biases, see the classic collection of readings: Daniel Kahneman, Paul Slovic, and Amos Tversky (editors), Judgment under uncertainty: Heuristics and biases (1982: Cambridge University Press). A more recent volume is: Thomas Gilovich, Dale Griffin, and Daniel Kahneman (editors), Heuristics and Biases: The Psychology of Intuitive Judgment (2002: Cambridge University Press).

Daniel Kahneman (of Princeton) was awarded the Nobel Prize in Economics in 2002 for his pioneering work on judgment under uncertainty. There are many articles about this award, including information from the Nobel Foundation, the press release from Princeton, and an article from The Economist.

For more wisdom from Charles Munger, click here for a speech he gave at Harvard Law School in 1995. Munger wrote a book called Poor Charlie's Almanack: The Wit and Wisdom of Charles T Munger.

For more on the Fermi problems I referred to in class.

One of the students sent me a Youtube video of Compuserve's advertisement response to AOL's capacity problem. Very good and very old school!

For more on prediction markets, check out the Wikipedia entry. For a New York Times article on Google's use of prediction markets, click here. An academic article on the same topic is available here. There used to be a betting market called Intrade, probably the best existing public prediction market. Intrade has had enormous success in calling the Presidential election in both 2004 and 2008.

Three

For more on hindsight bias, see Baruch Fischhoff, "Hindsight =/= foresight: The effect of outcome knowledge on judgment under uncertainty," Journal of Experimental Psychology: Human Perception and Performance (1975), 288-299. A great new recent article on hindsight appeared in The New Yorker, and concerns the hindsight bias concerning September 11 (and "connecting the dots"): Malcolm Gladwell (March 11, 2003), "Connecting the Dots".

For more on the low predictive quality of interviews, see Robyn Dawes, House of Cards (1994: Free Press), Chapter 3. The Dawes book is mostly a fascinating critique of the validity of psychotherapy. A popular New Yorker article on interviews, also by Malcolm Gladwell, is found here: Malcolm Gladwell (May 29, 2000), "The New-Boy Network". Those of you who haven't done so (and especially those of you who are interested in baseball) should read Michael Lewis' Moneyball: The Art of Winning an Unfair Game (2003: Norton). Click here for a list serve posting on academic articles on interviewing. The articles cited suggest that interviews are unreliable, "do not provide incremental validity over standardized tests of intelligence and personality," "there is no evidence for expertise in interviewing," but "interviews can be structured to enhance reliability."

For those of you who still believe in the hot hand, please read the article in your course package: Amos Tversky & Thomas Gilovich (1989). "The cold facts about the 'hot hand.'" Chance, 2, 16-21. The history of hot hand research is interesting, and if you are interested check out this link. You might also like to look at this article (by legendary investor Victor Niederhoffer), which talks about hot hand with respect to investment decision making. For an entire website on the hot hand, click here. Dusty Baker, the ex manager of the Chicago Cubs, is quoted in the October 16, 2003 Chicago Sun-Times as follows:

During the regular season, generally speaking, when a guy is hot, he stays hot for a while, and when he is cold, he stays cold,'' Baker said. "But I have seen in the playoffs and World Series that it changes from series to series. Just because a guy had a great series doesn't necessarily mean that he's going to have a great one next series, and whoever had a bad series doesn't mean he's going to do bad the next one.''

There's a little support for hotness in baseball, but not much once you correct for pitcher and park effects. But there is more than in basketball.

For those of you who haven't read this, you may like to read Jon Krakauer (1998), Into Thin Air (Anchor Books). An interesting academic article about the psychology of mountain climbing is: Loewenstein, George (1999). “Because It is There: The Challenge of Mountaineering ... for Utility Theory.” Kyklos 52: 315-344.

Finally, for those really hungry for climbing statistics, click here for statistics about the number of summits and deaths (through 2005). A more recent set of statistics is found here. The current count is approximately 6600 summits and 245 deaths, with 628 summits and 7 deaths in 2007, 290 summits and 1 death in 2008, 428 summits and 4 deaths in 2011. and 425 summits, 9 deaths in 2012, and 10 deaths and 600 summits in 2013. Through 2007, there have 149 ascents without oxygen by 122 individuals, with 5 in 2007 (2 by the same person 7 days apart!). 11 of the 149 died during descent. A fascinating page with statistics about summits by day and other arcane things is found here. A couple of other statistics. The webpage contains a chronological list of the 1500 summits through the end of 2001 (since then there have about over 2000 more summits). Of the first 1500 summits, 43 died during descent (of the 172 total Everest deaths). You might (or might not) wonder about statistics about the other 13 8,000 meter peaks. The death rate on Annapurna is by far the worst, with 153 summits and 58 deaths (through 2007). Finally, I referred to the Nova special on Everest. There is a description on the Nova website of the psychological tests they conduct..

One of the students sent a link to a Youtube video of the summit. It is pretty cool. You might also enjoy looking at the National Geographic photos of Everest.

Finally, Ed Viesturs is one of the great climbers in the world. He is one of 5 (and the sole American) climbers to summit all 14 8,000 meter peaks without oxygen. He is also known for his caution. There is a great interview with him, in which he discusses his risk tolerance.

For more on procedures for flying planes, see Atul Gawande (2009). The Checklist Manifesto.

Four

For more on prospect theory, read Daniel Kahneman and Amos Tversky (1979). “Prospect theory: An analysis of decision under risk.” Econometrica 47(2): 263-291, or Amos Tversky and Daniel Kahneman (1992). “Advances in Prospect Theory: Cumulative Representation of Uncertainty.” Journal of Risk and Uncertainty 5: 297-323. Some fun reading is an article that Michael Lewis (of Moneyball and Liar's Poker fame) wrote a few years ago for Vanity Fair.

For more on mental accounting, see Richard Thaler (1999). “Mental Accounting and Consumer Choice.” Marketing Science 4, 199-214.

For a primer (though somewhat academic) on behavioral finance, see Barberis, Nicholas and Richard Thaler (2003). "A Survey of Behavioral Finance." Handbook of the Economics of Finance. George M. Constantinides, Milton Harris and Rene M. Stulz (eds), Elsevier. A more accessible book is Hersh Shefrin's (2002), "Beyond Greed and Fear: Understanding Behavioral Finance and the Psychology of Investing". Some of the most interesting research is done by Terry Odean of Berkeley (he has the discount brokerage dataset). His website is here. You might to look at the article about the efficient markets debate between Fama and Thaler in the Wall Street Journal.

For a discussion of the utility of changing defaults (in organ donation or 401(k) enrollment), see Richard Thaler and Cass Sunstein (2008). Nudge (Yale University Press). Thaler and Sunstein's website.

Here's an article on how ski resorts can use a S'No Risk type of promotion to sell ski lift season passes. I've also posted three old Wall Street Journal articles on weather derivatives.

Five

For more on decision analysis, see Robert Clemen (1997), Making Hard Decisions, perhaps the best decision analysis text book around.

For a review of decision analysis software (for the hardcore!), click here.

For software for influence diagrams, there is Data by TreeAge (www.treeage.com). 

For a primer on the Strategic Decisions Group approach to decision analysis, see Ronald Howard, "Decision Analysis: Practice and Promise," Management Science (1988), 679-695.

For a book detailing the Smithkline-Beecham process described by Tom Keelin, see David Matheson and James Matheson, The Smart Organization (1998), Harvard Business Press. The book also has some data on the different processes firms use to make decisions and how these processes correlate with success.

For more on General Motor's Decision Dialogue Process, see Vince Barabba (1995), Meeting of the Minds (Harvard Business School Press).

For a primer on the Strategic Decisions Group (a leading decision consulting group) approach to decision analysis, see Ronald Howard, "Decision Analysis: Practice and Promise," Management Science (1988), 679-695.

For more on valuing information, you might be interested in reading a note that I wrote a few years ago. The note briefly discusses how to think about imperfect information, but for more detail on how to do so, check out Robert Clemen (1997), Making Hard Decisions. That book also details how you might do two-way sensitivity analysis, i.e., sensitivity analysis with two variables at once. Another way to do this that is easy to do is use the Data Table function in Excel (with a row and column input). I've put together an example spreadsheet for the make or buy decision.

For more on scenario analysis, refer to Russo and Schoemaker's Winning Decisions. Shell Oil is a major user of this method. For more on Shell's use of scenario analysis, see Pierre Wack, "Scenarios: shooting the rapids," Harvard Business Review (November-December 1985), 139-150 or Arie P. De Geus, "Planning as Learning," Harvard Business Review (March-April 1988), 70-74.

For a more conceptual approach to scenario analysis, see Paul Schoemaker, "Multiple Scenario Development: Its Conceptual and Behavioral Foundations", Strategic Management Journal (1993), 193-213.

Books

Some of you asked for additional books that might be interesting. Here are some suggestions.

• Smart Choices (Hammond, Keeney, and Raiffa, 2002): A good accessible primer on decision making. Provides a framework for helping your make better decisions.
• Blink (Gladwell, 2005): Explores the strengths and weaknesses of snap judgments. Provocative and interesting.
• The Wisdom of Crowds (Surowiecki, 2004): Discusses how and why combining information from lots of people (the wisdom of crowds) might help.
• Predictably Irrational (Ariely, 2008): A recent fast-paced account of the foibles of our decision making.
• Nudge (Thaler and Sunstein, 2008): A lively discussion of how choice architecture can make a difference.
• Decisive (Heath and Heath, 2013): A new and accessible book on how to make better decisions.