Resources

Links and references to extra readings will be posted here.

One

Here is a link to the Harvard Business Review issue on decision making (University of Chicago access required).

There are lots of websites with images of various visual illusions, including the one I presented in class. Here is one, but the best one is probably Roger Shepard's unbelievable table turning illusion.

There are many websites devoted to the Monty Hall problem. Here is the discussion from the Car Talk Guys. You can also check out the self-proclaimed, "Official Let's Make a Deal Website" here. Wikipedia also has an entry on the Monty Hall problem. Some of the comments sent to Parade are found here. If you want to know more about Marilyn vos Savant, click here.

For an excellent book on medical decision making, see Harold Bursztajn, Richard Feinbloom, Robert M. Hamm, Archie Brodsky (1990), Medical Choices, Medical Chances (Routledge). For an excellent (and readable) book on medical decision making, see Harold Bursztajn, Richard Feinbloom, Robert M. Hamm, Archie Brodsky (1990), Medical Choices, Medical Chances (Routledge). I also recommend that you read the book: Atul Gawande (2002), Complications (Metropolitan Books). The last chapter, "The Case of the Red Leg", is especially good.

My neurosurgeon friend is Jim Reese, Google employee number 19. Here is an except from an article from The New Yorker about Google about his interesting career change

I had lunch one day with a few of the company's researchers, including Jim Reese, who told me that he was employee No. 19. His business card describes him as chief operations engineer and head neurosurgeon. That's because, before coming to Google, he was a neurosurgeon, at Stanford. Reese spends a lot of his time at Google's "server farms," warehouses filled with computers that have the fastest connections to the Internet. One of Google's facilities is run by a company called Exodus, in nearby Santa Clara, and Google stores some of its network of nearly four thousand Linux computers there, each with eighty gigabytes of hard-drive space, on which it keeps constant downloads of the Web. (Many other companies, including Hotmail and eBay, use Exodus as their electronic storage vault.)

Two

For more detail on judgmental biases, see the classic collection of readings: Daniel Kahneman, Paul Slovic, and Amos Tversky (editors), Judgment under uncertainty: Heuristics and biases (1982: Cambridge University Press). A more recent volume is: Thomas Gilovich, Dale Griffin, and Daniel Kahneman (editors), Heuristics and Biases: The Psychology of Intuitive Judgment (2002: Cambridge University Press).

Daniel Kahneman (of Princeton) was awarded the Nobel Prize in Economics in 2002 for his pioneering work on judgment under uncertainty. There are many articles about this award, including information from the Nobel Foundation, the press release from Princeton, and an article from The Economist.

For more wisdom from Charles Munger, click here for a speech he gave at Harvard Law School in 1995. Munger wrote a book called Poor Charlie's Almanack: The Wit and Wisdom of Charles T Munger .

For more on the Fermi problems I referred to in class.

For more on prediction markets, check out the Wikipedia entry. For a New York Times article on Google's use of prediction markets, click here. An academic article on the same topic is available here. You should also check out Intrade, probably the best existing public prediction market. It is very addictive. Intrade has had enormous success in calling the Presidential election in both 2004 and 2008.

Three

For more on hindsight bias, see Baruch Fischhoff, "Hindsight =/= foresight: The effect of outcome knowledge on judgment under uncertainty," Journal of Experimental Psychology: Human Perception and Performance (1975), 288-299. A great new recent article on hindsight appeared in The New Yorker, and concerns the hindsight bias concerning September 11 (and "connecting the dots"): Malcolm Gladwell (March 11, 2003), "Connecting the Dots".

The story of Bruce Sutter comes from legendary tennis coach Vic Braden. For more on Vic Braden's approach to the study of sports, click here.

For an article on Thomas Edison and concrete houses, click here for an old New York Times article.

For more on the low predictive quality of interviews, see Robyn Dawes, House of Cards (1994: Free Press), Chapter 3. The Dawes book is mostly a fascinating critique of the validity of psychotherapy. A popular New Yorker article on interviews, also by Malcolm Gladwell, is found here: Malcolm Gladwell (May 29, 2000), "The New-Boy Network". Those of you who haven't done so (and especially those of you who are interested in baseball) should read Michael Lewis' Moneyball: The Art of Winning an Unfair Game (2003: Norton). Click here for a list serve posting on academic articles on interviewing. The articles cited suggest that interviews are unreliable, "do not provide incremental validity over standardized tests of intelligence and personality," "there is no evidence for expertise in interviewing," but "interviews can be structured to enhance reliability."

I also suggested that you might want to make atypical decisions as ways of obtaining feedback that you would not otherwise get. A more dramatic approach is to experiment. In an article in the Financial Times, my colleagues Steve Levitt and John List suggest that you shouldn't guess; you should experiment.

For those of you who still believe in the hot hand, please read the article in your course package: Amos Tversky & Thomas Gilovich (1989). "The cold facts about the 'hot hand.'" Chance, 2, 16-21. The history of hot hand research is interesting, and if you are interested check out this link. You might also like to look at this article (by legendary investor Victor Niederhoffer), which talks about hot hand with respect to investment decision making. For an entire website on the hot hand, click here. Dusty Baker, the ex manager of the Chicago Cubs, is quoted in the October 16, 2003 Chicago Sun-Times as follows:

During the regular season, generally speaking, when a guy is hot, he stays hot for a while, and when he is cold, he stays cold,'' Baker said. "But I have seen in the playoffs and World Series that it changes from series to series. Just because a guy had a great series doesn't necessarily mean that he's going to have a great one next series, and whoever had a bad series doesn't mean he's going to do bad the next one.''

There's a little support for hotness in baseball, but not much once you correct for pitcher and park effects. But there is more than in basketball.

For more on sports decision making, see the recent segment on NewHour with Richard Thaler and Joe Torre.

Four

For more on prospect theory, read Daniel Kahneman and Amos Tversky (1979). “Prospect theory: An analysis of decision under risk.” Econometrica 47(2): 263-291, or Amos Tversky and Daniel Kahneman (1992). “Advances in Prospect Theory: Cumulative Representation of Uncertainty.” Journal of Risk and Uncertainty 5: 297-323.

For more on mental accounting, see Richard Thaler (1999). “Mental Accounting and Consumer Choice.” Marketing Science 4, 199-214.

For a primer (though somewhat academic) on behavioral finance, see Barberis, Nicholas and Richard Thaler (2003). "A Survey of Behavioral Finance." Handbook of the Economics of Finance. George M. Constantinides, Milton Harris and Rene M. Stulz (eds), Elsevier. A more accessible book is Hersh Shefrin's (2002), "Beyond Greed and Fear: Understanding Behavioral Finance and the Psychology of Investing". Some of the most interesting research is done by Terry Odean of Berkeley (he has the discount brokerage dataset). His website is here. You might to look at the article about the efficient markets debate between Fama and Thaler in the Wall Street Journal.

For a discussion of the utility of changing defaults (in organ donation or 401(k) enrollment), see Richard Thaler and Cass Sunstein (2008). Nudge (Yale University Press). Thaler and Sunstein's website.

For those of you who haven't read this, you may like to read Jon Krakauer (1998), Into Thin Air (Anchor Books). An interesting academic article about the psychology of mountain climbing is: Loewenstein, George (1999). “Because It is There: The Challenge of Mountaineering ... for Utility Theory.” Kyklos 52: 315-344.

Finally, for those really hungry for climbing statistics, click here for statistics about the number of summits and deaths (through 2005). A more recent set of statistics is found here. The current count is 3969 summits and 211 deaths, with 628 summits and 7 deaths in 2007 and 290 summits and 1 death in 2008. Through 2007, There have 149 ascents without oxygen by 122 individuals, with 5 in 2007 (2 by the same person 7 days apart!). 11 of the 149 died during descent. A fascinating page with statistics about summits by day and other arcane things is found here. A couple of other statistics. The webpage contains a chronological list of the 1500 summits through the end of 2001 (since then there have about over 2000 more summits). Of the first 1500 summits, 43 died during descent (of the 172 total Everest deaths). You might (or might not) wonder about statistics about the other 13 8,000 meter peaks. The death rate on Annapurna is by far the worst, with 130 summits and 53 deaths (through 2004), compared to 2249 summits and 186 deaths for Everest (though 2004).

One of my oldstudents sent a link to a Youtube video of the summit. It is pretty cool.

Five

For more on decision analysis, see Robert Clemen (1997), Making Hard Decisions, perhaps the best decision analysis text book around.

For a review of decision analysis software (for the hardcore!), click here.

For software for influence diagrams, there is Data by TreeAge (www.treeage.com). 

For a primer on the Strategic Decisions Group approach to decision analysis, see Ronald Howard, "Decision Analysis: Practice and Promise," Management Science (1988), 679-695.

For a book detailing the Smithkline-Beecham process described by Tom Keelin, see David Matheson and James Matheson, The Smart Organization (1998), Harvard Business Press. The book also has some data on the different processes firms use to make decisions and how these processes correlate with success.

For more on General Motor's Decision Dialogue Process, see Vince Barabba (1995), Meeting of the Minds (Harvard Business School Press).

For a primer on the Strategic Decisions Group (a leading decision consulting group) approach to decision analysis, see Ronald Howard, "Decision Analysis: Practice and Promise," Management Science (1988), 679-695.

For more on valuing information, you might be interested in reading a note that I wrote a few years ago. The note briefly discusses how to think about imperfect information, but for more detail on how to do so, check out Robert Clemen (1997), Making Hard Decisions. That book also details how you might do two-way sensitivity analysis, i.e., sensitivity analysis with two variables at once. Another way to do this that is easy to do is use the Data Table function in Excel (with a row and column input). I've put together an example spreadsheet for the make or buy decision.

For more on scenario analysis, refer to Russo and Schoemaker's Winning Decisions. Shell Oil is a major user of this method. For more on Shell's use of scenario analysis, see Pierre Wack, "Scenarios: shooting the rapids," Harvard Business Review (November-December 1985), 139-150 or Arie P. De Geus, "Planning as Learning," Harvard Business Review (March-April 1988), 70-74.

For a more conceptual approach to scenario analysis, see Paul Schoemaker, "Multiple Scenario Development: Its Conceptual and Behavioral Foundations", Strategic Management Journal (1993), 193-213.

For more on creativity, try James L. Adams, Conceptual Blockbusting (2001: Addison-Wesley), Doug Hall and David Wecker, Jump Start Your Brain (2007, Clerisy Press), or Barry Nalebuff and Ian Ayres, Why Not? (2003: Harvard Business School Press). You might also enjoy reading Andrew Hargadon, How Breakthroughs Happen (2003: Harvard Business School Press), which discusses how innovation happens in firms. You may also like to see the ABC Nightline special on IDEO, a leading product development firm.