Big data and bad data: On the sensitivity of security policy to imperfect information

University of Chicago Law Review

Volumn 83, Issue 1, 2016, Pages 117-137

  Graves, James T ^a   Acquisti, Alessandro ^a   Christin, Nicolas ^b  

^a CARNEGIE MELLON UNIVERSITY   (United States)

^b CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84964501372     PISSN: 00419494     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (98)

1. Charles Babbage, Passages from the Life of a Philosopher 67 (Longman 1864) (quotation marks omitted).
2. See generally National Science and Technology Council, Trustworthy Cyberspace: Strategic Plan for the Federal Cybersecurity Research and Development Program (Executive Office of the President, Dec. 2011), archived at http://perma.cc/W456-7LGH.
3. Munindar P. Singh, Toward a Science of Security (Computing Now, Jan 2013), archived at http://perma.cc/3ED4-WGYZ.
4. See Fred B. Schneider, Blueprint for a Science of Cybersecurity, 19 Next Wave 47, 47(2012), archived at http://perma.cc/CMH7-QNJL.
5. Frederick R. Chang, Guest Editor's Column, 19 Next Wave i, i (2012), archived at http://perma.cc/Q4KV-LYXF ("There are some promising indications that a science of cybersecurity initiative is gaining momentum, including several workshops, conferences, and reports that point to the need for an interdisciplinary approach to addressing the problem.");
6. Robert Meushaw, NSA Initiatives in Cybersecurity Science, 19 Next Wave 8, 10-11(2012), archived at http://perma.cc/Q4KV-LYXF (discussing funding for cybersecurity research).
7. See Singh, Toward a Science of Security (cited in note 4).
8. Schneider, Blueprint at 53 (cited in note 5).
9. See Charles H. Brown, et al, The Science of Security: A Survey and Analysis ∗2 (AFCEA International Cyber Committee, June 2014), archived at http://perma.cc/39CH-XUJF (highlighting the various areas that are reliant on cybertechnologies and the broad motivations for developing a science of cybersecurity).
10. James R. Clapper, Worldwide Cyber Threats ∗2 (House Permanent Select Committee on Intelligence, Sept 10, 2015), archived at http://perma.cc/7XN6-9N5W.
11. See Cyberspace Policy Review: Assuring a Trusted and Resilient Information and Communications Infrastructure ∗17, archived at http://perma.cc/2J26-2SM7 ("The public and private sectors' interests are intertwined with a shared responsibility for ensuring a secure, reliable infrastructure.").
12. See Alessandro Acquisti, Allan Friedman, and Rahul Telang, Is There a Cost to Privacy Breaches? An Event Study ∗2 (Twenty Seventh International Conference on Information Systems, 2006), archived at http://perma.cc/4QM6-VHT2 (noting that some companies that have experienced "privacy debacles" were subjected to "public outrage and hard to quantify reputation losses").
13. See generally James T. Graves, Alessandro Acquisti, and Nicolas Christin, Should Payment Card Issuers Reissue Cards in Response to a Data Breach? (13th Annual Workshop on the Economics of Information Security, June 2014), archived at http://perma.cc/HJ6E-6Z6V.
14. See, for example, Visa International Operating Regulations ∗648-53 (Visa, Apr. 15, 2014), archived at http://perma.cc/79X7-EDGL.
15. See also Adam J. Levitin, Private Disordering? Payment Card Fraud Liability Rules, 5 Brooklyn J Corp, Fin & Comm L 1, 14-15 (2010) (outlining the general rules of liability for unauthorized credit card transactions);
16. Richard A. Epstein and Thomas P. Brown, Cybersecurity in the Payment Card Industry, 75 U Chi L Rev 203, 214-16 (2008) (describing the "elaborate systems to detect fraud" and the punishments prescribed for noncompliance).
17. See, for example, Pennsylvania State Employees Credit Union v Fifth Third Bank, 398 F Supp 2d 317, 322 (MD Pa 2005) (stating that the Pennsylvania State Employees Credit Union canceled and reissued 20, 029 cards at a total cost of $98, 128, or about $5 per card);
18. Maine Bureau of Financial Institutions, Maine Data Breach Study ∗18-20 (Maine Department of Professional & Financial Regulation, Nov. 24, 2008), archived at http://perma.cc/D8U9-CMA7 (finding reissuance costs totaling $1, 164, 200 across 246, 479 reissued cards, or an average cost to issuers of $4.72 per card);
19. Maria Aspan and Clare Baldwin, Sony Breach Could Cost Card Lenders $300 Mln (Reuters, Apr. 28, 2011), archived at http://perma.cc/W6YE-KCMK (reporting that "[e]ach customer request to replace a credit card would cost lenders about $3 to $5 per card", which includes "the new piece of plastic itself, postage, and various customer service costs");
20. Chris Churchill, TJX Reacts to Bank Lawsuit; T. J. Maxx Parent in Filing Says TrustCo Failed to Mitigate Injury from Data Breach, Times Union B9 (Aug. 30, 2008) (citing TrustCo as saying that its costs from the TJX breach, including reissuing four thousand debit cards, were up to $20 per affected account);
21. Mark Jewell, IDs Are a Steal; Thieves Looking for Credit Numbers Set Their Sights on Big Targets, Vancouver Columbian E1 (Aug. 23, 2004) (reporting that Sovereign Bank reissued eighty-one thousand cards twice at a total cost of about $1 million);
22. Denis Paiste, Compromised Credit Cards Top 100, 000, NH Union Leader B3 (Jan 31, 2007) (reporting that "[v]arious banks and credit unions have said it costs from $5 to $25 per card reissued");
23. Anne Ravana, Banks Start Credit Card Reissue; Breach of Databases Prompts Replacements, Bangor Daily News A4 (Feb. 8, 2007) (quoting a Merrill Bank executive as saying that the cost of replacing seventyone cards was about $14 per card);
24. Eric G. Stark, Computer Hackers Are Stealing Bank Card Information, but There Is Protection and Some Banks Have Been Aggressive, Lancaster Sunday News D1 (July 11, 2004) (reporting that Fulton Bank spent $100, 000 to replace twenty thousand cards).
25. See, for example, Harriet Pearson, Letter to the Office of the Attorney General (Sept 9, 2014), archived at http://perma.cc/YM3P-QUZ6 (stating that Home Depot "received reports from its banking partners and law enforcement that criminals may have hacked its payment data systems");
26. Brian Krebs, Staples: 6-Month Breach, 1.16 Million Cards (Krebs on Security, Dec. 19, 2014), archived at http://perma.cc/86YF-PUPG (implying that the office-supply store Staples may have been informed of a data breach by banks that noticed suspicious activity);
27. Jeff Goldman, Data Breach at TripAdvisor's Viator Impacts 1.4 Million Users (eSecurity Planet, Sept 24, 2014), archived at http://perma.cc/68TP-CXLU (reporting that Viator was notified "by its payment card service provider" of a breach).
28. See Lynn Langton, Identity Theft Reported by Households, 2005-2010 ∗1 (DOJ, Nov. 2011), archived at http://perma.cc/4BFK-MKQZ.
29. See, for example, id; Consumer Sentinel Network Data Book for January - December 2014 ∗12 (FTC, Feb. 2015), archived at http://perma.cc/7B3D-9Y2R;
30. Lynn Langton and Michael Planty, National Crime Victimization Survey Supplement: Victims of Identity Theft, 2008 ∗1 (DOJ, Dec. 2010), archived at http://perma.cc/3QAP-KDDU.
31. See generally, for example, Synovate, Federal Trade Commission - 2006 Identity Theft Survey Report (FTC, Nov. 2007), archived at http://perma.cc/BNJ3-PDF2;
32. Synovate, Federal Trade Commission - Identity Theft Survey Report (FTC, Sept 2003), archived at http://perma.cc/ZZD3-X5FC.
33. Security Breach Notification Laws (National Conference of State Legislatures, June 11, 2015), archived at http://perma.cc/86QR-KUXN.
34. See generally, for example, Maine Security Breach Reporting Form, archived at http://perma.cc/B6DX-62NS;
35. Guidelines for Businesses to Comply with the Maryland Personal Information Protection Act (Maryland Attorney General), archived at http://perma.cc/4JGG-HALD;
36. Database Breach Notification Requirements, archived at http://perma.cc/FEH3-XTDU (requesting information to be sent to the Virginia attorney general office).
37. See Chronology of Data Breaches: Security Breaches 2005 - Present (Privacy Rights Clearinghouse), archived at http://perma.cc/G7V9-QPTS;
38. Data Breach Notices, archived at http://perma.cc/4L2L-VUGE (listing breach notifications sent to the attorney general of Maine);
39. Maryland Information Security Breach Notices - 2015 (Maryland Attorney General), archived at http://perma.cc/U8C6-BL7T;
40. Security Breach Notifications (New Hampshire Department of Justice), archived at http://perma.cc/PW7M-24JN.
41. See Langton, Identity Theft at ∗5 (cited in note 20).
42. See Langton and Planty, National Crime Victimization Survey at ∗13 (cited in note 21);
43. Gary R. Gordon, et al, Identity Fraud Trends and Patterns: Building a Data-Based Foundation for Proactive Enforcement ∗3 (Center for Identity Management and Information Protection, Oct. 2007), archived at http://perma.cc/6ABG-ZZAH;
44. Synovate, 2006 Identity Theft Survey Report at ∗30 (cited in note 22).
45. See Linda Foley, et al, Identity Theft: The Aftermath 2009 ∗15 (ITRC, 2010), archived at http://perma.cc/GV2E-F4QV.
46. See Graves, Acquisti, and Christin, Should Payment Card Issuers Reissue Cards at ∗19 (cited in note 15).
47. Foley, et al, Identity Theft at ∗15 (cited in note 33).
48. Graves, Acquisti, and Christin, Should Payment Card Issuers Reissue Cards at ∗19 (cited in note 15).
49. See, for example, Gordon, et al, Identity Fraud Trends at ∗53 (cited in note 32).
50. For an introduction to Monte Carlo simulations, see W. K. Hastings, Monte Carlo Sampling Methods Using Markov Chains and Their Applications, 57 Biometrika 97, 97-98 (1970) ("For numerical problems in a large number of dimensions, Monte Carlo methods are often more efficient than conventional numerical methods.").
51. See Graves, Acquisti, and Christin, Should Payment Card Issuers Reissue Cards at ∗26-27 (cited in note 15).
52. See, for example, Shirley W. Inscoe, Global Consumers React to Rising Fraud: Beware Back of Wallet ∗17 (Aite Group, Oct. 2012), archived at http://perma.cc/TZ4B-QB2F ("33% of consumers who received replacement cards [after a breach] state that they used the new card less frequently than the original card."). It is unclear whether this is because a card was reissued or because of the card exposure regardless of reissue.
53. See Graves, Acquisti, and Christin, Should Payment Card Issuers Reissue Cards at ∗28-30 (cited in note 15).
54. See Tyler Moore, Richard Clayton, and Ross Anderson, The Economics of Online Crime, 23 J Econ Persp 3, 3-4 (Summer 2009) (describing the transition from "amateur hackers who defaced websites and wrote malicious software" in a "cottage industry" to "criminal networks" and "online black markets").
55. See id; Ross Anderson, et al, Measuring the Cost of Cybercrime, in Rainer Böhme, ed, The Economics of Information Security and Privacy 265, 296-97 (Springer 2013).
56. See, for example, Yuehong Yuan and Jonathan P. Caulkins, The Effect of Variation in High-Level Domestic Drug Enforcement on Variation in Drug Prices, 32 Socio-Econ Planning Sci 265, 266 (1998).
57. See, for example, Moore, Clayton, and Anderson, 23 J Econ Persp at 8 (cited in note 40) (describing the ability of researchers to "stud[y] the new criminal markets directly" by "monitor[ing] the public chat channels used by online criminals to contact each other", "infiltrat[ing]⋯ botnet[s]", and using related means).
58. Anderson, et al, Measuring the Cost at 267-73 (cited in note 41).
59. See Peter Maass and Megha Rajagopalan, Does Cybercrime Really Cost $1 Trillion? (Pro Publica, Aug. 1, 2012), archived at http://perma.cc/6UD3-GA3S (questioning McAfee's $1 trillion estimate, a figure cited by General Keith Alexander, then-director of the NSA, but published only in a McAfee news release and not in its cybercrime study);
60. Josh Rogin, NSA Chief: Cybercrime Constitutes the "Greatest Transfer of Wealth in History" (Foreign Policy, July 9, 2012), archived at http://perma.cc/G2UL-EPW8.
61. Anderson, et al, Measuring the Cost at 295 (cited in note 41).
62. Compare Ross Anderson, Unsettling Parallels between Security and the Environment, archived at http://perma.cc/A8Z6-L67K ("My intuition is that many firms get it about right, or if anything spend slightly too much [on network security].")
63. with Bruce Schneier, Computer Security: It's the Economics, Stupid (Workshop on Economics and Information Security, May 16, 2002), archived at http://perma.cc/RWR3-J4XQ (arguing that organizations spend too little on computer security but will spend more only if software companies improve their security abilities).
64. See, for example, Busted, but Not Broken: The State of Silk Road and the Darknet Marketplaces ∗1 (Digital Citizens Alliance), archived at http://perma.cc/9PLU-GGHG ("Approximately 13, 648 listings for drugs are now available on Silk Road.").
65. See Kyle Soska and Nicolas Christin, Measuring the Longitudinal Evolution of the Online Anonymous Marketplace Ecosystem ∗38-39 (USENIX, Aug. 2015), archived at http://perma.cc/N7V3-WW8M.
66. See, for example, Patrick Howell O'Neill, Dark Net Markets Offer More Drugs Than Ever Before (Daily Dot, May 13, 2015), archived at http://perma.cc/432X-46JT ("Dark Net markets have grown over 37 percent in product listings in the last year despite sweeping police actions and the constant threat of multimillion-dollar thefts looming large.");
67. Steven Nelson, Largest Online Drug Market Shuts in Massive Suspected Scam (US News, Mar. 18, 2015), archived at http://perma.cc/2MS4-JQLX ("The 15 largest deep web markets had about 42, 000 drug listings, indicating the overall deep web drug market had nearly recovered from last year's FBI-led raids.");
68. Jay Newton-Small, The Silk Road Is Back: The Dread Pirate Roberts Sails the Illicit Online Drug Trade Again (Time, Apr. 30, 2014), archived at http://perma.cc/DX9Q-DZPM ("There are approximately 13, 648 listings for drug on the Darknet website⋯ compared to the 13, 000 listed shortly before [Ross] Ulbricht's arrest.");
69. Matthew Mosk, Underground Website Used for Black Market Drug Sales Bigger Than the Original, Report Says (ABC News, Apr. 30, 2014), archived at http://perma.cc/K4UU-ESQ2 (noting that the "'darknet' drug economy as a whole contains 75 percent more listings for drugs" than it did before the FBI shut down the popular site Silk Road).
70. Peter Andreas and Kelly M. Greenhill, Introduction: The Politics of Numbers, in Peter Andreas and Kelly M. Greenhill, eds, Sex, Drugs, and Body Counts: The Politics of Numbers in Global Crime and Conflict 1, 19 (Cornell 2010).
71. See, for example, Andy Greenberg, Follow the Bitcoins: How We Got Busted Buying Drugs on Silk Road's Black Market (Forbes, Sept 5, 2013), archived at http://perma.cc/TWY2-VU29.
72. See How Blacklists Work (MailChimp, July 24, 2015), archived at http://perma.cc/534J-MHHL.
73. See Andreas Pitsillidis, et al, Taster's Choice: A Comparative Analysis of Spam Feeds ∗1 (ACM, Nov. 2012), archived at http://perma.cc/5FCJ-9XA3.
74. See Newton-Small, The Silk Road Is Back (cited in note 51).
75. Rainer Böhme, et al, Bitcoin: Economics, Technology, and Governance, 29 J Econ Persp 213, 222-23 (Spring 2015).
76. Sealed Complaint, United States v Ulbricht, Criminal Action No 13-2328, ∗15 (SDNY filed Sept 27, 2013) ("Ulbricht Complaint") ("The total revenue generated from these sales was 9, 519, 664 Bitcoins, and the total commissions collected by Silk Road from the sales amounted to 614, 305 Bitcoins. These figures are equivalent to roughly $1.2 billion in revenue and $79.8 million in commissions.").
77. See, for example, Nicolas Christin, Traveling the Silk Road: A Measurement Analysis of a Large Anonymous Online Marketplace ∗10 (International World Wide Web Conference Committee, May 2013), archived at http://perma.cc/55QM-4MKJ (estimating Silk Road's monthly sales volume at approximately $1.2 million).
78. See, for example, Joshuah Bearman, Silk Road: The Untold Story (Wired, May 23, 2015), archived at http://perma.cc/JL8R-3F43;
79. David Segal, Eagle Scout. Idealist. Drug Trafficker? (NY Times, Jan 18, 2014), archived at http://perma.cc/FG78-NJR8;
80. Saumya Vaishampayan, Silk Road Drug Market Handled $1.2 Billion of Transactions in 2.5 Years before FBI Seizure (MarketWatch, Oct. 2, 2013), archived at http://perma.cc/E4T2-JJ46.
81. Emily Flitter, U. S. Sharply Reduces Silk Road's Estimated Sales Volume (Reuters, Jan 20, 2015), archived at http://perma.cc/H5N7-P2AV.
82. See, for example, Soska and Christin, Measuring the Longitudinal Evolution at ∗40 (cited in note 50) (estimating Silk Road's gross sales at over $100 million per year, which "appears consistent with the (revised) US Government calculations of $214M of total grossed income by Silk Road over its lifetime").
83. See Ulbricht Complaint at ∗15 (cited in note 60).
84. See United States v Ulbricht, 31 F Supp 3d 540, 550 (SDNY 2014) ("According to the Indictment, the defendant pursued violent means, including soliciting the murderfor-hire of several individuals he believed posed a threat to that enterprise.') (quotation marks omitted).
85. See also Superseding Indictment, United States v Ulbricht, Criminal Action No 13-0222, ∗6-11 (D Md filed Oct. 1, 2013).
86. Andy Greenberg, Alleged Silk Road Boss Ross Ulbricht Now Accused of Six Murdersfor-Hire, Denied Bail (Forbes, Nov. 21, 2013), archived at http://perma.cc/XCR9-3ADT.
87. See United States Sentencing Commission, Guidelines Manual §§ 2B1.1 (b) (1), 2S1.1 (a) (2) (2014).
88. Andy Greenberg, Silk Road Creator Ross Ulbricht Sentenced to Life in Prison (Wired, May 29, 2015), archived at http://perma.cc/GAF2-LPNM.
89. World Drug Report 2014 ∗18 (UN Office on Drugs and Crime, June 2014), archived at http://perma.cc/4GKG-4TW8.
90. See, for example, Fred Chong, et al, National Cyber Leap Year Summit 2009: Co-Chairs' Report ∗9-12 (Sept 16, 2009), archived at http://perma.cc/4CAK-5TZ6 ("[I]t is imperative that appropriate models of cooperation are developed immediately to incentivize the participants to engage in research, development, and testing of technologies and approaches to achieve [security] goals.").
91. See, for example, Shirley S. Wang, How Much Should Scientists Check Other Scientists' Work? (Wall St J, Oct. 5, 2015), archived at http://perma.cc/WN46-JWGT (discussing one psychology journal's initiative to promote data sharing, as well as the advantages and disadvantages of "open science and data sharing").
92. For details about the OPM breach, see Andrea Peterson, OPM Says 5.6 Million Fingerprints Stolen in Cyberattack, Five Times as Many as Previously Thought (Wash Post, Sept 23, 2015), archived at http://perma.cc/EPZ4-XTVB.
93. See Kellan Howell, U. S. Will Retaliate against China for OPM Hacks (Wash Times, Aug. 1, 2015), archived at http://perma.cc/TEN9-9FUV.
94. See Soska and Christin, Measuring the Longitudinal Evolution at ∗33-34 (cited in note 50).
95. See Dozens of Online "Dark Markets" Seized pursuant to Forfeiture Complaint Filed in Manhattan Federal Court in Conjunction with the Arrest of the Operator of Silk Road 2.0 (DOJ, Nov. 7, 2014), archived at http://perma.cc/6VCE-GWCR (noting that the seizures were "part of a coordinated international law enforcement action").
96. TIAS No 13174, 2296 UNTS 167 (2001).
97. Chart of Signatures and Ratifications of Treaty 185: Convention on Cybercrime (Council of Europe, Oct. 19, 2015), archived at http://perma.cc/J72Y-3L6E.
98. See Nicolas Christin, On Critical Infrastructure Protection and International Agreements ∗4-5 (Center for International and Security Studies at Maryland, Mar. 2011), archived at http://perma.cc/H7Q3-WXQ2.