Privee: An architecture for automatically analyzing web privacy policies

Proceedings of the 23rd USENIX Security Symposium

Volumn , Issue , 2014, Pages 1-16

  Zimmeck, Sebastian ^a   Bellovin, Steven M ^a  

^a Fu Foundation School of Engineering and Applied Science   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

TRANSPARENCY;

AUTOMATIC CLASSIFICATION; CLASSIFIER PERFORMANCE; NATURAL LANGUAGES; NOTICE AND CHOICES; POLICY ANALYSIS; PRIVACY POLICIES; PRIVACY PRACTICES; PROOF OF CONCEPT;

WEBSITES;

EID: 85066762295     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (78)

1. Claridge v. RockYou, Inc., 785 F. Supp. 2d 855 (N.D. Cal. 2011).
2. P3P compact policy cross-reference. http://compactprivacypolicy.org/compacttokenreference.htm. Last accessed: July 1, 2014.
3. Privacy Icons. http://www.azarask.in/blog/post/privacy-icons/. Last accessed: July 1, 2014.
4. Privacychoice. http://www.privacychoice.org. Last accessed: July 1, 2014.
5. Terms of Service; Didn't Read (ToS;DR). http://tosdr.org/index.html. Last accessed: July 1, 2014.
6. Top sites in United States. http://www.alexa.com/topsites/countries/US. Last accessed: July 1, 2014.
7. TOSBack. http://tosback.org/. Last accessed: July 1, 2014.
8. TOSBack2. https://github.com/pde/tosback2. Last accessed: July 1, 2014.
9. Usable Privacy Policy Project. http://www.usableprivacy.org/home. Last accessed: July 1, 2014.
10. Platform for Internet Content Selection (PICS). http://www.w3.org/PICS/, 1997. Last accessed: July 1, 2014.
11. KnowPrivacy. http://knowprivacy.org/, June 2009. Last accessed: July 1, 2014.
12. eXtensible Access Control Markup Language (XACML) version 3.0. http://docs.oasis-open.org/xacml/3.0/xacml-3.0-core-spec-os-en.html, Jan. 2013. Last accessed: July 1, 2014.
13. ADREEVSKAIA, A., and BERGLER, S. Mining WordNet for fuzzy sentiment: Sentiment tag extraction from WordNet glosses. In 11th conference of the European chapter of the Association for Computational Linguistics (Stroudsburg, PA, USA, 2006), EACL '06, ACL, pp. 209-216.
14. AGRAWAL, R., KIERNAN, J., SRIKANT, R., and XU, Y. An XPath-based preference language for P3P. In Proceedings of the 12th international conference on World Wide Web (New York, NY, USA, 2003), www'03, ACM, pp. 629-639.
15. ÄIMEUR, E., GAMBS, S., and HO, A. UPP: User privacy policy for social networking sites. In Fourth International Conference on Internet and Web applications and services (Washington, DC, USA, 2009), ICIW '09, IEEE Computer Society, pp. 267-272.
16. AMMAR, W., WILSON, S., SADEH, N., and SMITH, N. Automatic categorization of privacy policies: A pilot study. Tech. Rep. CMU-ISR-12-114, CMU-LTI-12-019, Carnegie Mellon University, Dec. 2012.
17. ARTSTEIN, R., and POESIO, M. Inter-coder agreement for computational linguistics. Comput. Linguist. 34, 4 (Dec. 2008), 555-596.
18. ASHLEY, P., HADA, S., KARJOTH, G., POWERS, C., and SCHUNTER, M. Enterprise Privacy Authorization Language (EPAL 1.2). Tech. rep., IBM, Nov. 2003.
19. BIAGIOLI, C., FRANCESCONI, E., PASSERINI, A., MONTE-MAGNI, S., and SORIA, C. Automatic semantics extraction in law documents. In Proceedings of the 10th International Conference on Artificial Intelligence and Law (New York, NY, USA, 2005), ICAIL '05, ACM, pp. 133-140.
20. BREAUX, T. D., AND ANTÓN, A. I. Mining rule semantics to understand legislative compliance. In Proceedings of the 2005 ACM Workshop on Privacy in the Electronic Society (New York, NY, USA, 2005), WPES '05, ACM, pp. 51-54.
21. BREAUX, T. D., AND ANTÓN, A. I. Analyzing regulatory rules for privacy and security requirements. IEEE Trans. Software Eng. 34, 1 (Jan. 2008), 5-20.
22. BRODIE, C., KARAT, C.-M., KARAT, J., and FENG, J. Usable security and privacy: a case study of developing privacy management tools. In Proceedings of the 2005 Symposium On Usable Privacy and Security (New York, NY, USA, 2005), SOUPS '05, ACM, pp. 35-43.
23. BRODIE, C. A., KARAT, C.-M., and KARAT, J. An empirical study of natural language parsing of privacy policy rules using the SPARCLE policy workbench. In Proceedings of the second Symposium On Usable Privacy and Security (New York, NY, USA, 2006), SOUPS '06, ACM, pp. 8-19.
24. BYERS, S., and CRANOR., L. F., KORMANN, D., and MCDANIEL, P. Searching for privacy: design and implementation of a P3P-enabled search engine. In Proceedings of the 4th international conference on Privacy Enhancing Technologies (Berlin, Heidelberg, Germany, 2005), PET '04, Springer, pp. 314-328.
25. CIOCCHETTI, C. A. The future of privacy policies: A privacy nutrition label filled with fair information practices. J. Marshall J. Computer & Info. L. 26 (2008), 1-46.
26. COSTANTE, E., DEN HARTOG, J., and PETKOVIC, M. What websites know about you. In DPM/SETOP (Berlin, Heidelberg, Germany, 2012), R. D. Pietro, J. Herranz, E. Damiani, and R. State, Eds., Vol. 7731 of Lecture Notes in Computer Science, Springer, pp. 146-159.
27. COSTANTE, E., SUN, Y., PETKOVIĆ, M., AND DEN HARTOG, J. A machine learning solution to assess privacy policy completeness: (short paper). In Proceedings of the 2012 ACM Workshop on Privacy in the Electronic Society (New York, NY, USA, 2012), WPES '12, ACM, pp. 91-96.
28. CRANOR, L. F. Necessary but not sufficient: Standardized mechanisms for privacy notice and choice. J. on Telecomm. and High Tech. L. 10, 2 (2012), 273-307.
29. CRANOR, L. F., DOBBS, B., EGELMAN, S., HOGBEN, G., HUMPHREY, J., LANGHEINRICH, M., MARCHIORI, M., PRESLER-MARSHALL, M., REAGLE, J. M., SCHUNTER, M., and STAMPLEY., D. A., and WENNING, R. The Platform for Privacy Preferences 1.1 (P3P1.1) specification. World Wide Web Consortium, Note NOTE-P3P11-20061113, November 2006.
30. CRANOR, L. F., GUDURU, P., and ARJULA, M. User interfaces for privacy agents. ACM Trans. Comput.-Hum. Interact. 13, 2 (June 2006), 135-178.
31. CRANOR, L. F., LANGHEINRICH, M., and MARCHIORI, M. A P3P Preference Exchange Language 1.0 (APPEL 1.0). World Wide Web Consortium, Working Draft WD-P3P-preferences-20020415, April 2002.
32. CRANOR, L. F., LANGHEINRICH, M., MARCHIORI, M., PRESLER-MARSHALL, M., and REAGLE, J. M. The Platform for Privacy Preferences 1.0 (P3P1.0) specification. World Wide Web Consortium, Recommendation REC-P3P-20020416, April 2002.
33. CRANOR, L. F., and REIDENBERG, J. R. Can user agents accurately represent privacy notices? TPRC (Sept. 2002).
34. DE MAAT, E., KRABBEN, K., and WINKELS, R. Machine learning versus knowledge based classification of legal texts. In Proceedings of the 2010 conference on Legal Knowledge and Information Systems: JURIX 2010: The Twenty-Third Annual Conference (Amsterdam, The Netherlands, The Netherlands, 2010), IOS Press, pp. 87-96.
35. DE MAAT, E., and WINKELS, R. Automatic classification of sentences in dutch laws. In Proceedings of the 2008 conference on Legal Knowledge and Information Systems: JURIX 2008: The Twenty-First Annual Conference (Amsterdam, The Netherlands, The Netherlands, 2008), IOS Press, pp. 207-216.
36. DE MAAT, E., and WINKELS, R. A next step towards automated modelling of sources of law. In Proceedings of the 12th International Conference on Artificial Intelligence and Law (New York, NY, USA, 2009), ICAIL '09, ACM, pp. 31-39.
37. DI EUGENIO, B., and GLASS, M. The kappa statistic: a second look. Comput. Linguist. 30, 1 (Mar. 2004), 95-101.
38. FEDERAL TRADE COMMISSION. Protecting consumer privacy in an era of rapid change. http://www.ftc.gov/reports/protecting-consumer-privacy-era-rapid-change-recommendations-businesses-policymakers, Mar. 2012. Last accessed: July 1, 2014.
39. FISCHER-HÜBNER, S., AND ZWINGELBERG, H. UI prototypes: Policy administration and presentation - version 2. Tech. Rep. D4.3.2, Karlstad University, 2010.
40. FLEISS, J. Measuring nominal scale agreement among many raters. Psychological Bulletin 76, 5 (1971), 378-382.
41. FRANCESCONI, E., and PASSERINI, A. Automatic classification of provisions in legislative texts. Artif. Intell. Law 15, 1 (Mar. 2007), 1-17.
42. GODBOLE, S., and SARAWAGI, S. Discriminative methods for multi-labeled classification. In Proceedings of the 8th Pacific-Asia Conference on Knowledge Discovery and Data Mining (Berlin, Heidelberg, Germany, 2004), Springer, pp. 22-30.
43. HALL, M., FRANK, E., HOLMES, G., PFAHRINGER, B., REUTEMANN, P., and WITTEN, I. H. The WEKA data mining software: An update. SIGKDD Explor. Newsl. 11, 1 (Nov. 2009), 10-18.
44. HOFFMAN, P., and RALPH., M. L., and ROGERS, T. Semantic diversity: A measure of semantic ambiguity based on variability in the contextual usage of words. BRM 45, 3 (2013), 718-730.
45. HOLTZ, L.-E., NOCUN, K., and HANSEN, M. Towards displaying privacy information with icons. In Privacy and Identity Management for Life (Berlin, Heidelberg, Germany, 2011), S. Fischer Hübner, P. Duquenoy, M. Hansen, R. Leenes, and G. Zhang, Eds., Vol. 352 of IFIP Advances in Information and Communication Technology, Springer, pp. 338-348.
46. HRIPCSAK, G., and ROTHSCHILD, A. S. Technical brief: Agreement, the F-measure, and reliability in information retrieval. JAMIA 12, 3 (2005), 296-298.
47. KARJOTH, G., SCHUNTER, M., and WAIDNER, M. Platform for Enterprise Privacy Practices: privacy-enabled management of customer data. In Proceedings of the 2nd international conference on Privacy Enhancing Technologies (Berlin, Heidelberg, Germany, 2003), PET '02, Springer, pp. 69-84.
48. KELLEY, P. G. Designing a privacy label: assisting consumer understanding of online privacy practices. In CHI '09 Extended Abstracts on Human Factors in Computing Systems (New York, NY, USA, 2009), CHI EA '09, ACM, pp. 3347-3352.
49. KELLEY, P. G., BRESEE, J., CRANOR, L. F., and REEDER, R. W. A "nutrition label" for privacy. In Proceedings of the 5th Symposium On Usable Privacy and Security (New York, NY, USA, 2009), SOUPS '09, ACM, pp. 4:1-4:12.
50. KELLEY, P. G., CESCA, L., BRESEE, J., and CRANOR, L. F. Standardizing privacy notices: an online study of the nutrition label approach. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (New York, NY, USA, 2010), CHI '10, ACM, pp. 1573-1582.
51. KELLEY, P. G., and CRANOR., L. F., and SADEH, N. Privacy as part of the app decision-making process. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (New York, NY, USA, 2013), CHI '13, ACM, pp. 3393-3402.
52. KOBYLIŃSKI, L., AND PRZEPIÓRKOWSKI, A. Definition extraction with balanced random forests. In Proceedings of the 6th international conference on Advances in Natural Language Processing (Berlin, Heidelberg, Germany, 2008), GoTAL '08, Springer, pp. 237-247.
53. KRIPPENDORFF, K. Content analysis: An introduction to its methodology. SAGE, Beverly Hills, CA, USA, 1980.
54. KUHN, F. A description language for content zones of German court decisions. In Proceedings of the LREC 2010 Workshop on the Semantic Processing of Legal Texts (2010), SPLeT '10, pp. 1-7.
55. LEON, P. G., and CRANOR., L. F., MCDONALD, A. M., and MCGUIRE, R. Token attempt: The misrepresentation of website privacy policies through the misuse of P3P compact policy tokens. In Proceedings of the 9th Annual ACM Workshop on Privacy in the Electronic Society (New York, NY, USA, 2010), WPES '10, ACM, pp. 93-104.
56. MANNING, C. D., RAGHAVAN, P., AND SCHÜTZE, H. Introduction to Information Retrieval. Cambridge University Press, New York, NY, USA, 2008.
57. MAROTTA-WURGLER, F. Does contract disclosure matter? JITE 168, 1 (2012), 94-119.
58. MCDONALD, A. M., and REEDER., R. W., KELLEY, P. G., and CRANOR, L. F. A comparative study of online privacy policies and formats. In Proceedings of the 9th international symposium on Privacy Enhancing Technologies (Berlin, Heidelberg, Germany, 2009), PETS '09, Springer, pp. 37-55.
59. MOENS, M.-F., BOIY, E., and PALAU., R. M., and REED, C. Automatic detection of arguments in legal texts. In Proceedings of the 11th International Conference on Artificial Intelligence and Law (New York, NY, USA, 2007), ICAIL '07, ACM, pp. 225-230.
60. NATIONAL TELECOMMUNICATIONS and INFORMATION ADMINISTRATION. Short form notice code of conduct to promote transparency in mobile app practices. http://www.ntia.doc.gov/files/ntia/publications/july25codedraft.pdf, July 2013. Last accessed: July 1, 2014.
61. PASSONNEAU, R. Measuring Agreement on Set-valued Items (MASI) for semantic and pragmatic annotation. In Proceedings of the international Conference on Language Resources and Evaluation (2006), LREC '06.
62. PASSONNEAU, R. J., and CARPENTER, B. The benefits of a model of annotation. In Proceedings of the 7th Linguistic Annotation Workshop and Interoperability with Discourse (Stroudsburg, PA, USA, 2013), ACL, pp. 187-195.
63. PORTER, M. An algorithm for suffix stripping. Program: electronic library and information systems 14, 3 (1980), 130-137.
64. RAJARAMAN, A., and ULLMAN, J. D. Mining of massive datasets. Cambridge University Press, New York, NY, USA, 2012.
65. REEDER, R. W. Expandable Grids: a user interface visualization technique and a policy semantics to support fast, accurate security and privacy policy authoring. PhD thesis, Pittsburgh, PA, USA, 2008. AAI3321049.
66. REEDER, R. W., and KELLEY., P. G., MCDONALD, A. M., and CRANOR, L. F. A user study of the Expandable Grid applied to P3P privacy policy visualization. In Proceedings of the 7th ACM Workshop on Privacy in the Electronic Society (New York, NY, USA, 2008), WPES '08, ACM, pp. 45-54.
67. REIDENBERG, J. R. The use of technology to assure internet privacy: Adapting labels and filters for data protection. Lex Electronica 3, 2 (1997).
68. REIDSMA, D., and CARLETTA, J. Reliability measurement without limits. Comput. Linguist. 34, 3 (Sept. 2008), 319-326.
69. RUBINSTEIN, I. S. Privacy and regulatory innovation: Moving beyond voluntary codes. ISJLP 6, 3 (2011), 355-423.
70. STAMEY, J. W., and ROSSI, R. A. Automatically identifying relations in privacy policies. In 27th ACM International Conference on Design of Communication (New York, NY, USA, 2009), SIGDOC '09, ACM, pp. 233-238.
71. STEDE, M., and KUHN, F. Identifying the content zones of German court decisions. In Lecture Notes in Business Information Processing (Berlin, Heidelberg, Germany, 2009), Vol. 37 of BIS'09, Springer, pp. 310-315.
72. SYMANTEC CORPORATION. Complete privacy statement. http://web.archive.org/web/20131028120625/http://www.symantec.com/about/profile/policies/privacy.jsp. Last accessed: July 1, 2014.
73. SYMANTEC CORPORATION. Global privacy statement for Symantec. http://web.archive.org/web/19991012020231/http://symantec.com/legal/privacy.html. Last accessed: July 1, 2014.
74. TSOUMAKAS, G., and KATAKIS, I. Multi label classification: An overview. IJDWM 3, 3 (2007), 1-13.
75. WESTERHOUT, E. Definition extraction using linguistic and structural features. In Proceedings of the 1st Workshop on Definition Extraction (Stroudsburg, PA, USA, 2009), WDE '09, ACL, pp. 61-67.
76. WESTERHOUT, E. Extraction of definitions using grammar-enhanced machine learning. In Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop (Stroudsburg, PA, USA, 2009), EACL '09, ACL, pp. 88-96.
77. WU, G., GREENE, D., and CUNNINGHAM, P. Merging multiple criteria to identify suspicious reviews. In Proceedings of the Fourth ACM Conference on Recommender Systems (New York, NY, USA, 2010), RecSys '10, ACM, pp. 241-244.
78. YANG, J., YESSENOV, K., and SOLAR-LEZAMA, A. A language for automatically enforcing privacy policies. In Proceedings of the 39th annual ACM SIGPLAN-SIGACT symposium on Principles Of Programming Languages (New York, NY, USA, 2012), POPL '12, ACM, pp. 85-96.