Valuable patents redux: On the enduring merit of using patent characteristics to identify valuable patents

Texas Law Review

Volumn 85, Issue 7, 2007, Pages 1769-1797

  Allison, John R ^a,b   Sager, Thomas W ^a  

^a UNIVERSITY OF TEXAS AT AUSTIN   (United States)

^b The University of Texas at Austin

Author keywords

[No Author keywords available]

Indexed keywords

EID: 34547821917     PISSN: 00404411     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Note

References (73)

1. John R. Allison et al., Valuable Patents, 92 GEO. L.J. 435 (2004).
2. Id. at 437.
3. Id. at 445-46.
4. See id. at 446 ("[T]he large population study reveals a few facts about an extremely large number of patents.").
5. Id.
6. Id. at 451-55. When speaking of value, we refer to private value - the value of patents to their owners - and not to social value. Moreover, we, like almost all others, seek to identify patents that have stand-alone value. We do not attempt to identify patents that may have little or no value by themselves but that may have value in contributing to a patent portfolio in which the whole is greater than the sum of its parts. See, e.g., Gideon Parchomovsky & R. Polk Wagner, Patent Portfolios, 154 U. PA. L. REV. 1 (2005) (advancing a theory of patent value based on the aggregation of individual patents into a collection of patents).
7. Allison et al., supra note 1, at 460.
8. Id. at 461.
9. David E. Adelman & Kathryn L. DeAngelis, Patent Metrics: The Mismeasure of Innovation in the Biotech Patent Debate, 85 TEXAS L. REV. 1677 (2007).
10. Id. at 1708.
11. Id. at 1708-09.
12. Id. at 1719.
13. Id. at 1725-27.
14. Id. at 1723.
15. Allison et al., supra note 1, at 466.
16. See Bronwyn H. Hall et al., The NBER Patent Citations Data File: Lessons, Insights and Methodological Tools (Nat'l Bureau of Econ. Research, Working Paper No. 8498, 2001), available at http://www.nber.org/papers/ w8498 (describing the database of U.S. patents developed by the National Bureau of Economics).
17. See Zvi Griliches, Patent Statistics as Economic Indicators: A Survey, 28 J. ECON. LITERATURE 1661, 1680 (1990) (using patent-renewal rates to determine that "the majority of patents are either of low value, or that their value depreciates (obsoletes) rapidly, or both").
18. Adelman & DeAngelis, supra note 9, at 1724-25; see also F.M. Scherer, Firm Size, Market Structure, Opportunity, and the Output of Patented Inventions, 55 AM. ECON. REV. 1097, 1098 (1965); F.M. Scherer & Dietmar Harhoff, Technology Policy for a World of Skew-Distributed Outcomes, 19 RES. POL'Y 559, 559 (2000).
19. F.M. Scherer, The Innovation Lottery, in EXPANDING THE BOUNDARIES OF INTELLECTUAL PROPERTY: INNOVATION POLICY OF THE KNOWLEDGE SOCIETY 3, 15-19 (Rochelle Cooper Dreyfuss et al. eds., 2001).
20. Scherer & Harhoff, supra note 18, at 562-63, 565.
21. Allison et al., supra note 1, at 447 n.45.
22. Some would call this "bivariate" analysis because it relates an individual predictor to the dummy variable that distinguishes litigated from unlitigated patents. Id.
23. Id.
24. Adelman & DeAngelis, supra note 9, at 1709 n.138.
25. For the exponential family and sufficiency of the mean of a transformation of the data, see ROBERT V. HOGG & ALLEN T. CRAIG, INTRODUCTION TO MATHEMATICAL STATISTICS 333-35 (5th ed. 1995).
26. Allison et al., supra note 1, at 447 n.45.
27. Id.
28. Id. at 446-47, 447 n.45.
29. A stochastic shift means that all percentiles of one distribution exceed corresponding percentiles of the other. For example, if the fiftieth percentile of A is 17 then the fiftieth percentile of B might be 20 - there is some nonnegative gap for all percentiles that is consistently in favor of B. See JAROSLAV HÁJEK, A COURSE IN NONPARAMETRIC STATISTICS 32 (1969) (providing the definition of a stochastic shift); see also SHELDON M. ROSS, STOCHASTIC PROCESSES 404-05 ex. 9.1.1 (2d ed. 1996) (providing a definition to describe when a randomly distributed variable is larger than another randomly distributed variable).
30. Allison et al., supra note 1, at 447 n.45.
31. Allison et al. did not specifically state that small amounts were added to zero values before logging because this is such a commonly accepted practice.
32. See HÁJEK, supra note 29, at 52-55 (providing the Wilcoxon test).
33. See id. at 56-59 (providing the Median test).
34. See id. at 62-65 (providing the Kolmogorov-Smirnov test).
35. For an explanation of the variables used in this table and in the other tables in this Commentary, see the Appendix.
36. One can test for equality of variances with an F-test. See J. SUSAN MLLTON ET AL., INTRODUCTION TO STATISTICS 382-84 (1997) (discussing the use of the F-distribution in comparing variances). With our data, however, it makes little difference because the p-values are very similar with or without the assumption of group variance equality.
37. We refer to total small-entity owners, which is a combination of individual, small business, and nonprofit owners.
38. See Allison et al., supra note 1, at 445-48 (describing the analytical techniques used on the large study).
39. Adelman & DeAngelis, supra note 9, at 1715 n.171.
40. See id. ("Accordingly, if one is uncertain of the magnitude or nature of the effect, findings of statistical significance may have no physical or practical meaning.").
41. For an explanation of these variables, see the Appendix.
42. "AppTime" refers to prosecution time - the time that the patent application spent in the patent office before issuance. However, the NBER database used for the large population study, which Adelman and DeAngelis also used, only reports the filing date of the application that led immediately to the patent in question, and does not include the time from an original application for the very substantial percentage of patents for which continuation applications were filed after the original - a more meaningful measure that we used in the sample study.
43. See MICHAEL PATRICK ALLEN, UNDERSTANDING REGRESSION ANALYSIS 189 (1997) ("Logistic regression analysis was developed precisely for the purpose of estimating the parameters of regression models with binary dependent variables.").
44. See Allison et al., supra note 1, app. at 479 tbl.5.
45. See JOSEPH F. HAIR ET AL., MULTIVARIATE DATA ANALYSIS 318-20 (5th ed. 1998) (presenting a common version of a pseudo-R-square).
46. See DAVID W. HOSMER & STANLEY LEMESHOW, APPLIED LOGISTIC REGRESSION 156-60 (2d ed. 2000) (discussing classification tables at some length).
47. The user is free to set a cutoff probability at whatever level the user thinks appropriate for the objective at hand. Since different users may have different objectives, it is not surprising that different cutoffs may be used with the same data. Most of the patents with estimated probabilities of litigation close to 1 really are litigated; similarly, most of the patents with estimated probabilities of litigation close to 0 really are not litigated. Patents with probabilities in the middle of the range are more difficult to classify correctly. An investor seeking a small set of patents rich in value may well choose a block at the high end by setting a high cutoff. For triage purposes, a low cutoff probability could be used to select a block at the low end for deferred action on a lot of patents unlikely to have value. The 0.6 cutoff of this illustration was selected so that the proportion of patents classified as litigated would roughly match the hypothetical proportion of valuable patents in the reweighted analysis below.
48. See SAS INST., 2 SAS/STAT USER'S GUIDE, VERSION 6 1092 (4th ed. 1990) (defining sensitivity, specificity, false positive, and false negative through the use of an example).
49. Preemptive defensive strategies could include seeking reexamination of certain patents, modifying their own products, redirecting some of their research and development investments, and so on.
50. See Mark A. Lemley, Rational Ignorance at the Patent Office, 95 Nw. U. L. REV. 1495, 1507 (2001) ("I suspect the total number of patents litigated or licensed for a royalty (as opposed to cross-license) is on the order of five percent of issued patents.").
51. Setting the cutoff at 0.25 results in approximately equal sensitivity and specificity for these data, as reweighted.
52. See THE RANDOM HOUSE DICTIONARY OF THE ENGLISH LANGUAGE 347 (Jess Stein et al. eds., 1973) (defining cross-validation as "a process by which a method that works for one sample of a population is checked for validity by applying the method to another sample from the same population").
53. The jackknife is a general method for reducing bias in statistics. See Christopher Z. Mooney & Robert D. Duval, Bootstrapping: A Nonparametric Approach to Statistical Inference, in QUANTITATIVE APPLICATIONS IN THE SOCIAL SCIENCES, at 22-23 (Sage Univ., Paper Series on Quantitative Applications in the Social Sciences, Series No. 07-095, 1993) (discussing the use of the jackknife method). For assessing possible overfitting in logistic regression, the jackknife will: remove the trial [of a patent] to be classified from the data, re-estimate the parameters of the model, and then classify the trial based on the new parameter estimates. However, it would be too costly to repeat the estimation every time a trial is removed. The LOGISTIC procedure provides a one-step approximation to obtain new parameter estimates. SAS INST., supra note 48, at 1092.
54. See Allison et al., supra note 1, at 446 (comparing the extrapolative value of the large population study with the sample study).
55. Id. at 464.
56. Adelman and DeAngelis also briefly criticized an earlier study by Allison and Lemley that used various patent characteristics to produce findings about who gets U.S. patents, in which technology areas they get them, and a number of other variables. See Adelman & DeAngelis, supra note 9, at 1714-16 (criticizing John R. Allison & Mark A. Lemley, Who's Patenting What? An Empirical Exploration of Patent Prosecution, 53 VAND. L. REV. 2099 (2000)). Their criticisms of that article were very similar to their criticisms of Valuable Patents. Here, we do not comment on their passing critique of that article. If time and space allowed, we would demonstrate that those criticisms are at least as misplaced as their criticisms of Valuable Patents, and probably even more so because there was no potential base-rate problem in that study.
57. Allison et al., supra note 1, app. at 478 tbl.4.
58. Id. app. at 479 tbl.5.
59. Adelman & DeAngelis, supra note 9, at 1720 tbl.2.
60. Id. at 1721 tbl.3.
61. See MICHAEL O. FLNKELSTEIN & BRUCE LEVIN, STATISTICS FOR LAWYERS 458-60 (2d ed. 2001) (explaining some differences between linear models and logistic regression).
62. Adelman & DeAngelis, supra note 9, at 1720 n.201 (emphasis added).
63. Id. at 1720 n.203.
64. See FINKELSTEIN & LEVIN, supra note 61, at 448-49 (explaining some differences between linear models and logistic regression).
65. Adelman & DeAngelis, supra note 9, at 1709 n.138.
66. Id. at 1715 n.168.
67. Id. at 1715 n.170.
68. Id. at 1715 n.171 (emphasis added).
69. Id. (emphasis added).
70. Id. at 1722 n.209.
71. Id. at 1723 n.214 (citation omitted).
72. Id. at 1723 n.217.
73. See FINKELSTEIN & LEVIN, supra note 61, at 114-15 (explaining central limit theorems).