Statutory interpretation from the inside: Methods appendix

Stanford Law Review

Volumn 65, Issue 5, 2013, Pages 1-52

  Gluck, Abbe R ^a   Bressman, Lisa Schultz ^b  

^a The Earth Institute at Columbia University   (United States)

^b VANDERBILT UNIVERSITY LAW SCHOOL   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84879775221     PISSN: 00389765     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (72)

1. Abbe R. Gluck & Lisa Schultz Bressman, Statutory Interpretation from the Inside-An Empirical Study of Congressional Drafting, Delegation, and the Canons: Part I, 65 Stan. L. Rev. 901, 919-923 (2013).
2. Specifically, we asked eighty-five questions, with fifteen questions containing three to ten subparts. Two of these questions (Q77A and Q32A) were added after the first batch of interviews took place, as discussed below.
3. Each time we asked this question, at least 70% respondents reported that we were asking the right questions about the topic. See Q19; Q28; Q40; Q60; Q64. Almost no respondents said we were not asking the right questions. Those who did not answer "yes" typically offered additional insights into aspects of the process about which we had not inquired. We do not include one of these questions (Q49) in this tally, because most of the comments were simply elaborations on the previous questions, or the introduction of new elements we had not inquired about, rather than "yes" or "no" answers.
4. Gluck & Bressman, supra note 1, at 920 n.57 (citing methodological sources and discussing this point in more detail).
5. Telephone conversation between Research Assistant Alexandra Golden and House Clerk's Office (Oct. 2011).
6. R. Eric Petersen et al., Cong. Research Serv., R41366, House of Representatives and Senate Staff Levels in Member, Committee, Leadership, and Other Offices, 1977-2010, at 4 (2010).
7. Examples of congressional staff responsibilities include: conducting policy research, processing and responding to constituent correspondence, arranging travel for members of Congress, communicating members' policies and positions to the media and the public at large, conducting legislative and investigative hearings, and performing administrative tasks, among others. For a brief overview of congressional staff duties.
8. John S. Pontius & Faye M. Bullock, Cong. Research Serv., 93-380 GOV, Congressional Staff: Duties and Functions (2003).
9. R. Eric Petersen et al., Cong. Research Serv., RL34545, Congressional Staff: Duties and Functions of Selected Positions 6 (2008) (quoting a 1996 Senate Handbook as stating, "[t]hroughout the Senate, individuals with the same job title perform vastly different duties").
10. For example, a survey conducted by the Congressional Research Service ("CRS"), the nonpartisan research arm of the legislative branch, found that for fourteen common staff titles, the House and Senate staff produced 118 alternative titles for the same positions. According to that survey, in 2008 there were 500 separate entities in the House that hired staff and 135 similar entities in the Senate, none of which is required to follow a uniform standard for creating and filling staff positions. Id. at 2.
11. A closer examination of current staff positions on one committee in the House and one committee in the Senate provides a useful illustration. The House Agriculture Committee, for instance, employs forty-three staff members. Staff on the Committee hold a variety of unique titles, including "Policy Director," "Science Advisor," "Chief Economic Advisor," and "Senior Policy Advisor," in addition to the more commonly used titles of "Professional Staff," "Legislative Assistant," and "Staff Director." In the Senate, the Budget Committee uses staff titles such as "Performance Budget Specialist," "Budget Analyst," "Healthcare Analyst," and "General Counsel" that are similarly inscrutable for purposes of determining which staffers draft legislation.
12. Frequently Asked Questions, LegiStorm, http://www.legistorm.com/salaries/faq.html (May 20, 2013) (noting the difficulty of parsing staffer titles).
13. On the role of committees in the House.
14. John V. Sullivan, Parliamentarian, U.S. House of Representatives, How Our Laws Are Made, H.R. Doc. No. 110-49, at 9 (2007).
15. Senate Rule XXVI(8)(a) describes the role of committees as assisting the Senate in "its formulation, consideration, and enactment of.legislation" through "review and study.[of] the subject matter of which is within the legislative jurisdiction of that committee." Senate Rule XXVI; Pontius & Bullock, supra note 7, at 2 (describing the role of committee staff as, in part, "draft[ing] and analyz[ing] legislation and amendments.[and] prepar[ing] legislation for reporting to the House or Senate."). In comparison, the role of members' personal office legislative staff as described by CRS is to "conduct research and analysis on policy issues and assist in devising and carrying out strategies for accomplishing the legislative goals of the Member." Id.
16. Petersen et al., supra note 6, at 2. Because there is no publicly available source of staffing data in Congress, CRS compiled information from House and Senate staff directories to determine staffing levels for member offices, committees, leadership offices, and other officials in the House and Senate. These data have their own limitations-for example, not all staff are included in the telephone directories, and some staff who are included no longer work in Congress-however, CRS has found that this to be the most accurate assessment possible of staffing levels. For 2009, CRS reports that there were 1362 committee staff employed in the House, and 1153 in the Senate. Id. at 18-24.
17. Gluck & Bressman, supra note 1, at 921-22 & n.62.
18. Some committees in the House and Senate are oversight and investigative bodies, and thus employ primarily investigative staff, including the House Committee on Oversight and Government Reform and the House Committee on Ethics, among others. Additionally, as each standing committee has oversight responsibility over agencies and activities within the committee's jurisdiction, investigative staff who are employed in this capacity are less likely to participate in the legislative drafting process than other staff.
19. Frederick M. Kaiser et al., Cong. Research Serv., RL30240, Congressional Oversight Manual 15 (May 1, 2007).
20. Leadership offices in the House and Senate, such as those of the Speaker and Majority Leader of the House and the Majority Leader of the Senate, work closely with committees to set the legislative calendar, and serve as spokespeople for their party conferences about the legislative agenda.
21. Art & History: Majority and Minority Leadership and Party Whips, U.S. Senate, http://www.senate.gov/artandhistory/history/common/briefing/Majority_Minority_Leaders.htm (last visited May 20, 2013).
22. House History: Majority & Minority Leaders, U.S. House of Representatives, http://artandhistory.house.gov/house_history/leaders.aspx (last visited May 20, 2013).
23. A substantial number of staff in member offices participate in the legislative drafting process. Discussions with Senate staff reveal that senators who introduce legislation independently (without the input of a committee) rely heavily on their own staff to draft the bill. In contrast, bills introduced by a senator in his capacity as a committee member are more likely to be drafted by committee staff.
24. Congressional Staff Salaries, LegiStorm, http://www.legistorm.com/salaries/aboutcs.html (last visited May 20, 2013).
25. We used the most recently available data, which in the Senate spanned October 1, 2010 to March 31, 2011, and in the House spanned April 1, 2011 to June 30, 2011. Because Appropriations Committee staff does not list titles in LegiStorm, our research assistants used the Almanac of the Unelected Staff of the U.S. Congress (Suzanne Struglinksi & Lisa Friedman eds., 23d ed. 2011) to determine how many counsels were on that committee. They also included seventeen committee staffers who did not currently have counsel titles, but had in the past, on the theory that they had moved to more supervisory roles (like staff director) but were still doing counsel work.
26. Congressional Yellow Book, Leadership Directories, http://www.leadershipdirectories.com/Products/LeadershipPrintDirectories/Government/CongressionalYellowBook (last visited May 20, 2013).
27. Some counsels were duplicated within a directory, likely because they had changed committees. We removed those duplicates that we could determine to a high probability were likely the same person, but left multiple names if the names were common (e.g., John Matthews).
28. E-mail from James V. Saturno, Research Manager, Legislative & Budget Process Section, Gov't & Fin. Div., Cong. Research Serv., to Research Assistant Mallory Jensen (Mar. 23, 2012) (suggesting that best sources of data were LegiStorm and the Yellow Book but also suggesting that the disbursement documents might be a good confirming device as well).
29. Statement of Disbursements, U.S. House of Representatives, http://disbursements.house.gov (last visited May 20, 2013).
30. Report of the Secretary of the Senate, U.S. Senate, http://www.senate.gov/legislative/common/generic/report_secsen.htm (last visited May 20, 2013) (online searchable disbursement reports for each house).
31. For instance, the Senate Select Committee on Ethics, the House Committee on Oversight and Government Reform, and the Permanent Subcommittee on Investigations. See also Memorandum from Research Assistant Giselle Barcia to Abbe R. Gluck (describing research finding no introduction of bills by those committees on the Library of Congress "Thomas" website, and phone calls to each committee confirming the observation).
32. Using the number 794 instead would be far too high to estimate the current population of counsels at any one moment. For example, that number reflects staffers who overlapped but replaced one another during the same quarterly pay period. None of the sources that compute staffing levels compute a committee counsel level higher than 686, including Legislative Counsels.
33. E-mail from James V. Saturno to Research Assistant Mallory Jensen, supra note 21.
34. One committee chief counsel had heard about our study and asked us to send the email through him. We also omitted those counsels who already had been snowballed into our sample by others. See infra notes 30-31 and accompanying text.
35. E-mail from Columbia-Vanderbilt Law Schools Legislative Research Project (Nov. 3, 2011).
36. For these individuals, we e-mailed them using a different e-mail address. For example, if the person's name was Matthew Feder, our initial e-mail would have been sent to matthew.feder@mail.house.gov. If that e-mail bounced back, on this second round, we tried a different version of the first name, usually a nickname, e.g., matt.feder@mail.house.gov.
37. Specifically, included in that third round was the entire list of counsels from one committee that we had omitted from our original e-mail because one chief counsel asked us to go through him to reach his committee and promised us a representative cross-sample of committee staff. However, after several follow-up emails to him went unanswered, we decided to include his whole committee in our next e-mail blast.
38. Two of our potential interviewees were current students at Columbia Law School, where one of us was on the faculty at that time, so we deemed them ineligible. Several others had left congressional service before our cut-off date (all respondents had to have served within the last two years to be eligible for the survey).
39. Of this number, twenty-nine were "snowballed" in this manner from fourteen of our respondents, and sixteen snowballed from people we did not interview because they were ineligible for the survey or, in the case of one counsel, referred us to a colleague instead.
40. With respect to the Office of Legislative Counsel in the House, the task of selecting a representative group from those who volunteered was delegated to another counsel in the office.
41. These six are included in the totals from the first and third round of e-mails.
42. We are unable to confirm the identity of the individual who "snowballed" one of our respondents. As it is probable that this snowballer was included in our sample, we have made that assumption for purposes of this diagram.
43. Q2. Six of the 107 described themselves as serving both on committee and personal staff. We had respondents from the following House committees: Judiciary, Rules, Small Business, Energy and Commerce, Ways and Means, Homeland Security, Oversight and Government Reform, Financial Services, Transportation and Infrastructure, Intelligence, Education and the Workforce, Science, Space and Technology, and House Administration. From the Senate, we interviewed staffers from the following committees: Judiciary, Health, Education, Labor and Pensions, Rules and Administration, Appropriations, Small Business and Entrepreneurship, Homeland Security and Government Affairs, Banking, Housing and Urban Affairs, Agriculture, Nutrition and Forestry, Finance, Energy and Natural Resources, Commerce, Science and Transportation,and Foreign Relations. We also interviewed respondents from the Joint Committee on Taxation.
44. Leadership staff work for personal members (the leaders of the houses) but have more drafting responsibilities than ordinary personal staff.
45. The Appropriations Committee is one example of a committee that does not utilize counsel titles.
46. Thirteen were involved in three to six pieces, two were involved in two or fewer, and three did not know. Q6.
47. We ultimately discovered one minor difference between our coding practices. See infra note 44 (explaining that, for a few questions, one of us accepted answers such as "I guess" and marked such answers as "other," while the other pressed respondents to classify the "I guess" response as "yes" or "no").
48. There was a minority of questions for which there were statistically significant differences (at 95% confidence) between surveys conducted by phone and in person, and between surveys conducted by each of the two authors. Using phone/in-person and coder as dependent variables, respectively, we conducted multinomial logit regressions. We controlled for Legislative Counsel and Judiciary, given the specialized nature of those two committees and the fact that each author did not interview equal numbers of respondents from those two groups. We conducted various iterations of these regressions, as we tested using the finite 650 population as a base as well as the super population assumption. Because the low level of responses for many of the questions could potentially cloud the analysis, we dropped all responses that had fewer than twenty respondents answering that specific answer choice, resulting in forty-one questions being dropped (excluding demographic Questions 1 to 10, the Legislative-Counsel only Questions 79 to 81, and questions in which we asked whether these "were the right questions to be asking" about the topic at issue).
49. With respect to in-person versus phone surveys, there were statistically significant differences at 95% for fourteen questions using the super population assumption and seventeen questions using the finite 650 population-only 8% and 10% of 171 questions, respectively. When comparing interviewers, we found differences for twenty-six questions using the super population assumption, and twenty-seven using the finite population of 650. While we cannot rule out that the method of investigation or the identity of the interviewer made a difference, there are other plausible explanations for the divergences. For example, the surveys conducted in person were all conducted during recess, when staffers were generally more relaxed. The surveys conducted by phone took place when Congress was in session and so some staffers were more rushed. In addition, although most Legislative Counsel and Judiciary committee respondents were interviewed in person, a greater share of respondents from those groups were interviewed over the phone than respondents who were not Legislative Counsel or members of the Judiciary committee, and those populations tended to respond differently on certain kinds of questions. (For example, 35% of Judiciary respondents were interviewed by phone compared to 27% for the rest.) As another example, one of us interviewed more Legislative Counsel than others and one interviewer also interviewed more over the phone than the other interviewer. Although these differences do not explain the difference for some questions, other differences might. In particular, a differential coding practice was likely responsible for some of these differences. For example, for Questions 45-47 (questions where we did see some differences), concerning the textual canons and which each had many subparts, many respondents gave answers on the order of "I suppose" or "I guess so," one of us simply transcribed that phrase, in which case the answer was coded as "other," whereas the other one of us asked respondents to clarify if they meant yes or no. As noted in the text, we tried to avoid such differences by ensuring that each survey was identical-read from a script and respondents were given all answers choices from which to make a selection-and by conducting the first batch jointly. However, we also recognize that there is always a risk with interviews that respondents' answers might be affected by the personality of the interviewer or how respondents wish to be perceived by a particular interviewer.
50. Herbert M. Kritzer, Stories from the Field: Collecting Data Outside Over There, in Practicing Ethnography in Law: New Dialogues, Enduring Methods 143, 154 (June Starr & Mark Goodale eds., 2002).
51. Specifically, seventy-six respondents were asked one version of the survey, and sixty-one were asked a different version. We did not see statistically significant differences (at 95% confidence) between the two versions using either the super population assumption or the finite 650 population assumption, except with regard to eight questions (after controlling for Legislative Counsel and Judiciary)-a difference of only 5%, which one would expect to see even occurring as a result of natural variation. See Q17; Q34; Q43; Q54; Q68a, c-d; Q77a. We did not scramble subparts within questions. For example, Q50 asked about drivers of ambiguity in legislation, and provided seven answer options (respondents could mark as many as they wished). When we created the second version of the survey, we did not scramble the order of the answer choices within questions of this nature. Further, many of the surveys conducted early in the process were in the chronological format and chronological surveys were also more likely to be conducted in person to a statistically significant degree.
52. In some instances, our administrative assistants did the first round of transcribing, but we each checked every transcription for accuracy.
53. In total, we had 4817 questions in which a respondent said something more than the answer choice. A few hundred of these, however, were small notes, like "I think so." We include these in the total number because line-drawing between substantive and nonsubstantive responses is difficult and does not affect our discussion of the data.
54. Viewing the population as having been drawn from some represented super population, whether an individual was "snowballed" is simply one characteristic of that individual, so we can test for whether that characteristic has a statistically significant effect on that individual's responses to the survey. As noted, we observed no such effect. Controlling for Legislative Counsel, using the super population assumption, the snowball method was significant at 95% in only two questions in the entire survey: Q45d and Q46d.
55. To test the null hypothesis that snowballers and others were not systemically different, we conducted additional statistical tests. To do so, we first compiled a subset of our survey questions that contained only binary responses (such as "yes" or "no"), and excluded all such questions where ten or more respondents answered "other" or "don't know" (45 out of 171 questions). We then suppressed these "other" or "don't know" responses, and used only the dichotomous responses (i.e., the "yes" or "no" responses). We did this in order to be able to perform regression analyses using Ordinary Least Squares (OLS). We then randomly selected twenty survey questions from this subset in order to mitigate problems of multiple inference.
56. In order to test whether the individuals who recommended other participants ("snowballers") varied systematically from the rest of our sample, we ran OLS regressions, using the super population approach, on all twenty questions in our sub-sample simultaneously, without controls and found that the coefficient on the "snowballer" indicator was statistically significant for only one of these questions at a 95% significance level. Given that a 95% significance level implies that there is a 5% chance of a "false positive" result, the fact that we found that the coefficient on one of these regressions was significant is not strong evidence that "snowballer" was affecting our results. Recognizing that our Legislative Counsel respondents often gave responses that were different from the rest of our sample, we redid this analysis, omitting Legislative Counsels, and again found that only one question was significant at 95%. (Due to the limitations of Stata, we were unable to conduct a similar simultaneous analysis using the finite population of 650.) In order to ensure that non-random external "snowballees" (individuals who were recommended by people who did not themselves participate in the survey) were not polluting our results, we also conducted a third analysis in which we excluded these respondents. Our results in this second analysis were very similar to the first. Again, we found that the coefficient on the "snowballer" indicator was statistically significant for one of the questions at a 95% significance level, again using the super population approach. We therefore do not reject the null hypothesis that respondents who recommended additional respondents are indistinguishable from the rest of our sample.
57. We then conducted a fourth analysis in which we removed all snowballees (including those who were recommended by individuals who did participate in the survey). In this analysis, we found that the coefficient on "snowballer" was not significant at 95% for any of the questions. Finally, we conducted the analysis a fifth time, this time excluding all snowballees and the remaining Legislative Counsels. Again, we found that the coefficient on "snowballer" was not significant at 95% for any of the questions. Despite these results, we recognize that fully understanding the effects of the "snowball" effect on our sample would require even more, and more sophisticated, testing. We also note that we are unable to confirm the identity of the individual who "snowballed" one of our respondents. While it is probable that this snowballer was included in our sample, it is possible that we have omitted one snowballer from our analysis, and counted that individual among the "non-snowballer" respondents.
58. We also attempted, but were unable, to determine whether there were any relationships between the answers offered by a snowballer respondent and his/her particular snowballee (i.e., did people recommend others who were "like them"?). Unfortunately, there was not enough variation in our data to meaningfully test this hypothesis: most snowballers referred only a very small number snowballees (often just one). The largest group was the nine House Legislative Counsels who were snowballed by their snowballer-boss. Since our Legislative Counsel respondents already vary systematically from our non-Legislative Counsel respondents, we concluded that an analysis of this particular snowball group could lead to misleading results.
59. For example, those who responded might be more interested in interpretive methodology than others.
60. Westlaw Profiler did not contain information on most of the individuals in either the control or the survey sample, and was generally more useful to confirm and supplement results found on LegiStorm than as a primary data source. For individuals with records in both databases, we found no differences in the data reported with respect to the factors we investigated. The exception was for Legislative Counsels, about whom we were able to gather more information using Westlaw Profiler than we were on LegiStorm. In particular, we were able to supplement the Legislative Counsels' educational information by using Westlaw.
61. To compare the groups, we used chi square goodness of fit tests and then supplemented the analysis by doing t-tests, where appropriate. The results of the analyses done each of these ways was substantially the same for most of the data below.
62. For example, the House Agriculture, House Science and Technology, Senate Foreign Relations, the Senate Special Committee on Aging and Senate Veterans Affairs. House Budget, House Armed Services, House Administration, House Foreign Affairs, Senate Agriculture, Nutrition and Forestry, and Senate Rules and Administration all have six.
63. Statement of Disbursements of the House, as Compiled by the Chief Administrative Officer, from October 1, 2011 Through December 31, 2011, available at http://disbursements.house.gov/2011q4/2011q4_singlevolume.pdf.
64. E-mail from James Fransen, Legislative Counsel for the U.S. Senate, to Abbe R. Gluck (Apr. 19, 2012).
65. E-mail from James Fransen, Legislative Counsel for the U.S. Senate, to Lisa Schultz Bressman (Mar. 23, 2012).
66. E-mail from James Fransen to Abbe R. Gluck, supra note 55 (including letterhead illustrating counsel seniority).
67. E-mail from Megan Renfrew, Assistant Counsel, Office of Legislative Counsel, U.S. House of Representatives, to Abbe R. Gluck (Apr. 11, 2012).
68. E-mail from Megan Renfrew, Assistant Counsel, Office of Legislative Counsel, U.S. House of Representatives, to Abbe R. Gluck (Jan. 23, 2013).
69. Memorandum from Giselle Barcia, supra note 23.
70. Excluding Questions 1 to 10 and 82, which were all demographic questions, and Questions 78 and 83, which asked respondents what they would change if they could about the process and whether there was anything they wished to add. These figures also combine never and rarely responses and, separately, always and often responses when those options were offered.
71. Lisa Schultz Bressman & Abbe R. Gluck, Statutory Interpretation from the Inside-An Empirical Study of Congressional Drafting, Delegation, and the Canons: Part II, 66 Stan. L. Rev. (forthcoming 2014).
72. The e-mail was sent from a special joint Gluck-Bressman account at Vanderbilt Law School that was created and used solely for this project. Gluck was on the faculty at Columbia Law School at the time the survey was conducted.