A propensity score approach in the impact evaluation on scientific production in Brazilian biodiversity research: the BIOTA Program

Scientometrics

Volumn 101, Issue 1, 2014, Pages 85-107

  Colugnati, Fernando A B ^a,c   Firpo, Sergio ^b   de Castro, Paula F Drummond ^a   Sepulveda, Juan E ^a   Salles Filho, Sergio L M ^a  

^a UNIVERSITY OF CAMPINAS   (Brazil)

^b FUNDAÇÃO GETÚLIO VARGAS   (Brazil)

^c FEDERAL UNIVERSITY OF JUIZ DE FORA   (Brazil)

Author keywords

Bibliometrics; Biota program; Impact evaluation; Propensity score; Quasi experiment

Indexed keywords

EID: 84919832289     PISSN: 01389130     EISSN: 15882861     Source Type: Journal    
DOI: 10.1007/s11192-014-1397-1     Document Type: Article

References (42)

1. Abadie, A. (2005). Semiparametric difference-in-differences estimators. The Review of Economic Studies,72(1), 1–19.
2. Azoulay, P., Zivin, J. S. G., & Manso, G. (2011). Incentives and creativity: Evidence from the academic life sciences. RAND Journal of Economics,42(3), 527–554.
3. Böhmer, S., & von Ins, M. (2009). Different—Not just by label: Research-oriented academic careers in Germany. Research Evaluation,18(3), 177–184.
4. Carney, J., Smith, W., Parsad, A., Johnston, K., & Millsap, M. (2008). Evaluation of the Faculty Early Career Development (CAREER) Program. Bethesda, MD: Abt Associates Inc.
5. Chen, S., Mu, R., & Ravallion, M. (2008). Are there lasting impacts of aid to poor areas? Evidence for rural China. Policy Research Working Paper 4084, World Bank, Washington, DC.
6. Crump, R. K., Hotz, J., Imbens, W., & Mitinik, O. A. (2009). Dealing with limited overlap in estimation of average treatment effects. Biometrika,96(1), 187–199.
7. D’Agostino, R. B, Jr, & Rubin, D. B. (2000). Estimating and using propensity scores with partially missing data. Journal of the American Statistical Association,95, 749–759.
8. Ferraro, P. J. (2009). Counterfactual thinking and impact evaluation in environmental policy. New Directions for Evaluation 122 (Special Issue: Environmental Program and Policy Evaluation: Addressing Methodological Challenges).
9. Ferraro, P. J., & Pattanayak, S. K. (2006). Money for nothing? A call for empirical evaluation of biodiversity conservation investments. PLoS Biology,4(4), e105. doi:10.1371/journal.pbio.0040105.
10. Friedman, J. H. (2001). Greedy function approximation: A gradient boosting machine. Annals of Statistics,29(5), 1189–1232.
11. Frondel, M., & Schmidt, C. M. (2005). Evaluating environmental programs: The perspective of modern evaluation research. Ecological Economics,55(4), 515–526.
12. Hastie, T., Tibshirani, R., & Friedman, J. (2001). The elements of statistical learning. New York: Springer.
13. Hirano, K., Imbens, G. W., & Ridder, G. (2003). Efficient estimation of average treatment effects using the estimated propensity score. Econometrica,71, 1161–1189.
14. Hornbostel, S., Böhmer, S., Klingsporn, B., Neufeld, J., & von Ins, M. (2009). Funding of young scientist and scientific excellence. Scientometrics,79(1), 171–190.
15. Hoshino, T. (2008). A Bayesian propensity score adjustment for latent variable modeling and MCMC algorithm. Computational Statistics & Data Analysis,52(3), 1413–1429.
16. Ihaka, R., & Gentleman, R. (1996). R: A language for data analysis and graphics. Journal of Computational and Graphical Statistics,5(3), 299–314.
17. Inglesi-Lotz, R., & Pouris, A. (2011). Scientometric impact assessment of a research policy instrument: The case of rating researchers on scientific outputs in South Africa. Scientometrics,88(3), 747–760.
18. Jacob, B. A., & Lefgren, L. (2011). The impact of NIH postdoctoral training grants on scientific productivity. Research Policy,40, 864–874.
19. Jalan, J., & Ravallion, M. (2003). Estimating the benefit incidence of an antipoverty program by propensity-score matching. Journal of Business and Economic Statistics,21(1), 19–30.
20. Joly, C. A., Rodrigues, R. R., Metzger, J. P., Haddad, C. F. B., Verdade, L. M., Oliveira, M. C., et al. (2010). Biodiversity conservation research, training, and policy in São Paulo. Science,328, 1358–1359.
21. Khandker, S. R., Koolwal, G. B., & Samad, H. A. (2009). Handbook on impact evaluation quantitative methods and practices. Washington, DC: World Bank.
22. Liu, X., Zhang, L., & Hong, S. (2011). Global biodiversity research during 1900–2009: A bibliometric analysis. Biodiversity Conservation,20(807), 807–826.
23. Lynch, L., Gray, W., & Geoghegan, J. (2007). Are farmland preservation program easement restrictions capitalized into farmland prices? What can a propensity score matching analysis tell us? Applied Economic Perspectives and Policy,29(3), 502–509.
24. McCarey, D., Ridgeway, G., & Morral, A. (2004). Propensity score estimation with boosted regression for evaluating adolescent substance abuse treatment. Psychological Methods,9(4), 403–425.
25. Mena-Chalco, J. P., & Cesar, R. M, Jr. (2009). scriptLattes: An open-source knowledge extraction system from the Lattes platform. Journal of the Brazilian Computer Society,15(4), 31–39.
26. Mutz, R., & Daniel, H.-D. (2012). The generalized propensity score methodology for estimating unbiased journal impact factors. Scientometrics,92(2), 377–390.
27. Nelder, J., & Wedderburn, R. (1972). Generalized linear models. Journal of the Royal Statistical Society. Series A (General),135(3), 370–384.
28. Ravallion, M. (2008). Evaluating anti-poverty programs. In T. P. Schultz & J. Strauss (Eds.), Handbook of development economics (4th ed., pp. 3787–3846). Amsterdam: North-Holland.
29. Ridgeway, G. (1999). The state of boosting. Computing Science and Statistics,31, 172–181.
30. Ridgeway, G. (2005). GBM 1.5 package manual. http://cran.r-project.org/doc/packages/gbm.pdf.
31. Ridgeway, G. (2006). Assessing the effect of race bias in post-traffic stop outcomes using propensity scores. Journal of Quantitative Criminology,22(1), 1–29.
32. Ridgeway, G., McCaffrey, D., & Morral, A. (2006). twang: Toolkit for weighting and analysis of nonequivalent groups. Software for using matching methods in R. Available at http://cran.r-project.org/web/packages/twang/index.html.
33. Robins, J. M., & Rotnitzky, A. (2001). Comment on “inference for semiparametric models: Some questions and an answer” by P. Bickel and J. Kwon. Statistica Sinica,11, 920–936.
34. Rodrigues, R. R., & Bonomi, V. L. R. (Eds.). (2008). Diretrizes para conservação e restauração da biodiversidade no estado de São Paulo. São Paulo: SMA-SP & FAPESP.
35. Rosenbaum, P., & Rubin, D. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika,70(1), 41–55.
36. Rubin, D. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66(5), 688–701.
37. Rubin, D. (2001). Using propensity scores to help design observational studies: Application to the tobacco litigation. Health Services and Outcomes Research Methodology,2, 169–188.
38. Salles-Filho, S., Avila, F. D., Sepulveda, J., & Colugnati, F. A. B. (2010a). Multidimensional assessment of technology and innovation programs: The impact evaluation of INCAGRO-Peru. Research Evaluation,19(5), 361–372.
39. Salles-Filho, S., Bonacelli, M. B. M., Zackiewicz, M., Castro, P. F. D., & Bin, A. (2007). Development and application of a methodology for evaluating S,T&I Programs: The decomposition method. In Workshop Internacional sobre Inovações Metodológicas na Avaliação de Impacto da pesquisa Agropecuária, 2008, Brasília, XII Seminário Latino-Iberoamericano de Gestión Tecnológica - ALTEC 2007, Buenos Aires, 2007 (Vol. 1, pp. 2–6).
40. Salles-Filho, S., Castro, P. F. D., Bonacelli, M. B. M., Zeitoum, C., & Sá, F. P. (2010b). Indicators for evaluation of results and impacts of research program in conservation and sustainable use of biodiversity, the case of BIOTA/FAPESP. In International conference: Getting post 2010 biodiversity targets right, Bragança Paulista.
41. Stuart, E. A. (2010). Matching methods for causal inference: A review and a look forward. Statistical Science,25(1), 1–21.
42. White, H. (1980). A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica,48, 817–830.