Measuring systems sustainability with multi-criteria methods: A critical review

European Journal of Operational Research

Volumn 258, Issue 2, 2017, Pages 607-616

  Diaz Balteiro, L ^a   Gonzalez Pachon J ^a   Romero, C ^a  

^a UNIVERSIDAD POLITÉCNICA DE MADRID   (Spain)

Author keywords

Aggregation; Criteria; Indicators; Multiple criteria analysis; Sustainability

Indexed keywords

AGGLOMERATION; INDICATORS (INSTRUMENTS); SUSTAINABLE DEVELOPMENT;

ANALYTIC HIERARCHICAL PROCESS; CRITERIA; CRITERIA AND INDICATORS; GROUP DECISION MAKING; HYBRIDIZATION PROCESS; MULTIPLE CRITERIA ANALYSIS; MULTIPLE CRITERIA DECISION MAKING; SUSTAINABILITY CRITERIA;

DECISION MAKING;

EID: 85006154080     PISSN: 03772217     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ejor.2016.08.075     Document Type: Article

References (116)

1. Adrianto, L., Matsuda, Y., Sakuma, Y., Assessing local sustainability of fisheries system: A multi-criteria participatory approach with the case of Yoron Island, Kagoshima prefecture, Japan. Marine Policy 29 (2005), 9–23.
2. Afgan, N.H., Begic, F., Kazagic, A., Multi-criteria sustainability assessment: A tool for evaluation of new energy system. Thermal Science 11 (2007), 43–53.
3. Afgan, N.H., Darwish, M., Multi-criteria sustainability assessment of water desalination and energy systems – Kuwait case. Desalination and Water Treatment 25 (2011), 241–250.
4. Arce, M.E., Saavedra, A., Míguez, J.L., Granada, E., The use of grey-based methods in multi-criteria decision analysis for the evaluation of sustainable energy systems: A review. Renewable & Sustainable Energy Reviews 47 (2015), 924–932.
5. Aspen, D.M., Sparrevik, M., Fet, A.M., Review of methods for sustainability appraisals in ship acquisition. Environment Systems and Decisions 35 (2015), 323–333.
6. Atkisson, A., Sustainability is dead – long live sustainability. Keiner, M., (eds.) The future of sustainability, 2006, Springer, Dordrecht, The Netherlands, 231–243.
7. Azapagic, A., Perdan, S., An integrated sustainability decision-support framework Part II: Problem analysis. International Journal of Sustainable Development & World Ecology 12 (2005), 112–131.
8. Ballestero, E., Romero, C., Multiple criteria decision making and its applications to economic problems. 1998, Kluwer Academic Publishers, Boston.
9. Banker, R.D., Charnes, A., Cooper, W.W., Some models for estimating technical and scale inefficiencies in data envelopment analysis. Management Science 30 (1984), 1078–1092.
10. Blancas, F.J., Caballero, R., González, M., Lozano-Oyola, M., Pérez, F., Goal programming synthetic indicators: An application for sustainable tourism in Andalusian coastal counties. Ecological Economics 69 (2010), 2158–2172.
11. Blancas, F., González, M., Lozano-Oyola, M., Pérez, F., The assessment of sustainable tourism: Application to Spanish coastal destinations. Ecological Indicators 10 (2010), 484–492.
12. Bouyssou, D., Marchant, T., Pirlot, M., Tsoukiàs, A., Vincke, P., Evaluation and decision models with multiple criteria. 2006, Springer, New York.
13. Brans, J.P., Vincke, P., Mareschal, B., How to select and how to rank projects: The PROMETHEE methods. European Journal of Operational Research 24 (1986), 228–238.
14. Brunner, N., Starkl, M., Decision aid systems for evaluating sustainability: A critical survey. Environmental Impact Assessment Review 24 (2004), 441–469.
15. Büyüközkan, G., Çifçi, G., A novel fuzzy multi-criteria decision framework for sustainable supplier selection with incomplete information. Computers & Industrial Engineering 62 (2011), 164–174.
16. Caradonna, J.L., Sustainability: A history. 2014, Oxford University Press, New York.
17. Charnes, A., Cooper, W.W., Rhodes, E., Measuring the efficiency of decision making units. European Journal of Operational Research 2 (1978), 429–444.
18. Cinelli, M., Coles, S.R., Kirwan, K., Analysis of the potentials of multi criteria decision analysis methods to conduct sustainability assessment. Ecological Indicators 46 (2014), 138–148.
19. Cohon, J.L., Multiobjective programming and planning. 1978, Academic Press, New York.
20. Cooper, W.W., Origins, use, and relations between goal programming and data envelopment analysis. Journal of Multi-Criteria Decision Analysis 13 (2005), 3–11.
21. Cullen, J.G., Educating business students about sustainability: A bibliometric review of current trends and research needs. Journal of Business Ethics, 2016, 1–11, 10.1007/s10551-015-2838-3.
22. De Felice, F., Petrillo, A., Multi-criteria decision making: A mechanism design technique for sustainability. Luo, Z., (eds.) Mechanism design for sustainability: Techniques and cases, 2013, Springer, Dordrecht, 15–35.
23. Diaz-Balteiro, L., Romero, C., Making forestry decisions with multiple criteria: A review and an assessment. Forest Ecology and Management 255 (2008), 3222–3241.
24. Diaz-Balteiro, L., González-Pachón, J., Romero, C., Forest management with multiple criteria and multiple stakeholders: An application to two public forests in Spain. Scandinavian Journal of Forest Research 24 (2009), 87–93.
25. Diaz-Balteiro, L., González-Pachón, J., Romero, C., Goal programming in forest management: Customising models for the decision-maker's preferences. Scandinavian Journal of Forest Research 28 (2013), 166–173.
26. Egilmez, G., Gumus, S., Kucukvar, M., Environmental sustainability benchmarking of the U.S. and Canada metropolis: An expert judgment-based multi-criteria decision making approach. Cities 42 (2015), 31–41.
27. Elsevier. Scopus Website. (2015). http://www.elsevier.com/solutions/scopus (last accessed 10/09/2015).
28. Erol, I., Sencer, S., Sari, R., A new fuzzy multi-criteria framework for measuring sustainability performance of a supply chain. Ecological Economics 70 (2011), 1088–1100.
29. European Commission (2015). Statistical Classification of Economic Activities in the European Community, Rev. 2 (2008) (NACE Rev. 2). http://ec.europa.eu/eurostat/ramon/nomenclatures/index.cfm?TargetUrl=LST_CLS_DLD&StrNom=NACE_REV2&StrLanguageCode=EN&StrLayoutCode= (last accessed 10/12/2005).
30. Figueira, J., Greco, S., Ehrgott, M., Multicriteria decision analysis. State of the art surveys, 2005, Springer, Boston.
31. Foxon, T.J., McIlkenny, G., Gilmour, D., Oltean-Dumbrava, C., Souter, N., Ashley, R., et al. Sustainability criteria for decision support in the UK water industry. Journal of Environmental Planning and Management 45 (2010), 285–301.
32. Gómez-Limón, J., Sanchez-Fernandez, G., Empirical evaluation of agricultural sustainability using composite indicators. Ecological Economics 69 (2010), 1062–1075.
33. González-Pachón, J., Romero, C., Aggregation of ordinal and cardinal preferences: A framework based on distance functions. Journal of Multi-Criteria Decision Analysis 15 (2009), 79–85.
34. Gonvindan, K., Rajendran, S., Sarkis, J., Murugesan, P., Multi criteria decision making approaches for green supplier evaluation and selection: A literature review. Journal of Cleaner Production 98 (2015), 66–83.
35. Greco, S., Matarazzo, B., Słowiński, R., The use of rough sets and fuzzy sets in MCDM. Gal, T., Stewart, T., Hanne, T., (eds.) Multicriteria decision making, 1999, Springer Science+Business Media, New York, 397–455.
36. Greco, S., Matarazzo, B., Słowiński, R., Rough sets theory for multicriteria decision analysis. European Journal of Operational Research 129 (2001), 1–47.
37. Hacatoglu, K., Dincer, I., Rosen, M.A., A new model to assess the environmental impact and sustainability of energy systems. Journal of Cleaner Production 103 (2015), 211–218.
38. Hajkowicz, S., Collins, K., A review of multiple criteria analysis for water resource planning and management. Water Resources Management 21 (2007), 1553–1566.
39. Halog, A., Manik, Y., Advancing integrated systems modelling framework for life cycle sustainability assessment. Sustainability 3 (2011), 469–499.
40. Herrera, F., Herrera-Viedma, E., Chiclana, F., Multiperson decision-making based on multiplicative preference relations. European Journal of Operational Research 129 (2001), 372–385.
41. Herva, M., Roca, E., Review of combined approaches and multi-criteria analysis for corporate environmental evaluation. Journal of Cleaner Production 39 (2013), 355–371.
42. Hiltunen, V., Kangas, J., Pykäläinen, J., Voting methods in strategic forest planning – Experiences from Metsähallitus. Forest Policy Economics 10 (2008), 117–127.
43. Hovanov, N., Analysis and synthesis of parameters under information deficiency (in Russian). 1996, St. Petersburg State University Press St, Petersburg, Russia.
44. Huth, A., Drechsler, M., Köhler, P., Using multicriteria decision analysis and a forest growth model to assess impacts of tree harvesting in Dipterocarp lowland rain forests. Forest Ecology and Management 207 (2005), 215–232.
45. Hwang, C.L., Yoon, K., Multiple attribute decision making: Methods and applications. 1981, Springer-Verlag, New York.
46. Ibáñez-Forés, V., Bovea, M.D., Pérez-Belis, V., A holistic review of applied methodologies for assessing and selecting the optimal technological alternative from a sustainability perspective. Journal of Cleaner Production 70 (2014), 259–281.
47. Ignizio, J.P., Goal programming and extensions. 1976, Lexington Books, Massachusetts.
48. Ignizio. Generalized goal programming. An overview. Computers & Operations Research 10 (1983), 277–289.
49. Jalilova, G., Khadka, C., Vacik, H., Developing criteria and indicators for evaluating sustainable forest management: A case study in Kyrgyzstan. Forest Policy Economics 21 (2012), 32–43.
50. Janeiro, L., Patel, M.K., Choosing sustainable technologies. Implications of the underlying sustainability paradigm in the decision-making process. Journal of Cleaner Production 105 (2015), 438–446.
51. Jato-Espino, D., Castillo-Lopez, E., Rodriguez-Hernandez, J., Canteras-Jordana, J.C., A review of application of multi-criteria decision making methods in construction. Automation in Construction 45 (2014), 151–162.
52. Jones, D.F., Tamiz, M., Practical goal programming. 2010, Springer, New York.
53. Joro, T., Korhonen, P., Wallenius, J., Structural comparison of data envelopment analysis and multiple objective linear programing. Management Science 44 (1998), 962–970.
54. Keeney, R.L., Raiffa, H., Decisions with multiple objectives: Preferences and value trade-offs. 1976, John Wiley and Sons, New York.
55. Lai, E., Lundie, S., Ashbolt, N.J., Review of multi-criteria decision aid for integrated sustainability assessment of urban water systems. Urban Water Journal 5 (2008), 315–327.
56. Li, Y., Shi, Y., Qureshi, S., Bruns, A., Zhu, X., Applying the concept of spatial resilience to socio-ecological systems in the urban wetland interface. Ecological Indicators 42 (2014), 135–146.
57. Prete, L., Hobbs, C., Norman, B.F., Cano-Andrade, C.S., Fuentes, S., Spakovsky, A., et al. Sustainability and reliability assessment of microgrids in a regional electricity market. Energy 41 (2012), 192–202.
58. Lopez-Ridaura, S., Van Keulen, H., van Ittersum, M.K., Leffelaar, P.A., Multi-scale sustainability evaluation of natural resource management systems: Quantifying indicators for different scales of analysis and their trade-offs using linear programming. International Journal of Sustainable Development & World Ecology 12 (2005), 81–97.
59. Malczewski, J., Rinner, C., Multicriteria decision analysis in geographic information science. 2015, Springer, New York.
60. Mardani, A., Abbas, A., Zavadskas, E.K., Fuzzy multiple criteria decision-making techniques and applications – Two decades review from 1994 to 2014. Expert System with Applications 42 (2015), 4126–4148.
61. Martin, L., Incorporating values into sustainability decision-making. Journal of Cleaner Production 105 (2015), 146–156.
62. Maxim, A., Sustainability assessment of electricity generation technologies using weighted multi-criteria decision analysis. Energy Policy 65 (2014), 284–297.
63. Medel-González, F., García-Ávila, L., Acosta-Beltrán, A., Hernández, C., Measuring and evaluating business sustainability: Development and application of corporate index of sustainability performance. Erechtchoukova, E., Khaiter, P.A., Golinska, P., (eds.) Sustainability appraisal: Quantitative methods and mathematical techniques for environmental performance evaluation, 2013, Springer-Verlag, Berlin, 33–61.
64. Mendoza, G.A., Prabhu, R., Multiple criteria decision making approaches to assessing forest sustainability using criteria and indicators: A case study. Forest Ecology and Management 131 (2000), 107–126.
65. Michalopoulos, T., Hogeveen, H., Heuvelink, E., Public multi-criteria assessment for societal concerns and gradual labelling. Food Policy 40 (2013), 97–108.
66. Munda, G., Multicriteria evaluation in a fuzzy environment. 1995, Physica-Verlag, Heidelberg Contributions to economics series.
67. Munda, G., Measuring sustainability: A multicriterion framework. Environment, Development and Sustainability 7 (2005), 117–134.
68. Munda, G., Social multi-criteria evaluation for a sustainable economy. 2008, Springer-Verlag, Berlin.
69. Munda, G., Nardo, M., Saisana, M., Srebotnjak, T., Measuring uncertainties in composite indicators of sustainability. International Journal of Environmental Technology and Management 11 (2009), 7–26.
70. Munier, N., Introduction to sustainability. Road to a better future, 4th ed., 2005, Springer, Dordrecht, The Netherlands.
71. Neumayer, E., Weak versus strong sustainability. 2013, Edward Elgar, Cheltenham, UK, 22–29.
72. OECD, Organization for Economic Co-operation and Development. Handbook on constructing composite indicators. Methodology and user guide, 2008, OECD, Paris.
73. Oltean-Dumbrava, C., Watts, G., Miah, A., Transport infrastructure: Making more sustainable decisions for noise reduction. Journal of Cleaner Production 42 (2013), 58–68.
74. Opricovic, S., Multicriteria optimization in civil engineering (in Serbian). Faculty of civil engineering, 1998, Građevinski fakultet Univerziteta Belgrade.
75. Özdemir, E.D.Ã., Härdtlein, M., Jenssen, T., Zech, D., Eltrop, L., A confusion of tongues or the art of aggregating indicators – Reflections on four projective methodologies on sustainability measurement. Renewable & Sustainable Energy Reviews 15 (2011), 2385–2396.
76. Patel, A.D., Meesters, K., Uil, denH., de Jong, E., Blok, K., Patel, M.K., Sustainability assessment of novel chemical processes at early stage: Application to biobased processes. Energy & Environmental Science 5 (2012), 8430–8444.
77. Pérez, V., Hernández, A., Guerrero, F., León, M.A., Silva, C.L., Caballero, R., Sustainability ranking for Cuban tourist destinations based on composite indexes. Social Indicators Research 129:1 (2016), 425–444.
78. Pohekar, S.D., Ramachandran, M., Application of multi-criteria decision making to sustainable energy planning – A review. Renewable & Sustainable Energy Reviews 8 (2004), 365–381.
79. Pretzsch, J., Paradigms of tropical forestry in rural development. Pretzsch, J., Darr, D., Uibrig, H., Auch, E., (eds.) Forests and rural development, 2014, Springer, Heidelberg, 7–49.
80. Raison, R.J., Brown, A., Flinn, D., Criteria and indicators for sustainable forest management. 2001, CABI Publishing, Wallingford, UK.
81. Reisi, M., Aye, L., Rajabifard, A., Ngo, T., Transport sustainability index: Melbourne case study. Ecological Indicators 43 (2014), 288–296.
82. Romero, C., Handbook of critical issues in goal programming. 1991, Pergamon Press, Oxford.
83. Romero, C., Tamiz, M., Jones, D., Goal programming, compromise programming and reference point method formulations: Linkages and utility interpretations. Journal of the Operational Research Society 49 (1998), 986–991.
84. Romero, C., A general structure of achievement function for a goal programming model. European Journal of Operational Research 153 (2004), 675–686.
85. Rosén, L., Back, P.E., Söderqvist, T., Norrman, J., Brinkhoff, P., Norberg, T., et al. SCORE: A novel multi-criteria decision analysis approach to assessing the sustainability of contaminated land remediation. Science of the Total Environment 511 (2015), 621–638.
86. Roth, S., Hirschberg, S., Bauer, C., Burgherr, P., Sustainability of electricity supply technology portfolio. Annals of Nuclear Energy 36 (2009), 409–416.
87. Rowley, H., Peters, G., Lundie, S., Moore, S., Aggregating sustainability indicators: Beyond the weighted sum. Journal of Environmental Management 111 (2012), 24–33.
88. Roy, B., Classement et choix en présence de points de vue multiples (La méthode de ELECTRE). Revue Francaise d´Informatique et de Recherche Opérationnelle 6 (1968), 57–75.
89. Roy, B., The outranking approach and the foundations of ELECTRE methods. Theory and Decision 31 (1991), 49–73.
90. Roy, B., Słowiński, R., Questions guiding the choice of a multicriteria decision aiding method. EURO Journal on Decision Processes 1 (2013), 69–97.
91. Saaty, T.L., A scaling method for priorities in hierarchical structures. Journal of Mathematical Psychology 15 (1977), 234–281.
92. Saaty, T.L., The analytic hierarchy process. 1980, McGraw Hill, New York.
93. Sadok, W., Angevin, F., Bergez, J.E., Bockstaller, C., Colomb, B., Guichard, L., et al. Ex ante assessment of the sustainability of alternative cropping systems: Implications for using multi-criteria decision-aid methods. A review. Lichtfouse, E., Navarrete, M., Debaeke, P., Véronique, S., Alberola, C., (eds.) Sustainable agriculture, 2008, Springer, Dordrecht, 753–767.
94. Salvado, M.F., Azevedo, S.G., Matias, J.C.O., Ferreira, L.M., Proposal of a sustainability index for the automotive Industry. Sustainability 7 (2015), 2113–2144.
95. San Cristóbal Mateo, J.R., Multi-criteria analysis in the renewable energy industry. 2012, Springer, London.
96. Santos, S.F., Brandi, H.S., Model framework to construct a single aggregate sustainability indicator: An application to the biodiesel supply chain. Clean Technologies and Environmental Policy 17 (2015), 1963–1973.
97. Sheppard, S.R.J., Meitner, M., Using multi-criteria analysis and visualisation for sustainable forest management planning with stakeholder groups. Forest Ecology and Management 207 (2005), 171–187.
98. Shmelev, S.E., Powell, J.R., Ecological economic modelling for strategic regional waste management systems. Ecological Economics 59 (2006), 115–130.
99. Shmelev, S.E., Rodríguez-Labajos, B., Dynamic multidimensional assessment of sustainability at the macro level: The case of Austria. Ecological Economics 68 (2009), 2560–2573.
100. Shmelev, S.E., van den Bergh, J.C.J.M., Optimal diversity of renewable energy alternatives under multiple criteria: An application to the UK. Renewable and Sustainable Energy Reviews 60 (2016), 679–691.
101. Steuer, R.E., Multiple criteria optimization: Theory, computation and application. 1986, John Wiley and Sons, New York.
102. Stewart, T.J., Relationships between data envelopment analysis and multiple criteria decision analysis. Journal of the Operational Research Society 47 (1996), 654–665.
103. Thomson Reuters. Journal Citation Reports, 2015 http://about.jcr.incites.thomsonreuters.com/ (last accessed 10/09/2015).
104. Troldborg, M., Heslop, S., Hough, R.L., Assessing the sustainability of renewable energy technologies using multi-criteria analysis: Suitability of approach for national-scale assessments and associated uncertainties. Renewable & Sustainable Energy Reviews 39 (2014), 1173–1184.
105. Ugwu, O.O., Kumaraswamy, M.M., Wong, A., Sustainability appraisal in infrastructure projects (SUSAIP): Part 2: A case study in bridge design. Automation in Construction 15 (2006), 229–238.
106. von Carlowitz, C. (1713). Sylviculture Oeconomica oder haußwithliche nachricht und Naturgemäßige. Anweisung zur Wilden Baum-Zucht, Johann Friedrich Braun 2. Leipzig.
107. Wang, J.J., Jing, Y.Y., Zhang, C.F., Zhao, J.H., Review on multi-criteria decision analysis aid in sustainable energy decision-making. Renewable & Sustainable Energy Reviews 13 (2009), 2263–2278.
108. Wierzbicki, A.P., A mathematical basis for satisficing decision making. Mathematical Modelling 3 (1982), 391–405.
109. Wierzbicki, A.P., On completeness and constructiveness of parametric characterization to vector optimization problems. OR Spektrum 8 (1986), 73–87.
110. Wolfslehner, B., Vacik, H., Mapping indicator models: From intuitive problem structuring to quantified decision-making in sustainable forest management. Ecological Indicators 11 (2011), 274–283.
111. Yoon, K., A reconciliation among discrete compromise situations. Journal of the Operational Research Society 38 (1987), 277–286.
112. Yu, P.L., A class of solutions for group decision problems. Management Science 19 (1973), 936–946.
113. Yu, P.L., Multiple criteria decision making: Concepts, techniques and extensions. 1985, Plenum Press, New York.
114. Zeleny, M., A concept of compromise solutions and the method of the displaced ideal. Computers & Operations Research 1 (1974), 479–496.
115. Zeleny, M., Multiple criteria decision making. 1982, McGraw-Hill, New York.
116. Zimmerman, H.J., Fuzzy programming and linear programming with several objective functions. Fuzzy Sets and Systems 1 (1978), 45–55.