A comprehensive MCDM assessment for economic data: success analysis of maximum normalization, CODAS, and fuzzy approaches

Abstract

The approach of evaluating the final scores of multi-criteria decision-making (MCDM) methods according to the strength of association with real-life rankings is interesting for comparing MCDM methods. This approach has recently been applied mostly to financial data. In these studies, where it is emphasized that some methods show more stable success, it would be useful to see the results that will emerge by testing the approach on different data structures more comprehensively. Moreover, not only the final MCDM results but also the performance of normalization techniques and data types (fuzzy or crisp), which are components of MCDM, can be compared using the same approach. These components also have the potential to affect MCDM results directly. In this direction, in our study, the economic performances of G-20 (Group of 20) countries, which have different data structures, were calculated over ten different periodic decision matrices. Ten different crisp-based MCDM methods (COPRAS, CODAS, MOORA, TOPSIS, MABAC, VIKOR (S, R, Q), FUCA, and ELECTRE III) with different capabilities were used to better visualize the big picture. The relationships between two different real-life reference anchors and MCDM methods were used as a basis for comparison. The CODAS method develops a high correlation with both anchors in most periods. The most appropriate normalization technique for CODAS was identified using these two anchors. Interestingly, the maximum normalization technique was the most successful among the alternatives (max, min–max, vector, sum, and alternative ranking-based). Moreover, we compared the two main data types by comparing the correlation results of crisp-based and fuzzy-based CODAS. The results were very consistent, and the “Maximum normalization-based fuzzy integrated CODAS procedure” was proposed to decision-makers to measure the economic performance of the countries.

Introduction

Macroeconomic performance is one of the main locomotive powers that activate a country’s potential dynamics. In particular, countries with remarkable economic performance on a global scale almost directly determine numerous dynamics, such as technological and scientific development, welfare, political and social theories, human and demographic structures, the environment, sustainable policies, and social psychology. Moreover, as Adam Smith (1776/2005) emphasizes, economics is often the main underlying factor in many issues. At this point, it is important to define economic performance and how it should be measured. Many scientific studies have been conducted to measure economic performance by employing different techniques, and these studies have obtained performance results via the metrics they have created (Karahan et al. 2021). For this reason, rating indices are metrics that decision-makers frequently follow and benefit from in their evaluation processes, and these indices classify countries by ranking them in certain respects.

The recent global economic crisis has highlighted the importance of understanding the interdependencies between countries and their economies, assessing risks, and determining the economic performance of countries in order to produce new and effective results. Countries are rated using various metrics. The necessity of an accurate measurement metric is important in terms of showing how the economic power of a country is compared to other countries as well as the effect on the quality of decisions to be taken. In addition, countries are categorized based on these measurements. Classification as a strong- or low-risk economy is a motivating factor for a country; however, it also provides an opportunity to cooperate with other countries that are included in the same classification. In this context, countries that are together become stronger; they also provide protection against countries that are out of the union or cooperation. The G20 was formed within this framework and consists of 19 dominant countries with the highest GDP in the world economy and the European Union Commission. Considering their GDP, it may be appropriate to state that these countries could easily steer the global economic order. However, as the correct calculation of a country’s economic and financial performance also affects the cost of borrowing, it is even more critical for borrowing countries to make accurate assessments.

Obtaining statistical results by processing data on macroeconomic indicators is a classical evaluation approach used to develop economic policies. In this case, contradictory situations may arise, such as the policies implemented to ensure success in basic economic indicators, creating negative effects on other indicators (Koşaroğlu 2021). Similarly, it often does not provide sufficient information on a country’s economy. It is scientifically more meaningful and rational to evaluate many different effective criteria simultaneously rather than using a mere economic indicator (Chattopadhyay and Bose 2022). Accordingly, several indices have been derived in which many different criteria are used simultaneously in the evaluation of economic performance. Examples include the Economic Discontent Index developed by Okun (1970), the Calmfors Driffill Index created by Calmfors and Driffill (1988), the Macroeconomic Performance Index (MPI) formulated by the OECD (1987), and the Barro Misery Index (BMI) formed by Barro (1999) (Belke 2020).

Multi-criteria decision-making (MCDM) methods have been constantly developed and renewed for more than half a century, and there are more than 200 types (Cinelli et al. 2022). If the criteria consist of different criteria units with a cost or benefit focus, measurement difficulty increases. MCDM methodologists state that classical weighted aggregation methods have a high “compensatory” feature (Ziemba 2019). The disadvantage of additive methods is that they can assign higher-ranked undesirable or unacceptable alternatives (Daher and Almedia 2013). Significant progress has also been made regarding MCDM components such as normalization types, weighting methods, and initial decision matrices. For example, in the context of improving data, “fuzzy-based” data types are constantly being developed for the first decision matrix, in addition to crisp-based and interval-value-based ones (Petrović et al. 2019). In addition, more than 10 normalization types have been proposed as alternatives to create dimensionless criterion units (Ersoy 2022; Vafaei et al. 2022; Aytekin 2021). Suggesting more than 10 objective and subjective methods for criterion weighting can be cited as another example of methodological improvement (Mukhametzyanov 2021; Pamučar et al. 2018). Another of the many MCDM improvements that have taken place is that some MCDM methods suggest the use of preference functions and threshold values in computational algorithms (Behzadian et al. 2010). In addition, rating agencies that use classical weighted aggregation methods do not offer solutions to rank reversal problems. However, the MCDM methodology has been determined to eliminate the rank reversal problem completely. Considering the many cumulative methodological developments and innovations in calculation algorithms, MCDM methods are much more useful and sophisticated than classical weighted aggregation-based economic measurements of indices. In this case, it is reasonable to use MCDM methods to determine the best alternative for a particular decision-making problem that includes more than one criterion or alternative (Zavadskas and Turskis 2011). As a matter of fact, MCDM methods are used in engineering (Stojčić et al. 2019; Stević et al. 2022; Kiptum et al. 2022), health (Liu et al. 2019), energy (Kumar et al. 2017), logistics and transport (Dabić-Miletić and Raković 2023; Taletović 2023; Stanimirović et al. 2023), informatics (Wu et al. 2020; Chakraborty et al. 2023), finance (Baydaş 2022; Baydaş et al. 2023), traffic and transportation (Damjanović et al. 2022; Badi and Bouraima 2023; Huskanović et al. 2023), portfolio selection (Emamat et al. 2022), and sustainable environment (Wei 2021). Despite these improvements, the insistence of rating agencies to still use classical performance metrics implies a fundamental measurement problem related to the goodness and accuracy of the measurement. It is possible that the results of classical collection methods may reduce the quality of decisions and that falsifiable results may adversely affect information users’ decisions.

Despite these cumulative developments, the disagreement about which method to choose from among the MCDM methods still seems to be a chronic problem. This is because the MCDM methods (when their algorithms and components are considered together) can produce different hierarchical ranking results. Thus, the discussions about which results are more efficient continue (Sałabun and Piegat 2017; Petrović et al. 2019; Baydaş and Pamucar 2022). At this point, even the smallest step or suggestion to be taken objectively is essential. Although the early MCDM literature indicated that such selection is a near-impossible phenomenon or paradoxical problem (Triantaphylloui 2000), new developments objectively show that some MCDM improvement efforts can provide a fair comparison (Baydaş and Pamucar 2022). Earlier inferences were mostly related to the input-based computational algorithms of MCDM methods. To discuss the superiority of one MCDM algorithm over another, it makes sense to focus on its input-based mathematical power. This has been the case previously (Triantaphylloui 2000). However, for the final uncertainty to be determined by a purely input-oriented assessment, clear alternative solutions must also be introduced. Nowadays there are enormous repositories of what is referred to as “big data”. Data processing, machine learning, artificial intelligence, and data mining have significantly developed over time. Having a sophisticated structure far beyond what is expected (e.g., MCDM problems are not always linear) and the extent to which the theory is reflected in practice can be better understood using today’s data analytics findings. The extent to which any MCDM algorithm or theory represents real life can be understood heuristically using data analytics. It would be interesting to study how and to what extent these algorithms respond in real life, their validity, and the extent to which they are valid. In fact, the kinds of characteristics they exhibit in real-life conditions, the kinds of weaknesses they show, and in which subjects they excel can be revealed via data analytics. Recent studies of financial data have yielded interesting findings. For example, it has been repeatedly proven that the FUCA and PROMETHEE-2 methods provide higher correlations with their rankings of real-life factors (e.g., share price changes) (Baydaş and Elma 2021; Baydaş and Pamucar 2022; Baydaş et al. 2023). Moreover, although these studies claim that these methods have low compensatory efficiency, they highlight that the amount of information their scores produce and their resistance to the rank reversal phenomenon may be high. These findings suggest a highly innovative MCDM evaluation methodology; they clearly show that MCDM methods can be evaluated not only on the basis of input but also on the basis of output and data analytics results.

Research gap and originality of study

We recommend an approach that expands the scope of MCDM methods’ evaluation by testing the methods with other data types, as well as financial ones. In this study, an evaluation was made using the economic data of the countries and not financial data. It is essential to note this difference because, while financial data are highly dynamic and volatile, economic data are more static (in fact, the kurtosis and skewness of financial and economic data are different). A notable point in previous studies is that only one real-life sequence was considered as the reference anchor when comparing MCDM methods. However, in this study, we suggest using two different anchors (GDP per capita and EPI) simultaneously. The depletion of natural resources and the deterioration of ecological balance indicate that the development of countries should be evaluated with an environmental focus, along with economic development (Arsu and Ayçin 2021). As a matter of fact, the importance of green energy and clean environment for a country is increasing day by day. The authors emphasize that the optimal utilization of renewable energy sources is vital (Saqib et al. 2021; Bhuiyan et al. 2022; Li et al. 2022; Dinçer et al. 2022). In addition, while previous studies focused only on comparing the basic algorithm of MCDM, this study also compares other MCDM components (normalization and data type). Thus, all significant techniques in the MCDM calculation process are compared with two different anchor references, and the appropriate techniques are selected and suggested. In particular, useful information has been produced on whether the dominant superiority of the FUCA and PROMETHEE-2 methods in financial data is sustainable in economic data. We are keen on the methodology here, and the findings will feed much more comprehensive studies in the future, where the big picture will be revealed more clearly with “big data.”

Fuzzy numbers have been used extensively in recent MCDM studies to create the first decision matrices (Moiseev et al. 2023; Wang et al. 2023a, 2023b; Albahri et al. 2023; Kou et al. 2023a). However, it cannot be said that a satisfactory objective answer has been given to the question of whether the classical crisp or fuzzy-based MCDM is more successful. We believe that this study serves as a pioneer in this regard.

Aims of the study

In this study, we aimed to determine the most appropriate MCDM method, normalization technique, and data type. In this context, this study proposes an objective methodological framework for evaluating MCDM methods (comparison and selection) in real-life economics using GDP per capita and environmental orientation. The original approach of this study is the first in the literature. Here, the algorithm, normalization technique, and data type of MCDM methods are objectively compared, selected simultaneously, and presented to decision-makers to make an appropriate MCDM performance evaluation for national economies. Moreover, two independent anchors are used for the first time in this study. Thus, an improved MCDM framework is developed. According to the findings, the most appropriate MCDM method was CODAS (Combinative distance-based assessment), and the most appropriate normalization technique was maximum normalization, and the most appropriate data type was fuzzy-based data.

Paper structure

In line with the purpose, the remainder of this paper is organized as follows. The second section reviews the relevant literature. This section describes the progress in MCDM evaluation (comparison and selection) and presents a limited number of MCDM-based studies on countries’ economic performance. The third section presents the model, dataset, and methodology for empirical treatment. The fourth section describes the results of the empirical analysis. In the fifth section, the findings and predictive results are discussed by comparing them with the findings of related studies in the literature. The conclusion section presents suggestions in the context of the results obtained. In addition, the limitations of the study and suggestions for future research are included.

The flow chart of the methodology applied in this study is shown in Fig. 1.

Fig. 1

The flow chart of the methodology used in this research

Literature review

In the first part of the literature review, studies on MCDM-based country economic performances are examined. Then, the literature on the MCDM evaluation methodology (comparing and selecting methods), which is the most critical topic of this study, is reviewed. Thus, gaps are identified for improved performance measurements, and useful insights are presented to the reader.

Comparative performance analysis of countries and MCDM methods

Rating agencies compare countries in many aspects (e.g., environmental, economic, sustainability, education, culture, livability, democracy, and human development) with the data they collect and/or receive from reliable institutions and rank them in a certain hierarchy. These ranking efforts form the basis for development-based country classification. The G-8 countries (representing approximately 65% of the world economy) or the G-20 rankings for the 20 most powerful countries (accounting for 85%) can be given as examples. Another similar classification effort is the Environmental Performance Index (2023). While this ranking examines the environmental sensitivity of countries with various criteria, it is based on a doctrine that examines the relations of countries with the environment, countries that have reached a certain level of development on many different issues, such as culture, politics, sustainability, and demographics. In fact, the methodological measurement made by the rating agencies here is based on the “weighted simple aggregation” method, which is one of the oldest and simplest methods of MCDM. There are more than 200 alternative MCDM methods, and objective questioning of such measurements and comparisons with other advanced MCDM methods are important gaps in the literature. When the literature in which the economic performances of countries are measured on the basis of MCDM is examined, it will be better understood that the method adopted by the current rating agencies has weaknesses that cannot be ignored.

An early example of measuring economic performance through MCDM is Lovell et al. (1995). In this study, the economic performance of 19 OECD countries between 1970 and 1990 was measured using the data envelopment analysis (DEA) method. Podvezko (2011) used GDP growth, industrial production growth, average job wage, export/import ratio, and the unemployment rate with simple additive weighting (SAW) and Complex Proportional Assessment (COPRAS) methods for Estonia, Latvia, Lithuania, and Poland. Momeni et al. (2011) conducted a larger study using macroeconomic data from 54 countries. The priority of the criteria was determined using the analytic hierarchy procedure) method, and the countries were listed using the K-mean technique. Urfalıoğlu and Genç (2013) approached the issue from the perspective of the EU. They compared the macroeconomic performance of both EU member states, candidate countries, and Turkey using ELECTRE (ÉLimination Et Choix Traduisant la REalité), PROMETHEE (Organization method of preference order for enrichment evaluation), and TOPSIS (Order Preference Technique by Similarity to Ideal Solution) methods. Using the TOPSIS method, Chattopadhyay and Bose (2015) comparatively investigated the macroeconomic performance of 48 countries before and after the 2007 financial and economic crises between 2000 and 2012. Sweden, Singapore, Switzerland, Malaysia, and Luxembourg, were identified as the best-performing countries, while Greece, Poland, Portugal, and Romania were identified as the worst performers. Balcerzak and Pietrzak (2017) conducted research on macroeconomic performance using TOPSIS. When they examined 24 European Union countries between 2004 and 2010, they determined a direct relationship between the knowledge-based economy and quality of life in EU countries. Mitkova and Mlynarovič (2019) listed 17 European countries based on 16 macroeconomic indicators with Promethee II according to this evaluation; the Czech Republic and Poland were determined as the best countries.

Unlike previous studies, Ture et al. (2019) included sociodemographic and innovation indicators in a macroeconomic MCDM performance analysis. Twenty-seven EU member states were ranked using VIKOR (VIseKriterijumska Optimizacija I Kompromisno Resenje) and TOPSIS, and performance in many EU countries, including Slovenia and Romania (which are new members of the EU), has been calculated through both methods. Oguz et al. (2020) ranked seven EU candidate countries (five candidates and two potential candidates) in 2017, based on the Maastricht Economic Criteria. In a study using the TOPSIS method, Turkey ranked first. Arsu and Ayçin (2021) analyzed OECD countries with the MARCOS (Measurement of Alternatives and Ranking according to Compromise Solution) using 12 different economic, social, and environmental criteria. The findings revealed that the countries could be divided into two clusters. Denmark and Ireland were the most successful countries in the first cluster, whereas Slovenia, Spain, and Portugal were the best countries in the second cluster. Chattopadhyay and Bose (2022) also conducted research using OECD countries. They discussed the macroeconomic performance of 21 OECD countries for the pre- and post-pandemic periods and investigated January 2016–July 2020 via the TOPSIS method. It was observed that China and Poland had the highest ranks, whereas Russia and the Netherlands had the worst performance at the end of the analyzed periods.

Arsu (2022) also included different criteria in MCDM analyses for macroeconomic performance and measured the performance of BRICS and MINT countries using the COPRAS method. According to the findings, China, Russia, and Indonesia were the most successful countries in Scenarios 1 and 3, while Russia and Mexico were the most successful countries in Scenario 2. Stojanović et al. (2022) ranked the Western Balkan countries for 2019, 2020, and 2021 via CRADIS (Compromise Ranking of Alternatives from Distance to Ideal Solution). Among Western Balkan countries, Montenegro was found to have the best innovation performance, while Albania was cited as the worst-performing country. On the other hand, Starčević et al. (2022) aimed to make a purely macroeconomic evaluation of Bosnia and Herzegovina and Serbia using foreign direct investments, foreign trade, GDP, inflation rate, employment rate, and real exchange rate data. While DEA is a method applied for input-based efficiency, principal component analysis was performed for the exact values of the countries’ initial efficiency. The weight coefficients of the parameters used were determined using enhanced fuzzy Stepwise Weight Assessment Ratio Analysis (SWARA), and CRADIS was then applied. Akandere and Zerenler (2022) examined the 2022 environmental and economic indicators of Eastern European countries using the integrated CRITIC-TOPSIS method. Since it is known in the literature that a country’s environmental performance affects its economic performance in a meaningful and positive way, it is believed to help decision-makers develop comprehensive sustainability policies. Grossman and Krueger (1991) and Panayotou (1993) are among the first to study the inverted U-shaped relationship between environmental degradation and economic growth in the long run. From these studies, it has been determined that economic development increases environmental pollution in the first processes owing to the inverse relationship between growth and pollution, but the developments experienced with growth and the performance increases in the following processes contributed to reducing environmental pollution. In this study, we propose for the first time (supported by trends in the literature) two different MCDM performance metrics with an environmental focus and a GDP per capita focus. The purpose of this was to methodologically compare and select the MCDM algorithm components.

Among the aforementioned studies, TOPSIS, which is a distance-based and ideal value-oriented method, is adopted relatively more in economic performance measurements. However, there has two known weaknesses: over-caring about ideal values and rank reversal. Thus, it would not be accurate to be content with TOPSIS alone. CRADIS, which is a relatively new methodology, has also come to the fore in recent studies. However, the simple and old SAW method, which is closest to the techniques adopted by rating agencies, has rarely been used in MCDM studies. Previous studies have generally explored the development of the computational algorithm or the methods used in other studies in the literature, but it can be said that there is no objective selection framework for exactly why the MCDM methods they used were chosen. They did not mention the relationship between output-based scores and real-life situations. Moreover, in MCDM calculations, a limited number of periods were considered and an analysis was performed on several datasets. In addition, a few MCDM methods have been applied. Unlike the literature, in this study, we approached the subject comprehensively to gain a more insightful view and applied ten different MCDM methods for ten different periods and compared them with real life.

In addition, in contrast to the literature, normalization and data types (crisp or fuzzy) were compared in this study using two different anchors, which are real-life rankings. After choosing an appropriate MCDM method and normalization technique (for the first decision matrix), it is a question of whether the most appropriate data type should be “crisp” or “fuzzy.” Crisp values are static and based on unqueryable values. Fuzzy data is capable of linguistic interpretation and is then converted into multiple numeric values. In other words, comments such as low, medium, good, and very good can be made regarding the numerical criterion value of an alternative, which can then be converted into fuzzy numbers. In doing so, the expert interpreter can make an appropriate assessment of a country’s benchmark value based on the mean, standard deviation, and previous years’ values (increases or decreases), which can be predicted to be healthier. This may be more accurate for critical nonlinear or hard-to-model problems. Therefore, this study is comprehensive, original, and innovative compared to other studies.

MCDM evaluation methodology and using dynamics as an anchor for MCDM methods

Each MCDM method has advantages and disadvantages, and no single method is expected to be perfect for all problems. As an ongoing debate in the literature, concentrated efforts continue to propose an objective methodology for selecting the most appropriate method (Guarini et al. 2018). In MCMD studies, several methods are commonly used simultaneously to distribute risk. Among the more than 200 MCDM methods (Cinelli et al. 2022), there are simple, primitive, and algorithmically advanced methods. However, while the MCDM method selection or MCDM evaluation methodology is mostly associated with the theoretical capacity of input-based algorithms, it cannot be said that comprehensive data analytics have been conducted regarding the outputs (scores) they produce. As a matter of fact, at this point, classical ancient literature claims that this problem is significantly difficult to ease or has no definite solution (Triantaphyllou 2000). Various suggestions exist regarding the type of framework that should be adopted when selecting an appropriate MCDM technique. For example, some studies, such as Velasquez and Hester (2013), emphasize that SWOT analysis can provide an idea of which MCDM methods are approximately proper, since logically, it is a good idea to create new methods that eliminate weaknesses and effectively integrate strengths into the situation. For instance, according to some, European “outranking schools” such as PRMOTHEE-2 are less compensatory, which can be counted as a positive advantage. TOPSIS is dependent on ideal values (positive and negative), and can produce more rank-reversal problems, which are negative traits. Many MCDM methods transform and distort information through normalization, which is the most essential reason for reversing the order that should normally occur (Wu and Abdul-Nour 2020). In fact, no MCDM method is perfect; therefore, it is recommended to apply more than one method to the same problem as an alternative approach (Mulliner et al. 2016). On the other hand, sensitivity analysis (which also measures compensation level) has been proposed for the comparison problem of MCDMs (Haddad et al. 2020). MCDM selection can be flawed and sometimes problematic, and good guidance for MCDM selection is mandatory (Cinelli et al. 2022). There are many problematic components or calculation steps that complicate the selection of an MCDM. For example, it is unclear whether outranking, utility, or distance-based MCDM schools are preferred. The MCDM algorithm type, normalization, weighting technique, appropriate threshold values for some methods, and selection of the preference function type are other uncertainty problems. Moreover, “rank reversal” and “compensation” are undeniable problems with unfairness and producing inconsistent results.

As the search continues for a reference objective criterion(s) for the MCDM evaluation methodology, the strength of the outputs of MCDM methods to meet real-life scenarios (modeling of reality) may be a significant answer (Munier 2006). Therefore, for MCDM methods, both the computational and output-based (score) capabilities can be analyzed. Several studies have been conducted to determine which method outperforms others in terms of output. Convincing findings have come to the forefront of recent financial research. MCDM selection is determined through a real-life return-on-share anchor (Baydaş and Elma 2021; Baydaş et al. 2023; Baydaş and Pamucar 2022). Studies using this procedure have highlighted the success of certain MCDM methods. Therefore, this study aims to clarify the bigger picture and reveal the capabilities of different MCMD methods that are not commonly used in macroeconomic performance—environmental performance studies. In addition to the methods used in the literature, multiple new and popular methods were applied in this study, and the findings and their competencies were compared. For the study to be healthy, all schools in the literature (outranking, utility, distance-based, etc.) and many new, old, popular, and common MCDM methods were included.

Material and methods

In this study, all one-year periods between 2011 and 2020 for 19 (18 countries with data available for some periods) of the G-20 countries were examined separately for MCDM. Data on the countries’ macroeconomic indicators were obtained from the World Bank’s World Development Indicators database. Argentina’s inflation data are provided from the OECD (2023) database for the period 2017–2020. The Environmental Performance Index (2023), on the other hand, is a data set developed by Yale and Colombia Universities within the framework of the United Nations Millennium Development Goals, jointly by the World Economic Forum and the Joint Research Center of the European Commission for Knowledge, and these data were pulled from Yale University’s EPI database. In addition, 10 MCDM methods from different schools were used to measure the economic performance of countries over six macroeconomic metrics for the 10 years (terms) examined, and the countries were ranked with MCDM methods accordingly. In this study, the entropy weighting method was applied for each year to assign a weighting coefficient to the criteria. Excel was used to calculate the final MCDM scores, and MINITAB software was used for statistical Spearman rank correlation analysis. For the selection of the best normalization technique, four different normalization techniques and one alternative conversion criterion were examined. Finally, “crisp-based” and “fuzzy-based” matrices were compared for data type selection. All the methods, techniques, and data types were compared using two different anchors.

The depletion of natural resources and the deterioration of ecological balance indicate that the development of countries should be evaluated with an environmental focus along with economic development (Arsu and Ayçin 2021). In this context, this study comparatively evaluates the macroeconomics of countries, both GDP per capita and environmental focus. More importantly, GDP per capita and EPI were fixed as references to enable the comparison of MCDM methods and validate the results. Thus, sustainable, meaningful, and high-relationship methods can be developed by using these anchors. In addition, the fact that the correlation results of both anchors are common (confirmation of each other) can be considered a factor that increases the reliability of the study.

In this section, the performance criteria, weighting techniques, and MCDM methods are discussed in detail. Table 1 lists the metrics used in economic performance studies.

Table 1 MCDM methods, weighting technique, criteria, and MCDM comparison anchors used in this study

Decision criteria and anchors

It makes sense to use MCDM methods because economic performance has a multicriteria structure. To make an effective and efficient economic performance measurement, many variables must be evaluated and criteria must be determined. Economic welfare can be evaluated using the GDP per capita as an anchor, which is an important indicator of the economic performance of countries. Increasing GDP per capita leads people to spend on intangible components, such as education and health, as well as affects their ability to purchase goods and services (Bolt et al. 2014:58). The depletion of natural resources and the deterioration of ecological balance indicate that countries’ development should be evaluated with a focus on the environment (e.g., EPI) as well as economic welfare (Arsu and Ayçin 2021). It should be emphasized that MCDM methods are structured for the same purpose and provide very similar results. Therefore, although they are close to each other, some provide a relationship with an external criterion at a high rate, whereas others provide a low rate. This indicates that some MCDM methods provide better correlations with an external factor without disturbing its structure. The widely adopted criteria and references for the decision matrix are as follows. Table 2 lists the preferred criteria and calculation steps used in the study.

Table 2 Financial metrics used in this study

The Spearman rank correlation coefficient measures the degree of interdependence between two ranking sequences (Kou et al. 2012). In this study, the relationship between the MCDM rankings and the GDP-EPI was measured using the Spearman correlation coefficient.

Criterion weighting method: entropy

In this study, entropy weight coefficients were used for all periods and methods. This method, which was identified as one of the 17 equations that changed the world by Stewart (2012) and is based on Shannon information entropy, was also suggested for determining weight significance. Information entropy, which is a measure of uncertainty, was first introduced by Shannon (1948). According to the concept of information entropy, which is widely used in many fields, entropy can be used to measure the amount of useful information provided by data itself (Wu et al. 2011). Using the same logic, the entropy weighting method is based on the amount of objective information regarding the criteria in the decision matrix. The smaller the entropy value, the greater the entropy-based weighting coefficient; thus, more information is provided by a specific criterion (Li et al. 2011). The basic steps are as follows (Wang et al. 2020; Puška et al. 2023):

Step 1. Normalize the decision matrix with m rows (solutions) and n columns (targets) by applying the commonly used sum normalization:

$${F}_{ij}=\frac{{f}_{ij}}{\sum_{k=1}^{m}{f}_{kj}} i\in \left\{\mathrm{1,2},\dots ,m\right\}; j\in \{\mathrm{1,2},\dots ,n\}$$

(1)

Step 2. Entropy is calculated for each criterion column.

$${E}_{j}=-\frac{1}{{\text{ln}}\left(m\right)}{\sum }_{i=1}^{m}({F}_{ij}{\text{ln}}{F}_{ij}) j\in \{\mathrm{1,2},\dots ,n\}$$

(2)

Step 3. The weight coefficient of each criterion is determined.

$${w}_{j}=\frac{1-{E}_{j}}{\sum_{j=1}^{n}(1-{E}_{j})} j\in \{\mathrm{1,2},\dots ,n\}$$

(3)

Normalization methods for criterion conversion

In MCDM methods, the criteria values in the initial decision matrix may consist of different units. In this case, the values are made unitless by normalization. Thus, numerical evaluations, such as summing the criteria for the final score, are possible. Using the methodological framework in this study, different normalization techniques were applied to the best chosen method, and the most successful normalization method was determined. Sum normalization, vector normalization, maximum–minimum normalization, and maximum normalization (Ersoy 2022; Vafaei et al. 2022; Aytekin 2021; Wang et al. 2020; Saha et al. 2022) were the methods applied. As an alternative to these classical MCDM normalization techniques, a rank-based conversion technique applied to the FUCA method (Wang and Rangaiah 2017) has also been proposed.

Applied MCDM methods

The reason MCDM methods used in this study (out of more than 200 available) were preferred is that they were chosen from those with a high ability to represent their own schools and species. Many methods, ranging from common to popular, have been preferred. In particular, “outranking, utility, and distance-based methods” were chosen. The formulas for the MCDM methods used in this study are provided in the supplementary file. Fuzzy-based CODAS, which was the only successful method owing to space limitations, is explained below. Studies based on preferred MCDM methods and weighting techniques are presented in Table 3.

Table 3 References to relevant MCDM formulations and weighting techniques used in this study

For ELECETRE-3, by default, the indifference, preference, and veto thresholds were calculated as 10 percent, 20 percent, and 80 percent of the value range of each objective, respectively. A cutoff was not applied (i.e., zero cutoff). In this study, 0.5 was used as the default value for the “γ parameter” in the VIKOR method.

Combinative distance-based assessment (CODAS) and fuzzy CODAS

In CODAS, which is a relatively new method, the overall performance of an alternative is measured by its distance from the negative ideal point (Ghorabaee et al. 2016). Each pair of alternatives is compared based on their distance from the ideal value. In addition, with this method, the superiority of alternatives can be determined using two measurements. The priority criterion is the Euclidean distance between the considered alternatives and negative ideal. Hamming distance is preferred when Euclidean distance cannot be used. Therefore, in an alternative case, the other measure is the Hamming distance. Similarly, the Hamming distance from the negative ideal value is also considered. In this method, maximum normalization is often used to convert the different units.

Different spaces, ${l}^{1}$ and ${l}^{2}$ norms, are used in the CODAS method. These Euclidean and Hamming distances are used to assess alternatives to the CODAS method (Ghorabaee et al. 2017:7). The fuzzy CODAS method with fuzzy extensions of these distances was first proposed by Ghorabaee et al. (2017). In recent years, the fuzzy CODAS method has been used for supplier selection of education technology (Yazdani et al. 2023), supplier selection of furniture company (Ulutaş 2021), identifying the most critical failure causes for sustainable operation and environmental friendly production in coal-fired power industry (Panchal et al. 2022), selection of cryptocurrency investment alternatives (Katrancı and Kundakcı 2020), evaluating environmental quality (Ouhibi and Frikha 2020), and market segmentation (Ghorabaee et al. 2017). The application steps of the fuzzy CODAS method are as follows: Suppose that there are $q$ decision-makers in a decision problem with $n$ alternatives and $m$ criteria (Ogundoyin and Kamil 2022: 10; Ghorabaee et al. 2017:7–9; Chen 2000:5).

Step 1. Identify criteria and alternatives by decision-makers. Alternatives are compared with the linguistic scale shown in Table 4 for each criterion (Chen 2000:5; Vinodh and Wankhede 2021).

Table 4 Linguistic scale for the alternative ratings

Step 2. Construct the decision matrix $\left({\widetilde{X}}^{l}\right)$ for each decision-maker $\left(l=\mathrm{1,2},\dots , q\right)$ as shown in Eq. (4), using triangular fuzzy number equivalents of linguistic variables in Table 1. The average fuzzy decision matrix $\left(\widetilde{X}\right)$ of the decision-makers shown in Eq. (6) is created using Eq. (5).

$${\widetilde{X}}^{l}={\left[{\widetilde{x}}_{ij}^{l}\right]}_{n\times m}=\left[\begin{array}{cc}\begin{array}{cc}{\widetilde{x}}_{11}^{l}& {\widetilde{x}}_{12}^{l}\\ {\widetilde{x}}_{21}^{l}& {\widetilde{x}}_{22}^{l}\end{array}& \begin{array}{cc}\cdots & {\widetilde{x}}_{1m}^{l}\\ \cdots & {\widetilde{x}}_{2m}^{l}\end{array}\\ \begin{array}{cc}\vdots & \vdots \\ {\widetilde{x}}_{n1}^{l}& {\widetilde{x}}_{n2}^{l}\end{array}& \begin{array}{cc}\vdots & \vdots \\ \cdots & {\widetilde{x}}_{nm}^{l}\end{array}\end{array}\right]$$

(4)

$${\widetilde{x}}_{ij}=\begin{array}{c}q\\ \oplus\\ l=1\end{array}{\widetilde{x}}_{ij}^{l}$$

(5)

$$\widetilde{X}={\left[{\widetilde{x}}_{ij}\right]}_{n\times m}=\left[\begin{array}{cc}\begin{array}{cc}{\widetilde{x}}_{11}& {\widetilde{x}}_{12}\\ {\widetilde{x}}_{21}& {\widetilde{x}}_{22}\end{array}& \begin{array}{cc}\cdots & {\widetilde{x}}_{1m}\\ \cdots & {\widetilde{x}}_{2m}\end{array}\\ \begin{array}{cc}\vdots & \vdots \\ {\widetilde{x}}_{n1}& {\widetilde{x}}_{n2}\end{array}& \begin{array}{cc}\vdots & \vdots \\ \cdots & {\widetilde{x}}_{nm}\end{array}\end{array}\right]$$

(6)

where ${\widetilde{x}}_{ij}^{l}$ shows the fuzzy performance score of $i$th $\left(i=\mathrm{1,2}, \dots , n\right)$ alternative with respect to $j$th $\left(j=\mathrm{1,2},\dots , m\right)$ criterion for $l$th $\left(l=\mathrm{1,2},\dots , q\right)$ decision-maker, and ${\widetilde{x}}_{ij}$ denotes the average fuzzy performance value of $i$th alternative with respect to $j$th criterion.

Note: To justify a reasonable comparison in this study, the fixed weighting coefficient values determined by entropy were the same for both crisp- and fuzzy-based weightings.

Step 3. Obtain the normalized fuzzy decision matrix $\left(\widetilde{N}\right)$ as shown in Eq. (7), using Eq. (8). $B$ and $C$ in Eq. (8) represent the normalization of the benefit and cost criteria, respectively.

$$\widetilde{N}={\left[{\widetilde{n}}_{ij}\right]}_{n\times m}$$

(7)

$${\widetilde{n}}_{ij}=\left\{\begin{array}{cc}{\widetilde{x}}_{ij}/\underset{i}{{\text{max}}}\,\mathfrak{D}\left({\widetilde{x}}_{ij}\right)& if\,\,\, j\in B\\ 1-\left({\widetilde{x}}_{ij}/\underset{i}{{\text{max}}}\,\mathfrak{D}\left({\widetilde{x}}_{ij}\right)\right)& if\,\,\, j\in C\end{array}\right.$$

(8)

where ${\widetilde{n}}_{ij}$ shows the normalized fuzzy performance scores.

Step 4. Calculate the fuzzy weighted normalized decision matrix $\left(\widetilde{R}\right)$ as shown in Eq. (9). The fuzzy-weighted normalized performance scores $\left({\widetilde{r}}_{ij}\right)$ are calculated as shown in Eq. (10):

$$\widetilde{R}={\left[{\widetilde{r}}_{ij}\right]}_{n\times m}$$

(9)

$${\widetilde{r}}_{ij}={\widetilde{w}}_{j}\otimes {\widetilde{n}}_{ij}$$

(10)

where ${\widetilde{w}}_{j}$ shows the fuzzy weight of $j$th criterion, and $0<\mathfrak{D}\left({\widetilde{w}}_{j}\right)<1$.

Step 5. Define fuzzy negative-ideal solution as shown in Eq. (11).

$$\widetilde{NS}={\left[{\widetilde{ns}}_{j}\right]}_{1\times m}$$

(11)

$${\widetilde{ns}}_{j}=\underset{i}{{\text{min}}}\,{\widetilde{r}}_{ij}$$

(12)

where $\underset{i}{{\text{min}}}\,\,\,{\widetilde{r}}_{ij}=\left\{{\widetilde{r}}_{kj} | \mathfrak{D}\left({\widetilde{r}}_{kj}\right)=\underset{i}{{\text{min}}}\left(\mathfrak{D}\left({\widetilde{r}}_{ij}\right)\right),k\in \left\{\mathrm{1,2},\dots ,n\right\}\right\}$.

Step 6. Calculate the fuzzy weighted Euclidean $\left({ED}_{i}\right)$ and fuzzy weighted Hamming $\left({HD}_{i}\right)$ distances of alternatives from the fuzzy negative-ideal solution, as shown in Eqs. (13) and (14), respectively.

$${ED}_{i}=\sum\nolimits_{j=1}^{m}{d}_{E}\left({\widetilde{r}}_{ij},{\widetilde{ns}}_{j}\right)$$

(13)

$${HD}_{i}=\sum\nolimits_{j=1}^{m}{d}_{H}\left({\widetilde{r}}_{ij},{\widetilde{ns}}_{j}\right)$$

(14)

The fuzzy-weighted Euclidean and Hamming distances of the alternatives are calculated from the fuzzy negative-ideal solution using Eqs. (15) and (16) (Ogundoyin and Kamil 2022: 10).

$${d}_{E}\left({\widetilde{r}}_{ij},{\widetilde{ns}}_{j}\right)=\sqrt{\frac{{\left({a}^{l}-{b}^{l}\right)}^{2}+{\left({a}^{m}-{b}^{m}\right)}^{2}+{\left({a}^{u}-{b}^{u}\right)}^{2}}{3}}$$

(15)

$${d}_{H}\left({\widetilde{r}}_{ij},{\widetilde{ns}}_{j}\right)=\sqrt{\frac{\left|{a}^{l}-{b}^{l}\right|+\left|{a}^{m}-{b}^{m}\right|+\left|{a}^{u}-{b}^{u}\right|}{3}}$$

(16)

where ${\widetilde{r}}_{ij}$ and ${\widetilde{ns}}_{j}$ are two triangular fuzzy numbers defined by ${\widetilde{r}}_{ij}=\left({a}^{l},{a}^{m},{a}^{u}\right)$ and ${\widetilde{ns}}_{j}=\left({b}^{l},{b}^{m},{b}^{u}\right)$.

Step 7. Calculate the relative assessment matrix $\left(RA\right)$ as shown in Eq. (17), using Eq. (18).

$$RA={\left[{p}_{ik}\right]}_{n\times n}$$

(17)

$${p}_{ik}=\left({ED}_{i}-{ED}_{k}\right)+\left(t\left({ED}_{i}-{ED}_{k}\right)\times \left({HD}_{i}-{HD}_{k}\right)\right)$$

(18)

where $k\in \left\{\mathrm{1,2},\dots ,n\right\}$ and $t$ is a threshold function that is defined as shown in Eq. (19).

$$t\left(x\right)=\left\{\begin{array}{ccc}1& if& \left|x\right|\ge \theta \\ 0& if& \left|x\right|<\theta \end{array}\right.$$

(19)

Decision-makers determine the threshold parameter $\left(\theta \right)$ shown in Eq. (19). Generally, $\theta =0.02$ is used for the calculations.

Step 8. Calculate the assessment score $\left({AS}_{i}\right)$ for each alternative, as shown in Eq. (20).

$${AS}_{i}=\sum_{k=1}^{n}{p}_{ik}$$

(20)

Step 9. Rank the alternatives according to the assessment score. A higher score indicates a more desirable alternative.

Empirical analysis

In this study, based on the GDP per capita and environment-oriented economic performance measurement analysis of the G-20 countries, the rankings created by MCDM methods are compared and verified with real-life anchors, and the most appropriate methods are suggested for this research area. The calculation steps used in this study are as follows:

Step 1. Determination of economic performance criteria: The MCDM criteria of the 19 countries researched in this study are preferred because they are frequently used in the economic performance literature. Ten decision matrices were created using the performance metrics of the MCDM method. The weights were calculated using the entropy method.

Step 2. Determination of MCDM method ranking results: The ranking results of 19 countries in the G-20 covering 10 periods were calculated in Excel using 10 different MCDM methods.

Step 3. Comparison of GDP/EPI and MCDM ranking results to determine the most appropriate method for economic performance analysis: MCDM rankings were calculated separately for 10 separate years using 10 MCDM models, and six macroeconomic metrics were compared with the EPI and GDP rankings for the relevant period. In this study, in which only the entropy weighting method was preferred because it provided strong and meaningful results, the strength of the relationships between the rankings was determined using the Spearman rank correlation coefficient. Methods that produce stable and highly correlated results have been proposed for economic rating agencies, particularly for different types of decision-makers.

After the analysis, the GDP and EPI rankings were used to compare the results, as described in the following section. These 10 different MCDM equations were compared using two different anchors to determine the MCDM method that produced better results for economic performance analysis. Moreover, the normalization techniques and initial decision matrix data were compared using the same anchors.

Findings and results

In line with the step-by-step methodology described above, the final scores of 19 countries (18 in some years), which form the basis for the ranking of MCDM methods in the G-20, were calculated and evaluated. In this section, first, the selection of an MCDM method from among the alternatives (via anchors) is shown, supported by analysis. In the next step, the MCDM method, which provides the best results, is kept constant, and the normalization technique that yields the best results (with the same approach and over the anchors) is selected. In the final stage, the data type (clear or fuzzy) is determined (via anchors) using the MCDM method. Because determining the most appropriate MCDM type, normalization technique, and data type requires obtaining the most appropriate performance measurement tool, only the ranking of the final MCDM method obtained with this integrated metric based on countries was considered as the basis.

Selection of crisp/classical numbers based MCDM algorithm that produces the best results

The entropy technique is used to determine the weighting coefficients of the decision criteria in the periods considered as the basis for the two different economic performance rankings of the MCDM methods over the six preferred criteria. Accordingly, it is remarkable and interesting that the objective entropy weighting method automatically assigns the highest weighting importance (in all periods) to interest payments and official exchange rate criteria. This information should be evaluated and interpreted by decision-makers. By contrast, unemployment and export/import ratios have the lowest weights according to this technique. The criterion weights calculated using this method are listed in Table 5.

Table 5 Weight coefficients determined according to entropy technique for this study

In this study, which examined a period of 10 years, the CODAS and FUCA methods clearly came to the fore as the most successful methods in the 10 examined periods. As shown in Table 6, these two methods produced the highest Spearman’s correlation coefficients. CODAS using max normalization and the FUCA method independent of normalization were determined as methods to be recommended to decision-makers evaluating economic performance criteria.

Table 6 Spearman’s Rank correlation relationship between GDP per capita ranking and scores composed by MCDMs

As shown in Table 7, these two methods (CODAS and FUCA) produced the highest Spearman correlation coefficients. In this study, which examined a period of 10 years, the CODAS and FUCA methods were the most successful. The CODAS and FUCA methods have been determined as recommended methods for decision-makers evaluating economic performance measures.

Table 7 Spearman’s rank correlation relationship between EPI ranking and scores composed by MCDMs

Remarkably, the VIKOR’s S-based result was more successful than the other Q- and R-based results. The VIKOR-R and TOPSIS produced the lowest correlations for this problem in both cases. Although the data changed from year to year, the stable results are an interesting and consistent indicator. The ten-year average correlations in Fig. 2 show that the CODAS method was the most successful in this study.

Fig. 2

Similarity of mean correlations produced by GDP per capita and EPI anchors with different MCDM methods

It is no coincidence that MCDM methods that succeed, fail, or perform merely adequately are similar for both anchors. These findings indicate that the two independent anchors used here are suitable choices for evaluating MCDM methods. The mean-based results in Fig. 3 show that the results produced by the MCDM methods with the two different anchors confirm each other.

Fig. 3

Similarity of mean correlations produced by GDP per capita and EPI anchors with different MCDM methods

Selection of the normalization technique that produces the best results

Normalization is usually the most innocent part of MCDM methods; however, it can significantly affect the ranking results depending on the data type. According to the above findings, the most appropriate method that came to the fore was CODAS. CODAS was the MCDM method that produced the highest correlation with anchor-1 and anchor-2 in most periods and on average. Based on the following findings, we adopt the maximum normalization technique commonly used for CODAS. What could be the impact factors behind the success of CODAS? The CODAS algorithm, or normalization, is an important factor. Although it is relatively difficult to determine the strength of the algorithm, it is easier to measure the effect of normalization. It is known that MCDM methods operate with uniform normalization. However, other types of normalization have been used in the literature. Moreover, we do not know which method produces the most efficient results. To clarify this uncertainty, we can determine which normalization technique is more successful using anchor-1 and anchor-2. The task is simple. The normalization techniques were changed and tested individually while keeping the CODAS algorithm constant. That is, these techniques were integrated into the CODAS method, and the results were compared. As an alternative to normalization techniques, rank-based values are proposed for the first time in this study. Rank-based results show the rank of an alternative in a criterion, which is only one of the steps in the FUCA method. The results show that this method is a very good alternative. Table 8 clearly shows the correlation results indicating the relationship between anchor-1 and anchor-2 of the normalization techniques integrated into the CODAS method (only correlations with Anchor-1 are shown since Anchor-1 and Anchor-2 give the same hierarchical results).

Table 8 Spearman’s rank rho correlations between GDP per capita and scores produced by (different normalized) CODAS methods

The fact that rank-based values are second in success also proves that this proposal is a good alternative to normalization techniques. Moreover, the rank-based technique sufficiently explains why FUCA ranks second, as shown in the previous section. These results show that the normalization type is the most effective impact factor for the success of MCDM methods. We observed that other normalization techniques for CODAS significantly reduced the success rate (e.g., the min–max for CODAS). If we had chosen the normalization type, the correlation strength would have decreased. The classic and widely used maximum normalization method is more successful than other normalization methods. Figure 4 shows the change in the mean correlation results of the CODAS according to the normalization method.

Fig. 4

CODAS’s changing performance according to normalization techniques

Selection of data type

We chose a well-performing MCDM method and normalization technique, but did not want to settle for this. With the same approach, we wondered if the data type for CODAS (for the initial decision matrix) should be “classic/crisp” or “fuzzy.” The crisp values are static and based on only one value. Fuzzy data is capable of linguistic interpretation and is then converted into multiple numeric values (Kou et al. 2023b). That is, interpretations such as low, average, good, and very good can be made for the numerical criterion value of an alternative, and these values can then be converted into fuzzy numbers. While doing so, in the evaluation of the criterion value of a country, the commentator makes an evaluation based on the mean, standard deviation, and values of the previous years (increases or decreases can be observed), which can be predicted to be healthier. In Table 9, the results of the triangular-based fuzzy integrated maximum normalized CODAS and crisp-based maximum normalized CODAS are compared. As can be seen, the fuzzy-based results produced much higher correlation values for both anchors.

Table 9 Spearman’s rank rho correlations between GDP per capita & EPI rankings and scores produced by fuzzy CODAS and Crisp CODAS

Based on these results, Fig. 5 clearly shows that fuzzy numbers are more efficient not only in the absence but also in the presence of clear data.

Fig. 5

Comparison of fuzzy and crisp based decision matrices

Finally, according to the findings of these integrated calculations for the 10 MCDM methods, the maximum normalized and fuzzy-based integrated CODAS method for the improved economic performance measurement of the G-20 countries has been proposed for relevant decision-makers.

Discussion

This study’s original approach is the first in the literature. Here, the algorithm, normalization technique, and data type of MCDM methods are objectively compared, selected simultaneously, and presented to decision-makers to make an appropriate MCDM performance evaluation for national economies. Moreover, two independent anchors are used for the first time in this study. Thus, an improved MCDM evaluation framework is developed. According to the findings, the most appropriate MCDM method was CODAS, and the most appropriate normalization technique was maximum normalization, and the most appropriate data type was the fuzzy-based data. Some of the conclusions from these results can be summarized as follows:

• We determined that the basic algorithms of MCDM methods produce results at different levels.

• It should be emphasized that the important factors that affect the results are the basic algorithm of the MCDM, the normalization technique, and the data type (fuzzy or crisp).

• FUCA and PROMETHEE-2 came to the fore as the most successful method in the results of researchers (Baydaş et al. 2023; Baydaş and Pamucar 2022) who used this model in previous studies. FUCA was successful in this study, albeit in second place. This can be explained by the fact that financial data are very volatile and contain many negative values. This reduces the efficiency of the normalization techniques. As FUCA does not use normalization (it uses a rank-based value), it may have been more successful. Moreover, the compensability of this method is low, which is advantageous. Because the economic data contain more positive values and the dataset is more static, the successful ones among the MCDM methods and components (normalization and first decision matrix data type) are dominantly different in all periods. Table 10 lists the dominance and skewness of the economic datasets used in this study. As can be seen, the change in data from year to year is not very high. That is, the decision matrices are similar; therefore, the probability of encountering surprises is low.

Table 10 Average skewness and kurtosis values of the criteria (input data) in the initial decision matrix

These results also explain, to some extent, why the ranking results produced by the fuzzy- and maximum-based normalization-integrated CODAS methods were consistently successful. Certain methods are likely to be more successful for certain data types because they can model a problem better. As a matter of fact, while FUCA is very successful in financial data that contains a lot of negative data and has high skewness and kurtosis, on the contrary, the success of CODAS is low (Baydaş et al. 2023). FUCA also excels in terms of economic data, but CODAS performs better. Therefore, researchers should pay special attention to data types.

According to these sensitive measurements and methodological evaluation results, Australia and Canada are the countries with the best economies (both environmentally oriented and in terms of GDP per capita). Over a 10-year period, Australia achieved five first places, three second places, and two third places. Canada won four first places, three second places, and one third place.

Conclusion

This study proposes an objective methodological framework for evaluating MCDM methods (comparison and selection). Recently, real-life sequences have been used as references for evaluating MCDM methods. In this study, instead of calculating countries’ economic performance, we shifted the axis towards GDP per capita and environmental performance. Thus, we obtain two different MCDM-based economic performance measurements that are highly correlated with GDP per capita and environmental sensitivity, which are considered important outputs today. This perspective has provided the opportunity to determine which MCDM method produces a stronger relationship between GDP per capita and environmental sensitivity. In other words, the MCDM methods can be compared using two different anchors. Moreover, the components of the MCDM (normalization and data type) were evaluated using the same anchors.

Ten different MCDM methods were compared with the economic performance data of G-20 countries using two different anchors for 10-year periods (annual basis). Among the correlations between the obtained scores and anchor-1 and anchor-2, those that produced higher correlations between the MCDM methods and their components were identified. According to the obtained results, CODAS came to the fore as the method that provided the best relationships. Normalization techniques for CODAS were evaluated using a similar procedure. The results show that the maximum normalization type consistently produces higher correlations. Finally, the net CODAS results were compared with the fuzzy-based CODAS results, and it was observed that the fuzzy-based results mostly produced higher correlations with both anchors. Therefore, the results of this study are as follows. The maximum normalization type, CODAS MCDM method, and fuzzy data type produce the most successful result. When we combined them, fuzzy-based CODAS with integrated maximum normalization stood out as the best metric.

This study, which we believe makes an important contribution to the MCDM evaluation methodology, shows that the choice of the basic algorithm for normalization, data type, and MCDM methods is critical for decision-makers. In previous studies using this methodological procedure, two issues emerged regarding the success of FUCA and PROMETHEE-2. The structures of economic and financial data (kurtosis and skewness) are completely different from each other. The structure of the financial data is more variable and contains more negative data. However, the economic data are more stable and positive. This indicates that MCDM results can be affected by the data structure. Finally, another fact is that the transformation of the data type together with the normalization also influences the results.

Recommendations for future researchers

Our recommendations for future research are as follows: Researchers who wish to contribute to the MCDM evaluation methodology can evaluate datasets from different fields and MCDM methods using the approach proposed in this study. Researchers can also use rank reversal performance as a separate evaluation criterion. In addition, we recommend the development of a metric that measures compensation. Thus, they can improve their sensitivity analysis by evaluating them for this purpose. Furthermore, researchers can compare the anchor solutions and fuzzy types here.

Limitations of the study

The MCDM methods, normalization techniques, and data types used in this study are limited. If the results obtained are tested using other methods or techniques, better results may be discovered. In addition, it should be considered that the results obtained here are valid for economic data and the results may differ for different data. We consider the assessment methodology proposed in this study robust and applicable. However, it should also be emphasized that the methods and techniques that have proven successful may vary in different scenarios. It should be noted that the purposes of the MCDM evaluation methodology and MCDM methods are different from each other.