Discussions and Closures Discussion of “Application of the Firefly Algorithm to Optimal Operation of Reservoirs with the Purpose of Irrigation Supply and Hydropower Production” by Irene Garousi-Nejad, Omid Bozorg-Haddad, Hugo A. Loáiciga, and Miguel A. Mariño Downloaded from ascelibrary.org by 184.108.40.206 on 10/27/17. Copyright ASCE. For personal use only; all rights reserved. DOI: 10.1061/(ASCE)IR.1943-4774.0001064 Mojtaba Moravej 1 1 Academic Elite Soldier, National Elite Foundation, 1438833171 Tehran, Iran. ORCID: https://orcid.org/0000-0003-0347-7317. E-mail: Mojtaba.Moravej@gmail.com In the original paper, the authors made an effort to show the application of the firefly algorithm (FA) and its superiority to solve different reservoir operation problems in comparison with the genetic algorithm (GA). Although the paper seems interesting, the discusser would like to mention following points for future studies. The authors claimed that the FA has not been applied for the optimal operation of reservoir systems before. On the one hand, Hosseini-Moghari and Banihabib (2014) applied the FA to optimize the operation of the Bazoft Reservoir. Furthermore, there are numerous different kinds of lifeform throughout the world; this cannot mean that researchers should develop, modify, and apply almost infinite different algorithms based on insects, fish, mammals, and plants. Over the last decade, there has been an explosion in the development of new so-called metaheuristic optimization algorithms based on natural metaphors. These algorithms have attracted criticism globally (Weyland 2010, 2015; Lones 2014; Sörensen 2015; Swan et al. 2015). It has been argued that these algorithms just proliferate existing methods by using different terminology in order to hide their lack of novelty (Weyland 2010, 2015; Lones 2014; Sörensen 2015). The best-known algorithm that lacks novelty is harmony search. Weyland (2010) provided compelling evidence that the harmony search algorithm is nothing but a special case of (μ þ 1) of the evolution strategies, which was proposed by Rechenberg (1973). Another example of such algorithms is the FA. The difference between glowworm swarm-based optimization (Krishnanand and Ghose 2006), FA, fly optimization algorithm (Zainal Abidin et al. 2010), fruit-fly optimization algorithm (Pan 2012), multiswarm fruit-fly optimization algorithm (Yuan et al. 2014), and well-known particle-swarm optimization (PSO) (Eberhart and Kennedy 1995) seem negligible (Weyland 2015). The differences between these various social insect algorithms proved marginal at best (Sörensen 2015). It has been proven that the FA is sensitive to its parameter selection, converges slowly, and gets trapped in local optimum points (Yang 2012; Gandomi et al. 2013; Garousi-Nejad et al. 2016). Accordingly, many studies have focused on overcoming these limitations (Yang 2012; Gandomi et al. 2013; Kazem et al. 2013; Wang et al. 2016; Garousi-Nejad et al. 2016), which shows that the FA’s performance was insufficient. Therefore, just because an algorithm has not been applied to an optimization problem, it is not a praiseworthy work to use it. It is not clear how an application of such algorithms contributes in the reservoirs operation field, especially when more powerful tools have already been developed (Wardlaw and Sharif 1999; Jalali et al. 2007; Afshar 2013; Moravej and Hosseini-Moghari 2016). To support this idea, some empirical evidence is provided in this discussion. A comparison among the GA, PSO, and reported results in the original paper is made in Table 1. A two-dimensional Rosenbrock function is considered and solved using the mentioned algorithms. The GA is highly sensitive to its parameter selection. This means that if careful tuning of the GA parameters is neglected, it leads to unacceptable results. Table 1 demonstrates that this is the case in the reported results in Table 4 of the original paper. The GA results reported in Table 1 are calculated using a population of 50 individuals, 500 generations, 2 elites, and a 0.9 crossover probability. The stochastic uniform is chosen as selection function, and a heuristic crossover function with a ratio equal to 1.2 is assumed. Adaptive feasible mutation is selected for the mutation function. All of these functions are implemented in the global optimization toolbox of MATLAB. Therefore, they are readily available for researchers to solve their optimization problems. The PSO parameters were selected to equal to 0.78, 1.6, and 1.6 for ω, c1 , and c2 , respectively. The method and parameters definitions of the PSO have been given by Kennedy (2011). The GA and the PSO were executed until 25,000 function evaluations were reached (same as in the original paper). Table 1 indicates that the GA results are better than those of the FA. The same conclusion was made by Garousi-Nejad et al. (2016). They investigated the performance of the FA and modified firefly algorithm (MFA) to solve the continuous 10-reservoir operation benchmark problem. Their results indicated that both FA and MFA fall behind the GA results reported by Wardlaw and Sharif (1999). It is anomalous that how the GA can outperform the FA in the complex continuous 10-reservoir operation problem (Garousi-Nejad et al. 2016), but it cannot solve a simple two-dimensional Rosenbrock function (original paper). The only reasonable conclusion is that the GA in the original paper is not well tuned. The results in Table 1 support this conclusion. Table 1. GA and the PSO Calculated Objective Function of the Rosenvrock Function Run 1 2 3 4 5 Best (minimum) Average Worst (maximum) © ASCE GA PSO −29 1.75 × 10 9.39 × 10−27 1.85 × 10−27 1.66 × 10−27 1.18 × 10−28 1.75 × 10−29 2.61 × 10−27 9.39 × 10−27 GA (reported in original paper) FA (reported in original paper) 0.0044 0.0046 0.0041 0.5313 0.5000 0.0041 0.2089 0.5313 1.03 × 10−13 1.69 × 10−13 1.63 × 10−13 1.65 × 10−14 3.77 × 10−14 1.65 × 10−14 9.78 × 10−14 1.69 × 10−13 −26 1.53 × 10 4.76 × 10−22 9.59 × 10−22 2.85 × 10−25 4.80 × 10−27 4.80 × 10−27 2.87 × 10−22 9.59 × 10−22 07017019-1 J. Irrig. Drain Eng., 2018, 144(1): 07017019 J. Irrig. Drain. Eng. The authors did not consider a constraint on maximum allowable change in storage for each time step. According to Fig. 6 in the original paper, the reservoir storage fluctuates highly, which might detrimentally affect dam stability. Therefore, the following constraint is suggested for future studies in order to take into account dam stability: Downloaded from ascelibrary.org by 220.127.116.11 on 10/27/17. Copyright ASCE. For personal use only; all rights reserved. jStþ1 − St j < SC ð1Þ where Stþ1 = reservoir storage at t þ 1 time step; St = reservoir storage at t time step; and SC = maximum allowable change in storage within each month considering dam stability and safety conditions. The authors successfully solved the Aydoghmoush and Karun-4 reservoir operation problems using an exact method (i.e., nonlinear programming). When an exact method can solve a particular problem, it is not clear that why one should use heuristic methods like the GA or FA. Heuristic methods generally return solutions that are worse than optimal (Sörensen 2015). Exact methods guarantee finding the optimal solution. In order to do that, they not only have to locate this solution in the solution space but also have to prove that they are optimal. This process is exhaustively repetitive because the exact methods have to examine every single solution in the solution space. In some real-world optimization problems, this repetitive calculation is time-consuming and costly. Heuristic methods are best second alternatives in such problems. But, in a one-reservoir operation problem, exact methods perform sufficiently because the numbers of decision variables and constraints are limited. Researchers could perform novel studies on matheuristics instead of focusing on the application of different heuristic algorithms. The idea behind the matheuristics is to combine exact algorithms with a local search via heuristics. This idea leads to more computationally efficient exact methods. The authors assert that a sensitivity analysis was made for GA and FA parameters. But it is not transparent how it took place, and the results of the sensitivity analysis were not given. The main purpose of a sensitivity analysis is to show how parameter selection affects the final results’ accuracy. Therefore, the results of a sensitivity analysis should be the output uncertainty under different given assumptions. The authors could present the sensitivity analysis with box-plots to support and quantify their statements such as “the best solutions from the GA and FA strongly depend on the best settings of algorithmic parameters” of the GA and FA. Also, the way that the sensitivity analysis was carried out is vague throughout the original paper. If the authors had provided details on how the sensitivity analysis was performed, the discusser would be able to compare the results of Table 1 with the results of Table 4 in the original paper. The authors stated that “the process of producing an initial population in the GA and the FA is random. Therefore, the final value of the objective function differs each time the algorithm is run.” Thus, they performed five different runs for each algorithm. Considering the main goal, which is comparing the GA and FA, it is more convenient to exclude the effects of the initial random solutions. When the effects of the initial random solution are excluded, the underlying mathematic mechanism of the algorithm would be the only source that makes a difference in performance. Chaotic maps and fixed pseudorandom seeds can be applied to eliminate the initial random solution’s impacts. Different chaotic maps such as cubic (Xing et al. 2015), logistic (Ma 2012), and Lorenz (Ebrahimzadeh and Jampour 2013) have been used in metaheuristic algorithm so far. Also, the authors could simply use fixed pseudorandom seeds for both the GA and FA. It is highly recommended for future studies that researchers report the pseudorandom seeds when © ASCE they work with a random-based method. Reporting such information helps other researchers to reproduce exact outputs; it will standardize the methodology and help to make comparisons among different methods easier. Figures like Fig. 3 in the original paper and conclusions such as “it is concluded from Fig. 3 that the FA converges faster than GA” are redundant because it is obvious that with the same number of function evaluations, the algorithm that has produced a better answer has a steeper convergence slope (in other words, it converges faster). These same redundant conclusions are numerous in the literature (Bozorg-Haddad et al. 2014; Haddad et al. 2014; Asgari et al. 2015; Azizipour et al. 2016). Numerous published papers are engaged in playing the “up-thewall game” (Burke et al. 2009). The only aim in this game is to get higher up the wall (i.e., obtain better results) than previous studies or other methods. This game comes with no rules, just getting higher up the wall for the sake of publication. Science, however, is not a game. True innovation in metaheuristics research, therefore, does not come from yet another method that performs better than its competitors (Sörensen 2015). For example, the authors show in the original paper that the GA is unable to solve a simple twodimensional Rosenbrock function, but the FA can. Then, GarousiNejad et al. (2016) show that the GA performs better than the FA in the complex continuous 10-reservoir problem. Empirical evidence provided in this discussion supports that the GA used in the original paper had not been tuned well in favor of getting the FA higher up the wall. Publications that play the up-the-wall game are too many to list. Papers by Asgari et al. (2015), Azizipour et al. (2016), and Hamedi et al. (2016), the original paper, and Garousi-Nejad et al. (2016) are some recent examples of research that plays the up-the-wall game without bringing any contribution to water resources engineering. The water resources community is invited to drop and ban this detrimental direction of research. References Afshar, M. H. (2013). “Extension of the constrained particle swarm optimization algorithm to optimal operation of multi-reservoirs system.” Int. J. Electr. Power Energy Syst., 51(10), 71–81. Asgari, H. R., Bozorg Haddad, O., Pazoki, M., and Loáiciga, H. A. (2015). “Weed optimization algorithm for optimal reservoir operation.” J. Irrig. Drain. Eng., 10.1061/(ASCE)IR.1943-4774.0000963, 04015055. Azizipour, M., Ghalenoei, V., Afshar, M. H., and Solis, S. S. (2016). “Optimal operation of hydropower reservoir systems using weed optimization algorithm.” Water Resour. Manage., 30(11), 3995–4009. Bozorg-Haddad, O., Karimirad, I., Seifollahi-Aghmiuni, S., and Loáiciga, H. A. (2014). “Development and application of the bat algorithm for optimizing the operation of reservoir systems.” J. Water Resour. Plann. Manage., 141(8), 10.1061/(ASCE)WR.1943-5452.0000498, 04014097. Burke, E. K., et al. (2009). “Towards the decathlon challenge of search heuristics.” Workshop on Automated Heuristic Design—In Conjunction with the Genetic and Evolutionary Computation Conf., Association for Computing Machinery, New York. Eberhart, R. C., and Kennedy, J. (1995). “A new optimizer using particle swarm theory.” Proc., 6th Int. Symp. on Micro Machine and Human Science, Vol. 1, IEEE, Piscataway, NJ, 39–43. Ebrahimzadeh, R., and Jampour, M. (2013). “Chaotic genetic algorithm based on Lorenz chaotic system for optimization problems.” Int. J. Intell. Syst. Appl., 5(5), 19–24. Gandomi, A. H., Yang, X. S., Talatahari, S., and Alavi, A. H. (2013). “Firefly algorithm with chaos.” Commun. Nonlinear Sci. Numer. Simul., 18(1), 89–98. Garousi-Nejad, I., Bozorg-Haddad, O., and Loáiciga, H. A. (2016). “Modified firefly algorithm for solving multireservoir operation in continuous 07017019-2 J. Irrig. Drain Eng., 2018, 144(1): 07017019 J. Irrig. Drain. Eng. Downloaded from ascelibrary.org by 18.104.22.168 on 10/27/17. Copyright ASCE. For personal use only; all rights reserved. and discrete domains.” J. Water Resour. Plann. Manage., 10.1061 /(ASCE)WR.1943-5452.0000644, 04016029. Haddad, O. B., Moravej, M., and Loáiciga, H. A. (2014). “Application of the water cycle algorithm to the optimal operation of reservoir systems.” J. Irrig. Drain. Eng., 10.1061/(ASCE)IR.1943-4774.0000832, 04014064. Hamedi, F., Bozorg-Haddad, O., Pazoki, M., Asgari, H. R., Parsa, M., and Loáiciga, H. A. (2016). “Parameter estimation of extended nonlinear Muskingum models with the weed optimization algorithm.” J. Irrig. Drain. Eng., 10.1061/(ASCE)IR.1943-4774.0001095, 04016059. Hosseini-Moghari, S. M., and Banihabib, M. E. (2014). “Optimizing operation of reservoir for agricultural water supply using firefly algorithm.” J. Water Soil Resour. Conver., 3(4), 17–31. Jalali, M. R., Afshar, A., and Marino, M. A. (2007). “Multi-colony ant algorithm for continuous multi-reservoir operation optimization problem.” Water Resour. Manage., 21(9), 1429–1447. Kazem, A., Sharifi, E., Hussain, F. K., Saberi, M., and Hussain, O. K. (2013). “Support vector regression with chaos-based firefly algorithm for stock market price forecasting.” Appl. Soft Comput., 13(2), 947–958. Kennedy, J. (2011). “Particle swarm optimization.” Encyclopedia of machine learning, Springer, New York, 760–766. Krishnanand, K. N., and Ghose, D. (2006). “Glowworm swarm based optimization algorithm for multimodal functions with collective robotics applications.” Multiagent Grid Syst., 2(3), 209–222. Lones, M. A. (2014). “Metaheuristics in nature-inspired algorithms.” Proc., Companion Publication of the 2014 Annual Conf. on Genetic and Evolutionary Computation, Association for Computing Machinery, New York, 1419–1422. Ma, Z. S. (2012). “Chaotic populations in genetic algorithms.” Appl. Soft Comput., 12(8), 2409–2424. MATLAB version 7.1 [Computer software]. MathWorks, Natick, MA. Moravej, M., and Hosseini-Moghari, S. M. (2016). “Large scale reservoirs system operation optimization: The interior search algorithm (ISA) approach.” Water Resour. Manage., 30(10), 3389–3407. © ASCE Pan, W. T. (2012). “A new fruit fly optimization algorithm: Taking the financial distress model as an example.” Knowledge Based Syst., 26(2), 69–74. Rechenberg, I. (1973). Evolutionsstrategie: Optimierung technischer Systeme nach Prinzipien der biologischen Evolution, FrommannHolzboog, Stuttgart, Germany (in German). Sörensen, K. (2015). “Metaheuristics—The metaphor exposed.” Int. Trans. Operational Res., 22(1), 3–18. Swan, J., et al. (2015). “A research agenda for metaheuristic standardization.” Proc., 11th Metaheuristics Int. Conf., Lille Univ. of Science and Technology, Villeneuve-d'Ascq, France. Wang, H., et al. (2016). “Firefly algorithm with adaptive control parameters.” Soft Comput., 21(17), 1–12. Wardlaw, R., and Sharif, M. (1999). “Evaluation of genetic algorithms for optimal reservoir system operation.” J. Water Resour. Plann. Manage., 10.1061/(ASCE)0733-9496(1999)125:1(25), 25–33. Weyland, D. (2010). “A rigorous analysis of the harmony search algorithm—How the research community can be misled by a ‘novel’ methodology.” Int. J. Appl. Metaheuristic Comput., 1–2(2), 50–60. Weyland, D. (2015). “A critical analysis of the harmony search algorithm— How not to solve sudoku.” Oper. Res. Perspect., 2(12), 97–105. Xing, B., Gan, R., Liu, G., Liu, Z., Zhang, J., and Ren, Y. (2015). “Monthly mean streamflow prediction based on bat algorithm-support vector machine.” J. Hydrol. Eng., 10.1061/(ASCE)HE.1943-5584.0001269, 04015057. Yang, X. S. (2012). “Chaos-enhanced firefly algorithm with automatic parameter tuning.” Int. J. Swarm Intell. Res., 2(4), 125–136. Yuan, X., Dai, X., Zhao, J., and He, Q. (2014). “On a novel multi-swarm fruit fly optimization algorithm and its application.” Appl. Math. Comput., 233(5), 260–271. Zainal Abidin, Z., Ngah, U. K., Arshad, M. R., and Ong, B. P. (2010). “A novel fly optimization algorithm for swarming application.” IEEE Conf. on Robotics, Automation and Mechatronics, IEEE, New York, 425–428. 07017019-3 J. Irrig. Drain Eng., 2018, 144(1): 07017019 J. Irrig. Drain. Eng.