Japanese Language 日本語はこちら

Benchmark Problem Based on Real-World Car Structure Design Optimization
Mazda Bechmark Problem

August 1, 2018

Design problems in real world often have many constraints. Design optimization methods that can efficiently obtain an optimal design or Pareto-optimal designs for such problems are needed. Evolutionary algorithms can efficiently obtain optimal design(s) for such problems and have been studied using benchmark problems such as CDTLZ problems and real-world-like problems. However, Tanabe et al showed that those benchmark problems have inappropriate features [1]. For example, most of infeasible designs violate only one constraints.
For this reason, we decided to provide simultaneous optimization problem of multiple car models as a benchmark problem under a joint research by Mazda Motor Corporation, Japan Aerospace Exploration Agency, and Tokyo University of Science. This problem was solved on "K computer" [2].
Here, total weight of three types of cars is minimized while number of common thickness parts among the three types of cars is maximized. Design parameters are thickness of 222 structural parts. This problem has 54 constraint functions such as collision safety performances. This benchmark problem can be used as a single-objective design optimization benchmark problem (weight minimization problem) or multiobjective design optimization benchmark problem (weight minimization and number of common parts maximization).
In the optimization performed on K computer, structural simulation software LS-DYNA was used to evaluate the constraints on collision safety [3]. In the benchmark problem, constraints on collision safety are modeled by response surface approximation. Please refer to reference [4] for details. Sample results are presented in [5].
This benchmark problem is released as C ++ source code. Thus, it should work on any operating system. You can compile it on your own environment to use it. Executable file that is checked on windows 10 machine (64 bit version, Visual Studio 2015/2017) is also included.
We expect that this benchmark problem contributes to research and development of a robust and efficient design optimization algorithm for real-world problems.

[1] Ryoji Tanabe and Akira Oyama, "A Note on Constrained Multi-Objective Optimization Benchmark Problems," 2017 IEEE Conference on Evolutionary Computation, 2017.
[2] What is K?
[3] Akira Oyama, Takehisa Kohira, Hiromasa Kemmotsu, Tomoaki Tatsukawa, and Takeshi Watanabe, "Simultaneous structure design optimization of multiple car models using the K computer," 2017 IEEE Symposium Series on Computational Intelligence, 2017.
[4] Takehisa Kohira, Hiromasa Kemmotsu, Akira Oyama, and Tomoaki Tatsukawa, "Proposal of Benchmark Problem Based on Real-World Car Structure Design Optimization," The Genetic and Evolutionary Computation Conference (GECCO) 2018, 2018.
[5] Hiroaki Fukumoto and Akira Oyama, "Benchmarking Multiobjective Evolutionary Algorithms and Constraint Handling Techniques on a Real-World Car Structure Design Optimization Benchmark Problem," The Genetic and Evolutionary Computation Conference (GECCO) 2018, 2018.

How to download the benchmark problem
Download the benchmark problem from here. We appreciate it if you provide your name and affiliation to benchmark@flab.isas.jaxa.jp after you download. The provided information may be used for update notices and statistical analysis.

When you publish results
We appreciate it if you refer to the reference [4] in your publications.
We also appreciate it if you send the publication to benchmark@flab.isas.jaxa.jp

Frequently asked questions
(1) What is included in the provided file?
It includes
- explanation of design problem /Info_Mazda_CdMOBP.xlsx
- executable file for windows (64bit) /Mazda_CdMOBP/bin/win64
- executable file for linux(64bit) /Mazda_CdMOBP/bin/Linux
- R script for post-processing /Mazda_CdMOBP/post
- sample input/output files /Mazda_CdMOBP/sample
- source code (C++) /Mazda_CdMOBP/src
- sample optimization result /Calculation_sample_Mazda_CdMOBP
(2) How to understand the optimization problem?
Read the references [3], [4] and [5].
(3) Are design variables treated as discrete variables or continuous variables?
We recommned to treat the design variables as discrete variables because the original problem is a discrete variable optimization problem. See H column of the sheet named "Explain_DV_and_Const" in Info_Mazda_CdMOBP.xlsx for discretization. You may treat them as continuous variables if you like. However, clearly mention how the design parameters are treated in your publications because optimization result depends on how they are treated.
(4) Are there any constraints among design parameters?
Yes, there are. See reference [4].
(5) All constraints are functions of all design parameters?
No. Constraints on a car model is a function of design parameters of the corresponding car model.
(6) What are appropriate population size and number of generations?
In industrial design problems, objective/constraint function evaluation is often very expensive. For example, in Mazda's car structure design, one collision evaluation using LS-DYNA took 10 hours. Therefore we recommend maximum number of design evaluations to be equal or less than 30,000. Of course, smaller is better.
(7) How to define upper and lower boundaries of search space?
They are written in E and F columns of the sheet named "Explain_DV_and_Const" in Info_Mazda_CdMOBP.xlsx
(8) Can you provide sample data used to generate the approximated response surface?
We cannot provide the sample data.
(9) How to calculate hypervolume value?
Obective functions should be normalized as
mass: f1 = (f1-2.0)/(3.0-2.0) = f1-2.0
number of common parts: f2 = (f2-0)/(74-0) = f2/74 . Then calculate hypervolume value with the reference point (1.1, 0.0).

If you have further question, send e-mail to benchmark@flab.isas.jaxa.jp