Some experiments with CPLEX automated tuning tool

Tuning strategies to get the most out of a solver seem an important issue to me. I a recent post, I looked at a strategy consisting of emphasizing cutting plane types that were generated by CPLEX using default settings. Following a suggestion from Paul A. Rubin, I decided to give a try with CPLEX’s automated tuning tool. To do this, I used the same set of 20 instances of a multi-period capacitated lot-sizing model that was used in tuning strategy post. I just let the automated tool run and then I note what it suggests.

Conclusions to this experiment are:

  • Overall, the tuning tool seems to yield fairly consistent results for instances of the same model. Three strategies emerge across the 20 tests, one of which is to use the default settings. (See below for more details). These three strategies are rather similar and account for 85% of the runs.
  • The automated tuning can consume a lot of computer resources and time (from 3 minutes to ±18 hours for one model).
  • The tuning tool tends to need a lot more time on instances that are harder to solve. Unfortunately, it’s on these instances that I need good parameter values. If one instance solves in 1 minute, I don’t need optimal parameters to reduce that to 54.5 seconds.
  • Overall, it typically takes between 3 and 6 times more time to perform the tuning than to solve the model using CPLEX defaults. The average (time to perform tuning) / (time to solve with default settings) ratio is 4.73.
  • In console mode, if you run out of memory while in the tuning phase, CPLEX 12.4 will terminate without giving you an error message. Of the 20 tests, 2 made CPLEX crash. This issue, however, has been fixed in CPLEX 12.5.

Actual strategies proposed by tuning

  • Use default settings (7 times);
  • Set Mixed integer rounding MIR) cuts to moderate level (5 times);
  • Set maximum aggregation for Gomory cuts to 10,000, limit cuts to 10 passes and limit cut factors to 30 (5 times);
  • No answer (2 times);
  • Disable heuristics (1 time).
There does not seem to be any kind of obvious relationship between CPLEX’s ability to solve an instance quickly (or not) with default settings and  its ability to propose better parameters. That being said, there is not much that can be concluded on experiments of this size: 20 instances generated randomly using the same probability functions isn’t very general.
These tests were performed using CPLEX Optimization Studio 12.4 on a computer using Intel Xeon E5405 CPU (4 cores, 2 GHz), with 8 GB of RAM at 1333 MHz. The set of models used is P30-T21-C2 (30 products, 21 periods, with moderately tight capacities) and it can be downloaded here.

Comments

  1. Nice post!
    Do you have any idea if ParamILS (*) does a better tuning?

    (*) http://www.cs.ubc.ca/labs/beta/Projects/MIP-Config/

Trackbacks

  1. […] seeks to find good parameters for the solver for a given optimization model. The use of this tool is rather straightforward on easy models, it is less obvious to use it efficiently on harder models. I talked about this with Dr. Ed Klotz […]

Speak Your Mind

*