Code, Benchmarks and Experiment Data for the ICAPS 2023 Paper "Computing Domain Abstractions for Optimal Classical Planning with Counterexample-Guided Abstraction Refinement"

This bundle contains the code, scripts, and benchmarks necessary to recall or replicate all experiments detailed in the paper. Additionally, it includes the data generated for the paper.

kreft-et-al-icaps2023-fast-downward.zip contains our implementation based on Fast Downward (https://github.com/aibasel/downward).

The file kreft-et-al-icaps2023-experiment-scripts.zip contains experiment scripts used to run the planner configurations as reported in the paper, as well as scripts to aggregate the raw data in the desired format.

We conducted all experiments using Downward Lab (https://github.com/aibasel/lab). For convenience, we have included a copy of Lab version 7.3 in kreft-et-al-icaps2023-lab.zip.

The kreft-et-al-icaps2023-ipc-benchmarks.zip file contains the IPC benchmarks. It consists of the STRIPS IPC benchmarks used in all optimal sequential tracks of IPCs up to 2018 (suite optimal_strips from https://github.com/aibasel/downward-benchmarks).

The file kreft-et-al-icaps2023-raw-data.zip contains raw experimental data that is distributed over a subdirectory for each experiment. Each subdirectory for an experiment contains runs that are sorted into runs-* directories. Each planner run has its own directory with a unique number and contains symbolic links to the problem domain and task descriptions (domain.pddl and problem.pddl). These links can be resolved by placing the benchmarks in the correct location. The raw data also contains the run log files (run.log and run.err), the script used to start the run (run), and the properties file which contains data parsed from the run logs.

The file kreft-et-al-icaps2023-processed-data.zip contains experimental data as well. However, the directories ending with -eval consist of the properties file, which contains the combined data of all runs of the corresponding experiments. The eval-directories also hold the HTML report of that experiment and a directory called average_eval which contains the averaged data and report. In the average report and properties, all similar experiment configurations that differ only by the random seed are used to generate an average for that configuration.

Note that in order to obtain a working directory of experiments and data, one only needs to merge the domain-abstractions folders of the script and data zips.