[PDF][PDF] Standing on the shoulders of giants by managing scientific experiments like software

I Jimenez, M Sevilla, N Watkins, C Maltzahn, J Lofstead… - USENIX; login, 2016 - usenix.org
USENIX; login, 2016usenix.org
Independently validating experimental results in the field of computer systems research is a
challenging task. Recreating an environment that resembles the one where an experiment
was originally executed is a time-consuming endeavor. In this article, we present Popper [1],
a convention (or protocol) for conducting experiments following a DevOps [2] approach that
allows researchers to make all associated artifacts publicly available with the goal of
maximizing automation in the re-execution of an experiment and validation of its results. A …
Independently validating experimental results in the field of computer systems research is a challenging task. Recreating an environment that resembles the one where an experiment was originally executed is a time-consuming endeavor. In this article, we present Popper [1], a convention (or protocol) for conducting experiments following a DevOps [2] approach that allows researchers to make all associated artifacts publicly available with the goal of maximizing automation in the re-execution of an experiment and validation of its results.
A basic expectation in the practice of the scientific method is to document, archive, and share all data and the methodologies used so other scientists can reproduce and verify scientific results and students can learn how they were derived. However, in the scientific branches of computation and data exploration the lack of reproducibility has led to a credibility crisis. As more scientific disciplines are relying on computational methods and dataintensive exploration, it has become urgent to develop software tools that help document dependencies on data products, methodologies, and computational environments, that safely archive data products and documentation, and that reliably share data products and documentations so that scientists can rely on their availability.
usenix.org
Showing the best result for this search. See all results