ACM Transactions on Graphics (Proceedings of SIGGRAPH 2020) |
||
Code Replicability in Computer Graphics |
Nicolas Bonneel | David Coeurjolly | Julie Digne | Nicolas Mellado | |
Univ. Lyon - CNRS | Univ. Toulouse - CNRS |
Abstract | |
Being able to duplicate published research results is an
important process of conducting research whether to build
upon these findings or to compare with them. This process is
called “replicability” when using the original authors'
artifacts (e.g., code), or “reproducibility” otherwise
(e.g., re-implementing algorithms). Reproducibility and
replicability of research results have gained a lot of
interest recently with assessment studies being led in various
fields, and they are often seen as a trigger for better result
diffusion and transparency. In this work, we assess
replicability in Computer Graphics, by evaluating whether the code is available and whether it works properly.
As a proxy for this field we
compiled, ran and analyzed 151 codes out of
374 papers from 2014, 2016 and 2018 SIGGRAPH
conferences. This analysis shows a clear increase in the
number of papers with available and operational research codes
with a dependency on the subfields, and indicates a
correlation between code replicability and citation count. We
further provide an interactive tool to explore our results and
evaluation data.
|
@article{BCDM20, |
|
Acknowledgements | |
We thank Roberto Di Cosmo for insightful discussions and the reviewers for their constructive feedback. This work was funded in part by ANR-16-CE23-0009 (ROOT), ANR-16-CE33-0026 (CALiTrOp), ANR-15-CE40-0006 (CoMeDiC) and ANR-16-CE38-0009 (e- ROMA). | |
Copyright by the authors, 2020. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in ACM Transactions on Graphics: http://dx.doi.org/10.1145/1122445.1122456 | |