Degree of reproducibility

Since our laboratory deals with computational problems of various sizes using a diverse collection of software and hardware tools, not all result files are easily reproducible for every reader. Consequently, application makefiles typically define three result list variables: RESULTSER, RESULTSCR, and RESULTSNR. The endings ER, CR, and NR indicate to the reader the degree of reproducibility:

• ER: Easily reproducible result files can be regenerated within ten minutes on a standard workstation.
• NR: Non-reproducible result files, such as hand-drawn illustrations or scanned figures, cannot be recalculated by the reader.
• CR: Conditionally reproducible result files require proprietary data, licensed software, or more than 10 minutes for their recomputation. The author nevertheless supplies a complete set of source files and rules to ensure that a reader can reproduce the results if he possesses the necessary resources. Additionally the author lists these resources in a warning file (e.g. myresult.warning), which accompanies the conditionally reproducible result file (e.g., myresult.ps). In industrial-scale geophysical research, many interesting results are conditionally reproducible.

The standard make targets burn and build are complemented by targets burnER, burnCR, burnNR, burnall and buildER, buildCR, buildNR, buildall. For example burnCR burns all conditionally reproducible result files. The target burn is defaulted to burnER to restrict the standard removal of result files to easily reproducible ones. The target build is defaulted to buildER to recompute the result files that make burn removes.