Practice question · Put in order
Order the steps of putting an analysis on a reproducible footing.
- Write unit tests checking each function against known answers
- Put the analysis script under version control
- Publish the code and data alongside the result
- Record the library versions and parameters used
- Fix the random seed so runs are repeatable
Hints
- Tracking changes must come first, or the work you do next is untraceable.
- Publication is the last step, once everything it refers to is pinned down.
Show the answer
- Put the analysis script under version control
- Write unit tests checking each function against known answers
- Fix the random seed so runs are repeatable
- Record the library versions and parameters used
- Publish the code and data alongside the result
Why
Version control first, because everything afterwards is a change worth tracking; then correctness, then controlled randomness, then the environment, then release. Publishing code that is untested and unseeded spreads the problem rather than solving it.
Practise Reproducibility and Good Practice
The app has 5 more questions on this lesson, and keeps your place in the course. Mathematics I is free to start.
More questions on Reproducibility and Good Practice
- A NumPy vectorised operation and an equivalent Python loop compute identical results, and the vectorised one…
- A colleague sends a script and the number it printed. Select everything you still need before that result can…
- Vectorising usually gives a large speed-up, but sometimes almost none. Why does replacing a loop with an…
- Two runs of the same simulation with the same fixed seed and the same library versions should produce…
- Sort each practice by its effect on reproducibility.
- Complete the purpose of a test suite.
- A test suite contains 40 unit tests, of which 36 passed before a change. The change makes 3 previously…