# Reproducibility and Good Practice

Mathematics I · Scientific Programming · https://tryals.app/learn/mathematics-i/reproducibility-and-good-practice

## Making Computation Trustworthy

A scientific result produced by code is only as credible as the code that produced it. **Reproducibility** is the ability to re-run an analysis and obtain the identical result. It is a cornerstone of scientific integrity.

Good practice relies on these core elements:

| Practice | Purpose |
| --- | --- |
| **Version control** | Tracks changes, records why, and allows reverting to past states safely |
| **Testing** | Verifies code against known answers using **unit tests** |
| **Documentation** | Explains the **why** behind code for future reuse |

Code that runs without error is not automatically correct. A program can execute flawlessly yet still produce wrong answers from a logic bug.

## Controlling Code and Randomness

Simulations introduce **randomness**, which threatens reproducibility. Setting a **random seed** fixes the pseudo-random sequence, making stochastic computations completely reproducible across runs.

You must also **record the environment**, including exact library versions and parameters, because results can shift with software updates.

| Pitfall | Consequence |
| --- | --- |
| **Un-versioned scripts** | Untraceable changes and lost history |
| **No random seed** | Non-reproducible stochastic outputs |
| **Assuming error-free = correct** | Hidden logic bugs or **regressions** slip through |

Good practice separates a throwaway script from a reliable scientific instrument.

## Practice questions

8 of this lesson's 13 practice questions, with answers. The full set is in the app.

### 1. A NumPy vectorised operation and an equivalent Python loop compute identical results, and the vectorised one can be fifty times faster. Where does the difference come from?

A. From automatically running on the GPU rather than the CPU
B. From leaving the interpreter for compiled C over contiguous memory
C. From a mathematically superior algorithm used by the library
D. From performing fewer arithmetic operations overall

**Answer:** B. From leaving the interpreter for compiled C over contiguous memory

**Why:** Same arithmetic, different overhead: a Python loop pays bytecode dispatch and type checks per element, while one NumPy call crosses into C once and runs a tight loop over contiguous memory with SIMD. It is why 'vectorise it' is the first optimisation in scientific Python, and why the advice does not transfer to C.

Page: https://tryals.app/practice/mathematics-i/reproducibility-and-good-practice/a-numpy-vectorised-operation-and-an-equivalent-python-loop-compute

### 2. A colleague sends a script and the number it printed. Select everything you still need before that result can be called reproducible.

A. The exact version of the code that produced it
B. The random seed that was used
C. The library versions it ran against
D. A proof that the algorithm is optimal
E. A faster computer

**Answer:** A. The exact version of the code that produced it; B. The random seed that was used; C. The library versions it ran against

**Why:** Code version, seed and environment are exactly the three things that can change the number. Speed and optimality are worth caring about for other reasons, but neither affects whether the result can be reproduced.

Page: https://tryals.app/practice/mathematics-i/reproducibility-and-good-practice/a-colleague-sends-a-script-and-the-number-it-printed-select

### 3. Vectorising usually gives a large speed-up, but sometimes almost none. Why does replacing a loop with an array operation sometimes fail to help?

A. When each step depends on the previous result, or memory is the limit
B. When the arrays contain integers rather than floating-point numbers
C. When the loop body contains more than one arithmetic operation
D. Never: vectorising always produces a substantial speed-up

**Answer:** A. When each step depends on the previous result, or memory is the limit

**Why:** Vectorising removes per-iteration interpreter overhead. Where that overhead was not the bottleneck — a genuine dependency chain, or a memory-bound operation — there is nothing for it to remove.

Page: https://tryals.app/practice/mathematics-i/reproducibility-and-good-practice/vectorising-usually-gives-a-large-speed-up-but-sometimes-almost

### 4. Two runs of the same simulation with the same fixed seed and the same library versions should produce identical output.

**Answer:** True

**Why:** True. Pseudo-random values are computed deterministically from the seed, so fixing it fixes the whole sequence and hence the result. This is what makes a stochastic method reproducible without making it any less stochastic.

Page: https://tryals.app/practice/mathematics-i/reproducibility-and-good-practice/two-runs-of-the-same-simulation-with-the-same-fixed-seed-and-the-same

### 5. Sort each practice by its effect on reproducibility.

**Answer:**

- Makes the result reproducible: Committing the analysis code to version control, Setting a fixed random seed before the simulation, Recording the exact library versions used
- Leaves the result irreproducible: Editing the script by hand between runs without keeping copies, Relying on an unseeded random number generator

**Why:** Version control, a fixed seed and recorded versions together pin down exactly what produced a number. Untracked edits and unseeded randomness make the result unrepeatable even by its own author.

Page: https://tryals.app/practice/mathematics-i/reproducibility-and-good-practice/sort-each-practice-by-its-effect-on-reproducibility

### 6. Complete the purpose of a test suite.

**Answer:** Running the tests after every change catches **regressions** immediately.

**Why:** A regression is a bug introduced by a change, and catching it the moment it appears is what makes the suite worth running. Round-off and conditioning are properties of the mathematics, not defects a test can remove.

Page: https://tryals.app/practice/mathematics-i/reproducibility-and-good-practice/complete-the-purpose-of-a-test-suite

### 7. A test suite contains 40 unit tests, of which 36 passed before a change. The change makes 3 previously passing tests fail and fixes 1 previously failing test. How many tests pass now?

**Answer:** 34

**Why:** 36 - 3 + 1 = 34. The net loss of two hides the fact that something was also fixed, which is exactly why the suite reports individual test names rather than a single score.

Page: https://tryals.app/practice/mathematics-i/reproducibility-and-good-practice/a-test-suite-contains-40-unit-tests-of-which-36-passed-before-a

### 8. Order the steps of putting an analysis on a reproducible footing.

**Answer:**

1. Put the analysis script under version control
2. Write unit tests checking each function against known answers
3. Fix the random seed so runs are repeatable
4. Record the library versions and parameters used
5. Publish the code and data alongside the result

**Why:** Version control first, because everything afterwards is a change worth tracking; then correctness, then controlled randomness, then the environment, then release. Publishing code that is untested and unseeded spreads the problem rather than solving it.

Page: https://tryals.app/practice/mathematics-i/reproducibility-and-good-practice/order-the-steps-of-putting-an-analysis-on-a-reproducible-footing