Property-Based Testing (Shrinking & Oracles).

🧾Friday afternoon

A booking service merges a user's busy intervals (minutes since midnight) before computing free slots:

def merge(ranges):

rs = sorted(ranges)
out = [rs[0]]
for s, e in rs[1:]:
    if s < out[-1][1]:                 # overlap?
        out[-1] = (out[-1][0], max(out[-1][1], e))
    else:
        out.append((s, e))
return out

31 unit tests. 97% line coverage on the module. Ships.

Monday: a user with 09:00–10:00 and 10:00–11:00 sees TWO busy blocks instead of one. The free-slot engine computes gaps as (prev_end, next_start), finds a 0-minute "gap" at 10:00, offers it, and someone books it. Double booking.

🔍Why 31 tests missed it

Every test an engineer wrote had either a real overlap or a visible gap. Nobody wrote the touching case e1 == s2, because humans don't type that. Coverage measured lines executed — not input space explored. The < should be <=, and no amount of 97% tells you that.

🎲 Stop writing examples. State a law.

from hypothesis import given, strategies as st

pair = st.tuples(st.integers(0,1440), st.integers(0,1440))

@given(st.lists(pair, min_size=1))

def test_output_is_disjoint(ranges):

out = merge(ranges)
for (s,e),(s2,e2) in zip(out, out[1:]):
    assert e < s2        # no overlap, no touching

Runs in ~0.3s. Output:

Falsifying example: ranges=[(0, 0), (0, 0)]

🪓Shrinking is the actual product

The first failure it hit was more like [(338,1200),(1200,1201),(97,110)]. Hypothesis then re-runs with simpler inputs while the test still fails, and hands you the minimal one. That loop is why PBT output is debuggable instead of fuzzer noise — and it's the part your hand-rolled for _ in range(1000): random... doesn't have. (JS: fast-check, same idea.)

🧠Picking properties — where people actually stall

Four shapes that almost always apply:

• Round-trip — decode(encode(x)) == x
• Invariant — output sorted / disjoint / total preserved
• Oracle — agrees with a slow, obviously-correct version
• Metamorphic — merge(a+b) == merge(merge(a)+merge(b))

⚠️The four ways people make PBT useless

• Tautology. Asserting with the same logic the code uses tests nothing.
• Over-constrained generators. st.integers(0,1440).filter(lambda x: x > 0) would have excluded the very bug above. Generate wide, assume less.
• Calling a failure a flake and re-running CI. A PBT failure is real. Hypothesis stores counterexamples in .hypothesis/examples and replays them first — commit that dir, or use @settings(derandomize=True) so CI is reproducible.
• Deleting unit tests. PBT explores the space; examples remain the named regression tests for bugs you've shipped. Keep both.

💰 Rough trade: one property ≈ hundreds of examples you'd never finish writing, at ~100 generated cases per run by default (max_examples=100). Bump it in nightly CI, not on every push.

🏋️Your exercise

First fix the real bug: < becomes <=. Now break it a second way — drop the max(...) so the merge line reads out[-1] = (out[-1][0], e). Run the disjointness property again on input [(0,10),(1,2),(5,6)]: it returns [(0,2),(5,6)] and the property PASSES. Still sorted, still disjoint — and it silently threw away the minutes 2–10.

So write the oracle property: expand every range into a set of minutes, union them, and assert the merged output covers exactly that same set. Which bug does each property catch — and what does that tell you about invariants vs oracles?