Generating Realistic Test Data: Why "John Doe" Hides Your Bugs
What good mock data looks like, the edge cases every dataset should contain, locale and Unicode traps, referential integrity, and why you should never test with production dumps.
Fake data that is too clean is a liability
A table of "John Doe, john@example.com, 123 Main St" tests the happy path and nothing else. Real data has a customer named Zoë O’Brien-Nakamura, an address with three lines and no postcode, a phone number with an extension, and a birthday on February 29th. Software that only ever met clean fixtures fails on launch day — and the failures cluster exactly where the test data was lazy: truncated names, broken sorting, layout overflow, rejected valid input.
- Names: apostrophes, hyphens, diacritics, single-word names, very long names (60+ characters), non-Latin scripts.
- Strings in general: empty, whitespace-only, leading/trailing spaces, emoji, right-to-left text, HTML-looking text like
<b>. - Numbers: zero, negatives, huge values, high-precision decimals, and strings that look like numbers (
007). - Dates: leap days, DST transitions, end of month, far past and far future, timezone offsets.
- Optional fields: actually missing (
null), not just empty strings — the two behave differently in most code.
Referential integrity: rows must relate
Ten thousand random orders with random customer IDs will not exercise your joins — most IDs point nowhere. Good mock data is generated relationally: customers first, then orders that reference real customer IDs, with realistic distributions (a few customers with hundreds of orders, most with one or two). Distributions matter for performance testing too: a uniformly random dataset hides the hot-key problem that a power-law dataset reveals immediately.
Why not just use a production dump?
Because it is real people’s data. Copying a production database into staging, laptops and CI is one of the most common privacy failures there is — the dev environment has weaker access controls, gets shared in screenshots, and outlives its purpose. Under GDPR and similar regimes it is processing without a purpose. Pseudonymization ("just hash the emails") is weaker than it sounds; generated data with production-like *shape* and *volume* gives you the realism without the liability.
Determinism: make it reproducible
Random data that changes every run produces tests that flake. Seed the generator so the same seed yields the same dataset, commit the seed (or the generated fixture) next to the tests, and regenerate deliberately when the schema changes. A failing test should be reproducible from its seed alone — "it fails sometimes" usually means "the data is different sometimes."