John Hughes - Testing the Hard Stuff and Staying Sane
Channel:ClojureTV · Length:47:17 · Published:2014-03-26 · Views:39216
Summary
John Hughes, co-creator of QuickCheck, talks about property-based testing of hard systems. He shows production examples where it found very low-level bugs, including issues in the CAN bus standard.
Why HN recommends it
Recommended as the best introduction to property-based testing and cited for its examples of bugs found in the CAN standard.
John Hughes (not the filmmaker) gives a great talk about it — HN, 2022
Functional programming · Testing, debugging & developer tools
Subtitles (excerpt)
[Applause] Okay. Well, thank you very much for the invitation. It's been a great pleasure to be here. Um, I want to start by asking you all a question. Who really really loves testing? Oh, well, a few, but but not very many. So, I didn't really used to enjoy it very much either, but then I uh co-invented quick check. So, so why is testing hard fundamentally? Well, suppose you're you're writing automated tests for a system and you've got say n features that you want to test. What are you going to do? You're going to write maybe three or four tests per feature, right? That's okay. That's a linear amount of work. But we all know that you won't catch all of the bugs that way because there are some bugs that only appear when you use two features together. Oh, well, if you want to test for all of those possibilities, then you've got to write a quadratic number of test cases. That's starting to sound not so much fun. And actually, some bugs don't appear just when you use two things together. Sometimes you need to use three things together to provoke them. If you want to test for all of those, you're going to be writing a cubic number of test cases. And this is really starting to r sound like a lot of work.
On Hacker News
I'd recommend watching this [VIDEO] to see how powerful this can be. I've just discovered generative testing due to clojure.spec. Describing specs for your data works not only to help validate data structures but can be plugged in as properties for generative testing. After the "aha" moment it was followed by a feeling that I've wasted so much time writing specs just to validate data structures a… — Naomarik, 2016-12
One of the authors, John Hughes did a talk on property-based testing at Clojure West some number of years back. Worth a watch if you're interested: [VIDEO] — devin, 2026-04
I feel like this talk by John Hughes showed that there is real value in this approach with production systems of varying levels of complexity, with two different examples of using the approach to find very low level bugs that you'd never think to test for in traditional approaches. [VIDEO] — robertfw, 2025-11
While of course Haskell has more normal testing infrastructure available (eg. [LINK] ), my favorite bit of Haskell testing is QuickCheck, which IIUC started life in Haskell and has been reimplemented in other languages with various degrees of effectiveness and various degrees of connection to the original project. John Hughes (not the filmmaker) gives a great talk about it: [VIDEO] — dllthomas, 2022-10
Possibly due to amount of data coming from various sensors and systems. BTW, CAN standard is riddled with bugs: [VIDEO] at 26:18 — troupo, 2023-11
Related videos
- "Transducers" by Rich Hickey · Strange Loop Conference · 45:00
- Brian Goetz - Stewardship: the Sobering Parts · ClojureTV · 1:03:30
- Lenses, Folds, and Traversals · Edward Kmett · 1:54:45
- Can You Watch This Without Getting Angry? · CGP Grey · 6:16
- Ruby Conf 12 - Y Not- Adventures in Functional Programming by Jim Weirich · Confreaks · 53:21
- RailsConf 2014 - All the Little Things by Sandi Metz · Confreaks · 38:47
- The Soul of Erlang and Elixir • Sasa Juric • GOTO 2019 · GOTO Conferences · 42:02
- "Simple Made Easy" - Rich Hickey (2011) · Strange Loop Conference · 1:01:38
- "Making Impossible States Impossible" by Richard Feldman · elm-conf · 25:06
- William Byrd on "The Most Beautiful Program Ever Written" [PWL NYC] · PapersWeLove · 1:31:05
- "Testing Distributed Systems w/ Deterministic Simulation" by Will Wilson · Strange Loop Conference · 40:20
- 🚀 TDD, Where Did It All Go Wrong (Ian Cooper) · DevTernity Conference · 1:03:55
- Bret Victor - Inventing on Principle · Rui Oliveira · 54:20
- The Value of Values with Rich Hickey · InfoQ · 31:43
- Why Isn't Functional Programming the Norm? – Richard Feldman · Metosin · 46:09
- Maybe Not - Rich Hickey · ClojureTV · 1:03:28