Test

Building a configuration means making lots of small decisions: which release groups you trust, which sources you prefer, how much lossless audio is worth, and so on. The regular expressions, custom formats, and quality profiles you end up with are the results of those decisions. But a result can’t tell you why it looks the way it does, or which releases you had in mind when you made it. That part stays in your head.

Consider an example; you want a regular expression that matches lossless DTS audio. You have three releases in front of you, one lossless and two lossy, and you write DTS-HD.MA, the label you’ve seen on most lossless DTS releases. It matches the first and not the other two, which is what you want.

Movie.2019.2160p.UHD.BluRay.DTS-HD.MA.5.1-GROUP      lossless
Movie.2019.1080p.BluRay.DTS-HD.HRA.5.1-GROUP         lossy
Movie.2019.1080p.BluRay.DTS.5.1-GROUP                lossy

Weeks later a lossless release comes through labelled just DTS-HD, with no MA, and your expression misses it. You look at that release, shorten the expression to DTS-HD, and check it matches.

Movie.2019.1080p.BluRay.DTS-HD.5.1-GROUP             lossless

What you didn’t check was the lossy release from the first day. DTS-HD.HRA contains DTS-HD, so it matches now too.

Movie.2019.1080p.BluRay.DTS-HD.HRA.5.1-GROUP         lossy, wrong

This is a regression: something you had right, broken by a later change made while you were looking at something else. Nothing told you, because the decision that HRA must not match only ever existed in your head.

Testing takes those decisions out of your head. You write down a release and what should happen to it, and re-run that check after every change, so a regression shows up as a failed test instead of a wrong download weeks later. Each kind of entity has its own kind of test: