My Website Has a Test Suite

Deprecations, external dependency shutdowns, infrastructure changes, data changes and shifting standards. Websites rot: each broken piece sits there for months until a reader (or worse, a potential employer) finds it before you do. I got tired of discovering this stuff by accident, so I gave it a test suite.

This post covers a static audit script that reads every page like a very pedantic proofreader, and a headless browser test that loads the site like a very terminal focused nerd. These both run automatically on every push.

The proofreader

The first half is audit_site.py, a Python script with no dependencies beyond the standard library. It asks git for every tracked HTML file, parses each one, and complains about everything it doesn't like. It requires zero errors, zero warnings, on every page, on every commit.

What it checks, roughly in order of how often it has saved me:

• Broken links and images. Every internal link and image source is resolved to an actual file on disk. A renamed page or a missing thumbnail fails the audit immediately.
• Metadata. Every page needs a description of sensible length, a canonical URL that matches its filename, and the social preview tags that make links look respectable when shared. The audit also validates that the preview image actually exists (this caused me so many issues trying to understand how social media sites grab images).
• Structured data. Each blog post carries a machine-readable summary for search engines. The audit parses every one and checks each URL inside it, because invalid structured data fails silently in the real world.
• Accessibility basics. Images without alt text, duplicate element ids, missing or multiple h1 headings, and skipped heading levels all get flagged for accessibility requirements I set.
• The data files. The blog listing is driven by a JSON file: every entry must point at a real page and a real image, every date must be well-formed, and every tag must come from a fixed allowed list. That last one exists because the tag buttons are generated dynamically, so a typo like "phtography" wouldn't error; it would just mint a brand-new filter button and display it.
• The feeds. The RSS feed and the sitemap are parsed and reconciled against the post list, so publishing a post without wiring it up everywhere gets caught. Not much point in publishing a new blog that isn't shown anywhere!

Headless smoke tests

The audit reads files; it cannot tell you whether the page actually works. This site injects its header, footer, comments, and related posts with JavaScript, and it animates sections into view as you scroll. A one-character mistake in that JavaScript can leave a page technically valid and completely blank, and the audit wouldn't catch it.

So the second half is a smoke test built on Playwright, which drives a real, invisible Chromium. The script serves the repository over a local web server, loads six representative pages, waits a few seconds for the JavaScript and animations to settle, and then asserts on what a visitor would actually see:

# A few of the checks, one page each:
# index.html   - hero present, live-stats skeletons cleared, blog cards visible
# blog.html    - at least 3 post cards and 3 tag filter buttons rendered
# gallery.html - at least 12 photos in the grid
# a blog post  - body text visible (opacity != 0), table of contents built
# quotes.html  - at least 500 quote cards
# map.html     - the map initialised with at least 10 markers

The site reveals sections with a fade-in animation, which means a JavaScript failure doesn't remove content, it leaves content sitting in the page at opacity zero: present in the HTML, invisible to humans. That is the failure a static checker can never catch. The check is a Python tuple holding a line of JavaScript, evaluated inside the live page:

("post body visible",
 "(() => { const p = document.querySelector('.blog-post > p');"
 "  return p && getComputedStyle(p).opacity !== '0'; })()"),

The test also collects console errors, ignoring the third-party noise (analytics, embeds) and failing on anything from my own code.

The robot that runs it all

Both halves run in GitHub Actions on every push. The workflow has two jobs: one installs Python and runs the audit plus a check that the minified stylesheet is up to date (the CSS build is its own story); the other installs Playwright and runs the smoke tests. If either fails, I get an email and a red X on the commit within a few minutes. A local pre-commit hook runs the fast checks too, so most mistakes never even reach GitHub.

Is this overkill for a personal site? No: it's because the site is a hobby that I need automation. Nobody is paid to notice when this site breaks (or, you know, paid for writing blogs, creating the site, etc.). There is no QA department, no on-call rota, just me, and I would rather spend my evenings writing posts than manually clicking every link on fifty pages. The test suite is what lets a one-person site behave like it has staff.

The suite also enforces my rules on me. Zero errors and zero warnings sounds strict, but a clean baseline means any non-zero number is news.

What it still misses

The audit checks that every referenced image exists on disk. It does not check that the image is committed to git. I published a post whose hero image existed on my machine but had never been committed, every local check passed, and the live site served a grey void where a seascape should have been. A reader (fine: me) spotted it and fixed it later that day, but that's not great when I already had a few dozen readers.

Each gap you find becomes the next check you write. The fix here is a rule that any local file referenced by a tracked page must itself be tracked, which is one more function in the audit script. Here is that check as it runs today (lightly trimmed):

def check_target(url, line, what="target"):
    local = resolve_local(page, url)
    if local is None:
        return  # external URL, checked elsewhere
    if not local.exists():
        add("ERROR", page, line, "broken-link", f"missing {what}: {url}")
    elif page_is_tracked:
        relp = local.resolve().relative_to(ROOT).as_posix()
        if relp not in tracked:
            add("ERROR", page, line, "untracked-ref",
                f"{what} exists locally but is not tracked by git: {url}")

Fittingly, that check fired while I was preparing this very post: the hero image at the top of this page existed on my machine and had never been committed. Test suites are never finished, only extended.