What W3 Total Cache Is Really Storing on Disk (and a Plugin to Show You)

I run a large WordPress site that uses W3 Total Cache with the Disk: Enhanced page cache, and I recently started a crawler to keep the cache continuously primed. Fairly quickly something looked off: the cache folder on disk had grown to around 2GB and held roughly 13,000 files - far more than the site has pages, and well beyond the ~7,000 URLs my crawler had actually been through.

W3TC itself gives you almost no visibility into what’s sitting in that folder, so I built a small open-source plugin (more on that at the end) to look inside it. The moment I could see it, the picture got clearer and stranger at the same time: only about 3,600 of those files were live pages. The rest were something else entirely.

That sent me down a rabbit hole into how W3TC actually stores things on disk, and I learned a few things that aren’t documented anywhere obvious.

Disk: Enhanced mirrors your URL tree

The nice thing about the Enhanced page cache is that it isn’t a black box. It stores each cached page as a plain HTML file in a directory structure that mirrors your URLs:

wp-content/cache/page_enhanced/
  example.com/
    charter-yachts/
      some-page/
        _index_slash_ssl.html
        _index_slash_ssl.html_gzip

So the folder path is the URL path, and each cached page gets one or more files. On my site every page had two: an uncompressed variant (_index_slash_ssl.html) and a gzip variant (..._gzip). That already explains part of the gap - “13,000 entries” is a file count, not a page count, and files ≈ pages × variants.

But that only accounted for some of it. The real surprise was how many files ended in _old.

Meet the _old files

W3TC has a “serve stale content while a new version is generated” option. To do that, it keeps the previous copy of a page around with an _old suffix. I expected a handful of these. Instead, a big chunk of my 2GB was _old files - and many directories contained nothing but _old files, with no live page at all.

There are two ways these get created, and the second one is the interesting one.

1. On purge. When a page is flushed (say a post is edited), W3TC doesn’t delete the cached file if serve-stale is on. It renames it. From Cache_File_Generic::delete():

if ( ! $this->_use_expired_data ) {
    return @unlink( $path );        // serve-stale off: just delete
}
$old_entry_path = $path . '_old';
@rename( $path, $old_entry_path );  // serve-stale on: keep it as _old

2. On expiry. This is the one I didn’t expect. With Disk: Enhanced, nginx serves the cached .html straight off disk and never checks its age - so the only thing that enforces your cache lifetime is the garbage collector. And when GC finds a page that’s older than the lifetime, it doesn’t delete it either. It renames it to _old too (Cache_File_Cleaner_Generic::_clean_file()):

} elseif ( ! $this->is_valid( $full_path ) ) {   // primary is past its lifetime
    $old_entry_path = $full_path . '_old';
    @rename( $full_path, $old_entry_path );       // expired page -> renamed to _old
}

So every page that expires becomes an _old file at the next GC run. Those _old files then sit on disk until GC eventually hard-deletes them, which doesn’t happen until they’re older than five times your cache lifetime.

The 30-second twist

Here’s the part that genuinely surprised me. Those _old files are barely ever served. Looking at get_with_old(), an _old copy is only served to a visitor if it was created in the last 30 seconds:

$too_old_time = time() - 30;
...
if ( $file_time > $too_old_time ) {
    return array( $this->_read( $path_old ), $has_old_data );  // serve it
}

And because a rename preserves the file’s original timestamp, an _old file created by GC (from a page that was already hours or days old) is never within that 30-second window. So it can never be served. It just occupies disk for weeks until GC finally removes it.

In other words, for the expiry case, the rename-to-_old does nothing useful. W3TC might as well have deleted the file. It’s an artifact of reusing the same “keep a stale copy” mechanism for both purges and garbage collection.

This also puts the “serve stale content to users” worry to bed. The exposure is a ~30-second window during active regeneration, not the multi-week retention I’d assumed when I first saw the file dates. On a modern setup with fast regeneration, it’s a non-issue.

Why “live” is a rolling window

Once I understood the expiry path, the original mystery solved itself. Under a cache lifetime of 48 hours (which is what I’d had for years), a page stays “live” only if something re-primes it within 48 hours. My crawler takes longer than that to get all the way around ~7,000 pages. So:

• Pages primed in the last 48 hours = live (~3,600 of them).
• Everything the crawler primed earlier had aged past 48 hours, got renamed to _old by GC, and was waiting there as dead weight.

Nothing was broken. The lifetime was doing its job; my crawler simply couldn’t keep the whole site under the 48-hour window. The fix was to raise the cache lifetime well beyond the crawl cycle - I went to the maximum W3TC allows (30 days). With a long lifetime, a page primed once stays live for a month, the crawler has ample time to circle back, pages stop expiring between visits, and the _old churn stops at the source.

Two related notes if you’re tuning this yourself:

• Garbage collection interval doesn’t control any of this. It only sets how often GC runs. The number of _old files is driven by how many pages expire, i.e. your lifetime. Leave the interval alone.
• Check that GC actually runs. GC is a WordPress cron job. If you’ve disabled WP-Cron (DISABLE_WP_CRON) and rely on a system cron, make sure it’s actually firing w3_pgcache_cleanup, or expired files never get cleaned up at all.

The plugin: W3TC Cache Insights

While chasing this, I realised W3TC gives you almost no visibility into what’s actually on disk versus what’s live versus what’s dead. So I built a small plugin to surface it, reusing W3TC’s own measurement code rather than reinventing it. Importantly it only uses W3TC’s “live snapshot” APIs, so it needs no Pro licence and no debug logging - it works on the free version.

It adds a Tools → Cache Insights page with:

• A page cache explorer. Because Enhanced mirrors your URL tree, you can browse it as a folder tree. Each folder shows a recursive rollup: how many distinct pages are cached beneath it, their size, and a breakdown of live / expired / orphaned. So a folder like charter-yachts tells you at a glance how many of that section are cached - and how many are dead _old-only directories.
• Orphaned detection. Directories whose live page is gone and only _old copies remain get counted and totalled, so you can see exactly how much disk the debris is using.
• A cleanup action. A button that deletes the _old files under the current path (the whole cache at the root, or a single branch). It only ever removes files ending in _old - live pages, gzip variants and .htaccess are never touched - and it prunes the empty directories left behind. On my site that reclaimed the ~900MB of legacy _old files in one click.
• Backend stats, a cache-directory overview, recent flushes and the purge log, plus a compact dashboard widget.

Everything expensive (walking the cache directory) runs on demand and is time-bounded and briefly cached, so it never slows down a normal page load - which matters for a plugin that’s meant to help you go faster.

Get it

The plugin is free and GPL licensed. Grab it, use it, and send PRs if you find something to improve:

github.com/robwent/w3tc-cache-insights

If you’re running W3 Total Cache with the Disk: Enhanced engine and you’ve ever wondered what’s actually sitting in wp-content/cache, it’ll tell you - and give you a safe way to clear out the parts that aren’t doing anything.