← All case studies Forensic Troubleshooting

The Weekend Crash

Membership organisation, annual conference, registration window open

Down Live

Site status

4h+ 47min

Total downtime

The situation

The client ran the website for a professional membership organisation. Their annual conference was approaching and registration had opened four days earlier. Several hundred members were expected to register over the weekend.

The site went down on a Friday evening at approximately 18:40. The admin panel was inaccessible. The front-end was returning a PHP fatal error. The previous developer, who had done some work on the site earlier that week, was not reachable.

The client contacted me at 19:03. They had already been down for 23 minutes. Conference registrations were failing, and members were beginning to contact the organisation directly.

The diagnosis

The PHP fatal error message, once retrieved from the server error logs via SFTP, was an uncaught exception from a plugin — a type mismatch error on a function that expected an array but was receiving null.

The call stack pointed to a recent plugin update. Cross-referencing the file modification timestamps confirmed: the plugin in question had been auto-updated at 18:37, three minutes before the site went down. The update introduced a breaking change that the plugin developer had not tested against the PHP version on this server.

The previous developer's work earlier in the week was unrelated. The cause was a plugin update with a compatibility regression.

The recovery

Because the admin panel was inaccessible, the recovery had to be done via SFTP and WP-CLI directly.

  • Plugin disabled via filesystem

    The offending plugin's directory was renamed via SFTP, preventing WordPress from loading it. This restored site access without requiring database access or admin credentials.

  • Previous version restored

    The plugin's previous version was retrieved from the WordPress.org SVN repository and deployed via SFTP. The site was restored to full functionality at 19:50.

  • Root cause confirmed

    With the site live, the error was reproduced on a staging copy with the updated version to confirm the diagnosis. The compatibility issue was verified and documented.

  • Auto-updates disabled for this plugin

    Automatic updates for the offending plugin were disabled pending the developer issuing a fix. A note was added to the client's documentation to monitor the plugin changelog.

  • Permanent fix deployed

    The plugin developer released a patched version on the Monday. It was tested on staging first, then deployed to live with auto-updates re-enabled for this plugin specifically.

The outcome

The site was restored at 19:50 — 47 minutes after first contact, 70 minutes after the outage began. Conference registration continued over the weekend without further incident. No registration data was lost.

The client had not previously been aware that WordPress auto-updates could take their site down without warning. Following the incident, we implemented a staged update process — updates are now applied to a staging environment and verified before being applied to the live site.

WordPress auto-updates are a security feature that solve one problem by creating another. Applying them without a staging verification step is a calculated gamble. For a site with active revenue or time-sensitive traffic, that gamble is not worth taking.

From the blog

Further reading

All articles →

Site down? Contact me directly.

Emergency recovery for WordPress sites. I work systematically under pressure, communicate clearly throughout, and document everything for your records.