I do semi-apologize for that headline. But, you know, it's a real thing—in fact, that's how Scarecrow's file monitoring got started. A local business owner came to me with a question: Is there a way to prevent this from happening...again?
It was an interesting conversation! Sadly, the "prevention" part, which involved some server hardening, was not suitable for a general-use tool.
But! Detecting this sort of thing was absolutely feasible. So we built that piece, and gave the business owner a copy for free
We felt his pain, for sure. This poor guy had found out about his site being hacked...by having his customers tell him. Oof.
Here's how the history went:
- We already had a simple uptime-checking service. Just ran from one location, but it was sort of functional (single-location monitoring is a pet peeve of mine).
- We added a website content checker that actually used one FTP protocol or another to look at file listings. It saved the last listing, and notified by email if anything had changed. Neat!
...only that, it turned out, wasn't a complete solution either. Because, later, a customer came to me with a somewhat different scenario: his site was up. His files were unchanged. Sounds great, right?
Only he had some sort of billing dispute with his hosting provider; said provider helpfully put up a page indicating my no-good no-account customer hadn't paid his bill—AS THE SITE'S ONLY CONTENT.
Eep! So we added content-aware checking to cover not only this scenario but ALSO situations in which web page content was served by a database. If the database was down or corrupted? Notification!
Handy! Of course later on we discovered that "site down" didn't necessarily mean what we thought it did (in MOST cases, the outage is only a local hiccup—so single-location monitoring gives a ton of false positives, which is only one of its weaknesses).
We also added transparent backups (another pet peeve: black-box backups that you can't personally inspect) and restores, and multi-continent DNS & certificate monitoring got into the mix too. It all sort of fit together really.
At any rate: this is a real problem. We've seen it many times (of course we would, given what we do here). Ideally? This will never happen to you.
'Course if it does, it'd be nice to know, right? Maybe coupled with a service that not only notifies you, but allows you to restore to a the working site you had two days ago at 8pm via a few clicks?
Well, perhaps it's food for thought. Carry on.