Increased number of false-positive alerts

This incident has been resolved.

This weekend we rolled out a change to OnlineOrNot to migrate from a system that checks from a single region by default (us-east), to one that checks around the world (us-east -> eu-west -> singapore or sydney -> us-west) significantly more frequently (up to every 30 seconds for paid plans, 15 seconds for enterprise).

As part of this change, a bad deployment to Singapore resulted in elevated error alerts for almost all of our uptime checks from Singapore - we've since redeployed and limited checks to run from only the US and Europe.

To further mitigate the issue, we sped up free checks to once every 3 minutes, and defaulted to only sending an alert after a failed check in two regions.