Can’t access my projects at the moment.
I can’t also access https://anvil.works/build/
??
1 Like
Same for me, 504 error on my apps, and can’t access the Anvil homepage.
Yes, it is. Including a failure of the alerting system, which failed to wake us up until now (currently 5:30am UK time).
Posting this was clearly always going to bring down the wrath of the Uptime Gods, but I didn’t expect it to be this dramatic. Stand by please.
EDIT: And we’re back.
Update: Now we’ve had time to do a post-mortem, we’ve concluded that the proximate cause was in relatively minor (a deadlock bug exposed by scaling pressures, which was dislodged by a global restart and fixed within minutes - it would have been a couple of minutes of downtime, but not a major incident). The main cause of the prolonged outage was the alerting failure. We’ve identified the failure, have applied appropriate quantities of duct tape, are now doing a ground-up rework of our alerting system. Thank you for your patience!
5 Likes
It’s back now. Thanks for fixing the issue
1 Like
If for nothing else but your own sanity, you guys really need an ops team…
1 Like
This outage had me thinking that perhaps the build environment (IDE) should be separated from where you host Anvil apps to limit the downtime exposure across the entire ecosystem.
While Anvil was down, I couldn’t even continue to build new features and test them on a local Anvil app server.
Perhaps something to consider for the future.
5 Likes
I think it is a very good idea.
1 Like
True but unfortunately good ideas cost money to implement:)