Apps are intermittently loading on custom domains
We’re aware, we’re still working the problem, no need for any more IDs/domains right now.
5 Likes
Status update: We’ve deployed some more mitigations, and most apps we’re monitoring are now loading OK, at least for the initial page load. Still working the problem.
2 Likes
Nice looking app, by the way!
Thank you David! I appreciate you taking a look.
Thanks Meredydd for the update. We appreciate you and the rest of the team working on keeping things running!
Other apps are slow to start, but once started they work fine. This one is still unable to start: ESBTDT7U4BMPNWUQ
Update: We have currently mitigated this issue for most custom domains.
5 Likes
Now all the apps seem to be nice and zippy.
Ironically the one that was the slowest, was the only one (that I tested) that wasn’t doing any query at startup (other than for the user).
My users are receiving different results with some being down while others are not,
I just want to keep communication up. I’m still experiencing issues.
Are mitigations still being implemented?
Thanks!
Same here, random issues, api’s not working correctly
Bad day to test API’s I guess for me. I thought I was doing something wrong hehe… I guess till tomorrow, hopefully the experts at Anvil can resolve this.
1 Like
None of my applications are functioning. Please advise.
1 Like
I’m still suffering from my api’s failing, but I am able to redirect my custom domains to debug domains using cloudflare, for anyone looking for partial releif.
Yes the client does see the odd anvil.app url, but its directly redirected from your custom domain.
2 Likes
That’s a good workaround. Didn’t thought of that. Will do it.
1 Like
I’m assuming the attack is still not mitigated for some domains? I have one app out of several that is still not working.
We believe this attack has now been mitigated for all remaining custom domains. We are continuing to monitor the situation.
4 Likes
Would it be possible to get some triage info on what happened and how it broke? We been using Anvil for about almost 5 years now and this was biggest issue we’ve seen so far.
1 Like
Hi @chris.houston,
What happened was that last night (around 18:30 UTC), a denial of service attack began against one of our customers – essentially, throwing so much traffic at the target that it cannot respond to legitimate requests. This traffic overwhelmed a bunch of our shared infrastructure, and we spent the following few hours mitigating the attack, which is to say allowing access to as much of Anvil as possible while still excluding the malicious traffic. This proceeded in stages, with some apps being restored before others. For obvious reasons, there’s not a lot of detail we can give about exactly how this mitigation was accomplished, but at the time of posting we believe the attack has been fully mitigated and Anvil is back on the road.
5 Likes