[RESOLVED] Anvil Incident - Apps not loading

Hi everyone,

We’re aware that Anvil is currently down. This has been caused by a hardware failure in AWS. We’re working on solving this as quickly as possible!

1 Like

I get the list of apps when I fire the editor up, but when I try and load an app into the beta editor, I just get the spinny thing witth no app. All apps are affected (ie i can’t load any of them).

Anyone else suffering with this today?

edit - ooh, it appears to be affecting published apps as well, At least for me.

Yep, same here, server also lost connection to my uplink server 2 minutes ago

edit,
Anvil websocket open
Connected to “Default Environment” as CLIENT
Anvil websocket closed (code 1006, reason=Going away)

yes app are not loading.

yep me too! even in non beta

looks like all apps are down.

I’ve mailed support. I’m sure they’re already on it. I reckon a klaxxon goes off in their office when things like this happen.

this is the one thing that means using Anvil stops a business passing ISO 9001 stakeholder needs.

Is it possible that there is somewhere a problem?
it is impossible to load any of my apps, not in develop mode nor my private published ones
it keeps spinning on any app and after a few seconds I receive:
Screenshot 2023-01-27 12.10.22
I logged out and logged back in, still the same

Seems working again !!!

1 Like

We are currently experiencing a full outage for all apps.

It seems like apps are not loaded at all.

1 Like

Yeah, all back for me now.

yes all back now. thank you

Hi everyone,

Everything should now be back to normal.

I’m sorry for the interruption. You’re quite right that several klaxons go off when things like this occur, but it still happens more often than we would like.

There’s a lot of redundancy in the Anvil architecture, and we can cope with individual failures in many parts of the infrastructure. Unfortunately, a couple of single points of failure inevitably remain, and this hardware failure took out one of those: the server responsible for managing storage of your apps in git. This meant that many apps continued to work for a while, until caches elsewhere timed out, and then eventually all apps were inaccessible. By that time we were already well on our way to restoring the failed instance.

The maximum total downtime for any app was about 23 minutes, which is clearly much longer than we would like, and we recognise that this has a big impact on many of our customers. Please be assured that we are working to both reduce the number of single points of failure in our infrastructure, as well as recover more quickly when outages like this do occur.

If anyone is seeing any residual issues in the platform, please let us know by responding to this thread.

- Thanks, Ian.

6 Likes

Yes - my app also not working

“You may have been logged out - please refresh the page.”

@Ilqar Thanks for letting us know, sorry about that. Please try again now - I believe that issue is now resolved too.

Thanks. The app seems to load now, but calling something from the database is not possible yet.
11.47 UTC

@kempynck.jan All database operations are working okay as far as we can see at this end. Please can you start a new thread to describe the particular issue you’re seeing? If you can let us know what you’re trying to, and what error you’re seeing, we can investigate. Thanks!

Ok I deleted the latest added row of my database stored during the trouble time and now it works fine. thank you

1 Like

App is back again - Thank you very much Ian !