[Resolved] Announcement: Anvil outage

Announcements
21

As suspected, the attack continues to evolve. We’re obviously doing everything we can to bring things back!

9 Likes
22

Any Updates?
This situation is becoming increasingly critical for us.

23

Thank you for being on this. I know these are appalling to deal with in real-time. Good work all and keep it up!

24

We’re responding to changing attacks as fast as we can, and right now things are mostly up again. We continue to keep a close eye on all services and apps.

1 Like
25

I am still getting ‘anvil.server.RuntimeUnavailableError: Server resources not available for ‘python3-full’. If this error persists for more than a few minutes, please contact support@anvil.works.’ which I guess is related to the DOS attack.

26

It is indeed, but this should now be resolved too.

1 Like
27

Sorry mate I still have a few apps that are still not connecting to the backend?

28

Seems OK to me now. Have run my app in the IDE and via the published web version and both working ok and giving expected results.

I have already been asked by a senior colleague ‘is there a back up protocol needed if this happens again or proves to be a persistent occurrence?’. I don’t have an answer to that. Could Anvil Support provide some reassurance. I appreciate it is impossible to prevent a DOS attack starting.

Thanks for reacting so professionally Anvil Support.

Regards
Steve

29

Issues are still very much present on our App

30

Same here for all my apps.

31

Same here as well, none of our apps are working.

32

It looks like apps accessed through custom domains are still struggling. We’re continuing to do our best to mitigate the effects of the overwhelming traffic volume.

1 Like
33

In my case the issue doesn’t seem to be related to access through custom domains.

34

@kritzingerdewet Your app seems to use Basic Python 3, which is still experiencing some issues. Please try Full Python 3.
@stevenwu_n and @tomka I can’t see any errors in your apps - please can you post some details? At the very least, app ID and the error you’re seeing.

1 Like
35

Is it under control now?

36

:+1: DMed you the info.

37

@tomka Thank you for the info. The issue was a direct effect of the outage, not fully cleaned up yet. Your apps should be good to go again, and we’re now conducting a rolling restart of the rest of the infrastructure in order to catch any other components in an invalid state.

3 Likes
39

Hi all,

Just a quick update to say that we now believe everything should be back online. Needless to say, we’ll be investigating the attack and our response in some detail, but some of the things we’ll definitely be looking at will include:

  • How prepared we are for attacks like this on Anvil or apps built on Anvil
  • How we can respond effectively and rapidly when attacks do happen
  • How we can better isolate customers on Dedicated Hosting plans from attacks on our core infrastructure
  • How we can most effectively communicate status updates to our customers during incidents like these

Thank you all for your help, patience and feedback this morning. I’m going to close this thread now - please create new ones for any new issues you see.

13 Likes