[RESOLVED] Anvil server issues?

Hi all,

Is anyone else experiencing server issues? I’m getting heaps of 409 errors while using the editor, the save operation isn’t completing. I’ve recreated the same form three times, since changes got lost each time

I’ve not had any issues.

Thanks for the reply David! I was hoping it was only me.

I just wish I wasn’t losing work so frequently :frowning:

To paint a picture, in the last 24 hours, the time to go from clicking the Run to having the Saving,… spinner disappear feels about 20 seconds, and to go from there to the app running another 20 seconds. The worst handful of times was when it just wouldn’t stop saving.

Sometimes closing the browser tab and creating a new one helps.
Not closing the browser and reopening it, because it could remember the tabs and whatever they had slowing them down.

Eek! Yeah, I can confidently say this shouldn’t be happening. This is our detection system for concurrent conflicting edits (in git terminology, detecting a “force push”), which should never fire if there’s only one person using the editor. Some quick diagnostic questions:

  • Was anyone else editing the same app at the same time as you? (This is the “correct” reason for conflict errors.)

  • Can you confirm that you saw the big “You have edited this app in another session” dialog? (Or are the 409s just on the console, and the only symptoms you’re observing are slow saves and lost work?)

  • Do these conflict errors always occur at the same time as slow saves?

  • Are your apps particularly large? (Eg do they contain large assets?)

  • Do the slowdowns/conflicts occur at a particular time of day? (Perhaps we can correlate this with logs/metrics from our servers.)

In the meantime, I’m going to put some extra tracing in place, to try to clarify this further…

You are describing what is happening to me right now.

Today I am working from home and I have the exact same problem. In my case it is caused by the VPN. I turn it off and everything is as fast as if I were in the office. Our IT is investigating, but I’m sure the problem is on our side.

@stefano.menci - Aha! Could you try opening the Network tab in the Chrome developer tools (or the Firefox equivalent) and seeing what happens there? Eg how long do the save operations take?

My current hypothesis is that if the upstream network is slow enough, all the asynchronous “save” operations will all pile up on top of each other (which explains the 20s saving delay) and eventually overwhelm our deconfliction system (which explains the conflict errors).

My guess is that the VPN is slowing down connections once in a while, randomly. I have the same problem in any browser tab, Anvil, Google or any other domain.

This is what happens when I start an app and it waits 10-20 seconds:

This is what happens when I press Ctrl+F5:

Thank you for your help! Yes, a VPN randomly delaying requests by 10 seconds would cause a pile-up of requests behind it!

I think I have a fix. (Specifically, it detects and avoids “pile-ups” on a slow connection, as well as recovering from them without giving you a conflict error. Also, you now get an explicit “Saving…” spinner when you press Ctrl+S, so you can always “flush the pipe” and know for sure when your work has been saved.)

As this is something that’s getting in the way of your work, I’ve performed an emergency release. You’ll probably need to refresh your editors. Can you tell me whether this has improved things?

I will leave the answer to your last question to @phil because I did not notice problems with conflicts (unless I had two tabs open on the same app).

Just annoying delays that can be avoided by turning off the VPN.

The “pile-up” fix should still help you. Could you switch on your VPN for a couple of minutes and tell us how it performs?

(But yes, I’m looking forward to hearing from @phil - probably when Australia wakes up :slight_smile: )

Thanks for the thorough reply Meredydd.

  • noone else was editing, but I MIGHT have left a session open on my other computer
  • I saw the “you have edited this app in another sesion” dialog once, and then i closed the browser and started again, and didn’t see it after that. The 409’s persisted however, and the slow saves continued and some never finishing
  • Apps are very small
  • I haven’t noticed a pattern to the conflicts or saving issues yet. I’ll keep an eye out for it and get as much information to you as possible.

Is there perhaps some way I could tell Anvil “I’m really super duper sure the version i’m saving is the one I want, ignore all conflicts and let me keep working please” ?

.

Hi Phil,

Are you seeing this issues at all now? The “409 snowball” should now be nipped in the bud, and consequently saves should have speeded up.

If you want to save your current version, and be confirmed that this version has in fact saved, press Ctrl+S. When that spinner disappears, you know the save has completed, and all your edits to that point have been saved.

Thanks again Meredydd, I’ll keep trying it out. I haven’t noticed any issues in the last couple of days, but I will keep a note of saving behaviour over the next week or two.

@meredydd I did not reply with news about the VPN performance because after your post the VPN started working without delays (fortunately for me and unfortunately for your feedback). I left it on during the whole weekend and I had no problems with anvil, gmail, stackoverflow, etc.
I will let you know if I see any performance/reliability problem in the future.