Is it safe to use asyncio on the server?

Anvil Q&A
1

What I’m trying to do:
The code you see below basically makes about 10 network requests each taking about 1 second.
Currently this is done synchronous and thus takes about 10 seconds. By using a promise this would be reduced to 1-2 seconds. In JS I’d use Promise.all() for that.

I tried to use import concurrent.futures but got weird errors and I read a few forum posts that you should not be threading server side.

If I use standard asyncio I get a red syntax error which makes think that anvil does not want me to do that.

Anyway my question is, how do you properly and safely to asynchronous programming in Anvil (Server Side)?

(Also please do not suggest Background tasks for that - trust me I have tried that route. Its slower.)

image
image1230×654 163 KB

Thanks,

Mark

1 Like
2

My only input is that others years ago found the same thing, but also that the code still compiled an ran, despite the syntax errors in the code editor.
The same questions you have were raised, but I don’t remember them ever getting an answer, and I assume they either used it without problems or abandoned it and ended up working without it directly on the sever module (moved it to uplink, or whatever else they did) I do not know, only speculating.

Sorry that isn’t much help, just some context from the past.

3

I got the same impression from the existing forum posts on the topic - thanks for your input!

And yes I can confirm that the code is executed despite the syntax erros - but in order to use it in a production environment I will need something more official on the topic.

Maybe someone from anvil-central can bring some clarity on topic @daviesian ?
We are on quite some pressure to increase speed on this part of the code. And rebuilding it to use concurrency would be quite the effort - so I would like to prevent any gotchas at the end of the road.

thanks!

4

That’s an unusual thing to want to do…

If you’re making requests, what’s the reason for doing that server side? Why not do it client side where you have all that kind of functionality available in several different flavours?

5

I would use parallel threads.

There are two problems with creating new threads:

  • Your threads can’t use any Anvil globals, because you don’t know if they are thread safe
  • Your threads can be killed as soon as the request ends

But in this case you are waiting for all your threads to finish their job, and you can make your own returned data thread safe yourself.

EDIT
… or background tasks, which is the Anvil way to run parallel processes.
I’m guessing background tasks add some overhead, but I don’t know how much would that hurt.

Hmmm… no, I would still try parallel threads.

6

building on stefano, I successfully used threads to accomplish something similar. I was ignorant to its pitfalls, so maybe I have an issue I am an unware of but here is the general structure:


import threading
from queue import Queue
import get_data_function

def get_commisions():
  data_list = get_data_function()
  results_queue = Queue()
  def process_thread():
    while True:
      try:
        data = data_list .pop(0)
      except IndexError:
        break
      else:
        commissions= get_commission_data(data)
        for commission in commissions:
          results_queue.put(commission)

  num_threads = len(data_list)
  threads = []
  for _ in range(num_threads):
    thread = threading.Thread(target=process_thread)
    thread.start()
    threads.append(thread)

  for thread in threads:
    thread.join()

  processed_commissions = []
  while not results_queue.empty():
    result = results_queue.get()
    processed_commissions.append(result)

  return processed_commissions

EDIT

This is in my production and has had no issue. I also would like to get anvils input.

7

Thanks for your answers!

@owen.campbell I wouldn’t consider asynchronous programming on a python server particularly uncommon, especially since asyncio has been in the python core for many years.

But of course its a valid point to ask why this cannot be done client side. The short story is that sfpt servers, endpoints with private keys and computationally intesive tasks are involved.

@stefano.menci @anthonys
We tried to do this with individual background tasks but it was slower in every case and would result in tens of thousands of background tasks beeing lauched every day. Which even our dedicated server can’t handle.

I lean towards threads as well and we already have an implementation similar to @anthonys ready.
But I will not ship this to production unless a lot of people have tried this in a loaded production environment or I get some sort of thumbs up from anvil itself.

Thanks for your Help!

Mark

1 Like
8

Hi @mark.breuss,

The official answer here is yes, it is safe and supported to use asyncio in server modules. Multithreading isn’t safe, but async runs concurrent tasks that spend a long time waiting in the same thread – and if it’s all in the same thread, Anvil’s thread-local machinery doesn’t mind at all!

You can therefore launch async functions in both Server Functions and Background Tasks, with asyncio.run.

For example, the following code works as expected:

import asyncio

async def hello():
    print('Hello ...')
    await asyncio.sleep(1)
    print('... World!')
    return 42

@anvil.server.callable
def test_run():
    return asyncio.run(hello())

This will have the desired outcome: When you call this, the console will say Hello..., then wait a second, then ...World, and return 42.

So here, the @anvil.server.callable is an ordinary function that runs in a single thread, launching a self-contained single-threaded async runner with an async function in it, and returning its result.

This enables you to do patterns like this, if you have many tasks that wait for (eg) IO in an async-y way:

async def multi_hello():
    calls = [hello() for i in range(10)]
    return await asyncio.gather(*calls)

@anvil.server.callable
def test_run_multi():
    return asyncio.run(multi_hello())

This function will also work as expected - it will print ten Hello...s, and a second later ten ...Worlds, then return [42, 42, 42, 42, ...].

As for autocomplete, this is likely(?) a bug/oversight in our parsing and completion. The Python 3.7 grammar we currently use can deal with async just fine, so it should just be a bugfix, but I haven’t personally touched that code in a couple of years so can’t speak authoritatively.

For now, however, using async will break autocompletion in the module where it’s used, which I’m aware is frustrating. I’d suggest keeping your usage of async “small”, and confined to certain modules that need it, so autocomplete remains working elsewhere.

8 Likes