Accessing global variable from scheduled background task

Hi all,

I would like to run a scheduled task every morning before my users log in to the app. I’m on the business plan, so I make use of persistent servers. I have a global variable called globals_dict which contains a couple of large dictionaries and lists stored in memory.

Currently I log into the app every morning and trigger a server-side call which access the global dict, clears it and populates it with the latest information. I don’t want to keep logging in early every morning to manually trigger the recalculation, which is why I’d lke to do it via a scheduled task.

However, it seems that scheduled tasks can only trigger background functions, but background functions don’t have access to global variables stored in memory since each background task is launched in a separate python process according to this chat: It seems background tasks can’t see globals? - Anvil Q&A - Anvil Community Forum

Any suggestions on how I can access a global variable from a scheduled background task?


Managed to solve my own question, there might be better ways to do it, but this is what worked for me.

In the scheduled background task, you need to call a registered function on your server which accesses the global variable.

def get_global_variable():
   """This function is called from the background task to get the latest state of the global variable"""
   return some_global_variable

Then within your scheduled task you can do all the calcs and changes you need and then return it again to another server function which updates the global variable.

def update_global_variable(updated_global_variable):
   """This function is called from the background task to update the global variable"""
   some_global_variable = updated_global_variable

If anyone has another way of doing it, please let me know.

There are two ways I have done this which work nicely.

  1. Store the dicts in a data table. This would be my preference.
    This is good because you can always access it and it doesn’t matter where it is called from.
    You can have a row per customer if you like and update the data selectively. For large dicts the simple object would be the way to go.
  2. The second is to write it to a file and save that either to a data table or app file or asset file. Again data table would be the easiest. Seems like more work than necessary.

Is there a reason you haven’t used the data table service?

Thanks @rickhurlbatt

No specific reason for not using the datatables to do this. I’m using the datatables for many of my other function calls, so it’s definitely also an option (maybe even better that what I’m currently doing). I’m experimenting with all the different options for my own learning.

From a speed/security/best-practice view-point, would you say there is any problem in doing it the way I mentioned in my previous post?

I think there are users out there more qualified to speak about best practice than what I am but from my own experience data tables would be the right solution.

The biggest reason I would avoid using a variable in memory is that persistent servers aren’t guaranteed to, well, persist. What happens if the server drops this from memory? Then it has to be recreated. Presumably this is a slow process or you would be doing it on request. In which case now your users have to wait for the data to be reconstructed.

If this is data that your users rely on and there is a possibility that multiple users would want to access or edit this data at the same time and it is variable data, I would think that data tables would be a more robust solution. Robust is more often best.

From a speed perspective, using a background task to update the data and then store it in a data table would be fast. Accessing tables is very fast if there is not a length search. Just test out the speed of app_tables.table.get_by_id(). Its very fast.

1 Like

I agree with @rickhurlbatt.

You can’t rely 100% on a global dict, because once in a while a new server instance will be used. So you will need a fallback mechanism that creates the global dict when it’s not there. Then you will need to manage two concurrent accesses when there is not global dict and prevent them from creating the same global at the same time. Then…

Of course if your global dict takes a few minutes to generate and it contains objects that can’t be stored on a datatable, then that’s a valid option, and you will need to manage the cases described above.

But if your dict is json friendly and is smaller than a few megabytes, I would definitely store it in one simple object column.