Background Task for the Complete APP

Hi,

the timeout for a Server Function call is 30 seconds.
Performing longer-running tasks, the background-task is available and clear for me.

For some functions is fine but for a high number of functions and functions with a lot of arguments or a function calling other functions (with data transfer to tables and files) the background-task is not the nicest way to deal with it and you may have to split again to single server calls.

A) Is there a way to get for paid plans a bigger timeout for the complete app by sticking with the @anvil.server.callable

B) Is there a way to combine both calls (anvil.server.launch_background_task(‘nn’)/ @anvil.server.background_task) in one like @anvil.server.baground.callable

BR,
Gerhard

You have some particular patterns of use in mind, but didn’t show or describe any of them. So it’s really hard to tell how you’d want these to work. It’s hard to answer any of these questions, without seeing those details.

How about an example (of usage) of each one, along with a description of how you expect it to behave, in practice? Then we’ll have a clearer idea of what you’re after.

1 Like

Hi Phil,

thank you for the response.
Example: sure possible, I did not posted one because if answer A) can be solved there is no need for it.

Should I post it to Feature Requested:
@ Anvil Feature Requested: app timeout for server function call set by user (@anvil.server.callable)

Happy Holiday,
Gerhard

Hi @Gerhard,

As @p.colbert says, it would be great to have an idea of where Background Tasks aren’t suitable - our intention is that you can do (almost) anything you can do with server functions with Background Tasks, and if they are falling short in some way it would be really useful to know! Any use-cases you can share would be most appreciated.

You can run anything on an uplink for longer than 30 seconds, and a droplet from digital ocean is like 4$ USD a month. I can’t see a feature that you could pay for from anvil to eliminate the timeout costing less than $4…

In other words, they are not going to change how Anvil works because someone asks for it without even giving a solid use case.

Anvil has many limitations, and they are all there for a reason. Server functions are designed for quick tasks, not for slow ones. If they were allowed to run for a longer time without being killed when they are abused or when a bug keeps them running for too long, the Anvil servers would go down pretty quickly.

The 30 second limit is a feature, a good one, that allows Anvil to keep resources available and users to make sure their server functions are running smoothly.

Increasing the timeout limit for the whole app would be very dangerous. A bug could clog the server.

Increasing the timeout for a single call would make little sense, because there are background tasks for that. Launching a background task and immediately returning the control to the caller is much better than hanging there, executing a long running process without feedback.

An average app of mine has 50 server callables and 2 background tasks. It feels good knowing that I need to keep my eyes only on the 2 background tasks, because the 50 callables will never hang for longer than 30 seconds.

Hi Ian,

pls. find attachted the use case.
I took the background tasks out, so the case is more clear.
https://anvil.works/build#clone:TORUFBFX4REGIPMU=LPUUVWKLBQ5QFOVRRHSZUUPF

APP TASK USE CASE:

  • Set the Start ROW: eg. 0
  • Set the Stop ROW e.g. 10 …8760
  • Set the Location (Drop-Dow)
  • Press the Button ( if you go higher with the Stop Row up to 700 you will get the out of time error, no background task implemented)
  • Finally, a Plot and MAX, MIN., AVG Values of the Weather Data are passed to the client in the GUI

  • APP Final Goal: Dynamic Simulation from 0 to 8760 rows (get weather data from excel file, calculate thermodynnamic processes … row after row → all Data are taken or written to Tables e.g. Look Up / Parametric about 1000 columns). At the End of the Simulation about 30 Plots and an PDF of the Plots and 9 Forms e.g. GUI Windows must be generated).
  • Calculations should stay protected on the server side, plots are generated on the server side and passed to the client.

Would it make sense to call the plots with a timer >30s, in order to refresh/chache the plots as the rows are written?

MAX, MIN., AVG, SUM Values of the Data must be generated at the end of the row 8760 and written in the Table an then passed to the Client Side GUI (Is there an ANVIL DoLast Function).

Could you pls. give a look at the app and seeting the background tasks (not getting the curve in the plot by setting a background task on the plot call)?

Nice Holiday,
Gerhard

Ciao Stefano,

thank you for the detailed and clear answer as well sharing your expirience with Anvil.
If the 30 second limit can not be set higher, I have to find another solution.

Merry Xmas,
Gerhard

Hi Ian,

the Uplink is a solution I looked and took in conisderation for later on.
Thank you for the droplet idea.

On the longer run I do have to set up a fast server with more CPU’s.
May It make sence to do this right now.

Nice Holiday,
Gerhard

Hi Phil,

I droped an Use Case below.

Nice Holiday,
Gerhard

Hi, @Gerhard.

I don’t see a use-case in your forum message, nor any link to one. However, this link may be of use to you. Scroll down to Persistent Server Modules, and see if you can use them to do what you want.

I’ve found few programming tools that do exactly what I want right out of the box. But I can usually use them to build things that do. That’s likely to be the case here, for your uses.

Can you clarify in what way background tasks are not suitable for this use case? Where does using a background task fail for you?

Hi Jay,

appreciate it, pls. find attached the version with background task for the Server Plot Call and the AVG, MAX, MIN … Server Call

https://anvil.works/build#clone:DDJMHZ6L4677IK7Y=VTUQZFVVYC6BTBDZRSNOBPJA

  • DBT Curve is not shown in the Plot, however the Figure is loaded on the Client.
  • Out of time is appearing as well
  • May you have to take the anvil.server.launch_background_task(‘llocation’) out .
    @anvil.server.callable
    def location():
    task = anvil.server.launch_background_task(‘llocation’)
    return task

Thank you,
Gerhard

Hi Phil,

I answerd to Ian and Jay, with an APP Case, you should see them in the chat?

Best,
Gerhard

Just looking at the background task, you have odd looking code in there. I’ll comment on the key things I see, others may have more sophisticated comments.

  1. You do multiple searches without specifying an order by clause, yet you then take a slice from those results. Unless you specify the order, you’re relying on the database to give you back results in the order you want, but without specifying the order you can’t guarantee it’ll do that.

  2. You create a generator comprehension and then take a slice from it. There’s no need for the generator comprehension. For example:

([column['DBT'] for column in app_tables.data.search()][startrow:stoprow])

is designed to give you a list of a specific column between startrow and stoprow. This will do the same (the Anvil search results already functions like a generator):

[column['DBT'] for column in app_tables.data.search()[startrow:stoprow]]

I suspect the Anvil search results will do the slicing more intelligently than the generator, which will have to process all the items up to the ones you want.

  1. You do the exact same data tables search multiple times. Do it once and reuse the results, e.g.:
    results = app_tables.data.search()
    [FILE] = ([column['FILE'] for column in results][0:1])
    [STATION] = ([column['STATION'] for column in results][0:1])

That should speed things up some, since you don’t need to perform the same query multiple times.

The majority of your time is no doubt taken up by your DBT_AVG calculation, since it processes over 8,000 rows. I’m assuming there’s some magic reason that you know exactly how many rows will be in the table?

In any case, rather than calculate that average as needed, since you’re importing the contents of that data from an Excel file and are adding each row individually, you should be able to calculate the average as you’re already processing each row in the Excel file. That would prevent the need to calculate it later (unless the contents of the data table can change after the import).

  1. Your plot isn’t showing because you have code in the client like this:

self.plot_1.figure = anvil.server.call('get_plot')

But what the server function returns is not the plot, but the background task. You need to have some mechanism on the client that waits until the background task is complete before setting the figure. It’s pretty typical to use a timer on the form to check to see if the background task is complete.

When you make the server call that returns the task, you remember the task and start the timer, e.g.:

self.task = anvil.server.call('get_plot')
self.timer_1.interval = 1

Then in the timer function you can check to see if the task is complete:

if self.task.get_return_value():
  self.plot_1.figure = self.task.get_return_value()
  self.timer_1.interval = 0

That can get way more sophisticated by using the task state to return multiple items, but that gives you the basic idea.

3 Likes

Hi Jay,

thank you for looking over the codes, appreciate your feedback and solutions.
You are right I did not specify an order by clause in the use case.
I made a version were I took buttons for the order for testing not implementing a timer.
Proceeding manualy.
BTN 1: Excel to Table
BTN 2: Calculate AVG, MAX, MIN
BTN 3: Creating Plot

So, the timeout error was primary caused by the not exisitng order, this makes sense for me.
Since working for 500 rows I was assuming that the order of codes would be enough.
The cause a combination of order + calls to optimize and not primary background task related.

  • Combinig the Data Table search in once is a much more better way.
  • Timer, yes I will implement an order
  • Yes, I know the number of rows
  • Processing each row in the Excel file before writing to the table: good solution for some data I can do so, for others I have to adjust the data with a DELTA Value so the processing has to go on the tables.

Ok, I will implement the optimzed teable search an order + timer and may a background task for the plots. I will let you know the improvments.

Merry Christmas and Thank You!
Gerhard

What do you mean by DELTA Value?

I don’t see any reason why you should write to a table something you already have in memory.

Hi Stefano,

in this file are stored historical weather data.
I some cases (energy calulations) it need to adjust e.g. the temperature fron the file by adding a DELTA_DBT because you would like to perform the calulation with other parameters.
As well I need one table file with all the calculated values

Ciao
Gerhard

If you need to add a value, you can do it on memory, before writing to database.

If you need to do a complex query that includes both new and old data, then it may make sense to first write to database.