Anvil Background Tasks Timing Out

Anvil Q&A
1

Hi Everyone,

What I’m trying to do:

I have a web application that takes in a couple pieces of information from the client. The main piece of information being an Excel workbook along with some details. The web app then makes use of a callable function in the server code side that launches a background task that makes a couple request calls to a server that uploads the excel file and data.

What I’ve tried and what’s not working:

The issue that I am facing is that the web app times out when I make the call to the server code for the background task and doesn’t complete all the functions after the call to the server code. The client gets an error on screen and the logs show the same error: anvil.server.TimeoutError: Server code took too long. The line that it is referirng to is the line that launches the background task in the server code.

The flow is as follows:

  • Client adds details to Form and clicks submit.
  • Form calls an anvil.server.callable function in serverModule.
  • serverModule launches an anvil.server.background_task using anvil.server.launch_background_task

Code Sample:

# this is a formatted code snippet.

## Form Snippet
anvil.server.call("createWorkItem", self.file, self.sheet, self.fileName, self.fileSize, self.clientEmail, self.vendor)

## serverModule Snippet of callable function
@anvil.server.callable
def createWorkItem(file, sheet, fileName, fileSize, clientEmail, vendorName):
    anvil.server.launch_background_task('createWorkItems', file, sheet, fileName, fileSize, clientEmail, vendorName)

## serverModule Snippet of background task
@anvil.server.background_task
def createWorkItems(file, sheet, fileName, fileSize, clientEmail, vendorName):
    try:
        doStuff()
    except:
        doOtherStuff()
# paste your code between ```
2

Welcome to the Forum!

This error message has a rich history of discussion here.

Without further detail, my best guess is that your server-side code has a large startup delay, due to big imports. launch_background_task will start up a second instance of that code, so it will have its own (additional) startup delay. If the total delay exceeds 30 seconds, createWorkItem will time out.

It would be a waste of time and effort for anyone to repeat the many existing and fruitful discussions on this topic. Instead, I suggest that you use the :mag: icon (at the top of this web page) to find them. You can match the discussed situations with your own, and find relevant strategies and tactics you can use.

For example, if imports really are the problem, you’ll find ways to defer the worst-offending imports until they are actually needed.

With Search, you don’t have to wait for anyone.

3

Thanks, @p.colbert for the response!

I am using the standard, pre-given imports that come with the server code module:

import anvil.secrets
import anvil.email
import anvil.google.auth, anvil.google.drive, anvil.google.mail
from anvil.google.drive import app_files
import anvil.users
import anvil.tables as tables
import anvil.tables.query as q
from anvil.tables import app_tables
import anvil.server

And the only extra imports I have are requests and json (which both do not take long at all to import).

Do the standard anvil imports take longer than 30s to import if they were imported twice?

4

How many database tables do you have in your app? When you import app_tables all those get imported. If you only have a few then I would think it rather unlikely that its your imports.

5

Hi @duncan_richards12,

I do not have any. I have removed all the unused imports and I am left with the following:

import anvil.secrets
import anvil.email
import anvil.server
import requests
import json

Even after getting rid of the unused imports the server code still times out

6

Do you have many server modules?
All the modules are loaded, not only the one containing the function you are calling.

Are you able to provide a clone link?

7

Hi @stefano.menci,

There is only one server module and one form:
image

I can’t provide a clone link as there is some sensitive data in a users data table (used for authentication and logging in to the web app).

But I have included the code in the server module below:

import anvil.secrets
import anvil.email
import anvil.server
import requests
import json

# This is a server module. It runs on the Anvil server,
# rather than in the user's browser.
#
# To allow anvil.server.call() to call functions here, we mark
# them with @anvil.server.callable.

@anvil.server.callable
def createWorkItem(file, sheet, fileName, fileSize, clientEmail, vendorName):
    anvil.server.launch_background_task('createWorkItems', file, sheet, fileName, fileSize, clientEmail, vendorName)

@anvil.server.background_task
def createWorkItems(file, sheet, fileName, fileSize, clientEmail, vendorName):
    try:
      header = {"Authorization": f"RC-WSKEY {anvil.secrets.get_secret('authToken')}"}
      workspaceId = f"{anvil.secrets.get_secret('workspaceId')}"
      if vendorName == "Cotton On":
        processId = f"{anvil.secrets.get_secret('processId')}"
      elif vendorName == "H&M":
        processId = f"{anvil.secrets.get_secret('processId2')}"
      else:
        return "Invalid Vendor"
      url = f"https://api.eu1.robocorp.com/process-v1/workspaces/{workspaceId}/processes/{processId}/work-items"
      # Create New Work Item
      response = requests.post(url=url, headers=header, json={"payload": {"sheet": sheet, "email": clientEmail, "vendor": vendorName}})
      workItemData = response.json()
      workItemId = workItemData["id"]
  
      # Upload File
      fileData = {"fileName": fileName, "fileSize": fileSize}
      url = f"https://api.eu1.robocorp.com/process-v1/workspaces/{workspaceId}/processes/{processId}/work-items/{workItemId}/files/upload"
      response = requests.post(url=url, headers=header, data=json.dumps(fileData))
      uploadData = response.json()
      uploadURL = uploadData["url"]
      uploadFields = uploadData["fields"]
  
      files = {"file": (fileName, file.get_bytes())}
      response = requests.post(url=uploadURL, data=uploadFields, files=files)
      # Start Process
      url = f"https://api.eu1.robocorp.com/process-v1/workspaces/{workspaceId}/processes/{processId}/run-request"
      requestData = {"type": "workItemIds"}
      requestData["workItemIds"] = [workItemId]
      response = requests.post(url=url, headers=header, data=json.dumps(requestData))
      return response.status_code
    except Exception:
      anvil.email.send(from_name="SPB Web App", to="ops@otomatika.com", subject="App Failure", text="There was an issue with the bot.")

Hopefully this helps? The form calls the createWorkItem function which should start the createWorkItems background task. The error log mentions that the line that fails is the line that launches the background task.

8

It seems you are doing several api calls whose timing is going to be way more variable than any other part of the call and generally slower than just regular code. I suspect one or all of those are the bottleneck. To verify you can add print statements to see how far along the function gets before timing out. It would probably be better to split each api call into its own background task, maybe that call into each other, so that you don’t sink all the background task time into one timeout period

9

Hi @duncan_richards12,

The background tasks aren’t timing out, the error log states that the anvil.server.launch_background_task is timing out which is what launches the background task. Does this function wait for the background task to complete?

There have been occasions where the web app runs successfully (although it ran for close to 30s) where it successfully created the background task and completed the rest of the form code.

Here is an image of the App Sessions that failed due to it timing out (Note; I have cleaned up the code in the snippet above so the lines do not align):

image
image2724×1246 265 KB

You can see that when it does create the background task it completes the run but when it times out it doesn’t create the background task.

So the App session fails to even start the background task sometimes and that is where I am struggling to understand why.

11

Ahh I see. How big are the files you are sending to the server function? It could be that the timeout starts when the call is initiated on the client and includes sending the file to the server. To test if that’s the case, add a print statements before the background task launching to see if you are even getting into the body at all.

It also could be the file size timing out starting the background task as well by taking to long to send the file there

12

@duncan_richards12, that makes a lot of sense. The code sends the files (which tend to be fairly big 30mb+) between the form and the server function. Would it be wise to store the files somewhere from the form and then access it later in the background task?

13

It would certainly be worth a try to put it in a db table in a media column and have either the server function or the background task access the file that way (whichever works best to prevent the highest number of timeouts). I’m not 100% this will definitely prevent all the timeouts with files that big but trying it can’t hurt and the server function and background task are both a lot closer to the db than the client is to the server.

1 Like
14

I will definitely give this a try, thank you so much for all your assistance, @duncan_richards12! I really do appreciate it!

1 Like
15

@duncan_richards12, that did the trick! Got it working with no timeout issues at all. Thank you!

1 Like
16

Perfect, glad it worked out!

1 Like