Server timeout when uploading file

Anvil Q&A
#1

Once again I’m coming late to the party, but glad of the Forums here for capturing these questions for new generations of Anvil users (=me!).

So I’ve followed these docs on Media and Files, and the link to the Uploader component…

My initial code worked fine for a 20MB data file:

Client

def upload_address_lines(self, file, **event_args):
        """This method is called when a new file is loaded into this FileLoader"""
        file = self.file_loader_1.file
        file = anvil.server.call('_store_uploaded_media', file, "Address_Data_UK")
        self.file_loader_1.clear()

Server

@anvil.server.callable("_store_uploaded_media")
def _store_uploaded_media(media, custom_name):
    media_upload = app_tables.uploads.add_row(name=custom_name, media = media, datetime = datetime.datetime.now())  #  <<< This is "Line 38"; see error message below
    print(f"{media.name} saved to 'uploads' database as {custom_name}.")
    return media_upload

But presumably because of some timeout designed for reglar code execution, I get the following error consistently with a larger (40MB) data file even with my fast broadband connection which I use daily for video editing and transfers of several GB at a time:

anvil.server.TimeoutError: Server code took too long at [HomePage._DeveloperTools, line 38](javascript:void(0))

Is there a way of deactivating or lengthening the timeout for such an upload please? The only workaround I’ve found so far isn’t very satisfactory because I have to manually open up the Datatable for client writes and this obviously exposes the Uploader to potential abuse, even if temporarily:

    def upload_address_lines_client_only(self, file, **event_args):
        """This method is called when a new file is loaded into this FileLoader"""
        file = self.file_loader_1.file
        media_upload = app_tables.uploads.add_row(name=file.name, media = file, datetime = datetime.datetime.now())
        print(f"{file.name} saved to 'uploads' database.")
        self.file_loader_1.clear()

Alternatively, is there any way I can programmatically permit, then immediately restrict, Datatable writes from the client e.g. based on user rights I specify?

All the best,
Peter

Advice for securely uploading a large dataset
#2

In case it helps, please see this post.

2 Likes
#3

Aha! Thanks - Background tasks was next on my list to have a look at, so hopefully that’ll provide a solution.

All the best,
Peter

#4

You might also want to look at

Converting a media file to table data
#5

It is not the upload that times out, it’s the call to a server function that takes longer than 30 seconds: anvil.server.call('_store_uploaded_media.

You can upload large files by first calling a server function that returns a row that is writable to the client, then assigning media to its Media Object column, then calling another server function passing the row object.

#6

Thanks for the suggestion @stefano.menci. I tried this but unless I’m doing something wrong, it looks like your suggestion still requires me to give Client write access to the Data table I’m afraid:

anvil.server.InternalError: Permission denied: Cannot write in table 'uploads' from client code.

Client

    def upload_via_returned_row(self, file, **event_args):
        upload_row, row_id = anvil.server.call("_get_upload_row", "Address_Data_UK")
        file = self.file_loader_1.file
        upload_row['media'] = file  #  <<<< Error Row
        anvil.server.call("_write_upload_row", upload_row, row_id)
        self.file_loader_1.clear()

Server

@anvil.server.callable("_get_upload_row")
def _get_upload_row(custom_name):
    media_upload = app_tables.uploads.add_row(name=custom_name)
    return media_upload, media_upload.get_id()
  
@anvil.server.callable("_write_upload_row")
def _write_upload_row(upload_row, row_id):
    row = app_tables.uploads.get_by_id(row_id)
    row.update(media = upload_row['media'])
#7

That’s correct.
You either create a table that is writable from the client side or (better) you return a client_writable view containing the row you want the client to modify.

#8

Thanks for all the suggestions but at the risk of appearing conceited I’m going to record my own simple solution to this problem which uses the built-in Anvil Uplink feature and therefore only requires code in one place (my local machine) and doesn’t require the Data table to be opened up to Client code:

import anvil.server
from anvil.tables import app_tables
from pathlib import Path

anvil.server.connect("xxxxxxxxxxxxxx")

def upload_txt(file_path):
    """ Send a file directly to Anvil Data Table as media for onward emailing"""
    file_path = Path(file_path)
    with open(file_path,"r", encoding='utf-8') as file:
        blob = anvil.BlobMedia("text/plain", file.read(), file_path.name)
        row = app_tables.uploads.add_row(name = "Address_Data_UK", media = blob, datetime = datetime.datetime.now())

Simples! Thank you Anvil…

2 Likes
#9

If the upload starts always from the same computer and you have control over it, then the uplink is definitely the way to go.

1 Like