Converting a media file to table data

Anvil Q&A
#1

What I’m trying to do:
I’m having a difficult time uploading a larger file (+500kb), due to the 30 second server timeout issue.

What I’ve tried and what’s not working:
I’ve read, and used the code in this thread (Server timeout when uploading file - #4 by owen.campbell), and this thread (Timeout on large file upload - #8 by owen.campbell).

This leaves me with a file.xlsx media object in my data table. I’m getting around the upload timeout, so mission partially accomplished! But how would I interact the media object to “unpack” it and fill in my data table?

I thought that’s what the “_write_upload_row” function was supposed to do, but apparently I’m mistaken.

Code Sample:


@anvil.server.callable("_get_upload_row")
def _get_upload_row(custom_name):
    media_upload = app_tables.uploads.add_row(name=custom_name)
    return media_upload, media_upload.get_id()
  
@anvil.server.callable("_write_upload_row")
def _write_upload_row(upload_row, row_id):
    row = app_tables.uploads.get_by_id(row_id)
    row.update(media = upload_row['media'])

Clone link:
https://anvil.works/build#clone:47HEWUPPDYDPOUY7=GBIDO5VBC4COLD2VN4LH2J7Q

#2

I have not done this myself before, so follow this advice with caution. But it’s something I’d like to know how to do, so I looked at the docs and this is my understanding:

If you’re on a free plan, you might need to pivot a bit and follow the directions for doing this via Uplink here: Anvil Docs | CSV and Excel import

If you’re on a paid plan, you can get the media object from your uploads table and pass it to a store_data function like the one here: https://anvil.works/docs/data-tables/csv-and-excel#file-upload-in-a-running-app

#3

Hi @hugetim, I really appreciate the response. That’s where I started, and I was getting a server timeout at 30 seconds. Now I’m around the server timeout issue and wondering how to go about translating the media object into a table.

Does that make sense?

#4

Your code was simply complicating a pretty simple task. That _write_upload function was totally useless. When you set the upload_row['media] = file

The task was already done.

https://anvil.works/build#clone:TQL4SVPTTYDDLEXD=VSEGXLWMWJENZ2ZRMMH3ELQV

Here’s the corrected version.

I had to remove Pandas because I don’t have paid plan. You can add it back.

#5

Right, I saw that you started there. You now need to combine that with what you learned from Owen’s responses. I think that is the big idea you are not getting. But a few coding mistakes were getting in the way, also.

When I tried to run your code, the immediate error was caused by the way you were returning (only) a row object from “_get_upload_row” but trying to unpack it into both a row and a row_id.

Once that was fixed, I replaced your _write_upload_row code with the store_data(file) I was prescribing above. Then I saw you hadn’t checked the “Auto-create missing columns when adding rows” box like the Docs I linked to instruct you to. Then I saw that you hadn’t created a data table separate from the “uploads” table to store the contents of the Excel file.

Anyway, see if this works for you: (I tried to put ### at the end of lines I edited or added.)

https://anvil.works/build#clone:RK4S7TBLST3EXUHP=O56KBX7N7CWROQ7NB62JF7RQ

Unsolicited advice warning. Only click the arrow to expand if you're looking for advice.

My broader advice is: if you are having trouble with a big issue (like you don’t know how to load a large excel file into a Data Table), try to see if you can break it up into smaller pieces that you can figure out. In this case, you say you succeeded in getting the big file stored as a media object in a table. But it doesn’t seem like you ever tried seeing whether you could load a small excel file into a data table. Like, what I did was just try to load the sample excel file colours.xlsx from the docs. But maybe I’m mistaken to think that you hadn’t first tried that simpler task?

It seems like maybe you’re copying and pasting Python code without understanding what it means or does. Anvil is not really intended to be used that way. You can get away with it for certain things, but it’s going to be limiting. It takes time to understand what the code is doing, but it’s worth it.