🖋 Upload files directly to S3 from an Anvil app

Hi everyone!

If you want to upload very large files into an Anvil app, you might experience timeouts or exceed your Data Tables storage. Instead, it makes sense to store large files in a cloud storage service, such as Amazon S3.

Meredydd and I have written a guide on how to upload large files directly to S3 from an Anvil app! It uses Amazon’s boto3 library and the uppy.io JS widget (using Python).

Check it out:

7 Likes

That is great!

Question from a S3 noob, Is is just as easy to pull that file back from the S3 bucket?

Having never used boto3, I found this is the docs, so it looks like they make it pretty simple:

https://boto3.amazonaws.com/v1/documentation/api/latest/guide/s3-example-download-file.html

Since it accepts a file handler, I don’t see why you couldn’t shove it into a file stream and operate it in memory without even writing it to disk.
(Lets say you are going to immediately load it into a pandas dataframe or something, why write to disk)

1 Like

Thank you! I should’ve gone to the docs or chat gtp for this question :sweat_smile:

This is such a game changer!