What I’m trying to do:
Uploading files of more than 18 mb
What I’ve tried and what’s not working:
I have a professional plan and previously I was able to upload this files, now I am getting an error. I am building a startup and need to make a decision if I will scale up with anvil or get a new service. This is a new problem.
Thank you,
How are you uploading files? Are you using the client-writeable view workaround for file size limitations?
It would also help if you said what error you are getting, along with the details on how you are uploading the files.
The server is limiting the size of my files, I know this because the same method with a size < 17 mb runs perfectly, if I go above this it will give me this error:
ConnectionError: (‘Connection aborted.’, ConnectionResetError(10054, ‘An existing connection was forcibly closed by the remote host’, None, 10054, None))
when I do it in post man i get this: “request body 23364135; max request body 16777216”…
I still think we need answers to @jshaffstall 's question, are you using a file loader component? uplink? an anvil.server.call() object? Are you posting from the client to an http endpoint directly? (you mentioned postman, only reason why I’m asking)
Are you using media files? Are you trying to send a string? of text? of encoded bytes?
This is what I am doing
headers = {
“authorization”: f"Bearer {jwt_token}",
“Content-Type”: “application/octet-stream”,
“audio_duration_seconds”: str(audio_duration_seconds or “”),
}
with open(audio_path, 'rb') as audio_file:
audio_bytes = audio_file.read()
response = requests.post(LAUNCH_URL, headers=headers, data=audio_bytes)
Three forumers already tried to help, I’m the fourth.
They already asked for more information, I don’t understand why you are not providing it.
If you want help from volunteers, the minimum you can do is provide the information requested.
I don’t think you will get any help if you don’t put any effort in providing detailed information.
Thank you for your response. I appreciate your help. I understand that I may not have provided enough information initially, but I tried to provide all the relevant details, including my code, server responses, and the fact that this was working recently. I’m not sure what additional information you need. It seems that the server is rejecting my request, possibly due to changes in server logic to handle large files. If you’re unable to assist further, I understand, and I appreciate your time. Thank you!
Okay…
So we have a file, is it being sent from an anvil server module to somewhere else? Is it being sent by an uplink (aka anvil → another computer → python sends a request → other computer complains about file)?
a ConnectionResetError(10054... in python means that one or both machines you are connecting to/from are using windows, and they do not agree on either how long a communication should time out, or some sort of credential problem
(but that would happen on first connect, not on use of the connection as you have described)
Anvil is not running on a windows machine, so something somewhere is being limited to 16MB, if you google “16777216” you will find that this is a common misconfiguration for a webserver to receive files.
You’ve provided what looks like client code that sends data to an HTTP endpoint. Let’s see the code for the HTTP endpoint.
Ok so the code I provided, Is not from anvil this is the client code from my computer in python. In anvil I have the HTTP endpoint set up to receive audios like this…
#####################################################################################################################################################
#####################################################################################################################################################
@anvil.server.http_endpoint(‘/myendpoint’, methods=[‘POST’])
def proxy_audio_handler():
# Retrieve headers and audio data
auth_header = anvil.server.request.headers.get(‘authorization’)
audio_duration_seconds = anvil.server.request.headers.get('audio_duration_seconds')
audio_data = anvil.server.request.body.get_bytes()
# Check if audio data received
if not audio_data:
return {"status": "Error: No audio data received."}
# Create a tempfile to store incoming audio bytes
temp_file_name = os.path.join(tempfile.gettempdir(), "temp_audio.m4a")
with open(temp_file_name, 'wb') as temp_file:
temp_file.write(audio_data)
# Launch background task, passing collected data and tempfile path
task = anvil.server.launch_background_task('audio_handler', temp_file_name, auth_header, audio_duration_seconds)
return {"task_id": task.get_id()}
After this it goes into a background task that handles the audio. .
The error immediately if the file is more than 17 mb. if you split the file, everything works perfectly.
Anvil has a much easier and more secure way to pass data, objects, and files between a remote computer and an app.
You will connect via uplink, create a media object on your remote machine containing the audio file data, and simply pass it to a server callable function as an object, instead of creating an http endpoint.
and
…unless your goal is actually to create an endpoint that has absolutely no security and allows anyone to process audio… In that case, I would just break up your file into ordinally tagged 16MB chunks and reassemble them on the anvil server side using something like a data table and some sort of indicator to the endpoint that this is the last of the file being sent, to trigger reassembly.
Hi, the endpoint has a JWT token verification as you can see so will not be able to process audio. I understand the uplink but this approach was really cumbersome when you are processing audios from IOS, Android and web apps simultaneously.
Also the curious thing is that this is a new error. This code has at least 4 month old with no change and was working perfectly. Recently I cant process the file more than 17 mb anymore when we are getting ready to launch the start up app. I will like to see if you have any other solution or if some setting was changed from the ANVIL server side that is not allowing me to do this. Thanks again.
You can do this, sorting the order and storing the < 16MB file chunks in a data table, and having your audio_handler background task reassemble it and clean up the data table when it has completed successfully.
Got it. Does anybody happen to know what recent changes were made to the server that could have created this issue?. I recently upgraded to the Business account just to fix this to no avail.
Everyone you’re talking to in this thread are Anvil users. If anyone knew of a change we’d have said.
Rather than upgrading to a Business account you’d be better off paying for a month’s worth of a support contract and getting the technical folks at Anvil itself to help you solve the problem. They definitely know if anything has changed, and how it all works under the hood.
All we can do is suggest ways to work around the limit you’re clearly experiencing. Uplink is one way, chunking the data is another.
oh I thought the DEV also will help in these situations. Oh well, I guess Anvil is not the best for me with no support. Will migrate then. Thanks for all your help.
Anvil staff do respond on the forums, but since you are clearly in a time crunch that’s what the support plans are for, priority support.
I can wait for the answer. This is very important to decide what platform I will use for my backend. I already paid for the business plan (which will help me anyways because I want to see what happens with increased load of users), so I cant afford to pay more at the moment. Thanks for you help.
Hi all,
Thanks for jumping in to help here, and thanks @dr.peguero for letting us know that this limit is a problem. We’ve now increased the upload limit from 16 MB to 256 MB, and that change will go live in the next few days. We’ll post an update here to let you know when it is deployed.