Loading a model alongside the API endpoint server

Anvil Q&A
1

I think I have already heard this was possible with Anvil, and it’s possible it might have been a request from others in this forum. Please point me to another answer if this has already been asked before (I searched for a few keywords but got lots of responses and some didn’t look like were relevant to my query).

Really sorry if it sounds like a repeat question. Basically I have access to various hubs like TF or Pytorch, or some model zoo. And I’d like to put frontend before one of these models and Anvil is perfect for it.

I’m more curious about the disk space allowed per project - My models are in the areas of 500MB and higher all the way to 1 to 2GB. Also, would you cache such an asset, so that we don’t download it each time we try to access it.

Thanks.

2

My other questions on the back of the above one would be:

  • is it desirable to use Anvil to serve a model that you are also incrementally training (might add to latency), sort of like active-learning?
  • and are there any examples or has anyone used anvil to trigger model training and/or analysis

These above would bring the question about storage that you can read from directly (live) as opposed to storage that is just to save large data/files.

3

In general you may want to consider the details in this post which describe the DataTable storage limits on various plans. If that will not work for you, you could consider GoogleDrive integration. Also the Jupyter demo might be helpful as I believe the model is computed locally and then Anvil provides the front end. The app connects to the local computer via Anvil uplink. I realize you may want a different set up, but maybe that will give you some ideas in terms of how to proceed.

4

Well, I’m thinking of downloading (one-off) pre-trained models which can be pretty large at times and give the model an Anvil frontend. Not sure if the above you say covers it but Ill take a look anyways.

5

Great! Feel free to show what you’ve tried or clarify further and we’ll try to point you in the right direction.

6

Anvil staff can clarify whether space or computation will be an issue. But…

My two cents are that you should separate the front end (UI with code logic) from the back end (model inference) for this type of task and connect them with an API. If you will have more than a few users computation will be an issue I believe. Inference time for large models on CPU’s is something like 5-10x compared to a GPU. If this is really meant as more of a demo with only a few users at a time it may be ok. If you are intending to use this to provide a GUI self-serve interface to clients I would not recommend hosting the models on Anvil.

If you give a bit of context on what you are intending I may be able to give some more advice.

1 Like