Hi, if I want to host Ai model like hugging face model. (yanolja/EEVE-Korean-Instruct-10.8B-v1.0 · Hugging Face)
I am going to host it. what service should I use?
thank you!
I’m not sure Anvil is the right tool to host an AI model. This model is over 20GB in size and so would require substantial RAM and CPU power. Unless you’re on a dedicated plan, you won’t get nearly enough power.
Anvil is largely serverless so that’s another issue with cold starts.
You’re better off trying to host it using HuggingFace’s Inference Endpoints service, at least as a start. You can then call that endpoint from your Anvil app and do whatever you want with it.
1 Like
Or you can always connect it using Anvil uplink.
2 Likes
Or use the open source anvil-app-server
As far as i know anvil does not provide the capability to host powerful machine to host large models.
But take a look at anvil uplink
You can run your model on a dedicated machine (real machine or cloud instance) out side of anvil and use the uplink to connect to your anvil web application.
You can also host anvil yourself using the anvil-app-server. Its basically the same as the frontend but without the webeditor
1 Like