I am a retired hobbyist using Python for math and physics.
What I’m trying to do
I wish develop a cement recipe optimization app in Python. (yes: cement is fun for me)
There are typically 10 to 20 components needed to produce cement.
There are various kinds of constraints: chemical, technical, envornmental, … .
And of course, the cost has to be minimized.
What I’ve already tried in Python and JupyterLab
I know that the best approach is to use the Pyomo optimization modeling package.
I did or tried similar things using JulputerLab on my computer, on Colab, Azure notebook, on Datalore, …
The difficulties I expect to use Anvil for this project
First, the Pyomo package needs to be available.
As a fresh beginner, I have no idea about what I can do on Anvil to include Pyomo in the project.
This could be, I guess, mainly about the plan I would chose and about my (tiny) budget.
Or would that be possible on the “Free Plan”?
Second, that worries me more is that Pyomo relies on an “external solver”.
This is typically, for me, the IPOPT solver.
This is a binary file (for some versions it is several files).
This binary file needs to be installed separately somewhere on the computer. Pyomo should be able to access and execute this binary file.
My experience is that this is always tricky!
But there is hope! I have done that already on Datalore and other platforms.
How you could help me
In summary, I would like to be able to install Pyomo as well as the binary file that it needs.
I would appreciate your suggestions!
Getting a solver binary to Pyomo would depend on how Pyomo needs to get those binaries. Do you pass a file name? Do you pass the binary contents of the file? Do you pass a URL? Each of those has ways you can manage in Anvil.
I probably need a little more time to understand how Anvil works in this regard.
On Datalore it has been rather simple.
The required packages are listed in an “environment.yml” file
This can be done by editing this file or by the “Environment” menu.
And indeed the binary file can be included in the “Notebook files” folder which is intended for data.
As you guessed, the binary file is indeed accessed by Pyomo on the basis of its name.
So, to start with, I will “pip install pyomo” in the startup code of my toy app using “subprocess.run”.
I hope this will be allowed.
I will also add the Ipopt binary in the “Data Files service”.
Can I then get a path to the file and pass it to Pyomo?
Or is that not so easy because of the word “service” that troubles me?
Would you know this?
When you say binary, you mean an executable or binary data?
I don’t think the Anvil server will allow you to spawn a process or setup a service or demon. You are going to need to setup another machine wroth an uplink.
Yes I mean an executable.
Isn’t that common with Python?
What is special here is that Pyomo doesn’t come “batteries included”: you need to care for the external solver, even non-commercial solvers.
On Datalore as well as on Colab, I could bring the batteries myself.
Thanks
Michel
It isn’t common with web apps. The normal web app architecture has code executing for a limited time frame to produce response to the web browser. External executables aren’t a normal part of that.
With Anvil, @stefano.menci pointed you to the way to handle executables. You’d need an uplink program running on a computer you control (either your own actual computer, or one hosted somewhere) where you could safely install the binary and Pyomo and let them interact. Your Anvil client or server could both make calls to that Uplink as needed.
Maybe I misunderstood the meaning of “spawn a process or setup a service or demon”.
I just understand how Pyomo works.
Pyomo converts a model in a kind of data file for the solver.
It then “executes” the solver and reads the response file.
At that time, the solver is stopped.
On my Windows machine, when the solver needs a few seconds to execute, I have the time to see what happens in the Task Manager. The solver (IPOPT for me) first appears in the list of processes. I can see CPU load for the solver. Then the solver disappears from the list of processes when the solve if finished.
As a hobbyist, I can hardly consider to use an additional computer in the solution.
My dream was to develop something for the web with simple and cheap means.
I always expected the “external solvers” to pose some difficulties.
It was actually not too difficult on Colab, Azure Notebook (old), Datalore, Deepnote, Binder, Kaggle.
But these were all notebook-based aproaches.
Anvil was more attractive than notebooks for me.
I will still spend sometime on this question.
Maybe by calling the www.coin-or.org solvers server or the NEOS servers , but I know by experience this will not be very responsive because there is a long witing queue. It is not an attractive solution.
Thanks for this information which is quite logical.
However, I have investigated this package long ago.
The ipopt package on PyPI is a Python wrapper to the actual Ipopt solver.
Pyomo has its own wrappers for Ipopt as well as for other solvers.
This ipopt Python PyPI wrapper package also needs the Ipopt binary executable to be installed and it needs to be called by the ipopt wrapper.
The only difference with Pyomo, I guess, it that Pyomo communicates with Ipopt by using intermediate files (.nl files) that describe the (nonlinear programming) problem in a standard way. With the additional advantage of more mathematical automation for calculating gradients or hessians. With Pyomo, the user does not need to create functions to calculate the gradients and the hessian.
The ipopt Python PyPI wrapper communicates by passing “handles” to Python functions that calculate gradients and hessians as defined in Python by the user. It is therefore much more difficult to use for modeling large problems than Pyomo which is actually a modeling tool.
In summary, the Ipopt PyPI package is more another interface to the Ipopt executable than a way to avoid the Iopt executable altogether.
The funny thing is that even in Julia, the mathematical modeling tools rely on Ipopt or other solvers, based on external executables. There is still no native Julia version of Ipopt, and probably not before long.
I Python, of course, there is even less hope to avoid this use of external executables. This is by design.
They suggest to use Anaconda. I guess that, just like for pandas+numpy+other data analysis tools that heavily rely on c compiled code, also ipopt can be installed together with all the c libraries already compiled.
Anaconda is the best way to install those tools on windows, where their compilation would be likely to fail. Unfortunately you can’t use Anaconda on the Anvil server, and I don’t think they will allow you to install any executable or setup a service/demon.
But the base configuration comes with many packages already installed, including all the binaries required by numpy, pandas and co.
You could try to ask the Anvil team and see if they consider it a good candidate to be added to that configuration.