Just been testing my serializations from my jupytur notebook

Anvil Q&A
1

Hey Tony again!

I have so many questions! it baffles my mind with what anvil can do!

Right now, im messy around testing whether i call entire models i generated from my jupytur notebook or whether i can pass large amounts of data via APIs. I tried using the yfinance API to requests stock data to be processed in my jupytur notebook, but its saying “anvil.server.UplinkDisconnectedError: Data payload too big - please use Media objects to transfer large amounts of data.”. Also when i try to return my model from my jupytur notebook, it says "
anvil.server.SerializationError: Cannot serialize return value from function. Cannot serialize <class ‘xgboost.core.Booster’> object at msg[‘response’][5]"

What options do i have?

-Tonu

2

Hi Tony,

I’m getting started with this kind of work as well with Anvil. It’s very flexible but there are a few things I’ve learned.

If you need to pass large data from your app to your notebook then use media object. This can be done by converting the data to an object then sending it to your uplink function. Ideally though you would pass something smaller, like user defined parameters, and have the data pulled from an API by your Jupyter notebook.

As for returning predictions from your model, it would be recommended to do predictions on the uplink machine and return the values as a list or dictionary or even strings.

If you give a bit more description on the specifics of what you are trying to move around it would be helpful for everyone here.

1 Like
5

for right now, ive optimized a model from my jupyter notebook, saved it and im making predictions from my loaded xgboost booster model. i also request data from yfinance API from my jupyter notebook and my jupyter notebook dumps back to my app a numpy array.

Im thinking whether i can do the actual training and data visualizations from anvil server modules by importing libraries like xgboost and shap, if not just have my function return my boosted model

also, even call models from my client form into my server module function, create a shap image, then return it back to my client form and out put it to my UI.

-Tony

6

Does the code listed here return an error? Is it the size error you listed above?

7

There are lots of features im planning to add in the future. For now, i just want my app to be able to take in a stock name string, then output visualizations that i generate from my jupyter notebook

my jupyter notebook doesn’t return an error, anvil is saying “AttributeError: ‘NoneType’ object has no attribute ‘data’
at /home/tony/Desktop/github_repos/Dataquest-modules/venv/lib/python3.7/site-packages/shap/plots/force.py, line 257
called from , line 34”

i do find my media object in where i wanted to save it to

8

https://anvil.works/build#clone:YQJ35PBVIRDRBXRJ=GNHT3RIHRA6WWDOSKIES3ESC

9

I’m away from my computer right now but I can give it a try later.

Have you tried converting it this way:

11

How are you attempting to show the image on your app? Can you show the code you are using?

12

https://anvil.works/build#clone:YQJ35PBVIRDRBXRJ=GNHT3RIHRA6WWDOSKIES3ESC

this is what i have so far

this is my jupyter notebook repo

import anvil.media

@anvil.server.callable
def predictions(string):
    requested_stock = yf.Ticker(string)
    history = requested_stock.history(period='max')
    stock = pd.DataFrame(history)
    stock_changes = stock.pct_change()
    stock_changes.drop(stock_changes.index[0],inplace=True)
    stock_matrix = pd.concat([stock_changes.Close.shift(-i) for i in range(100)],axis=1)
    stock_matrix.drop(stock_matrix.index[-99:],inplace=True)
    stock_matrix.columns = [ f'Day {i+1}'for i in range(len(stock_matrix.columns))]
   
    X = stock_matrix.loc[:,'Day 2':]
    y = stock_matrix['Day 1']
    X_train,X_test,y_train,y_test = train_test_split(X,y,test_size=0.2)
    
    dtrain = xgb.DMatrix(X_train,y_train)
    dtest = xgb.DMatrix(X_test,y_test)
    loaded_models = xgb.Booster()
    loaded_models.load_model('xgbregression.model')
    prediction = loaded_models.predict(dtest)

    explainer = shap.TreeExplainer(loaded_model)
    shap_values_XGB_test = explainer.shap_values(X_test)
    shap_values_XGB_train = explainer.shap_values(X_train)

    df_shap_XGB_test = pd.DataFrame(shap_values_XGB_test)
    df_shap_XGB_train = pd.DataFrame(shap_values_XGB_train)
    shap.force_plot(explainer.expected_value,shap_values_XGB_test[0],X_test.iloc[[0]],show=False,matplotlib=True).savefig(
        '/home/tony/Desktop/github_repos/Dataquest-modules/My_Notebooks/media/shap.png')
#     shap.save_html('/home/tony/Desktop/github_repos/Dataquest-modules/My_Notebooks/media/best_model_bar.png',plot)
   
    
    return prediction,anvil.media.from_file('/home/tony/Desktop/github_repos/Dataquest-modules/My_Notebooks/media/shap.png','text/html')

this is the specific function im fixingPreformatted text

13

Hi @thefool363, nice to see you around again!

Here are a couple of sections of the reference docs that may be of help.

@robert has pointed out that if you want to pass large amounts of data around (anything > 4MB), you can turn them into Media Objects so they will be streamed. Here’s the docs about constructing Media Objects:

There is a limited set of things that can be returned from server functions. The docs say:

The arguments and return values of @anvil.server.callable functions must be very simple. They may only be strings, numbers, lists, dicts, datetime.date , datetime.datetime , None , Media objects, or rows from a data table. They may not be circular (for example, a dict may not contain itself).