Using SpaCy from within Anvil (instead of via Uplink)

I am looking to upgrade to the Individual plan and was hoping to port my NLP code into Anvil. I use SpaCy too, so will I experience this issue as well (referring to the issue posted [here])(Server error when loading Spacy lib)

Right now I use uplink to call SpaCy related functions. It’s not bad in terms of speed. Will I be better off using Uplink? My NLP code sits in my local machine, so its not ideal to depend on uplink.

Hi @i.rajeshwari,

You could activate your free trial of the Full Python 3 runtime and try it out. It will also depend on what SpaCy model you are trying to load into memory. In that other post you reference, they were loading the largest model, which requires a fair bit more memory than the medium and small models.

Thank you Brooke. Will give this a go as you suggest and report back. I think for now we should be OK with the small model which is 11MB.

Hi ya,

So I have now upgraded and moved to the full Python version in anvil.

In my server module, I have:

    import spacy
    ```
but when I try to load the model:

```python
    @anvil.server.callable
    def loadSpacyModel():
      nlp = spacy.load("en_core_web_sm")
      print('Loaded SpaCy')
    ```
I get the following error:

OSError: [E050] Can't find model 'en_core_web_sm'. It doesn't seem to be a shortcut link, 
a Python package or a valid path to a data directory. at /usr/local/lib/python3.7/site-packages/spacy/util.py, line 119 
called from /usr/local/lib/python3.7/site-packages/spacy/__init__.py, line 21 called from [ServerModule1, line 39](javascript:void(0)) called from [mockTest_template, line 16](javascript:void(0))

Not sure what I am missing. In my local machine, after installing spacy, this code was enough to load the model and work off it. Given it is already installed in Anvil's environment, I don't need to download and install again?

With some googling around, it seems the solution lies in downloading the model again.
For eg. in this post.
But I’m not sure where I’m supposed to do it…

Is somebody able to respond to this please? I am unable to make any progress if I do not have SpaCy working in my app. I was hoping that by upgrading, I will not have to depend on Uplink for the NLP aspects, however I have not any luck so far.

Any suggestions much appreciated!
Thank you!

Hi @i.rajeshwari,

After a little digging myself, I think you are correct that the model needs to be downloaded separately as is implied in the Spacy docs. I will poke the relevant person to download this for you during the (UK) workday tomorrow. Hopefully the Uplink will suffice until then!

Thank you so much Brooke, you are a star. Sorry if I’m being impatient, but my entire app depends on this :slight_smile: Will look forward to hearing from you tom.

Hi @i.rajeshwari,

Thanks for your patience! The en_core_web_sm model is now downloaded. I just tried loading it myself with your code, and it worked :slight_smile:

2 Likes

Yes I’m able to load it as well. And it doesn’t take too long.
:slight_smile: What joy!

THANK YOU SO VERY MUCH!!

2 Likes

Hello,

I had a couple users report that they are experiencing an error when they try using a feature in the app that uses SpaCy. Please refer to the conversation above.

I tried it now and indeed I get this error:

FileNotFoundError: [Errno 2] No such file or directory: ‘/usr/local/lib/python3.7/site-packages/spacy/data/en_core_web_sm/init.py’ at , line 916 called from , line 859 called from , line 724 called from /usr/local/lib/python3.7/site-packages/spacy/compat.py, line 135 called from /usr/local/lib/python3.7/site-packages/spacy/util.py, line 126 called from /usr/local/lib/python3.7/site-packages/spacy/util.py, line 112 called from /usr/local/lib/python3.7/site-packages/spacy/init.py, line 21 called from [ServerModule1, line 265](javascript:void(0)) called from [mockTest_template, line 35](javascript:void(0))

The error is at this line:
nlp = spacy.load(“en_core_web_sm”)

The last time this code was successfully accessed was around noon UK time, dec 15th.

I have not changed anything related to this feature since this version of the app got published. Any ideas please?

Just to update that I tried to use it now and all seems fine.

Please are you able to explain why this might have happened and how can I attend to this in the future?

Thank you

From this thread, I thought a Spacy load would work. When I run this:

import spacy
import en_core_web_sm
NER = spacy.load(“en_core_web_sm”)

I get this traceback:

ModuleNotFoundError: No module named '_swigfaiss'

Need an extra library installed? Click for more info

  • at /usr/local/lib/python3.7/importlib/__init__.py:127
  • called from /usr/local/lib/python3.7/site-packages/faiss/swigfaiss.py:16
  • called from /usr/local/lib/python3.7/site-packages/faiss/swigfaiss.py:17
  • called from /usr/local/lib/python3.7/site-packages/faiss/__init__.py:19
  • called from /usr/local/lib/python3.7/site-packages/transformers/models/rag/retrieval_rag.py:42
  • called from /usr/local/lib/python3.7/site-packages/transformers/models/rag/modeling_rag.py:29
  • called from /usr/local/lib/python3.7/site-packages/transformers/models/auto/modeling_auto.py:199
  • called from /usr/local/lib/python3.7/importlib/__init__.py:127
  • called from /usr/local/lib/python3.7/site-packages/transformers/models/auto/__init__.py:198
  • called from /usr/local/lib/python3.7/site-packages/transformers/file_utils.py:1699
  • called from /usr/local/lib/python3.7/site-packages/transformers/file_utils.py:1700
  • called from /usr/local/lib/python3.7/site-packages/transformers/__init__.py:2487
  • called from /usr/local/lib/python3.7/site-packages/spacy_transformers/util.py:4
  • called from /usr/local/lib/python3.7/site-packages/spacy_transformers/data_classes.py:12
  • called from /usr/local/lib/python3.7/site-packages/spacy_transformers/layers/listener.py:4
  • called from /usr/local/lib/python3.7/site-packages/spacy_transformers/layers/__init__.py:1
  • called from /usr/local/lib/python3.7/site-packages/spacy_transformers/architectures.py:6
  • called from /usr/local/lib/python3.7/site-packages/spacy_transformers/__init__.py:1
  • called from <frozen importlib._bootstrap>:219
  • called from <frozen importlib._bootstrap_external>:728
  • called from <frozen importlib._bootstrap>:677
  • called from <frozen importlib._bootstrap>:967
  • called from <frozen importlib._bootstrap>:983
  • called from <frozen importlib._bootstrap>:1006
  • called from <frozen importlib._bootstrap>:219
  • called from <frozen importlib._bootstrap>:953
  • called from <frozen importlib._bootstrap>:983
  • called from <frozen importlib._bootstrap>:1006
  • called from /usr/local/lib/python3.7/importlib/__init__.py:127
  • called from /usr/local/lib/python3.7/site-packages/catalogue/_importlib_metadata/__init__.py:94
  • called from /usr/local/lib/python3.7/site-packages/catalogue/__init__.py:124
  • called from /usr/local/lib/python3.7/site-packages/catalogue/__init__.py:109
  • called from /usr/local/lib/python3.7/site-packages/spacy/language.py:162
  • called from /usr/local/lib/python3.7/site-packages/spacy/language.py:1755
  • called from /usr/local/lib/python3.7/site-packages/spacy/util.py:531
  • called from /usr/local/lib/python3.7/site-packages/spacy/util.py:488
  • called from /usr/local/lib/python3.7/site-packages/spacy/util.py:422
  • called from /usr/local/lib/python3.7/site-packages/spacy/__init__.py:52
  • called from servicesExcel, line 20

(line 20 is the spacy.load statement)

Any help?

thanks!

You could wait for someone who knows better to answer but…
Have you heard about the new ability to install packages? You might want to try installing the newest version of SpaCy directly into your anvil server and see if that works?