How to use Open AI Whisper in an Anvil app

[pierreyohann16]

Hello,

I am aware that some Open AI libraries can be accessed from Anvil apps. Is there a way to have the same access to the Open AI Whisper newly released STT API?

Thank you for any idea about that subject.

All the best,

Pierre-Emmanuel

[jshaffstall]

Welcome to the forum!

Since Whisper seems like it’s a Python library, not an API, this post on installing custom Python libraries should apply: 📢 New Beta: Install Any Python Package Instantly

[pierreyohann16]

Thanks a lot! I will try to install it and I will give feedback to the community.

Pierre-Emmanuel

[pierreyohann16]

Hello,

I used the method you kindly recommended. I used the requirements.txt line for whisper on my local working install:

whisper @ git+https://github.com/openai/whisper.git@7f3e408e092e73d472036ae2e3fba1e7c68ca4e6

This gives me:

and the full terminal output is:

Installing custom packages...
Collecting whisper@ git+https://github.com/openai/whisper.git@7f3e408e092e73d472036ae2e3fba1e7c68ca4e6
  Cloning https://github.com/openai/whisper.git (to revision 7f3e408e092e73d472036ae2e3fba1e7c68ca4e6) to /tmp/pip-install-avcaaixs/whisper_5fce8535847143baaff8abff760d5f6d
  Running command git clone --filter=blob:none --quiet https://github.com/openai/whisper.git /tmp/pip-install-avcaaixs/whisper_5fce8535847143baaff8abff760d5f6d
  Running command git rev-parse -q --verify 'sha^7f3e408e092e73d472036ae2e3fba1e7c68ca4e6'
  Running command git fetch -q https://github.com/openai/whisper.git 7f3e408e092e73d472036ae2e3fba1e7c68ca4e6
  Running command git checkout -q 7f3e408e092e73d472036ae2e3fba1e7c68ca4e6
  Resolved https://github.com/openai/whisper.git to commit 7f3e408e092e73d472036ae2e3fba1e7c68ca4e6
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: numpy in ./.env/lib/python3.10/site-packages (from whisper@ git+https://github.com/openai/whisper.git@7f3e408e092e73d472036ae2e3fba1e7c68ca4e6->-r /home/anvil/custom-requirements.txt (line 1)) (1.23.2)
Collecting torch
  Downloading torch-1.13.0-cp310-cp310-manylinux1_x86_64.whl (890.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 890.1/890.1 MB 36.7 MB/s eta 0:00:00
Collecting tqdm
  Downloading tqdm-4.64.1-py2.py3-none-any.whl (78 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78.5/78.5 kB 37.6 MB/s eta 0:00:00
Collecting more-itertools
  Downloading more_itertools-9.0.0-py3-none-any.whl (52 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 52.8/52.8 kB 29.6 MB/s eta 0:00:00
Collecting transformers>=4.19.0
  Downloading transformers-4.25.1-py3-none-any.whl (5.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.8/5.8 MB 38.5 MB/s eta 0:00:00
Collecting ffmpeg-python==0.2.0
  Downloading ffmpeg_python-0.2.0-py3-none-any.whl (25 kB)
Requirement already satisfied: future in ./.env/lib/python3.10/site-packages (from ffmpeg-python==0.2.0->whisper@ git+https://github.com/openai/whisper.git@7f3e408e092e73d472036ae2e3fba1e7c68ca4e6->-r /home/anvil/custom-requirements.txt (line 1)) (0.17.1)
Requirement already satisfied: packaging>=20.0 in ./.env/lib/python3.10/site-packages (from transformers>=4.19.0->whisper@ git+https://github.com/openai/whisper.git@7f3e408e092e73d472036ae2e3fba1e7c68ca4e6->-r /home/anvil/custom-requirements.txt (line 1)) (21.3)
Collecting regex!=2019.12.17
  Downloading regex-2022.10.31-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (770 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 770.5/770.5 kB 57.4 MB/s eta 0:00:00
Collecting filelock
  Downloading filelock-3.8.0-py3-none-any.whl (10 kB)
Collecting huggingface-hub<1.0,>=0.10.0
  Downloading huggingface_hub-0.11.1-py3-none-any.whl (182 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 182.4/182.4 kB 30.9 MB/s eta 0:00:00
Requirement already satisfied: requests in ./.env/lib/python3.10/site-packages (from transformers>=4.19.0->whisper@ git+https://github.com/openai/whisper.git@7f3e408e092e73d472036ae2e3fba1e7c68ca4e6->-r /home/anvil/custom-requirements.txt (line 1)) (2.28.1)
Collecting tokenizers!=0.11.3,<0.14,>=0.11.1
  Downloading tokenizers-0.13.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.6/7.6 MB 30.7 MB/s eta 0:00:00
Collecting pyyaml>=5.1
  Downloading PyYAML-6.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (682 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 682.2/682.2 kB 63.0 MB/s eta 0:00:00
Collecting nvidia-cudnn-cu11==8.5.0.96
  Downloading nvidia_cudnn_cu11-8.5.0.96-2-py3-none-manylinux1_x86_64.whl (557.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 557.1/557.1 MB 33.7 MB/s eta 0:00:00
Collecting nvidia-cublas-cu11==11.10.3.66
  Downloading nvidia_cublas_cu11-11.10.3.66-py3-none-manylinux1_x86_64.whl (317.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 317.1/317.1 MB 29.7 MB/s eta 0:00:00
Collecting typing-extensions
  Downloading typing_extensions-4.4.0-py3-none-any.whl (26 kB)
Collecting nvidia-cuda-runtime-cu11==11.7.99
  Downloading nvidia_cuda_runtime_cu11-11.7.99-py3-none-manylinux1_x86_64.whl (849 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 849.3/849.3 kB 74.0 MB/s eta 0:00:00
Collecting nvidia-cuda-nvrtc-cu11==11.7.99
  Downloading nvidia_cuda_nvrtc_cu11-11.7.99-2-py3-none-manylinux1_x86_64.whl (21.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 21.0/21.0 MB 39.1 MB/s eta 0:00:00
Requirement already satisfied: setuptools in ./.env/lib/python3.10/site-packages (from nvidia-cublas-cu11==11.10.3.66->torch->whisper@ git+https://github.com/openai/whisper.git@7f3e408e092e73d472036ae2e3fba1e7c68ca4e6->-r /home/anvil/custom-requirements.txt (line 1)) (65.3.0)
Requirement already satisfied: wheel in ./.env/lib/python3.10/site-packages (from nvidia-cublas-cu11==11.10.3.66->torch->whisper@ git+https://github.com/openai/whisper.git@7f3e408e092e73d472036ae2e3fba1e7c68ca4e6->-r /home/anvil/custom-requirements.txt (line 1)) (0.37.1)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in ./.env/lib/python3.10/site-packages (from packaging>=20.0->transformers>=4.19.0->whisper@ git+https://github.com/openai/whisper.git@7f3e408e092e73d472036ae2e3fba1e7c68ca4e6->-r /home/anvil/custom-requirements.txt (line 1)) (3.0.9)
Requirement already satisfied: charset-normalizer<3,>=2 in ./.env/lib/python3.10/site-packages (from requests->transformers>=4.19.0->whisper@ git+https://github.com/openai/whisper.git@7f3e408e092e73d472036ae2e3fba1e7c68ca4e6->-r /home/anvil/custom-requirements.txt (line 1)) (2.1.1)
Requirement already satisfied: certifi>=2017.4.17 in ./.env/lib/python3.10/site-packages (from requests->transformers>=4.19.0->whisper@ git+https://github.com/openai/whisper.git@7f3e408e092e73d472036ae2e3fba1e7c68ca4e6->-r /home/anvil/custom-requirements.txt (line 1)) (2022.6.15)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in ./.env/lib/python3.10/site-packages (from requests->transformers>=4.19.0->whisper@ git+https://github.com/openai/whisper.git@7f3e408e092e73d472036ae2e3fba1e7c68ca4e6->-r /home/anvil/custom-requirements.txt (line 1)) (1.26.12)
Requirement already satisfied: idna<4,>=2.5 in ./.env/lib/python3.10/site-packages (from requests->transformers>=4.19.0->whisper@ git+https://github.com/openai/whisper.git@7f3e408e092e73d472036ae2e3fba1e7c68ca4e6->-r /home/anvil/custom-requirements.txt (line 1)) (3.3)
Building wheels for collected packages: whisper
  Building wheel for whisper (setup.py): started
  Building wheel for whisper (setup.py): finished with status 'done'
  Created wheel for whisper: filename=whisper-1.0-py3-none-any.whl size=1175210 sha256=fdfdf55571d76e2ec1e40f1b1b0224f6f261aad402c240b10c07f86254c2a436
  Stored in directory: /tmp/pip-ephem-wheel-cache-f8fj8ef7/wheels/eb/47/ff/4702f12d15399fd5feb975879fccb109e2353d2a7a4ffd7877
Successfully built whisper
Installing collected packages: tokenizers, typing-extensions, tqdm, regex, pyyaml, nvidia-cuda-runtime-cu11, nvidia-cuda-nvrtc-cu11, nvidia-cublas-cu11, more-itertools, filelock, ffmpeg-python, nvidia-cudnn-cu11, huggingface-hub, transformers, torch, whisper
Error: Failed to build image (timeout after 120000 ms)

So the collection and built seems to be ok but the install is doomed by a timeout.
I also need to find a way to get ffmpeg installed on the server.

I will keep on trying to install whisper and I will let the community benefit from my findings.

All the best !

Pierre-Emmanuel

[pierreyohann16]

Hello,

As per thread Unable to build custom package - timeout error

ffmpeg-python is not supported by python 3.10

I also found Google Issue Tracker

It seems that the only way to use ffmpeg_python is to use python 3.9 but this ersion cannot be used to install custom packages on Anvil.

I’ll keep on searching…

All the best,

Pierre-Emmanuel

[pierreyohann16]

Hello,

I have found a ffmpeg-python on pypi but I get:

I will try to update the package.

All the best !

Pierre-Emmanuel

[jshaffstall]

Another option, if you cannot get everything to install on the Anvil server, is to use an Anvil uplink server on your machine. That allows you to install whatever version of Python you want and whatever Python packages you want. You will need to have the uplink server running all the time, though.

[pierreyohann16]

Hello,

I think you are right since it is a python and not an Anvil issue. I will try your uplink server solution and I will keep the community updated.

Thanks again,

Pierre-Emmanuel

[zielinski.mark]

Hi Pierre-Emmanuel,
Did you find a solution to your issue?

[advancedhealthsys]

I am facing the same issue here (with different laborers: Transformer and torch) . Any solution without the need to use uplink?

Never mind , the libraries I need already installed in full python 3

[yahiakalabs]

I stumbled on this using whisper in my app and found out that ffmpeg is already installed on the Anvil server. You don’t actually need ffmpeg to use whisper but I needed it to convert different audio file formats.

You can also use the whisper API by just installing the openai package:

https://platform.openai.com/docs/guides/speech-to-text

Here’s a post that is helpful when using ffmpeg:

No sound from video file using default image component

Hi @joinlook, David’s example is based on HTML video tags, so there may be an issue with browser support. Some video formats are not supported by all browsers, so transcoding might solve your problem. Mozilla have a table showing which formats are supported by which browsers: [image] The most well-supported format seems to be H.264 video, AAC audio, in MPEG-4, as suggested by the table and by this StackOverflow answer. ffmpeg We’ve installed ffmpeg and python-ffmpeg in Server Modules. It’…