GET URL of uploaded image and pass to chat gpt

[Harry_Python]

What I’m trying to do:
I am trying to get the url from a file uploader and message the url out. Everything I tried so far has resulted in “none”. I might be doing something obviously wrong but I cannot seem to understand why its not displaying the url of the image? It is displaying as a image on the screen but not the url when i ask for it.

the image does get saved in the database. I just want to pull the url from the image on display or in the file uploader. It really doesnt matter at that point.

What I’ve tried and what’s not working:

 def button_save_click(self, **event_args):
    """This method is called when the button is clicked"""
    self.image_uploadedFile.source = self.file_loader_Office.file
    imgUrl = self.file_loader_Office.file.url
    anvil.server.call('saveFile',imgUrl)
    alert(imgUrl)
    #self.text_area_chatGPTOutput.text = anvil.server.call('definingPicture',imgUrl)
    
    pass

Code Sample:

# this is a formatted code snippet.
# paste your code between ``` 

Clone link:
share a copy of your app

[campopianoa]

Not sure if this helps or if you saw this post.

Get URL from uploaded Image

What I’m trying to do: I’m using a FileLoader component to upload an image. This image then needs to be passed to a Custom HTML form which has JCrop selection implemented such that the user can crop the image. The problem is that the FileLoader component provides an Anvil Media object which I cannot pass into my custom HTML form. I need a way of extracting a URL from the media object which is available to the front-end - and pass this URL to the HTML form.

[Harry_Python]

was not helpful.

Client Side

def button_save_click(self, **event_args):
    """This method is called when the button is clicked"""
    self.image_uploadedFile.source = self.file_loader_Office.file
    
    imgFile = self.file_loader_Office.file
    anvil.server.call('saveFile',imgFile)
    self.text_area_chatGPTOutput.text = anvil.server.call('definingPicture',imgFile)
    
    pass

Server Side

@anvil.server.callable
def definingPicture(img_file):
  response = client.chat.completions.create(
  
    model="gpt-4-vision-preview",
      messages=[
        {
          "role": "user",
          "content": [
            {"type": "text", "text": "identify all the furniture in the room"},
            {
              "type": "image_url",
              "image_url": {
                "url": **I need Image URL** ,
              },
            },
          ],
        }
      ],
      max_tokens=300,
    )
  return response.choices[0].message.content

I tried everything but I cant get the image url to pass to Chatgpt…

[campopianoa]

Just curious if you tried that temp url method, created client side, mentioned in the other post. Or if its client value still gives you None, or different error, etc.

[Harry_Python]

i tried the temporary method, but could not pass it to the server side. The server side code works with URL, the only part i am struggling with is getting a URL of the image into the server side.

[p.colbert]

This may help:

Get URL from uploaded Image

Just a guess here, but the temporary url is an object, not a string. If you’re trying to pass the object itself, that probably won’t work. The network protocol understands only some basic data types, not arbitrary object types. See Valid arguments and return values. The good news is that you can extract the url string from the object, so you could pass that instead. Follow Stu’s documentation link, above, for TempUrl, and read to the bottom. You’ll see how to extract the url string from th…

[Harry_Python]

@p.colbert I tried the temporary url object and turned it into a string. I test it with a print function and its the url. When i run my code, I get this error

BadRequestError: Error code: 400 - {'error': {'message': 'Invalid image.', 'type': 'invalid_request_error', 'param': None, 'code': None}}

The server side code is correct if I can pass a string ‘URL’ . I tested it with a random image online and it worked. What am I doing wrong?

Client Side

 def button_save_click(self, **event_args):
    """This method is called when the button is clicked"""
    self.image_uploadedFile.source = self.file_loader_Office.file
    imgFile = self.file_loader_Office.file
    temp_url = anvil.media.TempUrl(self.file_loader_Office.file) # takes a MediaObject
    imgUrl = str(temp_url.url)
    print(imgUrl)
    
  
    anvil.server.call('saveFile',imgFile)
    self.text_area_chatGPTOutput.text = anvil.server.call('definingPicture',imgUrl)
    temp_url.revoke()
    pass

Server Side

@anvil.server.callable
def definingPicture(imgUrl):
  response = client.chat.completions.create(
  
    model="gpt-4-vision-preview",
      messages=[
        {
          "role": "user",
          "content": [
            {"type": "text", "text": "identify all the furniture in the room"},
            {
              "type": "image_url",
              "image_url": {
                "url": imgUrl,
              },
            },
          ],
        }
      ],
      max_tokens=300,
    )
  return response.choices[0].message.content

[p.colbert]

Based on readings from past Forum posts, I suspect that the temporary url, created on the Client instance, is valid only for communication with that Client instance, not with any other program instance.

For a longer-lived URL, one that can be used by server-side code, you might send the image itself (as an Anvil Media Object) to the Server, where the Server could create its own url, to pass to other services. A Media Object stored to a database table can have a very long-lived url.

[Harry_Python]

I see. If I understood correctly, you are suggesting that;

1. I save the image
2. pull it up on the server side
3. then get the URL from that end?

How would i go about doing this?

[p.colbert]

@Harry_Python

How would i go about doing this?

As always, start with the documentation. Anvil’s is one of the best around! Here’s one starting point:
Files, Media and Binary Data

The Search bar at the top of the docs searches more than just the docs. It also searches this Forum, Tutorials, and other Anvil-supplied learning materials.

In my experience, self-service with Search is much faster than waiting for the volunteers on this Forum. We’re good people, but this isn’t our day job.

[Harry_Python]

I am still getting that same error message when i tried to this a completely different way.

When i print the url, it works. How come ChatGPT vision is giving me an error code 400? Seems like its not recognizing it as an image?

Error Message
BadRequestError: Error code: 400 - {‘error’: {‘message’: ‘Invalid image.’, ‘type’: ‘invalid_request_error’, ‘param’: None, ‘code’: None}}

Here is what I tried
Server Side

@anvil.server.callable
def saveFile(roomImg):
  row = app_tables.maindata.add_row(usersEmail=anvil.users.get_user()['email'],Office=roomImg)
  rowId = row.get_id()
  return rowId



@anvil.server.callable
def definingPicture(imgID):
  imgRow = app_tables.maindata.get_by_id(imgID)
  Img = imgRow['Office']
  ImgUrl= Img.get_url()
  print(ImgUrl)
  response = client.chat.completions.create( 
    model="gpt-4-vision-preview",
      messages=[
        {
          "role": "user",
          "content": [
            {"type": "text", "text": "identify all the furniture in the room"},
            {
              "type": "image_url",
              "image_url": {
                "url": str(ImgUrl),
              },
            },
          ],
        }
      ],
      max_tokens=300,
    )
  return response.choices[0].message.content

Client Side

def button_save_click(self, **event_args):
    """This method is called when the button is clicked"""
    self.image_uploadedFile.source = self.file_loader_Office.file
    roomImg = self.file_loader_Office.file
    
    
  
    imgID = anvil.server.call('saveFile',roomImg)
    self.text_area_chatGPTOutput.text = anvil.server.call('definingPicture',imgID)
    
    pass

[p.colbert]

@Harry_Python

When i print the url, it works.

As far as print() is concerned, a url is just a value of type string, so I would expect printing to work just fine. It’s not as if print() is trying to fetch the image from somewhere.

@Harry_Python

How come ChatGPT vision is giving me an error code 400?

Have you tried pasting the url into a web browser’s address bar? (In a new browser tab.) This might retrieve the image, or (if it fails) it might tell you more about where the problem is located.

[Harry_Python]

When I place the new URL into the browser, it downloads the image. So it appear to be working?

Here is clone of my app

Login

[Harry_Python]

I am trying to think outside of the box on this… When I use anvil.works URL it immediately downloads the image in the data tables. Can we just view the image instead of downloading it? Maybe ChatGPT needs to just see it and errors out when it downloads it?

[stefano.menci]

@Harry_Python

When I use anvil.works URL it immediately downloads the image in the data tables. Can we just view the image instead of downloading it?

I haven’t read the details of the thread, but looking only at this, it sounds like a problem with the MIME type.

Here is an HTTP endpoint that I use to upload images, in case it helps:

@anvil.server.http_endpoint('/add_bitmap/:wizard_name/:bitmap_name', methods=['POST'])
def add_bitmap(wizard_name, bitmap_name):
  extension = bitmap_name.split('.')[-1]
  if extension in ['gif', 'bpm', 'jpg', 'png']:
    mime = 'image/' + extension
  else:
    mime = 'application/octet-stream'

  bb = anvil.server.request.body.get_bytes()
  bb = base64.b64decode(bb)
  app_tables.bitmaps.add_row(wizard_name=wizard_name,
                             image_name=bitmap_name,
                             bitmap=anvil.BlobMedia(mime, bb, name=bitmap_name))

[Harry_Python]

It really did not work. I used Img.get_url(False) to disable the download. I got the image instead of the download, but ChatGPT still wont recognize it.

The funny thing is that when i place the URL from google of an image, I get the code to work.

Example: https://img.onmanorama.com/content/dam/mm/en/lifestyle/decor/images/2023/6/1/house-middleclass.jpg

@anvil.server.callable
def definingPicture(imgID):
  imgRow = app_tables.maindata.get_by_id(imgID)
  Img = imgRow['Office']
  ImgUrl= Img.get_url()
  print(ImgUrl)
  response = client.chat.completions.create( 
    model="gpt-4-vision-preview",
      messages=[
        {
          "role": "user",
          "content": [
            {"type": "text", "text": "identify all the furniture in the room"},
            {
              "type": "image_url",
              "image_url": {
                "url": **'https://img.onmanorama.com/content/dam/mm/en/lifestyle/decor/images/2023/6/1/house-middleclass.jpg'**,
              },
            },
          ],
        }
      ],
      max_tokens=300,
    )
  return response.choices[0].message.content

Replacing my string with a random image url, the code works flawlessly, but with my string it errors out. So weird!

[stefano.menci]

@stefano.menci

BlobMedia(mime, bb, name=bitmap_name)

Did you set the mime?

[Harry_Python]

@Harry_Python

Img.get_url(False)

No. I answered my own question by finding in the documentation that if I use

Img.get_url(False) 

It will not download the image. So I can view the image on the web browser.

[Harry_Python]

What I’m trying to do:
I am trying to pass an image from my datatable to chatGPT to analyze. I can view it in my browser in normal browser mode (chrome), however, when i use incognito mode, it gives me an error “Bad request: Invalid (Lazy) Media object”. Maybe that is why I keep getting this error?

BadRequestError: Error code: 400 - {'error': {'message': 'Invalid image.', 'type': 'invalid_request_error', 'param': None, 'code': None}}

Question 1:
How do I remove this security so i can pass my variable with url to ChatGPT without this.

Question 2:
Do you think this is the cause of the error? I am so baffled i cannot figure out why i am getting this error. The code is simple and works with URL off the web, but not with my own images from Anvil data table…

Code looks like this:
Client Side

  def button_save_click(self, **event_args):
    """This method is called when the button is clicked"""
    self.image_uploadedFile.source = self.file_loader_Office.file
    roomImg = self.file_loader_Office.file
    
    
  
    imgID = anvil.server.call('saveFile',roomImg)
    self.text_area_chatGPTOutput.text = anvil.server.call('definingPicture',imgID)
    
    pass

Server Side

@anvil.server.callable
def saveFile(roomImg):
  row = app_tables.maindata.add_row(usersEmail=anvil.users.get_user()['email'],Office=roomImg)
  rowId = row.get_id()
  return rowId



@anvil.server.callable
def definingPicture(imgID):
  imgRow = app_tables.maindata.get_by_id(imgID)
  Img = imgRow['Office']
  #ImgUrl= Img.get_url(False)
  #modified_url = ImgUrl[:-3]
  #print(modified_url)
  print(Img.get_url(False))
  NewImg = Img.get_url(False)
  response = client.chat.completions.create(
  model="gpt-4-turbo",
  messages=[
    {
      "role": "user",
      "content": [
        {"type": "text", "text": "What’s in this image?"},
        {
          "type": "image_url",
          "image_url": {
            "url": NewImg,
          },
        },
      ],
    }
  ],
  max_tokens=300,
)
  
  print(response.choices[0])

My clone link below

Clone link:

Login

[campopianoa]

Sorry to hear the core issue is the same. Out of curiosity, did you try setting the mime type like Stefano suggested? Worth a shot.