How to stream frames received by image source as media objects

I am trying to make a face recognition web app on anvil server linked via jupyter notebook:

**I am trying to Do real time Face recognition in my web app , I have tried making function call via “While” loop and keeps on over writing image source by the media object I am receiving from my function on the jupyter notebook .

Issue I am facing is : It is taking a time to switch from one received media object to other . I am not able to show it like real time face recognition , it is loading and over writing images again and again which is purely not continuous making my real time recognition fail :**

code at anvil -

**def track_button_click(self, **event_args):
track_image = anvil.server.call(“track_encoding”,self.file_loader_1.file)
self.stop=False
while not self.stop:
self.image_2.source= anvil.server.call(“track_criminal”,track_image)

def track_stop_click(self, **event_args):
self.stop=True **

Python code -

def track_criminal(track_face_encoding):
    face_locations = []
    face_encodings = []
    face_names = []
   
    video_capture = cv2.VideoCapture(0)
    while True:
        if cv2.waitKey(1) & 0xFF == ord('q'):
            video_capture.release()
            cv2.destroyAllWindows()
            break
        # Grab a single frame of video
        ret, frame = video_capture.read()
        
        if ret:
            # Resize frame of video to 1/4 size for faster face recognition processing
            small_frame = cv2.resize(frame, (0, 0), fx=0.25, fy=0.25)
            # Convert the image from BGR color (which OpenCV uses) to RGB color (which face_recognition uses)
            rgb_small_frame = small_frame[:, :, ::-1]

            face_locations = face_recognition.face_locations(rgb_small_frame)
            face_encodings = face_recognition.face_encodings(rgb_small_frame, face_locations)

            for face_encoding in face_encodings:
                matches = face_recognition.compare_faces([track_face_encoding], face_encoding)
                name = "Unknown"

                face_distances = face_recognition.face_distance([track_face_encoding], face_encoding)
                best_match_index = np.argmin(face_distances)
                if matches[best_match_index]:
                    name = "Detected"
                face_names.append(name)

            for (top, right, bottom, left), name in zip(face_locations, face_names):
                top *= 4
                right *= 4
                bottom *= 4
                left *= 4

                # Draw a box around the face
                cv2.rectangle(frame, (left, top), (right, bottom), (255, 153, 153), 2)

                # Draw a label with a name below the face
                cv2.rectangle(frame, (left, bottom - 35), (right, bottom), (255, 153, 153), cv2.FILLED)
                font = cv2.FONT_HERSHEY_DUPLEX
                cv2.putText(frame, name, (left + 6, bottom - 6), font, 1.0, (53, 255, 153), 1)
            cv2.imwrite("buffer.jpeg",frame)
            media_objects=anvil.media.from_file("buffer.jpeg",'image/jpeg') 
            return media_objects
        else:
            break 

Go to Livestream button and chose Track person then upload your image and click on track Person.

Clone Link :

Anvil | Login
https://caring-shocking-nail.anvil.app

I am not sure if you can, however other people claim to have done it? (It looks pretty complicated to me)

I think the anvil-extras package just started or already has implemented web workers, I don’t know how this would all fit together, though.

https://www.google.com/search?q=can+i+use+cv2+in+browser

Also your clone link seems to be broken now:
image

If you want to do face recognition in the browser, perhaps look at Javascript implementations: face-api.js : A way to build a Face Recognition system in the browser. | by Jeeva Saravanan | TheLeanProgrammer | Medium

1 Like

I am not much familiar with Javascript . How will I would I connect js with my server database to load the images

Anvil is python in the browser that uses another software called Skulpt. (On the server side it is full vanilla python)

What Skulpt does is transliterate the browser side python code directly into javascript to be run in the browser. (One of the members of the Skulpt project team is also a founding member of Anvil)

This means you also have the ability to make javascript code run directly in the browser along side anvil, while still using python to interact with your databases in a way that you are already familiar with.

You end up with python hooks to be able to interact with the javascript code. I am also unfamiliar with writing javascript, but have still been able to use this feature set to use javascript libraries even though I am uninterested in the full js implementation.

1 Like

I have pasted correct clone link now

1 Like

I am not clear with how would I call server functions to get images and their respective names from database that is stored in the server only that will help in face recognition

One way to go about it would be to use CV2 in the browser to get some bounding boxes and a set of a few clear images of the face, then submit those frames to your other external service that would use just those frames as search data to get your results.
The results can them be returned to the client where CV2 could update the bounding box to include a caption identifying the person and an accuracy percentage. (if that is what you are after, it is typical of this kind of app)

You would have to research how to do all of this and make it work together, the transferring data to and from the client through anvil should become one of the easiest parts.

There are plenty of references available on the internet on how this has been done in various ways, whitepapers, examples, etc.