An app to record drawings (with audio and animations)

It is available here: drawcast.app

And here is a (brief and bad!) example of what you can create: scast
(Click inside the frame to start it).

This project is not finished. There are still many bugs and you should not rely on it! Also, please do not try to save long recordings. All recording may be deleted at this stage - due to limited space!

Still, I would like to share. Both to demonstrate the power of Anvil and to get some feedback at this stage.

Background and lessons.

I wanted to be able to make screencasts for my students.

Starting with the signature app by @meredydd, I first build an app that recorded all events the user made on the canvas in a list.

For instance: (“line”, 100, 100, 150, 150, 0.1), (“draw”, ((100,100,102,103, 0.05), (100,100,102,103, 0.12)

The first element is the type of action, the second is the data needed to recreate the action, and the last data point is the time to sleep between or after an action.

The whole drawing could then saved and recreated as an animation by the player which just read from the list and did every action on the canvas.

At this point I should have stopped, but …

- Audio: I spent lots of time implementing audio recording and trying to make sure the drawing and the audio is synchronized. Recording had to be converted to base64 strings. This part is quite complex and requires some ugly javascript. A custom component for sound/camera recording would be perfect, but the forum is also very helpful. @stucork has several posts on this, and some of the more recent approaches are better than what I implemented.

- Smoothing: There are several algorithms to smooth curves and to make drawn words look nicer, this was useful.

- Animations: I got overly ambitious and tried to allow the user to move objects around on the screen after they had been drawn. You can draw a circle and click on it and drag it around. This part is still far from perfect. It requires the user to click at a specific point (end of a curve, or middle of a circle). It also introduced lots and lots (and lots) of complexity. But it was a cool feature! I am working on an improved algorithm that allows the user to click anywhere close to an object (not just a specific point). It required an ugly change to the list of events that describes the animation. Each event had to get a unique id. Only then was it possible to click on an event drawn ten steps ago, move it, and redraw the whole thing.

- Conversion to a video: Initially, the aim was simply to allow the user to record and play back the animation (and the sound) by accessing the app (with a unique hash for each animation). In this way, the animation could be deployed and shared right away. The user would get a link like drawcast.app#draw_id and going there they would see and hear the animation. Anvils ability to extract the name after the hash is perfect for this. The initial file would just look for the hash, extract the name, get it the list of event (and the audio) from a datatable and play it.

After implementing this, I, once again, got too ambitious: What if the final result could be turned into an mp4 or a webm file. A few days later, and after way too much javascript, it turns out that it is possible to record the events on a canvas (and the sound being played) and convert it to a video. Right now this is done automatically and it (automatically) downloads the file when the recording is finished. I know it looks a bit fishy (automatically downloading files) and I will revise it, but it was great for testing features.
Also, at this point I got close to throwing the computer out of the window, since the video introduced a black background and flickers. It took way too much effort - that introduced other bugs - to reduce this. In short, you have to fill the canvas with a white rectangle (at several points) to reduce flickers and the black background.

All in all, canvas and anvil proved to be very powerful. I struggled with the audio and the javascript. I was also surprised at how much time it takes to create boilerplate stuff like allowing users to store things, view all their animations, upgrade to pro account (experimental, not really implemented) retrieve and animation, delete it. I have not focused much on layout and being esthetically challenged, I would probably need help with this.

Feel tree to try, but - again - please ignore all bugs, do not rely on it, but I am happy to take feedback.

Because of some external time constraints, I may not be able to help explain all details, but I will try to answer questions.

9 Likes

By the way, the best info on audio recording: https://anvil.works/forum/t/pyaudio-client-end-using-microphone-user/9484

2 Likes

Wow, I’ve only peeked briefly so far, but this looks amazing. For now I mostly just wanted to ask: what kind of feedback are you interested in at this stage? It sounds like you don’t want bug reports or layout feedback, if I understand correctly. Is there a kind of feedback you are especially hoping for?

Thanks. Good question.

In addition to praise, flattery and general adulation, I was hoping to get some feedback on how intuitive it is to use, potential features that could be added (or deleted).

I should add some instructions, but I was also hoping that it would be almost self-explanatory. You can draw on the screen, insert shapes, click on objects to move them and everything is recorded (including the audio) and you can share the animation (you get a link). Finally, you can also get a video file of the animation (webm format, mp4 is also easy to add).

I know there are several (many!) bugs. The transparent fill seems to be solid (until it is moved!) now - after I made some other changes. But there might be other bugs as well and I am happy to get a note since I have not discovered all.) And moving objects sometimes fails. I just discovered that the link on the “player page” was wrong (Well, since it did not start with “http” is seems to be interpreted as a local link in anvil. Did not know that!)

If somebody know how to get rid of the flickering, I would be happy (I know there are some solutions - involving reset_contex and drawing at regular intervals) but so far they did not work in my case).

Also the black background when recording events from a canvas to create a (webm) video was a very annoying problem and I think my solution (to draw a white rectangle at strategic points in the script), is ad hoc and it introduced other bugs. Very happy if somebody know the solution.

But these things take lots of time, and I guess I just wanted to put it now out to give me some extra motivation and perhaps also collaborate with somebody.

1 Like