Make the world better? Remove some Javascript.

Back when my friend Ian and I were idealistic students, we sat down and decided to make the world a better place, as you do. And we decided that the best way two developers could make the world a better place was to remove some Javascript from it.

The web is the greatest application delivery platform on Earth, and frankly it’s a stain on our profession that you need to know five different programming languages and three different frameworks if you want to make it do anything. Really, all you should need is to be able to code - ideally, in a friendly, readable language that’s easy to learn and isn’t full of unpleasant surprises.

There was an obvious candidate.

So, we built a tool for building full-stack web applications with nothing but Python.

We built a graphical interface builder for designing your pages, and let you write your client-side code in Python - so all the buttons, text fields etc you put on your page are just Python objects. We built a back-end server environment, also in Python, and a database with a pure Python API, all deployed in the cloud in one click.

But today I want to talk about this front-end code.

If you want to drive the items on your web page as pure Python objects, you’re going to need to run Python in the browser. If you’re going to run something in the browser, it’s going to have to be in Javascript.

Thankfully, there’s a great open-source project called Skulpt, that compiles Python to Javascript. So you can write Python to drive your app, and we can compile it to Javascript so that when you publish it online, your code actually runs in a user’s browser. We give Skulpt Python objects, it represents them as Javascript objects. We give it a Python function, and it produces a Javascript function for the browser to run.

There is a problem, though. Javascript, in its infinite wisdom, is 100% non-blocking. If you kick off a process that finishes later, you’d better provide a callback.

Here’s some code you might write in Python. Go get a record from the database, and if it’s not there, throw an error.

And here’s how you’d do it in Javascript. I count three separate callbacks here. I’m not exaggerating - this is an example straight from the documentation of the postgres library!

OK, so we have a Python-to-Javascript compiler, and it’s open source. What if we could turn this into this, automatically?

So, we go “git clone”, and start exploring. It turns out that Skulpt has a classic compiler architecture, modeled quite closely on CPython: There’s a parser that turns Python code into a parse tree (these are all the files in the Skulpt source tree, by the way, if you want to follow along).

Then it walks over this parse tree and makes an abstract syntax tree, which is made of elements you might recognise - like variable assignments, function definitions, function calls, and so on. Then there’s a compiler that then walks this syntax tree and spits out Javascript.

So, this is what we need to modify to implement blocking code. We invent a new Javascript class that a function can return, to say “hey, I’m returning, but I’m not done yet; I’ve just blocked”. We call this a suspension.

So, when we’re compiling the AST, wherever we’re compiling a function call, we generate Javascript code that calls the function, and looks at the return value, and says, “is it a Suspension?”. Because if it returned a Suspension, we’re blocking - and I’m going to have to suspend myself. We save all our variables and temporary values, our exception state, and where we are in the function, and then we return our own suspension with all that information in it.

And so our caller will recognise that we have suspended, and so it’s going to have to suspend as well, and so on all the way back to the Javascript runtime.

When that operation completes - say the database gives us a response - we resume the Suspension. We call our function with the suspension, which then restores all our variables, jumps to the right place and resumes the original suspension, which restores all of its variables, and so on, until we can resume execution as normal.

So, that’s how we take simple blocking Python, and compile it into non-blocking Javascript so you don’t have to.

This has been a quick overview; if you’re interested you can check out Skulpt at skulpt.org - I’m one of the maintainers now, and we’re always looking for new contributors.

And if you’re fed up to the back teeth with all this Javascript, and you want to forget it all and write full-stack web apps with nothing but Python, please check us out at anvil.works.

Thank you very much!