“Python in the Browser”: The Promised Land?

Hanging out in the Python community, you’ll see periodic waves of excitement about Python in the browser. “Finally,” we think, “we can cast off the dead weight of Javascript! The world will be in harmony once more, and rainbows and butterflies will inhabit the land.” Unfortunately, that’s not generally how it turns out. In this post, I’m going to talk about why it’s tempting, why it doesn’t work, and what to do instead.

What’s the problem?

But first, why is “Python in the browser” so alluring in the first place? Well, for most people who aren’t web developers (and many who are), the web is a place of pain. It’s not just the parade of languages – HTML, Javascript, CSS, Python, SQL – it’s the endless frameworks that come with them – React, Redux, Bootstrap, Flask, SQLAlchemy (and don’t get me started on Webpack). There is a fairly infamous flowchart of the learning path. And, of course, there’s the massive, disruptive churn – are we using Svelte this week? Typescript? Tailwind? Vue is so last year, darling.

What is it about the web that causes this endless, churning proliferation of frameworks? It’s because the web stack is trying to do something really difficult – and when we understand what that is, we’ll understand why saying “boo Javascript, let’s use Python instead” doesn’t really help us.

Why is the web hard?

Let’s think about it from the perspective of your data. If you’re writing a traditional web app, your data exists as:

  • Rows in a database table, accessed by SQL, which you translate into:
  • Server side Python1 objects, accessed by methods and attributes, which you translate into:
  • JSON, accessed over HTTP endpoints with just a few verbs (GET, POST, PUT, DELETE), which you translate into:
  • Javascript objects, accessed by methods and attributes, which you translate into:
  • HTML DOM objects, accessed by their own API, which you render into:
  • Pixels on a screen

That’s a lot of translation. And indeed most of the day-to-day work of web development is translating information between these forms. This work is tedious and repetitive, and we all know what programmers do with tedious and repetitive tasks. We automate them!

So we build ORMs to translate SQL rows into objects, web frameworks to help us translate those objects into JSON and HTTP endpoints, front-end frameworks to help us fetch that JSON and translate it into JS objects, templating engines to turn those JS objects into HTML DOM, and CSS frameworks to turn them into pixels.

Now we have two problems: As well as four-or-five different programming languages, we now have to understand four-or-five frameworks to get anything done. Even worse, those frameworks are extremely leaky abstractions. Each framework is bridging two domains that have very different ways of representing the world, and so they have to twist the semantics of one layer in order to represent the adjacent one. This will bite you unless you understand both of those layers and how the framework chooses to do the translation.

An example

Four out of five of those translations are associated with the front end, so we often cast the blame on Javascript, but let’s change it up and take an example from the back end: SQL ⬌ Python. If you were querying a database to find inexpensive books, you could write SQL like this:

SELECT * FROM books WHERE price < 20

Or you could use SQLAlchemy, and write the query in Python:

books = DBSession.query(Book).filter(Book.price<20)

This is nice - you can write your query in Python, and get Python objects back. But how does that Python call get translated to a database query? Black magic, that’s how. It’s got metaclasses, it’s got overloaded operators…and if you don’t understand that, sooner or later you’ll come unstuck. It would be reasonable to assume that Book.price<20 is a numerical comparison operation, so you could write something like Book.price<20 and Book.is_hardcover. But if you try that, it won’t work, and you’ll have to learn that “<” is actually an overloaded operator that returns some intermediate object that generates code in another programming language, and that doesn’t play nicely with the and operator. Of course, you need to generate efficient SQL, so you’ll have to learn how to tickle that code generator just right… All that, and you still need to be fluent in SQL.

You’ve bought yourself some concise code, but you’ve paid for it with a lot of complexity. And that’s not because SQLAlchemy is badly designed – it’s a great library! It’s because you’re twisting a Python statement to represent an SQL query, in order to bridge between two very different representations of the world. A few stress fractures are inevitable.

(And if you’re a front-end developer feeling smug about those silly ORMs right now, I have two words for you: React. Hooks. Twisting Javascript into representing HTML DOM creates at least as many sharp edges.)

You could probably get away with this sort of complexity once or twice in your stack. But with a framework like this at every level? You’re going to produce, well, this flowchart.

What does this have to do with Python?

Python developers look at the jagged rock-face of web development, compare it to the landscape they’re used to, and rightly think, “I don’t have to deal with any of this in Python! Why can’t we just use Python instead of all this JS junk?”

But by now, you already know the answer: Javascript is not the problem. “Javascript” is just one of the representations you’re translating your data into. Projects like Transcrypt or Brython promise to replace all your Javascript with Python. But if you do that, you still have a data path that looks like this:

SQL ➔ Python ➔ HTTP ➔ ~~JS~~ Python ➔ DOM ➔ Pixels

And you still have all the same translations, all the same frameworks, and all the same pain. It’s like moving from one front-end framework to another: it might solve a few niggles, but the big picture hasn’t changed. And this is why those projects don’t take over the world.

So what’s the answer?

Now that we’ve identified the problem, and established Javascript’s innocence, is there any hope?

Well, if the real problem is all that translation between layers, what about trying to eliminate them? We looked for a representation that would work for as many layers of the stack as possible, and decided on Python.

So we built a genuinely full-stack Python web framework, called Anvil. Yes, it runs Python in the browser, but it also uses a Python GUI toolkit instead of HTML. Server side code is also in Python, but instead of translating everything into JSON, Anvil supports making function calls from client-side Python to server-side Python. We even built a database layer, constrained so that all rows are Python objects – which means you can safely pass those objects all the way from the datastore to the UI, without changing their representation.

The result is a consistent abstraction, where you never have to translate your data into anything except Python. Of course, the abstraction can’t be perfect – we had to build escape hatches aplenty to support advanced operations. And we had to build the abstraction itself: We built the UI toolkit with Javascript, HTML and CSS; we use a Python-to-JS compiler; we built an RPC mechanism, and a database layer on top of Postgres.

The result? Instead of this:
SQL ➔ Python ➔ HTTP ➔ JS ➔ DOM ➔ Pixels
You can have this:
Python ➔ Python ➔ Python ➔ Python ➔ Python ➔ Pixels

With a simpler application model, we can do some amazing things – or, to be less grandiose, we can bring back standard features from 90s development environments. We can build a drag and drop UI designer – which is a nightmare in HTML+CSS because of the complex logic those languages can express. We can build code completion that knows about your client code, and your server code, and your UI, and your database, everywhere – which is impossible if your back-end is a black box of an HTTP API, written in a different programming language.

One of the hallmarks of a good architecture is that nice properties like this “drop out”. We think this is a productive way to think about building web applications, and we’d love you to try it out. You can use our web editor at anvil.works, or if you’re a command-line-only type you can pip install the standalone framework.

Let us know what you think - on our forum, on Reddit or Hacker News, or by dropping us a line at contact@anvil.works.

Read more

Get Started with Anvil

Create an app that collects feedback and sends it to you by email, and publish it online, in ten minutes or less.
Go

Generate PDF Invoices with Python

In this example build, see how to generate and download PDF invoices
Go

Anvil Examples: What Can You Build?

Discover full-scale apps built with Anvil. Start with a template, get inspired by an example app or learn what real users are building.
Go

Rapid Prototyping: Building Calendly in 3 Hours

How long does it take to prototype a startup? I found out by rebuilding a working version of Calendly, the invaluable meeting scheduling app.
Go

  1. Or Ruby, or Java, or even Javascript. My whole point is that the choice of language here doesn’t make much difference. ↩︎