Making the Web More Pythonic

The Web, I assert, is seriously un-Pythonic. It’s also famously a pain to program for. Are these two things related? Could we make the Web a nicer place to program, by making it more Pythonic?

Well, to start with, what do I mean by “the web is un-Pythonic”?

Think about a typical web app. You’re going to have to turn your data into a bunch of different shapes along the way:

1. Your data exists as rows in a database, accessed via SQL.
2. You then transform it into model objects on the server, with methods and attributes to interact with them.
3. Now, you have to represent these objects as JSON, and provide a whole bunch of REST endpoints for getting or manipulating them.
4. On the client, you then transform these HTTP requests into Javascript objects, which have methods of their own.
5. And then you have to transform these client objects into HTML DOM…
6. …and style that to produce pixels on a screen.

Oof.

At each of these boundaries, a bunch of boring and repetitive translation work happens. And that is an invitation to exactly the wrong sort of magic.

Let’s take an entirely unfair pot-shot at SQLAlchemy, which is a library that manages the transformation of an SQL database into Python objects (and back). It’s actually really good at it. You can even write neat query expressions like this: filter(Book.value<20).

Of course, the process of turning that Python expression into SQL is black magic. It involves metaclasses, and overloading standard Python operators to do something completely different to what they normally do.

And that’s cool, if you do it once. But if you have this amount of magic at every boundary in this stack, you’re setting yourself up for a bad time.

But of course, that’s exactly what we do:

• We have ORMs to translate databases into objects;
• We have REST frameworks to help us represent objects as JSON;
• We have JS frameworks (like Angular’s resources) to help us turn patterns of HTTP requests into Javascript objects;
• We have templating engines to represent JS objects as HTML DOM;
• And we have CSS frameworks to help us display this DOM as the right pixels.

And they’re all extremely leaky abstractions! To be a reasonably advanced user of any of these frameworks, you need to understand everything they do, on both sides of the transformation.

Aside: This is a leading cause of the “Framework of the Week” anti-pattern. You have a tool which is inherently unsatisfactory, because it’s spanning one or more of these transitions and suffers from all these impedance mismatches. And just in order to use this tool, you need to know more or less everything you need to build one yourself. This presents an irresistible temptation. But of course the new one still isn’t quite right…and around and around we go.

So, how does this situation stack up against The Zen of Python?

“There should be one obvious way to do it”?

Hoo, boy. Look at all these frameworks!

“Explicit is better than implicit”?

Transforming data implicitly is these frameworks’ job.

“If you can’t explain the implementation, it’s a bad idea”?

Again, look at the sheer amount of magic in every level of this stack!

So, what might something more Pythonic look like?

At Anvil, we start by putting Python everywhere — even in the browser. (We use the Skulpt Python-to-Javascript compiler; check out my previous talk about it.)

OK, so if we’re in Python, and making an HTTP request to a Python server, what happens?

Well, we make a function call into the requests library, and then some time later it emerges as a function call to a Flask endpoint.

Wait a second. If a function call was all we wanted, why not explicitly turn the whole thing into function calls?

So that’s what we do. Have a function on the server, decorate it with @anvil.server.callable, and call it from the client with a function call.

Now we can pass arguments (even keyword arguments), and return values from the function. We don’t have to marshal everything into an HTTP request any more — it’s just a Python function call.

(The data still passes over HTTP, of course. Actually, we use an encrypted WebSocket.)

OK, so the next question is, “What sorts of data should you be able to pass into, or return out of, these functions?”

Well, strings, numbers, dicts, and lists are easy. That’s just JSON. But we want to avoid translations, so we want to pass proper objects from the server to the client.

Unfortunately, this is a web server, serving lots of clients, so it has to be stateless: the server just can’t afford to keep these objects in RAM.

So we support passing a special sort of object: a stateless server object.

What’s in a stateless object? Well, it has an immutable ID, some method names, and some permissions. That’s all. (So there’s nothing else the server has to remember.)

We can call methods on this object from the client: it’s just a server function call, like we saw on the last slide. We make sure the client can’t call methods on arbitrary objects by signing the ID and permissions, and requiring that signature to call a method on an object. So the client can only call methods on an object that has been returned from a server function.

An excellent use case for this is…database rows! The object’s ID is the unique ID of that row in the database; the methods are things like update() and delete(). And by implementing __getitem__() and __setitem__(), we can even use square-bracket lookup to get and set column values.

If we put this together, we can:

1. Make a simple function call to the server,
2. Look up a database row,
3. Return that row to the client as a Python object,
4. …and use the values from that row in client code.

We’ve skipped a whole bunch of these translation steps! We’ve got one object, passed all the way from the database to client code.

And that’s a little contribution to making the Web more Pythonic.

For more about Anvil, check out https://anvil.works.

Thank you very much.