When you need to escape an abstraction, how violent is your exit?

Abstraction is essential for development. But what happens when it goes wrong?

We live in a golden age of abstraction. When I write a program, I don’t have to use physical switches to enter machine code – I write Python in a comfortable text editor. I’m not even writing this post in HTML – I’m using Markdown.

All of development is built on abstractions like these. Very early in the history of computing, we recognised that building abstractions was the only way to get anything done at all. And very soon after that, we discovered that abstraction comes with dangers:

Every problem in computer science can be solved by another layer of abstraction.
– David Wheeler

…except for the problem of too many layers of abstraction.
– Anon

Today, I’m talking about one of those dangers - and it’s one that the creators of abstractions, in love with their creations, don’t think about enough. The question is: What happens when you hit the limits of your abstraction?

Perhaps an example will help:

Option 1: Go down with the ship

Let’s say it’s 1999, you’ve written a Windows desktop application, and now someone is telling you that it needs to be available on the web. Too bad – Windows apps can’t do that. You’re going down with the ship.

Your only choices are to rewrite your app on a new platform, or sit tight and hope that feature isn’t that critical after all. Often it really is that critical: Many products died at the turn of the millennium, left behind by the Web.

My condolences.

Option 2: The Ejector Seat

If you’re lucky, your abstraction is more helpful than that.

Example: Create-react-app is a neat way to start a web app with React. It’s a black box that replaces your build manager, a Javascript transpiler, a minifier, and all the other build tools you need for modern web development.

Of course, sooner or later you’ll hit the limits of what create-react-app can do: you’ll need something it doesn’t support. The authors have thought of this situation, and there’s a button you can push: EJECT.

Ejecting from create-react-app opens up the black box. It writes out a set of build scripts representing the current state of your app. Now you can edit those scripts yourself – the hard way.

But create-react-app has given up. It protected you for a while – but as soon as you needed one advanced feature, you’ve been dropped in the Mojave desert with a parachute, a first-aid kit, and a copy of the Webpack manual.

Good luck!

It’s better than going down with the ship, of course. But should you have to discard an abstraction entirely when you touch its limits?

Option 3: The Escape Hatch

It’s much more helpful to acknowledge that your system will always be incomplete, and make it usable even then. Ideally, your users can use complex external tools for the tricky corners of their project, without making the other 95% of their work more complicated. I call this pattern an “escape hatch”, rather than an “ejector seat” – you don’t need to jump all the way out at once.

The Rust programming language is a great example. Rust is a memory-safe low-level language: The compiler enforces rules that make sure your code can’t access memory it shouldn’t. But there are some things you can’t express within those rules. So Rust has an escape hatch, called unsafe. You mark a block of code unsafe, and you can break those rules. A few lines later, you return to writing the rest of your app – with compiler-guaranteed memory safety.

Anvil is an abstraction over all of web development. We provide a visual UI designer (no HTML/CSS), and you write your front-end and back-end code in Python (no JS, no REST frameworks). It’s even got an optional Python-native database (no SQL). This enables a data scientist, a sysadmin, or a developer in a hurry to build full-stack web apps with nothing but Python.

This is a pretty ambitious goal: The Web is a huge platform, with a large set of existing libraries, and the browser vendors add new features all the time. Let’s face it: Our new abstraction isn’t going to cover it all, any time soon.

We could have provided an ejector seat: “Export this Anvil app as a React+Flask app”. This, bluntly, would suck. The moment you edited the generated code, you’d lose all the benefits of Anvil’s abstractions!

So instead, we built escape hatches into every part of Anvil. For example:

• If you want finer control over appearance, you can leave the drag-and-drop designer and edit the HTML template it uses. And then… flip right back to the drag-and-drop designer, where you can drop components into your new page layout.

• If you need a browser feature we don’t support yet, you’ll need to use Javascript. You can call from Python to Javascript and back again. And then… wrap that code up as a Python component your colleagues can use. (Here’s an example using the Web Location API.)

• If you want to use HTTP after all, that’s fine! You can consume HTTP APIs or make your own.

• If you want to run code outside our serverless environment, it’s easy. Just install our Uplink library on your server, and run your code there. And then… call that code from your app, exactly the same as calling serverless code. Your server code can use all of Anvil’s server-side APIs. (Here’s an example with a Raspberry Pi.) Of course, if you need everything on site, you can get an on-site installation.

• If you want to edit your code outside Anvil, just check it out in Git! Every Anvil app is a Git repository. Pull, make some changes, push it back. You can automate this, and integrate with whatever code review or CI/CD system you like. And then… jump right back into Anvil, and edit your app there.

The web is a complex platform – often too complex – but every single feature is there for a reason.

Are you building an abstraction? If so, think about your escape hatches: How can your users exploit features of the underlying platform, without ejecting into the wild blue yonder?