Blog

Build a Business Dashboard With Python
22nd of August 2017

You make what you measure.
— Joe Kraus
Dashboards in Python: Visualise Your Performance

Business is like driving - what you’re looking at is what you steer towards. A dashboard is a way to make sure you’re looking at the most important parts of your business. The good news is that they’re quite straightforward to build – especially if you have Python to help you.

I’m going to put the goodies up front, for the geeky and impatient. This is a video of me building a dashboard that pulls data from a business database, and displays it on the web. Watch it now, or read on as we take a step-by-step guide to creating your own business dashboard.

See the code

So, how do I do it?

Step 1: Ask the right question

This is the most important part of measuring your business. You have to ask yourself what information is so important, you should see it every morning.

When you’re building a business, the most important questions aren’t always obvious. There are some great guides out there. I’d recommend Startup Metrics for Pirates (not just for start-ups!), and Adam D’Angelo’s talk on measurement from Y Combinator’s Startup School.

Broadly, though, there are two sorts of questions: Strategic metrics that show you where you want to go, like user acquisition rate, retention, or revenue growth; and tactical metrics that monitor how you’re getting there, such as A/B tests and short-lived marketing initiatives. At very least, you should have a dashboard that shows you your strategic metrics, and how they compare to historical performance.

Don't be afraid of hard questions.

Sometimes, the answer you need is right there in your database, and all you need is to query it. It's great when that happens, and we'll cover this simple case in our walkthrough.

But it's often more complicated than that: Perhaps you'll need data from your application database, your CRM system and an external analytics provider. Because we're using Python to compile the data for our dashboard, we have the flexibility to do that. So you have no excuse: Don't measure the wrong thing, just because it's easier.

Step 2: Get the answer from your database

Let’s imagine we’ve thought about it, and decided that our primary business concern is acquisition: How many new users are we signing up, and how is that changing from week to week?

For 99% of online businesses, this information will be in an SQL database somewhere. Connect with your command-line tool of choice, and write your query. Our example table looks like this, using Postgres:

myapp=> \d users
                 Table "public.users"
   Column    |            Type             | Modifiers                      
-------------+-----------------------------+-----------
 id          | integer                     | 
 email       | text                        | 
 signup_date | timestamp without time zone | 

A little trial and error, and we have a query that gives us the number of user sign-ups by week, for the last three months:

myapp=> SELECT COUNT(*), DATE_TRUNC('week', signup_date) AS d
             FROM users
             WHERE signup_date > NOW() - INTERVAL '3 months'
             GROUP BY DATE_TRUNC('week', signup_date)
             ORDER BY d;
Step 3: Put it online

We want to make this data visible on a web page. Anvil lets us create web apps with nothing but Python. We’re going to create an Anvil app: It will have a server module that uses the standard Python tools to extract the data from our database, and a client-side page to display it on the web (also in pure Python).

To connect to a Postgres database, we use the standard Psycopg2 library. We create a server module and write:

import psycopg2

conn = psycopg2.connect("host=db.myapp.com dbname=my_app user=postgres password=secret")

Now, we want to run that SQL query on demand. We’ll define a function that gets our data and returns it as a list:

@anvil.server.callable
def get_user_signups():
    cur = conn.cursor()
    cur.execute("""
        SELECT COUNT(*), DATE_TRUNC('week', signup_date) AS d
             FROM users
             WHERE signup_date > NOW() - INTERVAL '3 months'
             GROUP BY DATE_TRUNC('week', signup_date)
             ORDER BY d;
    """)
    return list(cur)

We’ve marked the function @anvil.server.callable. That’s all we need to make it accessible from client-side code when we build our web page. (No web server required!)

Calculating complex metrics

Remember earlier, when we said you probably want more than just a user count? This is where Python shines.

Cross-reference with a NoSQL database. Query your CRM via its API. Run statistical models with NumPy. Python is the swiss army knife of data analysis – why wouldn't you use it for your dashboard?

Step 4: Draw the plot

All that’s left is to display this data as a graph in our web app. We open Anvil’s visual designer and add a Plot component to our app’s main page. When the app opens, we call the query function we’ve just defined, and construct a line graph with the data:

signups = anvil.server.call('get_user_signups')

# Anvil plots use the Plot.ly API: https://plot.ly/python/#basic-charts
scatter = go.Scatter(x = [signup_time for (count,signup_time) in signups],
                     y = [count for (count,signup_time) in signups],
                     fill = 'tozeroy')

We want that code to run when the page first opens, so we put it in the __init__ method of our form. (We also import the plot API.)

Here’s the entire page source code, including the parts Anvil provides for you:

from plotly import graph_objs as go

class Form1(Form1):
  def __init__(self, **properties):
    self.init_components(**properties)

    # This code will run when the form opens.
    signups = anvil.server.call('get_user_signups')

    # Make a line plot of this data
    scatter = go.Scatter(x = [signup_time for (count,signup_time) in signups],
                         y = [count for (count,signup_time) in signups],
                         fill = 'tozeroy')

    # Display that plot on our page
    self.plot_1.data = scatter

That’s it! Here’s what it looks like, now we’re finished:

Screen-shot of the finished app

See source code in the Anvil editor

Step 5: Actually watch it

It feels silly to say it, but I know from personal experience: A dashboard you don’t look at is as bad as no dashboard at all. It’s actually worse, because knowing it’s there gives you a false sense of security.

Set your dashboard as your home page - or if you have a spare screen, display it in the corner of your office. And then, when it tells you something interesting, you might actually react!



I hope you’ve found this guide helpful. You can follow us on Twitter, or email me any questions or comments at meredydd@anvil.works. Finally, you can see the full source code for this example here:

See the code

Interactive plots for your apps
26th of July 2017

New: Graphs and charts for your Anvil app

Python is the world’s favourite language for data processing and visualisation – and when you use Anvil, Python is all you need to build web apps. Today, we’re making it even easier to present your data on the web.

You can now offer interactive charts right inside your Anvil apps, built and customised in just a few lines of code. The new Plot component is powered by the Plotly Python API, so you have a wealth of examples already online to get you started.

Plots like this are simple to create, and simple to customise:

You can open this example in Anvil right now, or read the docs to learn more.

You can also find more examples in the Plotly library docs .


Try this demo now




HTTP endpoints for your apps
17th of July 2017

New: Make an API for your Anvil app

Sometimes we want to connect other programs with our Anvil apps. Whether it’s a native mobile app, a shell script, or a web service like Twilio, most things expect to make REST or plain HTTP requests.

As of today, this is all it takes to publish an HTTP API that returns some JSON:

@anvil.server.http_endpoint("/greet/:name")
def greet(name, **qs):
  return {'message': 'Hello, ' + name + '!'}

We can demonstrate it with the curl command-line program:

$ curl https://http-endpoint-demo.anvilapp.net/_/api/greet/Joanne
{"message":"Hello, Joanne!"}

You can open this example in Anvil right now, or read the docs to learn more.


Try this demo now




Make the world better? Remove some Javascript.
27th June 2017

Compiling Python to Javascript

To write full-stack web apps in nothing but Python, you need to run Python code in the browser. Watch my talk at PyCon 2017, or scroll down for a transcript:

For more information about Skulpt, check out skulpt.org.

Or you can check out the pull request that implemented these changes.

Transcript:

Back when my friend Ian and I were idealistic students, we sat down and decided to make the world a better place, as you do. And we decided that the best way two developers could make the world a better place was to remove some Javascript from it.

The web is the greatest application delivery platform on Earth, and frankly it’s a stain on our profession that you need to know five different programming languages and three different frameworks if you want to make it do anything. Really, all you should need is to be able to code - ideally, in a friendly, readable language that’s easy to learn and isn’t full of unpleasant surprises.

There was an obvious candidate.

So, we built a tool for building full-stack web applications with nothing but Python.

We built a graphical interface builder for designing your pages, and let you write your client-side code in Python - so all the buttons, text fields etc you put on your page are just Python objects. We built a back-end server environment, also in Python, and a database with a pure Python API, all deployed in the cloud in one click.

But today I want to talk about this front-end code.

If you want to drive the items on your web page as pure Python objects, you’re going to need to run Python in the browser. If you’re going to run something in the browser, it’s going to have to be in Javascript.

Thankfully, there’s a great open-source project called Skulpt, that compiles Python to Javascript. So you can write Python to drive your app, and we can compile it to Javascript so that when you publish it online, your code actually runs in a user’s browser. We give Skulpt Python objects, it represents them as Javascript objects. We give it a Python function, and it produces a Javascript function for the browser to run.

There is a problem, though. Javascript, in its infinite wisdom, is 100% non-blocking. If you kick off a process that finishes later, you’d better provide a callback.

Here’s some code you might write in Python. Go get a record from the database, and if it’s not there, throw an error.

And here’s how you’d do it in Javascript. I count three separate callbacks here. I’m not exaggerating - this is an example straight from the documentation of the postgres library!

OK, so we have a Python-to-Javascript compiler, and it’s open source. What if we could turn this into this, automatically?

So, we go "git clone", and start exploring. It turns out that Skulpt has a classic compiler architecture, modeled quite closely on CPython: There’s a parser that turns Python code into a parse tree (these are all the files in the Skulpt source tree, by the way, if you want to follow along).
Then it walks over this parse tree and makes an abstract syntax tree, which is made of elements you might recognise - like variable assignments, function definitions, function calls, and so on. Then there’s a compiler that then walks this syntax tree and spits out Javascript.

So, this is what we need to modify to implement blocking code. We invent a new Javascript class that a function can return, to say “hey, I’m returning, but I’m not done yet; I’ve just blocked”. We call this a suspension.

So, when we’re compiling the AST, wherever we’re compiling a function call, we generate Javascript code that calls the function, and looks at the return value, and says, "is it a Suspension?". Because if it returned a Suspension, we’re blocking - and I'm going to have to suspend myself. We save all our variables and temporary values, our exception state, and where we are in the function, and then we return our own suspension with all that information in it.

And so our caller will recognise that we have suspended, and so it's going to have to suspend as well, and so on all the way back to the Javascript runtime.

When that operation completes - say the database gives us a response - we resume the Suspension. We call our function with the suspension, which then restores all our variables, jumps to the right place and resumes the original suspension, which restores all of its variables, and so on, until we can resume execution as normal.

So, that’s how we take simple blocking Python, and compile it into non-blocking Javascript so you don’t have to.

This has been a quick overview; if you’re interested you can check out Skulpt at skulpt.org - I’m one of the maintainers now, and we’re always looking for new contributors.

And if you're fed up to the back teeth with all this Javascript, and you want to forget it all and write full-stack web apps with nothing but Python, please check us out at www.anvil.works.

Thank you very much!




Smart Python Code Completion
15th of February 2017

That's right. It's here.

Every coder knows the pain: You know what you need to do, but you can’t remember the name of that function, or what order it takes its parameters. It’s time to fix that problem.

We are excited to announce that from today, the Anvil web IDE includes smart code completion. Anvil knows about your forms, your components, your variables - even your data tables. Now we’re giving that information to you in real-time, as you type. Our API documentation is great, but the best docs are the ones you never have to look at. Code completion means the options are always at your fingertips.

Smart code completion is available right now - just log in and start creating!


Start Now




Python widgets in HTML layouts
5th October 2016

Speed, meet beauty.

Building a web page with drag and drop is much faster than fighting with HTML, CSS and Javascript. When we set out to build Anvil, we wanted to make it as easy to design a web app as it is to lay out a Powerpoint slide. We’ve combined drag-and-drop design with a library of prebuilt components, a built-in database and a simple Python programming environment. So far, we’re making web app development quicker and easier for people all over the world, from entrepreneurs to doctors. (Want to know more about Anvil? Read our introduction here.)

But sometimes, you need to put your best foot forward. You want to re-use your existing page design and brand assets. Or if you’re building those assets from scratch, you want pixel-perfect control over your page header. In short, you want the flexibility of traditional web design.

So, we asked: What if you could have both?

Introducing HTML templates

We’re excited to announce support for HTML templates in Anvil. You can choose from our menu of ready-to-go templates, or use your existing web design assets. Once you’ve loaded your template, development is as easy as ever: just drag and drop Anvil components into the page, and arrange them how you like. And you can drive all these components with Anvil’s straightforward Python APIs. (No Javascript-framework-of-the-week necessary.)

Try it out
Custom HTML

Anvil’s built-in templates make it easier than ever to produce a beautiful web app in record time. But if you know HTML, or have existing web assets, you can go beyond our built-in templates.

If you’re an Anvil Pro user, you can control your page down to the pixel, with any HTML or CSS you like. All you need is a couple of special attributes to tell Anvil where you can drag and drop components.

I can code my web app in Python, and employ a web designer to help me with the HTML/CSS. Anvil is without a doubt the easiest and fastest way to bring my app to life!
— Peter, Startup founder
(Yes, he actually said this)

Here’s all you need to build a drag-and-drop layout. You just need to specify where each group of components goes, and where to drag-and-drop them:

<link rel="stylesheet" href="https://anvil.works/template-assets/flower-card.css">

<div class="header" anvil-drop-slot="title">
    <div anvil-slot-repeat="title"></div>
</div>

<div class="card" anvil-slot-repeat="default" anvil-drop-here>
</div>

(Want to know how that works? Check out our documentation.)

And here’s how it looks in action:

Available now

To build an app with Anvil templates, sign up and try some of our examples, or start from scratch. We’ve got lots of video tutorials to help you out. If you’re into DIY, our reference documentation describes how to use your own HTML and CSS with Anvil.

Try it out

We’d love to hear what you think. Drop us a line at contact@anvil.works, or you can use my personal address: meredydd@anvil.works.

Usable configuration with Git
27th July 2016

The configuration dilemma

As developers, almost every app we write has configuration. Often, that configuration should really be accessible to our less technical colleagues: feature flags, rate limits, deployment signoffs, and so forth.

However, these changes also need to be tracked and audited. “The app just broke. Did someone change the config?” “Quick, revert it to last week’s settings!”

As programmers, we know exactly the right tool for this: Text files in version control. They’re diffable, trackable and comprehensible, and if anything goes badly wrong we can dive in with a text editor.

The problem comes when we present this solution to our non-technical colleagues. “So, I open the terminal? And then I type git clone and a string of gibberish you made me memorise?”

It’s tempting to give up and say, “I’ll do it for you”. Developers end up as gatekeepers, with every change coming through us.

This isn’t great either. Years ago, I used to develop SMS-based services for a mobile carrier in south-east Asia. This was the bad old days, before Twilio and friends, and the carrier had to sign off on every minor UI change – often at the very last minute. I spent many late nights waiting for a meeting on the other side of the world to finish, just so I could change one line in a config file.

GitHub API to the rescue

We can fix this. With the GitHub API, we can build an app in minutes that empowers our colleagues to change configuration on their own – with all the power of Git’s versioning and auditing.

Here’s a simple app, hosted on Heroku (source at github.com/anvil-ph-test/edit-demo). It has a configuration file (called config.json) that determines vital parameters such as the font and background colour.


Open in new tab

Here’s how I built an Anvil app to edit that configuration, with less than a dozen lines of code:

Getting the config

First, we need to grab the latest version of our config file:

self.gh_record = anvil.http.request("https://api.github.com/repos/anvil-ph-test/edit-demo/contents/config.json", json=True, username="abc", password="123")

Github returns some information about this file, and its content in base64:

{
  "name": "config.json",
  "encoding": "base64",
  "size": 67
  ...several other bits omitted...
  "content": "eyJ1cHBlcmNhc2UiOnRydWUsImZvbnQiOiJIZWx2ZXRpY2EiLCJiYWNrZ3Jv\ndW5kIjoiYmxhbmNoZWRhbG1vbmQifQ==\n",
  "sha": "bfb17ee5edf43a54f6756f032603872ca7dce320",
}

The content is what we care about:

self.data = json.loads(base64.b64decode(self.gh_record["content"]))

The decoded data looks like this:

{
  "background": "blanchedalmond",
  "font": "Helvetica",
  "uppercase": true
}

All we need now is to design our configuration interface. With data bindings, it’s all drag-and-drop - we can just specify which JSON key (in self.data) each text-box or check-box corresponds to. That’s all we need for a read-only interface:


See source code for this app
Committing our config

Now we have read-only access to our configuration, the next step is to save our changes. First, we write the contents of the form back into self.data:

self.save_data()

Now we just push it back to the server, with an HTTP PUT request to the same URL. All GitHub needs is the new content for the file, a commit message, and the previous SHA hash of this file:

new_record = {'content', base64.b64encode(json.dumps(self.data)),
              'message', 'Edit from the web'}
              'sha': self.gh_record["sha"]}

r = anvil.http.request("https://api.github.com/repos/anvil-ph-test/edit-demo/contents/config.json", method="PUT", data=new_record, json=True, username="abc", password="123")

self.gh_record = r["content"]

And here’s the working app. Why not try changing some settings?


See source code for this app

Once you’ve saved your changes, scroll up and refresh the example app. Be patient - it may take a few seconds to re-deploy with the new config.

Increased security

OK, we’re not quite done. So far, we’re doing everything on the client side, which means everyone with the URL can see our authentication information! Even if we only give that URL out to people we (mostly) trust, it’s far too easy for it to end up in the wrong hands.

Instead, we’ll do our GitHub API calls on the server side, and expose only two functions to the client: save_config and load_config. All the rest is safely on the server, where the user can’t see it:

# This code runs on the server

@anvil.server.callable
def load_config():
  gh_record = anvil.http.request("https://api.github.com/repos/anvil-ph-test/edit-demo/contents/config.json", json=True, username="abc", password="123")

  return (gh_record['sha'], json.loads(base64.b64decode(gh_record['content'])))


@anvil.server.callable
def save_config(data, last_sha):
  new_record = {'content': base64.b64encode(json.dumps(data)),
                'message': 'Edit from the web',
                'sha': last_sha}
  r = anvil.http.request("https://api.github.com/repos/anvil-ph-test/edit-demo/contents/config.json", method="PUT", data=new_record, json=True, username="abc", password="123")

  return r['content']['sha']
Copy the final app to see its full source code

(In fact, I’ve been using this version of the code for all the example apps embedded in this blog post. I’m afraid wasn’t feeling generous enough to share my GitHub authentication tokens with anyone who can View Source. Sorry to disappoint anyone who tried.)

Job done.

There you have it - a secure, functional configuration editor, ready for our non-technical colleagues to use. You don’t need to know Git to use it, but it does have full tracing and history of every change.

Conclusions

#1: We don’t have to sacrifice the benefits of Git for our configuration, just in order to get a user-friendly admin interface. We can have both!

#2: The GitHub API is awesome.

#3: Anvil lets you build useful web apps very, very quickly.



Why not clone this example app, read its source code, and try it out yourself?


Version control and Anvil
11th July 2016

Beta: Git access for your apps

Rapid development is great, and Anvil lets you build web apps amazingly fast. But sometimes you need more. You need tracking, collaboration, code review, versioning. In short, you need source control.

Today, we’re announcing beta availability of Git access for your Anvil apps. It’s simple: each Anvil app is its own Git repository. Just clone the repository, and you can pull, push, edit and merge Anvil apps right from your workstation.

Now you can collaborate on multi-person teams, manage deployment and staging environments, and integrate Anvil into your code review process.

You can sign up for the beta program from the Anvil app editor. Just select Git Access from the drop-down menu, and click the button to request access.

To learn more about Anvil, watch our video tutorial series. (Or, you can sign up and start building right away!)

Watch a tutorial




Anvil On-Site Installation
12th April 2016

Using Anvil behind your firewall

Anvil is, by default, a cloud-hosted service. This makes it incredibly easy to create web-apps that are live in the cloud, accessible from anywhere, and integrate with other cloud services.

If you’re in a corporate environment, your web app may need to access local resources. For example, you might want to use Anvil to query a database on your corporate network. For this, you will normally use the Anvil Uplink. This lets you securely give your Anvil app access to the relevant parts of your database:

The Anvil Uplink is available to all users, and does not store any data in the Anvil cloud. To learn more, watch our 4-minute video or read our documentation.


Sometimes, that's not enough.

Certain enterprise users, however, require more assurance. For example, organisations dealing with healthcare data may not transfer patient records to third-party services without special agreements.

For these users, we offer Anvil On-Site Installation. This allows you to develop and run your app entirely behind your corporate firewall, on servers you control:

An on-site Anvil installation requires no connection to the outside internet, giving you maximum assurance that your data is under your control.

Anvil On-Site Installation runs as a Docker container. It typically takes less than five minutes to get Anvil On-Site working on your network - and Anvil staff will be there to help you every step of the way.

If you want to run Anvil apps on your own network, please get in touch to find out more or arrange a free trial:

Arrange a trial




Introducing Anvil
20th February 2016

It's 2016. Making an interactive website should be easy, right?

Not so much. Let’s say you want a site to take orders for your new widget, or keep track of your customers, or schedule your local football league. Traditionally, you’ll need to know an alphabet soup of languages and technologies: HTML, CSS, JS, PHP, SQL - the list goes on. And that’s before we start on the complex frameworks required to make them usable.

This makes web development slow and complicated for professionals, difficult for other engineers, and entirely inaccessible for beginners.

We need to do better than this. So we built Anvil.

Web apps made simple

Anvil is a tool for making interactive websites in Python. Build your site with drag and drop, placing text, buttons, input boxes, images and more. Then double-click a button and write the Python that executes when that button is clicked.

You can make something really quick this way. Watch us build a page that greets you by name, in 45 seconds flat:

Want to try this yourself? Check out our tutorials


Simple built-in database

Anvil’s built-in database has a simple, spreadsheet-like interface for editing your data. Searching or editing it from your code is a no-nonsense Python statement. You can build a working database-backed to-do list app in five minutes - watch us do it!

If you already have a database, no problem - Anvil can connect to that too. (For those with special requirements, we even offer an on-site solution)

Use the rest of the web

No app is an island, and you shouldn’t have to build things from scratch. Anvil makes it easy for your apps to use services from the rest of the web:

Authenticate with Google login
Video
Accept credit cards with Stripe
Video
Store files in Google Drive
Video
Store data in Google Sheets
Tutorial
Use local resources

You might want to use something that’s only available on your network, or your computer. Perhaps you want to use your company database, or special hardware, or files stored on your computer.

With Anvil, that’s a snap. Just import a library, mark the functions you want to call from Anvil, and away you go.

Watch us control a Raspberry Pi from the web in three minutes.


Try it out

Anvil is free for personal use, and we can’t wait to see what you will build with it. Why not sign up for free and try it out?


Build an app




Learn More

Get the best out of Anvil with our free email course.