Blog

Decoding Drone Flight Logs
12th July 2016

Hacking flight telemetry, because we can

I recently acquired a new toy: The amazing DJI Phantom 4 drone. Naturally, as soon as I’d mastered the art of not flying backwards into unsuspecting trees, buildings and ‘volunteers’, I started to poke around inside the software. The DJI GO app shows me all my flight logs on a map - wouldn’t it be fun to export that data and, say, display it in real-time alongside the video? Unfortunately, DJI haven’t made this terribly easy. Ah well, I like a challenge. Here’s what I came up with:



Pretty neat, really. You may want to look again to notice the live telemetry without being distracted by the wonder of the Northumberland coastline in the sun.

Finding and decoding the flight records

The DJI GO app helpfully stores all the flight logs (at least on Android) in /DJI/dji.pilot/FlightRecord/. Rather less helpfully, they are in an undocumented, proprietary binary format. Discovering this turned out to be something of a red rag to a bull, so I fired up my favourite hex editor and stared at the chaos until some sense began to emerge.

You can find all the gory details here on GitHub, but the short version is that the files contain a stream of frames, each with a header declaring the type and length. Some frames contain position information, some battery levels, others camera settings and controller positions. There’s a wealth of information, much of it recorded 10 times per second for the duration of the flight. Some data frames took more effort to decode than others - I’m particularly proud of spotting the home point altitude (a 16-bit float, recording the Pressure Altitude of the take-off point) and the battery manufacture date (a 16-bit packed DOS date, with the bytes in reverse order!).

Incidentally, there are a couple of websites that will decode these files for you (usually for a fee), but I’m not hugely enthusiastic about uploading details of exactly where and when I have ever flown to random websites. Let’s come up with something better.

Displaying decoded telemetry

Now that we’ve worked out the file format, let’s have a go at displaying the data. Anvil is a great tool for building software in Python, in the browser, and fast. If you haven’t used it before, you can sign up here.

Starting with a blank app, we add my decoder as a server module. There’s nothing magic about the Python code behind an Anvil app, so this Just Works.

Dragging and dropping a few components onto the page is all it takes to build the UI, then we add a timer to keep our telemetry updated around 10 times per second. We keep a pointer into our decoded array of flight data frames and advance through the array until we reach the current playback point of the video. Then we display the most recent altitude, heading and speed that we saw while scanning the array. It’s that simple.

Click the button below to clone the app and see the source.

Clone the app




Smart Python Code Completion
15th of February 2017

That's right. It's here.

Every coder knows the pain: You know what you need to do, but you can’t remember the name of that function, or what order it takes its parameters. It’s time to fix that problem.

We are excited to announce that from today, the Anvil web IDE includes smart code completion. Anvil knows about your forms, your components, your variables - even your data tables. Now we’re giving that information to you in real-time, as you type. Our API documentation is great, but the best docs are the ones you never have to look at. Code completion means the options are always at your fingertips.

Smart code completion is available right now - just log in and start creating!


Start Now




Python widgets in HTML layouts
5th October 2016

Speed, meet beauty.

Building a web page with drag and drop is much faster than fighting with HTML, CSS and Javascript. When we set out to build Anvil, we wanted to make it as easy to design a web app as it is to lay out a Powerpoint slide. We’ve combined drag-and-drop design with a library of prebuilt components, a built-in database and a simple Python programming environment. So far, we’re making web app development quicker and easier for people all over the world, from entrepreneurs to doctors. (Want to know more about Anvil? Read our introduction here.)

But sometimes, you need to put your best foot forward. You want to re-use your existing page design and brand assets. Or if you’re building those assets from scratch, you want pixel-perfect control over your page header. In short, you want the flexibility of traditional web design.

So, we asked: What if you could have both?

Introducing HTML templates

We’re excited to announce support for HTML templates in Anvil. You can choose from our menu of ready-to-go templates, or use your existing web design assets. Once you’ve loaded your template, development is as easy as ever: just drag and drop Anvil components into the page, and arrange them how you like. And you can drive all these components with Anvil’s straightforward Python APIs. (No Javascript-framework-of-the-week necessary.)

Try it out
Custom HTML

Anvil’s built-in templates make it easier than ever to produce a beautiful web app in record time. But if you know HTML, or have existing web assets, you can go beyond our built-in templates.

If you’re an Anvil Pro user, you can control your page down to the pixel, with any HTML or CSS you like. All you need is a couple of special attributes to tell Anvil where you can drag and drop components.

I can code my web app in Python, and employ a web designer to help me with the HTML/CSS. Anvil is without a doubt the easiest and fastest way to bring my app to life!
— Peter, Startup founder
(Yes, he actually said this)

Here’s all you need to build a drag-and-drop layout. You just need to specify where each group of components goes, and where to drag-and-drop them:

<link rel="stylesheet" href="https://anvil.works/template-assets/flower-card.css">

<div class="header" anvil-drop-slot="title">
    <div anvil-slot-repeat="title"></div>
</div>

<div class="card" anvil-slot-repeat="default" anvil-drop-here>
</div>

(Want to know how that works? Check out our documentation.)

And here’s how it looks in action:

Available now

To build an app with Anvil templates, sign up and try some of our examples, or start from scratch. We’ve got lots of video tutorials to help you out. If you’re into DIY, our reference documentation describes how to use your own HTML and CSS with Anvil.

Try it out

We’d love to hear what you think. Drop us a line at contact@anvil.works, or you can use my personal address: meredydd@anvil.works.

Usable configuration with Git
27th July 2016

The configuration dilemma

As developers, almost every app we write has configuration. Often, that configuration should really be accessible to our less technical colleagues: feature flags, rate limits, deployment signoffs, and so forth.

However, these changes also need to be tracked and audited. “The app just broke. Did someone change the config?” “Quick, revert it to last week’s settings!”

As programmers, we know exactly the right tool for this: Text files in version control. They’re diffable, trackable and comprehensible, and if anything goes badly wrong we can dive in with a text editor.

The problem comes when we present this solution to our non-technical colleagues. “So, I open the terminal? And then I type git clone and a string of gibberish you made me memorise?”

It’s tempting to give up and say, “I’ll do it for you”. Developers end up as gatekeepers, with every change coming through us.

This isn’t great either. Years ago, I used to develop SMS-based services for a mobile carrier in south-east Asia. This was the bad old days, before Twilio and friends, and the carrier had to sign off on every minor UI change – often at the very last minute. I spent many late nights waiting for a meeting on the other side of the world to finish, just so I could change one line in a config file.

GitHub API to the rescue

We can fix this. With the GitHub API, we can build an app in minutes that empowers our colleagues to change configuration on their own – with all the power of Git’s versioning and auditing.

Here’s a simple app, hosted on Heroku (source at github.com/anvil-ph-test/edit-demo). It has a configuration file (called config.json) that determines vital parameters such as the font and background colour.


Open in new tab

Here’s how I built an Anvil app to edit that configuration, with less than a dozen lines of code:

Getting the config

First, we need to grab the latest version of our config file:

self.gh_record = anvil.http.request("https://api.github.com/repos/anvil-ph-test/edit-demo/contents/config.json", json=True, username="abc", password="123")

Github returns some information about this file, and its content in base64:

{
  "name": "config.json",
  "encoding": "base64",
  "size": 67
  ...several other bits omitted...
  "content": "eyJ1cHBlcmNhc2UiOnRydWUsImZvbnQiOiJIZWx2ZXRpY2EiLCJiYWNrZ3Jv\ndW5kIjoiYmxhbmNoZWRhbG1vbmQifQ==\n",
  "sha": "bfb17ee5edf43a54f6756f032603872ca7dce320",
}

The content is what we care about:

self.data = json.loads(base64.b64decode(self.gh_record["content"]))

The decoded data looks like this:

{
  "background": "blanchedalmond",
  "font": "Helvetica",
  "uppercase": true
}

All we need now is to design our configuration interface. With data bindings, it’s all drag-and-drop - we can just specify which JSON key (in self.data) each text-box or check-box corresponds to. That’s all we need for a read-only interface:


See source code for this app
Committing our config

Now we have read-only access to our configuration, the next step is to save our changes. First, we write the contents of the form back into self.data:

self.save_data()

Now we just push it back to the server, with an HTTP PUT request to the same URL. All GitHub needs is the new content for the file, a commit message, and the previous SHA hash of this file:

new_record = {'content', base64.b64encode(json.dumps(self.data)),
              'message', 'Edit from the web'}
              'sha': self.gh_record["sha"]}

r = anvil.http.request("https://api.github.com/repos/anvil-ph-test/edit-demo/contents/config.json", method="PUT", data=new_record, json=True, username="abc", password="123")

self.gh_record = r["content"]

And here’s the working app. Why not try changing some settings?


See source code for this app

Once you’ve saved your changes, scroll up and refresh the example app. Be patient - it may take a few seconds to re-deploy with the new config.

Increased security

OK, we’re not quite done. So far, we’re doing everything on the client side, which means everyone with the URL can see our authentication information! Even if we only give that URL out to people we (mostly) trust, it’s far too easy for it to end up in the wrong hands.

Instead, we’ll do our GitHub API calls on the server side, and expose only two functions to the client: save_config and load_config. All the rest is safely on the server, where the user can’t see it:

# This code runs on the server

@anvil.server.callable
def load_config():
  gh_record = anvil.http.request("https://api.github.com/repos/anvil-ph-test/edit-demo/contents/config.json", json=True, username="abc", password="123")

  return (gh_record['sha'], json.loads(base64.b64decode(gh_record['content'])))


@anvil.server.callable
def save_config(data, last_sha):
  new_record = {'content': base64.b64encode(json.dumps(data)),
                'message': 'Edit from the web',
                'sha': last_sha}
  r = anvil.http.request("https://api.github.com/repos/anvil-ph-test/edit-demo/contents/config.json", method="PUT", data=new_record, json=True, username="abc", password="123")

  return r['content']['sha']
Copy the final app to see its full source code

(In fact, I’ve been using this version of the code for all the example apps embedded in this blog post. I’m afraid wasn’t feeling generous enough to share my GitHub authentication tokens with anyone who can View Source. Sorry to disappoint anyone who tried.)

Job done.

There you have it - a secure, functional configuration editor, ready for our non-technical colleagues to use. You don’t need to know Git to use it, but it does have full tracing and history of every change.

Conclusions

#1: We don’t have to sacrifice the benefits of Git for our configuration, just in order to get a user-friendly admin interface. We can have both!

#2: The GitHub API is awesome.

#3: Anvil lets you build useful web apps very, very quickly.



Why not clone this example app, read its source code, and try it out yourself?


Version control and Anvil
11th July 2016

Beta: Git access for your apps

Rapid development is great, and Anvil lets you build web apps amazingly fast. But sometimes you need more. You need tracking, collaboration, code review, versioning. In short, you need source control.

Today, we’re announcing beta availability of Git access for your Anvil apps. It’s simple: each Anvil app is its own Git repository. Just clone the repository, and you can pull, push, edit and merge Anvil apps right from your workstation.

Now you can collaborate on multi-person teams, manage deployment and staging environments, and integrate Anvil into your code review process.

You can sign up for the beta program from the Anvil app editor. Just select Git Access from the drop-down menu, and click the button to request access.

To learn more about Anvil, watch our video tutorial series. (Or, you can sign up and start building right away!)

Watch a tutorial




Anvil On-Site Installation
12th April 2016

Using Anvil behind your firewall

Anvil is, by default, a cloud-hosted service. This makes it incredibly easy to create web-apps that are live in the cloud, accessible from anywhere, and integrate with other cloud services.

If you’re in a corporate environment, your web app may need to access local resources. For example, you might want to use Anvil to query a database on your corporate network. For this, you will normally use the Anvil Uplink. This lets you securely give your Anvil app access to the relevant parts of your database:

The Anvil Uplink is available to all users, and does not store any data in the Anvil cloud. To learn more, watch our 4-minute video or read our documentation.


Sometimes, that's not enough.

Certain enterprise users, however, require more assurance. For example, organisations dealing with healthcare data may not transfer patient records to third-party services without special agreements.

For these users, we offer Anvil On-Site Installation. This allows you to develop and run your app entirely behind your corporate firewall, on servers you control:

An on-site Anvil installation requires no connection to the outside internet, giving you maximum assurance that your data is under your control.

Anvil On-Site Installation runs as a Docker container. It typically takes less than five minutes to get Anvil On-Site working on your network - and Anvil staff will be there to help you every step of the way.

If you want to run Anvil apps on your own network, please get in touch to find out more or arrange a free trial:

Arrange a trial




Introducing Anvil
20th February 2016

It's 2016. Making an interactive website should be easy, right?

Not so much. Let’s say you want a site to take orders for your new widget, or keep track of your customers, or schedule your local football league. Traditionally, you’ll need to know an alphabet soup of languages and technologies: HTML, CSS, JS, PHP, SQL - the list goes on. And that’s before we start on the complex frameworks required to make them usable.

This makes web development slow and complicated for professionals, difficult for other engineers, and entirely inaccessible for beginners.

We need to do better than this. So we built Anvil.

Web apps made simple

Anvil is a tool for making interactive websites in Python. Build your site with drag and drop, placing text, buttons, input boxes, images and more. Then double-click a button and write the Python that executes when that button is clicked.

You can make something really quick this way. Watch us build a page that greets you by name, in 45 seconds flat:

Want to try this yourself? Check out our tutorials


Simple built-in database

Anvil’s built-in database has a simple, spreadsheet-like interface for editing your data. Searching or editing it from your code is a no-nonsense Python statement. You can build a working database-backed to-do list app in five minutes - watch us do it!

If you already have a database, no problem - Anvil can connect to that too. (For those with special requirements, we even offer an on-site solution)

Use the rest of the web

No app is an island, and you shouldn’t have to build things from scratch. Anvil makes it easy for your apps to use services from the rest of the web:

Authenticate with Google login
Video
Accept credit cards with Stripe
Video
Store files in Google Drive
Video
Store data in Google Sheets
Tutorial
Use local resources

You might want to use something that’s only available on your network, or your computer. Perhaps you want to use your company database, or special hardware, or files stored on your computer.

With Anvil, that’s a snap. Just import a library, mark the functions you want to call from Anvil, and away you go.

Watch us control a Raspberry Pi from the web in three minutes.


Try it out

Anvil is free for personal use, and we can’t wait to see what you will build with it. Why not sign up for free and try it out?


Build an app




Learn More

Get the best out of Anvil with our free email course.