Best Version Control Options

So I’m curious how people here with production grade apps deal with having say a test/prod version of the app especially if you have a database, etc. I’m trying to think of the best way to add updates/enhancements to the site without screwing up the existing site, not sure if I should just clone the app and then update it that way and only bring back over modules and not the db or what.

Curious how others approach this.

Thanks

1 Like

Search the Forum for the term “Publish” or “Published”. You’ll see that you can “freeze” the app (as seen by users) while still working on improvements, step by step.

This leverages Anvil’s use of the Git version control system. Each app has its own independent repository of files, with a history of prior versions. You can mark any version as “Published”, and that’s what end-users will see. (Only one version can be Published at a time.) Meanwhile, you can continue to improve the code, creating new Development versions step by step.

As I understand it, this use of Git applies only to the app’s code, user interface, permissions, and services. Changes to your database structure (tables and columns) and content (rows) take place immediately, and are visible to all versions of the app, including the Published version.

Thus, if you need to make backwards-incompatible changes to database structure, then you’ll need to make them in a clone of the original app. As a distinct app, the clone can have its own, distinct set of tables, columns, and rows.

After you’ve done this a few times, you may discover a stable set of subroutines and/or data, things that absolutely don’t (or shouldn’t) change from one app to the next. You may then want to make a “library” out of them, usable from all the apps. This gives you a single point of maintenance for these routines.

Again, this applies only to code and user interface. Library code does not inherit the library app’s tables, permissions, or services. Those must be provided by whatever app uses the library.

2 Likes

See also:


One more thing to say: If you’re using the same code in different environments, you can make two apps (your “development app” and your “production app”), and use Git on your machine to pull code from one and push to the other. That way, you have separate environments with separate data tables.

(Note: Beware that this will break App Secrets. By design, an App Secret created in one app cannot be accessed by another app. However, you can access your dependencies’ App Secrets, so if you want secrets that are available in both environments, put them in a dependency they both share.)

2 Likes

I attempted to implement this idea: performing testing in a separate Anvil app and using git to move code between them, but I found it very awkward at first. In the process of writing this post, though, I came to see a more sensible workflow.

Maybe other Git noobs will find it helpful to learn from the mistakes, so I kept my initial draft here.

Here is what I did in my first attempt:

  1. Clone with Git my “production app” onto my local machine.
  2. Create a clone of my production app within Anvil to serve as the “development app.” (This copies only the files over to the new app, not the internal Git repository, so the version history is lost.)
  3. Test and update the development app in the Anvil IDE, committing changes as I go.
  4. Fetch the development app to my local production app Git repository from step #1.
  5. Graft the development app commits onto the tip of my local production app repository (using “git replace --graft”)
  6. Make the graft permanent using “git filter-branch,” which rewrites the development app commit ids. (I can no longer push back to the Anvil version of the development app, I think, so it will be discarded.)
  7. Fast-forward merge this development branch into master.
  8. Undo the changes to anvil.yaml (which otherwise causes an error like “cannot edit these data tables,” tracing from users.py)
  9. Push to origin, the production app on Anvil.

A post I found subsequently seems to suggest this (less involved but still clunky) alternative, after steps 1 and 2 as above:
3. Clone with Git my development app to a separate local repository.
4. For each commit to the development app, pull it down to my local development repository, copy the changed files to my local production app directory and commit the change to my local production app repository.
5. Repeat step 4 for each commit.
6. When ready to make a change to the live production app, push my local repository up to Anvil.

In a different place, meredydd speaks of “force-push,” which suggests:

  1. Clone with Git my “production app” onto my local machine.
  2. Create a clone of my production app within Anvil (via this) to serve as the “development app.” (This copies only the files over to the new app, not the internal Git repository, so the version history is lost.)
  3. Force push my local production app repository to the development app on Anvil, so it will have the full repository there now, restoring the version history. (This step is needed for step 4 to work smoothly. Caveat: I’m not clear on how the anvil.yaml issue discussed in the thread linked at top is resolved.)
  4. a) do code editing and commits locally, pushing to the development app on Anvil for testing; or b) edit the development app in the Anvil IDE, pulling new commits down to my local repository.
  5. Push the local repository to the production app on Anvil when ready.
1 Like

That sounds like a good workflow. Once you’ve done the first 3 steps, it’s just

  • Push and pull as appropriate to sync local with cloud (step 4)
  • Push to Prod when you want to deploy (step 5)

And you could automate step 4 with a Bash script…

Here's a start on that
#!/bin/bash
##
# Anvil local repository watcher.
# Syncs with the cloud version of the app.
# Just commits; you need to add a push and a pull, but it's a start.
# Also, if you modify both the local and cloud version during the watch
# period, you might get merge conflicts. So don't do that.
#

# Where the app is checked out
APP_DIR="$1"
# The message to use for the autosave commits
AUTOSAVE_MESSAGE='Local autosave'
# How many seconds to wait between autosaves
WATCH_PERIOD=5

cd "$APP_DIR"

while true; do
  LAST_COMMIT_MESSAGE="$(git log -1 --pretty=%B)"
  if [[ "$LAST_COMMIT_MESSAGE" == "$AUTOSAVE_MESSAGE" ]] ; then
    # If the latest commit is an autosave, overwrite it.
    git commit -a --amend -m "$AUTOSAVE_MESSAGE"
  else
    # If the latest commit was done manually, make a new autosave commit.
    git commit -a -m "$AUTOSAVE_MESSAGE"
  fi
  sleep $WATCH_PERIOD
  echo "."
done

cd -
1 Like

(how did you do that expanding code block?)

1 Like
[details="Isn't this cool?"]
Your words go here.
[/details]
Isn't this cool?

Your words go here.

I wouldn’t use it for anything crucial though, because it’s easy to mistake it for a static triangle next to some static text!

Thanks to @hugetim for popularising that tag!

5 Likes

We use the two app approach for development and production, but we don’t use any fancy git commands.

On your step 2 you mention the loss of history as a problem, we consider it a feature!

Initial setup

  1. Clone the app in Anvil
  2. Call the app with the full history App Dev and the new one App Prod
  3. Git clone both the apps to your computer. At this point you have two apps with the same files but with different history
  4. On one of the two apps delete the tables that need to be identical in development and in production and add them again as shared tables

Everyday workflow

  1. Work on App Dev either on the browser or on the desktop, creating dozens of commits, most of them just noise automatically committed by the online IDE
  2. When App Dev works pull it to my local repository
  3. Copy the files from App Dev to App Prod (manually or with a batch script)
  4. Commit and push App Prod

We never use git commands directly, we only use PyCharm pull and push buttons (and create change lists and commit them inside PyCharm too).

At this point the history of App Prod is clean and only shows the intended commits.

Access to tables via Anvil ‘app_tables’ will work seamlessly as long as the apps either share the same tables or use different tables with the same name. Access via SQL requires a line at the beginning of the module to set the correct table name.

9 Likes