Git cloning app including data tables locally

[cam]

Hello!

I’m currently trying to git clone my app from the Anvil online site. I am able to successfully clone the source code with git over ssh, however all the data that is currently in the data tables doesn’t seem to come with it. There’s an option to clone the app to a new anvil-hosted instance, which seems to include the data; has anyone figured out the best way to clone a local copy of all the tables?

Thanks!

[jshaffstall]

Backup to SQLite via Uplink

Here’s my updated version of this to work with the modern (as of January 2022) anvil.yaml file format. I only use this with a Git version of the file, not with an exported one. from __future__ import annotations # standard-lib modules: import pathlib import sqlite3 as dblib import datetime import json import shutil # third-party modules: import yaml # i.e., py -m pip install pyyaml # web-service imports import anvil.server from tables import app_tables # Note: Reading a full-project .yam…

Backing up to a local SQLite db is the closest I’ve seen

And welcome to the forum!

[cam]

Thanks mate. Do you know if this preserves all the links between tables (where we use the actual ‘row’ object as a link to another table)? By preserve I mean can you effectively recreate the state within the anvil DB after backing up?

[jshaffstall]

The code linked is backup only. It does maintain links for restoring, but nobody has written the code to restore the db yet.

[p.colbert]

Restoring can be a bit tricky, if there are any saved row ids (“link” columns). The existing row ids won’t be valid anymore, and will have to be translated. The catch is, there’s no way to tell what the new row id will be; you actually have to create the corresponding record, then ask for its id.

This has to be done as a two-phase process: first, create the new records, without saving any old row ids into them, and record both the old and new row ids of each record; then, using that old-to-new-id translation table, fill in the missing row ids.

It’s also possible to store row ids in non-link columns, e.g., as strings in Text or SimpleObject columns. Anvil’s database schema knows nothing of these cross-references. If you have them (I do), then you’ll need to note, separately from Anvil’s schema, each place where the old ids can be found, and handle those cases in a similar manner.

In practice, one is often “restoring” data into an existing database, i.e, where there already is data. This opens its own can of worms, as this could come from any number of scenarios. How much of the existing data (in the receiving database) needs to be overwritten, or deleted? How do you establish a correspondence, if necessary, between existing and incoming records? If data within a SimpleObject cell (column in row) changes, can it be copied all-or-nothing, or does the cell value need to be selectively updated?

Answers will vary, case by case. And that will dictate the “restore” code we have to write.

I’m starting to think that we need something like Git for our app’s data, as well as for its code.

[nickantonaccio]

@p.colbert

I’m starting to think that we need something like Git for our app’s data, as well as for its code.

This is a useful suggestion for anvil.works. +1