Local database management clarification

I built an app on Anvil parapeet and I would like to self host it on my home PC as an experiment.

What I’ve tried
The data used in the app comes from the database with multiple tables, including user accounts.
I git cloned the app and installed the app server. It works to launch the app, but once I try to get data from the database it says it is empty.

What I’m trying to do:
View the contents of my database and read write to it in my personal IDE on my machine.

Is this doable and what tools would I need?

This might help:

Or this, after you download your tables to csv on your local machine you might use this code:

Either way, what I am suggesting is using some sort of backup to download your tables data, then loading it to the data tables system located in your local self-hosted anvil environment.

4 Likes

Thank you for this answer, but I am finding it hard to understand how this can help me see and modify the database contents of a Git cloned Anvil project.

There is no standard method used to move data from an app hosted on anvil.works servers, to your self-hosted installation. What ianbuywise has linked are some well known solutions devised by the community for backing up and transporting table data. In some cases, exporting CSV data files manually from the visual table editor, and then importing with pandas can be useful, as introduced here: Anvil Docs | CSV and Excel import. Importing CSV can get messy, with lots of value transformations needed during the import routine. Some users choose to connect apps with uplink or API code, and perform batch transfers over the network. Some users move data to a third party database, as in the Sqlite example linked above. I prefer exporting and importing JSON.

No matter what, you’re going to need to craft a custom migration solution to handle your particular data table schema, especially for linked columns.

I often treat anvil.works hosted apps as a ‘development’ server version of the app. Only dummy data is ever saved to the development server tables - the development server is just used to demonstrate functionality before being installed on the production anvil-app-server. Production data is only ever entered into the production app. That way, production data never needs to be migrated from the anvil hosted environment (that’s my workflow, your case is different).

Here’s some code I’ve used to export and import multiple data tables, each with more than 100 columns, which included every type except media objects and links. It properly handles None values (which can be troublesome, with default CSV dumps generated by Anvil), empty values, text, number, boolean, date, datetime, and simple objects (you should be able to add media objects by encoding to base64, for example). Notice that the import routine wipes the datatable with table.delete_all_rows(), so be careful using it:

@anvil.server.callable
def export_adult_table_to_json():
    table = app_tables.adult_eval_form_data
    rows = table.search()
    data = []
    for row in rows:
        row_data = dict(row)
        for key, value in row_data.items():
            if isinstance(value, datetime.date):
                row_data[key] = value.strftime('%Y-%m-%d')  # Convert date to date format
            elif isinstance(value, datetime.datetime):
                row_data[key] = value.strftime('%Y-%m-%d %H:%M:%S')  # Convert datetime to string
        data.append(row_data)
    json_data = json.dumps(data)
    json_data_bytes = json_data.encode('utf-8')
    media_object = anvil.BlobMedia("application/json", json_data_bytes, name="adult.json")
    return media_object

@anvil.server.callable
def import_adult_table_from_json(file):
    json_data = file.get_bytes().decode('utf-8')
    data = json.loads(json_data)
    table = app_tables.adult_eval_form_data
    table.delete_all_rows() 
    for row_data in data:
        for key, value in row_data.items():
            if value is not None and key in ['Patient_DOB', 'Date']:
                try:
                    parsed_value = datetime.datetime.strptime(value, '%Y-%m-%d').date()
                except ValueError:
                    parsed_value = None
                row_data[key] = parsed_value
            if value is not None and key in ['Timestamp']:
                try:
                    parsed_value = datetime.datetime.strptime(value, '%Y-%m-%d %H:%M:%S')
                except ValueError:
                    parsed_value = None 
                row_data[key] = parsed_value
        table.add_row(**row_data)
    return "Data imported successfully"

You can adjust the app_tables.<tablename> to import/export as many tables as needed.

On the front end, there’s a button and a fileloader widget:

  def button_export_adults_click(self, **event_args):
    media_object = anvil.server.call('export_adult_table_to_json')
    anvil.media.download(media_object)

  def file_loader_import_adults_change(self, file, **event_args):
    if confirm('ONLY RUN IF ON ANVIL-APP-SERVER'):   # just a note to ensure I don't copy data sensitive data back to the development server
      if file is not None:
        response = anvil.server.call('import_adult_table_from_json', file)
        alert(response)

To deal with linked columns, the Sqlite solution which ianbuywise suggested, parses the anvil.yaml config file to reconstruct schema.

2 Likes

This is brilliant.

I envision building applications with an admin panel (django style). Admin would be able to run an a version of the app that can manipulate the database in local deployment.

If this was also media file compatible it would be a complete solution.

Thank you, I saw the edits showing you care about your answer. Members like you make this an excellent community!

Hey man, did you get the answer? Because i am facing the same issue now. Anxiously waiting for the solution. Any help would be appreciated. Thanks in advance

While in the original app on anvil.works create a button called upload data to server. Maybe with a password. Then once you are locally deployed, click that button and upload the data you will need.
I didn’t find a way to modify the data with some desktop tool sadly, but I will be making my own “Admin panel” like django does when i need it in the future.

Somethign like this:
image

I am still openif someone suggets a tool that already exists. But for now you have to make one yourself.