[FIXED] App server auto migration problem

Bug Reports
1

What I’m trying to do:
Update anvil app server to latest (1.6.0)

What I’ve tried and what’s not working:
I have updated the packages via pip and the latest jar file has downloaded.
When running the app server it reports that the schema has changed (it hasn’t).
When running with --auto-migrate the schema held in app_storage_tables is messed up. The users schema has been removed and replaced by the schema for another table (which now appears twice).
The raw data in app_storage_data is intact.

Code Sample:

# this is a formatted code snippet.
# paste your code between ```

Clone link:
share a copy of your app

1 Like
3

To investigate further, I have wiped the database to start from a clean slate. Here is what I have found:-
The schema definition stored within the database does not match the schema definition in anvil.yaml, the unique identifiers for each column have changed. Each time the app server is started, the schema is validated and reported to have changed and require migration. Below is the schema from anvil.yaml, followed by a dump of what is in app_storage_tables.

db_schema:

  • name: users
    id: 65525
    python_name: users
    columns:
    AupfJ4vEZj0=:
    name: roles
    type: liveObject
    backend: anvil.tables.Row
    admin_ui: {order: 1.75, width: 200}
    table_id: 67429
    gNx8x+NMB34=:
    name: email_confirmation_key
    type: string
    admin_ui: {order: 6, width: 200}
    s4sY+aEHUNo=:
    name: refresh_countdown
    type: number
    admin_ui: {order: 11, width: 200}
    Vst9OcgP+Q4=:
    name: remembered_logins
    type: simpleObject
    admin_ui: {order: 10, width: 200}
    SU3tqBZsHuM=:
    name: n_password_failures
    type: number
    admin_ui: {order: 13, width: 200}
    f6ZRigBiesI=:
    name: password_hash
    type: string
    admin_ui: {order: 4, width: 200}
    Ei7IjoxoJCw=:
    name: signed_up
    type: datetime
    admin_ui: {order: 2, width: 200}
    nFMJN_cbXgo=:
    name: confirmed_email
    type: bool
    admin_ui: {order: 5, width: 200}
    GsEWULURmTE=:
    name: firstname
    type: string
    admin_ui: {order: 7, width: 200}
    tzX3WhRqias=:
    name: it_only
    type: bool
    admin_ui: {order: 12, width: 200}
    xUOTG6dAmYE=:
    name: surname
    type: string
    admin_ui: {order: 8, width: 200}
    rlVoRmmk0DM=:
    name: email
    type: string
    admin_ui: {order: 0, width: 296}
    iYklbrK0aS4=:
    name: enabled
    type: bool
    admin_ui: {order: 1, width: 100}
    LlKGcPZ+U_w=:
    name: last_login
    type: datetime
    admin_ui: {order: 3, width: 200}
    access: {python_name: users, app_id: XZQO735PL7EXEFEJ, server: full, client: none,
    table_mapping_name: null, table_mapping_id: null, table_id: 65525}

  • name: roles
    id: 67429
    python_name: roles
    columns:
    TZhFfJ3mDho=:
    name: role
    type: string
    admin_ui: {order: 0, width: 200}
    access: {python_name: roles, app_id: XZQO735PL7EXEFEJ, server: full, client: none,
    table_mapping_name: null, table_mapping_id: null, table_id: 67429}

    1 | users | {
    “1p_vwQ0FqHY=”: {“name”: “it_only”, “type”: “bool”, “admin_ui”: {“order”: 12, “width”: 200}},
    “ElBXbsaOIws=”: {“name”: “surname”, “type”: “string”, “admin_ui”: {“order”: 9, “width”: 200}},
    “IFgm5_5Vo_Q=”: {“name”: “roles”, “type”: “liveObject”, “backend”: “anvil.tables.Row”, “admin_ui”: {“order”: 2, “width”: 200}, “table_id”: 2},
    “JSVlylIAefM=”: {“name”: “remembered_logins”, “type”: “simpleObject”, “admin_ui”: {“order”: 10, “width”: 200}},
    “OIoztpiF94E=”: {“name”: “confirmed_email”, “type”: “bool”, “admin_ui”: {“order”: 6, “width”: 200}},
    “WhVtlqJ+hF8=”: {“name”: “password_hash”, “type”: “string”, “admin_ui”: {“order”: 5, “width”: 200}},
    “h4ZCAHCl47I=”: {“name”: “email”, “type”: “string”, “admin_ui”: {“order”: 0, “width”: 296}},
    “ke3ieNdPK5Q=”: {“name”: “enabled”, “type”: “bool”, “admin_ui”: {“order”: 1, “width”: 100}},
    “kx0+rc4h8nU=”: {“name”: “email_confirmation_key”, “type”: “string”, “admin_ui”: {“order”: 7, “width”: 200}},
    “qpmZja1DIsw=”: {“name”: “signed_up”, “type”: “datetime”, “admin_ui”: {“order”: 3, “width”: 200}},
    “sTCZ6ZjTzQo=”: {“name”: “refresh_countdown”, “type”: “number”, “admin_ui”: {“order”: 11, “width”: 200}},
    “tACU_zurTF8=”: {“name”: “last_login”, “type”: “datetime”, “admin_ui”: {“order”: 4, “width”: 200}},
    “xrwA6TuwsZQ=”: {“name”: “firstname”, “type”: “string”, “admin_ui”: {“order”: 8, “width”: 200}},
    “yHOPU8Z9viw=”: {“name”: “n_password_failures”, “type”: “number”, “admin_ui”: {“order”: 13, “width”: 200}}
    }

    2 | roles | {
    “PI5worey3o8=”: {“name”: “role”, “type”: “string”, “admin_ui”: {“order”: 0, “width”: 200}}
    }
    (2 rows)

1 Like
5

This doesn’t seem to be an update issue but a 1.6.0 issue itself.

I run 1.6.0 on a container with a host directory volume for the database, and on first start it successfully creates the database and everything works, but if I kill the container and start it again it doesn’t recognize the schema and suggests all kinds of crazy changes.

Same procedure on anvil-app-server version 1.5.5 works fine.

I have created an issue on github: Incorrect database schema migration on 1.6.0 · Issue #45 · anvil-works/anvil-runtime · GitHub

2 Likes
6

Hi @gorka.eguileor & @ahawes,

It seems like something funny is going on here – it would really help if one of you could provide all of the following for one app:

  1. An anvil.yaml (or just the db_schema fragment) that causes this problem on an empty database (if you’re putting it into a forum post, please post between ```, otherwise the Markdown formatter will eat it)

  2. A description of how to reproduce the error on a blank database (I think that what you’re saying is that “start the App Server on this app, shut it down, then start it again” is enough, but it would be good to have that confirmed)

  3. A copy/paste of the output from the App Server when it’s trying to migrate a database it’s already set up (like in @gorka.eguileor’s GH issue)

So far I have half of #1 and maybe #2 from @ahawes, and #3 from @gorka.eguileor – if someone could fill in the gaps and give us a full house, we can get this replicated and fixed!

7

Hi @meredydd ,

Thank you for the quick reply.

I was able to reproduce it with a new blank project, then added data tables and users services and created a simple table with a field that links to the users.

I created a reproducer and pushed to https://github.com/Akrog/anvil-issue-45.

DB schema can be found in the Material_Design_1/anvil.yaml file

Cheers,
Gorka.

8

Perfect, thank you! Moved to Bug Reports :slight_smile:

9

We have now released the App Server v1.6.1, with a fix for this bug!

Thanks in particular for your help, @gorka.eguileor and @ahawes :slight_smile: