(Unofficial) Anvil Version Control for Dummies

[david.wylie]

@AWSOM

is there a prize for longest forum post ever?

I nearly took you up on that semi-implied challenge, but cut my post down when even I couldn’t be bothered to read it all back

@AWSOM

I do think it’s a shame that it takes so long (relatively speaking) to settle on good working practices for version control and collaboration, and that although Git/Github integration is supported in Anvil, it doesn’t just ‘work’ right out of the box.

One of the “problems” with programming languages and their environments becoming easier to use is that one can lose sight of the fact that developing is hard. Anvil has lowered the bar to web development significantly but to do that it has had to hide and simplify some quite complex processes, as well as prioritise features. Team development moves you into non-trivial land.

I would love to see a version control built in that allows you to check out a package, blocking anyone else from working on it, and then merges all the changes, highlighting errors, creating a test environment just for you then…etc., etc. The problem is, I’ll bet there’ll be as many competing, incompatible ideas from others of varying development experience. And actually, that’s only what I think I want. If I ever got that I might end up saying, “Oh, no, that’s not as good as it sounded in my head!”

It’s a tough one, but I think when you are working on a project with a team, you have stepped onto the next rung of skills expectations and need to bite the bullet (which you’ve done).

I’ll bet that once you’ve worked through your list for the 40th+ time on large & non-trivial projects you won’t even notice the “Git incantations” (great phrase, by the way) as being anything other than just “what you need to do to make things work”.

What I would like to see is a really, really clear set of instructions, with pictures, of a team project set up where, for example, 2 people are working on the UI and 2 people are working on the back end. Advice on how to ensure no one clashes, what to do if they do, how to merge all the code, best practises. A proper guide that idiots like me could stick up on my wall and use as a bible, from someone(s [sic]) who really, really understand the process.

[AWSOM]

Thanks for your comments @david.wylie.

@david.wylie

Team development moves you into non-trivial land.

I would love to see a version control built in that allows you to check out a package, blocking anyone else from working on it, and then merges all the changes, highlighting errors, creating a test environment just for you then…etc., etc. The problem is, I’ll bet there’ll be as many competing, incompatible ideas from others of varying development experience.

… actually I think this problem is exactly what Github seeks to solve while giving you flexibility in the way you structure and control your packages. I only stumbled on it today but it seems that’s already partly integrated and partly available within Anvil’s View History. Please see (and comment/like) my recent Feature Request on the topic?

[AWSOM]

@david.wylie

I’ll bet that once you’ve worked through your list for the 40th+ time on large & non-trivial projects you won’t even notice the “Git incantations” (great phrase, by the way) as being anything other than just “what you need to do to make things work”.

Hehehe so true, and even over these last few days in my journey from ‘newbie’ to ‘not so newbie’ I’m starting to warm to the idea of a one line Git incantation rather than logging in through my browser and lots of menu clicking. However, since I started off with a pathological aversion to Git incantations, I think other newcomers are likely to be in the same boat, hence the idea of writing a more prescriptive and (hopefully) simpler Tutorial for first-time ‘dummies’ like me…

[AWSOM]

@david.wylie

What I would like to see is a really, really clear set of instructions, with pictures, of a team project set up where, for example, 2 people are working on the UI and 2 people are working on the back end. Advice on how to ensure no one clashes, what to do if they do, how to merge all the code, best practises. A proper guide that idiots like me could stick up on my wall and use as a bible, from someone(s [sic]) who really, really understand the process.

I’d like to see that too but certainly don’t “really really understand the process”!

I think the foundations of an answer will be obvious if my Feature Request is actioned, at which point you could simply create new branches “Dev1 - Back End”, “Dev2 - Back End”, “Dev3 - UI”, and “Dev4 - UI” in Github after inviting collaborators to the master Anvil remote. Each developer would create their own branch and when ready create a pull request for the app owner to review, discuss, and decide whether to merge using organic Github functionality.

[david.wylie]

I like being told what to do by people who really understand something. Then when I think I really understand it, then I go off piste

[jshaffstall]

I’d done some experimentation on multi-app projects with dev/prod versions and data tables. It was a bit of a mess, but some of the gotchas are shown in this thread: Cloning apps and data tables for Production and Development Apps

[AWSOM]

Hi all,

Does this diagram I’ve created for my Tutorial look right to you?

If you prefer to show rather than tell, please feel free to copy and amend the original PowerPoint slide here: https://github.com/PFython/WGACA-UK/blob/master/project%20management/Anvil%20Git%20Github.pptx

[p.colbert]

I see a common thread to every topic in (Unofficial) Anvil Version Control for Dummies

1. Identification of a specific feature that varies from one [version of an] app to another.
2. Identification of the source-code-level artifacts that go into that feature (app feature or Anvil feature), and where they can be found. In some files, these are just a fragment of the entire file.
3. Using 2 to manage those artifacts, especially when Anvil’s default management does not do what’s appropriate in your specific case. This includes defining extra steps to perform, before and/or after an Anvil or Git action.

Do I have this abstracted properly?

[jshaffstall]

I think so, with the caveat that there might not be source-code-level artifacts (e.g. for data tables) to use.

[p.colbert]

@jshaffstall, good catch! Thanks!

Yes, I can envision several app instances sharing some tables (e.g., to define prompts and help text) but not others (e.g., users). The distinction between shared and non-shared features/artifacts really is app-specific.

Revise that to “artifacts (source-code-level and otherwise)”.

[jshaffstall]

Yes, the complexities add up. Combine that with some of the resources needing duplicated for dev/prod splits and others being shared, and it’s tough to come up with a one-size-fits-all mechanism.

Putting all the resources into source code artifacts would help a lot, as would supporting more robust Git support in Anvil itself.

[AWSOM]

Thanks for taking the time to read and think about my Tutorial, both.

I’m afraid your discussion has gone over my head though… Is there a Dummies’ level summary/recommended actions you think might be worth adding to the post? It’s now a forum Wiki so you should be able to edit directly… Maybe a new Level 12 entry?

[meredydd]

Hi all,

I’ve merged a couple of other discussions into this thread that belong here, and reopened this thread now that it’s a wiki. Have at it!

[p.colbert]

Thank you, @meredydd. I would happily have posted here, had the topic been open.

[p.colbert]

This is more of an overview, a conceptual framework that your process fits into.

Let’s imagine that you have 2 Production apps, one for Customer A, and one for Customer B, each in its own Anvil repository. They are identical except for some customer-specific detail X, so A’s X and B’s X are different. (X may be a set of Secrets, a database table, a function or module, a logo, etc.)

When you need to update these apps (e.g., for a bug fix), you will need to preserve those distinct copies of feature X. To do that, you will need to know how Anvil represents them and/or refers to them, within each instance of the app, and how an update (e.g., git push) will affect those things. You may need to take steps before and/or after the update. The details will vary depending on how the feature is implemented, but this is the “big picture” that (I hope) every feature fits into.

Sometimes I need to see the trees. Sometimes I need to see the forest in order to understand the trees.

[jshaffstall]

@AWSOM

I’m afraid your discussion has gone over my head though… Is there a Dummies’ level summary/recommended actions you think might be worth adding to the post? It’s now a forum Wiki so you should be able to edit directly… Maybe a new Level 12 entry?

It’s tough to make a one-size fits all recommendation, since every app is a bit different.

The complexities in my case came from having a single “app” that was composed of multiple Anvil apps. This is a pretty typical pattern used with Anvil. One of my apps, for example, was for authors, another was for customers buying things, another was for readers, etc. These apps all shared some data tables (Users, for example, among others).

Splitting this combined app into Dev/Prod versions (at that time I wasn’t thinking in terms of test versions, since publishing Dev in Anvil worked fine for my testing) highlighted the complexities really well for me.

When you clone an Anvil app, the clone gets copies of the data tables used by the project it’s cloned from. In the case of a multi-app Anvil app that shares data tables, that means every Production clone of the various apps has its own copy of shared tables. You have to go through manually and fix that up. Luckily, that’s a one-time thing, but it’s complicated especially if you have data in the tables already and have linking fields to other tables.

Since data tables aren’t stored in the git repo you download, though, changes to those don’t get copied when you update Prod from Dev. You have to manually update the Prod data tables with changes made in Dev.

Specific steps I identified are in Cloning apps and data tables for Production and Development Apps I can’t even say that’s all of what would need done.

Anyway, it ends up being a mess, and I don’t think it’s in the For Dummies category. It was enough of a mess that I kind of stalled on development because migrating to Prod would have been error prone and time consuming.

There have been some feature requests that Anvil handle the Dev/Prod split more natively. I’m really hoping something along those lines is adopted because of the complexities in doing it manually.

[p.colbert]

Yes. My aim was to provide a way of thinking about an app’s features in a way that makes such reasoning easier. Maybe even amenable to automation.

For apps that “just happen” or are grown “organically”, it can be hard to identify where one feature begins and another leaves off. Even where features are added systematically, it sometimes happens that one body of code serves several interlocking features. So it’s not perfect. But the more we can isolate and separate things, the better chance we have of managing them deliberately (instead of mangling them accidentally!).

[hugetim]

Until recently I had a setup similar to your diagram, with my DEV-TEST version shared to GitHub. But my latest thinking was that it would be better to have the PROD version on GitHub, for transparency’s sake. Otherwise no one would really know what code the PROD version is running, which I thought was part of the idea of open source?

Mine is just a solo project for the time being, but thinking ahead I guess pull requests to the PROD version on GitHub could be cloned to a new DEV/TEST version and tried out there before merging into the published branch on PROD?

[stefano.menci]

Another reason for having the PROD in GitHub is that you have full control over the which commits are included, because you have one commit every time you merge the changes from the DEV version, while having the DEV version on GitHub would include tons of “Saved Automatically” and other useless commits.