Discussing: Best practices: Test-driven development with Anvil

(Wiki link)

Are there situations in which putting the logic in client rather than server modules would be inadvisable for security reasons? In your example, I guess the risk is limited by the server functions only accessing rows associated with the logged in user.

2 Likes

In my experience I never had security concerns of this kind.

The server returns only dictionaries that include only certain columns, so there is no way to access the database from the client or abuse or break anything.

Please let me know if I have missed something! :slight_smile:

The only problems with using the logic on the client side that I can think of are:

  • It could be impossible because you may need libraries that are not available on the client
  • It could be slow. I have never compared the performance of some CPU intensive task running on the server vs on Skulpt. I never had problems with performances, but I wouldn’t be surprised if Skulpt was slower than the server
  • You may expose some aspects of your logic that should be kept secret
1 Like

You’d still need some logic on the server, though, right? Any validation for example, would happen on the client, but needs checked again on the server (to guard against people who are messing with the app’s Javascript to circumvent the validation).

Edit: that was one of the advantages of portable classes, right? That the same code could be shared between client and server in a secure way.

2 Likes

Yes, the server will always be the place where you make sure things happen safely.

I have not mentioned anything about safety in my post because so far I have used this technique only in apps intended for internal use, my users are signed in, and the first line of all the server functions is:

@anvil.server.callable
def something():
    if 'Internal User' not in CommonFunctionsServer.user_permissions():
        # either raise some sort of unauthorized exception or return something
        # safe to tell the client that something went wrong

The CommonFunctionsServer module is part of a custom component that I use to manage login, logout, user registration and user permissions. In the wiki post I have not addressed user management and permission at all because I do all with that custom component. I thought this might be missing in the post, and that’s one of the reasons why I made it a wiki: if someone out there will adopt this system and will have something to add, for example about user permission, please add.

I will eventually work on apps for external users (we have those too), but I haven’t done that yet.

Perhaps using portable classes with their capabilities is the correct way to manage this kind of issues. But, as I said, I never had this kind of issues yet, so I haven’t worked on a solution.

I will add a bullet about this on the “What to do section”.

I haven’t had the need yet to fully understand portable classes and the capabilities feature, but it does seem like the sort of thing they’re designed to handle. Maybe making your data classes portable and using capabilities would be the best of both worlds? Just thinking out loud, my current project is way too far along to refactor this much, but I like what you’re doing with your data classes.

It goes onto my list of “think about this when designing your next project” items.

I did refactor an old app, because it was very small and the changes I wanted to do were large, but I think this technique is especially useful if you adopt it from inception.

This started as a way to organize the code in my apps, then became a way to test them, now it’s a way to develop them.

I love test-driven development in general, and it turned out to be a good way to increase the coverage of my tests in Anvil apps (which in many cases were non existing).

Maybe they could be the solution, but for the time being I haven’t really seen any problem to solve.

On my first tries I was importing Globals both on the client and on the server side, so the same classes were used on both the sides. Calling some methods on the client would trigger a server call that would recreate the class on the server just like it was in the client so it could save itself.

They were not @anvil.server.portable_class portable, but they were my way portable.

I remember trying to think what methods should work on the client and what methods should work on the server side. I started with making the loading and saving working on the server side and most of the logic on the client side, which meant changing the behavior depending on where the code was running. The result was something similar to this (I’m making it up oversimplified based on my faint memory) (the numbers in parenthesis show the sequence of execution when calling boxes.save() on a form):

# Server module - all it does is instantiate and initialize, then
# tell the instance itself to do its job
@anvil.server.callable
def save(json_blob):
    # create the singleton on the server
    boxes = Boxes.from_json(json_blob) # (2)
    # let it do the job
    boxes.save() # (3)
# Globals
class Boxes(AttributeToKey):
    def save(self, boxes=False, candies=False, cookies=False):
        if anvil.server.context.type == 'browser':
            # (1) - working on data loaded and modified on the client
            anvil.server.call('save', self.to_json())
        else:
            # (4) - working on data just reconstructed on the server
            app_tables.boxes.add_row(**self.to_boxes_columns())
            app_tables.candies.add_row(**self.to_candies_columns())
            app_tables.cookies.add_row(**self.to_cookiees_columns())

My first versions of Globals were also interacting with the forms. For example they would change the background color of the template form of a repeating panel when the box was full.

The first thing I removed was all interactions with forms. I replaced the changing the background from the class with a background_color property ready to use with databinding.

Testing still wasn’t easy, because Globals was using tables and other services, and I needed acrobatics during the import and to avoid touching stuff that wouldn’t work on my PC.

Slowly I realized that removing interactions with Anvil services from Globals was the way to go.

I started using Globals only on the client side, only one singleton instantiated once and used by all the forms, set a few rules to follow almost religiously (the dos and the don’ts on the wiki post) and everything became much clearer.

I thought one day I would add some logic to the classes to make the server side functions simpler, but I never needed to do that. Both the number of server side functions and their complexity has been very low and everything seems to be working well as I described it.

Now my Globals have only one import (import anvil.server) used for calling the server functions. This makes me feel good and makes the testing much easier.

3 Likes

The example Boxes class already has the basics of a portable class in place: just rename the to_data method to __serialize__ (iirc).

This (early-stage) anvil_labs extension strikes me as similar in spirit:

I had the same thought.

This snippet from the description of the atom applies 100% to my test-driven singleton:

The library aims to move state management to global objects (aka an atoms)
this leave Forms to handle displaying the UI.
since an atom is global any form can import it
this prevents trying to pass state up the form hierarchy

Then I looked at redux and I found this:

Centralizing your application's state and logic enables powerful capabilities...

so far, we are similar, but then:

...like undo/redo, state persistence, and much more.

I haven’t looked at the capabilities of our atom and I don’t know if it’s already completed, but the goal of the test-driven singleton is limited to keeping the state and managing the logic, and doing it in one location that is not dependent from Anvil UI and services, so it can be tested offline.

1 Like

An atom from the atomic module doesn’t depend on the UI.
It’s based largely based on a subscriber model.

When using an atom - methods called in forms (that use the @render decorator) implicitly subscribe to specific atom attributes that they access.
When the atom's attribute is updated, any render methods that subscribed to that attribute are re-rendered.

But of course, if there is no UI then there are no subscribers. So in the case of testing in isolation, it should just work since atom's don’t really know anything about the render methods that subscribed to them.

e.g. in the count_atom example:

def test_count_atom():
    count_atom.value = 0
    assert count_atom.get_count() == 0
    count_atom.update_count(1)
    count_atom.update_count(1)
    assert count_atom.get_count() == 2

It’s definitely early days for the module.
But early adopters will help shape its direction :wink: .

(side note: we explored redux - but ended up with a more mobx inspired state manager)

4 Likes

I finally bit the bullet and rewrote a portion of my code according to these principles so that I could more feasibly set up automatic tests for it. It’s been a painful, time-consuming process, but it feels great to have the tests in place. And I’ve learned a lot along the way that should hopefully make the process quicker next time.

3 Likes

I hope the pain was caused by the rewriting part, not by following my guidelines :slight_smile:

1 Like

Subtopic: Identifying global objects, for design and implementation.
Disclaimer: I haven’t fully studied the Wiki.
Illumination: I come from a multi-language background, including C++, which sometimes puts ideas in a new light.

In Python, one is often not terribly concerned with object lifetimes. The garbage collector largely takes care of that for you.

In C++, however, it’s something you usually have to plan for, and code for. Whatever resources an object may own, it is responsible for cleaning them up, so it’s the programmer’s responsibility to make sure that it does.

Lifetime thus becomes an essential ingredient in an object’s identity. Things with different lifetimes must be different objects.

And when they are singleton types, which occur at very high levels in an application, then they must also be of different types. Thus, this notion can help us discern what high-level types (classes?) we may need.

With this in mind, I am working on a (tentative) description of high-level types, for your consideration. Containment is indicated by indentation.

  • BrowserSession
    • ClientApp
    • ServerConnection
      • UserSession

Each of these may have a different lifetime. A UserSession may terminate (via logout) before the ServerConnection does. All would terminate upon tab/page close (or reload).

It seems to me that the above are defined by Anvil’s app architecture.

That’s just a starting point, of course. Browsers may offer supplemental types (e.g., storage). Add-in libraries, likewise. And any real App may need its own, additional types.

With this model, one might direct each kind of event to the highest-level object that must respond to it. That would let it coordinate the responses of its constituents.

1 Like

Definitely the rewriting part. I have nothing but gratitude for your helpful guidelines. I have been wanting to separate the logic from the UI for a long time (particularly in a buggy part of my project that I have spent weeks on but still didn’t have working consistently), but it was only your guidelines that gave me the concrete advice (and push) I apparently needed to finally do it. It was still difficult to wrap my head around how to structure the class needed–it was just a totally new way of thinking relative to how I’ve been using Anvil for years. And writing tests first was also a new thing to (belatedly) wrap my head around.

I guess the only exception is that I wasn’t able to get my case to work with a single, simple server function. I tried to follow that pattern at first but ultimately decided it was more trouble than it was worth. Because the server functions need to access data tables that are not accessible to the user, things ended up being a lot more complicated than in your example.

1 Like

For anyone interested in this stuff, you might want to Google and read about:

  • Domain Driven Design
  • Clean Architecture
2 Likes

And for any other uber geeks, “Business Objects: Re-Engineering For Re-use” by Chris Partridge is a revelatory read if you can find a copy

Aristotle to OO Design via Descartes!

2 Likes

I found this helpful for understanding TDD better (and motivating me to try really doing it): Clean Code - Uncle Bob / Lesson 4 - YouTube

At one point he talks about how it’s a skill that takes time (3 months?) to learn. He says not to even try doing it at your job until you’ve practiced it elsewhere and gotten good at it.