Question on data binding with Data Grid

Anvil Q&A
1

Hello dear anvil people !

I have a few questions on how data grids work…

Let’s dive directly into it:

When i use a data grid component, i can manually add data from any data table, EVEN for tables that don’t provide access to client (“client code has no access to this table”).

I find this weird and…

Anyway, since it does work… when data bindings are set in this data table (more precisely, the repeating panel), if the wright back option is enabled, does it wright on the data table or only on the self.items of the repeating panel ?
I think i know the answer but since the data grid has been populated directly from a data table, it would be natural that the write back also works to the data table.
I’m not saying it should be, but given how things work, it “looks like” so, and i was just wondering how this works…

2

A DataGrid doesn’t know anything about data tables.

A DataGrid shows the values stored on a list of dictionaries.

The list doesn’t need to be a list. It can be any object that can be iterated like a list. A SearchIterator object created by searching a data table can be iterated, so it can be used by a DataGrid.

A dictionary doesn’t need to be a dictionary. It can be any object with key/value pairs. A Row object returned by iterating a SearchIterator has key/value pairs (one key per column), so it will work.

The DataGrid itself is not going to write back anything, because all it does is rendering stuff, not editing it. But if will assign each dictionary (or dictionary-like) object from its list (or list-like) to the item of its template form, and the template form could have controls that will change the values of the dictionary keys.

I hope that clarifies what a DataGrid does.

Now we can focus on the SearchIterator and Row objects. A Row object is like a dictionary on steroids: it can be iterated and return values to be shown in the UI just like any dictionary, but when you (or any component or any line of code or whatever) assign a new value to a key, it will update the column of the row its original value was read from. And it will do this whether it is on the client or on the server, pretty cool!

I rarely use DataGrids to show data from tables. I either build lists of dictionaries, or, in more complex apps, I create my own classes to interact with DataGrids. For example the class Shipment can be iterated just like a list, and each returned item is an instance of the class Crate which can be used just like a dictionary.

3 Likes
3

Thanks @stefano.menci for clarification !
This is more or less what i was thinking a DataGrid would do with data bindings and write-back activated.
Regarding Row object, does it mean if i return a datatable row from server side to client side, and edit it in the client, will it directly update the datatable row ?

Being new to app development (been a little more than 5 months now as a freelancer, more of a scientific python background for my studies in geophysics), it also comforts me to see your approach of using DataGrids is quite similar to mine for my business management app :
During the custom user authentication process (server side), i create client side global variables (dictionnaries) containing copies of rows from data tables, based on if the user has a role with read-access granted (in the app, i have roles such as admin, director, agent, …).
I then use these global variables to populate the DataGrids.
If the user edits the items of the datagrid and has a role with write-access granted, then and only then is made a call to a server side function to update the related row of the proper data table.

4

Not necessarily. Your Server-side code can choose to mark the source of the row as Client read-only. Then the row would be readable, but not writable. See Views

1 Like
5

Yes.
That’s the point of using Row objects. (One of the points.)
And that’s why I don’t use them.

# this triggers three round trips (if executed on the client)
row['column1'] = value1
row['column2'] = value2
row['column3'] = value3

# this is more efficient, because it updates 3 columns in one round trip
row.update(column1=value1, column2=value2, column3=value3)

A Row object is more than a smart database updater. It also caches / lazily loads some values. This can be managed by the fetch_only query arguments and by the accelerated table settings.

If you don’t want to expose some columns, you can use a view of a table, so the row contains only the columns of that view. And you can use a view to make a row read only.

I never send Row objects to the client. My server callables return data structures, usually lists and dictionaries, that the client uses to build some class instances. At this point the client has an object that knows what it needs to know to manage the UI and to call other server callables to save any edited data. I only save the whole object when I decide to save, for example when a user clicks on the Save button, and all the data related to that object, which often is spread across multiple tables, is saved in the same transaction.

The following code works because the Shipment class is iterable like a list, and returns dictionary-like objects (of the Crate class) that can be queried with both attributes and keys (see the AttributeToKey paragraph here):

# in the main form
self.shipment = Shipment(shipment_id)
self.data_grid_1.items = self.shipment

# in the template either
self.name_label.text = self.item.name
# or
self.name_label.text = self.item['name'] # this is DataGrid friendly

def save_button_click(self, **eventargs):
    self.shipment.save()

In the code above, the Shipment class constructor makes one server call to load anything it needs in one round trip. It is iterable, so it can be used in a DataGrid, and the Crate objects it returns are dictionary-like, so they can be used with databinding.

The save method will send all the modified data to the server in one single round trip, and update all the rows of all the tables that need to be udpated.

This is the right thing to do, but it is not enough. You need to do the same permission check on the server side. Any respectable hacker can see and modify your client side code, and bypass any check and make any server call. But no one will be able to fool the checks made on the server side.

3 Likes
6

@stefano.menci @p.colbert thanks to both of you for clarification about restricting the view of a row object. I read it in the docs so it’s not a complete discovery to me, but i also never used it so i’m also not familiar with this feature.
However, i don’t think i will ever use this feature or even use rows directly in the client (at least for apps that require a certain level of security and with multiple users or group of users -businesses for instance-).
Because i never used the view feature of data tables, my current approach was simply to :

  • explicitly name the columns i don’t want to share to users as _column_name (for instance, an object id, a user id, or a login date, …)
  • build a Global variable containing data from tables and from columns which name doesn’t start with “_”
  • only send this Global variable (a dict) to the client

i’m sure the view approach would be more efficient, but at least here i know what goes in the client and what stays in the server. Maybe i will add it later in my app (or in future ones).

@stefano.menci :

A Row object is more than a smart database updater. It also caches / lazily loads some values. This can be managed by the fetch_only query arguments and by the accelerated table settings.

Well… will have to look deeper into this if i decide to use em in client someday.

I never send Row objects to the client. My server callables return data structures, usually lists and dictionaries, that the client uses to build some class instances. At this point the client has an object that knows what it needs to know to manage the UI and to call other server callables to save any edited data. I only save the whole object when I decide to save, for example when a user clicks on the Save button, and all the data related to that object, which often is spread across multiple tables, is saved in the same transaction.

Yup, that’s also something i was trying to be aware of. Most of what you describe is what i was trying to figure out a few weeks ago and what i finally came with. As someone with zero background as a dev, I wasn’t sure it was “like that” that devs do it, but it seems to be the most convenient and logical approach.

Didn’t want users to have direct access to rows, so i used global variables (dict) containing the data from tables.
Same for fetching or saving data in tables, i made a first test app with multiple round trips to server functions and understood it was not the way to go.
So yeah, i try to minimize as much as possible the server function calls.

For instance, i do all this in one function during my custom login:

  • check user permissions and read-access to data tables
  • based on user access, create a global variable containing all accessible data
  • store also the UI access of the user (what components should not be displayed), even though someone qualified enough could manipulate it, this would not cause major issues. a user that is not supposed to see something may see it, for instance a “remove item” button, but in the server i double check the user permission so…
  • return global variable to the client, which is used to set the UI.

My global variable being a simple dict, it can be tiring sometimes to update the global variable after a server function call (edit, remove, or add).

This is the right thing to do, but it is not enough. You need to do the same permission check on the server side. Any respectable hacker can see and modify your client side code, and bypass any check and make any server call. But no one will be able to fool the checks made on the server side.

Fortunately i was aware of that, and i always check the user permissions in the server functions. Here is an example:

def check_user_db_access(db_name, writeable=False):
    role = anvil.server.session['role']
    offer = anvil.server.session['offer']
    db_access_by_role = app_tables._db_readable_by_role.get(role=role)[db_name]
    if writeable:
        db_access_by_role = app_tables._db_writeable_by_role.get(role=role)[db_name]
    db_access_by_offer = app_tables._db_access_by_offer.get(offer=offer)[db_name]
    db_access = db_access_by_offer and db_access_by_role
    return db_access

@anvil.server.callable
@anvil.tables.in_transaction
def _delete_item_from_database(db_name, id):
    # check user access:
    if check_user_db_access(db_name, writeable=True):
        # user access granted
        row = getattr(app_tables, db_name, None).get(
            _business_id=anvil.server.session['business_id'],
            _id=id)
        if row:
            row.delete()
            return True, "L'objet a bien été supprimé."
        else:
            return False, "L'objet n'a pas pu être supprimé: objet introuvable dans la base de données."
    else:
        # no access, return cancel message
        return False, "Vous n'avez pas la permission requise pour cette action."

so basically i have databases to define user permissions based on their role and based on the plan (offer) they have subscribed to (which have been stored to the server.session during the custom login process).
Then, when a call to server function from client is made (such as _delete_item_from_database here) user access is always checked.
Then i return True or False if the call is successfull or not, with a success or error message that i display as a notification.

see the AttributeToKey paragraph here

Wow Okay ! now that’s some cool stuff ! Pretty cool to see someone’s compilation of best practices and dos and don’ts.
I’m glad to see my approach is on the good path, even though there’s lot of things i need to rethink.
Will definitely take a closer look at this in the next days !

2 Likes
7

If by “global” you mean a variable that is defined at top level scope on the server side, visible by all modules, then you need to be very careful: if you enable the persistent server, the globals could be shared across all the calls.

You may not have a plan that allows the persistent server today, but as soon as you upgrade your plan, you should also enable the persistent server on all the apps, so they will immediately become much more responsive. And they will also fail if they rely on globals.

If by “global” you mean on the client, then there is no problem. Each client has its own interpreter, hence its own globals.

1 Like
8

Is a client-session id available, to keep them distinct?

9

I do see the use of anvil.server.session in the snippets above, so there may be a key for a dictionary that keeps the globals for different sessions.

But this would have two problems:

  • Even with persistent server enabled, there are reasons why the server could decide to spin a new instance to respond to the next request, so you can’t rely on the globals being there
  • You would need to design some cleanup to prevent that globals from growing too large and running out of memory, either over time or because you have many clients

Using globals with persistent server may make sense only if building those globals would take a long time, but you should always be ready to rebuild them if they don’t exist because the request is being handled by a new instance. And accesses to those globals should be thread safe, because you could have multiple requests accessing them concurrently.

I never have the slow global problem, so I don’t have globals at all. All my apps have persistent server enabled and none of them sets globals.

1 Like
10

In short, a cache optimization only. And beware premature optimization.

1 Like