Repeating panel is slow to populate

Hi everyone,

I’m setting the items of a repeating panel with rows from my data table. The repeating panel is just displaying a bunch of labels.

It takes like 15 seconds for the page to load (and my print statements tell me that this is occurring as the repeating panel is being populated).

The repeating panel is displaying about 75 sentences that are stored in my data table.

Is there some reason why this takes so long? I must be doing something wrong.

How many items are stored in the data table? 75 or many more?

There are several hundred records in the table. I’ve been experimenting with simply just dumping the whole set of sentences into one row rather than having a row per sentence. I can display it just the same by using a BlobMedia object on a label’s text rather than using a repeating panel. So far that loads in the blink of an eye. I’ll see if I hit any snags.

Okay, rather than spreading the text data I have across rows in a repeating panel, I just store it all in one cell in the table (where each row corresponds to a set of themed text data) and display it as a single item (i.e., convert it to BlobMedia and set it to a label’s text).

Storing and displaying the data in this way allows the text-based data to display instantly when the page loads. Much better than 15 seconds of loading time. This seems to solve my issue and so I’m left thinking that perhaps the repeating panel is better for a very small amount of rows (that need to be kept separate in the db). Feel free to correct me of course.

Allan

1 Like

Have you tried with a Simple Object column rather than BlobMedia?
It allows you to store a list or other JSON data without the need to convert it.

If the list of items is long is better to create a server function that builds a list of dict(row) on the server side and return the list. This will prevent the repeating panel from doing a round trip per row.

1 Like

Oh cool idea. I didn’t know you could dict(row). I will try this.

What do you mean by the repeating table having to do a “round trip”? Is it not doing as much work when passing it as a dict?

If you give a query to a repeating panel, you don’t have the data yet. The data will be fetched when the repeating panel asks for it. I don’t know the details, but Anvil might fetch one line at a time or a block of lines.

Also, if you have a column with a link to another table, the other table row will be accessed only on request. For example a user['friends'] that returns a list of rows from another table will require another query.

The work is the same, but collecting the data for a dict on the server side is much faster than doing the same on the client side.

If you create a dict on the server side and you make sure to expand the list of friends and read all the rows you need and skip the ones you don’t, the form receives the dict and will never go back to the server to ask for anything else.

Yes, sometimes the magic of data binding between the form and the database is sexier, but often is better to make a server side function that prepares all the data that you might need and return it in one shot.

1 Like

Thanks for the detailed and helpful response. Really appreciate it.

Hi all,

I got back to trying this conversion to a dictionary for instances of a repeating panel in my app; however, even for very small repeating panels I still get quite a lag before it appears.

My server function returns only a list of dictionaries rather than rows from the data table, and yet there is still a noticeable lag before populating the repeating panel.

@anvil.server.callable
def get_list(my_search_text):

   # get list of rows from a table
   rows=app_tables.my_table.get(text=my_search_text)['links_to_another_table']
   
   # for each item in the list, make it a dictionary
   rows = [dict(x) for x in rows]

return rows

The dictionary has just two keys with a few strings as values, nothing massive, and no Anvil rows inside. The lag is about 2 seconds just by eyeballing, the spinny ball goes around a little more than once.

@meredydd, @stefano.menci have any ideas?

Have you tried to print(datetime.now()) before each step is executed in both the server side and the form code?

It will help you narrow down which step is the slow one.

Hi @stefano.menci,

By doing as you’ve suggested, I’ve determined that actually populating the repeating panel with repeating_panel.items=my_rows is not the issue as that seems to happen very fast.

What is taking the time is simply just returning data from any server function. I want to emphasise that inside the server function itself, retrieving anything from my data tables is extremely fast. It is only when I measure the “round trip” time for the server to return something back to the client that I see the speed issue.

For example, look at the timings here (units are in seconds, not ms):

The server calls are very simple and the data tables are very small. Server_call_1 is just returning 4 or so rows from a linked table as follows:

@anvil.server.callable
def get_themes(question):
  rows=app_tables.questions.get(question=question)['link_to_themes']
  rows = [dict(x) for x in rows]
 
  return rows

The client call looks like this:

start_time=time.time()
rows=anvil.server.call('get_themes', question)
print('server_call_1 takes ' + str(time.time() - start_time) + ' ms')

Each line in the server function runs very quickly and doesn’t come close to the 700ms you see above.

Is it normal to wait over a second for a few simple round trip server calls to be made? I’m sure I’m just doing something wrong but I haven’t figured it out. Any help is much appreciated.

For those watching, Shaun and I continued outside this thread and he helped me to optimize the app so that the speed is acceptable. I cannot share a clone at this time but in his response you can get a sense of the techniques he used to speed things up. From what I understand, he removed unnecessary server calls by returning much of what I needed inside of a single server-side function. Part of this optimization required one of my smaller tables to be de-normalized.

Great thread I haven’t seen any slowness myself but now I’m curious and will be using some of said things in this post to check mine.

Yes, lots of good little tips in here. The people here are a great resource.

Hi Allan

I’m very intrigued by what you wrote here. I’m coming across the same issue - I want to give users the option to load either a month, a quarter or a year’s worth of info. The month is generally ok…about 3-4 seconds to load. Quarter is barely acceptable and a year takes up to 50 seconds for the repeating panel to load! I pass the RP a list of dictionaries (i.e. no server calls anywhere), each panel has about 10 labels to populate.

Can you send me an example of an app where Blobmedia object is used to the same effect as a RP? Or explain to me how it works?

My app downloads a string from server, uses json.loads to convert into list of dictionaries, and then i filter that into a month/quarter/year’s worth of info before passing it to the RP.items = list code.

Many thanks in advance
Bruce

The speedup from using a dict() rather than a table row is quite remarkable; so much so that is hard to see why it should ever be done any other way when the point of the webpage is to simply deliver, reformatted, the results of a table. And the ability to do so, on either the server or the browser side, is a single line of code. Instead of:

 repeating_panel_1.items=app_tables.[tablename].search(whatever)

all you need is:

repeating_panel_1.items= [dict(list(row) for row in app_tables[tablename].search(whatever)]

if the browser can retrieve the table, or the equivalent in a server function (with dict(row) instead of dict(list(row)) if it isn’t.

You give up lazy search of the datatables, but gain so many fewer trips to the server that it seems to me the use case for the first method is really restricted

I discovered them when a 40 line table was coming up in 50 seconds, reduced to 2.

1 Like

I have just done some testing with my app which relies heavily on linked rows and this simple change has had a massive increase on performance. The server code and timing below shows seconds taken across the server call, the repeating panel population and the drop down list population respectively

Before:
data['docs'] = app_tables.documents.search()

Docs starting call 3.63272
repeat panel 13.77057
pop dds 0.002875

After:

data['docs'] = app_tables.documents.search()
data['docs'] = [dict(row) for row in docs]

Docs starting call 3.593985
repeat panel 2.95227
pop dds 0.00303

Epic!!

Edit: Updated code to reflect @jonathan.falk and @stucork’s updates above and below. Timing is still accurate

3 Likes

You don’t need the call to list anymore
[dict(row) for row in docs] will work just fine now - both on the client and server…

4 Likes