REST api how to get nextPage results?

Anvil Q&A
1

Hello,

Currently I have configured an HTTP endpoint (https://anvil.works/blog/http-api-endpoints) and am able to get results. However now I have too many entries and the nextPage param is no longer null.

How to I format my request to get the next set of results? Is this something that is built into the framework?

Thanks
Michael

2

Hi Micheal,

It sounds like you’re doing something like:

@anvil.server.http_endpoint("/greet/:name")
def greet(name, **qs):
  return app_tables.test.search()

app_tables.test.search() creates a SearchIterator object, which is designed to do lazy loading of data from Data Tables.

This HTTP endpoint is dumping the SearchIterator to JSON and returning the dump. This is not intended as a built-in paginated API, in fact it’s not really something that we’d suggest doing.

SearchIterators can be sliced, so you can create a paginated response like this:

PAGE_SIZE = 10

@anvil.server.http_endpoint("/an-api-endpoint/:page")
def an_api_endpoint(page, **qs):
  # Clean the input
  page = int(page)
  if page < 0:
    raise ValueError('Page numbers begin at 0.')
    
  # Convert between page number and row index
  start = page * PAGE_SIZE   # page == 0 is the first page
  end = (page + 1) * PAGE_SIZE

  # Convert the SearchIterator to a list and each Row object to a dict. 
  return [dict(x) for x in app_tables.test.search()[start:end]]

requesting /an-api-endpoint/0 results in:

[
  {"testcol":"0"},
  {"testcol":"1"},
  {"testcol":"2"},
  {"testcol":"3"},
  {"testcol":"4"},
  {"testcol":"5"},
  {"testcol":"6"},
  {"testcol":"7"},
  {"testcol":"8"},
  {"testcol":"9"}
]

and requesting /an-api-endpoint/1 results in:

[
  {"testcol":"10"},
  {"testcol":"11"},
  {"testcol":"12"},
  {"testcol":"13"},
  {"testcol":"14"},
  {"testcol":"15"},
  {"testcol":"16"},
  {"testcol":"17"},
  {"testcol":"18"},
  {"testcol":"19"}
]

The slicing still needs to iterate over the table, so the response is O(n) - later pages will be slower than earlier pages. So for large data sets, you may still find this slow. There is a feature on the way that will make this scalable for very large data sets.

Also, on the business plan, you can have a persistent server that allows you to store state between calls, so if you have a large dataset you could use caching to paginate it scalably.

Click this link to clone my example app that has a paginated endpoint:

https://anvil.works/ide#clone:Q4B42M7MH2WQCSI7=VACWHMI263N4QHT73UYHMBU2

Let me know if that helps :slight_smile:

Shaun

1 Like