Programmatically access data about my Anvil apps

AWSOM · July 29, 2021, 5:43pm

What I’m trying to do:

I’ve been building a simple dashboard which allows a user to easily browse and (if authorised) launch one of many Anvil apps I’ve been tinkering with over the previous months. The cloned app below shows the basic concept.

As you’ll see from the clone, I’m currently hard coding the App Name and URL inside the client code, which is tedious and will need to be maintained if I change either the name or url in future.

So… I was just wondering if there was a way to programmatically fetch a list of my apps from https://anvil.works/build#page:apps along with other metadata such as the URL, date created etc. or even an Anvil API?

If not, please could this question be moved to Feature Requests?

What I’ve tried and what’s not working:

I’ve tried searching this forum for “my apps” “new apps” etc but no joy so far.

I’ve considered periodically scraping the data using selenium but that defeats the appeal of offering a web app in the first place and at my current skill level would take longer to code than manually copy/paste.

Clone link:

https://anvil.works/build#clone:J4ZFO5XHKWSVPK3K=4VG6JPDPZC2SVWMROS4VVDLF

AWSOM · August 10, 2021, 12:26am

I guess my question has drawn a blank from the community… Could this be considered as a feature request please?

p.colbert · August 10, 2021, 2:35pm

You’re in good company. If memory serves, at least two others have requested the same thing over the years.

stefano.menci · August 10, 2021, 3:04pm

Long time ago I played with an app that creates a dependency chart across all my apps.

I get the list of available apps with https://anvil.works/ide/apps. Then I use https://anvil.works/ide/apps/export/(app-id) to get the yaml of each app.

This would give me tons of information on each app, but something was still missing for a complete dependency chart (I don’t really remember what was missing, I haven’t used it in a while).

One problem with this is that you can only see the apps shared with the team and the apps on the account that is executing it, you don’t see unshared apps from other team users. To decrease this problem, I store in a table the result of every execution, so if user1 executes it first, then user2 executes it later, user2 will see the info cached by the first execution.

AWSOM · August 10, 2021, 6:32pm

That’s really helpful thanks so much @stefano.menci

p.colbert · August 10, 2021, 7:06pm

Holy smokes! I haven’t seen this documented anywhere, so I suppose it’s subject to change without notice. I would also expect that this only works after you successfully log in as an Anvil user. Even so…

I tried the 1st link, got a JSON listing of all of my apps! That’s incredibly helpful.

I wasn’t sure how to use the 2nd link. But if you append the app’s “id” (from the 1st link) immediately after the trailing “/”, the response is a copy of the corresponding app as a .yaml file. In a browser, it begins downloading immediately.

This is something we can automate and build on. Thank you, indeed, @stefano.menci !

stefano.menci · August 10, 2021, 9:10pm

Yes.
Here is a snippet of the old code used to manage the login (after stripping out tons of stuff, hopefully not too much):

import requests
LOGIN_URL = 'https://anvil.works/login'
URL = 'https://anvil.works/ide/apps'
@anvil.server.background_task
def update_dependency_json(user_email, user_password):
  session = requests.Session()
  session.post(LOGIN_URL, data={'email': user_email, 'password': user_password})
  r = session.get(URL)
  user = anvil.users.get_user()

The job is done by a background task because the app UI allows to display and browse the dependencies using cached data, while the background task is running. If you don’t need updates about the most recent apps, you can start exploring without waiting for the background task to be finished. As soon as the background task has finished, the UI will use fresh data.

stefano.menci · August 10, 2021, 9:27pm

I was sure had mentioned it, so I checked and… I was kind of right. Here is a snippet of the editor:

I edited it now, replacing <...> or <...> with (...), and now it’s readable.

aldo.ercolani · August 16, 2021, 10:29am

Thanks @stefano.menci this is incredibly helpful and on this I can build a tool to answer my own question posted on the forum thread: Search APP by string in code.
Once done I’ll update that post and link to this one.

aldo.ercolani · August 16, 2021, 1:43pm

Just tried this code to log into my Anvil account from server code:

import anvil.secrets
import anvil.server
import requests

LOGIN_URL = 'https://anvil.works/login'
URL = 'https://anvil.works/ide/apps'
  
@anvil.server.callable
def get_apps():
  session = requests.Session()
  r=session.post(LOGIN_URL, 
               data={
                 'email': anvil.secrets.get_secret('anvil_account_username'), 
                 'password': anvil.secrets.get_secret('anvil_account_password')
               }
              )
  print(r.cookies)
  for cookie in r.cookies:
    print("------------------")
    print(repr(cookie))
  return(r.json())

and:

when running from Anvil server environment I am not authenticated and I get the cookie:

Cookie(
  version=0, 
  name='anvil_first_referrer', 
  value='', 
  port=None, 
  port_specified=False, 
  domain='anvil.works', 
  domain_specified=False, 
  domain_initial_dot=False, 
  path='/',
  path_specified=True, 
  secure=True, 
  expires=3776604326, 
  discard=False, 
  comment=None, 
  comment_url=None, 
  rest={
    'SameSite': 'None', 
    'Secure,anvil_first_landing': '%2Flogin', 
    'Secure,anvil_first_landing_params': '%7B%22then%22%3A%22%5C%2Fbuild%22%7D',
    'Secure,anvil_first_landing_time': '1629120679713'
  }, 
  rfc2109=False
)

when running from Uplinked server code I am authenticated and I get the cookie:

Cookie(
  version=0, 
  name='anvil_auth', 
  value='aldo.ercolani%2540gmail.com%3B4MCNVRZWHONF36RVPWGQPN24', 
  port=None, 
  port_specified=False, 
  domain='anvil.works',
   domain_specified=False, 
  domain_initial_dot=False, 
  path='/', 
  path_specified=True, 
  secure=True, 
  expires=2838720755, 
  discard=False, 
  comment=None, 
  comment_url=None, 
  rest={
    'HttpOnly': None, 
    'SameSite': 'None'
  }, 
  rfc2109=False
)

when issuing that call from Postman, I am authenticated and I get 2 cookies:
- anvil_auth set in the same way of the uplinked code
- ring-session set to something like RG35VADZ3XTTAFYLLKGO7RQLBMTLIQ74%3DQWIBLCB36JGSIFFPB4SU745B

I can’t understand the differences in these 3 cases.

stefano.menci · August 16, 2021, 2:12pm

I can’t think (or remember) of any reason for the different behavior.

Looking at the code I realize now that I am using requests in a background task launched by a form that is already signed in with the same account that the app is signing in in the requests call.

So I think that the snippet I pasted earlier is confusing because the user = anvil.users.get_user() doesn’t care about you signing in with requests, they are two different sessions. One is the official app session compatible with the anvil.users service, the other is the parallel session running in requests.

Does this help?
Or do I really need to do some archaeological digging in the old app?

aldo.ercolani · August 16, 2021, 2:24pm

Yes I noticed that, but I don’t need the user signed in the front-end at all and I removed that line (and didn’t add the Users Service at all indeed).
I appreciate very much your offer to dig in the old code but honestly I doubt that could be helpful.
Question is: how can the same code making a requests call give different results.
Requests is 2.26.0 on both systems.
Actually I can’t understand what the anvil_first_referrer cookie stands for.