HashRouting: Routing, navigation with URL Hash

Availble as part of anvil_extras:

 from anvil_extras import routing

Or as a standalone dependency, which mirrors anvil_extras
Both available as Third Party Dependencies:

  • anvil_extras - C6ZZPAPN4YYF5NVJ
  • HashRouting - ZKNOF5FRVLPVF4BI

Edit:
This dependency has been through some editing, feature additions and testing and I think it’s good to be tested/used by anyone who is keen. thanks to @stefano.menci for collaboration on this!

You can find all the details here including a README.md which has examples how-tos:

GitHub - s-cork/HashRouting: HashRouting - a dependancy for anvil.works that allows navigation in apps

Here’s the dependency and an example clone link
https://anvil.works/build#clone:ZKNOF5FRVLPVF4BI=JHFO3AIV2GTM5ZP4FPFL3SMI

And the live example: https://hash-routing-example.anvil.app

aAde62kKf9

The aim is to abstract away routing from forms as seen in this forum post:

Deep linking and back button

It’s an extension of the docs navgation type app

Anvil Docs | Navigation

The Premise
Use decorators to extend the form classes and then everything should just work :wink:

  • @main_router decorator goes above your MainForm
  • @route decorator goes above any form that is loaded into the MainForm's content_panel
  • @error_form decorator goes above your error_form

NB: Much of the discussion below is no longer relevant for the dependency!
For up to date details see: GitHub - s-cork/HashRouting: HashRouting - a dependancy for anvil.works that allows navigation in apps

17 Likes
Summary

Hi Stu,

I tried to understand what’s going on and I need some help. Perhaps I don’t understand because I don’t know django/flask routing? The self.url_patterns = [...] seems to be the heart of the whole thing, but I don’t understand what it does and how to use it.
Question number 1: Can you please elaborate on how to define self.url_patterns?

I don’t see any on_history_back / on_history_forward event.
Question number 2: How does it work / what happens when the user clicks on the back / forward button?

In my experience a form similar to blog post always needs more than one blog table row, so my server function that retrieves the article details always receives a dictionary with the article row plus a few other pieces of information like the author, list of tags, etc.
I think this component would be more flexible if, instead of passing a table, I could pass a function. Something similar to:

@routing.route('blog_post', keys=['id'], on_update_form=BlogPostForm.get_blog_post)

I feel comfortable with adding such a change to my cloned version (once I understand how the whole thing works), but I think this would be a good improvement for any generic users, so…
Question number 3: Can you add an on_update_form parameter that expects a caller?

I tried going through the code and I found some comments that I don’t understand, like this one:

# push states are the result of loading forms from button/link clicks and does not need to be explicitly 
# replace states are the result of changing the URL hash 
# if you open a form and set replace state to true than this will replace the most recent form in history

Question number 4: What is the difference between push and replace? And do I care?

I have cloned and tried the example, and it seems to be working smoothly.
Question number 5 (maybe not for you): This thing is great (thank you!), why isn’t it in the component library?

2 Likes

Awesome! Thanks for having a go at using it!

Edit: Much of the information below is no longer relevant

Summary

Question number 1 : Can you please elaborate on how to define self.url_patterns ?

Some context:
The first time main router loads it will do the following.

  • run __init__
  • check the url hash (convert this to a url_pattern + url_dict e.g. #articles?id=3 → ‘article’, {‘id’,‘3’)
  • of course, the first url hash from the user will probably just be an empty string but maybe not if they always end up at #dashbard or whatever.
  • The main_router then loops through the list self.url_patterns
  • It checks each pattern to see whether the current url_hash has the correct string pattern and all the correct keys from the url_dict.
  • if there is no match then it will load your error form (if you have declared one in the main_router's init method. Else it will just raise an exception.
  • if there is a match then it will load the form you told it to load having declared this in self.url_patterns.

To declare self.url_patterns use the router.Path class which you get when you do from Hash_Routing import routing

    self.url_patterns = [routing.Path(Home, pattern=''),
                         routing.Path(ListBlogPostsForm, pattern='blogposts'),
                         routing.Path(BlogPostForm, pattern='blog_post', keys=['id']),
                         routing.Path(ListArticlesForm, pattern='articles'),
                         routing.Path(ArticleForm, pattern='article', keys=['id']),
                        ]   

NB: You will need to have imported all these forms earlier e.g.
from .BlogPostForm import BlogPostForm
Path takes the arguments

"""
form: Anvil Form to load, 
pattern: string pattern to match from url hash
keys: list of keys (strings) to match from the url dict
properties: dict - extra stuff that you normally pass to that particular form. Example the user e.g. properties = {'user': user}
"""

To declare an error form do this in the main_router init function like:

self.error_form = Error_Form


I don’t see any on_history_back / on_history_forward event.
Question number 2 : How does it work / what happens when the user clicks on the back / forward button?

This is in the JS Native libraries.
There are three JS functions that do the work.

pushFormState - //appends the state to the window history
replaceState -  //replaces the last window history state with the current state
onpopstate -    //when there is a navigation change or a url_hash change then this js function 
                //is called automatically - it then sends the call to the main_router method js_onpopstate

so on history forward and back or on a url change the js_onpopstate function is called…

There are two options -

either there is some state information that we stored in the window’s history when we originally loaded the form - e.g. the item property.
NB: If there is history state information then js_onpopstate was because of back/forward navigation

or

There is no history state information when js_onpopstate is called because this was a url_hash change (the result of set_url_hash being called - or manually typing a url in the addressbar) i.e. it’s new to the window history… and so no state information.

Both situations result in something similar:

  • main_router loops through self.url_patterns to match the url_pattern again
  • The correct form will be loaded
  • if it was from back_forward navigation additional properties will be passed to the form that were stored in the window history state information. (like the item)


@routing.route('blog_post', keys=['id'], on_update_form=BlogPostForm.get_blog_post)
Question number 3: Can you add an on_update_form parameter that expects a caller?

I think it’s possible for sure.

some more context:

if self.item is a dictionary (and can be converted to json) then it should just work. This dictionary will be stored in the history state so on back/forward navigation it should load with the form no issues.

The problem with a live table row is that you can’t store that in the window history because the history doesn’t know how to store this type of object. (That’s why Hash_Routing converts a live_table_row to its row_id when it’s stored to the history state)

Note that you should always aim to be able to get the properties you need for that form only by the url_hash/url_dict. This is because your user might navigate to that page as a new url by manually changing address bar.

Example:
You can see this in the gif above when I manually change the id.
My form loads with no item property just the url_dict and so I need to retrieve this article by id using a server call.
The code was:

if not self.item: 
      try:
        self.item = anvil.server.call('get_blog_post_by_id',self.url_dict['id'])
      except:
        raise Exception(f'no blogpost with id {self.url_dict["id"]}')

If you have an example where this breaks let me know and I’ll try and help/add the functionality.

Since I was thinking about forms as loading with a self.item property I didn’t see a reason to add an extra function call to distinguish between different types of navigation… Either my forms has a self.item property or it doesn’t so retrieve it… and all this could happen in the init function.



Question number 4: What is the difference between push and replace? And do I care?

Not really. Hopefully this is taken care of for you but there are some instances where it’s useful to know the distinction.
push appends to window history
replace replaces in window history
push is the default for link clicks or whenever you just load the form by doing

content_panel.clear()
content_panel.add_component(MyForm()) # the form will load and be appended to history

I have found it useful is when, say, you are on a page that the user can delete then you would want to load a different form and set replace_state=True to improve the navigation experience. Otherwise, they’d probably get an error from navigating back to the deleted page

def trash_button_clicked(self, **event_args):
  from ..DashboardForm import DashboardForm
  self.item.delete()
  get_open_form().content_panel.clear()
  get_open_form().content_panel.add_component(DashboardForm(replace_state=True))

I also did this above with the login form because I didn’t want the login form to be part of the history
so I loaded the dashboard with replace_state=True

I also found it useful to load a form with set_history=False if I was loading a form that was a route form but in this situation I didn’t want it to be part of the history…

More aggressive would be to use route=False which will just ignore all the behaviour that would usually come with a route form. (useful if say you want a form to behave like a component sometimes and a route form other times - say you load the form in an alert)

def button_click(self, **event_args):
  from ..ExampleForm import ExampleForm
  alert(ExampleForm(route=False)) # ignores the usual route behaviour

Keep me posted - curious to hear how it goes and what works and doesn’t…

4 Likes

edit - HashRouting no longer uses the state to store properties… forms are stored in a cache so the history stack for the app will just look like a load of urls and null history states…

Summary

I’m sure my explanations are a bit ropey…

Think of your browser history as a stack
each item in the stack has a "url"
each item in the stack also has a "state" which is a javascript object.

usually the history stack state object is not used and so the browser history stack might just be a load of urls and null state objects…

# how you might think about it
history_stack
[{'url': 'google.com', 'state': null},
 {'url': 'anvil.works', 'state': null}
 {'url': 'twitter.com', 'state': null}
 {'url': 'google.com', 'state': null}
]

the way Hash_Routing works is that it stores properties of the form in the history stack as a state object.

history_stack
[{'url': 'hash-routing-example.anvil.app/#articles',     'state': {'url_dict':{},'url_pattern':'articles'}
 {'url': 'hash-routing-example.anvil.app/#article?id=1', 'state': {'url_dict':{'id':'1'},'url_pattern':'article', 'item':item}}
 {'url': 'google.com', 'state': null}
]

You might also just go to different websites open up chrome dev tools and do
console.log(history.state) to see the state object at the top of the stack…

if you do this at https://hash-routing-example.anvil.app
you’ll see the state objects at the top of the stack as you navigate around the website…


Work with?

so when browser navigation is used. The top of the stack gets called and Hash_Routing will load the form with all the properties that were stored in the state object.

1 Like

This a great explanation!! Thanks Stu, this will help when I trial this technique in my app. Much appreciated.

2 Likes

Ok, so here is the summary of the things I didn’t know and I finally understand (with the exception of the last two items, it’s still dark over there).

Summary
  • some background about how URLs work usually out there and in Anvil with this module:

    == How the URL is commonly used ==
    [   specifies document content on server side    ] [scroll position or other info used on the client side]
    [   domain   ] [  path   ] [    query string     ] [ hash  ]
    sub.domain.com/path1/path2?key1=value1&key2=value2#id_of_tag
    
    == How the URL is managed by Anvil with this routing module ==
    [never changes] [ managed by the routing module ]
    [   domain   ]  [ page  ] [ anvil query string  ]
    sub.domain.com/#form_name?key1=value1&key2=value2
    
  • window.history - a javascript object containing the list of URLs visited

  • state - a generic javascript object that can be associated to each URL in window.history and contains info about the state of the page

  • hash - the text following # on the URL

  • self.url_patterns - must be defined on the main module to tell the router module which pages are valid

  • pattern - a text identifying a form (usually similar to the form name)

  • table - used to get a row from the server and populate form.item when the user navigates back to a previously visited page

  • keys - this is sometimes a list and sometimes a key?

  • properties - is this the same as the **properties defined in the form?

While looking at how the routing module works, I kept thinking that the whole thing seems unnecessarily complex. Perhaps my vision is oversimplified because I haven’t used it yet, please let me know if I am missing something. Here are the three things I kept thinking about:

  • The automatic population of item is not necessary
    • The form already knows how to do it because all the required info is in the hash (which makes my third point useless)
    • It makes both the decorator and form code more complex: the decorator needs to manage the form item and the form needs to check if the decorator has already taken care of it
    • One generic server function that loads one row from one table is seldom enough (never in my experience)
    • I would use window.history for the minimum (only the URL) and cache the forms in a global Python dictionary, using the URL as a key; it is faster to get the form back from the global cache dictionary than transforming some data, saving it in the history state, then getting the history state back, transforming it back to Python format and telling the form to use it to rebuild itself
  • I don’t see the added value of self.url_patterns
    • I would like to use a simpler @routing.route('article') decorator in front of every form that should be managed
    • When the app starts and imports all the forms, a global dictionary should be populated automatically as the decorators are executed (rather than explicitly defining self.url_patterns inside the main form)
  • I would like to add the management of the query_close event
    • If an app uses this routing module, then all the form management should be done via this module (no more get_open_form().content_panel.add_component())
    • The routing module knows when a form is about to be unloaded, so it can (try to) call its query_close event and decide whether to interrupt the process of closing this form and navigating to the new one
    • The query_close event should be called when:
      • The app navigates to another form
      • The URL changes for any other reason
      • The browser tab/window is closed

Today I had no time to spend on it, but I will eventually find the time to play with it.
Thanks again for showing us this module and for your help.

1 Like

Edit - Much of the discussion in this post is no longer relevant

Summary

table - used to get a row from the server and populate form.item when the user navigates back to a previously visited page

This is definitely optional. It’s only really for a very simple case when your form is just a row from a table.
A lot of the Anvil Examples display a form where the item is a table row and I was trying to work it into the system. I based the routing initially on this example thinking how I might go about it…

https://anvil.works/learn/examples/hr-workflow

But maybe it overcomplicates the process…
In my main app I actually never used this function I just retrieve my item from a server call whenever the form is loaded based on the url_dict


keys - this is sometimes a list and sometimes a key?

keys is always a list - it might be a list of 1 item (or an empty list) - but always a list. (could also be a tuple)
It’s also optional since your url might be text only.
keys was designed for a situation like article?id=1
Essentially - A form that needs to get a unique item based on parameters.
I use it a lot in my main app in the same way.

Maybe this is trying to be too clever. But let’s say that you open an article form because someone set the url to be /#article?id=3
then here it’s obvious that the url_dict should be {'id':3}

But what if you just do:

get_open_form.content_panel.clear()
get_open_form.content_panel.add_component(ArticleForm(id=3, user=user))

or even

get_open_form.content_panel.add_component(ArticleForm(item=item))
# and item includes item['id'] --> 3

then routing will go about searching the properties you passed to the form and (assuming) your form was sent that property - which it should, otherwise, what was the point in making it a key in the first place…?

It will then create the url_dict based on these properties you passed to the form…

But maybe you change the url_dict, because…
I did something in my main app like this - let’s say you have a create new article button
and do:

def create_new_article_button_click(self, **event_args):
  get_open_form.content_panel.clear()
  get_open_form.content_panel.add_component(ArticleForm(id=None))
...
@routing.route('article',keys=['id'])
class ArticleForm(ArticleFormTemplate):
  def __init__(self, **properties):
    init_components(**properties)

    if not self.url_dict['id']:
      # create a new article
      self.item = anvil.server.call('create_new_article')
      self.url_dict['id'] = self.item['id']
    else:
      self.item = anvil.server.call('get_article', self.url_dict['id'])

in other words you’ve changed the url_dict from in the init_method
You wouldn’t want the url to be /article?id=None
so after your form init method Hash_Routing will update the url_dict to ensure that the url_hash matches the url_hash you intended…


properties - is this the same as the **properties defined in the form?

yeah this is the same as **properties.
So it’s basically a dictionary of all the kwargs you might usually pass into a particular form.
Again you might never use this option. I haven’t, in my main app since there aren’t any properties that I always pass to a particular form.


The automatic population of item is not necessary

agree - you could totally disregard this in your routing behaviour. I do in my main app.
My items are typically Live table rows and I just don’t provide a table.

I can imagine that if the item is a dictionary then this might be more annoying so we’ll cross that bridge when it comes… :upside_down_face:

But I can’t foresee it being too much of an issue since you can just assign your self.item in the usual way based on the url_dict and all will be fine…


I don’t see the added value of self.url_patterns

  • I would like to use a simpler @routing.route('article') decorator in front of every form that should be managed
  • When the app starts and imports all the forms, a global dictionary should be populated automatically as the decorators are executed (rather than explicitly defining self.url_patterns inside the main form

Ooooh I like that. I’ll give it a try later!


I would like to add the management of the query_close event

  • If an app uses this routing module, then all the form management should be done via this module (no more get_open_form().content_panel.add_component() )

I think this point is separate point…
That sort of is the case in that you can just do set_url_hash('article') and everything is taken care of by routing. You don’t need to ever do get_open_form().content_panel.add_component() if you don’t want to…

Similarly, if you do call get_open_form().content_panel.add_component() you don’t actually need to do get_open_form().content_panel.clear() before it as Hash_Routing should just clear the current form when a new route Form is opened…

In the example app I shared I think I only called this once when I loaded the ArticleForm passing the item as a kwarg and I didn’t call the clear method…
but I could also have done.

set_url_hash(f"article?id={self.item['id']}")

  • The routing module knows when a form is about to be unloaded, so it can (try to) call its query_close event and decide whether to interrupt the process of closing this form and navigating to the new one
  • The query_close event should be called when:
  • The app navigates to another form
  • The URL changes for any other reason
  • The browser tab/window is closed

I think I follow…
something like the form hide event but it should be raised just before the form is cleared?

# pseudo code
on navigation
current_content_panel_form.raise_event('x-query-close')

then in that form you could do something like

self.set_event_hadler('x-query-close', self.query_close)

def query_close(self, **event_args):
  if unsaved_changes:
    if confirm('are you sure you want to close there are unsaved changes'):
      pass
    else:
      cancel_navigation_somehow....?

What I was doing for the typical save changes was just saving the changes on the form hide event.
Which worked well for me but I can see how this would be important for other apps…

Also note that the app has no control over the user deciding to close the tab because that’s outside of the window (I think)

1 Like

Update:

As you suggested @stefano.menci - there is now no need to set self.url_patterns in the main_router this is all taken care of with global variables and much nicer!!!

Now all you need to do is put the decorators above the forms!
And everything should work! :crossed_fingers:

I’ve updated the clone links to reflect this!

At some point I’ll edit the first post text to reflect the changes!

1 Like

So - @stefano.menci - I’ve done a major rehaul building on your excellent suggestions

Summary

I’ve tried it on my main project and with a couple of tweaks to my navigation style, it works great.

The major change in thinking is that really you should only load forms with:
set_url_hash()

and never:

get_open_form().content_panel.clear()
get_open_form().content_panel.add_component(Form())

If you try to do the above you’ll get an error unless you do Form(route=False) i.e. you don’t want to use the route behaviour! Same comments as earlier - route = False useful if you want to put a route Form in an alert and behave like a component.

I’ve added a method - which also takes care of loading a route form:

routing.load_form(Form, **properties)

which is useful for loading a form with properties and still uses the cache if possible

you can see in the example the difference between the title_link_click in the ListArticlesForm and ListBlogPostsForm ItemTemplates

One uses

set_url_hash(f'blog_post?id={self.item["id"]}')

and the other uses

routing.load_form(AritcleForm, id=self.item['id'], item=self.item)

Let me know your thoughts - the clone links above should be up to date with changes… @david.wylie :smile:

1 Like

Wow, I was getting ready to create my own decorators inspired by yours, but at this point I think I will try to use yours as is.

I will try to integrate it with my first app and let you know how it goes.

Thank your for making it, and even more for adopting my suggestions.

1 Like

For anyone keen :smile: / brave :man_superhero: enough to use this dependency - I’ve moved the details to a GitHub repo.

https://github.com/s-cork/HashRouting

There are explanations and lots of examples of code in the README.md that will be up to date and I hope will make it easy/simple for anyone to try! (thanks @stefano.menci again for the collaboration efforts)

Let me know if you have a play / get stuck and how I can help. :smile:

Lots of the above discussion is no longer relevant - so no need to search through this thread.

3 Likes

When I get a spare hour or two, I’m gonna give this a good bashing. My apps come in for a bit of criticism over the lack of a working back button, just because “all the others [my rivals] have one”.

Now, where’s that spare hour or two …

2 Likes

@stucork, instead of writing my own decorator inspired by yours, you ended up with rewriting your decorator just the way I like it. Thanks!

Now:

  • the back and forward buttons work as expected by any new user
  • users can copy the URL to any page
  • I finally have the query_close event that I requested long time ago (you can clone the example app and see how it works)
  • automatically updates the browser tab title
  • etc.
3 Likes

I’ve been too busy to be much part of this, so I’m late to the party. But I just had to create a small web site to allow students to register for a programming camp, and took the hash routing for a test drive.

Well done! It works well and is easy to use. It’s everything I’d have wished for, except for canonical URLs (which Anvil needs to change to support).

Thanks for your hard work!

5 Likes

I second @jshaffstall’s report. It was much easier to implement than I expected. Error messages are clear. Github documentation is great. Thank you!!!

3 Likes

Hey so I have been working with the dependency in my app from your GitHub page.

First of all, it’s awesome, so nice work!

Second, my app uses a custom HTTP endpoint in a server module to handle an OAuth callback URI. I want users to get redirected to a particular place in my app after the callback happens, but when I try calling routing.set_url_hash('authentications') from server code, I get:

<class ‘AttributeError’>

module ‘anvil’ has no attribute ‘js’

Google.py

131

I suppose what I am wondering is if there is a way to direct the app to a specific url hash from server code?

1 Like

Awesome! pleased it’s working out :smile:

I don’t think there is a way to redirect from server code. It’s only intended for client side
i’ve checked and the anvil builtin set_url_hash also doesn’t fire…

Perhaps you could use some sort of background task and check its state from the client.
The background task can check whether the user has an oauth key in their datatable row, at a sensible regular time interval.
When the background task state return true you can kill the background task and set the url hash on the client side…

I actually figured out a real-time workaround! When the oauth callback happens, I set a specific user attribute to true. Next time they log in, they are redirected to the appropriate page, and then the oauth attribute is cleared from their user record.

3 Likes

@stucork, hopefully this is an easy thing to tell me what I’m doing wrong.

Just started using the errorForm feature of the library. When I go to a route that doesn’t exist, it does successfully direct me to the error form instead. But, I also get a runtime error: KeyError: authorsss does not exist

On my site, authors is a valid route, so I just added a few characters to make it invalid to test the error redirection.

My routing error form is this:

from HashRouting import routing

@routing.error_form
class RoutingError(RoutingErrorTemplate):
  def __init__(self, **properties):
    # Set Form properties and Data Bindings.
    self.init_components(**properties)

  def form_show(self, **event_args):
    routing.set_url_hash('')

And I import that form in my main routing form. The redirect on error to the home page works just fine, except for that pesky KeyError showing up.

What am I doing wrong?

1 Like

Think you might have spotted a bug actually. If you go into your version of the dependency and change line 116 to

else if _anvil.get_open_form():

Let me know if that fixes it and I’ll make the fix.

2 Likes