Ignorance may be Strength : February 2025

Thursday, February 13, 2025

Client-side Routing in a Javascript Application

This is a small thing that I'm just doing here so that I can nail down some issues in my own mind. For me at least, there is nothing really "new" here, just a consolidation of things that I think I know into one place.

Client-side routing is key to having "single page web applications" behave like "normal web applications" in that: they should you what you expect when you visit a URL; the URL updates as you navigate through the app; you can bookmark individual pages within the app; and the back button works in a "logical" way, if not necessarily the way in which you would expect.

I don't think perfection can currently be achieved though. In particular, annoying as it is to me, if you specify a "brand new" URL (either by typing it in the address bar or clicking on a bookmark), it seems to me that the browser takes over and will not let the current application intercept the new path within its own context. If it turns out that I have just missed something - or if it changes - I will issue a retraction. In the meantime, let's get on with what I do know.

I have put a "live" version of this on a website, jshistory.gmmapowell.com, which has all the versions next to each other (called v1, v2, etc.). You can look at the examples I've shown here by visiting those pages. If you look at the git repo, it's all in one directory, but the tag (e.g. JSHISTORY_V3) will tell you which is the corresponding version on the website. I'm doing that somewhat manually, so hopefully it all stays in sync.

The Basic Setup

I'm not going to do anything too amazing. In particular, when we do something to "navigate" to a "page" (by chainging the URL), the actual page layout is not going to change. All that is going to happen is a log message is going to be put in the box in the middle of the screen. The box should just have more and more messages in it - if it ever goes back to just application loaded, that will be because the application has been reloaded from the server, which is something we are trying to avoid.

The HTML outlines our basic setup and page layout:

<!DOCTYPE html>
<html>
  <head>
    <title>Experiments with History and Paths</title>
    <link rel="stylesheet" type="text/css" href="css/history.css">
  </head>
  <body>
    <div class="main">
        <div class="title">Experiments with history and paths</div>
        <div class="logging"></div>
    </div>
    <script src="js/history.js" type="module" charset="UTF-8"></script>
  </body>
</html>

JSHISTORY_V1:jshistory/index.html

Apart from the boilerplate, this specifies that we want to load the CSS file css/history.css and the Javascript module js/history.js. I'm not actually using modules, but I've got into the habit now.

The actual HTML consists of a "main" frame, which is going to take up the whole screen and have a blue background; a title element across the top; and an inner frame with a lighter background for all the logging information to go in.

The CSS is CSS and basically passes without comment, except for a few notes:

body {
    margin: 0px;
}
.main {
    width: 100vw;
    height: 100vh;
    background-color: lightblue;
}
.title {
    text-align: center;
    font-size: 20px;
    height: 10%;
    padding-top: 10px;
}
.logging {
    position: relative;
    background-color: aliceblue;
    height: 80%;
    margin-left: 10%;
    margin-right: 10%;
    padding: 1px;
    overflow-y: scroll;
}
.message {
    margin: 10px 10px 0px 10px;
    border: solid black 1px;
    padding: 7px;
    font-family: monospace;
}

JSHISTORY_V1:jshistory/css/history.css

The body of an HTML document is set by default to have a margin of 16px because that gives white space around a "just text" document. But it's ugly if you want a solid background, so it's best to override it.

The logging class has a position of relative, not because we want to move it from its default position, but because we want it to become a "positioned" element so that it is the "positioned" parent of the elements we will add to it (I will explain why below). It also has a padding of 1px to avoid the collapsing margins problem in which the top margin of the first nested element is applied to the outer box. The overflow-y is set to scroll so that this box will scroll (rather than the whole page) when we have too many messages.

The message class is not used directly in the HTML but is applied to the messages we generate in JavaScript. It has a top border but not a bottom border since these will be collapsed into each other anyway, so it's better to be explicit about what we expect to happen.

Turning to the JavaScript code:

var logArea = document.querySelector(".logging");

function scrollTo(elt) {
    logArea.scrollTo(0, elt.offsetTop + elt.offsetHeight - logArea.clientHeight);
}

function write(msg) {
    var elt = document.createElement("div");
    elt.className = "message";
    elt.append(document.createTextNode(msg));
    logArea.append(elt);
    scrollTo(elt);
}

write("application loaded");

JSHISTORY_V1:jshistory/js/history.js

This first makes sure that it can find the element tagged logging in order to write to it. The scrollTo function enables us to scroll to any given element within logging. The offsetTop returns the distance from the "inside" of the first ancestor element which is explicitly positioned (which is logging because we specified its position: relative; in the CSS above) and the offsetHeight is the height of the element. The clientHeight of logArea is the inside height of the containing element and thus the calculation we do here should scroll the contents of the container to align the bottom of elt with the bottom of the container.

The write method takes a text message, creates an element and a text node, styles the element to be of class message, and then appends it to the logArea and scrolls the logArea to ensure that it is showing.

This all works fine as is, but in order to be able to specify "interesting" paths as URLs and still return the correct webapp HTML, we need to do a little monkeying. Since I'm using Apache, this goes in the .htaccess file using the rewrite_mod. I'm not going to explain this in detail here, but this is the file:

RewriteRule (js/.*) 1 [L]
RewriteRule (css/.*) 1 [L]
RewriteRule ^.*$ index.html

JSHISTORY_V1:jshistory/HTACCESS

The third line is the one that says "rewrite any path to index.html", but we don't want it to be any file. If the path is js/ or css/, we will want to return those files specifically. While this may seem "obvious", it's easily overlooked, and I did just that this morning and wasted half an hour trying to figure out why the mime type on my JS file was wrong and unacceptable (it wasn't, in fact, just the mime type that was wrong: it was the whole file).

The first two lines handle the js/ and css/ paths and basically say "serve the file the user asked for" and the third line says "anything else, send back index.html". This file goes under v1, etc. so doesn't apply to favicon.ico and is separate for each version of the webapp. Although I've put it in the repository as HTACCESS, my deployment script renames it to .htaccess.

You can check all this works properly by visiting these pages:

the obvious one - jshistory.gmmapowell.com/v1
a nested link - jshistory.gmmapowell.com/v1/hello/world

Adding Some Links

So we have an application, but it isn't very interesting. What we need is to be able to attempt to navigate around the site. So I'm now going to add a "button bar" to the top (like a toolbar or whatever), and I'm going to put four different gadgets in it.

<!DOCTYPE html>
<html>
  <head>
    <title>Experiments with History and Paths</title>
    <link rel="stylesheet" type="text/css" href="css/history.css">
  </head>
  <body>
      <div class="main">
        <div class="title">Experiments with history and paths</div>
        <div class="buttons">
            <button onclick="url('first')">first</button>
            <button onclick="url('down/again')">down/again</button>
            <a href="linkto">linkto</a>
            <span class="likelink" onclick="url('textlink')">textlink</span>
        </div>
        <div class="logging"></div>
    </div>
    <script src="js/history.js" type="module" charset="UTF-8"></script>
  </body>
</html>

JSHISTORY_V2:jshistory/index.html

Two of them are buttons, called "first" and "inner/again". These will attempt to link to "/first" and "/inner/again". Then I have a "regular" anchor link called "linkto". And then I have something that looks similar, called "textlink", which is styled to make it look like a hyperlink but is actually just text, for which I have added a click event.

Apart from the "regular" anchor link (linkto), these all give the appearance of working: the message appears in the logging area as you would expect. What doesn't happen is that the URL does not change. We need to fix that. The regular anchor link causes the page to refresh and go back to just showing "application loaded". This is always a clue we have done something "wrong". (This needs quite a bit more effort and we will return to that later; for now, ignore it.)

The CSS for this isn't interesting. And on the JavaScript side, all I've done is added the url function, which for now just logs the message:

function url(goto) {
write(goto);
}

JSHISTORY_V2:jshistory/js/history.js

Introducing the History API

Everybody talks about the History API as if it is some amazing thing. It isn't, and it's very frustrating what it can't do. I understand that this is for security reasons (it doesn't want "this app" being able to see where else the user has been) but from the perspective of a webapp developer, it would be nice if it solved that problem by only telling you about the user history within your application, but told you everything within your application.

Winding back to the beginning, the History API is basically a programmatic version of the "back" and "forward" buttons on your browser. You can call history.back() and history.forward() to literally simulate the pushing of those buttons. You can also call history.go(n) where n is an integer which is negative to go back a number of pages, positive to go forward, or 0 to reload the current page (not something we want to do, obviously - we are trying to avoid reloads).

You can also see how big the history is using history.length, but this isn't as useful as it might first appear, since you cannot see where in that you are, and any number of entries (especially at 0 and history.length-1) might well not be in your application.

So that might all be useful in certain situations, but not for us. The final property on history is called state and by default is null. We will look at how to set it in a moment, and it may be useful to see the current state to decide on a course of action. What you can't do is see the state or url for any other page in the history stack.

But you can manipulate the stack. You can add items to it by calling pushState and change the current top item using replaceState. Because of the lack of transparency from the API (even in the debugger), it's hard to tell exactly what happens when you do these, however, and I don't feel the documentation makes it as clear as it could. But hopefully we can experiment and attempt to characterize the API. For example, when we push a new state, does it insert it or clear the rest of the stack and replace that with the new entry?

So let's make the url function we are already calling using pushState to try and update the URL (we should see the URL in the address bar change) and report on that and the new length of the stack. We'll also get it to store something in the state variable: an auto-incrementing ID that we can then read back. This will also appear in the tracing.

There are complications in JavaScript around dealing with relative paths and resolving them which I don't want to get into yet (although I may come back to it). So to avoid that, I am going to build an explicit path. To do that, I need to first figure out which version you are using, so I added this to the top of my JavaScript:

var pushid = 1;
var version = figureVersion(window.location.pathname);

function figureVersion(path) {
    var idx = path.indexOf('/', 1);
    if (idx == -1) {
        return path;
    } else {
        return path.substring(0, idx);
    }
}

JSHISTORY_V3:jshistory/js/history.js

This is my first attempt to implement url, and I was a little surprised at what happened:

function url(goto) {
    var url = window.location;
    url.pathname = version + '/' + goto;
    write("pushing " + pushid + " to " + goto + ": url = " + url);
}

JSHISTORY_V3:jshistory/js/history.js

You can try it for yourself by visiting jshistory.gmmapowell.com/v3 and clicking on first. That's right. The page reloads, which, as I've said before, is always a losing lottery ticket in the single-page webapp game.

The reason, of course, is that assigning to window.location forces a refresh. And even though I'd assigned window.location to another variable, it isn't what you call it that matters, but what value it has. It looks like a URL, but it's actually a more active thing, a Location. So let's try that again.

function url(goto) {
    var url = new URL(window.location.href);
    url.pathname = version + '/' + goto;
    write("pushing " + pushid + " to " + goto + ": url = " + url + "; stack = " + window.history.length);
    window.history.pushState({unique: pushid++}, null, url);
}

JSHISTORY_V4:jshistory/js/history.js

That's better. By copying the URL before we attempt to write to it, we have detached it from the Location object. We can now set the URL in the address bar by calling pushState. And we are done: all the buttons (except the anchor link) work in the way we expect and report an ever-increasing stack size.

Not that we're done yet.

(If you're wondering, the null middle argument to pushState is a weird beast. It is described as title but is not used as the window title. In fact, it doesn't seem to be used at all. So I always set it to null.)

Handling the Page Load

There are two things we need to consider on the initial page load. Recording where we are, and putting something on the stack to go back to.

In some ways, everything we're doing here is a classic "synchronization" or "caching" problem. We have two things that need to be kept in sync, but they are independent, and we need to make sure that we know which has changed and then change the other. Above, we looked at what happened when our app wanted to change the current route: and then we updated the address bar to follow suit. Here, we are looking at when the app first loads with a specific route set; then we need to update our app accordingly. We are just going to log a message about where we are, but a "real" application would obviously need to show the desired content for the specified route.

If the user selects a route "within" our app, it is perfectly possible that they would then expect that "going back" would take us somewhere else in our app (they might also expect to go back exactly where they came from; it's hard to know what users expect). We will assume that they want us to completely take over and have "back" go back to our home page. This means we need to put an extra element into the history stack, "below" as it were, the one that we're at. You can't actually do that, so what we are going to do is to "replace" this one with our home url, and then push the current url, and log both of these actions.

Of course, if a user selects the home page of our app, we just log that and we're done.

On the upside, there is no problem knowing when the page has loaded: it is the only time when it runs through all of our initialization JavaScript (which is why the "Application Loaded" message only comes out once).

Clear? Then let's begin by adding a new method call at the end to our "initialization" phase:

write("application loaded: " + version);
handleLoad(window.location.origin, window.location.pathname.substring(version.length));

On loading, we disassemble the window.location and pass the origin (https://jshistory.gmmapowell.com/) and the path (shorn of the version identifier) to a new method handleLoad:

function handleLoad(origin, route) {
    if (route == "/") {
        write("loaded with root; setting state to " + pushid + "; stack = " + window.history.length);
        window.history.replaceState({unique: pushid++}, null);
    } else {
        var nested = new URL(window.location.href);
        var root = new URL(window.location.href);
        root.pathname = version + "/";
        write("loaded with route " + route + "; replacing with " + root + " as " + pushid + "; stack = " + window.history.length);
        window.history.replaceState({unique: pushid++}, null, root);
        window.history.pushState({unique: pushid}, null, nested);
        write("pushing " + nested + " as " + pushid + "; stack = " + window.history.length);
        pushid++;
    }
}

JSHISTORY_V5:jshistory/js/history.js

This divides into two main cases: has the user asked for the "root" of the application (the home page, as it were) or a nested page?

Not much needs doing if this was the root page: we replace the state with a state object (in lieu of the default null) but the path stays the same.

On the other hand, if the user has selected a nested page, we want to replace the current state AND url with an appropriate state and the "root" url; and then push a new state and the requested url. I want to repeat that this is a choice: we could just update the state or we could push multiple in-between urls representing "the most common way" of getting to the request url. This is only really possible to determine in the context of both the application and the intended user experience.

One point of note: the moment you replace the state (or push a new one), the value of window.location is automatically updated, so it is possible to capture the value of nested before we perform the call to replaceState.

I also made a couple of minor changes to try and make the value of pushid always line up with the message displayed in the box.

function url(goto) {
    var url = new URL(window.location.href);
    url.pathname = version + '/' + goto;
    window.history.pushState({unique: pushid}, null, url);
    write("pushing " + pushid + " to " + goto + ": url = " + url + "; stack = " + window.history.length);
    pushid++;
}

JSHISTORY_V5:jshistory/js/history.js

Handling that Pesky <a> Link

It is, of course, possible to build your complete application without using any explicit <a> links. It is also possible to do it with them, and it would be good if our application could cope with that choice. Since we have an <a> link which doesn't currently work, let's try and wire it up.

Once processing of an <a> link has started, it's too late and there's basically nothing we can do to bring ourselves back from the brink. Instead, we need to target all the <a> links in the document and make sure that they call our function, not exhibit the default behaviour. We can do this by adding a click event handler onto the document, and then checking if it's a local <a> link. The code is not that hard:

function captureLocalAHref(origin) {
    document.addEventListener('click', (ev) => {
        var t = ev.target;
        if (t.tagName === 'A' && t.origin === origin) {
            ev.preventDefault();
            url(t.pathname.substring(version.length+1));
        }
    });
}

JSHISTORY_V6:jshistory/js/history.js

The listener checks that the link being clicked is an <a>, and that the origin in the referenced URL matches our origin (if it doesn't, we can't handle the link locally). If both these criteria are met, preventDefault is called on the event to make sure that the link is not automatically followed, and then our url method is called to perform the local routing.

We install this handler at the end of the script:

captureLocalAHref(window.location.origin);

JSHISTORY_V6:jshistory/js/history.js

Can I Just Go Back?

If you try pushing the back button on your browser, you should see the URL in the address bar change and, magically, the page does not reload. Once you've gone back, you should be able to go forward, and everything should behave just as you wanted it to. At least, as far as the address bar goes. But the logging area is not being updated. Since this is a proxy for updating the content on the screen, this means that while back and forward update the URL, they would not cause the content on the screen to change. Not ideal.

Let's fix that.

There is a window event called popstate which we can capture to handle this. We can add a handler at the end of our script like so:

capturePopstate();

JSHISTORY_V7:jshistory/js/history.js

This will then add the event handler onto the window, like so:

function capturePopstate() {
    window.addEventListener("popstate", (ev) => {
        ev.preventDefault();
        moveTo(new URL(window.location.href), ev.state);
    });
}

JSHISTORY_V7:jshistory/js/history.js

The event handler stops the "default" back action from happening, and then calls the moveTo function:

function moveTo(url, state) {
write("moved to " + state.unique + " at " + url + "; stack = " + window.history.length);
}

JSHISTORY_V7:jshistory/js/history.js

Note that this is a new function, not the url function we have been using up until now. In the real world, these two functions will have a lot in common that should be extracted (in fact, moveTo might well be the extraction). Specifically, the url function wants to change the History stack; moveTo does not - it just wants to use it. Our implementation simply logs another message that it has moved up or down the stack and reports the unique id it found there along with the current url (which is passed in from window.location). It also reports the current length of the stack, which should never be different from what it was before the method was called.

(It is at this point where you are frustrated that you cannot see more information, even just how many frames are above and below you in the stack, let alone whether they are part of your app or not. The unique id in this example is intended as a proxy for that, and I think it could be extended to keep track of where you are in the stack if you were that way inclined.)

This is worth playing around with for a while, because it's really quite instructive. Start here and click on a couple of the links. You should see the URLs and unique numbers pop up. Press back a couple of times and check that the unique numbers match the same URLs as they did going up. Now press a link and then back. You'll notice that a gap has now appeared in the unique numbers, and that the stack size has gone down. "Forward" history is destroyed when you click on a new link.

Suppressing History

There are occasions when you don't actually want all of your history to be present. As an example, if you are in a "shopping cart" experience, you want the different pages to be available while you are making your selections, etc. but once you have finished with the cart you want the whole thing to be suppressed and erased from history and just be left with a confirmation screen. While I'm not going to change the overall shape of the application, I'm going to add another row of buttons that shows how this could work.

You can see this here. (For full disclosure, this is pointing to the fully working version, so that I don't lock up your browser. But v8 does exist if you want to look at that as well; but it will lock up on confirm.)

The idea here is that you push the cart button (on the main row) when you want to enter the cart experience. To make it somewhat realistic, it is only when you do this that the second row of buttons light up. You can then go between the details, payment and review steps to your heart's content, and back works all the way through this. But when you push the confirm button, all of those steps are collapsed and you can't get back to any of the intermediate steps. Pushing the back button takes you back to where you were when you pushed cart.

How does this work? Well, it all ends up being a little complicated. Certainly more complicated than I had expected.

Part of that complication is due to the fact that I want the buttons to light up and turn off at the appropriate moments, but part of it is because of the History api.

Let's start with the UI: we can add the extra buttons to the HTML:

        <div class="buttons">
            <button onclick="url('first')">first</button>
            <button onclick="url('down/again')">down/again</button>
            <a href="linkto">linkto</a>
            <span class="likelink" onclick="url('textlink')">textlink</span>
            <button onclick="launchCart()" class='launch-cart'>cart</button>
        </div>
        <div class="cart buttons">
            <button onclick="cartStep('details')" disabled>details</button>
            <button onclick="cartStep('pay')" disabled>pay</button>
            <button onclick="cartStep('review')" disabled>review</button>
            <button onclick="cartStep('confirm')" disabled>confirm</button>
        </div>

JSHISTORY_V8:jshistory/index.html

When the cart button on the original row is pressed, the launchCart method is invoked:

function launchCart() {
    cartStep("details");
    var s = window.history.state;
    s.launchCart = true;
    window.history.replaceState(s, null);
}