Metrolink Application on the Watch

So I now have a working widget on the watch. Just not the one I wanted.

There are a number of ways I can go from here, but it amounts to understanding how the app I have works (and by extension the platform), and then to figure out how to build the app I want.

On one hand, I like working with "platforms", inasmuch as all the boilerplate code is done for you, and you are then just asked to provide the missing pieces. On the other hand, the flow of control is out of your hands, and if something "doesn't happen", it's not at all clear why not: did I not configure some XML correctly? did I not place my code in the right place? did it not implement some interface? was it called and then did it cause an error? And very often it's hard to investigate and debug.

So my first step is going to be to try and add in some debugging statements to the current code to see if the simulator will spit out information about what it's doing and in what order. I'm not sure this will help me any more than reading the documentation, but at least it will be there as I try to put my own logic in place.

The Hello, Monkey C! page suggests that it is possible to print values to "the debug console" by using System.println. I copy their code into getApp() in metrolink-watchApp.mc:

function getApp() as metrolink_watchApp {
  System.println( "get app" );
  return Application.getApp() as metrolink_watchApp;
}

This gives me an error on build:

/Users/gareth/Projects/IgnoranceBlog/metrolink-watch/source/metrolink-watchApp.mc:27,4: Cannot find symbol ':System' on type 'self'

It turns out that System needs to be explicitly referenced at the top of the file:

import Toybox.Application;
import Toybox.Lang;
import Toybox.WatchUi;
using Toybox.System;

(it's interesting to me that this uses using where all the lines already there use import - what's the difference?).

But even fixing this does not produce any output. The simulator appears, and the window appears, and the monkey appears, but no output appears.

I put a breakpoint on this line and try Start Debugging. The breakpoint is not hit, and the output does not appear. What else could be going wrong?

My experience with Visual Studio in the past - and particularly with languages the compile to binary form - is that they use two versions - release and debug - and (obviously) you can't debug a release build. I've seen some references to this here, including when "building for device", so I think that may be related. It's also worth noting that the output I am getting includes the message "building for device", so it's not clear that the simulator version is updating at all. Looking in the bin/ and vivoactive/ directories however, the version which is being updated is bin/metrolinkwatch.prg and its corresponding .debug.xml. There is also a test_ version which has not been updated, but that is test and not debug, so I am going to ignore that for now.

Let's try something else. The configuration has a property stopAtLaunch which by default is false. I am going to set that to true and see what happens. Wow! That works. And puts the breakpoint at line 8 in my code, which is the function initialize. So maybe my biggest problem was just guessing incorrectly that getApp() was the root of the program. In which case, I wonder what it does and what it's there for.

OK, I'm going to step through this, and then try adding some more println statements and see what happens.

import Toybox.Application;
import Toybox.Lang;
import Toybox.WatchUi;
using Toybox.System;

class metrolink_watchApp extends Application.AppBase {

  function initialize() {
    System.println("initialize");
    AppBase.initialize();
  }

  // onStart() is called on application start up
  function onStart(state as Dictionary?) as Void {
    System.println("start");
  }

  // onStop() is called when your application is exiting
  function onStop(state as Dictionary?) as Void {
    System.println("stop");
  }

  // Return the initial view of your application here
  function getInitialView() as [Views] or [Views, InputDelegates] {
    System.println("get view");
    return [ new metrolink_watchView() ];
  }

}

function getApp() as metrolink_watchApp {
  System.println( "get app" );
  return Application.getApp() as metrolink_watchApp;
}

So now it is behaving in the way I'd expected. It really just was as simple as that it wasn't going through the code that I had assumed was the top level. And now I see messages in the DEBUG CONSOLE corresponding to my println statements:

initialize
start
get view

I can't figure out how to tell the simulator that I have finished with my widget, so I can't get the stop message to come out.

Turning to the View, I see there are methods in there for initialize, layout, show, update and hide, which I'm assuming are in the order you would expect and (logically) all come after getView.

class metrolink_watchView extends WatchUi.View {

  function initialize() {
    System.println("init view");
    View.initialize();
  }

  // Load your resources here
  function onLayout(dc as Dc) as Void {
    System.println("layout");
    setLayout(Rez.Layouts.MainLayout(dc));
  }

  // the state of this View and prepare it to be shown. This includes
  // loading resources into memory.
  function onShow() as Void {
    System.println("show");
  }

  // Update the view
  function onUpdate(dc as Dc) as Void {
    System.println("update");
    // Call the parent onUpdate function to redraw the layout
    View.onUpdate(dc);
  }
  // state of this View here. This includes freeing resources from
  // memory.
  function onHide() as Void {
    System.println("hide");
  }

}

Having added these diagnostic statements, the following output appears:

initialize
start
get view
init view
layout
show
(2)update

Interestingly, onUpdate is called twice in rapid succession.

This is all checked in and tagged as METROLINK_MC_DEBUGGING.

As an aside, by selecting the Run/Debug perspective (the fourth item on the left hand navigation bar in VSCode), I see at the top a RUN AND DEBUG dropdown which enables me to select the launch configuration to be used. I don't know how it chooses the one it chooses, but it does enable me to choose the one I want to choose. Excellent. Now, can I remember that for when it comes up in future?

My Big Plan

In the short term at least, I basically want to hardcode all the data about routes I am interested in. But I want to hardcode it as if it were coming from some configuration file.

If I understand the documentation correctly, the "initial view" is there just to get you started. Because of navigation constraints, it cannot be "scrolled" but it can be "tapped" to launch another view, or it can offer a menu of choices (which launch other views) and it can display information. I think for now I would like it to display as much "next tram" information for your current query as will fit on the screen, with the option to somehow (swipe left/right?) move to a view that will allow you to select different routes. Whichever one you leave it on will then become the default. Tapping should keep the same route but allow you to scroll through all the data.

Breaking this down, it seems to me that I have six separate tasks right now:

• Provide the hardcoded configuration of routes;
• Get the data from the PHP server we wrote earlier;
• Figure out how to define a layout;
• Figure out how to populate the layout;
• Figure out how to have a dynamic layout, or change the layout, or something so that the layout fits the data;
• Wire everything up.

Obviously the last one must be done last, and the three layout tasks flow between each other, but whether to tackle the configuration, data or layout first is just a choice. I generally prefer to work in the direction of flow of data (unless I feel that something else is driving my choices), so I am going to proceed in exactly the order given here.

Encoding the Routes

One of the issues I have here is that I'm not entirely sure what code is supposed to go where. I've read a lot of the documentation, so I think I know, but I always feel that this is something that only really comes with experience. I also (obviously) feel a little hamstrung by not understanding either the data model or the file layout model. But then, of course, that is why I am doing this: to find those things out.

So, for now, I'm going to say that I am going to put the configuration of the data into initialize(), and the request for the data into update(). One of my concerns is that the call to request the data is asynchronous, and it is common in languages and frameworks for the callback to be somewhere where the UI cannot be updated, so it may be that update() is not really the right place to request the data. We will see what happens.

So, encoding the routes is very easy (assuming I do it correctly). For now, all I want is a list of from station codes and a list of to station codes. I think I also want each route to have a name as a handle to refer to it later (although this may not be the case). And I want to put them all in a list.

class metrolink_watchApp extends Application.AppBase {
  var routes as Array<Route>;

  function initialize() {
    System.println("initialize");
    AppBase.initialize();
    routes = configureRoutes();
  }

...

function configureRoutes() as Array<Route> {
  var altiRoute = new Route("ALTI", ["MKT", "PCG"], ["ALT"]);
  var traffRoute = new Route("TRC", ["TRC"], []);
  return [ altiRoute, traffRoute ];
}

class Route {
  var name as String;
  var from as Array<String>;
  var to as Array<String>;

  public function initialize(name as String, from as Array<String>, to as Array<String>) {
    self.name = name;
    self.from = from;
    self.to = to;
  }
}

This is checked in as METROLINK_MC_ROUTES.

I learnt a few things here. First of all, declaring a class is easy, using the class keyword. Members are declared using the var keyword; types are optional but are specified afterwards using as type notation. The constructor is a function called initialize and it's very important to spell this correctly, otherwise you will keep getting errors about mismatched argument lists. Lists are called Arrays, and can be typed using alligator notation (e.g. Array<String>).

So far, so good.

Making the WebRequest

I'm going to make the web request (for now at least) from the update() function in the view. I said earlier that we could do these things in any order and then wire them up at the end. So, for now, I am just going to make a web request and ignore the routes that we just configured.

Making a web request is described in the manual and I'm basically going to just try and copy that.

import Toybox.Graphics;
import Toybox.WatchUi;
import Toybox.Communications;

class metrolink_watchView extends WatchUi.View {

  // Update the view
  function onUpdate(dc as Dc) as Void {
    System.println("update");
    var params = { "from[]" => "FIR" };
    var options = {
      :method => Communications.HTTP_REQUEST_METHOD_GET,
      :headers => { "Content-Type" => Communications.REQUEST_CONTENT_TYPE_URL_ENCODED },
      :responseType => Communications.HTTP_RESPONSE_CONTENT_TYPE_URL_ENCODED
    };
    var responseCallback = method(:onReceive);
    Communications.makeWebRequest("https://gmmapowell.com/metrolink-data.php", params, options, responseCallback);
    // Call the parent onUpdate function to redraw the layout
    View.onUpdate(dc);
  }

  function onReceive(responseCode, data) {
    if (responseCode == 200) {
      System.println("Request Successful");
    } else {
       System.println("Request failed, code: " + responseCode);
    };
  };

  // Called when this View is removed from the screen. Save the
  // state of this View here. This includes freeing resources from
  // memory.

When I try and compile this, I get the message:

ERROR: vivoactive3: /Users/gareth/Projects/IgnoranceBlog/metrolink-watch/source/metrolink-watchView.mc:29,8: Permission 'Communications' required for '$.Toybox.Communications'.

Apparently, I need to provide some specific permission. I saw something about that in the manifest.xml, and going back to that, I can see that there is indeed a Permissions section with a checkbox for Communications. Let's select that.

There are a number of other problems with this code as well, which is surprising for something that I basically copied from the documentation. First off, there are semicolons after the closing braces at the end of onReceive. I have no idea what those are doing there, but I am going to assume there was a version of the compiler that either allowed them or required them and the documentation has not been updated.

More subtly, the function onReceive apparently needs to be typed. The error that appears is truly horrifying and suggests an awful type is required, but it is just very accepting and says that the method must take a Number as its first argument, return Void and then has a range of options for its second argument - so I chose String.

import Toybox.Lang;

...

  function onReceive(responseCode as Number, data as String) as Void {
    if (responseCode == 200) {
      System.println("Request Successful");
    } else {
       System.println("Request failed, code: " + responseCode);
    }
  }

The code then compiles (see tag METROLINK_MC_WEB_REQUEST_COMPILES), but when it runs, returns a code of -400. I don't know if this is supposed to be 400 and a minus sign has just been put on the front or if this is a special code. Anyway, looking back at the code, I know that I really want JSON back, but I have specified that I want HTTP_RESPONSE_CONTENT_TYPE_URL_ENCODED, so that is possibly the problem. Changing that to HTTP_RESPONSE_CONTENT_TYPE_JSON leads to a successful call, but (I remember from reading the documentation) the response is then parsed and I should be getting a dictionary back. Some quick checks confirm that I should in fact have declared the argument as a Dictionary and then I can check that I can extract the key Firswood - the station I requested as the station to depart from.

    var options = {
      :method => Communications.HTTP_REQUEST_METHOD_GET,
      :headers => { "Content-Type" => Communications.REQUEST_CONTENT_TYPE_URL_ENCODED },
      :responseType => Communications.HTTP_RESPONSE_CONTENT_TYPE_JSON
    };

...

  function onReceive(responseCode as Number, data as Dictionary) as Void {
    if (responseCode == 200) {
      System.println("Request Successful");
      System.println(data["Firswood"]);
    } else {
      System.println("Request failed, code: " + responseCode);
    }
  }

Rejoice! And don't forget to check in (METROLINK_MC_WEB_REQUEST_SUCCESSFUL).

The Initial View

Moving on to the view. This seems to me the hardest part, both because I'm so bad at visual things and because I'm really not sure what my options are for generating what I want. Looking at the section on layouts in the documentation, it seems that the easiest thing to do is probably to make my layout just a text-area and then scribble want I want into it and hope for the best. Hopefully, this at least gets me to the point where I can wire up one route, and then I can come back to the rest.

So, creating the layout and attaching it to the view does not seem so hard:

In route.xml:

<layout id="RouteLayout">
  <text-area width="fill" height="fill" />
</layout>

In the view:

  function onLayout(dc as Dc) as Void {
    System.println("layout");
    setLayout(Rez.Layouts.RouteLayout(dc));
  }

(see METROLINK_MC_DEFINE_LAYOUT).

I could not find anything in the developer guide to explain how to bind some text to this resource; after some Googling around, I found an example which used View.findDrawableById and confirmed that I could use that to display some simple text on the watch face.

class metrolink_watchView extends WatchUi.View {
  var textArea;

...

  function onLayout(dc as Dc) as Void {
    System.println("layout");
    setLayout(Rez.Layouts.RouteLayout(dc));
    textArea = self.findDrawableById("routeInfo") as TextArea;
  }

...

  function onUpdate(dc as Dc) as Void {
    System.println("update");
    textArea.setText("hello, world");

(This is tagged as METROLINK_MC_SHOW_TEXT_IN_AREA.)

Quickly, then, can I display multiple lines of data in this box using standard line-feed characters?

  function onUpdate(dc as Dc) as Void {
    System.println("update");
    textArea.setText("hello, world\nthis is a test\nof multiple lines\n");

Yes, I can (see METROLINK_MC_SHOW_MULTIPLE_LINES).

Finally, I raised the question earlier of whether or not this code would work inside the callback. Moving the code there led to a blank screen. A quick bit of research indicates that it is necessary to call WatchUi.requestUpdate. Adding this in does indeed cause the text to appear, although it creates a loop: requesting an update calls onUpdate which makes another request, which responds, causing a new request for an update. This is not an error, as such, but it will use an excessive amount of battery. Anyway, that is a problem to come back to, since what we will probably want to do is defer repeated updates until a later time, and I don't know how to do that yet.

    if (responseCode == 200) {
      System.println("Request Successful");
      System.println(data["Firswood"]);
      textArea.setText("hello, world\nthis is a test\nof multiple lines\n");
      WatchUi.requestUpdate();
    } else {
      System.println("Request failed, code: " + responseCode);
    }

Checked in and tagged as METROLINK_MC_SHOW_IN_RESPONSE.

Wire it all up

So, from here, it should be a simple matter of programming. I want to pass the list of configured routes from the "app" to the "view", use the first of those to make the web request, and then format the response into my text box. Should be easy.

Some of it was, some wasn't. Passing the route around was easy enough:

In the app:

  function getInitialView() as [Views] or [Views, InputDelegates] {
    return [ new metrolink_watchView(routes[0]) ];
  }

In the view:

class metrolink_watchView extends WatchUi.View {
  var route;
  var textArea;

...

  function initialize(route as Route) {
    self.route = route;
    View.initialize();
  }

Passing the parameters to the back end was not so easy. The PHP code requires us to pass the parameters as from[] and to[] and to do so multiple times. There does not seem to be a way of handling this in Monkey C's makeWebRequest, and I did extensive research on this. But, pushing back on the PHP end, it seems that any parameters that start from[ will be considered identical, even if they are different after that. So I can pass the parameters as from[0], from[1], etc. That requires a bit of code, but I'm not scared:

  function onUpdate(dc as Dc) as Void {
    var params = {};
    if (route.from) {
      for (var i=0;i<route.from.size();i++) {
        var key = "from[" + i + "]";
        params[key] = route.from[i];
      }
    }
    if (route.to) {
      for (var i=0;i<route.to.size();i++) {
        var key = "to[" + i + "]";
        params[key] = route.to[i];
      }
    }

At the other end, we need to convert the Dictionary we get back (which is parsed from the JSON) into a readable message. It seems to me that the easiest way to do this is just to list it all out vertically, with the "from" station first, then the "to" station, then a line with all the times on, and repeat. Over time, it's possible that I can find ways to improve this, but for now I'm just trying to get something to work.

Monkey C doesn't appear to have a StringBuilder class or equivalent, but it does have a toCharArray() method on a String, and StringUtil has a charArrayToString() method; and an Array is flexible enough to grow with the add() and addAll() methods. In short, we can do this:

  function assembleMessage(data as Dictionary) as String {
    var chars = [];
    var keys = data.keys();
    for (var i=0;i<keys.size();i++) {
      var fromX = keys[i];
      var toX = data[fromX];
      assembleFrom(chars, fromX as String, toX);
    }
    return StringUtil.charArrayToString(chars);
  }

  function assembleFrom(chars as Array<Char>, from as String, to as Dictionary) {
    chars.addAll(from.toCharArray());
    chars.add('\n');

    var dests = to.keys() as Array<String>;
    for (var i=0;i<dests.size();i++) {
      var toX = dests[i];
      chars.addAll("=> ".toCharArray());
      chars.addAll(toX.toCharArray());
      chars.add('\n');
      var times = to[toX];
      assembleTimes(chars, times);
    }
  }

  function assembleTimes(chars as Array<Char>, times as Array<String>) {
    for (var i=0;i<times.size();i++) {
      chars.add(' ');
      chars.add(' ');
      chars.addAll(times[i].toCharArray());
    }
    chars.add('\n');
  }

And then it's very easy to call this from the onReceive code:

  function onReceive(responseCode as Number, data as Dictionary) as Void {
    if (responseCode == 200) {
      textArea.setText(assembleMessage(data));
      WatchUi.requestUpdate();
    } else {
      System.println("Request failed, code: " + responseCode);
    }
  }

Too much of the text is missing for my liking, due to the circular nature of the watch face. The simplest "fix" is to centre everything:

<layout id="RouteLayout">
  <text-area id="routeInfo" width="fill" height="fill" justification="Gfx.TEXT_JUSTIFY_CENTER" />
</layout>

Yeah, I'm pretty happy with that. I'll tag that as METROLINK_MC_WIRING_UP.

Fixing the Timing Issue

Earlier, I noted that we were effectively in an infinite loop and that, while it wasn't the end of the world, it seemed a bad plan. So let's fix that by using a Timer.

My plan is that onUpdate will be called when the system feels that it is appropriate (this is already happening). When this happens, we will add a timer to say, "try again in 30s". When we come into onUpdate again, we will test if this timer is still active and, if so, we will not fire the web request. When the timer fires, we will clear it and then call onUpdate, thus making another web request and setting up another timer. It doesn't seem like that should be too hard, does it?

And indeed it isn't.

class metrolink_watchView extends WatchUi.View {
  var route;
  var textArea;
  var timer as Timer.Timer?;

...

  function onUpdate(dc as Dc) as Void {
    if (timer == null) {

...

      timer = new Timer.Timer();
      timer.start(method(:timerCallback), 15000, false);

...

  function timerCallback() as Void {
    timer = null;
    WatchUi.requestUpdate();
  }

Probably the most interesting thing of note is the syntax to declare a variable to either have a value or be null: Timer.Timer?. Also, the existence of a Timer class inside a Timer package caused me some confusion (I originally wrote var timer as Timer, and had two errors to fix).

It's also interesting how hard it is to make sure that everything is actually working; I had to add various debugging statements to make it clear that the request was being invoked and the display updated (not shown), but in the end I convinced myself I had done it correctly. So now it updates every 15s - probably that's a little too often, but I'll experiment and see if it causes me any issues with battery life.

I also added code to cancel the timer in onHide(). Unlike the lifecycle methods in the app class, there is a clear menu item (Settings>Force onHide) that causes this to be called. I suspect that some lifecycle operation will cancel it for me anyway, but I thought this would probably be on the safe side. In my experiments, it certainly doesn't seem to do any harm. The corresponding Settings>Force onShow starts the widget running again.

  function onHide() as Void {
    if (timer != null) {
      timer.stop();
      timer = null;
    }
  }

Time to deploy it to the actual device again. Cool. Tagged as METROLINK_MC_WITH_TIMER.

A PHP Bug

At this point, I tried adding some more routes to my configuration and suddenly encountered no data coming back. A little investigation revealed that the problem was I never finished the work of handling multiple "to" stations, and so if there were no trams going to the first destination, no data came back. Fortunately not at all subtle.

I went back and fixed that.

    // Apply the destination filters
    if ($this->to) {
    if (!array_key_exists($d, $item)) return false;
    $isTo = $item[$d];
    $anyTo = false;
    foreach ($this->to as $f) {
      if ($isTo == $f)
      $anyTo = true;
    }
    if (!$anyTo) return $anyTo;
    }

    // Apply the origin filters

Apparently, I didn't write any tests. Oh, well. Checked in as METROLINK_PHP_MULTIPLE_TO.

Getting started with Monkey C

OK, now onto the fun part. I've written lots of PHP code before and I generally find it painful. I have to say, the combination of a small project, the project being essentially greenfield, using the latest PHP features and "doing it right" (rather than just hacking something together) actually made that a relatively pleasant experience.

But what I really want to do is to build an app for the watch; as I said, it's something I've been wanting to do for about seven years now. So, let's get started.

Installing the SDK

I'm not sure there's any particular merit to my decisions, but I opted to do all the PHP work on my linux box (where I have PHP installed and it was easy to install other components such as PHPUnit) but have opted to do the watch side of things on my Mac, since Mac is a supported environment for Monkey C (so is linux, for what it's worth).

There is a Getting Started page which I am basically paraphrasing here, although similar instructions are on the download page.

Head over to the Garmin SDK Download page and agree to their terms and click to download the installer. This requires you to have an account and log in - I have done this previously when trying to get things to work, so I can just sign in.

It then downloads the current SDK and a whole bunch of metadata about devices. Fortunately, it downloads the SDK first, so I think we can carry on while it downloads all the devices.

Switching our attention to VSCode, we now install the "Monkey C" extension, complicated a little by the fact that there seem to be a few with the same name; I'm going to install the official Garmin one. It apparently then needs VSCode to restart, and then wants us to "verify the installation" by running the command "Monkey C: Verify Installation". I can do that. I think. It turns out that this requires me to have a "developer key" which was not mentioned in the instructions and is not obviously implied by "verify installation" (in the way it would be by "complete" or "configure" installation). I guess everything depends on context.

I may have generated a key 7 years ago, but if so, I've probably lost it or at least misplaced it (I don't take these things as seriously as I should to begin with, before even thinking about the fact I was just messing around, and that is two computers ago now), so I opt to generate a new one and save it in "a safe place". Hopefully I'll be able to find it again. Then everything seems fine.

The final set of instructions are about installing the SDK for your phone, which I don't want to do right now, so I won't.

Let's Build Something

So there is documentation at the Monkey C page which describes a simple application, but I'm going to follow the instructions at Your First App in order to build something quickly.

So, using CMD-SHIFT-P and Monkey C: New Project, I create a project called metrolink-watch which will be a widget. It then asks what API level I want to use. Normally, I would pick the highest number to be compatible as far into the future as possible; but on this occasion, I would like it to be compatible with the watch I already own. Looking at the compatibility page, I can see that my watch uses API level 3.1.0, so I will pick that one. It then asks me which devices I'm interested in supporting, so I select the "vivoactive 3". It then asks me for a directory. This is presumably the directory where it is going to put my project. Is it going to create a subdirectory, or just put the project there? I don't know, so I play it safe and create a new directory for it ... but no, it creates its own subdirectory. So now I have metrolink-watch/metrolink-watch. OK, delete all of that and start again.

Very good. We now have a project on the disk, in VSCode and incorporated in the workspace. Just what I wanted. The manifest file has also been opened in a special manifest editor. I'm not sure whether I wanted that or not, but I suppose it's not unreasonable.

(By now, the installer has finished its work so I have closed it.)

The continuing instructions on getting started tell me how to configure the manifest, but all of those options are already taken care of, so that's fine. So let's try running it in the simulator and see what happens. The instructions say use CMD-F5 on mac, but that seems to launch "VoiceOver" on my machine. VSCode suggests trying CTRL-F5, but that tells me something about npm not being available. From the Run menu, I get the same message. I try the context menu, and that has an option to "Build Current Project", so I try that, which seems to work, but still doesn't offer me the ability to run the program (it does have the option to run tests). Even when I point at the "binary" (metrolinkwatch.prg), it doesn't have an option to "run me". What am I missing?

(At this point I notice the instructions say, "if all goes well..."; presumably they aware that things not going well is a possibility, but not one they choose to dwell on).

In desperation, I try Run Unit Tests, which at least pops up a window, but doesn't seem to do anything else, presumably because I don't yet have any unit tests (maybe I should write some?).

I still feel I struggle a lot with VSCode and what you need to set it up. I feel that I am missing some kind of launch configuration or task, just like I was with running the phpUnit, and I don't know where to configure it. So let's go google.

On the page about the Monkey C Visual Studio Code Extension, there is a reference to launch.json, which attracted my attention (well, Google's), but before I get to the section on "Editing the Launch Configuration", I see this line:

Before running the program, make sure you have one of your source files (In the source folder with the .mc extension) open and selected in the editor.

which, I have to confess, is also in the instructions I was claiming to follow, but somehow escaped my attention. But no, that makes no difference.

OK, so where would I look for a launch.json? Well, in my .vscode directory in my workspace. It turns out I have four in other projects. I have no idea how VSCode decides which one(s) to use, so I move them all out of the way. Now when I push CTRL-F5, things happen and (eventually) a watch with a monkey in it appears on my screen. Woohoo!

(Interestingly, no launch.json seems to have been generated; if it has, I don't see where it has been put.)

I realize that I have not spent the time I should understanding all of the interactions between system installations, workspaces and projects that I should in VSCode, but this confuses me no end. If it has found five launch configurations, it should have some algorithm to figure out which one to use. Asking me seems like the obvious way, but I would accept "the one in this project". Randomly picking one of the four other projects and then complaining it doesn't work seems ridiculous in the extreme.

So I did some research, and it seems that launch configurations need to be defined in their various launch.json files, but then these can be named within the workspace file and possibly this allows you to see a menu of available launch configurations. But at the same time, I noticed that there is a menu option to create a new launch configuration for a project (Run>Add Configuration...).

When I run this with all my launch.json files moved out of the way, it sensibly creates one in metrolink-watch. But when I do it with an existing launch.json in the workspace, it attempts to add it to that one. So I'm forced into hacking things. I create a new .vscode/launch.json in the metrolink-watch directory, and put in some completely boilerplate content (the version and configuration tags, with minimal content). Now when I select the Run>Add Configuration... and choose Monkey C: Run App, it creates something reasonable for me - but it still put it in the existing one. Still, I can cut-and-paste with the best of them.

OK, for the time being at least, that seems to work, even with the other configurations in place, as long as I am in one of the .mc files.

{
  "version": "0.2.0",
  "configurations": [
    {
      "type": "monkeyc",
      "request": "launch",
      "name": "Run App",
      "stopAtLaunch": false,
      "device": "${command:GetTargetDevice}"
    }
  ]
}

Before going any further, I'm going to check that in and tag it as METROLINK_MC_SAMPLE.

On the Watch

The final part of the instructions gives information about "sideloading" an app: that is, loading onto your device without going through the store. So let's try that.

The first step is, reasonably enough, to plug the watch in to the computer. Easy enough.

Then I need to "build for device". I'm confused why the existing build is not enough. Does it need different packaging for simulators and devices? That seems a poor design choice. The instructions say that it will ask you to choose a device, but it didn't ask me (maybe because I only selected one during configuration?) and then asks for an output directory. I'm not sure why it doesn't just put it in bin/ but presumably there's a reason. So I create a new folder called vivoactive3 and click open. Then, it quietly puts another question at the top of the screen: Debug or Release. I didn't notice this the first couple of times through and was very confused as to what was going on (which seemed to be nothing). Eventually I selected Debug and my vivoactive3 directory was populated with a metrolinkwatch.prg file.

Following the instructions, I then copied that across to the device:

$ cp vivoactive3/metrolinkwatch.prg /Volumes/GARMIN/GARMIN/APPS/

Again, this seems a little confusing: the instructions say copy to your device's GARMIN/APPS directory but doesn't give the path to the device. This is apparently /Volumes/GARMIN/, so the full directory path ends up having two GARMIN segments.

And now I can do an ls on that directory:

$ ls -1 /Volumes/GARMIN/GARMIN/APPS/
DATA
LOGS
MAIL
OUT.BIN
SETTINGS
TEMP
metrolinkwatch.prg

OK, this is further I have ever managed to get before.

Turning to the watch, I find it seems "locked" in a weird mode. Presumably this is it's "I am currently connected to a PC" mode. I try unmounting the device, but that doesn't work. I then unplug it and it says "Verifying ConnectIQ apps". Very good. Then it goes back to normal mode. I slide through the widget carousel and the "last" or "bottom" one (as I think of it) says "IQ metrolink-watch" and then less than a second later shows me a picture of a monkey. Excellent. Not quite "hello, world", more sort of "hello, monkey".

But now I can start developing the app.

PHP to read metrolink data

I already have PHP installed on my linux box; I think this is the default. I have previously installed Visual Studio Code (VSCode) and I think that's a fairly common thing to have installed. I don't already have a plugin for PHP on VSCode, but I'm installing the DevSense PHP plugin right now from the "extensions" tab of VSCode.

So I can now write what I would consider the minimal code to read the metrolink data feed:

<?php
  require('metrolink_key.php');

  $ch = curl_init("https://api.tfgm.com/odata/Metrolinks");
  curl_setopt($ch, CURLOPT_HTTPHEADER, array($metrolink_key));
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, false);
  $response = curl_exec($ch);
?>

The require statement incorporates the security key you need in order to access the OData feed. This is not present in the repository since it is a secret, but the samples directory has a file metrolink_key_template.php which shows you what it would look like. Then the next four lines access the OData feed, giving the URI, the security key, specifying that we would like the output to be returned as a value from the function call (rather than being sent directly to the client) and then executing the request.

In theory at least, this should run on our local machine:

$ php metrolink-data.php
PHP Fatal error: Uncaught Error: Call to undefined function curl_init() in /home/gareth/Projects/IgnoranceBlog/metrolink-php/php/metrolink-data.php:4

So it would seem that while I have php installed, I don't have the PHP curl module installed. It would seem that the required fix on linux is:

$ sudo apt-get install php-curl

After this, I can try again and get the expected absence of any output:

$ php metrolink-data.php

Checked in and tagged as METROLINK_PHP_MINIMAL.

Transforming the data

The next step is to try and do this transformation. I want to write some PHP tests around this (I've never used phpUnit before, so this will be interesting in that way, too), but at the top level I want to translate the JSON response into objects, and then I want to transform the resultant objects into JSON. None of that is going to be tested, but the core is a transform() function which I do want to test, and to keep things straight, I'll put that in its own file.

The main program now looks like this:

<?php
  require('metrolink_key.php');
  require('transform.php');

  $ch = curl_init("https://api.tfgm.com/odata/Metrolinks");
  curl_setopt($ch, CURLOPT_HTTPHEADER, array($metrolink_key));
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
  $response = curl_exec($ch);

  $data = json_decode($response);
  $transformed = transform($data->value);
  print json_encode($transformed);
?>

and the transform() function at this stage just returns some sample data:

<?php
  function transform($odata) {
  return array("Firswood" => array("Victoria" => ["19:59", "20:09"]));
  }
?>

Checked in and tagged as METROLINK_PHP_TRANSFORM.

Installing and using phpUnit

The first step is to install phpUnit, which is described here, but on closer inspection it seems that I can install it with apt:

$ phpunit
Command 'phpunit' not found, but can be installed with:
sudo apt install phpunit

I can do that.

I can now write a simple (failing) test:

<?php
use PHPUnit\Framework\TestCase;

require("../php/transform.php");

final class Transform_test extends TestCase
{
  public function test_something() {

    $foo = transform(array());
    // $this->assertInstanceOf("", $transformer);
    $this->fail("hello, world");
  }
}

and run it on the command line:

$ phpunit *
PHPUnit 9.5.10 by Sebastian Bergmann and contributors.


F 1 / 1 (100%)


Time: 00:00.004, Memory: 4.00 MB


There was 1 failure:


1) Transform_test::test_something
hello, world


/home/gareth/Projects/IgnoranceBlog/metrolink-php/test/transform_test.php:12


FAILURES!
Tests: 1, Assertions: 1, Failures: 1.

But this doesn't seem to run inside VSCode. Googling around brought me to this article, which has cryptic instructions I don't understand, but I can try following:

$ composer require --dev phpunit/phpunit
Command 'composer' not found, but can be installed with:
sudo apt install composer
$ sudo apt install composer
...
0 to upgrade, 22 to newly install, 0 to remove and 9 not to upgrade.
Need to get 930 kB of archives.
...
$ composer require --dev phpunit/phpunit
Using version ^10.5 for phpunit/phpunit
./composer.json has been created
Running composer update phpunit/phpunit
Loading composer repositories with package information
Updating dependencies
...

And now I can run the tests in VSCode, using C-S-P and typing PHPUnit Test:

Executing task in folder vscode-java: "{php}" {phpargs} "{phpunit}" {phpunitargs} '/home/gareth/Projects/IgnoranceBlog/metrolink-php/vendor/bin/phpunit' --colors=always '/home/gareth/Projects/IgnoranceBlog/metrolink-php/php/transform.php'
/usr/bin/bash: line 1: : command not found

This is still not what I was expecting. It would seem that there are some configuration parameters that still need setting. I tried fiddling with various properties and settings, but nothing I did made it work.

Finally I gave up and decided to add a task. But I could not find where to put my tasks.json file. It turns out that in a VSCode "workspace" you don't put them in tasks.json but directly in the workspace, like so:

      "path": "metrolink-php"
    }
  ],
  "settings": {},
  "tasks": {
    "version": "2.0.0",
    "tasks": [
      {
        "label": "Metrolink PHP Tests",
        "type": "shell",
        "command": "phpunit",
        "args": ["*.php"],
        "group": "test",
        "options": {
          "cwd": "${workspaceFolder}/../metrolink-php/test"
        },
        "presentation": {
          "reveal": "always"
        }
      }
    ]
  }
}

And then it can be run with CTRL-SHIFT-P followed by Run Task followed by Metrolink PHP Tests. This is too much hassle for me, so I created a keyboard shortcut:

[
    {
        "key": "alt+f11",
        "command": "workbench.action.tasks.runTask",
        "args": "Metrolink PHP Tests"
    }
]

and placed this in the file ~/.config/Code/User/keybindings.json, which is where it would seem it is looked for on my system (I couldn't find a reference to this anywhere; I just created one in the editor to bind ALT-F11 to runTask and then searched for keybindings.json and edited it to add the task name).

OK, so now we can see our test fail inside VSCode:

Executing task in folder test: phpunit *.php


PHPUnit 9.5.10 by Sebastian Bergmann and contributors.


F 1 / 1 (100%)


Time: 00:00.004, Memory: 4.00 MB


There was 1 failure:


1) Transform_test::test_something
hello, world


/home/gareth/Projects/IgnoranceBlog/metrolink-php/test/transform_test.php:12


FAILURES!
Tests: 1, Assertions: 1, Failures: 1.
 * Terminal will be reused by tasks, press any key to close it.

And that is all checked in and "working" and tagged as METROLINK_PHP_PHPUNIT_FAIL.

Transforming the Data

The PHP function json_decode automatically transforms all of the JSON constructs to built-in PHP ones, and, in particular converts a JSON array to a PHP array and a JSON object to a PHP associative array. So by the time we have decoded the JSON and extracted the value property from the top level (which is done in the main code already shown), we pass an array of associative arrays to the transform() function, each of which has the information for one PID in it.

So the code for transform wants to iterate through this array, extracting and filtering the station location and destinations, and building up a map of trams between the two.

Hang on a sec! As I write "filter" I realize that I am missing a key part of code here, in that the top-level code needs to process the incoming request to identify the stations you want to travel between. That is, it needs to handle requests like:

metrolink-data.php?from=FIR&to=VIC&to=ROC

and when I say "like", I think over the course of time many more options will emerge, but for now the only requirement is that there must be at least one from and it is possible to have multiple from and multiple to parameters and to create a "grid" or "matrix" of times between the two.

That also means that it probably makes more sense to expose the Transformer class (which I was going to bury inside transform()) to the top level to take these parameters.

  $transformer = new Transformer($_GET);

  $ch = curl_init("https://api.tfgm.com/odata/Metrolinks");
  curl_setopt($ch, CURLOPT_HTTPHEADER, array($metrolink_key));
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
  $response = curl_exec($ch);

  $data = json_decode($response);
  $transformed = $transformer->transform($data->value);

Here we are passing the PHP superglobal $_GET into the Transformer constructor. What is that and what does it look like? It's an associative array of parameter names to values. You can see this in operation on the command line by dumping them out in the code:

  var_dump(value: $_GET);

and then running the program using the php-cgi binary:

$ php-cgi -f metrolink-data.php from[]=FIR from[]=VIC
array(1) {
  ["from"]=>
  array(2) {
    [0]=>
    string(3) "FIR"
    [1]=>
    string(3) "VIC"
  }
}

In order for this to work with multiple arguments with the same name, you will see that we have to give the argument name the array extension, [].

So, we can now rewrite our unit test to take this into account:

final class Transform_test extends TestCase
{
  public function test_extract_from() {
    $transformer = new Transformer(["from" => ["FIR", "VIC"]]);
    $this->assertEquals(["FIR", "VIC"], $transformer->from);
  }
}

And we can check all of that in as METROLINK_PHP_FIRST_TEST.

Some negative tests

It's fairly easy to knock out some "negative" tests, that is, tests for which the answer is always the empty map. These don't help us very much, but they do "ground" the code inasmuch as they stop us writing something that does not meet these criteria, and make sure that we have an empty array as the "default" value.

  public function test_no_pids_produces_no_output() {
    $transformer = new Transformer(["from" => ["FIR", "VIC"]]);
    $out = $transformer->transform([]);
    $this->assertEquals([], $out);
  }

  public function test_requiring_from_fir_excludes_from_air() {
    $transformer = new Transformer(["from" => ["FIR"]]);
    $out = $transformer->transform(["TLAREF" => "AIR"]);
    $this->assertEquals([], $out);
  }

  public function test_requiring_to_fir_excludes_to_air() {
    $transformer = new Transformer(["to" => ["FIR"]]);
    $out = $transformer->transform(["TLAREF" => "AIR"]);
    $this->assertEquals([], $out);
  }
}

And we need a trivial implementation of transform():

  function transform($odata) {
    return [];
  }

And this is all checked in as METROLINK_PHP_NEGATIVE_TESTS.

Testing the Loop

So we can now assert that if we receive a single entry with matching criteria, it is properly transformed.

  public function test_from_fir_includes_a_single_matching_entry() {
    $transformer = new Transformer(["from" => ["FIR"]]);
    $out = $transformer->transform([[
      "TLAREF" => "FIR",
      "Dest0" => "Victoria",
      "Wait0" => "6",
      "LastUpdated" => "2024-09-17T19:52:55Z"
    ]]);
    $this->assertEquals([["FIR" => ["VIC" => [ "19:58" ]]]], $out);
  }

(Note that in writing this test, I discovered that the previous tests had only provided one member, not a list, so changed them.)

Checked in as METROLINK_PHP_LOOP_TEST.

Filtering

For me, unit testing is always about how the code you've written is factored. In this case, there is basically a loop (which was tested above), then filtering, then the actual transformation. Having dealt with the loop, I now want to nail down (almost) all the cases to do with filtering.

Initially, at least, I want to offer three different cases of filtering: the from parameter should be a 3-letter code matching the TLAREF field, which is also the first three letters of the PIDREF field; the to field can be either the full name of the destination, i.e. one of Dest0, Dest1 or Dest2 (assuming those fields are there) or else its corresponding 3-letter code; or the to field can be a combination of the direction and line. For this last field, if you look at the data, you can see these two fields: the line is something like "Altrincham" or "South Manchester" and the direction is either "Incoming" or "Outgoing". In order to query this, the user must combine both into a single field, e.g. I-South Manchester or else O-Oldham & Rochdale. The idea of supporting this format is that you often know you want to go somewhere (e.g. Queens Road) but you don't know whether the tram is going to Bury, Heaton Park or Crumpsall - and you don't care. It's going the right way.

However, handling the 3-letter destination codes and the the "line" destinations requires us to do some pre-processing on the data: we need to find the actual destination names corresponding to the input.

As I go deeper and deeper into this, I discover more and more wrinkles. As I work towards processing this destination data, I realize that I need to consider the three destinations "separately", and so it makes sense to add an additional parameter to both filter and transformOne to specify that I want to deal with each destination index in turn, and then I need to merge these answers together to end up with the array I originally thought of. So much code, so little time.

The tests come out looking like this:

  public function test_firswood_has_two_trams_to_Victoria() {
    $transformer = new Transformer(["from" => ["FIR"]]);
    $out = $transformer->transform([[
      "TLAREF" => "FIR",
      "Dest0" => "Victoria",
      "Wait0" => "6",
      "Dest2" => "Victoria",
      "Wait2" => "18",
      "LastUpdated" => "2024-09-17T19:52:55Z"
    ]]);
    $this->assertEquals(["FIR" => ["VIC" => [ "19:58", "20:10" ]]], $out);
  }

  public function test_tram_to_Altrincham_passes_with_empty_filter_in_Dest0() {
    $transformer = new Transformer([]);
    $matches = $transformer->filter([
      "TLAREF" => "SPS",
      "Dest0" => "Altrincham",
      "Wait0" => "6",
      "LastUpdated" => "2024-09-17T19:52:55Z"
    ], 0);
    $this->assertTrue($matches);
  }

  public function test_to_Altrincham_passes_filter_in_Dest0() {
    $transformer = new Transformer(["to" => ["Altrincham"]]);
    $matches = $transformer->filter([
      "TLAREF" => "SPS",
      "Dest0" => "Altrincham",
      "Wait0" => "6",
      "LastUpdated" => "2024-09-17T19:52:55Z"
    ], 0);
    $this->assertTrue($matches);
  }}

and they pass with this implementation, calling filter, transformOne and merge repeatedly:

    foreach ($odata as $item) {
    for ($i=0;$i<3;$i++) {
      if ($this->filter($item, $i))
      $ret = $this->merge($ret, $this->transformOne($item, $i));
    }

and the filter implementation itself:

  function filter($item, $dst) {
    // This tram must be going *somewhere* (a lot of them are blank)
    $d = "Dest{$dst}";
    if (!array_key_exists($d, $item) || $item[$d] == "") return false;

    // Apply the destination filters
    if ($this->to) {
    if ($item[$d] != $this->to[0]) return false;
    }

    // Apply the origin filters
    if ($this->from) {
    if (!array_key_exists("TLAREF", $item)) return false;
    if ($item["TLAREF"] != "FIR") return false;
    }

    // If it didn't fail, it can be included
    return true;
  }

  function transformOne($pid, $dst) {
    if (array_key_exists("TLAREF", $pid)) {
    return [ "FIR" => [ "VIC" => [ $dst == 0 ? "19:58" : "20:10" ]]];
    }
  }

  function merge($ret, $incl) {
    foreach ($incl as $k => $v) {
    if (!array_key_exists($k, $ret)) {
      $ret[$k] = $v;
    } else {
      foreach ($v as $k2 => $v2) {
      foreach ($v2 as $time) {
        $ret[$k][$k2][] = $time;
      }
      }
    }
    }
    return $ret;
  }
  }
?>

(see METROLINK_PHP_FIRST_FILTERS).

Pre-processing the data

In order to handle the pre-processing, I am adding an extra "top-level" analysis step in metrolink-data.php to tell the transformer to analyze all the input data. Apart from anything else, this makes it easier to unit test.

I can then fairly easily write the code that handles the conversion of 3-letter codes to full names:

Testing this with:

  public function test_tla_collection() {
    $transformer = new Transformer(["to" => ["ALT"]]);
    $tlas = $transformer->collectTLAs([[
      "TLAREF" => "ALT",
      "StationLocation" => "Altrincham"
    ],[
      "TLAREF" => "SPS",
      "StationLocation" => "St Peter's Square"
    ],[
      "TLAREF" => "ALT",
      "StationLocation" => "Altrincham"
    ],[
      "TLAREF" => "WST",
      "StationLocation" => "Weaste"
    ]]);
    $this->assertEquals(["ALT" => "Altrincham", "SPS" => "St Peter's Square", "WST" => "Weaste"], $tlas);
  }

  public function test_analysis_can_turn_ALT_to_Altrincham() {
    $transformer = new Transformer(["to" => ["ALT"]]);
    $transformer->analyze([[
      "TLAREF" => "ALT",
      "StationLocation" => "Altrincham"
    ]]);
    $this->assertEquals(["Altrincham"], $transformer->to);
  }

we can write:

  function analyze($odata) {
    $tlas = $this->collectTLAs($odata);
    $input = $this->to;
    $this->to = [];
    foreach ($input as $dest) {
    $this->analyzeOne($tlas, $dest);
    }
  }

  function collectTLAs($odata) {
    $tlas = [];
    foreach ($odata as $pid) {
    // Obviously we need to have a TLAREF and a StationLocation
    if (!array_key_exists("TLAREF", $pid)) continue;
    if (!array_key_exists("StationLocation", $pid)) continue;

    // Once it's done, don't do it again
    if (array_key_exists($pid["TLAREF"], $tlas)) continue;

    // map it
    $tlas[$pid["TLAREF"]] = $pid["StationLocation"];
    }
    return $tlas;
  }

  function analyzeOne($tlas, $dest) {
    if (array_key_exists($dest, $tlas))
    $this->to[] = $tlas[$dest];
    else
    $this->to[] = $dest;
  }

and wire this into $metrolink-data.php" as:

  $data = json_decode($response);
  $transformer->analyze($data->value);
  $transformed = $transformer->transform($data->value);
  print json_encode($transformed);

and check in as METROLINK_PHP_STATION_ANALYSIS.

Processing for "Routes"

The "routes" feature is slightly more complicated because each input can translate into multiple outputs, but otherwise the principle is the same: we look at each entry in the input and assemble a set of "line names" consisting of the direction and line, and then map that to an array of full station names from the StationLocation field. It's also important to note that any given station is not on one exclusive "line" in this sense; in general, it will be on at least two: one "Incoming" and one "Outgoing".

Again, the tests are like this:

  public function test_line_collection() {
    $transformer = new Transformer([]);
    $lines = $transformer->collectLines([[
      "StationLocation" => "Firswood",
      "Line" => "South Manchester",
      "Direction" => "Outgoing",
    ],[
      "StationLocation" => "Firswood",
      "Line" => "South Manchester",
      "Direction" => "Incoming",
    ],[
      "StationLocation" => "Central Park",
      "Direction" => "Outgoing",
      "Line" => "Oldham & Rochdale",
    ],[
      "StationLocation" => "Failsworth",
      "Direction" => "Outgoing",
      "Line" => "Oldham & Rochdale",
    ],[
      "StationLocation" => "Freehold",
      "Direction" => "Outgoing",
      "Line" => "Oldham & Rochdale",
    ],[
      "StationLocation" => "Weaste",
      "Direction" => "Incoming",
      "Line" => "Eccles",
    ]]);
    $this->assertEquals([
      "O-South Manchester" => ["Firswood"],
      "I-South Manchester" => ["Firswood"],
      "I-Eccles" => ["Weaste"],
      "O-Oldham & Rochdale" => ["Central Park", "Failsworth", "Freehold"]
    ], $lines);
  }

  public function test_analysis_will_expand_outbound_oldham() {
    $transformer = new Transformer(["to" => ["O-Oldham & Rochdale"]]);
    $transformer->analyze([[
      "StationLocation" => "Firswood",
      "Line" => "South Manchester",
      "Direction" => "Outgoing",
    ],[
      "StationLocation" => "Firswood",
      "Line" => "South Manchester",
      "Direction" => "Incoming",
    ],[
      "StationLocation" => "Central Park",
      "Direction" => "Outgoing",
      "Line" => "Oldham & Rochdale",
    ],[
      "StationLocation" => "Failsworth",
      "Direction" => "Outgoing",
      "Line" => "Oldham & Rochdale",
    ],[
      "StationLocation" => "Freehold",
      "Direction" => "Outgoing",
      "Line" => "Oldham & Rochdale",
    ],[
      "StationLocation" => "Weaste",
      "Direction" => "Incoming",
      "Line" => "Eccles",
    ]]);
    $this->assertEquals(["Central Park", "Failsworth", "Freehold"], $transformer->to);
  }

and the code looks like this:

  function collectLines($odata) {
    $lines = [];
    foreach ($odata as $pid) {
    // Obviously we need the things we want to map
    if (!array_key_exists("Line", $pid)) continue;
    if (!array_key_exists("Direction", $pid)) continue;
    if (!array_key_exists("StationLocation", $pid)) continue;

    $lineName = substr($pid["Direction"], 0, 1) . "-" . $pid["Line"];
    $lines[$lineName][] = $pid["StationLocation"];
    }
    return $lines;
  }

(along with some associated changes to make everything work), which is checked in as METROLINK_PHP_LINE_NOTATION.

Transforming

Up until now, we have just been hacking in the transformed data. Now it's time to actually calculate this. There are two separate parts to this: figuring out the station names; and figuring out the expected arrival times. I have to admit, up until now I have been using the 3-letter codes, but I want to return the actual station names, so some re-work will be necessary. But other than that, getting the correct station names is not hard. Figuring out the arrival time is not hard, per se, but it is, shall we say, complicated. Or possessed of a number of cases and steps.

The time comes from two factors, one is the number of minutes to wait, in the Wait0 or equivalent field; and then other is the LastUpdated field, which is a timestamp. Basically, we need to add the number of minutes to the timestamp. What we decide to do here basically depends on how easy it is to parse and reformat times: if it's easy, we'll use the date logic; if it's not, we'll use an icky and hacky string manipulation. A little experimentation shows that a combination of strtotime and date makes the whole thing very easy, so a few unit tests later we have the ability to format the time, and can include that in the transformOne function.

  public function test_adding_0_minutes_is_easy() {
    $transformer = new Transformer([]);
    $when = $transformer->date("2024-09-17T19:52:55Z", 0);
    $this->assertEquals("19:52", $when);
  }

  public function test_adding_5_minutes() {
    $transformer = new Transformer([]);
    $when = $transformer->date("2024-09-17T19:52:55Z", 5);
    $this->assertEquals("19:57", $when);
  }

  public function test_adding_18_minutes_will_wrap() {
    $transformer = new Transformer([]);
    $when = $transformer->date("2024-09-17T19:52:55Z", 18);
    $this->assertEquals("20:10", $when);
  }

  public function test_can_wrap_to_tomorrow() {
    $transformer = new Transformer([]);
    $when = $transformer->date("2024-09-17T23:55:12Z", 6);
    $this->assertEquals("00:01", $when);
  }

And the corresponding code is:

  function transformOne($pid, $dst) {
    if (array_key_exists("TLAREF", $pid)) {
    return [ "FIR" => [ "VIC" => [ $this->date($pid["LastUpdated"], $pid["Wait{$dst}"]) ]]];
    }
  }

  function date($from, $wait) {
    $unix = strtotime($from);
    $when = $unix + $wait * 60;
    $ret = date('H:i', $when);
    return $ret;
  }

All this is checked in with the tag METROLINK_PHP_EXPECTED_TIME.

And so, I can finally finish it all off by implementing the rest of the transformation code (including the re-work on the station names).

The new test looks like this:

  public function test_a_tram_from_Freehold_to_Shaw_is_correctly_reported() {
    $transformer = new Transformer([]);
    $route = $transformer->transformOne([
      "StationLocation" => "Freehold",
      "Dest0" => "Shaw and Crompton",
      "Wait0" => "4",
      "LastUpdated" => "2024-09-17T19:52:55Z"
    ], "0");
    $this->assertEquals(["Freehold" => [ "Shaw and Crompton" => [ "19:56"]]], $route);
  }

(and there are a couple of amendments to other tests to include the StationLocation and update the expectation to be the station name); and the code looks like this:

  function transformOne($pid, $dst) {
    $from = $pid["StationLocation"];
    $to = $pid["Dest{$dst}"];
    $now = $pid["LastUpdated"];
    $wait = $pid["Wait{$dst}"];
    return [ $from => [ $to => [ $this->date($now, $wait) ]]];
  }

And I have wrapped up by checking this in as METROLINK_PHP_FINISH_TRANSFORM.

In the Real World

It almost invariably happens that once I have all my unit tests working, I go to try my code "in the real world" and it just doesn't work. The same is true here. So let's fix the bugs.

First off, it simply doesn't run and I see an error:

$ php-cgi -f metrolink-data.php from[]=FIR from[]=VIC
PHP Fatal error: Uncaught TypeError: arraykeyexists(): Argument #2 ($array) must be of type array, stdClass given in /home/gareth/Projects/IgnoranceBlog/metrolink-php/php/transform.php:25

I've seen this before, so I know what to do. When you decode JSON in PHP, you can either end up with "objects" or you can end up with arrays. I've been thinking about arrays, and testing with those, but I haven't called json_decode in the main code to obtain arrays, so I need to do that. Of course, then that gives me the reverse problem because I have use the object syntax to extract the value from the top-level object. Fixing both of these in one go addresses those issues:

  $data = json_decode($response, true);
  $transformer->analyze($data["value"]);

And the code appears to work:

$ php-cgi -f metrolink-data.php from[]=FIR from[]=VIC
{"Firswood":{"Manchester Airport":["10:29","10:43"],"East Didsbury":["10:40"],"Rochdale Town Centre":["10:32","10:45"],"Victoria":["10:40"]}}

While this may not be obviously wrong where you are, I can see that it's 09:29 here, and these trams all seem to be running an hour in the future. They're not, of course, it's just that something in this code hasn't accounted for GMT/BST. Specifically, I don't even think it's in my code. Looking at the LastUpdated timestamp, it claims to be UTC (the Z on the end), but is in fact in BST. I knew my date code processing had been too easy.

Time for a horrible hack. I want to be quite clear that I am blaming this hack on the fact that the data I am receiving is just wrong. This is not me being lazy. I simply cannot handle the fact that it is giving me a BST date and claiming it is UTC. So I need to say that I am just working in UTC so that the date function gives me back what I'm putting in. I am, if you will, perpetuating the lie.

  require('metrolink_key.php');
  require('transform.php');

  date_default_timezone_set('UTC');

And now it works (yes, 15 minutes have gone by ...)

$ php-cgi -f metrolink-data.php from[]=FIR from[]=VIC
{"Firswood":{"Manchester Airport":["09:59"],"East Didsbury":["09:59","10:10"],"Rochdale Town Centre":["09:45","10:00"],"Victoria":["09:55"]}}

And one more bug

It turns out that I never correctly tested (or implemented) the "from" filter, so I only ever see trams leaving Firswood. Now I look at it, that was obvious from my experiments in the previous section, but I was distracted by the times all being so wrong. This is why real world, end-to-end testing is so important!

OK, let's fix that one too, shall we? It's quite simple to write both a passing and a failing test:

  public function test_from_Firswood_passes_filter_in_Dest0() {
    $transformer = new Transformer(["from" => ["FIR"]]);
    $matches = $transformer->filter([
      "TLAREF" => "FIR",
      "Dest0" => "Victoria",
      "Wait0" => "6",
      "LastUpdated" => "2024-09-17T19:52:55Z"
    ], 0);
    $this->assertTrue($matches);
  }

  public function test_from_Victoria_passes_filter_in_Dest0() {
    $transformer = new Transformer(["from" => ["VIC"]]);
    $matches = $transformer->filter([
      "TLAREF" => "VIC",
      "Dest0" => "Altrincham",
      "Wait0" => "6",
      "LastUpdated" => "2024-09-17T19:52:55Z"
    ], 0);
    $this->assertTrue($matches);
  }

And then we can change the filter implementation:

    // Apply the origin filters
    if ($this->from) {
    if (!array_key_exists("TLAREF", $item)) return false;
    $isFrom = $item["TLAREF"];
    $anyFrom = false;
    foreach ($this->from as $f) {
      if ($isFrom == $f)
      $anyFrom = true;
    }
    if (!$anyFrom) return $anyFrom;
    }

And now let's see if we can get data starting at Victoria:

$ php-cgi -f metrolink-data.php from[]=VIC
{"Victoria":{"East Didsbury":["09:53","10:06"],"Piccadilly":["10:04"],"Manchester Airport":["09:55","10:10"],"Bury":["09:55","10:10"],"Rochdale Town Centre":["10:04"]}}

Yes, that's better.

Monkey C

I bought my first smartwatch back in 2011, but I wasn't at all happy with it. As I recall, there were three reasons:

• The watch face was black most of the time, and you had to "do something" to get it to light up;
• The battery barely lasted for a day;
• It wasn't as smart as they made out, and you couldn't program it.

I said I wouldn't buy another one until they sorted all this out. In 2017, I thought they had and bought a Garmin vivosport, which had a continuously-on display, a battery that lasted a week, and could allegedly be programmed in a language called Monkey C. I remember sitting in a hotel room trying to "sideload" a sample application only to find that the vivosport did not actually support installing custom apps, just the ones that it came with. Oh, well, two out of three ain't bad.

When that watch died two years later, I bought a vivoactive, making sure when I did that it supported custom apps. By that time, I had started writing this blog and thought that further investigation of the watch and its ecosystem would be something good to do here. And then didn't.

But over the years, I've thought of a number of applications that I would be interested in having run on the watch, one of which keeps on coming up with regularity: since I live in the centre of Manchester, I would like to be able to know which trams are going where on a frequent basis, especially the thorny question of whether I should walk to Market St, Piccadilly Gardens or St Peter's Square to catch a tram to Altrincham.

A couple of years ago, while ruminating on this problem, I found that Metrolink had a real-time data feed that can return all the information on all the departure boards in the city. These boards are apparently called PIDs (Platform Information Displays) and there is an OData feed to access them. I was going to include a link to how to access this yourself, but apparently the Bee Network has decided to go in a different direction, and has deprecated this feed. At the time I wrote a (simple?) and hacky PHP script to interrogate the data and show the times of the next trams leaving certain stations in the city centre. I won't show this code here, although I have checked it in to the repository as samples/metrolink.php if you want to use it as evidence against me. No, there are no prizes for figuring out what it does or how, but perhaps there should be.

Fortunately for me, in spite of the feed being deprecated, I can still gather the data, but sadly you cannot follow along with the code that directly accesses the Metrolink data. On the other hand, I have decided to access that from a web server and have that return a simplified form of the data to the watch - so you can follow along with the watch code accessing my server.

The Plan

So, here's the plan. Garmin watches have things they call "widgets". A widget is something that you can interact with to a limited extent and that you can access by swiping up or down the "widget carousel". Once there, you can swipe (I believe) left or right to see different "views" of the same widget, or tap on a selectable area to choose a specific area of information.

My thought is to have a predetermined number of "well-known" routes - city centre to Altrincham, leaving the Trafford Centre, Oldham to Manchester, for example - that I can swipe between. The "default view" will always be the one I looked at last, so that when I am thinking about catching a tram, I can "prime" it with the route I want and then it will appear more quickly when I swipe at my watch.

For now, I plan to have these hardcoded into the watch software, but have that send a request to the server stating the starting stations and lines that I am interested in. The server will then return a list of pairs (station and line) with a list of times of next trams. The server code (in PHP) will be responsible for contacting the Metrolink OData server and processing all the data in the system to produce this minimal amount of data.

This amount of code would ensure that I know how to:

• Get up and running with Monkey C, including setting up a development environment and sideloading to the watch;
• Build a basic "widget" app;
• Access the internet from Monkey C and process a JSON resource.

An extension activity would be to move the hardcoded routes into an app on the phone: the app enables the user to customise the routes they think they would like to take and to have each of these be one of the views in the widget. This would obviously require writing a whole Android app, but it would allow investigation of the communication between phone and watch, and more investigation of storing information on the watch.

The Data

I know I struggle a lot with other people's data decisions. My own intuitions about how to represent a real world system in a formal system are so strong that I often struggle to relate to the approaches that other people take, particularly when I cannot sit down with them and try to understand their viewpoint. This is a case in point.

In my mind, the key concepts I would model on a transit network would be the idea of a "tram", "location" and "station". And I would at any time identify where a "tram" was by giving a "location" which consisted of the amount it was (probably in time units) past the last "station" it stopped at. I would then have a network topography that indicated the connections (and expected time) between each pair of stations.

The Metrolink data is not like that. Presumably they have some "raw" data like this somewhere, from which they calculate all the data they do give you. But the data that is streamed is literally the data that is presented on the departure boards on all the stations. And when I say "literally", I mean that literally. You can ask for the contents of any of the departure boards around the city and you can see what is displayed on it, down to the memo line across the bottom (apologies to anybody who is lost by this description; take a trip to Manchester and you'll understand). For stations with multiple platforms (like St Peter's Square), there are multiple boards you can access. In fact, almost all stations have at least two boards - one for each direction. And each board has up to three trams on it. So how much information you can obtain about the network depends on how many lines go through a station.

I have considered a number of times whether it is possible to reverse engineer the data that they give you and build a complete map of the network including where all the trams are right now (by seeing how far each tram is from various stations along its line). I have also considered recently whether it would be possible to get an AI to do this. But it has never seemed worth the effort.

I have checked in a sample of the PID data you get back as samples/metrolink.json. As far as I can tell, this is just a json object wrapped around a value field which is an array of 231 PID displays. Here is one of these (chosen not quite at random):

{
      "Id": 1309608672,
      "Line": "South Manchester",
      "TLAREF": "FIR",
      "PIDREF": "FIR-TPID02",
      "StationLocation": "Firswood",
      "AtcoCode": "9400ZZMAFIR1",
      "Direction": "Incoming",
      "Dest0": "Victoria",
      "Carriages0": "Single",
      "Status0": "Due",
      "Wait0": "6",
      "Dest1": "Rochdale Town Centre",
      "Carriages1": "Single",
      "Status1": "Due",
      "Wait1": "10",
      "Dest2": "Victoria",
      "Carriages2": "Single",
      "Status2": "Due",
      "Wait2": "16",
      "Dest3": "",
      "Carriages3": "",
      "Status3": "",
      "Wait3": "",
      "MessageBoard": "On Tuesday 17th September Manchester United welcome Barnsley to Old Trafford. Kick Off is 8pm and services will be busier tha
n usual. Please allow extra time for travel.",
      "LastUpdated": "2024-09-17T19:52:55Z"
},

This is the departure board the the Incoming platform at Firswood station, which is on the East Didsbury-Rochdale and Manchester Airport-Victoria lines. The board shows up to four trams coming along: 0, 1, 2 and 3. Tram 0 is going to Victoria, has just one unit and is due in six minutes. So if I'm interested in trams from Firswood going to Victoria, I would expect that my server would query this, process it, and return something like the following:

{
  "Firswood": {
    "Victoria": [ "19:59", "20:09" ]
  }
}

And then the watch will be expected to convert that into a user-friendly display.

Let's Get Started

As I write this, I have done quite a bit of reading about ConnectIQ and Monkey C, and I think I know what I'm doing. But apart from my small foray into it seven years ago have no practical experience. Some of the decisions I have already "made" have been influenced by that "small amount of knowledge". I am assuming I want to do as little as possible on the watch, but for now I don't want to involve the phone more than necessary (i.e. I don't want to write a specific app on the phone), so I want to have the watch communicate directly with my web server and then have that pull back the data from TfGM's servers.

So I'm going to start by writing some PHP code to do that, and then write the code for a widget in Monkey C.

Let's get started!