PHP to read metrolink data

I already have PHP installed on my linux box; I think this is the default. I have previously installed Visual Studio Code (VSCode) and I think that's a fairly common thing to have installed. I don't already have a plugin for PHP on VSCode, but I'm installing the DevSense PHP plugin right now from the "extensions" tab of VSCode.

So I can now write what I would consider the minimal code to read the metrolink data feed:

<?php
  require('metrolink_key.php');

  $ch = curl_init("https://api.tfgm.com/odata/Metrolinks");
  curl_setopt($ch, CURLOPT_HTTPHEADER, array($metrolink_key));
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, false);
  $response = curl_exec($ch);
?>

The require statement incorporates the security key you need in order to access the OData feed. This is not present in the repository since it is a secret, but the samples directory has a file metrolink_key_template.php which shows you what it would look like. Then the next four lines access the OData feed, giving the URI, the security key, specifying that we would like the output to be returned as a value from the function call (rather than being sent directly to the client) and then executing the request.

In theory at least, this should run on our local machine:

$ php metrolink-data.php
PHP Fatal error: Uncaught Error: Call to undefined function curl_init() in /home/gareth/Projects/IgnoranceBlog/metrolink-php/php/metrolink-data.php:4

So it would seem that while I have php installed, I don't have the PHP curl module installed. It would seem that the required fix on linux is:

$ sudo apt-get install php-curl

After this, I can try again and get the expected absence of any output:

$ php metrolink-data.php

Checked in and tagged as METROLINK_PHP_MINIMAL.

Transforming the data

The next step is to try and do this transformation. I want to write some PHP tests around this (I've never used phpUnit before, so this will be interesting in that way, too), but at the top level I want to translate the JSON response into objects, and then I want to transform the resultant objects into JSON. None of that is going to be tested, but the core is a transform() function which I do want to test, and to keep things straight, I'll put that in its own file.

The main program now looks like this:

<?php
  require('metrolink_key.php');
  require('transform.php');

  $ch = curl_init("https://api.tfgm.com/odata/Metrolinks");
  curl_setopt($ch, CURLOPT_HTTPHEADER, array($metrolink_key));
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
  $response = curl_exec($ch);

  $data = json_decode($response);
  $transformed = transform($data->value);
  print json_encode($transformed);
?>

and the transform() function at this stage just returns some sample data:

<?php
  function transform($odata) {
  return array("Firswood" => array("Victoria" => ["19:59", "20:09"]));
  }
?>

Checked in and tagged as METROLINK_PHP_TRANSFORM.

Installing and using phpUnit

The first step is to install phpUnit, which is described here, but on closer inspection it seems that I can install it with apt:

$ phpunit
Command 'phpunit' not found, but can be installed with:
sudo apt install phpunit

I can do that.

I can now write a simple (failing) test:

<?php
use PHPUnit\Framework\TestCase;

require("../php/transform.php");

final class Transform_test extends TestCase
{
  public function test_something() {

    $foo = transform(array());
    // $this->assertInstanceOf("", $transformer);
    $this->fail("hello, world");
  }
}

and run it on the command line:

$ phpunit *
PHPUnit 9.5.10 by Sebastian Bergmann and contributors.


F 1 / 1 (100%)


Time: 00:00.004, Memory: 4.00 MB


There was 1 failure:


1) Transform_test::test_something
hello, world


/home/gareth/Projects/IgnoranceBlog/metrolink-php/test/transform_test.php:12


FAILURES!
Tests: 1, Assertions: 1, Failures: 1.

But this doesn't seem to run inside VSCode. Googling around brought me to this article, which has cryptic instructions I don't understand, but I can try following:

$ composer require --dev phpunit/phpunit
Command 'composer' not found, but can be installed with:
sudo apt install composer
$ sudo apt install composer
...
0 to upgrade, 22 to newly install, 0 to remove and 9 not to upgrade.
Need to get 930 kB of archives.
...
$ composer require --dev phpunit/phpunit
Using version ^10.5 for phpunit/phpunit
./composer.json has been created
Running composer update phpunit/phpunit
Loading composer repositories with package information
Updating dependencies
...

And now I can run the tests in VSCode, using C-S-P and typing PHPUnit Test:

Executing task in folder vscode-java: "{php}" {phpargs} "{phpunit}" {phpunitargs} '/home/gareth/Projects/IgnoranceBlog/metrolink-php/vendor/bin/phpunit' --colors=always '/home/gareth/Projects/IgnoranceBlog/metrolink-php/php/transform.php'
/usr/bin/bash: line 1: : command not found

This is still not what I was expecting. It would seem that there are some configuration parameters that still need setting. I tried fiddling with various properties and settings, but nothing I did made it work.

Finally I gave up and decided to add a task. But I could not find where to put my tasks.json file. It turns out that in a VSCode "workspace" you don't put them in tasks.json but directly in the workspace, like so:

      "path": "metrolink-php"
    }
  ],
  "settings": {},
  "tasks": {
    "version": "2.0.0",
    "tasks": [
      {
        "label": "Metrolink PHP Tests",
        "type": "shell",
        "command": "phpunit",
        "args": ["*.php"],
        "group": "test",
        "options": {
          "cwd": "${workspaceFolder}/../metrolink-php/test"
        },
        "presentation": {
          "reveal": "always"
        }
      }
    ]
  }
}

And then it can be run with CTRL-SHIFT-P followed by Run Task followed by Metrolink PHP Tests. This is too much hassle for me, so I created a keyboard shortcut:

[
    {
        "key": "alt+f11",
        "command": "workbench.action.tasks.runTask",
        "args": "Metrolink PHP Tests"
    }
]

and placed this in the file ~/.config/Code/User/keybindings.json, which is where it would seem it is looked for on my system (I couldn't find a reference to this anywhere; I just created one in the editor to bind ALT-F11 to runTask and then searched for keybindings.json and edited it to add the task name).

OK, so now we can see our test fail inside VSCode:

Executing task in folder test: phpunit *.php


PHPUnit 9.5.10 by Sebastian Bergmann and contributors.


F 1 / 1 (100%)


Time: 00:00.004, Memory: 4.00 MB


There was 1 failure:


1) Transform_test::test_something
hello, world


/home/gareth/Projects/IgnoranceBlog/metrolink-php/test/transform_test.php:12


FAILURES!
Tests: 1, Assertions: 1, Failures: 1.
 * Terminal will be reused by tasks, press any key to close it.

And that is all checked in and "working" and tagged as METROLINK_PHP_PHPUNIT_FAIL.

Transforming the Data

The PHP function json_decode automatically transforms all of the JSON constructs to built-in PHP ones, and, in particular converts a JSON array to a PHP array and a JSON object to a PHP associative array. So by the time we have decoded the JSON and extracted the value property from the top level (which is done in the main code already shown), we pass an array of associative arrays to the transform() function, each of which has the information for one PID in it.

So the code for transform wants to iterate through this array, extracting and filtering the station location and destinations, and building up a map of trams between the two.

Hang on a sec! As I write "filter" I realize that I am missing a key part of code here, in that the top-level code needs to process the incoming request to identify the stations you want to travel between. That is, it needs to handle requests like:

metrolink-data.php?from=FIR&to=VIC&to=ROC

and when I say "like", I think over the course of time many more options will emerge, but for now the only requirement is that there must be at least one from and it is possible to have multiple from and multiple to parameters and to create a "grid" or "matrix" of times between the two.

That also means that it probably makes more sense to expose the Transformer class (which I was going to bury inside transform()) to the top level to take these parameters.

  $transformer = new Transformer($_GET);

  $ch = curl_init("https://api.tfgm.com/odata/Metrolinks");
  curl_setopt($ch, CURLOPT_HTTPHEADER, array($metrolink_key));
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
  $response = curl_exec($ch);

  $data = json_decode($response);
  $transformed = $transformer->transform($data->value);

Here we are passing the PHP superglobal $_GET into the Transformer constructor. What is that and what does it look like? It's an associative array of parameter names to values. You can see this in operation on the command line by dumping them out in the code:

  var_dump(value: $_GET);

and then running the program using the php-cgi binary:

$ php-cgi -f metrolink-data.php from[]=FIR from[]=VIC
array(1) {
  ["from"]=>
  array(2) {
    [0]=>
    string(3) "FIR"
    [1]=>
    string(3) "VIC"
  }
}

In order for this to work with multiple arguments with the same name, you will see that we have to give the argument name the array extension, [].

So, we can now rewrite our unit test to take this into account:

final class Transform_test extends TestCase
{
  public function test_extract_from() {
    $transformer = new Transformer(["from" => ["FIR", "VIC"]]);
    $this->assertEquals(["FIR", "VIC"], $transformer->from);
  }
}

And we can check all of that in as METROLINK_PHP_FIRST_TEST.

Some negative tests

It's fairly easy to knock out some "negative" tests, that is, tests for which the answer is always the empty map. These don't help us very much, but they do "ground" the code inasmuch as they stop us writing something that does not meet these criteria, and make sure that we have an empty array as the "default" value.

  public function test_no_pids_produces_no_output() {
    $transformer = new Transformer(["from" => ["FIR", "VIC"]]);
    $out = $transformer->transform([]);
    $this->assertEquals([], $out);
  }

  public function test_requiring_from_fir_excludes_from_air() {
    $transformer = new Transformer(["from" => ["FIR"]]);
    $out = $transformer->transform(["TLAREF" => "AIR"]);
    $this->assertEquals([], $out);
  }

  public function test_requiring_to_fir_excludes_to_air() {
    $transformer = new Transformer(["to" => ["FIR"]]);
    $out = $transformer->transform(["TLAREF" => "AIR"]);
    $this->assertEquals([], $out);
  }
}

And we need a trivial implementation of transform():

  function transform($odata) {
    return [];
  }

And this is all checked in as METROLINK_PHP_NEGATIVE_TESTS.

Testing the Loop

So we can now assert that if we receive a single entry with matching criteria, it is properly transformed.

  public function test_from_fir_includes_a_single_matching_entry() {
    $transformer = new Transformer(["from" => ["FIR"]]);
    $out = $transformer->transform([[
      "TLAREF" => "FIR",
      "Dest0" => "Victoria",
      "Wait0" => "6",
      "LastUpdated" => "2024-09-17T19:52:55Z"
    ]]);
    $this->assertEquals([["FIR" => ["VIC" => [ "19:58" ]]]], $out);
  }

(Note that in writing this test, I discovered that the previous tests had only provided one member, not a list, so changed them.)

Checked in as METROLINK_PHP_LOOP_TEST.

Filtering

For me, unit testing is always about how the code you've written is factored. In this case, there is basically a loop (which was tested above), then filtering, then the actual transformation. Having dealt with the loop, I now want to nail down (almost) all the cases to do with filtering.

Initially, at least, I want to offer three different cases of filtering: the from parameter should be a 3-letter code matching the TLAREF field, which is also the first three letters of the PIDREF field; the to field can be either the full name of the destination, i.e. one of Dest0, Dest1 or Dest2 (assuming those fields are there) or else its corresponding 3-letter code; or the to field can be a combination of the direction and line. For this last field, if you look at the data, you can see these two fields: the line is something like "Altrincham" or "South Manchester" and the direction is either "Incoming" or "Outgoing". In order to query this, the user must combine both into a single field, e.g. I-South Manchester or else O-Oldham & Rochdale. The idea of supporting this format is that you often know you want to go somewhere (e.g. Queens Road) but you don't know whether the tram is going to Bury, Heaton Park or Crumpsall - and you don't care. It's going the right way.

However, handling the 3-letter destination codes and the the "line" destinations requires us to do some pre-processing on the data: we need to find the actual destination names corresponding to the input.

As I go deeper and deeper into this, I discover more and more wrinkles. As I work towards processing this destination data, I realize that I need to consider the three destinations "separately", and so it makes sense to add an additional parameter to both filter and transformOne to specify that I want to deal with each destination index in turn, and then I need to merge these answers together to end up with the array I originally thought of. So much code, so little time.

The tests come out looking like this:

  public function test_firswood_has_two_trams_to_Victoria() {
    $transformer = new Transformer(["from" => ["FIR"]]);
    $out = $transformer->transform([[
      "TLAREF" => "FIR",
      "Dest0" => "Victoria",
      "Wait0" => "6",
      "Dest2" => "Victoria",
      "Wait2" => "18",
      "LastUpdated" => "2024-09-17T19:52:55Z"
    ]]);
    $this->assertEquals(["FIR" => ["VIC" => [ "19:58", "20:10" ]]], $out);
  }

  public function test_tram_to_Altrincham_passes_with_empty_filter_in_Dest0() {
    $transformer = new Transformer([]);
    $matches = $transformer->filter([
      "TLAREF" => "SPS",
      "Dest0" => "Altrincham",
      "Wait0" => "6",
      "LastUpdated" => "2024-09-17T19:52:55Z"
    ], 0);
    $this->assertTrue($matches);
  }

  public function test_to_Altrincham_passes_filter_in_Dest0() {
    $transformer = new Transformer(["to" => ["Altrincham"]]);
    $matches = $transformer->filter([
      "TLAREF" => "SPS",
      "Dest0" => "Altrincham",
      "Wait0" => "6",
      "LastUpdated" => "2024-09-17T19:52:55Z"
    ], 0);
    $this->assertTrue($matches);
  }}

and they pass with this implementation, calling filter, transformOne and merge repeatedly:

    foreach ($odata as $item) {
    for ($i=0;$i<3;$i++) {
      if ($this->filter($item, $i))
      $ret = $this->merge($ret, $this->transformOne($item, $i));
    }

and the filter implementation itself:

  function filter($item, $dst) {
    // This tram must be going *somewhere* (a lot of them are blank)
    $d = "Dest{$dst}";
    if (!array_key_exists($d, $item) || $item[$d] == "") return false;

    // Apply the destination filters
    if ($this->to) {
    if ($item[$d] != $this->to[0]) return false;
    }

    // Apply the origin filters
    if ($this->from) {
    if (!array_key_exists("TLAREF", $item)) return false;
    if ($item["TLAREF"] != "FIR") return false;
    }

    // If it didn't fail, it can be included
    return true;
  }

  function transformOne($pid, $dst) {
    if (array_key_exists("TLAREF", $pid)) {
    return [ "FIR" => [ "VIC" => [ $dst == 0 ? "19:58" : "20:10" ]]];
    }
  }

  function merge($ret, $incl) {
    foreach ($incl as $k => $v) {
    if (!array_key_exists($k, $ret)) {
      $ret[$k] = $v;
    } else {
      foreach ($v as $k2 => $v2) {
      foreach ($v2 as $time) {
        $ret[$k][$k2][] = $time;
      }
      }
    }
    }
    return $ret;
  }
  }
?>

(see METROLINK_PHP_FIRST_FILTERS).

Pre-processing the data

In order to handle the pre-processing, I am adding an extra "top-level" analysis step in metrolink-data.php to tell the transformer to analyze all the input data. Apart from anything else, this makes it easier to unit test.

I can then fairly easily write the code that handles the conversion of 3-letter codes to full names:

Testing this with:

  public function test_tla_collection() {
    $transformer = new Transformer(["to" => ["ALT"]]);
    $tlas = $transformer->collectTLAs([[
      "TLAREF" => "ALT",
      "StationLocation" => "Altrincham"
    ],[
      "TLAREF" => "SPS",
      "StationLocation" => "St Peter's Square"
    ],[
      "TLAREF" => "ALT",
      "StationLocation" => "Altrincham"
    ],[
      "TLAREF" => "WST",
      "StationLocation" => "Weaste"
    ]]);
    $this->assertEquals(["ALT" => "Altrincham", "SPS" => "St Peter's Square", "WST" => "Weaste"], $tlas);
  }

  public function test_analysis_can_turn_ALT_to_Altrincham() {
    $transformer = new Transformer(["to" => ["ALT"]]);
    $transformer->analyze([[
      "TLAREF" => "ALT",
      "StationLocation" => "Altrincham"
    ]]);
    $this->assertEquals(["Altrincham"], $transformer->to);
  }

we can write:

  function analyze($odata) {
    $tlas = $this->collectTLAs($odata);
    $input = $this->to;
    $this->to = [];
    foreach ($input as $dest) {
    $this->analyzeOne($tlas, $dest);
    }
  }

  function collectTLAs($odata) {
    $tlas = [];
    foreach ($odata as $pid) {
    // Obviously we need to have a TLAREF and a StationLocation
    if (!array_key_exists("TLAREF", $pid)) continue;
    if (!array_key_exists("StationLocation", $pid)) continue;

    // Once it's done, don't do it again
    if (array_key_exists($pid["TLAREF"], $tlas)) continue;

    // map it
    $tlas[$pid["TLAREF"]] = $pid["StationLocation"];
    }
    return $tlas;
  }

  function analyzeOne($tlas, $dest) {
    if (array_key_exists($dest, $tlas))
    $this->to[] = $tlas[$dest];
    else
    $this->to[] = $dest;
  }

and wire this into $metrolink-data.php" as:

  $data = json_decode($response);
  $transformer->analyze($data->value);
  $transformed = $transformer->transform($data->value);
  print json_encode($transformed);

and check in as METROLINK_PHP_STATION_ANALYSIS.

Processing for "Routes"

The "routes" feature is slightly more complicated because each input can translate into multiple outputs, but otherwise the principle is the same: we look at each entry in the input and assemble a set of "line names" consisting of the direction and line, and then map that to an array of full station names from the StationLocation field. It's also important to note that any given station is not on one exclusive "line" in this sense; in general, it will be on at least two: one "Incoming" and one "Outgoing".

Again, the tests are like this:

  public function test_line_collection() {
    $transformer = new Transformer([]);
    $lines = $transformer->collectLines([[
      "StationLocation" => "Firswood",
      "Line" => "South Manchester",
      "Direction" => "Outgoing",
    ],[
      "StationLocation" => "Firswood",
      "Line" => "South Manchester",
      "Direction" => "Incoming",
    ],[
      "StationLocation" => "Central Park",
      "Direction" => "Outgoing",
      "Line" => "Oldham & Rochdale",
    ],[
      "StationLocation" => "Failsworth",
      "Direction" => "Outgoing",
      "Line" => "Oldham & Rochdale",
    ],[
      "StationLocation" => "Freehold",
      "Direction" => "Outgoing",
      "Line" => "Oldham & Rochdale",
    ],[
      "StationLocation" => "Weaste",
      "Direction" => "Incoming",
      "Line" => "Eccles",
    ]]);
    $this->assertEquals([
      "O-South Manchester" => ["Firswood"],
      "I-South Manchester" => ["Firswood"],
      "I-Eccles" => ["Weaste"],
      "O-Oldham & Rochdale" => ["Central Park", "Failsworth", "Freehold"]
    ], $lines);
  }

  public function test_analysis_will_expand_outbound_oldham() {
    $transformer = new Transformer(["to" => ["O-Oldham & Rochdale"]]);
    $transformer->analyze([[
      "StationLocation" => "Firswood",
      "Line" => "South Manchester",
      "Direction" => "Outgoing",
    ],[
      "StationLocation" => "Firswood",
      "Line" => "South Manchester",
      "Direction" => "Incoming",
    ],[
      "StationLocation" => "Central Park",
      "Direction" => "Outgoing",
      "Line" => "Oldham & Rochdale",
    ],[
      "StationLocation" => "Failsworth",
      "Direction" => "Outgoing",
      "Line" => "Oldham & Rochdale",
    ],[
      "StationLocation" => "Freehold",
      "Direction" => "Outgoing",
      "Line" => "Oldham & Rochdale",
    ],[
      "StationLocation" => "Weaste",
      "Direction" => "Incoming",
      "Line" => "Eccles",
    ]]);
    $this->assertEquals(["Central Park", "Failsworth", "Freehold"], $transformer->to);
  }

and the code looks like this:

  function collectLines($odata) {
    $lines = [];
    foreach ($odata as $pid) {
    // Obviously we need the things we want to map
    if (!array_key_exists("Line", $pid)) continue;
    if (!array_key_exists("Direction", $pid)) continue;
    if (!array_key_exists("StationLocation", $pid)) continue;

    $lineName = substr($pid["Direction"], 0, 1) . "-" . $pid["Line"];
    $lines[$lineName][] = $pid["StationLocation"];
    }
    return $lines;
  }

(along with some associated changes to make everything work), which is checked in as METROLINK_PHP_LINE_NOTATION.

Transforming

Up until now, we have just been hacking in the transformed data. Now it's time to actually calculate this. There are two separate parts to this: figuring out the station names; and figuring out the expected arrival times. I have to admit, up until now I have been using the 3-letter codes, but I want to return the actual station names, so some re-work will be necessary. But other than that, getting the correct station names is not hard. Figuring out the arrival time is not hard, per se, but it is, shall we say, complicated. Or possessed of a number of cases and steps.

The time comes from two factors, one is the number of minutes to wait, in the Wait0 or equivalent field; and then other is the LastUpdated field, which is a timestamp. Basically, we need to add the number of minutes to the timestamp. What we decide to do here basically depends on how easy it is to parse and reformat times: if it's easy, we'll use the date logic; if it's not, we'll use an icky and hacky string manipulation. A little experimentation shows that a combination of strtotime and date makes the whole thing very easy, so a few unit tests later we have the ability to format the time, and can include that in the transformOne function.

  public function test_adding_0_minutes_is_easy() {
    $transformer = new Transformer([]);
    $when = $transformer->date("2024-09-17T19:52:55Z", 0);
    $this->assertEquals("19:52", $when);
  }

  public function test_adding_5_minutes() {
    $transformer = new Transformer([]);
    $when = $transformer->date("2024-09-17T19:52:55Z", 5);
    $this->assertEquals("19:57", $when);
  }

  public function test_adding_18_minutes_will_wrap() {
    $transformer = new Transformer([]);
    $when = $transformer->date("2024-09-17T19:52:55Z", 18);
    $this->assertEquals("20:10", $when);
  }

  public function test_can_wrap_to_tomorrow() {
    $transformer = new Transformer([]);
    $when = $transformer->date("2024-09-17T23:55:12Z", 6);
    $this->assertEquals("00:01", $when);
  }

And the corresponding code is:

  function transformOne($pid, $dst) {
    if (array_key_exists("TLAREF", $pid)) {
    return [ "FIR" => [ "VIC" => [ $this->date($pid["LastUpdated"], $pid["Wait{$dst}"]) ]]];
    }
  }

  function date($from, $wait) {
    $unix = strtotime($from);
    $when = $unix + $wait * 60;
    $ret = date('H:i', $when);
    return $ret;
  }

All this is checked in with the tag METROLINK_PHP_EXPECTED_TIME.

And so, I can finally finish it all off by implementing the rest of the transformation code (including the re-work on the station names).

The new test looks like this:

  public function test_a_tram_from_Freehold_to_Shaw_is_correctly_reported() {
    $transformer = new Transformer([]);
    $route = $transformer->transformOne([
      "StationLocation" => "Freehold",
      "Dest0" => "Shaw and Crompton",
      "Wait0" => "4",
      "LastUpdated" => "2024-09-17T19:52:55Z"
    ], "0");
    $this->assertEquals(["Freehold" => [ "Shaw and Crompton" => [ "19:56"]]], $route);
  }

(and there are a couple of amendments to other tests to include the StationLocation and update the expectation to be the station name); and the code looks like this:

  function transformOne($pid, $dst) {
    $from = $pid["StationLocation"];
    $to = $pid["Dest{$dst}"];
    $now = $pid["LastUpdated"];
    $wait = $pid["Wait{$dst}"];
    return [ $from => [ $to => [ $this->date($now, $wait) ]]];
  }

And I have wrapped up by checking this in as METROLINK_PHP_FINISH_TRANSFORM.

In the Real World

It almost invariably happens that once I have all my unit tests working, I go to try my code "in the real world" and it just doesn't work. The same is true here. So let's fix the bugs.

First off, it simply doesn't run and I see an error:

$ php-cgi -f metrolink-data.php from[]=FIR from[]=VIC
PHP Fatal error: Uncaught TypeError: arraykeyexists(): Argument #2 ($array) must be of type array, stdClass given in /home/gareth/Projects/IgnoranceBlog/metrolink-php/php/transform.php:25

I've seen this before, so I know what to do. When you decode JSON in PHP, you can either end up with "objects" or you can end up with arrays. I've been thinking about arrays, and testing with those, but I haven't called json_decode in the main code to obtain arrays, so I need to do that. Of course, then that gives me the reverse problem because I have use the object syntax to extract the value from the top-level object. Fixing both of these in one go addresses those issues:

  $data = json_decode($response, true);
  $transformer->analyze($data["value"]);

And the code appears to work:

$ php-cgi -f metrolink-data.php from[]=FIR from[]=VIC
{"Firswood":{"Manchester Airport":["10:29","10:43"],"East Didsbury":["10:40"],"Rochdale Town Centre":["10:32","10:45"],"Victoria":["10:40"]}}

While this may not be obviously wrong where you are, I can see that it's 09:29 here, and these trams all seem to be running an hour in the future. They're not, of course, it's just that something in this code hasn't accounted for GMT/BST. Specifically, I don't even think it's in my code. Looking at the LastUpdated timestamp, it claims to be UTC (the Z on the end), but is in fact in BST. I knew my date code processing had been too easy.

Time for a horrible hack. I want to be quite clear that I am blaming this hack on the fact that the data I am receiving is just wrong. This is not me being lazy. I simply cannot handle the fact that it is giving me a BST date and claiming it is UTC. So I need to say that I am just working in UTC so that the date function gives me back what I'm putting in. I am, if you will, perpetuating the lie.

  require('metrolink_key.php');
  require('transform.php');

  date_default_timezone_set('UTC');

And now it works (yes, 15 minutes have gone by ...)

$ php-cgi -f metrolink-data.php from[]=FIR from[]=VIC
{"Firswood":{"Manchester Airport":["09:59"],"East Didsbury":["09:59","10:10"],"Rochdale Town Centre":["09:45","10:00"],"Victoria":["09:55"]}}

And one more bug

It turns out that I never correctly tested (or implemented) the "from" filter, so I only ever see trams leaving Firswood. Now I look at it, that was obvious from my experiments in the previous section, but I was distracted by the times all being so wrong. This is why real world, end-to-end testing is so important!

OK, let's fix that one too, shall we? It's quite simple to write both a passing and a failing test:

  public function test_from_Firswood_passes_filter_in_Dest0() {
    $transformer = new Transformer(["from" => ["FIR"]]);
    $matches = $transformer->filter([
      "TLAREF" => "FIR",
      "Dest0" => "Victoria",
      "Wait0" => "6",
      "LastUpdated" => "2024-09-17T19:52:55Z"
    ], 0);
    $this->assertTrue($matches);
  }

  public function test_from_Victoria_passes_filter_in_Dest0() {
    $transformer = new Transformer(["from" => ["VIC"]]);
    $matches = $transformer->filter([
      "TLAREF" => "VIC",
      "Dest0" => "Altrincham",
      "Wait0" => "6",
      "LastUpdated" => "2024-09-17T19:52:55Z"
    ], 0);
    $this->assertTrue($matches);
  }

And then we can change the filter implementation:

    // Apply the origin filters
    if ($this->from) {
    if (!array_key_exists("TLAREF", $item)) return false;
    $isFrom = $item["TLAREF"];
    $anyFrom = false;
    foreach ($this->from as $f) {
      if ($isFrom == $f)
      $anyFrom = true;
    }
    if (!$anyFrom) return $anyFrom;
    }

And now let's see if we can get data starting at Victoria:

$ php-cgi -f metrolink-data.php from[]=VIC
{"Victoria":{"East Didsbury":["09:53","10:06"],"Piccadilly":["10:04"],"Manchester Airport":["09:55","10:10"],"Bury":["09:55","10:10"],"Rochdale Town Centre":["10:04"]}}

Yes, that's better.