Wednesday, September 27, 2023

Polishing things off (for now)


In order to finish up this project, at least for a first phase, which is all this is, we need to tidy up the UI. This consists of two things: first of all, adding in some CSS; secondly to add actions on the tab buttons so that clicking on the tab labels will cause the associated tab to be shown; and thirdly to clean up the error handling so that the error tab only appears (and is automatically selected) when there are errors.

It turns out that in order to do all of this, we need to restructure the HTML somewhat and break apart the tab titles and the tab bodies. This adds more complexity all over the place, but none of it is really interesting. For those scoring at home, it is this change that the ensureTabs method depends on.

In fact, none of this work is interesting, so feel free to dig in if you want to, it's really just essential to making it look acceptable.

Future Work

What isn't in the future? This really is just a question of scratching the surface of what can be done. To progress, this tool really needs to be used in anger, with the intent of drawing diagrams that mean something. Earlier in this blog, I have drawn some diagrams by hand, but I'm not going to draw those again just to test this tool out. But next time this blog calls for diagrams, you can be sure I'm going to dig this tool out. It will also be interesting to see how I go about incorporating such a diagram into the blog (I could experiment with that here using the samples, but I'm not going to).

Among other things that still need to be done:
  • No complex connectors have yet been addressed, and these obviously come up all the time in real diagrams.
  • In spite of the claims I may have implied for my algorithm, I think there are quite a few cases where it will be necessary to move nodes after they have been placed.
  • So far we only have "nodes" and "edges". I would like to have a much richer semantic description of the diagrams I want to draw, but I'm not really sure what that would look like until I try it (but would have a domain-specific vocabulary of things like "server", "queue", "notification", "processing", "store" for a cloud-based software system). In addition to improving the clarity of the diagram description, I would expect this to provide information to the layout algorithm.
  • I definitely want to be able to handle very large diagrams - let's say 200 nodes - and be able to expand and compress them by having the tool offer an "overview" or "schematic" of a whole system with just (say) five nodes, each of which can be expanded to a subsystem of more nodes. Obviously, this would be hierarchical. The most important point would be that the actual truth of all the diagrams is only presented once and the other diagrams are just defined in terms of what is included or left out.
  • It seems to me that in certain cases, there needs to be the ability for humans to "override" the layout decisions, for example to indicate that certain nodes should be aligned horizontally or vertically, or that a node should be deflected slightly from its normal place, or that two nodes should have more space between them. On the other hand, I consider all such things "hacks" and wonder what semantic information is missing from the diagram that they are necessary.
  • There should be a mechanism for adding comments to diagrams so that users can collaborate on a diagram, and probably connections to sharing tools such as github and Google Docs.
Interested? Get in touch and let's see what we can do.

Rendering the Diagram


Now we've laid out the diagram, we can move on to the final step of drawing, or rendering it. This is mainly dull and technical, relating to getting what we have in mind onto the canvas. Inasmuch as there is anything interesting to the task, it is figuring out how much space to leave in the gaps between the shapes, and how to label both shapes and connectors so that nothing overlaps. And while that may not be boring, it is tricky.

In order to try and break up some of the complexity, I have added another layer into this, which abstracts away the canvas. This also has the benefit of making the render algorithm testable. The abstraction layer, of course, can only be tested using complex image-comparison tools.

Creating the Diagram Tabs

Anyway, first off, we need to dispose of something that's been hanging over us for a while, to wit, finding the tab and the canvas to draw into. As I said before, I want to make this as stable as possible, so we want each tab to be named according to the first node laid out in the diagram (at least until we have some better way of naming things), and then to have the diagrams remain in the same sequence based on this.

Then, as we render, we reuse the tabs and canvas objects that already existed, and delete or add any to reflect bigger changes in the overall portfolio of diagrams.
  ensureTabs(tabrow, tabdisplay) {
    this.tabs = {};
    var titles = tabrow.querySelectorAll(".diagram-tab");
    var bodies = tabdisplay.querySelectorAll(".diagram-tab");
    var toRemove = [];
    for (var i=0;i<titles.length;i++) {
      toRemove[titles[i].dataset["diagramFor"]] = { title: titles[i], body: bodies[i] };
    }
    for (var i=0;i<this.diagrams.length;i++) {
      var d = this.diagrams[i];
      var t = findTabFor(titles, d.named);
      var b = findTabFor(bodies, d.named);
      if (!t) {
        t = addDiagramTab(tabrow, tabdisplay, d.named);
      } else {
        delete toRemove[d.named];
        t = { title: t, diagram: b };
      }
      this.tabs[d.named] = t;
    }
    var keys = Object.keys(toRemove);
    for (var i=0;i<keys.length;i++) {
      tabrow.removeChild(toRemove[keys[i]]);
    }
  }
(For complicated reasons, this code does not quite align with its history, and only works with the code that comes from polishing the UI which follows).

The function to add a new diagram tab looks like this:
function addDiagramTab(titles, display, name) {
  var t = document.createElement("div");
  t.className = "diagram-tab tab-title";
  t.dataset["diagramFor"] = name;
  t.appendChild(document.createTextNode(name));
  titles.appendChild(t);
  clickFor(ev => selectDiagramTab(name)) (t);

  var d = document.createElement("div");
  d.className = "diagram-tab tab-body";
  d.dataset["diagramFor"] = name;
  var cd = document.createElement("div");
  var c = document.createElement("canvas");
  c.className="diagram";
  c.setAttribute("width", "1200");
  c.setAttribute("height", "800");
  cd.appendChild(c);
  d.appendChild(cd);
  display.appendChild(d);

  return { title: t, diagram: d };
 }

export default Portfolio;

Rendering the Diagram

Then we can come back and implement the render code. The first step is to implement the Render interface, and collect all the information from Layout about the shapes and connectors. As we do this, we collect metadata about the rows and columns which are implied in the layout.
  shape(x, y, s) {
    if (x >= this.maxx) this.maxx = x+1;
    if (y >= this.maxy) this.maxy = y+1;

    if (!this.rows[y]) {
      this.rows[y] = new RowInfo();
    }
    if (!this.columns[x]) {
      this.columns[x] = new ColumnInfo();
    }
    this.rows[y].include(s);
    this.columns[x].include(s);

    this.shapes.push({ x: x, y: y, s: s });
  }
Ultimately, the connector method needs to do similar analysis, figuring out how wide individual canals will need to be, and thus the spacing between the grid. But we haven't got that far yet, so it's just a question of recording the fact that the connector needs to be drawn.
  connector(pts) {
    this.connectors.push(pts);
  }
When layout is complete, it calls the done method which figures out what the actual size of all the elements, including the overall diagram, will be and then draws all the shapes and connectors in the right places, using the canvas abstraction.
  done() {
    this.figureGrid();

    // draw all the shapes
    for (var i=0;i<this.shapes.length;i++) {
      var si = this.shapes[i];
      this.drawShape(si.x, si.y, si.s);
    }

    // draw the connectors
    for (var i=0;i<this.connectors.length;i++) {
      var pts = this.connectors[i];
      this.drawConnector(pts);
    }
  }
  figureGrid() {
    // based on the rows and columns, we can figure out what the grid should be
    // because it's a grid, which just need two sets of values, one that maps x units to x locations and one for y
    var xpos = minCanal; // the left margin
    for (var i=0;i<this.maxx;i++) {
      if (!this.columns[i]) continue; // this should not be possible, but safety first ...
      var c = this.columns[i];
      c.from = xpos;
      xpos += c.maxwidth;
      c.to = xpos;
      c.channels = [];
      xpos += c.right;
    }
    this.totalWidth = xpos;

    var ypos = minCanal; // the top margin
    for (var i=0;i<this.maxy;i++) {
      if (!this.rows[i]) continue; // this should not be possible, but safety first ...
      var c = this.rows[i];
      c.from = ypos;
      ypos += c.maxheight;
      c.to = ypos;
      c.channels = [];
      ypos += c.below;
    }
    this.totalHeight = ypos;
  }

Tuesday, September 26, 2023

Laying out the diagram


Now we know we have a diagram (possibly a set of diagrams, but we will deal with them one at a time) which has a set of fully connected nodes and we want to lay it out on a two-dimensional grid.

As I said before, I am no expert on graph theory or layout, so I'm not going to do anything complicated. I am going to lay out the nodes on a rectangular grid with "standard" coordinates starting at 0 and going right and down. It will be the job of the rendering engine to make that look nice by adapting those coordinates to something in real space.

The abstract model will have "shapes" and "connectors". A shape is simply at a point in this 2D space, and connectors run between them in the "gaps". This is slightly more difficult to describe, but I view the space between the shapes as being like rivers or canals, and in that there are "channels". The rule is that any given channel (that is, a specific segment of a specific vertical or horizontal line between the shapes) may only have one connector line running down it. They may, however, cross an arbitrary number of times. A connector will always join at least two shapes (or quite possibly more); each limb will ultimately reach a shape; in between it will always run down channels between the shapes, never crossing a shape; the various limbs of a connector connecting more than 2 shapes must be connected in some way at some point so that they can be seen not to be crossing.

My approach for trying to get the graph as "tight" as possible (that is, as many shapes close to their friends as possible, thus reducing the length of the connectors) is to place first the shape with the most number of connectors to it, then to try and place all the shapes connected to it (again, going from the most connections to the least), and then (recursively) working our way down the list. This may or may not work out well. It may also present additional challenges as and when we introduce constraints (such as a desire to have certain shapes all be on the same row).

Before we get started, I should (perhaps) clarify that I did not write this in the order in which I presented it, but did some layout and some rendering and slowly worked my way to this algorithm. Also, I worked up to it using unit tests for some simple cases and, if and when I come back to add more complex cases, the algorithm will change again.

Layout is kicked off from inside the Diagram, but it basically just delegates to the LayoutAlgorithm to first lay the diagram out and then to render it. Obviously to do that, it passes in the knowledge it has collected about the diagram.
  layout(render) {
    var alg = new LayoutAlgorithm(this.errors, this.nodes, this.nodeNames, this.edges);
    alg.layout();
    alg.render(render);
  }
The layout algorithm comes in two parts: we first place the nodes on a hypothetical grid, and then we connect them with the edges.
  layout() {
    // handle the empty diagram
    if (this.nodes.length == 0) return;

    this.placeNodes();
    this.connectNodes();
  }
Placing the nodes is, of course, where all the meaty stuff happens. We want to lay the nodes out in such a way that nodes which are connected are close together and then connecting them via the edges should be relatively easy. We need to place the nodes one at a time; the first node is chosen to be the one with the most connections; after that, we can choose any node which is connected to a node already in the diagram. In practice, we try to choose the one with the most connections to nodes already in the diagram.

To make this code simpler, we delegate the process of tracking which nodes are eligible for selection (and the appropriate order) to a class called PushFrontier (so named because it is continually pushing forward the frontier of the graph which has been chosen) and the process of placing the nodes to a class called Placement.

With these in place, the function to place the nodes is not too awful:
  placeNodes() {
    // The frontier model is designed to provide us with the nodes in a "most connected" order
    var frontier = new PushFrontier(this.nodes, this.edges);

    // First place the most connected node
    var name = frontier.first();
    this.placement.place(0, 0, this.nameMap[name]);

    // Now, iteratively consider all the remaining nodes
    // Each node we receive will be connected to one or more (preferably more) nodes already in the diagram
    while (frontier.push(name)) {
      name = frontier.next();

      // try and figure out where it's near ...
      var avgpos = this.findNear(frontier, name);

      // noew place it somewhere near there that isn't occupied
      this.placement.place(avgpos.x, avgpos.y, this.nameMap[name]);
    }
  }

Selecting the next node

The PushFrontier code works by first analyzing all the nodes and edges to build an inverted table of all the nodes with their connections. It then sorts these to find the nodes with the greatest number of connections and then identifies the list of connection arities. It turns out that this is a very good fit for a radix sort.
class PushFrontier {
  constructor(nodes, edges) {
    this.conns = this.groupByConnections(nodes, edges);
    this.sorted = this.sortConnections(this.conns);
    this.radices = this.gatherRadices(this.sorted);
    this.done = [];
    this.frontier = [];
  }
The code to choose the first node simply looks for the first node in the list of nodes with the highest radix.
  first() {
    var r = this.radices[0];
    var node = this.sorted[r][0];
    return node;
  }
Once the node has been added to the graph, the push method is called, which adds it to the list of nodes that have been used (both so that it will not be returned again, and so that we can see which nodes are already in the graph). It then looks at all the nodes which are connected to this node (and which have not already been placed) and adds those to the expanding frontier.
  push(node) {
    this.done.push(node);
    var add = this.conns[node];
    for (var i=0;i<add.length;i++) {
      if (!this.done.includes(add[i]))
      this.frontier.push(add[i]);
    }
    return this.frontier.length > 0;
  }
And finally, when we are asked to provide the next node, we scan through the list of frontier nodes (all of which have the property that they have not yet been placed, but are connected to a node which has been placed) and choose the one which:
  • has the most connections to a node in the graph; or, failing that
  • is tied for the most connections in the graph and, among those, has the most total connections; or, failing that
  • is tied for the most number of connections, and has the alphabetically first name.
  next() {
    var mostInGraph = -1;
    var mostTotal = -1;
    var ret = null;
    var pos = -1;
    for (var i=0;i<this.frontier.length;i++) {
      var node = this.frontier[i];
      var total = this.conns[node].length;
      var inGraph = this.chooseDone(this.conns[node]);
      var chooseMe = false;
      if (inGraph > mostInGraph) { // it is connected to the most already in the graph
        chooseMe = true;
      } else if (inGraph == mostInGraph && total > mostTotal) { // it is connected to the most other total nodes
        chooseMe = true;
      } else if (inGraph == mostInGraph && total == mostTotal && node < ret) { // it is at least as good and has an earlier name
        chooseMe = true;
      }
      if (chooseMe) {
        mostInGraph = inGraph;
        mostTotal = total;
        ret = node;
        pos = i;
      }
    }
    this.frontier.splice(pos, 1);
    return ret;
  }

Placing the Nodes on a Grid

The other half of the task is to figure out where to put each node once we have identified it as the next node to be placed on the grid. For the first node, this is easy: we just say we want to place it at (0,0) and we're done. For all subsequent nodes, what we really want to do is to place them at the average position of all the nodes they are connected to.
  // Look at all the nodes connected to this one which have already been placed and figure out their average position
  // We want to be somewhere near there
  findNear(frontier, name) {
    var sx = 0, sy = 0, cnt = 0;
    var conns = frontier.connectedTo(name);
    for (var i=0;i<conns.length;i++) {
      var c = conns[i];
      var pl = this.placement.isPlaced(c);
      if (pl) {
        sx += pl.x;
        sy += pl.y;
        cnt++;
      }
    }
    return { x: sx/cnt, y: sy/cnt };
  }
Of course, there is a significant probability that that space is already taken, and we need to put it somewhere else "near" there. The Placement class is responsible for handling that. For the cases I have added tests for so far, this code is adequate, but clearly there will be the need to do more later.
  // find a slot for the node to go in, ideally the one it asked for
  findSlot(x, y) {
    // rounding is good from the perspective of trying something, but we possibly should try "all 4" (if not an integer) before trying neighbouring squares.
    x = Math.round(x);
    y = Math.round(y);

    // if that slot is free, go for it!
    if (!this.haveNodeAt(x, y)) return { x, y };

    // first try the four cardinal points
    if (!this.haveNodeAt(x+1, y)) return { x: x+1, y: y };
    if (!this.haveNodeAt(x, y+1)) return { x: x, y: y+1 };
    if (!this.haveNodeAt(x-1, y)) return { x: x-1, y: y };
    if (!this.haveNodeAt(x, y-1)) return { x: x, y: y-1 };

    this.errors.raise("can't handle this case in layout");
  }

Connecting Nodes

Once we have placed all the nodes, we need to go back and look at all the edges and figure out how to connect them. My thought for a connector is that it is a series of line segments which are joined together in such a way as to make a continuous line between two nodes. In order to trace such a path, it must have a series of points, all of which are relative to the grid. The ultimate idea (not yet in code) is that between the main grid nodes are a set of "canals" divided into "channels" through which the edges can flow. This means that there are two distinct types of points: those where the point is attached to a side of a node; and those where the point is in a canal. For now, we only have the points connected to the side of a node (called a ShapeEdge). This point is identified by the (x,y) grid position of the node, which side of the node it is attached to, and which "channel" it is going to use (here, the same idea of a channel is to be able to distinguish between two different edges both coming into the same side of a node).

For now, I have only written the code to handle connections between two nodes which are on the same horizontal or vertical line (the only cases I have considered as yet). In this case, the connector has exactly two points, each of which is a ShapeEdge. I have also not handled the case with multiple connectors from the same side of the same node.
  connectNodes() {
    for (var i=0;i<this.edges.length;i++) {
      var e = this.edges[i];
      if (e.ends.length != 2) {
        this.errors.raise("cannot handle this case yet");
        continue;
      }
      var f = e.ends[0];
      var t = e.ends[1];
      var fn = this.placement.isPlaced(f.name);
      var tn = this.placement.isPlaced(t.name);
      // This is nothing like sophisticated enough for 90% of cases. But it passes all current unit tests
      this.placement.connect([ new ShapeEdge(fn.x, fn.y, tn.x-fn.x, tn.y - fn.y, 0), new ShapeEdge(tn.x, tn.y, fn.x - tn.x, fn.y - tn.y, 0) ]);
    }
  }

Partitioning the Diagram


In some ideal world, the resultant diagram from parsing would be fully connected.

This isn't true for a number of reasons. The most obvious one is just user "error". As the user is entering information, they simply may not be aware of one or more connections, or unsure how to describe them. It is better to handle this case than to complain and force the user to take corrective action.

Another is that we want to explicitly support the idea of "modules" or "subsystems" which deliberately target a set of nodes.

And it seems to me that if you have two diagrams that aren't connected, then you should have two separate diagrams drawn, not put all the information onto one diagram.

So the next phase is to try and follow the edges between the nodes and partition the diagram into a number of smaller, independent diagrams. In the case of subsystems (which are not yet supported), the expectation would be to create a "duplicate" diagram where we copy across the requested nodes and edges, allowing a number of edges to be "dangling" with no nodes to attach them to.

(Note that dangling edges should not normally be allowed, but as yet we have not done any verification on our model: this also covers the fact that we shouldn't allow multiple nodes to have the same name.)

We are going to create a number (which may be 1) of diagrams based on the information in the original parsed diagram in the partitionInto method in DiagramModel. Each one will be given a name so that we can identify them, and this name will be the name of the (alphabetically) first node in the diagram. I am hoping this will be (relatively) tolerant to change when we go through the relayout-redraw cycle so that the visual display is relatively static.

The process by which we are going to do this is really very simple. We add a node to a diagram, keeping track of the nodes we've added as we go. We then find all the edges that connect to that node, and ask them for all the nodes they want to connect to. We then add all those nodes in turn, repeating the process until no more nodes turn up. We then move on to the next diagram, starting with the first unused node, until there are no more unused nodes. At that point, we are done.
  partitionInto(c) {
    // Collect together all the node names in order, and create a map back to the actual nodes
    var unseen = [];
    var map = {};
    for (var i=0;i<this.nodes.length;i++) {
      var n = this.nodes[i];
      unseen.push(n.name);
      map[n.name] = n;
    }

Parsing the Diagram Description


The core of what we are trying to do here is to describe a diagram (or set of diagrams) in a declarative, textual way. As much as possible, we want that description to be abstract and close to the semantics of what is going on. But there must obviously be some means by which that is mapped onto a diagram consisting of boxes and lines.

That being the case, I'm going to start from a very generic notion from graph theory that we have a graph consisting of nodes and edges. Each of these items may have metadata of various forms attached to them, which will inform how we lay them out and render them. Some of these may be semantic and need further interpretation; others may be more direct.

It would be great if the layout were fully automatic and exactly right every time; however, such an algorithm has not yet been invented, so there are going to have to be commands in the description which enable us to describe the manner in which nodes and edges should be connected.

And this is the right point to add the disclaimer that I am not any kind of expert in the fields of graph theory or layout, and that I'm going to come up with a layout algorithm that sort of works. I'll be happy to work with anybody who can do better - I'm just interested in a "tool" that can produce decent-ish pictures from a completely textual description.

The description language

The basis of the description language is a set of lines with nesting. Each "top level" line either introduces a new node or a new edge; adds metadata to an existing node or edge; or adds a constraint between multiple existing items. Because the language is declarative, the order of the declarations is irrelevant. If needed, additional information is included in nested lines.

Let's start with a very simple example:
node producer
    label "Producer"
node consumer
    label "Consumer"
edge
    from producer
    to consumer
        cap arrow
So this introduces two nodes, a "producer" and a "consumer" (and gives them labels) and then defines an arrow leading from the producer to the consumer.

Analyzing the description

The way in which I'm approaching this problem is much the same as at the top level: I'm going to build a TDA pipeline in which the parser is mainly responsible for breaking the input up into lines, and then calling a tokenizer on each of them, and having that pass any "interesting" (non-blank, non-comment) lines on to the next step.
import { tokenize } from "./tokenize.js";
import Blocker from "./blocker.js";

function parser(top, errors) {
  return function(text) {
    var lines = text.split(/\n/);
    var blocker = new Blocker(top);
    for (var i=0;i<lines.length;i++) {
      tokenize(lines[i], blocker, errors);
    }
  }
}

export default parser;
The next step is to try and "group" these by indentation level. Any lines with no indentation go directly to the "top level parser", which creates a model object (attaching it to the top level model) and returns a parser which will handle any nested items. Lines with greater indentation are assessed according to a "stack": it is passed to the deepest parser with an indentation strictly less than the current line.

The parser which is called is responsible for understanding the meaning of the tokenized line in its context and updating the model it possesses, and then returning a new parser which is capable of handling any nested content. If it does not wish to support nested content, returning null or undefined will cause the blocker to automatically raise an error.
class Blocker {
  constructor(top, errors) {
    this.top = top;
    this.errors = errors;
    this.stack = [];
  }

  line(tok) {
    var current = this.top; // the default

    // find which parser to use, closing and shifting others more deeply indented
    while (this.stack.length > 0) {
      var first = this.stack[0];
      if (tok.indent > first.indent) {
        current = first.handler;
        break;
      }
      if (first.handler && first.handler.complete) {
        first.handler.complete();
      }
      this.stack.shift();
    }

    // call the current parser
    if (!current) {
      // if there is no handler specified at this level, it's because no further nesting is allowed
      this.errors.raise("no content allowed here");
    } else {
      var nested = current.line(tok);
    
      // record this parser for everything under this nesting level
      this.stack.unshift({indent: tok.indent, handler: nested});
    }
  }

  complete() { // on end of input
    this.top.complete();
  }
}

While we have been doing this, we have quietly introduced modules into the code, and started writing simple tests of the tokenizer and blocker. We have also introduced some rudimentary error handling.

The Top Level Parser

We're now ready to try building a top level parser. Three lines of input should be presented to it: the two node lines and the edge line. It should be able to identify these, reject anything else, build a model entry for the item and add it to the model, and then return a parser to handle any configuration items that they may have (a NodeConfigurationParser and an EdgeConfigurationParser respectively). How hard can this be?
import NodeConfigParser from "./nodeconfig.js";
import EdgeConfigParser from "./edgeconfig.js";
import Node from "./model/node.js";
import Edge from "./model/edge.js";

class TopLevelParser {
  constructor(model, errors) {
    this.model = model;
    this.errors = errors;
  }

  line(l) {
    var cmd = l.tokens[0];
    switch (cmd) {
      case "node": {
        switch(l.tokens.length) {
          case 1: {
            this.errors.raise("node command requires a name");
            break;
          }
          case 2: {
            var node = new Node(l.tokens[1]);
            this.model.add(node);
            return new NodeConfigParser(node);
          }
          default: {
            this.errors.raise("node: too many arguments");
            break;
          }
        }
        break;
      }
      case "edge": {
        switch(l.tokens.length) {
          case 1: {
            var edge = new Edge();
            this.model.add(edge);
            return new EdgeConfigParser(edge);
          }
          default: {
            this.errors.raise("edge: not allowed arguments");
            break;
          }
        }
        break;
      }
      default: {
        this.errors.raise("no command: " + cmd);
      }
    }
  }
}

Each of the nested parsers go much the same way: the node parser supports the label property and the edge parser supports the from and to properties. The latter, in turn, allows further configuration using the EdgeEndParser. We now have the ability to parse the input script outlined above.

Handling Errors

I've quietly added some (primitive) error handling into the code above, but it still just prints on the console. This doesn't seem the best idea, so I'm going to add an error tab and put the errors in there. We still haven't implemented any CSS, so we don't have tabs as such, but the idea will be that normally this error tab will be hidden, but that when there are errors all the diagram tabs will be hidden, the error tab will be visible and selected.

This follows much the same pattern as the happy path in the code. We add a block in the tabs for the errors to appear:
          <div class="error-tab">
            <div class="tab-title">Errors</div>
            <div class="error-messages">
            </div>
          </div>
In the pipeline, we check if we have seen any errors. If so, we recover the error-tab and pass it to the show method on the ErrorReporter object, otherwise we continue with the normal flow, making sure to clear any previous errors.
function pipeline(ev) {
  var errors = new ErrorReporter();
  var model = new DiagramModel(errors);
  readText("text-input", parser(new TopLevelParser(model, errors), errors));
  if (errors.hasErrors()) {
    applyToDiv("error-messages", tab => errors.show(tab));
  } else {
    applyToDiv("error-messages", tab => tab.innerHTML = '');
    var portfolio = new Portfolio();
    model.partitionInto(portfolio);
    applyToDiv("tabs-row", ensureTabs(portfolio));
    portfolio.each((graph, tab) => graph.layout(d => d.drawInto(tab)));
  }
}
And here is the ErrorReporter class, now pulled out into its own file:
class ErrorReporter {
  constructor() {
    this.errors = [];
  }

  hasErrors() {
    return this.errors.length > 0;
  }

  raise(s) {
    console.log("error: " + s);
    this.errors.push(s);
  }

  show(elt) {
    elt.innerHTML = '';
    for (var i=0;i<this.errors.length;i++) {
      var msgdiv = document.createElement("div");
      msgdiv.className = "error-message";
      var msgtext = document.createTextNode(this.errors[i]);
      msgdiv.appendChild(msgtext);
      elt.appendChild(msgdiv);
    }
  }
}

export default ErrorReporter;

Drawing Diagrams

Way back in the day, I used to draw diagrams with a tool called pic. This only worked with troff. Then I used fig when I was working with latex. Since the early '90s, I've been forced into hideously painful WYSIWYG diagram editors (Powerpoint is only the worst example; Visio, Framemaker, Illustrator and the like are all painful).

I was reminiscing earlier this week with a colleague who has similar values and a need to produce documentation on a large system he is building. I discussed pic and fig and how much easier it was back in the day, although at the same time noted the many issues these tools had. And how I would pay good money for a tool that made it easy again.

After we'd finished talking, I realized how what I'd described matched exactly the sort of problem I like to tackle … and furthermore the sort of problem I like to tackle here. So here goes.

Because in the interim the world has gone Web, my default output is going to be for the Web, so I'm going to build everything in JavaScript. Because I want a lot of flexibility, I'm going to go straight for a Canvas model and draw everything by hand.

So, as always(?), I want to start by building something that approaches "the whole application" so that we can see the overall picture and then fill in the details as they emerge. In this case, before we start getting fancy, I basically want to have a div with a couple of tabs: one with a text area to enter the description, and one with a canvas to show the "final" diagram (we will return to the notion of how many diagrams, later). So this requires a simple HTML file and a bunch of stubbed JavaScript in main.js. For those playing at home, this is checked in with the tag DIAGRAM_TOOL_FIRST_STEP.
<html>
  <head>
    <title>Diagrammer</title>
    <script type="text/javascript" src="js/main.js"></script>
  </head>
  <body onload="initialize()">
    <div>
      <div class="toolbar">
        <button class="toolbar-update">Update</button>
      </div>
      <div class="tabbed-window">
        <div class="tabs-row">
          <div class="input-tab">
            <div class="tab-title">Input</div>
            <div>
              <textarea class="text-input"></textarea>
            </div>
          </div>
          <div class="output-tab">
            <div class="tab-title">Output</div>
            <div>
              <canvas class="diagram"></canvas>
            </div>
          </div>
        </div>
      </div>
    </div>
  </body>
</html>
function initialize() {
  var updateButton = document.getElementsByClassName("toolbar-update")[0];
  updateButton.addEventListener("click", pipeline);
}

function pipeline(ev) {
  var model = new DiagramModel();
  readText("text-input", parser(model));
  var portfolio = new Portfolio();
  model.partitionInto(portfolio);
  tabModel("tabs-row", ensureTabs(portfolio));
  portfolio.each((graph, tab) => graph.layout(d => d.drawInto(tab)));
}

// TODO: everything below here needs to broken out into modules

// jstda.js
function readText(label, processor) {
  var input = document.getElementsByClassName(label)[0];
  processor(input.value);
}

function tabModel(label, processor) {
  var tabrow = document.getElementsByClassName(label)[0];
  processor(tabrow);
}

function ensureTabs(portfolio) {
  return function(tabrow) {
    portfolio.ensureTabs(tabrow);
  }
}

// parser.js
function parser() {
  return function(text) {
    console.log(text);
  }
}

// model.js
class DiagramModel {
  partitionInto(c) {
    console.log("partition model into", c);
  }
}

// portfolio.js
class Portfolio {
  ensureTabs(tabrow) {
    console.log("need to ensure tabs");
  }

  each(f) {
    console.log("iterate over graphs and provide tabs");
  }
}
As always, the output from that is unpleasant on the eye and really needs some CSS to make it palatable. I'll come back to that, but for now, let's consider what that pipeline means.

First off, as regular readers know, I'm a TDA (tell-don't-ask) enthusiast. Mainly this stems from doing TDD with complex systems in Java; the mock support is not as good in JavaScript, but my brain remains wired up the same way - if it can't be functional, let it be TDA. And you never know, I may get around to actually writing some tests this time!

Consequently, everything is coupled up in pipeline and that's the only function I'm going to talk about. Almost everything else is plumbing that is just there so that the pipeline doesn't throw any errors.

There are four "phases" to the pipeline:
  • We convert the input text into a model of the graph in memory;
  • We split the graph into connected chunks (if the graph is completely connected, there will only be one such chunk) and build a Diagram for each chunk which are collected in a Portfolio;
  • We make sure that there is a tab and a canvas for each Diagram;
  • We go through each Diagram in the portfolio, first laying it out and then rendering it into the canvas.
What could be simpler?