Wednesday, July 19, 2023

Handling paper sizes

It's now time to turn our attention to the other really important part of this process: actually printing something. I have lived in both the UK and the US, and so I've had to deal with the whole A4/Letter thing on a personal basis. And, from time to time, I've also wanted to print calendars on A3 and tabloid paper (mainly to fit more on). So I definitely want to be able to handle this, because it's a terrible waste of paper and space if you have to assume you have paper 11" x 210mm (mismatched units intentional).

So, how do we tackle this? Well, I think the theory is that you use @media queries to handle it. Although most of the standards bodies seem more concerned in your ability to specify what page size you (as a document designer) wants than handling the page size the user has chosen to print on. (See the @page size descriptor for more details, including a (to me) humorous comparison between A4 and letter paper).

So there doesn't seem to be a nice simple
@media  print  and(paper-size:a4)  {  …  }
query, but rather we need to use something like:
@media  print  and  (min-width:  208mm)  and  (max-width:  212mm)  {  …  }
which is incredibly unclear and quite hacky. The idea here is that you can examine the width of the paper and if it is in the suitable range you can choose to use the settings for (in this case) A4.

Except, as far as I can tell, in Chrome at least, it doesn't work. I created a separate test case in pagesize.html and pagesize.css to test out these things, and I found I needed to test whether the paper size was 519mm, at which point it seemed to work the same for all different paper sizes.

Googling suggested that this is in fact a widespread problem, and thus I don't want to go near it (even though I do). But not supporting different paper sizes is not an option, so we are going to have to ask the user what their paper size is and then generate the relevant values based on that. I know for myself I will get this wrong more than once. Oh, and we also have to ask if they want calendars printed portrait or landscape (in my experience, for calendars of less than about four weeks, landscape is preferable; over four weeks generally demands a portrait layout, but it probably depends on what your application is). Sadly, this also limits the range of paper sizes we can offer (although we could offer a "custom" option; pull requests are always welcome).

In conclusion then, we are just going to create a second set of stylesheets which simply use @media print, and then use our own dropdown to determine the appropriate values. All of this work is tagged PLANNING_CALENDAR_PAPER_SIZES.
      <label for="page-size">Page Size:</label>
      <select id="page-size" onchange="redraw()">
        <option value="a4">A4</option>
        <option value="letter">Letter</option>
        <option value="a3">A3</option>
        <option value="tabloid">Tabloid</option>
      </select>
      <input type="checkbox" id="landscape" onchange="redraw()">
      <label for="landscape">Landscape</label>

CSS from JavaScript

Right, now we come to the meat of this whole project. In the basic CSS presented in the last episode, I opted for fixed cell measurements and completely flexible page measurements. This is exactly the opposite of what we actually want: the paper size for the calendar is fixed (even if we offer multiple options) and we want the cell size to vary to fill the available space.

Rather than deal with the complexities of the printer right up front (I hope I don't regret this decision…), I'm going to spend this episode looking at doing that on the screen in the available space (where it is easy to debug everything) and then concern myself with the complexities of printing next time.

Before we go any further, yes, I am aware of calc in CSS, and use it frequently. However, I have convinced myself that what I want to do here goes beyond calc because of the multiple constraints that apply to any given element. If you can figure out how to do all of this just with calc, do let me know.

Contrariwise, I believe it is possible to do everything I want to do here with JavaScript and hacking at elements in the DOM. But that is painful and error-prone and, quite frankly, not as exciting as what I'm going to try and do here, which is to create a new stylesheet at run-time whose values are calculated dynamically. I'm then going to split the styling between true "styling" (which will stay in the static stylesheets) and the measuring (which will be calculated dynamically).

I've chosen to put this code in a separate JS file, mainly for clarity, but not to do all the work that I should do to make everything modular (also for clarity). So I've defined a separate initStyling method, and called that from the init method in controls.js, passing through the fbdiv which I'm then storing as styledDiv. The more important thing, though, is that I create a new CSSStyleSheet and assign it to the list of adoptedStyleSheets in the document. Note that this only "works" if you are in complete control of the set of sheets to be adopted; if you're not, you will need to do the work to merge your sheets with everyone else's.
    <link rel="stylesheet" href="css/controls.css">
    <link rel="stylesheet" href="css/feedback.css">
    <script src="js/controls.js"></script>
    <script src="js/styling.js"></script>
  </head>
  <body onload="init()">
    <div class="controls">
  start.valueAsDate = new Date();
  end.valueAsDate = new Date();

  initStyling(fbdiv);

  redraw();
var styledDiv;
var sheet;

function initStyling(fbdiv) {
  styledDiv = fbdiv;
  sheet = new CSSStyleSheet();
  document.adoptedStyleSheets = [sheet];
}
Setting the rules depends on figuring out the size of the area to be filled and then updating the rules. This is information which should be available in redraw, so we'll call the function to set the dimension rules from there. At the end of redraw, I've added a call to fitToPageSize, which is a method to be defined in styling.js.
  fitToPageSize();
}
function fitToPageSize() {
  while (sheet.cssRules.length > 0)
    sheet.deleteRule(0);
  sheet.insertRule(".body-day { width: 25mm; height: 25mm; margin: 2mm }");
  sheet.insertRule(".body-day-date { top: 3mm; left: 3mm; font-size: 5mm }");
}
Before we go any further, I'm going to remove the sizing properties from the static CSS, so that there is no conflict and it is clear whether the dynamic stylesheet is working. And my first pass in getting the dynamic sheet working is just to try and add the same (fixed) properties to it. If nothing changes, we have a working dynamic spreadsheet.

And thus we have what we wanted: a dynamic stylesheet that can be continually updated. We are just left with the task of calculating the correct values.

This is tagged PLANNING_CALENDAR_DYNAMIC_STYLESHEET.

This next bit is something I never like doing. I think the root of the problem is just that it completely violates everything that is innate to HTML which is that you have an infinite scroll of paper, and thus there is no such notion as the "height" of the page. So you have to take the window size, and then go around subtracting all sorts of random measurements (body margin, other divs, etc) until it seems to fit. And then you try it on a mobile device only to find …

So bear with me if this doesn't "quite" work for you, or if the numbers (such as 16, double the "default" body margin of 8) seem to have been plucked out of nowhere. Imagine, if you will, that this code is in fact just going to return the available size for the feedback div. To help you, I will bury it in a function with that name and put all the nastiness in that, and then delegate the layout work to another function which takes the result.

Now, having done all that, I have a size in pixels. Up until now, I've been using mm, but there's no real difference between them for what follows, as long as you are consistent about it.

Let's tackle the horizontal first. We know that we have exactly 7 days in a week, and we also have 8 "spaces" (one on each end and six in between). Because this is CSS, each of the spaces consists of one or two margins, so actually we have 7 left margins and 7 right margins. So, if we say these will be equal, we can consider that we have 14 margins. If we say that we want a ratio of 12:1 for the days to the spaces, then we have 84 "units" in days and 7 units in spaces, for a total of 91 units. Each unit is then the width divided by 91. And then I'll round down to make sure it fits. We then need to set the width of a "day" to 12 units and the margin to be 0.5 units.
var styledDiv;
var sheet;
var controlPane;

function initStyling(fbdiv) {
  styledDiv = fbdiv;
  controlPane = document.getElementById('controls');
  sheet = new CSSStyleSheet();
  document.adoptedStyleSheets = [sheet];
}

function fitToPageSize() {
  // delte the old rules
  while (sheet.cssRules.length > 0)
    sheet.deleteRule(0);

  // calculate the page size
  var pageSize = calculateSizeOfFeedbackDiv();
  var unitIn = "px";

  // calculate desired box sizes
  var xunit = Math.floor(pageSize.x / 91);
  var xday = xunit*12;
  var xmargin = xunit/2;



  // generate new rules
  sheet.insertRule(".feedback { border-width: 0.1mm; width: " + pageSize.x + "px; height: " + pageSize.y + "px; }");
  sheet.insertRule(".body-day { border-width: 0.1mm; width: " + xday + unitIn + "; height: 25mm; margin: " + xmargin + unitIn + " }");
  sheet.insertRule(".body-day-date { top: 3mm; left: 3mm; font-size: 5mm }");
}

function calculateSizeOfFeedbackDiv() {
  var viewx = window.innerWidth;
  var viewy = window.innerHeight;

  var fbx = viewx - 16 - 5; // 16 for double body margin // allow, say, 5 for border
  var fby = viewy - controlPane.clientHeight - 16 - 5;

  return { x : fbx, y : fby };
}
When I do this, I find that the line "wraps": it's longer than I have calculated. Inspecting the DOM in Chrome, I discover there is extra space between the elements. Ah, yes, I remember adding that now when we were all text based. So I removed it from the DOM.

This is tagged PLANNING_CALENDAR_X_CORRECT.

Now, for the vertical dimension, we need to figure out how many rows there are. In fact, we have already done this when drawing the rows, which is where this function is called from, so we'll get it to count them for us and hand the number over.
  if (leftDate > from) {
    leftDate.setDate(leftDate.getDate() - 7);
  }
  var numRows = 0;
  do {
    console.log(" showing week with", leftDate, "in the left column");


    // advance to next week
    leftDate.setDate(leftDate.getDate() + 7);
    numRows++;
  } while (leftDate <= to);

  fitToPageSize(numRows);
}
Then the calculation is basically the same: we have a given number of rows, with a margin above and below each one, and we want the ratio to be about 12:1. So we divide the page size by the number of rows multiplied by 13, and use that as the y unit size. Oh, and don't forget that CSS measurements for things like margins use the "compass points" notation so that y comes before x.

Finally, we want to calculate the position and size of the date number. This wants to be about a fifth of the way into the box and about an eighth of its size (a completely arbitrary and aesthetic judgment on my part, you understand). More importantly, the size wants to be the smaller of what x and y will allow (the positions are independent). However, there are a couple more constraints: it would be good if the number were legible, so I would like it to always be at least 8px, but it can't be more than the available space in the box, which implies a font size of about three quarters the height of the box.
function fitToPageSize(rows) {
  var xpos = xday / 5;
  var xsize = xday / 8;

  var yunit = Math.floor(pageSize.y / (rows * 13));
  var yweek = yunit*12;
  var ymargin = yunit/2;
  var ypos = yweek / 5;
  var ysize = Math.min(Math.max(yweek / 8, 8), yweek * 3/4);
  
  var dateSize = Math.min(xsize, ysize);

  // generate new rules
  sheet.insertRule(".feedback { border-width: 0.1mm; width: " + pageSize.x + "px; height: " + pageSize.y + "px; }");
  sheet.insertRule(".body-day { border-width: 0.1mm; width: " + xday + unitIn + "; height: " + yweek + unitIn + "; margin: " + ymargin + unitIn + " " + xmargin + unitIn + " }");
  sheet.insertRule(".body-day-date { top: " + ypos + unitIn + "; left: " + xpos + unitIn + "; font-size: " + dateSize + unitIn + " }");
}
So far, so good. Checked in and tagged as PLANNING_CALENDAR_XY_CORRECT.

Now, what's bugging me is that the rounding that I did to make sure that the calendar doesn't overflow is working perfectly in the horizontal direction, but is slightly too aggressive in the vertical dimension. Simply removing it causes the box to overflow, as I'd expect, as does rounding to a single decimal place. I hate figuring these things out, but I guess I have to.

The problem, "of course", is that I have been ignoring the effect of the border widths in these calculations and just thinking that it will cancel out. Which it kind of does, but it's too ugly.

So the only solution is actually to take that into account, calculate the correct offsets given the border width, and remove the Math.floor. Then it will all work correctly.

This is too dull and testy to show here, but you can see it in the repository; it's checked in and tagged as PLANNING_CALENDAR_BORDER_INCLUDED.

One last thing: we also want to redraw the calendar when the window size changes (e.g. going full screen or removing the developer tools). To handle this, we need to add a handler for the resize event. This is tagged PLANNING_CALENDAR_HANDLE_RESIZE.

  initStyling(fbdiv);

  addEventListener("resize", redraw);
  redraw();
}

Tuesday, July 18, 2023

Basic CSS

How much styling we want to add is always a difficult question. For me, it tends to be somewhat "creeping" - I'll start with "just enough" and then keep adding a bit at a time while I'm working on other features and using the code. On here, though, I tend to keep it minimal, especially since I'm trying to make it clear what is important and what isn't. So while I might usually do things such as styling the date pickers, all I'm going to do here is make it clear what's included in the "feedback" box and then style it to look like a calendar.

So, in this case, I add a border around the feedback area to make it clear what the "page" looks like, then a border around each "date" box, and then style the number in the date box. The only oddity (IMHO) is styling body-day as position: relative which has nothing to do with what it wants, but just to make it the frame of reference for the position: absolute in body-day-date.

Simples. You may also notice that I added a css class onto the feedback box, changed the id to a class for the controls box, and fixed a bug in where the date value was being added in controls.js.
.feedback {
  border: 0.1mm solid black;
}

.body-day {
  display: inline-block;
  border: 0.1mm solid black;
  margin: 2mm;
  width: 25mm;
  height: 25mm;
  position: relative;
}

.body-day-date {
  position: absolute;
  top: 3mm;
  left: 3mm;
  font-family: 'Courier New', Courier, monospace;
  font-weight: bold;
  font-size: 5mm;
}
This code is in PLANNING_CALENDAR_BASIC_CSS.

Wiring up the Basics

Now that we have the outline of the calendar, we want to actually generate the calendar that has been asked for. As you may have guessed, 90% of this blog is going to be about how to get the calendar to look the way you want; the actual generation is a few simple lines of JavaScript. Certainly I'm doing this more than anything else (well, I guess I'm actually interested in the end product, but as I already have a functioning desktop app…) to investigate building CSS stylesheets dynamically in JavaScript.

That being the case, I'm going to gloss over most of this with a couple of classic handwaves. As always, if you feel I've left out significant details, get back to me and I'll elaborate.

The outline has three ui elements, each of which controls what is seen in the feedback window, and when any of them change, we want to generate an appropriate calendar based on the current set of three values. So, we add a listener on the appropriate change event for each control, all of which invoke the same method to redraw the calendar.

While I'm about setting that up, I have also added an init method which initializes the default start and end dates to be today and, for good measure, then generates a "default" feedback pane to be this week, starting on Monday. It's not a lot, but it gets you started.
    <script src="js/controls.js"></script>
  </head>
  <body onload="init()">
    <div id="controls">
      <label for="start-date">Start date:</label>
      <input type="date" id="start-date" onchange="redraw()">
      <label for="end-date">End date:</label>
      <input type="date" id="end-date" onchange="redraw()">
      <label for="first-day">First Day:</label>
      <select id="first-day" onchange="redraw()">
        <option value="1">Monday</option>
        <option value="2">Tuesday</option>
        <option value="3">Wednesday</option>
        <option value="4">Thursday</option>
        <option value="5">Friday</option>
        <option value="6">Saturday</option>
        <option value="0">Sunday</option>
      </select>
    </div>
What we want to draw is basically the same as what I put in the outline: a nested structure consisting of the whole calendar containing weeks, containing days, containing information about that day. redraw needs to clear out the existing feedback calendar and draw a new one starting the week of the start date from the first day until the end date is in the past.

A couple of notes on this code which is, otherwise, fairly straightforward:
  • The calculation of the offset for the start of the week works most of the time, but when the start date is a day of the week before the "left column day", the first week might be missed, and it is important to check for this and start the calendar a week earlier.
  • It's important to copy the value of from into leftDate (and leftDate into cellDate), otherwise, because dates in JavaScript are references, both leftDate and from will be updated, making this check pointless.
  • In order to make this work correctly, I changed the values of the options in the select statement from what I had initially used.
  • I have used a "do" loop to make sure at least one week comes out, but after that it is just a question of deciding whether the end date is in the future or the past. This automatically handles the case where the end date is before the start date in a satisfactory way.
  • There are comments logged to the console to clarify what is happening.
Obviously, we have not provided any styling yet. Because of this, by default, all the dates "run together" on a line to make a mess. This did not happen in the outline because of the white space inherent in the layout of the document. To replicate this, I have added a white space character between successive date boxes which will be removed when we add the borders.
var start, end, first, fbdiv;

function init() {
  start = document.getElementById('start-date');
  end = document.getElementById('end-date');
  first = document.getElementById('first-day');
  fbdiv = document.getElementById('feedback');

  start.valueAsDate = new Date();
  end.valueAsDate = new Date();

  redraw();
}

function redraw() {
  var from = new Date(start.value);
  var to = new Date(end.value);
  var leftColumn = parseInt(first.value);
  console.log("Generating calendar from", from, "to", to, "based on", leftColumn);

  fbdiv.innerHTML = '';
  var leftDate = new Date(from);
  leftDate.setDate(leftDate.getDate() - leftDate.getDay() + leftColumn);
  if (leftDate > from) {
    leftDate.setDate(leftDate.getDate() - 7);
  }
  do {
    console.log(" showing week with", leftDate, "in the left column");

    // create a div for the whole week
    var week = document.createElement("div");
    week.className = "body-week";
    fbdiv.appendChild(week);
    for (var i=0;i<7;i++) {
      var cellDate = new Date(leftDate);
      cellDate.setDate(cellDate.getDate() + i);
      console.log(" cell", i, "has date", cellDate);

      // create a div for each day, to contain all the aspects we will have
      var day = document.createElement("div");
      day.className = "body-day";
      week.appendChild(day);

      // the first aspect is the date
      var date = document.createElement("div");
      date.className = "body-day-date";
      day.appendChild(date);

      // and set the date text in here
      var dateValue = document.createTextNode(cellDate.getDate());
      day.appendChild(dateValue);

      // add a space between days until we style this
      var space = document.createTextNode(" ");
      week.appendChild(space);
    }

    // advance to next week
    leftDate.setDate(leftDate.getDate() + 7);
  } while (leftDate <= to);
}
The code is under the tag PLANNING_CALENDAR_BASIC_WIRING.

Monday, July 17, 2023

Planning Calendars

I am, by nature, a planner.

Many who meet me through the agile community are surprised by this, but, in fact, the nature of my planning fits well with agile: it is small scale, and can be repeated endlessly as new information emerges.

As such, I am always looking for tools to help me achieve this or communicate this. And because the things I do are generally small scale and "cheap", the tools I am looking for are generally not high priced enterprise things.

One such tool is a calendar. As I always like to say when planning projects, "you can always get your money back; but once the time is gone, it's gone". In my experience, most people don't have a good grasp of time in general, and, in particular, that only one thing fits in a given spot.

The origins of this tool go back to a "Filofax replacement" project I was building on NeWS (if you're under 50, Google is your friend here) back in the late 80s. Not much came of that, but the PostScript header file is the basis of the subsequent calendar programs I've written over the past 30 years. The plan here is to replace all of those with one that doesn't use that PostScript header file because it is 100% web based (HTML and CSS with the appropriate print media definitions).

The key to this tool is threefold:
  • Most projects I have worked on do not run on a nice, comfortable "exactly one calendar month" basis for which a standard calendar is a good fit;
  • Most people I know work on a Monday-Friday schedule, and then enjoy a weekend, whereas most calendars I can buy seem to think the week runs from Sunday to Saturday;
  • The month breaks on standard calendars are extremely distracting and often make people count the same week twice.
So, what I want is a calendar whose first week is the first week of the project, whose last week is the last week of the project and exactly fills one sheet of A4 (or "letter paper" if you happen to live in the dark ages). Sounds simple? OK, well, in that case, for good measure, I'd like to be able to choose which day of the week it starts on (although the default should be Monday) and whether you arrange it in landscape or portrait orientation (obviously, this is just on the print menu, but I want the calendar to re-layout when I do). And obviously, I want a lot of controls and visual feedback on the screen, but just the calendar on the output.

Oh, and over the past 30 years I have acquired a few new requirements that would be good to include:
  • The option to display the calendar in different "styles", in particular, how and where the month/year names are displayed, colours and shading for dates, weeks and months, and so on;
  • Sometimes I do want "long range plans": not plans as such, but if I have certain fixed engagements over the next few months, I'd like to have a calendar that has those on it so that I can see what chunks of work I might be able to commit to in that span. Experience has taught me that 13 weeks (i.e. a quarter of a year) on a calendar works, but that 26 (half a year) does not. So I'd like to be able to have multiple pages with (approximately) the same number of weeks on each.
  • There are a lot of calendars out there in ICS form that I'd like to include so that I can have them show up in order to avoid conflicts and to know when holidays are; and I'd like to be able to configure how they show up.
All good? Let's get started.

The basics

I believe in keeping things simple. So what I basically want is a single page that has a control area and a feedback area. The control area allows you to select a start date (which will be in the first row of the calendar) and an end date (which should be later than the start date, in which case it will be in the end row; otherwise you will just get a one-week calendar with the start date) and which day of the week is going to appear in the left-hand column.

The feedback area shows you approximately how the calendar will display, but without laying it out "on paper", or applying any fancy styling.

And then the "print" button does all the hard work through the application of a print media stylesheet.

Let's get started. We won't get everything we want done today, but let's at least get that outline in place.

The code for this can be found tagged PLANNING_CALENDAR_OUTLINE in the directory calendar. We're just getting started here, so nothing really works as such, but hopefully it's a useful guide to you as to where we're going with this - I know it is to me.

<html>
  <head>
    <title>Planning Calendar</title>
    <link rel="stylesheet" href="css/controls.css">
    <link rel="stylesheet" href="css/feedback.css">
  </head>
  <body>
    <div id="controls">
      <label for="start-date">Start date:</label>
      <input type="date" id="start-date">
      <label for="end-date">End date:</label>
      <input type="date" id="end-date">
      <label for="first-day">First Day:</label>
      <select id="first-day">
        <option value="0">Monday</option>
        <option value="1">Tuesday</option>
        <option value="2">Wednesday</option>
        <option value="3">Thursday</option>
        <option value="4">Friday</option>
        <option value="5">Saturday</option>
        <option value="6">Sunday</option>
      </select>
    </div>
    <div id="feedback">
      <div class="body-week">
        <div class="body-day">
          <div class="body-day-date">17</div>
        </div>
        <div class="body-day">
          <div class="body-day-date">18</div>
        </div>
        <div class="body-day">
          <div class="body-day-date">19</div>
        </div>
        <div class="body-day">
          <div class="body-day-date">20</div>
        </div>
        <div class="body-day">
          <div class="body-day-date">21</div>
        </div>
        <div class="body-day">
          <div class="body-day-date">22</div>
        </div>
        <div class="body-day">
          <div class="body-day-date">23</div>
        </div>
      </div>
    </div>
  </body>
</html>
At the moment, this website is 100% static, so you can just open index.html in your browser, and everything will just work fine. This is going to be true for a while, until we start opening ICS calendars using AJAX. Then we'll need to have it running on/in a server. So that I'm prepared, I just run this command in the web directory:
python  -m  SimpleHTTPServer  8080
And then visit localhost:8080 in my browser. I know there are 2,000 ways of doing this, so feel free to use your own. I have just got into this habit, and it keeps working for me.

Wednesday, April 26, 2023

Populating a Side View with a Notification

Following on from my previous post, it has come to my attention that it is also possible to populate the view by using a notification from the back end to the front end.

The purpose of notifications is to provide updates without being asked. In our case, it makes sense when we have updated the repository (after parsing one or more files) to send such a notification. Otherwise, the front end needs to "guess" when the information may have changed and request it again.

So the task here is to try and turn the process around, and instead of sending a request for the information we requested, for the back end to automatically send it when it is ready.

Reading through the specification, it seems that this is possible, but that it requires us to "extend" the protocol. But I'm not entirely sure how to do that. It turns out that while it's possible, it doesn't seem to me that it's "genuinely" supported at all.

On the server side, we need to define a new interface which extends LanguageClient and specify the new notification methods that we want, tagged with the @JsonNotification attribute, which specifies the command string that will be used in sending the message.
package ignorance;

import org.eclipse.lsp4j.jsonrpc.services.JsonNotification;
import org.eclipse.lsp4j.services.LanguageClient;

public interface ExtendedLanguageClient extends LanguageClient {
  @JsonNotification("ignorance/tokens")
  void sendTokens(Object object);

}
It should then just be a case of telling the LSPLauncher that this is the protocol we want to use, but none of the factory methods takes an interface - they all assume that you want to use LanguageClient. But fortunately, there is nothing magic about any of these methods, so we can extract it locally and make the change we need.
    Launcher<ExtendedLanguageClient> launcher = createServerLauncher(server, in, out);
  private static Launcher<ExtendedLanguageClient> createServerLauncher(IgnorantLanguageServer server, InputStream in, OutputStream out) {
    return new LSPLauncher.Builder<ExtendedLanguageClient>()
      .setLocalService(server)
      .setRemoteInterface(ExtendedLanguageClient.class)
      .setInput(in)
      .setOutput(out)
      .create();
  }
Likewise, on the client side, it should be just as simple as calling the onNotification method of the client, but this can only be done once the client is ready, and thus after communication has already started up - meaning that some messages might be sent (and lost) before the handler is installed. I tried a number of ways of working around this (there appears to be a notion of Features which can install handlers) but nothing fixed the fundamental problem, so I ended up having to add my own synchronization around this. But for now, here's the code that sets up the handler in the onReady method.
  client.onReady().then(() => {
    client.onNotification("ignorance/tokens", () => {
      console.log("received token notification");
    });
So to handle the synchronization, we add another command with the message ignorance/readyForTokens. In the onReady method we call this AFTER configuring the notification handler.
    client.onNotification("ignorance/tokens", () => {
      console.log("received token notification");
    });
    client.sendRequest(ExecuteCommandRequest.type, {
      command: 'ignorance/readyForTokens',
      arguments: [ ]
    });
      
    const tokensProvider = new TokensProvider();
On the server side, we can then start sending messages when appropriate.
      public CompletableFuture<Object> executeCommand(ExecuteCommandParams params) {
        switch (params.getCommand()) {
        case "java.lsp.requestTokens": {
          System.err.println("requestTokens command called for " + params.getArguments().get(0));
          return repo.allTokensFor(URI.create(((JsonPrimitive) params.getArguments().get(0)).getAsString()));
        }
        case "ignorance/readyForTokens": {
          System.err.println("readyForTokens");
          amReady = true;
          return CompletableFuture.completedFuture(null);
        }
        default: {
          System.err.println("cannot handle command " + params.getCommand());
          return CompletableFuture.completedFuture(null);
        }
        }
      }

Which just leaves the process of actually sending a message, which we want to do once we have finished parsing. There is a certain amount of off-camera wiring to make this happen reliably.
  private void parseAllFiles() {
    File file = new File(workspaceRoot.getPath());
    parseDir(file);
    client.sendTokens("hello, world");
  }
All the complex wiring is once again left as an exercise for the reader (see VSCODE_NOTIFICATIONS_COMPLEX_WIRING).

Conclusion

It is certainly possible to configure both client and server to handle custom notifications, but there are enough tricky issues involved that I don't think that it can honestly be described as "supported". But if you're an event-driven freak like me, you just have to bite it off.

Tuesday, April 25, 2023

Adding a side view to LSP



It seems like a while since I last worked on my LSP server (*checks git history*: it's a very long while, OK).

But right now I find myself wanting to add a new view with some meta data to my LSP server in "the real world". And so I come back here to figure out how to support that.

I was originally thinking I could add a window at the bottom (akin to Problems, or Output) but it doesn't seem possible to add something there. I considered adding a virtual document but then I realized that was a lot of hard work for not much gain and what I think I probably want is a view.

So here goes …

The basic setup is that we want to add a view which is capable of presenting some hierarchical information as a tree. The thought would be that the server has some way of deducing "global" information about a project which it can then feed back to you (a class explorer, for example).

Looking back at what I did before, I can see that for each project in the workspace, I collected together a list of tokens (called Names) which have a file (in the form of a URI), and a location. For the purposes of testing what I want to, I am going to use this a source of information and build a tree which has a list of Names, and then for each Name, it will return a list of URIs, and within each URI a list of locations. If I understand what I did previously, each name will only have one URI and location, but that is not all that interesting: it's the fact that it's presented as a list which I care about.

But before we get to that, we need to start on the front end, and add a new contribution to our existing package.json.
    "views": {
      "explorer": [
        {
          "id": "ignorantTokens",
          "name": "Ignorant Tokens"
        }
      ]
    }
And run the extension using F5. Well, yes, that was easy. The "Ignorant Tokens" view appears at the bottom of the left hand column. Click on it and it expands to show … nothing.

So now we need to put some content there. For now, we're not going to do the hard work of getting the content from the server, we're just going to hack something in, say a list of items with lists in them. We'll use multiple items here to make sure our code works, even if that isn't something the Java backend is going to offer.

Start small, Gareth, start small. So let's just start with a list of the currently active folders in the workspace.

The key to providing the content is to call the window.registerTreeDataProvider method and to provide it with an object which can provide all the data that you need. That, in itself, is not that hard.
  const tokensProvider = new TokensProvider();
  window.registerTreeDataProvider('ignorantTokens', tokensProvider);
Now we can move on to actually implementing the provider, along with the data model representing the tree. The provider has to be an instance of the interface vscode.TreeDataProvider, and this has basically two methods: one to get the children of an item and one to get the "tree item data" itself. The Microsoft example I'm working from assumed it was, in fact, possible for the item to be an instance of the TreeItem class, and I have followed suit. This is my implementation of ProjectTokens, which basically just says that it's not hard to represent a workspace folder as a tree item by providing the super constructor with the folder name and saying that it should initially be in the "collapsed" state (click to show children).
import { TreeItem, TreeItemCollapsibleState, WorkspaceFolder } from "vscode";

export class ProjectTokens extends TreeItem {
  constructor(wf : WorkspaceFolder) {
    super(wf.name, TreeItemCollapsibleState.Collapsed);
  }
}
The provider has a constructor which looks at all the folders in the workspace and creates an item for them. It's not actually clear to me whether it is better to put this code in the constructor, or to put it in the code which returns the top-level list of items. I have put it in the constructor here to provide for a clearer separation of responsibilities, but it is not clear to me how well this will adapt to changes.

On the subject of which, I think this is the case which is supposed to be handled by the Event onDidChangeTreeData, but I have to confess that I am ignorant about this and I could not (at this juncture) be bothered to go and check it out, nor am I sure who would do the checking to generate this event (a later juncture occurred while tidying up at the end; see below for how this works and is used).

The first method that VSCode calls when trying to generate the tree view is, oddly, getChildren. But it does so passing in undefined or null or something which gives you the clue that it doesn't know whose children it wants: and the children of nobody must be the top level items, which we have to hand and can return.

Then for each of these items that you pass back, it turns around and asks you to hand it a TreeItem. As noted above, we chose to implement our ProjectTokens as TreeItems, so we can just return the very thing we are given. In other situations, you could return a member of the object, or presumably create one on the fly.
import { Event, ProviderResult, TreeDataProvider, TreeItem } from "vscode";
import { ProjectTokens } from "./tokenlocation";
import * as vscode from 'vscode';

export class TokensProvider implements TreeDataProvider<ProjectTokens> {
  locations: ProjectTokens[];
  constructor() {
    this.locations = [];
    if (vscode.workspace.workspaceFolders == null)
      return;
    for (var wf=0;wf<vscode.workspace.workspaceFolders.length;wf++) {
      this.locations.push(new ProjectTokens(vscode.workspace.workspaceFolders[wf]));
    }
  }

  onDidChangeTreeData?: Event<ProjectTokens | null | undefined> | undefined;
  getChildren(element?: ProjectTokens | undefined): ProviderResult<ProjectTokens[]> {
    if (!element) { // it wants the top list
      if (!this.locations) {
        return Promise.resolve([]);
      } else {
        return Promise.resolve(this.locations);
      }
    }
  }
  getTreeItem(element: ProjectTokens): TreeItem {
    return element;
  }
  getParent?(element: ProjectTokens): ProviderResult<ProjectTokens> {
    throw new Error("Method not implemented.");
  }
}
To support multiple levels of nesting, we now need to provide two more data classes (implementing TreeItem): Token and TokenLocation. The idea here is that Token gives you the name of the token that has been defined and TokenLocation gives you the location where it can be found.
import { TreeItem, TreeItemCollapsibleState, WorkspaceFolder } from "vscode";

export class ProjectTokens extends TreeItem {
  tokens: Token[];
  constructor(wf : WorkspaceFolder, tokens : Token[]) {
    super(wf.name, TreeItemCollapsibleState.Collapsed);
    this.tokens = tokens;
  }

  children() : Promise<Token[]> {
    return Promise.resolve(this.tokens);
  }
}

export class Token extends TreeItem {
  locations: TokenLocation[];
  constructor(name: string, locations: TokenLocation[]) {
    super(name, TreeItemCollapsibleState.Collapsed);
    this.locations = locations;
  }

  children() : Promise<TokenLocation[]> {
    return Promise.resolve(this.locations);
  }
};

export class TokenLocation extends TreeItem {
  constructor(where: string) {
    super(where, TreeItemCollapsibleState.None);
  }

  children() : Promise<Token[]> {
    return Promise.resolve([]);
  }
};
In the provider, we need to make three sets of changes. First, we need to change the signatures of the provider and all the methods to enable all three TreeItem types to be returned. The second, and most important, change is that in the getChildren method, when an element is provided, we must ask that element for its children. And finally, because we are hacking things together, we must update the constructor to provide the appropriate tokens.
import { Event, ProviderResult, TreeDataProvider, TreeItem } from "vscode";
import { ProjectTokens, Token, TokenLocation } from "./tokenlocation";
import * as vscode from 'vscode';

export class TokensProvider implements TreeDataProvider<ProjectTokens | Token | TokenLocation> {
  locations: ProjectTokens[];
  constructor() {
    this.locations = [];
    if (vscode.workspace.workspaceFolders == null)
      return;
    const tmp = [
      new Token("List", [
        new TokenLocation("46.2"),
        new TokenLocation("53.1")
      ]),
      new Token("Map", [
        new TokenLocation("15.7"),
        new TokenLocation("28.9")
      ])
    ];
    for (var wf=0;wf<vscode.workspace.workspaceFolders.length;wf++) {
      this.locations.push(new ProjectTokens(vscode.workspace.workspaceFolders[wf], tmp));
    }
  }

  onDidChangeTreeData?: Event<ProjectTokens | Token | TokenLocation | null | undefined> | undefined;
  getChildren(element?: ProjectTokens | Token | TokenLocation | undefined): ProviderResult<ProjectTokens[] | Token[] | TokenLocation[]> {
    if (!element) { // it wants the top list
      if (!this.locations) {
        return Promise.resolve([]);
      } else {
        return Promise.resolve(this.locations);
      }
    } else {
      return element.children();
    }
  }
  getTreeItem(element: ProjectTokens | Token | TokenLocation): TreeItem {
    return element;
  }
  getParent?(element: ProjectTokens | Token | TokenLocation): ProviderResult<ProjectTokens | Token | TokenLocation> {
    throw new Error("Method not implemented.");
  }
}

The back end

Now it's time to turn our attention to the back end and how we can generate this data on the server side and pass it across. You may recall (or, if you're more like me, may not - I had to go and look at the code) that the communication between client and server is handled by a language client (called client in extension.ts).

Now, the key to doing this is to be able to send a command across from the client to the server when we would otherwise generate the data ourselves - i.e. in the constructor of the data provider. But what message should we send? The documentation doesn't seem wildly clear on this, but it would seem that we can repurpose executeCommand for this purpose. The documentation states that "usually" this will return an applyEdit message, but it seems entirely possible that we could return something else - maybe "hello, world"? Let's give it a try, shall we?

First things first: we need to pass the client to the token provider, and that in turn means that we need to move the declaration down to the bottom. There are a couple of other caveats here: because we are going to call a method in client, we need to wait to make sure that it has initialized, so we attach this code to its onReady method. And because this call is asynchronous, we will want to call await, so it cannot be located in the constructor, so we need to move it out into a separate method. Note that because of this, the constructor could be outside the onReady callback if you needed to store it somewhere; I don't, and it looks neater to me all grouped together.
  client.onReady().then(() => {
    const tokensProvider = new TokensProvider();
    tokensProvider.loadTokens(client);
    window.registerTreeDataProvider('ignorantTokens', tokensProvider);
  });
Inside the tokensprovider, we need to call sendRequest specifying that we want an ExecuteCommandRequest and providing a unique string as the command identifier and then any arguments we might want (in this case the URI of the workspace folder). When we get the response back, we need to process it but this is left as an exercise for the reader.
  async loadTokens(client: LanguageClient) : Promise<undefined> {
    if (vscode.workspace.workspaceFolders == null)
      return;
    for (var wf=0;wf<vscode.workspace.workspaceFolders.length;wf++) {
      let uri = vscode.workspace.workspaceFolders[wf].uri.toString();
      const result = await client.sendRequest(ExecuteCommandRequest.type, {
        command: 'java.lsp.requestTokens',
        arguments: [ uri ]
      })
      // this.locations.push(new ProjectTokens(vscode.workspace.workspaceFolders[wf], result));
    }
  }

On the back end, we need to make two changes. First, we need to say that we now support executeCommand and to list the unique command identifiers we support. Then, in the WorkspaceService, we need to provide a trivial implementation of the command execution - in this case, returning "hello, world".
        ServerCapabilities capabilities = new ServerCapabilities();
        ExecuteCommandOptions requestTokens = new ExecuteCommandOptions(Arrays.asList("java.lsp.requestTokens"));
    capabilities.setExecuteCommandProvider(requestTokens);
        return new WorkspaceService() {
          @Override
          public CompletableFuture<Object> executeCommand(ExecuteCommandParams params) {
            System.err.println("execute command called for " + params.getArguments().get(0));
            return CompletableFuture.completedFuture("hello, world");
          }
I thought that was all I needed to do, but it turns out that it's important to take the final step of updating the list, because we need to make sure that we send notifications when we do.

I had commented earlier about the Event object that was part of the interface and I wasn't quite sure what to do about it. The answer would appear to be that it is an idiom that you define another, similar object called an EventEmitter and then wire up the Event to be the event field of this, and then to call the fire() method on the EventEmitter when you want to the tree to be updated.
      this.locations.push(new ProjectTokens(vscode.workspace.workspaceFolders[wf], result));
    }
    this._onDidChangeTreeData.fire();
  }

  private _onDidChangeTreeData: vscode.EventEmitter<ProjectTokens | Token | TokenLocation | null | undefined> = new vscode.EventEmitter<ProjectTokens | Token | TokenLocation | null | undefined>();
  readonly onDidChangeTreeData: vscode.Event<ProjectTokens | Token | TokenLocation | null | undefined> = this._onDidChangeTreeData.event;
In order to support this, I had to change "hello, world" to be an empty JSON array:
          public CompletableFuture<Object> executeCommand(ExecuteCommandParams params) {
            System.err.println("execute command called for " + params.getArguments().get(0));
            return CompletableFuture.completedFuture(new JsonArray());
          }

Exercise

As I always go back and read my own work, I also have to follow through on the exercise of wiring up the dots. I'm not going to bore you with the details here, but you can check it out in the repository (tag VSCODE_SIDEVIEW_EXERCISE).

Conclusion

It is possible to create side views in VSCode and populate them from a repository of information that can be found on the server side.

Late in the game, I also came across ​​https://code.visualstudio.com/api/extension-guides/tree-view which contains explanations for some of the things I found confusing while doing this work. It might be worth a read.