Connecting VSCode to Java

The next step in the process of integrating my compiler with VSCode is to get the LanguageClient inside VSCode talking to an LSP server running inside a Java process.

I was pinning my hopes on deriving this code from what  Adam Voss had done. Sadly, I could not reverse engineer what he had done on the client side, so I started trying to research for myself. Sadly, although there appears to be good "explanatory" documentation for VSCode, it doesn't seem like there is very much in the way of "reference" documentation, so I ended up looking  at the code.

Now, I'm not a typescript expert, but going through this, it seems that what I really want to do is to provide a hash for the "server options" containing the field command in it. OK, I can do that. Everything else I'm going to liberally borrow from the lsp-sample from Microsoft.

Let's get coding

Another problem rears its head at this point. I have one project; lsp-sample seems to have two. Logically, it has a client and a server. I want to copy the client and do my own server (in Java). But the top level directory also has a package.json and the relevant items seem to be spread across the two. I don't really understand how this works, but I will just steal what I need to go in my package.json and hope for the best.

First off, I need the dependency on language-client:

"dependencies": {
  "vscode-languageclient": "^6.1.3"
},

Then I need to define activationEvents, which if I understand it, is the way in which you tell VSCode that your extension is willing to take on a particular file (possibly along with other situations).

So we declare two activation events (one for each of the two languages declared in package.json) which notice when an editor is opened which meets their criteria for editing.

"activationEvents": [
  "onLanguage:flas",
  "onLanguage:flas-st"
],

When I tried this, it didn't work and I received an error message that

properties `activationEvents` and `main` must both be specified or must both be omitted

I didn't see anything about this in the documentation, but by reference to the sample, it would seem that you have to specify a value for main in package.json, pointing to where the extension.ts file is found.

"main": "out/extension.js",

This may not immediately appear to be where extension.ts will be found, and it's not. Because in the real world node uses JavaScript, the "out/" is required because tsc is generating the JavaScript file (also note the .js extension here).

Defining the Extension

So that's the configuration. But what goes in extension.ts? I'm not going to reproduce it all here, but it is complicated enough - and took me long enough to figure out - that I think it's worth digging into a little bit.

Working backwards, we need to create and start a LanguageClient:

// Create the language client and start the client.
client = new LanguageClient(
  'IgnorancePlugin',
  'Plugin through Ignorance',
  serverOptions,
  clientOptions
);
        
// Start the client. This will also launch the server
client.start();

The four fields here are:

• the id of the plugin which will come up from time to time later;
  
• the title of the plugin which is displayed in the output and extensions windows in VSCode;
  
• the options about how to run the LSP server;
  
• the options about how the client is configured.
  

The client options seem fairly easy, although I have to say that I didn't dig too far into what all the possibilities were.

// Options to control the language client
let clientOptions: LanguageClientOptions = {
  // Register for our languages
  documentSelector: [
    { scheme: 'file', language: 'flas' },
    { scheme: 'file', language: 'flas-st' }
  ]
};

This seems to me somewhat duplicative of what we configured in the package.json but it may not be.

Finally, we have the server options, which, as noted above, can come in one of several varieties. To specify an external server which needs to be launched each time, the server options need to be an object with a command specified. args, env and cwd may also be specified. In the absence of cwd the current workspace root is used.

Thus I end up with these options:

let serverOptions: ServerOptions = {
  command: "java",
  args: [
    "-jar",
    path.resolve(context.extensionPath, '..', 'lsp-java', 'build', 'libs', 'lsp-java-all.jar')
  ]
};

Here, context.extensionPath is the path where the extension is found. Because I know that the Java binary is going to be found relative to the extension, I can specify this here. I'm not sure what happens when you come to package the extension for distribution, but that's a topic for another day.

Oh, and don't think that I figured all this out a priori. I spent a lot of time running sample scripts that were outputting relevant information and causing errors in VSCode to find out all the information I needed.

The Java Server

On the server side, I simply repackaged the Adam Voss server, putting it into my own package (under the lsp-java directory), and made it operate over standard input/output rather than using a socket.

So now it works, right? How do we know?

It appears that there is a mechanism to  view the communication between client and server, but how easy is that to actually do? Actually, not too hard.

First off, you need a block like this in the contributes hash of package.json.

"configuration": {
  "type": "object",
  "title": "Ignorance Settings",
  "properties": {
    "IgnorancePlugin.trace.server": {
    "scope": "window",
    "type": "string",
    "enum": [
      "off",
        "messages",
        "verbose"
      ],
      "default": "off",
      "description": "Traces the communication between VS Code and the language server."
    }
  }
}

Overall, the "configuration" block defines all the settings that your extension has. You can define anything (within reason) here and access it from both the client and the server.

The settings block is automatically configured as a properties "window" inside Settings. If you go to Code > Settings and then select Extensions, you will see a sub-block called Ignorance Settings (the name comes from the title above).

The property defined here is interpreted directly by VSCode as defining the trace level of the communication. This works by knowing exactly the plugin name (the id of the plugin as specified in the LanguageClient constructor in extension.ts) followed by .trace.server. From the settings window, it is possible to change the value to messages or verbose and see the communication between client and server. This is obviously vital for understanding what is going on.

Once turned on, you can quit the Extension window and restart it. As you do so, you should see the messages appear in the Output window. If you can't see the Output window anywhere, you can make it pop up by selecting View>Output from the main menu.

When I do this and restart with contract.st open, I see three messages sent across to the server: initialize - (0), initialized and textDocument/didOpen. It's a lot of output, so I'm not going to reproduce it all here, but a number of fields are interesting.

In initialize, a rootPath and a rootUri are passed across, which appear to be the location of the workspace folder. The response contains information about the features that are implemented - presumably derived from the code I have copied across to get started.

The initialized message is empty.

The textDocument/didOpen message contains a full uri, the languageId which VSCode has identified it having, a version number (which is updated every time you make a change - i.e. type anything), and the full text of the original document.

Every time the document changes, a textDocument/didChange message is sent across: the textDocument element describes the uri and updated version of the document, and the contentChanges contains the full text of the document. I believe that advanced usage exists to say that you only want to see some sub-ranges of the changes, but for me right now, getting the whole document every time feels like a win. In fact, it would seem that this is a setting configured during the initialization step (see IgnorantLanguageServer.java).

Wiring up a "Real" Compiler

In my first server-side check in, I simply accepted the code as I found it, but it didn't really do what I wanted. Here I am going to rework this code and, as I do so, describe how it now works.

Now, I don't want to wire up my full compiler at the moment (well, actually I do, but here is not the place and now is not the time). But I definitely want to check out how to deal with errors and report messages if something has gone wrong. So I am going to write a very simple compiler to handle the flas and flas-st languages without dealing with all their complexity.

FLAS is an indentation-based language, so I'm going to start by saying that we can have two top-level elements in FLAS files, contract and card (mainly because that is what is in the sample I have here). Any other keyword (for example error) is going to be an error and a message needs to come back to that effect. Lines that are not indented are considered comments; lines that are indented more than one tab stop are ignored by this simple parser.

The main function is in LSPServer. This simply creates an IgnorantLanguageServer and creates an LSP server using this and the combination of standard input and output.

The top level of the LSP server is IgnorantLanguageServer. It is responsible for setting up the connection and wiring things together. The initialize method is the first method called from the client side and passes in the expectations from the client side; the response is the set of capabilities that the server is prepared to offer.

The connect method is called from LSPServer when the client connects and enables the server to respond.

Because we want to implement the text document service, we do that and return a wrapper around our simple parser, the ParsingTextDocumentService. While there are many methods this could implement, we are basically just interested in the client opening or changing documents. Every time this happens, we parse the document using our SimpleParser; in the process of parsing, it sends back any errors it encounters.

Having done all of that, restarting the extension window of VSCode produces messages about the errors in our code. Great!

Conclusion

In this episode, we successfully wired up a Java based back end for the LSP and enabled it to read and parse documents sent from the editor. In doing so, we were able to send back errors as we came across them.

That's most of what I want to do. There are just two more things I want to try in this fake universe - can I navigate to a definition from a reference and can I complete a typename?