A lot of the software I design uses a "pluggable" architecture: that is, I build a core application and then allow it to be extended by loading in "modules" that provide additional functionality.
This works very well in Java - you simply drop a new ".jar" into your classpath and the classes are there. The mechansims for finding the service implementations could be improved dramatically, but it is possible to make them work if you put the effort in.
Can the same thing be done in Go?
One of the advantages (IMHO) of Go is that it is possible to build a binary executable that you can distribute without the need for any runtime or supporting code. This advantage "pushes against" the idea of pluggable applications, in which you assume that you will need a whole directory (or more) of parts in order to get it working.
But, in what appears to be a recent change, there is now a buildmode option to build a project into a plugin, and then a plugin package to import that plugin into a container application.
I naturally ask myself: OK, how does that actually work? And this is the place where I answer such questions. Hopefully I can manage that in just one post
The Setup
As always, I feel the need to have a "sample application" in mind. I certainly don't want to do anything complicated enough to actually warrant doing this, but on the other hand, it is necessary for it to actually act like it is doing something.A common pattern I use is to have something which has a text command mode and the plugin will offer new commands. Each of these commands obviously needs to offer information about itself when the user asks for help. So let's just do that. We will write a base program that expects to see --help as its principal argument. If it is not there, it will print a usage message. The other arguments must all be repeated -m file for the various modules to be loaded. The application will load all the listed modules and then discover all the commands they offer and then ask each of them to issue some help text. I feel if I can get that working, I can do just about anything with modules, although there may be some issues with sharing types and values that I will want to double check that I have covered.
Let's Get Started
So I'm going to be doing this in my normal ignorance blog at git@github.com:gmmapowell/ignorance-may-be-strength.git, and I'm going to put everything under go-plugins.% mkdir appThe idea here is that we have an application that is capable of doing some set of tasks (such as running servers, or deploying code) and that it can have various modules to help it with, for example, database engines and front end servers. Or something. The details here do not matter. We have created three directories; app is for the core application and database and frontend will hold the plugins.
% cd app
% go mod init gmmapowell.com/app
go: creating new go.mod: module gmmapowell.com/app
% cd ../
% mkdir database
% cd database/
% go mod init gmmapowell.com/database
go: creating new go.mod: module gmmapowell.com/database
% cd ..
% mkdir frontend
% cd frontend/
% go mod init gmmapowell.com/frontend
go: creating new go.mod: module gmmapowell.com/frontend
So here's the outline code for the main app which is just processing the arguments and issuing a usage message if it's not happy. If it is happy, it lists out the modules.
package main
import (
"fmt"
"log"
"os"
)
func main() {
hasHelp := false
noMod := false
modules := make([]string, 0)
i := 1
for i < len(os.Args) {
switch os.Args[i] {
case "--help":
if hasHelp {
fmt.Println("can only specify --help once")
return
}
hasHelp = true
case "--module":
fallthrough
case "-m":
i++
if i < len(os.Args) {
modules = append(modules, os.Args[i])
} else {
noMod = true
}
}
i++
}
if !hasHelp || noMod {
fmt.Println("Usage: app --help [-m module]")
return
}
for _, m := range modules {
log.Printf("have module %s\n", m)
}
}
GOMOD_APP_ARGS:go-plugins/app/cmd/main.go
While I really don't have an idea what I'm up to here, my instinct tells me I am going to want to define some sort of "top level" object that the plugin can hand me. So I'm going to define this under pkg, because that's where I believe things should go if I want to export them, which I certainly do in this case.package api
type PluginTopLevel interface {
Methods() []PluginMethod
}
type PluginMethod interface {
Help() string
}
GOMOD_TOPLEVEL:go-plugins/app/pkg/api/toplevel.go
The plan here is that "somehow" the app will be able to ask the module to hand it something that implements PluginTopLevel. It can then ask that for all the methods, and then ask each method for its help text. From there, everything should be easy.Defining a plugin
Now we need to switch to a plugin. I'm going to start with database, but since they're both going to be basically the same, it doesn't really matter. The purpose of having two is just to make sure that everything doesn't fall apart the moment you try and load more than one module; I'm presuming that the logic of dealing with each of them is the same.So the documentation for the go build command relative to the --buildmode=plugin says:
-buildmode=pluginSo I'm assuming that I need to build the thing that I want in a package called "main". Now, normally, I would put those in a subdirectory of cmd, but this isn't really cmd, so I'm going to put it in plugin.
Build the listed main packages, plus all packages that they
import, into a Go plugin. Packages not named main are ignored.
This is my first cut at this:
package main
import "gmmapowell.com/app/pkg/api"
type Database struct {
}
func ExportMe() api.PluginTopLevel {
return &Database{}
}
func (db *Database) Methods() []api.PluginMethod {
return nil
}
GOMOD_DB_PLUGIN:go-plugins/database/plugin/export_me.go
Of course, that doesn't work. It seems that it can't find the module gmmapowell.com/app/pkg/api, even though it's right there. It seems that it would find it if it could be located (in some form) at the URL I've given, but, of course, that is a fraudulent URL. But even if it were a legitimate URL - such as at github, where I am posting this, I want to develop concurrently, not have to keep on pushing partial versions. What gives?It turns out that there is a "replace" directive in the go.mod file that says something like "when you want this module, don't go and find it, just look here". Using this enables me to say "look right next door":
module gmmapowell.com/database
go 1.23.4
require gmmapowell.com/app v0.0.0
replace gmmapowell.com/app => ../app
GOMOD_DB_PLUGIN:go-plugins/database/go.mod
I'm not quite sure how to get any of this working inside VSCode, but from the command line I can build both of these:$ (cd app/cmd/app ; go build)And I end up with an app in one directory and a plugin.so in the other. So far, so good.
$ (cd database/plugin/ ; go build --buildmode=plugin)
(Note that I also moved main.go down a level under app in this commit; it was under cmd, but that built an executable cmd, which isn't what I wanted, so I added the extra directory.)
The commit also contains a couple of .gitignore files to make sure we don't check in the binaries we produced.
Using the plugin
Given that we have collected the arguments from the command line that list modules, we should be able to open them as per the documentation for the plugin package. So let's do that:for _, m := range modules {
log.Printf("have module %s\n", m)
p, err := plugin.Open(m + "/plugin.so")
if err != nil {
panic(err)
}
init, err := p.Lookup("expose_me")
if err != nil {
panic(err)
}
log.Printf("have expose_me function %v", init)
}
GOMOD_OPEN_PLUGIN:go-plugins/app/cmd/app/main.go
(There is a typo here in that the symbol I am looking for is expose_me where it should be ExportMe; I failed to check in the change after I'd made it; it is correct in later commits. The output below uses the correct version.)And we can build it again and run it:
$ (cd app/cmd/app ; go build ; ./app --help -m ../../../database/plugin)OK, so far so good. I'm now going to go back and duplicate database as frontend.
2025/03/06 17:34:15 have module ../../../database/plugin
2025/03/06 17:34:15 have ExportMe function 0x7596f2943500
I can build and test everything again:
$ (cd frontend/plugin/ ; go build --buildmode=plugin)(In doing this, I discovered that I hadn't handled the case where I just provided the plugin name without -m, so I added a case to trap that.)
$ (cd app/cmd/app ; go build ; ./app --help -m ../../../database/plugin -m ../../../frontend/plugin)
2025/03/06 17:38:04 have module ../../../database/plugin
2025/03/06 17:38:04 have ExportMe function 0x736693743500
2025/03/06 17:38:04 have module ../../../frontend/plugin
2025/03/06 17:38:04 have ExportMe function 0x736692d43500
And Wrap This Up ..
OK, so it should be fairly simple to wrap all this up. We have the ExportMe method, and we should be able to call that. It turns out that we need a somewhat hideous cast to make that happen, but we only need to do that once. That should return us an instance of the PluginTopLevel interface. We should then be able to call Methods() to obtain all the methods, and then call Help() on all of them. I am going to assume that there is a sensible format for these strings; consequently, I can put them all in a single slice, sort it and then show the full help in some sort of order.toplevels := make([]api.PluginTopLevel, 0)
for _, m := range modules {
p, err := plugin.Open(m + "/plugin.so")
if err != nil {
panic(err)
}
init, err := p.Lookup("ExportMe")
if err != nil {
panic(err)
}
tl := init.(func() api.PluginTopLevel)()
toplevels = append(toplevels, tl)
}
texts := make([]string, 0)
for _, tl := range toplevels {
for _, meth := range tl.Methods() {
h := meth.Help()
texts = append(texts, h)
}
}
slices.Sort(texts)
for _, t := range texts {
fmt.Println(t)
}
GOMOD_FINISHED_UP:go-plugins/app/cmd/app/main.go
This does, of course, depend on me adding some methods to the plugins:package main
import "gmmapowell.com/app/pkg/api"
type Database struct {
}
func ExportMe() api.PluginTopLevel {
return &Database{}
}
func (db *Database) Methods() []api.PluginMethod {
return []api.PluginMethod{&InsertMethod{}, &QueryMethod{}}
}
type InsertMethod struct{}
func (m *InsertMethod) Help() string {
return " insert - insert data into database"
}
type QueryMethod struct{}
func (m *QueryMethod) Help() string {
return " query - interrogate the database"
}
GOMOD_FINISHED_UP:go-plugins/database/plugin/export_me.go
package main
import "gmmapowell.com/app/pkg/api"
type FrontEnd struct {
}
func ExportMe() api.PluginTopLevel {
return &FrontEnd{}
}
func (db *FrontEnd) Methods() []api.PluginMethod {
return []api.PluginMethod{&BrowseMethod{}, &QuitMethod{}}
}
type BrowseMethod struct{}
func (m *BrowseMethod) Help() string {
return " browse - open brower"
}
type QuitMethod struct{}
func (m *QuitMethod) Help() string {
return " quit - quit brower"
}
GOMOD_FINISHED_UP:go-plugins/frontend/plugin/export_me.go
OK, and now when we run it (having removed all the other tracing statements):$ (cd database/plugin/ ; go build --buildmode=plugin)Sweet.
$ (cd frontend/plugin/ ; go build --buildmode=plugin)
$ (cd app/cmd/app ; go build ; ./app --help -m ../../../database/plugin -m ../../../frontend/plugin)
browse - open brower
insert - insert data into database
query - interrogate the database
quit - quit brower
No comments:
Post a Comment