The Client
I'm going to start with the client, in large part because it's a nice simple thing. I'm going to present the client files one at a time, in their entirety, with commentary among the various parts.cmd/ledgerclient/main.go
The first section of the file is mainly boilerplate.package mainThe package statement identifies the package that this file claims to be in. In order to become a command executable, it must be declared as being in the package main; if you don't do this the function main() will be flagged as not being used anywhere.
import (
"fmt"
"hash/maphash"
"log"
"net/url"
"github.com/gmmapowell/ChainLedger/internal/api"
"github.com/gmmapowell/ChainLedger/internal/client"
)
The import statement lists all the other packages which are going to be used in this file. If a package is going to be used, it must be listed and, conversely, if it is listed it must be used.
The fmt package is where Go places things like Printf, which will be using to confirm that we have submitted a transaction. We will be using the hash/maphash package to generate an acceptable hash to submit as the (alleged) hash of the contents of the document we are submitting. log contains all the statements to do logging and net/url contains the definition of the URL struct which we will be using everywhere to validate our strings are valid URLs.
The main module delegates most of its work to two internal packages (i.e. packages elsewhere in this same project), api and client. We will cover those when we get there.
func main() {As is the case with most C-based languages, go defines a function called main which is the entry point for the entire program. These opening lines show the basic setup of the client. We create an internal repository for things to do with users and nodes (the client and node respositories will be similar, but will be different). For now, we are just using this to hide the details of creating and managing the initial user's private key, but in the log run will store more and more information in there. For now, we are not using any external sources of information (e.g. command line arguments) but we will come back and do that later.
repo, e1 := client.MakeMemoryRepo()
if e1 != nil {
panic(e1)
}
uid := "https://user1.com/"
uu, e2 := url.Parse(uid)
if e2 != nil {
panic(e2)
}
pk, e3 := repo.PrivateKey(uu)
if e3 != nil {
panic(e3)
}
One such thing that should be an argument is the submitting user id, which is hardcoded to be https://user1.com/. I am using URLs as IDs for the simple reason that they have some structure and it makes it easy to have user ids with given relationships. In the long run, I expect that I will want this to be "real" URLs that return a JSON document that describes the user's key information, such as the public key for their signing key.
And then we ask the repository for the private key for the current user. Note that the repository is not going to have ALL the private keys for ALL the users; any given instantiation of the client repository will probably only have ONE private key (for the given user), although it is not going to be a hardwired constraint (mainly because I want to violate it in order to build a stress test). Ultimately, I expect the repository to read from files, databases or something like Amazon SSM parameters, depending on the deployment environment. In that case, you have access to whatever would be available.
Here in this code it is possible to see how much of the code is dominated by error handling in the absence of exception handling. I am optimistic that in the fullness of time I will become better at dealing with this and it will look less ugly.
One of the things that this does surface, however, is just how many things can go wrong, and it does (to a certain extent) force you to think about them. I'm not really sure what to do with any of these errors - for now, at least, nothing should go wrong here, because I am not depending on either external data or external services. I used the panic command because I saw it and thought it probably did what I wanted here: made it clear there was a problem.
cli, err := client.NewSubmitter("http://localhost:5001", uid, pk)In order to connect to the server, we need a "transaction submitter". This is a module which takes the URL of a node to connect to, a the URL id of a user who is going to be submitting the transactions, and their private key. It is then ready to submit all of the transactions on their behalf.
if err != nil {
log.Fatal(err)
return
}
If anything goes wrong, the error is logged and the main function returns. This is subtly different handling from the above, which is because there really are a number of things that can go wrong here, most notably that the server isn't running.
var h maphash.HashThe rest of the code comes here in a rush, in which we create a new transaction message which is defined by the URL http://tx.info/msg1 and for which we provide the hash of the alleged contents "hello, world". As I said in the design, it is not assumed that either the client or the server can read the URL, and indeed it may not exist, so the URL and the hash must both be provided.
h.WriteString("hello, world")
tx, err := api.NewTransaction("http://tx.info/msg1", &h)
if err != nil {
log.Fatal(err)
return
}
err = tx.SignerId("https://user2.com")
if err != nil {
log.Fatal(err)
return
}
err = cli.Submit(tx)
if err != nil {
log.Fatal(err)
return
}
Before the message can be hashed and signed, the transaction must be given a list of all the users who will sign it. Now, obviously, it is not possible for any one user to sign for all the parties, so the transaction will only have one signature - for the submitting party - and, given that the current party is submitting it, the submitting user does not need to explicitly provide their user id.
Finally, by calling Submit, on the cli reference (the submitter), the transaction is signed and uploaded to the server.
All of these methods can reasonably fail, but even so, all I'm doing is logging the errors. In the fullness of time, I probably want to revisit at least the most common errors.
fmt.Printf("submitted transaction: %v", tx)Finally, if all went well, we report to the user that the transaction was (successfully) submitted.
}
internal/client/submit.go
The usual boilerplate starts us off:package clientNow, I'm new to Go, so I'm going to cause offence and say we are going to declare a class. It's not a class, it's a struct, and you are (I am) going to hobble yourself if you thing that classes and structs are the same thing (in either direction). But anyway, we are going to declare a class to submit transactions to the server.
import (
"crypto/rsa"
"net/http"
"net/url"
"github.com/gmmapowell/ChainLedger/internal/api"
)
type Submitter struct {This declares a struct with three fields. Because the field names start with lowercase letters, they are all private fields, which means that they can be seen in the methods in this file, but not elsewhere (I think, I'm not entirely sure what the rules are).
node *url.URL
iam *url.URL
pk *rsa.PrivateKey
}
The three fields are:
- node is the URL of the server we are intending to connect to;
- iam is the URL which identifies the submitting user;
- pk is the private signing key of the submitting user.
func NewSubmitter(node string, id string, pk *rsa.PrivateKey) (*Submitter, error) {Go does not have constructors as such. Instead a new instance of a struct is created using the syntax shown on the last line of this function: the struct name, followed by field assignments in curly braces. The equivalent to constructors are the kind of functions shown here, which take arguments and then create the appropriate object. So here, we have a function that takes string versions of node and user id, and confirms that they are in fact valid urls before placing them in a Submitter structure.
nodeAddr, e1 := url.Parse(node)
if e1 != nil {
return nil, e1
}
iam, err := url.Parse(id)
if err != nil {
return nil, err
}
return &Submitter{node: nodeAddr, iam: iam, pk: pk}, nil
}
Note that in order to return a *Submitter, we have to use the & operator to convert an instance into a pointer (or some such technical language).
func (s *Submitter) Submit(tx *api.Transaction) error {This is how methods are defined and attached to a struct. The parentheses after the func keyword indicate that there is a "target object" (i.e. the instance) which is implicit in the function call, but other than that has an argument name just like any other (there is no built-in this variable, although there would be nothing stopping you calling the variable here this or self everywhere if you wanted to).
var e error = tx.Signer(s.iam)
if e != nil {
return e
}
e = tx.Sign(s.iam, s.pk)
if e != nil {
return e
}
json, e2 := tx.JsonReader()
if e2 != nil {
return e2
}
cli := http.Client{}
_, e3 := cli.Post(s.node.JoinPath("/store").String(), "application/json", json)
return e3
}
This first makes sure that the submitting user is listed as a Signer, and then uses their private key to Sign the key fields in this block. Note that it is the responsibility of the Transaction code to make sure that all the users sign exactly the same block, although this first version of the code doesn't do that: we'll come back to that in a bit.
Finally, the code submits a JSON version of the Transaction struct to the server. Once again, it feels like the error handling stops me writing the fluid code I would want to; it is not clear to me how to put the tx.JsonReader() in the call to cli.Post because I cannot handle the error return.
internal/api/transaction.go
Once again, the boilerplate for completeness so that you can't say I don't show you everything.package apiThe purpose of the Transaction struct is to manage the lifecycle of a transaction on the client side for submission to a server. It will also be stored on the server side until the server has one copy from each of the signatories. We'll get to that later. Quite a bit later.
import (
"bytes"
"crypto"
"crypto/rand"
"crypto/rsa"
"crypto/sha512"
"encoding/json"
"fmt"
"hash"
"io"
"net/url"
"github.com/gmmapowell/ChainLedger/internal/types"
)
type Transaction struct {A transaction consists of three things: the link to the content, ContentLink, the hash of said content ContentHash and the signature block Signatories.
ContentLink *url.URL
ContentHash hash.Hash
Signatories []*types.Signatory
}
For now, I have made all of these public fields, but I think that just reflects the fact that I am still at the experimental phase: do not be surprised if they are made private at some point in the future.
func NewTransaction(linkStr string, h hash.Hash) (*Transaction, error) {Again, we have here a "constructor pattern" where NewTransaction is responsible for taking the expected arguments, validating them, and returning a pointer to the Transaction block.
link, err := url.Parse(linkStr)
if err != nil {
return nil, err
}
return &Transaction{ContentLink: link, ContentHash: h, Signatories: make([]*types.Signatory, 0)}, nil
}
Note the use here of make to create a 0-length array of Signatory items. We will add the signatories individually below, so for now we don't have any.
func (tx *Transaction) SignerId(signerId string) error {Each of the signers for the transaction needs to be identified. During submission, the Signer method is called with the submitter's URL. All the others need to be explicited added, as happens in main.
signer, err := types.OtherSignerId(signerId)
return tx.addSigner(signer, err)
}
func (tx *Transaction) Signer(signerURL *url.URL) error {
signer, err := types.OtherSignerURL(signerURL)
return tx.addSigner(signer, err)
}
func (tx *Transaction) addSigner(signer *types.Signatory, err error) error {
if err != nil {
return err
}
tx.Signatories = append(tx.Signatories, signer)
return nil
}
The code here has two possible paths, with either with a string or a URL being passed in. The appropriate method in the types module (see below) is called to obtain a Signatory struct, and then the common path in addSigner (its name begins with a lower case letter, because it's an internal method) is called to add the Signatory to the Transaction.
The append function extends the array slice to make room for an additional element at the end, and then puts this new Signatory there.
func (tx *Transaction) Sign(signerURL *url.URL, pk *rsa.PrivateKey) error {This is responsible for doing all of the work to create a signature - and even then it doesn't sign the right thing!
return tx.doSign(signerURL, pk, nil)
}
func (tx *Transaction) doSign(signer *url.URL, pk *rsa.PrivateKey, e1 error) error {
if e1 != nil {
return e1
}
h, e2 := tx.makeSignableHash()
if e2 != nil {
return e2
}
sign, e3 := makeSignature(pk, h)
if e3 != nil {
return e3
}
done := false
for _, signatory := range tx.Signatories {
if signatory.Signer == signer {
signatory.Signature = sign
done = true
break
}
}
if !done {
return fmt.Errorf("there is no signatory %v", signer)
}
return nil
}
func (tx *Transaction) makeSignableHash() (hash.Hash, error) {
var h = sha512.New()
h.Write([]byte("hello, world"))
return h, nil
}
func makeSignature(pk *rsa.PrivateKey, h hash.Hash) (*types.Signature, error) {
sum := h.Sum(nil)
sig, err := rsa.SignPSS(rand.Reader, pk, crypto.SHA512, sum, nil)
if err != nil {
return nil, err
}
var ret types.Signature = sig
return &ret, nil
}
The first method follows the same pattern as the block above in order to allow for a Sign method with just a string, but that method does not exist.
The second method (doSign), first calls makeSignableHash to figure out what to sign, then calls makeSignature to sign that hash. Finally, it scans through all the signatory blocks and finds the one whose signatory is the submitting user. It is, of course, an error for there not to be one, and the fmt.Errorf is a special version of Printf which creates an error object.
The makeSignableHash object is supposed to lay out and organise the contents of the Transaction object into a consistent block, regardless of how it was constructed, and then produce a SHA-512 hash of it. At the moment, it does not do this, but rather hashes "hello, world". We'll come back to that later.
Finally, makeSignature takes a private key and the hash, and generates the Sum of the hash and calls SignPSS to sign it. This is all very easy code to write, although I'm not 100% convinced I actually understand what I've done. At some point we will write code on the server side to verify the signatures. If it turns out that we encounter some problems, we will come back and revisit it.
Signature is my own internal type (we'll see it below) and it is for all intents and purposes "the same as" []byte, which is what we get back from SignPSS. However, I cannot directly return the address of sig but have to first pass it to an explicitly typed variable ret before I can take the address. I am sure that there are good reasons for this (that probably involve the word 'contravariance') but I do not know what they are yet.
func (tx *Transaction) JsonReader() (io.Reader, error) {Finally we have a couple of methods to return a Transaction in a readable format. The JsonReader function simply marshals the transaction to JSON, which is very nicely handled in Go.
json, err := json.Marshal(tx)
if err != nil {
return nil, err
}
return bytes.NewReader(json), nil
}
func (tx *Transaction) String() string {
return fmt.Sprintf("Tx[%s]", tx.ContentLink)
}
Meanwhile the String function is just like a toString method in Java or JavaScript: for any given object, it will convert the contents to a string on demand. This behaviour is defined by the Stringer interface, which Transaction implicitly implements by providing the String method. Clever.
internal/client/repo.go
The client repository is a placeholder for a configurable information store which is intended to hold all of the information needed by the client. For now, we are just using it as a place to hide the public/private key for a user.We start with the boilerplate:
package clientBecause we will want to have many different ways of collecting the data, we are going to define ClientRepository as an interface:
import (
"crypto/rand"
"crypto/rsa"
"fmt"
"net/url"
)
type ClientRepository interface {For now, this interface is very simple, just allowing us to ask for the private key of a given user URL.
PrivateKey(user *url.URL) (*rsa.PrivateKey, error)
}
Internally, we are going to store records about all the users in the system, which are in ClientInfo structs. As I write this, I wonder if the name might want to change to UserInfo. Don't be surprised if it does at some point.
type ClientInfo struct {Each user has three fields associated with them for now - their unique URL id, their private key and their public key. I am storing all three of these, even though for now, we will only use the private key.
user *url.URL
privateKey *rsa.PrivateKey
publicKey *rsa.PublicKey
}
type MemoryClientRepository struct {MemoryClientRepository is a concrete implementation of ClientRepository and is one that is intended for "toy" use: testing or demonstration purposes. We will come back and build better things as we need them. This basically stores in memory a map of URLs to ClientInfo records. The MakeMemoryRepo function is a constructor of sorts, but it also initializes the repository with our chosen user.
clients map[url.URL]*ClientInfo
}
func MakeMemoryRepo() (ClientRepository, error) {
mcr := MemoryClientRepository{clients: make(map[url.URL]*ClientInfo)}
mcr.NewUser("https://user1.com/")
return mcr, nil
}
Note that the key of the map here is a url.URL not a *url.URL. That was my original implementation, but it did not work. Why not? Because the keys in a map have to match by equality, and if you use pointers, they don't. I'm not entirely sure how equality works in Go (yet), but it is reasonable that you say that two pointers are equal if they are the same pointer, but that two URLs are the same if they have the same "value", i.e. represent the same address. That's what seems to happen here.
func (cr MemoryClientRepository) PrivateKey(user *url.URL) (pk *rsa.PrivateKey, e error) {This method recovers the private key for a given user by accessing the map for the URL passed in. Again, note that we are using *user as the key, since we are given a pointer. This still seems weird in my head, but I'm sure I'll wrap my head around it in the end.
entry := cr.clients[*user]
if entry == nil {
e = fmt.Errorf("there is no user %s", user.String())
} else {
pk = entry.privateKey
}
return
}
If we don't find an entry in the map, we will return an error; otherwise we extract the private key. This assumes that there is a private key associated with the user; we should probably (and probably will at some point) check if the user has a private key and return an error if they do not. Of course, we could say that returning the pair nil, nil means that we did find the user, but they didn't have a private key - test for that case! It depends on how we want to code things.
func (cr *MemoryClientRepository) NewUser(user string) error {The final method is the one that adds a new user to the repository. Since it takes a string url, it must first parse that (and returns an error if it does not parse correctly). We check that this is not a duplicate user, because that is kind of the definition of a "new" user. Then we generate a key pair and store the new triple of user, public key and private key in the list of clients.
u, e1 := url.Parse(user)
if e1 != nil {
return e1
}
if cr.clients[*u] != nil {
return fmt.Errorf("user %s already exists in the repo", user)
}
pk, e2 := rsa.GenerateKey(rand.Reader, 2048)
if e2 != nil {
return e2
}
cr.clients[*u] = &ClientInfo{user: u, privateKey: pk, publicKey: &pk.PublicKey}
return nil
}
internal/types/signatory.go
The signatory type is just a struct combining the URL id of a signer and their (optional) signature.We start with the usual boilerplate.
package typesThe actual signatory struct is very simple.
import (
"net/url"
)
type Signatory struct {And then there are two constructor methods, one that takes a URL signer id and one that parses it from a string. In both cases, the Signature is initally left blank.
Signer *url.URL
Signature *Signature
}
func OtherSignerURL(u *url.URL) (*Signatory, error) {
return &Signatory{Signer: u}, nil
}
func OtherSignerId(id string) (*Signatory, error) {
u, err := url.Parse(id)
if err != nil {
return nil, err
}
return OtherSignerURL(u)
}
internal/types/signature.go
This final client file is not that complicated. It basically amounts to a type alias.package typesThis just says that I can write types.Signature anywhere and it is basically the same as writing []byte, although, as we saw when we used it, that is not true when using the & operator.
type Signature []byte
The Server
Moving on from the client side, we are going to have a server which initially doesn't do very much. It starts up and makes itself available for clients to submit transactions. It reports that it has done so.cmd/chainledger/main.go
As with the client, we put the main() function in a file called main.go in the directory cmd/chainledger which causes a binary chainledger to be produced with go build.It starts with the usual boilerplate:
package mainand then has the main() function which depends heavily on code included from internal:
import (
"errors"
"fmt"
"log"
"net/http"
"github.com/gmmapowell/ChainLedger/internal/clienthandler"
)
func main() {Because this is a server, I will be very inclined to log everything that happens. So, we start by acknowledging that we have started.
log.Println("starting chainledger")
storeRecord := clienthandler.NewRecordStorage()
cliapi := http.NewServeMux()
cliapi.Handle("/store", storeRecord)
err := http.ListenAndServe(":5001", cliapi)
if err != nil && !errors.Is(err, http.ErrServerClosed) {
fmt.Printf("error starting server: %s\n", err)
}
}
The NewRecordStorage class is the class which will be responsible for handling the receipt and processing of a transaction coming from the client. I'm not entirely sold on the name at this point, but the whole thing is really just a placeholder at the moment, so I'll probably change it when I feel the need.
The rest of the code just sets up a web server. cliapi is a http.ServeMux, which I think is what you need in order to have different services listen on different ports (but I'm not sure). There is something to be said for the fact that this is a case of "YAGNI", but since I believe I am more likely to need it (and forget that it exists and waste time trying to figure out what's going wrong) that I am to not need it, I have put it there.
This ServeMux then takes the Handle method to associate a web path with a handler (which must implement the ServerHTTP method in order to be an http.Handler); this is what the RecordStorage class does.
The ListenAndServe method opens up a socket to listen on the named port and dispatches any incoming requests to the identified muxer. It's important to note that this method blocks dispatching the incoming messages, so it will be important to call this method in a goroutine if we want to have multiple running concurrently.
The final couple of lines deal with error cases, the most likely of which is that we already have the server running on the identified port.
internal/clienthandler/recordstorage.go
This is the handler for the /store web path.The usual boilerplate gets us going:
package clienthandlerWe need to declare a struct and have a constructor for it.
import (
"log"
"net/http"
)
type RecordStorage struct {This is all very simple, mainly because it is a dummy implementation at the moment. We will come back to it next time when we start doing some serious implementation.
}
func NewRecordStorage() RecordStorage {
return RecordStorage{}
}
// ServeHTTP implements http.Handler.The one thing I did manage to get the VSCode plugin to "Quick Fix" for me was declaring RecordStorage and then using it where an http.Handler was needed. It told me it didn't have this method and implemented it for me (including adding the comment). I then added the one line "implementation" which is basically logging the fact that we have received a request to store a record, the said request being a JSON object in the body of the request. Again, we will come back to this next time.
func (r RecordStorage) ServeHTTP(http.ResponseWriter, *http.Request) {
log.Println("asked to store record")
}
internal/records/storedtransaction.go
Start with the boilerplate:package recordsAt the moment, this is a bit speculative, because it's not used anywhere, but it feels so key to me in terms of everything we are trying to accomplish that I couldn't imagine not defining it early. Apologies to all the YAGNI folks, but I am going to need it, and I want it there early. Having said that, I believe more fields will ultimately be added to this (in particular the node will sign it), so I'm certainly not going all waterfall on you.
import (
"hash"
"net/url"
"github.com/gmmapowell/ChainLedger/internal/types"
)
type StoredTransaction struct {This is the version of the transaction that we are going to log on the permanent record. In order to make this happen, we need to collect all the copies that are sent across from the clients and match them together. The last three fields obviously match the fields that are in the Transaction object that will be coming over from the client in the body of the request; the other two will be attached when we get around to processing it.
txid hash.Hash
whenReceived types.Timestamp
contentLink url.URL
contentHash hash.Hash
signatories []types.Signatory
}
internal/types/timestamp.go
Again, this is a very simple type alias at the moment, although I will probably add more functionality to it later.The purpose of this is to allow me to store a timestamp using the semantics of JavaScript (milliseconds centred around midnight at the start of Jan 1, 1970 GMT) in a single int64.
package types
type Timestamp int64
No comments:
Post a Comment