In retrospect, I think I was a little disingenuous with the ending of the last post.
Obviously, what I want to do is to replace reading the sample file with a full-on protocol communication with the test server. But I can't just do that. I need the flexibility that comes from being able to change the file (most urgently, I need to put my credentials in it). I also need to get my credentials from somewhere. But the shortest path to making a meaningful change is to replace reading the sample XML with generating it.
Having read the documentation in Transaction.pdf, I now feel I have a better understanding of how this whole thing works. This is not a CT600-specific service, but can handle many government forms. So there is a wrapper (an "envelope" that needs to be assembled) around the actual submission.
And, as I suspected before from my brief overview, it also has methods to check on the status of submissions, to obtain the outstanding submissions and to delete "dead" submissions (presumably once you have extracted some kind of authentication code).
(For those who haven't heard this before, I'm really bad at reading almost all kind of documents "in the abstract": specs, descriptions, APIs etc. One of the reasons this blog exists is as my attempt to build a "bridge" between those dry, disconnected documents and the real world; as much for me to look back at as to educate anyone else. Remember that I don't know what I'm doing here, and freely admit it. People often don't believe this, because once I have understood something, I can often quote chapter and verse of the spec in question.)
I mentioned in generating the GnuCash files that I was really unhappy with the class structures that I ended up with and questioned if I could do something more general to represent XML and have my classes be what I would expect them to be with a convenience method AsXML attached. So this time I'm going to try that. I think that also works really well with this whole "envelope"/"body" model here.
Let's get started. I'm not going to list out the whole of the sample message here (it's 126 lines), but I am going to show you excerpts. In particular, I'm not going to worry about the Body portion for now but just try and get an "Envelope" to be accepted.
Changing the Test
So we need to start by updating the test to replace the call to read the file with one that generates the file. Simples:func TestTheMinimalFileWorks(t *testing.T) {
send, err := submission.Generate()
if err != nil {
log.Fatalf("error generating xml file: %v", err)
}
cli := &http.Client{}
resp, err := cli.Post("https://test-transaction-engine.tax.service.gov.uk/submission", "application/x-binary", send)
if err != nil {
log.Fatalf("error posting xml file: %v", err)
}
body, err := io.ReadAll(resp.Body)
if err != nil {
log.Fatalf("error reading response: %v", err)
}
msg := string(body)
fmt.Printf("%s", msg)
if strings.Contains(msg, "GovTalkErrors") {
t.Fatalf("there was a GovTalkErrors block")
}
}
CT600_START_GENERATION:accounts/internal/ct600/submission/submission_test.go
Obviously, we also need to provide an implementation of Generate:package submission
import (
"bytes"
"encoding/xml"
"fmt"
"io"
"github.com/gmmapowell/ignorance/accounts/internal/ct600/govtalk"
)
func Generate() (io.Reader, error) {
msg := govtalk.MakeGovTalk()
bs, err := xml.MarshalIndent(msg.AsXML(), "", " ")
if err != nil {
return nil, err
}
fmt.Printf("%s", string(bs))
return bytes.NewReader(bs), nil
}
CT600_START_GENERATION:accounts/internal/ct600/submission/generate.go
It seems that using completely general XML structures simply doesn't allow attributes to be rendered properly. Given how important attributes are in XML, this surprises me, but I dug into the code and I still couldn't make any way to make it work. I left my attempt in genxml so that you can check my work, but this is the code I came up with:package genxml
import (
"encoding/xml"
)
type Element struct {
XMLName xml.Name
Attributes `xml:",innerxml"`
Elements
}
type Attributes any
type Elements []any
type GTAttrs struct {
XMLNS string `xml:"xmlns,attr"`
}
func LocalElement(name string) *Element {
return &Element{XMLName: xml.Name{Local: name}, Attributes: GTAttrs{XMLNS: "http://www.govtalk.gov.uk/CM/envelope"}}
}
CT600_START_GENERATION:accounts/internal/ct600/genxml/xml.go
and this is the output it gives me:<GovTalkMessage>So the whole thing is a non-starter.
<Attributes xmlns="http://www.govtalk.gov.uk/CM/envelope"></Attributes>
</GovTalkMessage>
But I am going to enforce a two-tier AsXML structure, I'm just going to have to implement 15,000 classes all of which look very similar (although I am going to use a general one if it doesn't have any attributes, which seems to be quite common in this protocol). Somewhat related, I discovered this tool out there on the internet, which will generate Go classes from XML. Useful if you want to go down a path of having one class per Element type.
So this is what I have now:
package govtalk
import "encoding/xml"
type GovTalk interface {
AsXML() any
}
func MakeGovTalk() GovTalk {
return &GovTalkMessage{}
}
type GovTalkMessage struct {
}
func (gtm *GovTalkMessage) AsXML() any {
ret := &GovTalkMessageXML{}
ret.XMLNS = "http://www.govtalk.gov.uk/CM/envelope"
ret.XSI = "http://www.w3.org/2001/XMLSchema-instance"
env := &SimpleElement{XMLName: xml.Name{Local: "EnvelopeVersion"}}
env.Text = "2.0"
ret.Elements = append(ret.Elements, env)
return ret
}
CT600_START_GENERATION:accounts/internal/ct600/govtalk/govtalk.go
The GovTalkMessage looks boring, but that's just because we haven't really done anything yet. This is where most of the logic is going to (eventually) go. In the meantime, all it really does is turn around and create a couple of XML Elements. GovTalkMessageXML is a "specific" struct because it needs the xmlns attributes, while SimpleElement is the general element I am going to use whenever we don't need that.Both of these are defined in the xml.go file:
package govtalk
import "encoding/xml"
type SimpleElement struct {
XMLName xml.Name
Text string `xml:",chardata"`
Elements
}
type GovTalkMessageXML struct {
XMLName xml.Name `xml:"GovTalkMessage"`
XMLNS string `xml:"xmlns,attr"`
XSI string `xml:"xmlns:xsi,attr"`
Elements
}
type Elements []any
CT600_START_GENERATION:accounts/internal/ct600/govtalk/xml.go
OK, I'm going to dive into a bunnyhole now and generate a whole bunch of those things.Here we go:
func (gtm *GovTalkMessage) AsXML() any {
ret := &GovTalkMessageXML{}
ret.XMLNS = "http://www.govtalk.gov.uk/CM/envelope"
ret.XSI = "http://www.w3.org/2001/XMLSchema-instance"
env := &SimpleElement{XMLName: xml.Name{Local: "EnvelopeVersion"}}
env.Text = "2.0"
header := &SimpleElement{XMLName: xml.Name{Local: "Header"}}
msgDetails := &SimpleElement{XMLName: xml.Name{Local: "MessageDetails"}}
test := &SimpleElement{XMLName: xml.Name{Local: "GatewayTest"}}
test.Text = "1"
msgDetails.Elements = append(msgDetails.Elements, test)
sndrDetails := &SimpleElement{XMLName: xml.Name{Local: "SenderDetails"}}
header.Elements = append(header.Elements, msgDetails, sndrDetails)
gtDetails := &SimpleElement{XMLName: xml.Name{Local: "GovTalkDetails"}}
keys := &SimpleElement{XMLName: xml.Name{Local: "Keys"}}
utr := &KeyElement{Type: "UTR", Value: "8596148860"}
keys.Elements = append(gtDetails.Elements, utr)
gtDetails.Elements = append(gtDetails.Elements, keys)
body := &SimpleElement{XMLName: xml.Name{Local: "Body"}}
ret.Elements = append(ret.Elements, env, header, gtDetails, body)
return ret
}
CT600_XML_ELEMENTS:accounts/internal/ct600/govtalk/govtalk.go
And over in the xml.go file, I defined the struct Key since it needs a Type attribute:type KeyElement struct {
XMLName xml.Name `xml:"Key"`
Type string `xml:"Type,attr"`
Value string `xml:",chardata"`
}
CT600_XML_ELEMENTS:accounts/internal/ct600/govtalk/xml.go
That all "works" (as far as it goes), but while I don't know about you, I don't like how much repetition it has in it. Let's fix that.func (gtm *GovTalkMessage) AsXML() any {
env := ElementWithText("EnvelopeVersion", "2.0")
test := ElementWithText("GatewayTest", "1")
msgDetails := ElementWithNesting("MessageDetails", test)
sndrDetails := ElementWithNesting("SenderDetails")
header := ElementWithNesting("Header", msgDetails, sndrDetails)
utr := Key("UTR", "8596148860")
keys := ElementWithNesting("Keys", utr)
gtDetails := ElementWithNesting("GovTalkDetails", keys)
body := ElementWithNesting("Body")
return MakeGovTalkMessage(env, header, gtDetails, body)
}
CT600_CLEANED_XML:accounts/internal/ct600/govtalk/govtalk.go
Which obviously requires a little bit of help and support in terms of the functions we are using:func ElementWithNesting(tag string, elts ...any) *SimpleElement {
env := &SimpleElement{XMLName: xml.Name{Local: tag}}
env.Elements = elts
return env
}
func ElementWithText(tag, value string, elts ...any) *SimpleElement {
env := &SimpleElement{XMLName: xml.Name{Local: tag}}
env.Text = value
env.Elements = elts
return env
}
func MakeGovTalkMessage(nesting ...any) *GovTalkMessageXML {
return &GovTalkMessageXML{
XMLNS: "http://www.govtalk.gov.uk/CM/envelope",
XSI: "http://www.w3.org/2001/XMLSchema-instance",
Elements: nesting,
}
}
func Key(ty, value string) *KeyElement {
return &KeyElement{Type: ty, Value: value}
}
CT600_CLEANED_XML:accounts/internal/ct600/govtalk/xml.go
OK, now that is a bit tidier, I'm going back into my bunnyhole.This is what I ended up with. I have used a mixture of styles here, from moving the body off into its own function, having things in temporary variables, and doing things inline. I may not have made the best tradeoffs, but it seemed a good idea at the time.
func (gtm *GovTalkMessage) AsXML() any {
env := ElementWithText("EnvelopeVersion", "2.0")
msgDetails := ElementWithNesting(
"MessageDetails",
ElementWithText("Class", "HMRC-CT-CT600"),
ElementWithText("Qualifier", "request"),
ElementWithText("Function", "submit"),
ElementWithText("Transformation", "XML"),
ElementWithText("GatewayTest", "1"),
)
sndrDetails := ElementWithNesting(
"SenderDetails",
ElementWithNesting(
"IDAuthentication",
ElementWithText("SenderID", "Provided by the SDST"),
ElementWithNesting(
"Authentication",
ElementWithText("Method", "clear"),
ElementWithText("Role", "Principal"),
ElementWithText("Value", "Provided by the SDST"),
),
),
)
gtDetails := ElementWithNesting(
"GovTalkDetails",
ElementWithNesting("Keys", Key("UTR", "8596148860")),
ElementWithNesting("TargetDetails", ElementWithText("Organisation", "HMRC")),
ElementWithNesting(
"ChannelRouting",
ElementWithNesting(
"Channel",
ElementWithText("URI", "Vendor ID"),
ElementWithText("Product", "Product Details"),
ElementWithText("Version", "Version #"),
),
),
)
return MakeGovTalkMessage(env,
ElementWithNesting("Header", msgDetails, sndrDetails),
gtDetails,
gtm.makeBody())
}
func (gtm *GovTalkMessage) makeBody() any {
body := ElementWithNesting("Body")
return &body
}
CT600_HEADER:accounts/internal/ct600/govtalk/govtalk.go
This leaves us with the "envelope" portion much as we had it in the sample, and the body just empty. And when we submit it, it again tells us we can't authenticate. Let's tackle that next time.
No comments:
Post a Comment