So, ultimately I gave up and submitted my accounts by hand. This is the final year for which this is an option - the submission tool is going away soon - so either I need to get this working by next year or I need to sign up for some "professional" software. Obviously, that would be cheaper - especially as there seem to be some free alternatives out there - but I'm generally not one to give up on things like this, especially if there's a chance to learn something.
I left it a couple of months, and then idly googled around the issue.
I came across the site https://microaccounts.uk which is an iXBRL accounts generator. When I first tried this, it crapped out, but then it worked and generated an iXBRL file which actually passes the Companies House validation. Excellent! So let's see if we can submit this to the CT600 test service.
So, no, it comes back with the same kinds of errors - the problem seems to be that the schema headers - which are being removed by the (required) canonicalization, are, in fact, required. Let's go off and research canonicalization.
In doing this, I came across the concept of "inclusive" vs "exclusive" canonicalization. It seems that this is a concept which is based on whether you are including the canonicalized document in an existing scope or want it to be more standalone. It would seem that in spite of including our document in a bigger context, we do, in fact, want it to be treated as more standalone. And this means that we want to use "inclusive canonicalization".
The library we are using does not have that option - it is a straightforward exclusive canonicalization library. So we need another one. I'm guessing here, but this one seems to do the job.
Interestingly, it depends on its own XML implementation, which feels like it is a fundamentally better fit for what I want to do than the standard library, so if I can get this to work, I will rework everything here with the etree library.
package govtalk...
import (
"bytes"
"crypto/sha1"
"encoding/base64"
"encoding/xml"
"fmt"
"log"
"strings"
"github.com/unix-world/smartgoext/xml-utils/c14n"
"github.com/unix-world/smartgoext/xml-utils/c14n/etree"
)
func canonicaliseBody(from *SimpleElement) (string, error) {
body := MakeBodyWithSchemaMessage(from.Elements...)
// Generate a text representation
bs, err := xml.MarshalIndent(body, "", " ")
if err != nil {
return "", err
}
bs, err = placeBefore(bs, "<Sender>", "\n ")
if err != nil {
return "", err
}
/*
// now canonicalise that
decoder := xml.NewDecoder(bytes.NewReader(bs))
out, err := c14n.Canonicalize(decoder)
if err != nil {
return "", err
}
*/
/*
xmldsig := chilkat.NewXmlDSig()
canonXml := xmldsig.CanonicalizeXml(strXml, "C14N", false)
fmt.Println(*canonXml)
*/
canon := c14n.MakeC14N11Canonicalizer()
doc := etree.Document{}
_, err = doc.ReadFrom(bytes.NewReader(bs))
if err != nil {
return "", err
}
out, err := canon.Canonicalize(doc.Element.ChildElements()[0])
if err != nil {
return "", err
}
// Generate a SHA-1 encoding
hasher := sha1.New()
_, err = hasher.Write([]byte(out))
if err != nil {
return "", err
}
sha := hasher.Sum(nil)
// And then turn that into Base64
w := new(bytes.Buffer)
enc := base64.NewEncoder(base64.StdEncoding, w)
enc.Write(sha)
enc.Close()
// The string of this is the IRmark
b64sha := w.String()
// remove the "fake" schema
// bs, err = deleteBetween(out, "<Body", ">")
// if err != nil {
// return "", err
// }
// Add the IRmark
bs, err = placeBefore(bs, "\n <Sender>", `<IRmark Type="generic">`+b64sha+"</IRmark>")
if err != nil {
return "", err
}
// Fix up whitespace around Body
ret := " " + string(bs) + "\n"
return ret, err
}
CT600_INCLUSIVE_CANON:accounts/internal/ct600/govtalk/govtalk.go
I'm not going to say "that worked", but it got us a lot closer. We are now seeing realistic errors, rather than lots of random errors about the schemas not being defined; and the ones we are seeing are because the schemas aren't defined.I was then able to fix up the remaining issues in the HTML (iXBRL) files that I was submitting and everything worked. Until it stopped working.
As I came to write this up, and tried to tidy the code up, I found that it was no longer working, with similar (but different) errors to the ones I'd seen before, but that made no sense in the context.
3 error(s) reported:I abandoned it for a few hours, and then it started working again. I hate flaky systems. How will I ever know if it is working when it shouldn't, or failing when it should work?
Code Raised By Location Type Message
3001 Department business The submission of this document has failed due to departmental specific business logic in the Body tag.
0 ChRIS Accounts xbrl.core.xml.SchemaValidationError.cvc-complex-type24_a cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.xbrl.org/uk/cd/business/2009-09-01":NameProductionSoftware}'. One of '{"http://www.xbrl.org/2003/instance":item, "http://www.xbrl.org/2003/instance":tuple, "http://www.xbrl.org/2003/instance":context, "http://www.xbrl.org/2003/instance":unit, "http://www.xbrl.org/2003/linkbase":footnoteLink}' is expected.
0 ChRIS Computations xbrl.core.xml.SchemaValidationError.cvc-complex-type24_a cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.govtalk.gov.uk/uk/fr/tax/uk-hmrc-ct-comp/2013-10-14":CompanyIsAPartnerInAFirm}'. One of '{"http://www.xbrl.org/2003/instance":item, "http://www.xbrl.org/2003/instance":tuple, "http://www.xbrl.org/2003/instance":context, "http://www.xbrl.org/2003/instance":unit, "http://www.xbrl.org/2003/linkbase":footnoteLink}' is expected.
Hopefully it's just a one-off and it will work correctly in the future.
Conclusion
It turns out that our problems have been with the canonicalization library we have been using, and its only offering "exclusive" canonicalization, which removes all the schema definitions.It would seem we are now in good shape to try and submit something more like our actual tax return.
