Friday, January 30, 2026

Time to Start Generating the iXBRLs

Given that we now have something that works, it's time to start trying to generate "that" (or something like that) from the actual accounts data that we've downloaded and calculated from Google Sheets.

I am obviously going to do this by using an iXBRL abstraction and then use that to generate the iXBRL (as XML) from the data using an appropriate class. I'll need two such classes: one for the accounts and one for the computations.

To avoid having to deal with too many errors at once, I'm going to deal with the accounts first, and then come back to the computations; so while I do that, I'm going to go back to giving pathetic excuses for not submitting computations.

Starting at the beginning, we have to change submit to use an AccountsGenerator:
func Submit(conf *config.Config) error {
    utr := conf.Utr
    if utr == "" {
        utr = conf.Business.TaxNum
    }
    ctr := &govtalk.IRenvelope{Business: conf.Business, ReturnType: "new",
        Sender: "Company", // the type of business we are, I believe.  The schema limits it to a handful of options

        UTR:         utr,
        PeriodStart: "2025-01-01", PeriodEnd: "2025-12-31",
        Turnover: 100000.0, TradingProfits: 0, LossesBroughtForward: 0, TradingNetProfits: 0,
        CorporationTax: 0,

        AccountsGenerator: conf.AccountsGenerator(),
        // AccountsIXBRL:     "ct600/micro-accounts.xml",
        // ComputationIXBRL: "ct600/sample-ctcomp.xhtml",
        NoComputationsReason: "Not within charge to CT",
    }

CT600_IXBRL_GENERATION:accounts/internal/ct600/submission/submit.go

What is this AccountsGenerator? It's an interface (one of a pair) that I've introduced in gnucash/config (note that because of the code's history, there's also a ct600/config).
package config

import "github.com/gmmapowell/ignorance/accounts/internal/ct600/ixbrl"

type AccountsGenerator interface {
    Generate() *ixbrl.IXBRL
}

type ComputationsGenerator interface {
}

CT600_IXBRL_GENERATION:accounts/internal/gnucash/config/generators.go

And it's created in the config object:
package config

import "fmt"

type Configuration struct {
    APIKey       string
    OAuth        string
    Token        string
    RedirectPort int
    Spreadsheet  string
    Output       string

    Business Business
    Accounts []Account
    Verbs    []Verb
    VerbMap  map[string]*Verb
}

...
func (config *Configuration) AccountsGenerator() AccountsGenerator {
    return GnuCashAccountsIXBRLGenerator{config: config}
}

CT600_IXBRL_GENERATION:accounts/internal/gnucash/config/config.go

The actual implementation class is GnuCashAccountsIXBRLGenerator which can be found in config/accountsixbrl.go:
package config

import "github.com/gmmapowell/ignorance/accounts/internal/ct600/ixbrl"

type GnuCashAccountsIXBRLGenerator struct {
    config *Configuration
}

// Generate implements AccountsGenerator.
func (g GnuCashAccountsIXBRLGenerator) Generate() *ixbrl.IXBRL {
    return ixbrl.NewIXBRL()
}

CT600_IXBRL_GENERATION:accounts/internal/gnucash/config/accountsixbrl.go

But I don't imagine it will be long before I move that off into its own package.

At the moment, of course, it isn't doing very much; just returning a new IXBRL object, which is itself a new concept:
package ixbrl

import (
    "github.com/gmmapowell/ignorance/accounts/internal/ct600/xml"
    "github.com/unix-world/smartgoext/xml-utils/etree"
)

type IXBRL struct {
}

func (i *IXBRL) AsEtree() *etree.Element {
    return xml.ElementWithNesting("html")
}

func NewIXBRL() *IXBRL {
    return &IXBRL{}
}

CT600_IXBRL_GENERATION:accounts/internal/ct600/ixbrl/ixbrl.go

This class is going to (eventually) do a lot of the heavy lifting around iXBRL generation, but for now, it's just returning an empty <html> node.

Also, for reasons of project circularity, I had to move the Element* code out of govtalk and into its own package, xml. This makes the checkin quite a bit bigger that would otherwise be necessary.

Not surprisingly, it doesn't work all that well:
2 error(s) reported:
  Code Raised By Location Type Message
  3001 Department business The submission of this document has failed due to departmental specific business logic in the Body tag.
  4065 ChRIS /hd:GovTalkMessage[1]/hd:Body[1]/ct:IRenvelope[1]/ct:CompanyTaxReturn[1]/ct:AttachedFiles[1]/ct:XBRLsubmission[1]/ct:Accounts[1]/ct:Instance[1]/ct:InlineXBRLDocument[1]/ct:html[1] schema Invalid content found at element 'html'
submission failed: 2 error(s) reported

Generating an Outline

Regardless of the content of an iXBRL file, it will have some standard features: a set of schemas in the html element; a head; a body; a hidden ix:header element; and a series of pages. We are going to encode this structure into our IXBRL class.

Here's a start:
package ixbrl

import (
    "github.com/gmmapowell/ignorance/accounts/internal/ct600/xml"
    "github.com/unix-world/smartgoext/xml-utils/etree"
)

type IXBRL struct {
    title string
}

func (i *IXBRL) AsEtree() *etree.Element {
    metaContent := xml.ElementWithNesting("meta")
    metaContent.Attr = append(metaContent.Attr, etree.Attr{Key: "content", Value: "application/xhtml+xml; charset=UTF-8"}, etree.Attr{Key: "http-equiv", Value: "Content-Type"})
    title := xml.ElementWithText("title", i.title)
    head := xml.ElementWithNesting("head", metaContent, title)
    body := xml.ElementWithNesting("body", i.ixHeader())
    ret := xml.ElementWithNesting("html", head, body)
    ret.Attr = append(ret.Attr, etree.Attr{Key: "xmlns", Value: "http://www.w3.org/1999/xhtml"}, etree.Attr{Space: "xmlns", Key: "xsi", Value: "http://www.w3.org/2001/XMLSchema-instance"})

    ret.Attr = append(ret.Attr, etree.Attr{Space: "xmlns", Key: "ix", Value: "http://www.xbrl.org/2013/inlineXBRL"})
    ret.Attr = append(ret.Attr, etree.Attr{Space: "xmlns", Key: "ixt", Value: "http://www.xbrl.org/inlineXBRL/transformation/2010-04-20"})
    ret.Attr = append(ret.Attr, etree.Attr{Space: "xmlns", Key: "ixt2", Value: "http://www.xbrl.org/inlineXBRL/transformation/2011-07-31"})
    ret.Attr = append(ret.Attr, etree.Attr{Space: "xmlns", Key: "xbrli", Value: "http://www.xbrl.org/2003/instance"})
    ret.Attr = append(ret.Attr, etree.Attr{Space: "xmlns", Key: "xbrldi", Value: "http://xbrl.org/2006/xbrldi"})
    ret.Attr = append(ret.Attr, etree.Attr{Space: "xmlns", Key: "link", Value: "http://www.xbrl.org/2003/linkbase"})
    ret.Attr = append(ret.Attr, etree.Attr{Space: "xmlns", Key: "xlink", Value: "http://www.w3.org/1999/xlink"})
    ret.Attr = append(ret.Attr, etree.Attr{Space: "xmlns", Key: "iso4217", Value: "http://www.xbrl.org/2003/iso4217"})

    return ret
}

func (i *IXBRL) ixHeader() *etree.Element {
    ixhidden := xml.ElementWithNesting("ix:hidden")
    ixrefs := xml.ElementWithNesting("ix:references")
    ixresources := xml.ElementWithNesting("ix:resources")
    ixheader := xml.ElementWithNesting("ix:header", ixhidden, ixrefs, ixresources)
    ret := xml.ElementWithNesting("div", ixheader)
    ret.Attr = append(ret.Attr, etree.Attr{Key: "style", Value: "display: none"})
    return ret
}

func NewIXBRL() *IXBRL {
    return &IXBRL{}
}

CT600_IXBRL_GENERATE_OUTLINE:accounts/internal/ct600/ixbrl/ixbrl.go

I'm going to say it's an improvement, even though we now have three errors rather than just 2:
3 error(s) reported:
  Code Raised By Location Type Message
  3001 Department business The submission of this document has failed due to departmental specific business logic in the Body tag.
  0 ChRIS Accounts xbrl.core.xml.SchemaValidationError.cvc-complex-type24_b cvc-complex-type.2.4.b: The content of element 'ix:hidden' is not complete. One of '{"http://www.xbrl.org/2013/inlineXBRL":footnote, "http://www.xbrl.org/2013/inlineXBRL":fraction, "http://www.xbrl.org/2013/inlineXBRL":nonFraction, "http://www.xbrl.org/2013/inlineXBRL":nonNumeric, "http://www.xbrl.org/2013/inlineXBRL":tuple}' is expected.
  0 ChRIS Accounts xbrl.core.xml.SchemaValidationError.cvc-complex-type24_b cvc-complex-type.2.4.b: The content of element 'ix:references' is not complete. One of '{"http://www.xbrl.org/2003/linkbase":schemaRef, "http://www.xbrl.org/2003/linkbase":linkbaseRef}' is expected.
submission failed: 3 error(s) reported

Generating Accounts Structure

To make further progress, we need to start adding more abstractions and providing those from the Accounts Generator. You may have noticed that we already slipped a title member into the IXBRL struct. We'll now add a schema to that (which is the schema to be defined in ix:references) and add lists for custom schema definitions and hidden properties
type IXBRL struct {
    schema string
    title  string

    hidden  []*IXProp
    schemas []*ixSchema
}

type ixSchema struct {
    id, schema string
}

type IXProp struct {
    Type     int
    Context  string
    Name     string
    Format   string
    Decimals int
    Unit     string
    Text     string
}

const (
    NonNumeric = iota
)

func (i *IXBRL) AddSchema(id, schema string) {
    i.schemas = append(i.schemas, &ixSchema{id: id, schema: schema})
}

func (i *IXBRL) AddHidden(h *IXProp) {
    i.hidden = append(i.hidden, h)
}

CT600_GENERATE_ACCOUNTS_STRUCTURE:accounts/internal/ct600/ixbrl/ixbrl.go

We can now make sure to include those schemas along with the pre-defined ones:
func (i *IXBRL) AsEtree() *etree.Element {
    metaContent := xml.ElementWithNesting("meta")
    metaContent.Attr = append(metaContent.Attr, etree.Attr{Key: "content", Value: "application/xhtml+xml; charset=UTF-8"}, etree.Attr{Key: "http-equiv", Value: "Content-Type"})
    title := xml.ElementWithText("title", i.title)
    head := xml.ElementWithNesting("head", metaContent, title)
    body := xml.ElementWithNesting("body", i.ixHeader())
    ret := xml.ElementWithNesting("html", head, body)
    ret.Attr = append(ret.Attr, etree.Attr{Key: "xmlns", Value: "http://www.w3.org/1999/xhtml"}, etree.Attr{Space: "xmlns", Key: "xsi", Value: "http://www.w3.org/2001/XMLSchema-instance"})

    ret.Attr = append(ret.Attr, etree.Attr{Space: "xmlns", Key: "ix", Value: "http://www.xbrl.org/2013/inlineXBRL"})
    ret.Attr = append(ret.Attr, etree.Attr{Space: "xmlns", Key: "ixt", Value: "http://www.xbrl.org/inlineXBRL/transformation/2010-04-20"})
    ret.Attr = append(ret.Attr, etree.Attr{Space: "xmlns", Key: "ixt2", Value: "http://www.xbrl.org/inlineXBRL/transformation/2011-07-31"})
    ret.Attr = append(ret.Attr, etree.Attr{Space: "xmlns", Key: "xbrli", Value: "http://www.xbrl.org/2003/instance"})
    ret.Attr = append(ret.Attr, etree.Attr{Space: "xmlns", Key: "xbrldi", Value: "http://xbrl.org/2006/xbrldi"})
    ret.Attr = append(ret.Attr, etree.Attr{Space: "xmlns", Key: "link", Value: "http://www.xbrl.org/2003/linkbase"})
    ret.Attr = append(ret.Attr, etree.Attr{Space: "xmlns", Key: "xlink", Value: "http://www.w3.org/1999/xlink"})
    ret.Attr = append(ret.Attr, etree.Attr{Space: "xmlns", Key: "iso4217", Value: "http://www.xbrl.org/2003/iso4217"})

    for _, s := range i.schemas {
        ret.Attr = append(ret.Attr, etree.Attr{Space: "xmlns", Key: s.id, Value: s.schema})
    }

    return ret
}

CT600_GENERATE_ACCOUNTS_STRUCTURE:accounts/internal/ct600/ixbrl/ixbrl.go

Then in the ix:header, we can add the hidden elements:
func (i *IXBRL) ixHeader() *etree.Element {
    ixhidden := xml.ElementWithNesting("ix:hidden")
    for _, ixp := range i.hidden {
        ixhidden.AddChild(ixp.AsEtree())
    }
And then the ix:references section should include the schema link:
    schemaLink := xml.ElementWithNesting("link:schemaRef")
    schemaLink.Attr = append(schemaLink.Attr, etree.Attr{Space: "xlink", Key: "href", Value: i.schema})
    schemaLink.Attr = append(schemaLink.Attr, etree.Attr{Space: "xlink", Key: "type", Value: "simple"})
    ixrefs := xml.ElementWithNesting("ix:references", schemaLink)
    ixresources := xml.ElementWithNesting("ix:resources")
    ixheader := xml.ElementWithNesting("ix:header", ixhidden, ixrefs, ixresources)

CT600_GENERATE_ACCOUNTS_STRUCTURE:accounts/internal/ct600/ixbrl/ixbrl.go

Then we need to go to the AccountsGenerator and make it generate the appropriate elements:
// Generate implements AccountsGenerator.
func (g *GnuCashAccountsIXBRLGenerator) Generate() *ixbrl.IXBRL {
    ret := ixbrl.NewIXBRL(g.config.Business.Name+" - Financial Statements", "https://xbrl.frc.org.uk/FRS-102/2025-01-01/FRS-102-2025-01-01.xsd")
    ret.AddSchema("bus", "http://xbrl.frc.org.uk/cd/2025-01-01/business")
    ret.AddHidden(&ixbrl.IXProp{Type: ixbrl.NonNumeric, Context: "CY", Name: "bus:NameProductionSoftware", Text: "Ziniki HMRC"})
    return ret
}

CT600_GENERATE_ACCOUNTS_STRUCTURE:accounts/internal/gnucash/config/accountsixbrl.go

And then we can try submitting again:
2 error(s) reported:
  Code Raised By Location Type Message
  3001 Department business The submission of this document has failed due to departmental specific business logic in the Body tag.
  0 ChRIS Accounts xbrl.ixbrl.UnknownContext Cannot find context definition with id 'CY' on element '{http://xbrl.frc.org.uk/cd/2025-01-01/business}NameProductionSoftware'.
submission failed: 2 error(s) reported
So, yes, we have referencede the context CY (for current year) but we haven't defined it. Let's do that.

Defining Contexts

Contexts are by far the most confusing aspects of this whole iXBRL thing (in my opinion, at least). I have gained some insight by reducing the computations iXBRL file to a minimal skeleton. In spite of the fact that the computations only cover one year, I still need four contexts. There appear to be two reasons for this: one is that there are things that want an "instant" (or at least a single date) when a fact is true, while others want a date range (such as a financial year); and the other is that there are multiple "dimensions" which are needed for a given fact, and the context must include those dimensions explicitly.

I also notice from the micro-accounts.xml file which I'm currently trying to recreate computationally, that there is an xbrli:entity element with an xbrli:identifier and an optional xbrli:segment with an explicitMember. Hopefully, we'll discover why these exist as we go through the process.

But for now, let's just define the context that we need:
// Generate implements AccountsGenerator.
func (g *GnuCashAccountsIXBRLGenerator) Generate() *ixbrl.IXBRL {
    ret := ixbrl.NewIXBRL(g.config.Business.Name+" - Financial Statements", "https://xbrl.frc.org.uk/FRS-102/2025-01-01/FRS-102-2025-01-01.xsd")
    ret.AddSchema("bus", "http://xbrl.frc.org.uk/cd/2025-01-01/business")
    ret.AddContext(&ixbrl.Context{ID: "CY", IdentifierScheme: "http://www.companieshouse.gov.uk/", Identifier: g.config.Business.ID, FromDate: ixbrl.NewDate(g.config.Ranges["Curr"].Start), ToDate: ixbrl.NewDate(g.config.Ranges["Curr"].End)})
    ret.AddHidden(&ixbrl.IXProp{Type: ixbrl.NonNumeric, Context: "CY", Name: "bus:NameProductionSoftware", Text: "Ziniki HMRC"})
    return ret
}

CT600_FIRST_CONTEXT:accounts/internal/gnucash/config/accountsixbrl.go

Which obviously depends on the existence of a context object:
type IXBRL struct {
    schema string
    title  string

    hidden   []*IXProp
    schemas  []*ixSchema
    contexts []*Context
}
...
type Context struct {
    ID               string
    IdentifierScheme string
    Identifier       string
    FromDate         Date
    ToDate           Date
}
...
func (i *IXBRL) AddContext(c *Context) {
    i.contexts = append(i.contexts, c)
}
...
func (cx *Context) AsEtree() *etree.Element {
    ret := xml.ElementWithNesting("xbrli:context")
    ret.Attr = append(ret.Attr, etree.Attr{Key: "id", Value: cx.ID})
    entity := xml.ElementWithNesting("xbrli:entity")
    identifier := xml.ElementWithNesting("xbrli:identifier")
    identifier.Attr = append(identifier.Attr, etree.Attr{Key: "scheme", Value: cx.IdentifierScheme})
    entity.AddChild(identifier)
    ret.AddChild(entity)
    period := xml.ElementWithNesting("xbrli:period")
    sd := xml.ElementWithText("xbrli:startDate", cx.FromDate.isoDate)
    period.AddChild(sd)
    ed := xml.ElementWithText("xbrli:endDate", cx.ToDate.isoDate)
    period.AddChild(ed)
    ret.AddChild(period)
    return ret
}

CT600_FIRST_CONTEXT:accounts/internal/ct600/ixbrl/ixbrl.go

Most of the work here is obviously done in AsEtree, which converts the context created above into the appropriate XML.

Finally, we have previously hacked in the date ranges, but that is no longer acceptable, so they are now part of the configuration file:
type Configuration struct {
    APIKey       string
    OAuth        string
    Token        string
    RedirectPort int
    Spreadsheet  string
    Output       string

    Business Business
    Ranges   map[string]DateRange
    Accounts []Account
    Verbs    []Verb
    VerbMap  map[string]*Verb
}
...
type DateRange struct {
    Start string
    End   string
}

CT600_FIRST_CONTEXT:accounts/internal/gnucash/config/config.go

And then we can try submitting again:
16 error(s) reported:
  Code Raised By Location Type Message
  3001 Department business The submission of this document has failed due to departmental specific business logic in the Body tag.
  3312 ChRIS Accounts business Company Reference Number (bus:UKCompaniesHouseRegisteredNumber) is missing.
  3318 ChRIS Accounts business The period to which this Return's Accounts applies does not coincide with the effective from/to dates of the referenced Accounts Taxonomy. Please correct and re-submit.
  3312 ChRIS Accounts business Legal form of entity (bus:LegalFormEntity) is missing.
  3312 ChRIS Accounts business Accounts type, full or abbreviated (bus:AccountsType) is missing.
  3312 ChRIS Accounts business Balance Sheet Date of Approval (core:DateAuthorisationFinancialStatementsForIssue) is missing.
  3312 ChRIS Accounts business Description of principal activities (bus:DescriptionPrincipalActivities) is missing.
  3312 ChRIS Accounts business Accounts status, audited or unaudited (bus:AccountsStatusAuditedOrUnaudited) is missing.
  3312 ChRIS Accounts business Trading/non-trading indicator (bus:EntityTradingStatus) is missing.
  3312 ChRIS Accounts business Accounting standards applied (bus:AccountingStandardsApplied) is missing.
  3312 ChRIS Accounts business Period Start Date (bus:StartDateForPeriodCoveredByReport) is missing.
  3312 ChRIS Accounts business Balance Sheet Date (bus:BalanceSheetDate) is missing.
  3312 ChRIS Accounts business Period End Date (bus:EndDateForPeriodCoveredByReport) is missing.
  3312 ChRIS Accounts business Company Name (bus:EntityCurrentLegalOrRegisteredName) is missing.
  3312 ChRIS Accounts business Dormant/non-dormant indicator (bus:EntityDormantTruefalse) is missing.
  3312 ChRIS Accounts business Name of Director Approving Balance Sheet (core:DirectorSigningFinancialStatements) is missing.
submission failed: 16 error(s) reported
After months of struggling with errors that make no sense, I finally feel a sense of relief at the appearance of this list of errors. It makes sense! Yes, that is a fairly accurate summary of some (all?) of the things I still have to do.

Conclusion

We have done the outline of the submission. I think most of the next chunk of work is fairly boilerplate, so I'm going to do it off camera (although I'll still check in publicly). I'll probably return with the next episode when it's time to put the actual accounts and computations into the iXBRL.

Thursday, January 29, 2026

Now Let's Figure out the iXBRL


So now we have two accounts documents: one reliably succeeds; the other reliably fails. So, what (apart from the obvious) is the difference between them?

One of the things that I like about having two files, one of which works, is that you can keep modifying the file that works until it stops working. And that tells you exactly what the problem is. It's much more reliable than having something that doesn't work.

You may ask: does it matter? And part of the answer is "no, it doesn't". With a working file, that's all we need to generate something that looks like that. But I'm still in the dark with regard to a lot of this XBRL stuff, and we still don't have a working computations file, so I want to try and understand it.

A few minutes later ...

On second thoughts, maybe it doesn't matter. It would seem that the accounts generator the government has been using spits out accounts that are simply incompatible with the schema (maybe this is why they're getting rid of it?). But at least we have the microaccounts to work from, so let's see what we can do with the computations.

Computations

I have struggled to find a valid computations file, but I recently came across the government document CT_Online_XBRL_Technical_Pack_2.0.pdf, which includes a set of links at the end. Included amongst these is: CT 2015v1.0 – example iXBRL Simple test case showing mandatory items for v2.1 (=CT600v3). And this expands to CTv3-2015v1.xml, which claims to be a valid computations file. Let's put that to the test shall we?

The first time I run it through, I get the following errors:
2 error(s) reported:
  Code Raised By Location Type Message
  3001 Department business The submission of this document has failed due to departmental specific business logic in the Body tag.
  1607 ChRIS Computations Business The UTR on the Computation must match the UTR on the CT600 AND the 'date for the end of the period' in the Computation must match the 'Period covered to' date in the CT600
submission failed: 2 error(s) reported
Yeah, fair enough. I need to modify those details.

Sadly, when I do that, the number of errors increases:
3 error(s) reported:
  Code Raised By Location Type Message
  3001 Department business The submission of this document has failed due to departmental specific business logic in the Body tag.
  3320 ChRIS Computations business The period to which this Return's Computation applies does not coincide with the effective from/to dates of the referenced Computation Taxonomy. Please correct and re-submit.
  1607 ChRIS Computations Business The UTR on the Computation must match the UTR on the CT600 AND the 'date for the end of the period' in the Computation must match the 'Period covered to' date in the CT600
submission failed: 3 error(s) reported
OK, so probably the most important error here is 3320: I am using an out of date taxonomy. That isn't going to fly, given that I want to submit future accounts. Let's see if we can track down another example somewhere with the correct taxonomy.

Google was able to find this: taxonomies accepted by HMRC, which contains a link to CT2024-v1.0.0.zip, which contains another sample file iXBRL-sample-ct-comp-2024.xhtml. So let's try that.

Not surprisingly, we get the same errors back:
2 error(s) reported:
  Code Raised By Location Type Message
  3001 Department business The submission of this document has failed due to departmental specific business logic in the Body tag.
  1607 ChRIS Computations Business The UTR on the Computation must match the UTR on the CT600 AND the 'date for the end of the period' in the Computation must match the 'Period covered to' date in the CT600
submission failed: 2 error(s) reported
Let's try and fix that. This took a little longer than I was expecting, given that it had a number of dates in it that were "clearly" wrong, for periods such as 2024-01-03 to 2024-01-04. Presumably this was filled in from a spreadsheet that was automatically generated with monotonically increasing dates.

Anyway, I was able to fix all of that and voilĂ : I was able to submit a tax return with accounts and computations:
2026/01/29 08:44:23 <GovTalkMessage xmlns="http://www.govtalk.gov.uk/CM/envelope">
  <EnvelopeVersion>2.0</EnvelopeVersion>
  <Header>
    <MessageDetails>
      <Class>HMRC-CT-CT600</Class>
      <Qualifier>response</Qualifier>
      <Function>submit</Function>
      <TransactionID/>
      <CorrelationID>189B1F57FAAE4E57A54A5D1F3D14C690</CorrelationID>
      <ResponseEndPoint PollInterval="10">https://test-transaction-engine.tax.service.gov.uk/submission</ResponseEndPoint>
      <Transformation>XML</Transformation>
      <GatewayTimestamp>2026-01-29T08:44:23.599</GatewayTimestamp>
    </MessageDetails>
    <SenderDetails/>
  </Header>
  <GovTalkDetails>
    <Keys/>
  </GovTalkDetails>
  <Body>
    <SuccessResponse xmlns="http://www.inlandrevenue.gov.uk/SuccessResponse">
      <IRmarkReceipt>
        <dsig:Signature xmlns:dsig="http://www.w3.org/2000/09/xmldsig#">
          <dsig:SignedInfo>
            <dsig:CanonicalizationMethod Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315"/>
            <dsig:SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1"/>
            <dsig:Reference>
              <dsig:Transforms>
                <dsig:Transform Algorithm="http://www.w3.org/TR/1999/REC-xpath-19991116">
                  <dsig:XPath>(count(ancestor-or-self::node()|/gti:GovTalkMessage/gti:Body)=count(ancestor-or-self::node())) and (count(ancestor-or-self::node()|/gti:GovTalkMessage/gti:Body/[name()=&apos;IRenvelope&apos;]/[name()=&apos;IRheader&apos;]/*[name()=&apos;IRmark&apos;])!=count(ancestor-or-self::node()))</dsig:XPath>
                </dsig:Transform>
                <dsig:Transform Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315#WithComments"/>
              </dsig:Transforms>
              <dsig:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"/>
              <dsig:DigestValue>AIxIEmcusqrDPubQ3uJwrIs98zw=</dsig:DigestValue>
            </dsig:Reference>
          </dsig:SignedInfo>
          <dsig:SignatureValue>w8RMS+vhauwMoZNxNpeQCBIGmBX4ZrNLswRCH+5tNooB1zjB+gdon7xFgzNo6NYRqZhKBjcBQ3fD
8nvOM/gMlsnTzgwsAvxroF2F2ea4nNQdS7xH4ONQzX3YI2FNo6lsB9+Dg55LYrm5vPyVF4STIrKS
MbdN8aJRmbPonNCihVR52/eJ9G6GV9DAYPlKX+qcX3i2L6O6mmLCyfEw+i0+OHRpYZLaNg/23bva
ui4GuhU9LKv4m78lEGOyC6RIUEGpZk1HFSlPfhU98hkHJuPuHLVl/j7DJxhOtXbyTNMGLMQKtws7
zYWTRwRVdkHtLabysDGo++a3P6L1i5dc5d83/g==</dsig:SignatureValue>
          <dsig:KeyInfo>
            <dsig:X509Data>
              <dsig:X509Certificate>MIIG1DCCBbygAwIBAgIQAXlGdOaNS07zg7WUCv1YHDANBgkqhkiG9w0BAQsFADBZMQswCQYDVQQG
EwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMTMwMQYDVQQDEypEaWdpQ2VydCBHbG9iYWwgRzIg
VExTIFJTQSBTSEEyNTYgMjAyMCBDQTEwHhcNMjUwNzI1MDAwMDAwWhcNMjYwODI1MjM1OTU5WjBe
MQswCQYDVQQGEwJHQjEPMA0GA1UEBxMGTG9uZG9uMR8wHQYDVQQKExZITSBSZXZlbnVlIGFuZCBD
dXN0b21zMR0wGwYDVQQDExRpcmE3NDgzNy5obXJjLmdvdi51azCCASIwDQYJKoZIhvcNAQEBBQAD
ggEPADCCAQoCggEBAOsinZ11A32r4dVz3yX63ePFY7uabzWncwKR4ZeHDp50b9LsFS1/4+H50vsc
V73BD/WL7Dkq0x8tYaQKBzj3A9E5KQNLhtlPR/HbjmL91c8C/PVvEwoxWUOMMJSvt9ie0YVX8nKa
VJfRILfqh8RyfWgxqT/6o5VFHmo233Dmx1TKPqPNzTqsu+ereAcHk9jLy63GXBVf0UZ64i1/dSIm
47VixZogwHkxqgcAIRhI9hZNdSzpBAishe5tdYW2mDwGSjsT4s/XXAItztrfsQgsEim1y4QylYRK
bis1bemYknj/rOfShviF270NI1D05hvi6/p3QHwWmnkCRT+mGYa/nLcCAwEAAaOCA5EwggONMB8G
A1UdIwQYMBaAFHSFgMBmx9833s+9KTeqAx2+7c0XMB0GA1UdDgQWBBQ3M7FFraa3yHrDZrOtwvdZ
J0+WKjAfBgNVHREEGDAWghRpcmE3NDgzNy5obXJjLmdvdi51azA+BgNVHSAENzA1MDMGBmeBDAEC
AjApMCcGCCsGAQUFBwIBFhtodHRwOi8vd3d3LmRpZ2ljZXJ0LmNvbS9DUFMwDgYDVR0PAQH/BAQD
AgWgMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjCBnwYDVR0fBIGXMIGUMEigRqBEhkJo
dHRwOi8vY3JsMy5kaWdpY2VydC5jb20vRGlnaUNlcnRHbG9iYWxHMlRMU1JTQVNIQTI1NjIwMjBD
QTEtMS5jcmwwSKBGoESGQmh0dHA6Ly9jcmw0LmRpZ2ljZXJ0LmNvbS9EaWdpQ2VydEdsb2JhbEcy
VExTUlNBU0hBMjU2MjAyMENBMS0xLmNybDCBhwYIKwYBBQUHAQEEezB5MCQGCCsGAQUFBzABhhho
dHRwOi8vb2NzcC5kaWdpY2VydC5jb20wUQYIKwYBBQUHMAKGRWh0dHA6Ly9jYWNlcnRzLmRpZ2lj
ZXJ0LmNvbS9EaWdpQ2VydEdsb2JhbEcyVExTUlNBU0hBMjU2MjAyMENBMS0xLmNydDAMBgNVHRMB
Af8EAjAAMIIBfwYKKwYBBAHWeQIEAgSCAW8EggFrAWkAdQDXbX0Q0af1d8LH6V/XAL/5gskzWmXh
0LMBcxfAyMVpdwAAAZhCMS7MAAAEAwBGMEQCICXglyjFEQqtO4utp9X05bRcKqX2IRSbRARNQerW
M7ldAiBzNoFRirdcAF8AbSAvr2RS0IK4qlkq66eQAijyw5j8ggB3AMIxfldFGaNF7n843rKQQevH
wiFaIr9/1bWtdprZDlLNAAABmEIxLwoAAAQDAEgwRgIhAJ7N3qWEzN5sRjeiStCNiARFHFFfZE8P
vJqnSArOnWytAiEA6rBJlBVQedIu/rEKeybndcGSt75Emne634xyrH/dNA4AdwCUTkOH+uzB74Hz
GSQmqBhlAcfTXzgCAT9yZ31VNy4Z2AAAAZhCMS8eAAAEAwBIMEYCIQC+TF9LUDY8I+2IlguM3a9E
wtHuhYMop3/K/qTh0428fAIhALQcptxsZTQte36EU7ulrFu8SZejwtI7NHJ+KbM2z8ZoMA0GCSqG
SIb3DQEBCwUAA4IBAQBHFr3J/EH0xKZl6Lsn8HClBCnRvJKz2QzRfuVtIGxVenISLdS11h5gEhyG
T2mE5gvlz9Eop9GkFNwGa7AOMq4hUaDZrBhEFUxqPHqrQDzhlnQdnLSNlynw3I4LA/CaUQH4yy06
9TkHJf4tGx2n/uMnYljLt0Ipn8qXRUu2U/M1WQnlPLzeudNpinI5MIC/0yVofvpRWFh5k/YK+cbz
GE8nPDvlpxakYT3hVrCLGHTQwPlKKzb+i2B2UPtcD4JdY8Za4KRDcl4UH3FDEaEw7IyIsCTU3NA/
C0n9Y/lRZpWoFF9CSYy0lS+3UhJK+HNIXIKNnfGC1rqxGhs+9Wim/UVB</dsig:X509Certificate>
            </dsig:X509Data>
          </dsig:KeyInfo>
        </dsig:Signature>
        <Message code="0000">HMRC has received the HMRC-CT-CT600 document ref: 8596148860 at 08.44 on 29/01/2026. The associated IRmark was: ACGEQETHF2ZKVQZ643IN5YTQVSFT34Z4. We strongly recommend that you keep this receipt electronically, and we advise that you also keep your submission electronically for your records. They are evidence of the information that you submitted to HMRC.</Message>
      </IRmarkReceipt>
      <Message code="077001">Thank you for your submission</Message>
      <AcceptedTime>2026-01-29T08:44:04.091</AcceptedTime>
    </SuccessResponse>
  </Body>
</GovTalkMessage>

Conclusion

I have finally found iXBRL files that work for submitting as part of my tax return. Moving on!

Tuesday, January 27, 2026

Figuring out the Errors

I'm at the point where I just can't take the errors any longer, and I need to understand once and for all what is going on.

It occurs to me that I could simplify things and go back to the set of micro-accounts that I generated on microaccounts.uk and not submit any computations. Then we can see whether that works more consistently.
func Submit(conf *config.Config) error {
    utr := conf.Utr
    if utr == "" {
        utr = conf.Business.TaxNum
    }
    ctr := &govtalk.IRenvelope{Business: conf.Business, ReturnType: "new",
        Sender: "Company", // the type of business we are, I believe.  The schema limits it to a handful of options

        UTR:         utr,
        PeriodStart: "2025-01-01", PeriodEnd: "2025-12-31",
        Turnover: 100000.0, TradingProfits: 0, LossesBroughtForward: 0, TradingNetProfits: 0,
        CorporationTax: 0,

        AccountsIXBRL: "ct600/11110000_accounts.html",
        // AccountsIXBRL:    "ct600/accounts-section.xml",
        // ComputationIXBRL: "ct600/comps-section.xml",
        // ComputationIXBRL: "ct600/ixbrl-sample-2.xml",
        NoComputationsReason: "Not within charge to CT",
    }
    submitOptions := &govtalk.EnvelopeOptions{Qualifier: "request", Function: "submit", IncludeSender: true, IncludeKeys: true, IncludeBody: true, IRenvelope: ctr}
    send, err := govtalk.Generate("submit.xml", false, conf, submitOptions)
    if err != nil {
        return err
    }

    msg, err := transmit(send)

    if err != nil {
        return err
    }

    decoder := xml.NewDecoder(bytes.NewReader(msg))
    var data string
    var waitFor time.Duration = 10
    pollOn := config.MakeBlankConfig()
    for tok, err := decoder.Token(); err == nil; tok, err = decoder.Token() {
        switch tok := tok.(type) {
        case xml.StartElement:
            switch tok.Name.Local {
            case "ResponseEndPoint":
                for _, a := range tok.Attr {
                    if a.Name.Local == "PollInterval" {
                        log.Printf("ResponseEndPoint PollInterval: %s\n", a.Value)
                        tmp, err := strconv.Atoi(a.Value)
                        if err != nil {
                            log.Printf("failed to parse number %s\n", a.Value)
                        } else {
                            waitFor = time.Duration(tmp)
                        }
                    }
                }
            }
        case xml.EndElement:
            switch tok.Name.Local {
            case "Function":
                log.Printf("Function: %s\n", data)
            case "Qualifier":
                log.Printf("Qualifier: %s\n", data)
            case "CorrelationID":
                log.Printf("CorrelationID: %s\n", data)
                pollOn.CorrelationID = data
            case "ResponseEndPoint":
                log.Printf("ResponseEndPoint: %s\n", data)
                pollOn.PollURI = data
            }
        case xml.CharData:
            data = string(tok)
        }
    }

    time.Sleep(waitFor * time.Second)
    Poll(pollOn)

    return nil
}

CT600_SUBMIT_MICRO:accounts/internal/ct600/submission/submit.go

And when I do this, I get a different result again:
​​2026/01/21 19:25:24 Qualifier: acknowledgement
2026/01/21 19:25:24 Function: submit
2026/01/21 19:25:24 CorrelationID: 886AE85484A14569B43358844F8EF515
2026/01/21 19:25:24 ResponseEndPoint PollInterval: 10
2026/01/21 19:25:24 ResponseEndPoint: https://test-transaction-engine.tax.service.gov.uk/poll
2026/01/21 19:25:34 <?xml version='1.0' encoding='UTF-8'?><GovTalkMessage xmlns="http://www.govtalk.gov.uk/CM/envelope"><EnvelopeVersion>2.0</EnvelopeVersion><Header><MessageDetails><Class>HMRC-CT-CT600</Class><Qualifier>response</Qualifier><Function>submit</Function><TransactionID></TransactionID><CorrelationID>886AE85484A14569B43358844F8EF515</CorrelationID><ResponseEndPoint PollInterval="10">https://test-transaction-engine.tax.service.gov.uk/submission</ResponseEndPoint><Transformation>XML</Transformation><GatewayTimestamp>2026-01-21T19:25:34.815</GatewayTimestamp></MessageDetails><SenderDetails/></Header><GovTalkDetails><Keys></Keys></GovTalkDetails><Body><SuccessResponse xmlns="http://www.inlandrevenue.gov.uk/SuccessResponse"><IRmarkReceipt><dsig:Signature xmlns:dsig="http://www.w3.org/2000/09/xmldsig#"><dsig:SignedInfo><dsig:CanonicalizationMethod Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315"></dsig:CanonicalizationMethod><dsig:SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1"></dsig:SignatureMethod><dsig:Reference><dsig:Transforms><dsig:Transform Algorithm="http://www.w3.org/TR/1999/REC-xpath-19991116"><dsig:XPath>(count(ancestor-or-self::node()|/gti:GovTalkMessage/gti:Body)=count(ancestor-or-self::node())) and (count(ancestor-or-self::node()|/gti:GovTalkMessage/gti:Body/[name()='IRenvelope']/[name()='IRheader']/*[name()='IRmark'])!=count(ancestor-or-self::node()))</dsig:XPath></dsig:Transform><dsig:Transform Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315#WithComments"></dsig:Transform></dsig:Transforms><dsig:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"></dsig:DigestMethod><dsig:DigestValue>MLuPohVdnhN+5rSvo3GdyT8PYFo=</dsig:DigestValue></dsig:Reference></dsig:SignedInfo><dsig:SignatureValue>w8RMS+vhauwMoZNxNpeQCBIGmBX4ZrNLswRCH+5tNooB1zjB+gdon7xFgzNo6NYRqZhKBjcBQ3fD
8nvOM/gMlsnTzgwsAvxroF2F2ea4nNQdS7xH4ONQzX3YI2FNo6lsB9+Dg55LYrm5vPyVF4STIrKS
MbdN8aJRmbPonNCihVR52/eJ9G6GV9DAYPlKX+qcX3i2L6O6mmLCyfEw+i0+OHRpYZLaNg/23bva
ui4GuhU9LKv4m78lEGOyC6RIUEGpZk1HFSlPfhU98hkHJuPuHLVl/j7DJxhOtXbyTNMGLMQKtws7
zYWTRwRVdkHtLabysDGo++a3P6L1i5dc5d83/g==</dsig:SignatureValue><dsig:KeyInfo><dsig:X509Data><dsig:X509Certificate>MIIG1DCCBbygAwIBAgIQAXlGdOaNS07zg7WUCv1YHDANBgkqhkiG9w0BAQsFADBZMQswCQYDVQQG
EwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMTMwMQYDVQQDEypEaWdpQ2VydCBHbG9iYWwgRzIg
VExTIFJTQSBTSEEyNTYgMjAyMCBDQTEwHhcNMjUwNzI1MDAwMDAwWhcNMjYwODI1MjM1OTU5WjBe
MQswCQYDVQQGEwJHQjEPMA0GA1UEBxMGTG9uZG9uMR8wHQYDVQQKExZITSBSZXZlbnVlIGFuZCBD
dXN0b21zMR0wGwYDVQQDExRpcmE3NDgzNy5obXJjLmdvdi51azCCASIwDQYJKoZIhvcNAQEBBQAD
ggEPADCCAQoCggEBAOsinZ11A32r4dVz3yX63ePFY7uabzWncwKR4ZeHDp50b9LsFS1/4+H50vsc
V73BD/WL7Dkq0x8tYaQKBzj3A9E5KQNLhtlPR/HbjmL91c8C/PVvEwoxWUOMMJSvt9ie0YVX8nKa
VJfRILfqh8RyfWgxqT/6o5VFHmo233Dmx1TKPqPNzTqsu+ereAcHk9jLy63GXBVf0UZ64i1/dSIm
47VixZogwHkxqgcAIRhI9hZNdSzpBAishe5tdYW2mDwGSjsT4s/XXAItztrfsQgsEim1y4QylYRK
bis1bemYknj/rOfShviF270NI1D05hvi6/p3QHwWmnkCRT+mGYa/nLcCAwEAAaOCA5EwggONMB8G
A1UdIwQYMBaAFHSFgMBmx9833s+9KTeqAx2+7c0XMB0GA1UdDgQWBBQ3M7FFraa3yHrDZrOtwvdZ
J0+WKjAfBgNVHREEGDAWghRpcmE3NDgzNy5obXJjLmdvdi51azA+BgNVHSAENzA1MDMGBmeBDAEC
AjApMCcGCCsGAQUFBwIBFhtodHRwOi8vd3d3LmRpZ2ljZXJ0LmNvbS9DUFMwDgYDVR0PAQH/BAQD
AgWgMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjCBnwYDVR0fBIGXMIGUMEigRqBEhkJo
dHRwOi8vY3JsMy5kaWdpY2VydC5jb20vRGlnaUNlcnRHbG9iYWxHMlRMU1JTQVNIQTI1NjIwMjBD
QTEtMS5jcmwwSKBGoESGQmh0dHA6Ly9jcmw0LmRpZ2ljZXJ0LmNvbS9EaWdpQ2VydEdsb2JhbEcy
VExTUlNBU0hBMjU2MjAyMENBMS0xLmNybDCBhwYIKwYBBQUHAQEEezB5MCQGCCsGAQUFBzABhhho
dHRwOi8vb2NzcC5kaWdpY2VydC5jb20wUQYIKwYBBQUHMAKGRWh0dHA6Ly9jYWNlcnRzLmRpZ2lj
ZXJ0LmNvbS9EaWdpQ2VydEdsb2JhbEcyVExTUlNBU0hBMjU2MjAyMENBMS0xLmNydDAMBgNVHRMB
Af8EAjAAMIIBfwYKKwYBBAHWeQIEAgSCAW8EggFrAWkAdQDXbX0Q0af1d8LH6V/XAL/5gskzWmXh
0LMBcxfAyMVpdwAAAZhCMS7MAAAEAwBGMEQCICXglyjFEQqtO4utp9X05bRcKqX2IRSbRARNQerW
M7ldAiBzNoFRirdcAF8AbSAvr2RS0IK4qlkq66eQAijyw5j8ggB3AMIxfldFGaNF7n843rKQQevH
wiFaIr9/1bWtdprZDlLNAAABmEIxLwoAAAQDAEgwRgIhAJ7N3qWEzN5sRjeiStCNiARFHFFfZE8P
vJqnSArOnWytAiEA6rBJlBVQedIu/rEKeybndcGSt75Emne634xyrH/dNA4AdwCUTkOH+uzB74Hz
GSQmqBhlAcfTXzgCAT9yZ31VNy4Z2AAAAZhCMS8eAAAEAwBIMEYCIQC+TF9LUDY8I+2IlguM3a9E
wtHuhYMop3/K/qTh0428fAIhALQcptxsZTQte36EU7ulrFu8SZejwtI7NHJ+KbM2z8ZoMA0GCSqG
SIb3DQEBCwUAA4IBAQBHFr3J/EH0xKZl6Lsn8HClBCnRvJKz2QzRfuVtIGxVenISLdS11h5gEhyG
T2mE5gvlz9Eop9GkFNwGa7AOMq4hUaDZrBhEFUxqPHqrQDzhlnQdnLSNlynw3I4LA/CaUQH4yy06
9TkHJf4tGx2n/uMnYljLt0Ipn8qXRUu2U/M1WQnlPLzeudNpinI5MIC/0yVofvpRWFh5k/YK+cbz
GE8nPDvlpxakYT3hVrCLGHTQwPlKKzb+i2B2UPtcD4JdY8Za4KRDcl4UH3FDEaEw7IyIsCTU3NA/
C0n9Y/lRZpWoFF9CSYy0lS+3UhJK+HNIXIKNnfGC1rqxGhs+9Wim/UVB</dsig:X509Certificate></dsig:X509Data></dsig:KeyInfo></dsig:Signature><Message code="0000">HMRC has received the HMRC-CT-CT600 document ref: 8596148860 at 19.25 on 21/01/2026. The associated IRmark was: GC5Y7IQVLWPBG7XGWSX2G4M5ZE7Q6YC2. We strongly recommend that you keep this receipt electronically, and we advise that you also keep your submission electronically for your records. They are evidence of the information that you submitted to HMRC.</Message></IRmarkReceipt><Message code="077001">Thank you for your submission</Message><AcceptedTime>2026-01-21T19:25:24.748</AcceptedTime></SuccessResponse></Body></GovTalkMessage>
And, suddenly, everything becomes clear: those "positive" results I've been seeing haven't been positive at all. They were just deferred negatives: that is, they were saying "I don't have an answer for you yet; come back in another 10s". An actual positive result has a completed Body element with a signature and messages.

OK, let's avoid running into this in future by handling the "deferred" error message.

This turns out to be a significantly bigger change than I had expected, because the etree library makes it much easier to parse the responses from the government's website. So that pushes me into refactoring the code that processes the response in submit.

So, for all the actual changes, this is really just a question of adding a for loop around the polling code and checking whether a message body came back:
    for {
        var waitFor time.Duration = 1
        pollOn := config.MakeBlankConfig()

        details := elt.FindElement("/GovTalkMessage/Header/MessageDetails")
        qualifier := details.FindElement("Qualifier")
        log.Printf("Qualifier: %s", qualifier.Text())
        function := details.FindElement("Function")
        log.Printf("Function: %s", function.Text())
        pollOn.CorrelationID = details.FindElement("CorrelationID").Text()
        log.Printf("CorrelationID: %s\n", pollOn.CorrelationID)
        rep := details.FindElement("ResponseEndPoint")
        if rep == nil {
            panic("not found")
        }
        a := rep.SelectAttr("PollInterval")
        tmp, err := strconv.Atoi(a.Value)
        if err != nil {
            log.Printf("failed to parse number %s\n", a.Value)
        } else {
            log.Printf("ResponseEndPoint PollInterval: %s\n", a.Value)
            waitFor = time.Duration(tmp)
        }
        pollOn.PollURI = rep.Text()
        log.Printf("ResponseEndPoint: %s\n", rep.Text())

        time.Sleep(waitFor * time.Second)
        elt, err = Poll(pollOn)

        if elt == nil || err != nil {
            return err
        }
    }

CT600_CHECK_RESPONSE_BODY:accounts/internal/ct600/submission/submit.go

Of course, now it "works" (read fails) every time, but I can work around that by hacking in a value of waitFor (say 1s) and check that it repeats (about 7 times at the moment, it would seem).

But, ah, that feels so good.

Conclusion

The apparent flakiness in the system wasn't "flakiness" at all ... it was just a factor of how loaded the system was and whether the estimate of 10s was long enough. Very good.

Wednesday, January 21, 2026

Refactoring to etree


Having included the etree package in order to use its canonicalization function, it makes sense to replace all of our hokey XML handling with etree.Element. This is the smallest refactoring I can do, keeping the same basic structure of functions in xml.go but changing the return types. These types then propagate through the rest of the code, along with any necessary modifications.
package govtalk

import (
    "log"
    "os"
    "reflect"

    "github.com/unix-world/smartgoext/xml-utils/etree"
)

func addElements(to *etree.Element, elts []etree.Token) {
    for _, e := range elts {
        if e != nil && !reflect.ValueOf(e).IsNil() {
            to.AddChild(e)
        }
    }
}

func ElementWithNesting(tag string, elts ...etree.Token) *etree.Element {
    env := etree.NewElement(tag)
    addElements(env, elts)
    return env
}

func ElementWithText(tag, value string, elts ...etree.Token) *etree.Element {
    env := etree.NewElement(tag)
    env.AddChild(env.CreateText(value))
    addElements(env, elts)
    return env
}

func ContentFromFile(filename string) *etree.Element {
    fp, err := os.Open(filename)
    if err != nil {
        log.Fatalf("Could not read %s: %v", filename, err)
    }
    defer fp.Close()
    doc := etree.NewDocument()
    _, err = doc.ReadFrom(fp)
    if err != nil {
        log.Fatalf("Could not read %s: %v", filename, err)
    }
    return doc.Element.ChildElements()[0]
}

func MakeGovTalkMessage(nesting ...etree.Token) *etree.Element {
    ret := etree.NewElement("GovTalkMessage")
    ret.Attr = append(ret.Attr, etree.Attr{Key: "xmlns", Value: "http://www.govtalk.gov.uk/CM/envelope"}, etree.Attr{Space: "xmlns", Key: "xsi", Value: "http://www.w3.org/2001/XMLSchema-instance"})
    addElements(ret, nesting)
    return ret
}

func MakeIRenvelopeMessage(nesting ...etree.Token) *etree.Element {
    ret := etree.NewElement("IRenvelope")
    ret.Attr = append(ret.Attr, etree.Attr{Key: "xmlns", Value: "http://www.govtalk.gov.uk/taxation/CT/5"}, etree.Attr{Space: "xmlns", Key: "xsi", Value: "http://www.w3.org/2001/XMLSchema-instance"})
    addElements(ret, nesting)
    return ret
}

func MakeCompanyTaxReturn(ty string, nested ...etree.Token) *etree.Element {
    ret := etree.NewElement("CompanyTaxReturn")
    ret.Attr = append(ret.Attr, etree.Attr{Key: "ReturnType", Value: ty})
    addElements(ret, nested)
    return ret
}

func Key(ty, value string) *etree.Element {
    ret := etree.NewElement("Key")
    ret.Attr = append(ret.Attr, etree.Attr{Key: "Type", Value: ty})
    ret.AddChild(ret.CreateText(value))
    return ret
}

CT600_REFACTOR_ETREE:accounts/internal/ct600/govtalk/xml.go

And when I run it again, it still works (although I can't guarantee that will continue forever).
2026/01/20 10:35:10 Qualifier: acknowledgement
2026/01/20 10:35:10 Function: submit
2026/01/20 10:35:10 CorrelationID: 73350F413E2546D4B84ECC988508A2CD
2026/01/20 10:35:10 ResponseEndPoint PollInterval: 10
2026/01/20 10:35:10 ResponseEndPoint: https://test-transaction-engine.tax.service.gov.uk/poll
2026/01/20 10:35:20 <?xml version="1.0" encoding="UTF-8"?>
<GovTalkMessage xmlns="http://www.govtalk.gov.uk/CM/envelope">
    <EnvelopeVersion>2.0</EnvelopeVersion>
    <Header>
        <MessageDetails>
            <Class>HMRC-CT-CT600</Class>
            <Qualifier>acknowledgement</Qualifier>
            <Function>submit</Function>
            <TransactionID></TransactionID>
            <CorrelationID>73350F413E2546D4B84ECC988508A2CD</CorrelationID>
            <ResponseEndPoint PollInterval="10">https://test-transaction-engine.tax.service.gov.uk/poll</ResponseEndPoint>
            <GatewayTimestamp>2026-01-20T10:35:20.157</GatewayTimestamp>
        </MessageDetails>
        <SenderDetails/>
    </Header>
    <GovTalkDetails>
        <Keys/>
    </GovTalkDetails>
    <Body/>
</GovTalkMessage>

Simplifying the Logic

In doing this refactoring, I had to duplicate the placeBefore method, because it ended up in two different packages. This is a code smell - on this occasion that submission/generate.go is in the wrong package and should be with govtalk.

But while I'm here, I'm not really happy with the way in which placeBefore is used. We start off with abstract objects, then we generate XML, then we format that and then we edit the formatted XML. Except we don't. We use the placeBefore method in three places: once to add a "blank" line where the <IRmark> is going to go; once to insert the <IRmark>, and then once to insert the body into the <GovTalkMessage> envelope. And none of this is really clear from the structure (and names) of Generate. I know this, because I keep struggling to find anything.

So, I'm going to move Generate from submission to govtalk to be with all the other like-minded things, and then I can share the placeBefore method. And then I'm going to simplify that by straightening out the canonicalizing of the body. And thus I end up with this:
func Generate(file string, runlint bool, conf *config.Config, options *EnvelopeOptions) (io.Reader, error) {
    gtxml, err := assembleGovTalkXML(conf, options)
    if err != nil {
        return nil, err
    }

    gtbs := writeXML(gtxml)

    if options.IncludeBody {
        bd := makeBody(options.IRenvelope)
        bs, err := canonicaliseBody(bd)
        if err != nil {
            return nil, err
        }
        body, err := insertIRmark(bs)
        if err != nil {
            return nil, err
        }
        gtbs, err = attachBodyTo(gtbs, body)
        if err != nil {
            return nil, err
        }
        gtbs = []byte(string(gtbs) + "\n")
    }

    if file != "" {
        err = checkAgainstSchema(file, runlint, gtbs)
        if err != nil {
            return nil, err
        }
    }

    return bytes.NewReader(gtbs), nil
}

CT600_REFACTOR_GENERATE:accounts/internal/ct600/govtalk/generate.go

And everything continues working in the same flip-floppy way.

Conclusion

I have managed to re-work the code to use the etree library which comes with the canonicalization library we are now using. This makes everything significantly simpler and clearer. I've taken the opportunity to tidy up some of the other code smells.

Monday, January 19, 2026

Submitting Real Accounts and Computations


So now, I have some fake accounts I generated from microaccounts.uk, and I have some real accounts, computations and a tax return that I submitted last year, which all appears to be ixbrl.

I can therefore break this up into pieces and include the relevant portions in the relevant spots in the submission, right?

Breaking it up was easy enough. I now have two files called accounts-section.xml and comps-section.xml (you can't see these because, again, it has private information in it). I can put these files in the relevant spots in the IRenvelope struct and then, hopefully, everything will work.

Sadly not. In our pursuit of trying to get something to work, we have only worked with the Accounts attachment, so now we need to allow both computations and accounts to be submitted, and, what's more, we need to submit them in that order (which seems backwards to me) in order to match the schema.
func (ire *IRenvelope) AsXML() any {
    if ire.NoAccountsReason == "" && ire.AccountsIXBRL == "" {
        log.Fatalf("Must give accounts or a reason not to")
    }
    if ire.NoAccountsReason != "" && ire.AccountsIXBRL != "" {
        log.Fatalf("Must EITHER give accounts OR a reason not to")
    }
    if ire.NoComputationsReason == "" && ire.ComputationIXBRL == "" {
        log.Fatalf("Must give computations or a reason not to")
    }
    if ire.NoComputationsReason != "" && ire.ComputationIXBRL != "" {
        log.Fatalf("Must EITHER give computations OR a reason not to")
    }
    keys := ElementWithNesting("Keys", Key("UTR", ire.UTR))
    pe := ElementWithText("PeriodEnd", ire.PeriodEnd)
    dc := ElementWithText("DefaultCurrency", "GBP")
    manifest := ElementWithNesting("Manifest",
        ElementWithNesting("Contains",
            ElementWithNesting("Reference",
                ElementWithText("Namespace", "http://www.govtalk.gov.uk/taxation/CT/5"),
                ElementWithText("SchemaVersion", "2022-v1.99"),
                ElementWithText("TopElementName", "CompanyTaxReturn"),
            )))
    irh := ElementWithNesting("IRheader", keys, pe, dc, manifest)
    irh.Elements = append(irh.Elements, ElementWithText("Sender", ire.Sender))
    ci := ElementWithNesting("CompanyInformation", companyInfo(ire.Business, ire.UTR, ire.PeriodStart, ire.PeriodEnd))
    summary := ElementWithNesting("ReturnInfoSummary", ire.accounts(), ire.computations())
    turnover := ElementWithNesting("Turnover", ElementWithText("Total", fmt.Sprintf("%.2f", ire.Turnover)))
    calc := ElementWithNesting("CompanyTaxCalculation", ire.taxCalc())
    too := ElementWithNesting("CalculationOfTaxOutstandingOrOverpaid", ire.cotoo())
    decl := ElementWithNesting("Declaration", decl())
    attachments := ire.figureAttachments()
    var attach any
    if attachments != nil {
        attach = ElementWithNesting("AttachedFiles", attachments)
    }
    ctr := MakeCompanyTaxReturn(ire.ReturnType, ci, summary, turnover, calc, too, decl, attach)
    return MakeIRenvelopeMessage(irh, ctr)
}
Here, we removed the XBRLsubmission element, because that is now part of the attachments element returned from figureAttachments:
func (ire *IRenvelope) figureAttachments() []any {
    ret := []any{}
    if ire.ComputationIXBRL != "" {
        cxml := ElementWithNesting("Computation", ElementWithNesting("Instance", ContentFromFile("InlineXBRLDocument", ire.ComputationIXBRL)))
        ret = append(ret, cxml)
    }
    if ire.AccountsIXBRL != "" {
        acxml := ElementWithNesting("Accounts", ElementWithNesting("Instance", ContentFromFile("InlineXBRLDocument", ire.AccountsIXBRL)))
        ret = append(ret, acxml)
    }
    return []any{ElementWithNesting("XBRLsubmission", ret)}
}

CT600_SUBMIT_IXBRL:accounts/internal/ct600/govtalk/irenvelope.go

This now creates two parallel attachments, one for the computations (which has the tag Computation for some reason) and one for the accounts.

And a little taffing later, that submits successfully (with the usual caveat that it flip-flops backwards and forwards between whether it does submit or not).

Conclusion

Inserting the appropriate iXBRL was relatively easy once you have the submission process sorted out.