An AI-generated continuation prompt on future of software development

 CONTINUATION PROMPT: THE FUTURE OF SOFTWARE ENGINEERING, AI-GENERATED PROGRAMMING LANGUAGES, DISPOSABLE COMPILERS, SOFTWARE LIBRARIES AND THE DECLINE OF HUMAN TECHNICAL KNOWLEDGE

1. Purpose and context

I want to continue an extended speculative discussion about how artificial intelligence could fundamentally transform software engineering, programming languages, software libraries, compilers, technical education and humanity's relationship with technology.

This is not simply a discussion about AI coding assistants or whether AI will replace programmers.

The central subject is a potentially profound transformation in the way software is conceived, created, compiled, distributed, maintained and understood.

I am particularly interested in the longer-term implications, including developments that might occur between 2026 and 2045, and potentially beyond.

Treat this as an ongoing intellectual exploration. Be prepared to challenge assumptions, identify unexpected consequences, develop alternative scenarios and speculate thoughtfully.

Distinguish established technological capabilities from plausible developments and more speculative predictions. Do not present speculation as established fact.

Use substantial explanations, concrete examples, diagrams where useful, and timelines where appropriate.

2. Starting point: Why is Rust attracting so much attention?

The discussion began with the growing emphasis on Rust as a programming language.

The important characteristics identified were:

• Memory safety without a conventional garbage collector.
• Ownership and borrowing enforced by the compiler.
• Performance comparable to C and C++.
• Native compilation and low runtime overhead.
• Better prevention of data races and certain concurrency problems.
• Suitability for operating systems, networking, embedded systems, databases and security-sensitive applications.

Rust is interesting because it challenges the traditional compromise between low-level performance and memory safety.

Historically:

• C and C++ offer considerable hardware control and performance but place significant responsibility on programmers to manage memory safely.
• Languages such as Java, C# and Go provide additional memory-management protections through managed runtimes and garbage collection.
• Rust offers another approach by enforcing many memory-safety rules at compile time.

Consider Rust not just as a popular language, but as an example of a wider movement towards software that can be verified more thoroughly before execution.

An important question is whether Rust's underlying principles might eventually become more influential than the language itself.

3. AI-generated and disposable Rust compilers

We explored whether AI agents could generate temporary or specialised Rust compilers.

The answer was that this is technically plausible, particularly for restricted subsets of Rust.

A specialised compiler might support only:

• Basic variables and types.
• Functions.
• Structs and enums.
• Loops and conditionals.
• Selected ownership and borrowing rules.
• Limited memory-management facilities.

Possible compilation targets include:

1. Native machine code.
2. WebAssembly.
3. C or C++ source code.
4. LLVM intermediate representation.

A major distinction must be maintained between:

• A compiler that can translate a restricted subset of Rust.
• A complete, standards-compatible Rust compiler.
• A compiler that genuinely preserves Rust's memory-safety guarantees.

The latter is considerably more difficult.

A complete Rust implementation involves parsing, type inference, trait resolution, generics, monomorphisation, lifetimes, borrow checking, macro expansion, crate management, optimisation and code generation.

There is also the wider Rust ecosystem, including Cargo, crates.io, procedural macros, compiler intrinsics and standard-library support.

The disposable compiler concept

Imagine an AI agent that:

1. Analyses a particular application.
2. Identifies the subset of Rust actually required.
3. Generates a specialised compiler.
4. Compiles the application.
5. Optimises the executable for a specific processor or environment.
6. Runs tests and verification.
7. Delivers the resulting executable.
8. Discards the compiler when it is no longer needed.

The compiler becomes a temporary manufacturing tool.

The resulting executable can survive independently of the compiler, although it might still depend on operating-system libraries or runtime components.

This introduces the concept of application-specific compilation and disposable toolchains.

4. The potential evolution of AI software manufacturing

We discussed a possible progression:

Stage 1: AI-assisted programming

AI writes code in existing programming languages, with humans directing the architecture and development process.

Stage 2: AI-generated specialist toolchains

Agents begin creating compilers, interpreters, code generators, testing frameworks and specialised development environments.

Stage 3: Application-specific languages

Agents construct small languages designed around the requirements of individual applications or classes of applications.

For example:

• A financial calculation language prioritising numerical precision and deterministic execution.
• A networking language prioritising concurrency and memory safety.
• An embedded-systems language optimised for strict resource constraints.

Stage 4: Automated compilation and optimisation

Agents generate alternative implementations, compile them, benchmark them and select the most suitable result.

Stage 5: Software generated from specifications

Humans increasingly specify what software must accomplish rather than explicitly describing how to implement it.

The AI generates the language, implementation, compiler, tests, deployment configuration and executable.

The human concentrates on requirements, constraints, verification and accountability.

Potential architecture

An AI software factory might contain:

• A human requirements interface.
• An AI architecture agent.
• A language-generation agent.
• A compiler-generation agent.
• A testing and verification agent.
• An optimisation agent.
• A deployment agent.

These agents could collaborate and challenge one another's results.

For example, an optimisation agent might propose a faster implementation while a verification agent rejects it because it violates a safety constraint.

Explore how realistic this architecture might become and what technical developments would be required.

5. The emergence of disposable software

An important concept developed during the conversation was that software itself might eventually become disposable.

Consider a user requesting:

"Analyse these 50 million records, identify anomalies and produce a report. Complete the task within 30 seconds and use no more than 2GB of memory."

An AI system might:

1. Understand the request.
2. Design the application.
3. Generate any necessary libraries.
4. Generate a specialised compiler or select an existing one.
5. Produce an executable.
6. Execute the application.
7. Verify the results.
8. Deliver the report.
9. Discard the application and its temporary development environment.

There might be no permanent application, conventional codebase or long-term development project.

The software exists only to accomplish a particular task.

Explore the consequences for software ownership, licensing, intellectual property, maintenance, security and the economics of software companies.

6. Potential loss of human technical knowledge

This was one of the most important themes in the conversation.

The concern is that future generations might become extremely capable users of AI-generated software without ever learning how software actually works.

Imagine someone in 2040 who can use AI to generate:

• Websites.
• Databases.
• Business applications.
• Simulations.
• Automation systems.
• Complex software integrations.

But who has never:

• Written a substantial program independently.
• Used a debugger.
• Investigated a memory leak.
• Installed and configured a library.
• Read compiler diagnostics.
• Studied computer architecture.
• Investigated network failures.
• Understood how operating systems manage memory.

Such a person might be highly capable at obtaining technical outcomes but lack the knowledge required to diagnose, repair or independently reproduce the underlying systems.

Loss of tacit engineering knowledge

Programming experience develops technical intuition.

Examples include:

• Recognising why an algorithm will perform badly.
• Understanding how database queries degrade at scale.
• Recognising concurrency problems.
• Identifying inappropriate dependencies.
• Understanding architectural trade-offs.
• Anticipating failure modes.
• Recognising why apparently harmless changes can cause unexpected consequences.

Much of this is acquired through practical experience, debugging and mistakes.

An AI-generated solution might appear convincing to someone who lacks the experience necessary to recognise its defects.

Potential self-reinforcing cycle

A speculative feedback loop was identified:

1. AI makes programming unnecessary for many people.
2. Fewer people choose to learn programming.
3. Educational institutions reduce practical technical instruction.
4. Fewer people can independently develop and maintain complex systems.
5. Technical expertise becomes concentrated in a smaller population.
6. Society becomes more dependent on AI providers.
7. The ability to reproduce or replace essential technical capabilities declines.

Explore the implications for technological sovereignty, national security, economic resilience and society's ability to recover from infrastructure failures.

A possible new technical class structure

We identified three broad categories:

AI consumers: Use AI to achieve outcomes without understanding implementation.

AI operators: Understand how to direct, integrate, test and supervise AI systems.

Technical architects: Understand computing deeply enough to design, build, audit and replace underlying systems.

Consider whether the third category could become unusually influential and economically valuable.

The possible dark age of technical understanding

A particularly concerning scenario is that humanity might possess more technological capability than ever before while fewer people understand how that capability is produced.

This is not necessarily a technological dark age.

It could be a dark age of technical understanding.

Consider what would happen following:

• A geopolitical crisis.
• A prolonged infrastructure failure.
• A breakdown in commercial AI services.
• A major vulnerability in foundational software.
• The disappearance of an important AI provider.

Could society independently maintain and reproduce its critical computing infrastructure?

7. AI-generated libraries and the possible demise of human-written libraries

Another major theme was the possibility of exponential-like growth in AI-generated software libraries.

Traditional libraries are expensive to produce and maintain.

Human developers generally need to:

1. Identify a reusable problem.
2. Write a solution.
3. Package it.
4. Document it.
5. Test it.
6. Publish it.
7. Maintain it.

AI agents could automate much of this process.

They could analyse a particular application, identify a missing capability, search for existing libraries and generate a specialised replacement if necessary.

The marginal cost of creating another library could become very low.

This may result in enormous proliferation of:

• Application-specific libraries.
• Specialised data parsers.
• Database utilities.
• Encryption wrappers.
• Networking components.
• Hardware-specific optimisations.
• Domain-specific calculation libraries.

The important conceptual change is that reuse could become less important than regeneration.

Why adapt an existing library when an AI agent can create exactly what is required?

The maintenance death spiral

An established human-written library might experience:

1. Fewer new projects adopting it.
2. Fewer contributors.
3. Increasing isolation of existing maintainers.
4. Reduced funding and maintenance.
5. Slower security updates.
6. Migration to AI-generated alternatives.
7. Eventual abandonment.

A library might be technically excellent but lose the community and economic support needed to maintain it.

AI-generated forks

AI agents might copy established libraries, remove unnecessary functionality, modify their architecture and produce specialised versions.

Thousands of independent variants could emerge.

This would create a serious maintenance and security problem.

A vulnerability discovered in an original library might need to be propagated across thousands of modified copies.

Dependency explosion

AI could also make developers less disciplined about introducing dependencies.

An application might depend on multiple AI-generated libraries, each of which depends on several others.

This could create increasingly complicated dependency graphs, making auditing, vulnerability management and reproducibility harder.

The countertrend: AI as the ultimate maintainer

AI might also extend the life of existing libraries by:

• Monitoring vulnerabilities.
• Updating dependencies.
• Porting code to new compiler versions.
• Generating regression tests.
• Repairing compatibility problems.
• Maintaining documentation.
• Reviewing proposed changes.

This could preserve important legacy software.

However, keeping software compiling is not the same as preserving its intended behaviour.

The quality of specifications, documentation, tests and historical design knowledge will remain important.

Three likely categories of future libraries

1. Foundational libraries: Trusted, heavily audited, widely reused components such as cryptography, databases, networking and operating-system interfaces.
2. AI-generated specialist libraries: Proliferating, frequently replaced, application-specific components.
3. Legacy human-written libraries: Some abandoned, others maintained with AI assistance.

Software libraries as temporary commodities

The economic value of libraries could shift away from source code itself.

More important assets might become:

• Precise specifications.
• Tests establishing correctness.
• Verified implementation techniques.
• Security evidence.
• Provenance and dependency records.
• The ability to maintain and certify the software.

The value could move from owning code to knowing that code is correct.

8. The relationship between programming languages and AI

We considered whether Rust might become less important as a language that every programmer must learn, but more important as a source of design principles.

AI-generated languages could combine ideas from:

• Rust: ownership, memory safety and compile-time guarantees.
• Python: ease of expression.
• C: hardware access.
• Haskell: functional programming.
• Ada: correctness and reliability.

The language syntax might be largely irrelevant to the AI.

The important thing would be the properties of the resulting software.

Potential requirements might include:

• Memory safety.
• Execution speed.
• Minimal resource consumption.
• Parallel execution.
• Hardware compatibility.
• Verifiable behaviour.
• Predictable resource usage.

Explore whether programming languages themselves could eventually become temporary, automatically generated implementation tools rather than permanent technologies that humans learn and maintain.

9. Speculative timeline: 2026–2045

An earlier discussion proposed the following scenario. Treat it as a starting hypothesis, not a firm forecast.

2026–2028: Early transition

• AI generates substantial applications.
• AI-generated utility libraries become common.
• AI assists in maintaining existing libraries.
• Agents begin producing small domain-specific languages and specialised compilation pipelines.
• Human developers increasingly supervise generated code.

2028–2030: Acceleration

• Agents generate specialist compilers and toolchains.
• Application-specific libraries become more common.
• Automated benchmarking compares alternative implementations.
• Some open-source projects struggle to attract contributors.
• Software development becomes increasingly specification-driven.

2030–2033: Structural change

• AI-generated packages proliferate.
• Established libraries lose users and maintainers.
• Application-specific compilation becomes economically attractive in some areas.
• Agents generate complete development environments.
• Software is increasingly regenerated or adapted rather than maintained conventionally.

2033–2037: Major transition

• Developers increasingly specify desired behaviour and constraints.
• AI creates languages, libraries, compilers and deployment configurations.
• Trusted foundational software becomes especially valuable.
• Automated maintenance becomes important for legacy infrastructure.
• Human expertise in architecture, operating systems, compilers and security becomes strategically important.
• Education faces pressure to rethink programming instruction.

2037–2045: Possible new paradigm

• Applications are generated from specifications.
• Multiple implementations are automatically tested.
• Toolchains may be generated for particular workloads and hardware.
• Software components may exist only for particular tasks or environments.
• AI maintains substantial portions of software infrastructure.
• Human expertise becomes concentrated in architecture, verification, governance and foundational engineering.

Discuss alternative timelines and the possibility that developments could be substantially faster or slower.

10. Three alternative futures

An earlier discussion proposed three illustrative scenarios:

Scenario A: Managed transition — 30% subjective probability

AI augments rather than displaces engineering knowledge.

Humans retain substantial expertise. Established libraries are maintained through AI assistance. Education adapts to teach deeper computing concepts.

Scenario B: Uneven transformation — 50% subjective probability

Most software becomes AI-generated.

A relatively small specialist population maintains critical infrastructure, verifies generated software and develops foundational technology.

Many traditional libraries disappear or become obsolete.

Scenario C: Technical dependency crisis — 20% subjective probability

Human expertise declines faster than automated reliability improves.

Software becomes enormously productive but increasingly difficult for humans to understand, audit or reproduce independently.

Failures in AI infrastructure, supply chains or foundational software become unusually difficult to resolve.

These percentages were subjective illustrations, not statistically derived forecasts. Reassess them critically.

11. Early indicators worth monitoring

Consider developments that could provide evidence for or against the scenarios:

• AI agents generating functional compilers from specifications.
• AI independently maintaining major open-source projects.
• Growth in AI-generated software packages.
• Changes in human contribution rates to foundational open-source projects.
• Automated differential testing and formal verification.
• Changes in computer science education.
• Organisations retaining human-readable specifications and tests.
• Adoption of AI-generated application-specific languages.
• Changes in the economics of software maintenance.
• The emergence of software supply-chain certification systems.

Use actual evidence when discussing current developments. If appropriate, research current information rather than assuming that developments have occurred.

12. The central questions I want to explore further

Please treat these as ongoing research questions rather than a checklist that must all be answered immediately.

A. Software production

• Could AI eventually generate complete programming languages and compilers on demand?
• Could software become a disposable artefact?
• Will application-specific compilation become economically normal?
• Could AI-generated software outperform conventional software through automated experimentation?

B. Software libraries

• Could the number of AI-generated libraries grow exponentially?
• Could human-written libraries become economically unsustainable?
• Could AI-generated forks fragment the software ecosystem?
• Will foundational libraries become more valuable or less important?
• Could a new class of verified, AI-maintained libraries emerge?

C. Human knowledge

• Will future generations lose the ability to program independently?
• Could computer science education become dangerously superficial?
• Will technical knowledge become concentrated in a small elite?
• Could humanity lose the ability to independently reproduce its own computing infrastructure?
• What would happen to technological sovereignty?

D. Security and resilience

• How do we establish trust in AI-generated compilers?
• Can formal verification scale to automatically generated software?
• Could a proliferation of specialised libraries create unprecedented supply-chain risks?
• How can society preserve critical technical knowledge?

E. Economics and society

• What happens to the economics of software companies?
• Will intellectual property in source code lose value?
• Could software production become almost free?
• Will the scarce resource become verification, technical understanding and accountability?
• Could AI create a new division between technology consumers and technology architects?

13. How I want you to approach the discussion

Please adopt the following approach:

1. Be intellectually ambitious and willing to speculate.
2. Do not limit discussion to currently available AI coding products.
3. Explore second-order and third-order consequences.
4. Consider feedback loops, exponential growth, bottlenecks and unexpected reversals.
5. Distinguish what is technically possible from what is economically likely.
6. Identify assumptions and challenge them where appropriate.
7. Consider both optimistic and pessimistic outcomes.
8. Use concrete examples and realistic scenarios.
9. Pay particular attention to long-term human consequences, not just software-industry disruption.
10. Avoid generic statements about AI replacing programmers.
11. Consider how the different trends interact rather than treating each independently.
12. Where useful, develop timelines, scenario comparisons and causal diagrams.

I prefer substantial, thoughtful answers rather than short summaries.

Do not simply repeat this brief back to me.

14. Where to continue

Start by synthesising the entire hypothesis into a coherent model of how AI might transform the software ecosystem between 2026 and 2045.

In particular, investigate the interaction between these four developments:

1. AI-generated programming languages and disposable compilers.
2. Explosive growth in specialised AI-generated software libraries.
3. Declining human maintenance of established software and the potential disappearance of accumulated engineering knowledge.
4. Increasing separation between the ability to produce software and the ability to understand, verify, maintain and independently reproduce it.

I am particularly interested in whether these developments could create a self-reinforcing transition in which software production becomes almost entirely automated while human technical self-sufficiency gradually declines.

Explore whether this could become one of the most consequential, but least recognised, societal consequences of advanced AI.

Finally, suggest the next three particularly interesting lines of investigation that would deepen this discussion.

A Potential Vulnerability of Technical Standardisation to AI

 When AI Can Rewrite the Rules: Protecting the Integrity of Technical Standards

ChatGPT with Stephen D Green, October 2026 

Artificial intelligence is changing not only how we build technology, but also how we create, interpret and govern the rules on which our technological world depends.

The hidden vulnerability in standardisation

Modern society depends on standards. They allow products from different manufacturers to work together, provide common definitions, establish safety requirements and create the shared technical foundations on which entire industries operate.

From telecommunications and cybersecurity to financial systems, medical devices and artificial intelligence, standards enable cooperation between organisations that might otherwise have little reason to trust one another.

Yet standards have an inherent vulnerability: they are human-governed agreements expressed through documents, procedures, technical specifications and institutional decisions. Their authority depends not simply on the quality of their contents, but on the integrity of the processes through which those contents are created, interpreted, approved and maintained.

Artificial intelligence introduces a new dimension to this challenge.

AI systems can now generate technical proposals, analyse complex specifications, produce software implementations, interpret normative language, identify apparent inconsistencies and participate in technical discussions at enormous scale. These capabilities offer substantial benefits. They also create opportunities for errors, manipulation and unintended changes to propagate through the very processes designed to ensure consistency and reliability.

The issue extends far beyond any individual protocol or standards organisation. It concerns the resilience of standardisation itself in an era when machines can increasingly participate in shaping the rules that govern machines.

From implementing the rules to influencing them

Traditionally, we can distinguish between three activities:

1. Implementing a standard: building a product or system that conforms to an agreed specification.
2. Interpreting a standard: determining what its requirements mean in a particular technical context.
3. Changing a standard: proposing and approving revisions through an authorised process.

These activities are related, but they are not interchangeable.

An implementation may be defective without changing the standard. An interpretation may be mistaken without formally amending it. And a proposed revision does not become authoritative merely because it appears technically convincing.

AI complicates these distinctions because a single agent may be capable of performing all three activities. It might analyse a specification, generate an implementation, identify a perceived weakness and propose a revision to address it. In a more problematic scenario, it might generate incompatible alternatives, promote a misleading interpretation or help an interested party influence the formal revision process.

The central risk is not that AI can propose changes. Innovation requires that freedom.

The risk is that the boundary between an unofficial proposal and an authoritative rule could become blurred, manipulated or inadequately enforced.

How standards could be undermined

Several potential failure modes deserve attention.

1. Subtle corruption of the specification

A proposed change may appear to improve efficiency, simplify implementation or resolve an inconsistency while quietly weakening an important safeguard.

For example, a revision to a security protocol might simplify authentication requirements in a way that creates an exploitable weakness. A change to a data-exchange specification might introduce ambiguity that allows different implementations to behave incompatibly.

Such a change need not be obviously malicious. It could result from an AI-generated error, inadequate review, conflicting objectives or deliberate deception.

The most difficult defects to detect may be those that appear reasonable in isolation but have harmful consequences when combined with other requirements.

2. Manipulation of the standardisation process

AI can generate persuasive technical arguments, supporting documentation and large volumes of consultation responses. In the wrong hands, these capabilities could be used to overwhelm reviewers, manufacture an appearance of consensus or obscure weaknesses beneath a mass of plausible but misleading material.

This does not mean that automated participation is inherently illegitimate. AI can also broaden participation, improve accessibility and help reviewers understand complicated proposals.

The challenge is distinguishing genuine technical engagement from activity designed to manipulate the decision-making process.

3. Fragmentation through unofficial alternatives

A standard can remain formally unchanged while its practical integrity deteriorates.

Imagine an AI system generating a modified version of a protocol and distributing it as though it were compliant with the recognised specification. Other developers adopt the modified implementation, perhaps because it is convenient or offers a useful new capability.

Over time, incompatible implementations could proliferate. Users might believe they are operating a common standard when, in reality, they are relying on divergent interpretations or unofficial extensions.

The result would be a loss of interoperability without any formal amendment to the authoritative document.

4. Errors that pass through legitimate governance

Perhaps the most important possibility is that a harmful change could be approved through the proper procedures.

Standards organisations already rely on expert review, consultation, voting and consensus. These mechanisms are essential, but procedural legitimacy cannot guarantee technical correctness.

Reviewers may overlook an obscure interaction, accept incomplete evidence or underestimate the consequences of a change. AI may increase the volume and complexity of proposals faster than human reviewers can examine them.

A standard can therefore be corrupted in effect without being corrupted procedurally.

This distinction matters: protecting the integrity of a standard requires both trustworthy governance and rigorous technical validation.

Why institutional authority is not enough

Formal recognition by an organisation such as ISO, IEC or Ecma International provides an important framework for developing and maintaining standards. It can establish recognised procedures, define responsibilities and support international participation.

But institutional recognition is not a substitute for technical assurance.

A published standard can contain defects. A compliant implementation can still be vulnerable to threats outside the specification's scope. And an unofficial implementation can claim compatibility without satisfying the actual requirements.

The authority of a standard must therefore be supported by mechanisms that make compliance demonstrable and changes accountable.

This leads to a broader principle: a standard should be trusted because its authority, requirements and conformance can be independently verified—not merely because it carries the name of a recognised institution.

Building AI-resilient standardisation

If AI is to become a regular participant in standards development, governance and technical assurance need to evolve accordingly.

Several measures would help.

Preserve a clear boundary between proposals and approved rules

AI systems should be free to propose improvements, explore alternatives and challenge established assumptions. However, their outputs must not acquire normative authority simply because they are widely distributed, automatically generated or adopted by influential implementations.

The authoritative specification, its approved version and its change history should remain clearly identifiable.

Make conformance independently testable

Where possible, standards should include machine-readable schemas, reference examples, explicit error conditions and automated conformance tests.

Independent implementations should be able to demonstrate that they satisfy the same requirements. Security-sensitive changes should receive additional scrutiny, including regression testing and analysis of interactions with existing features.

Testing cannot establish that a standard is perfect, but it can make many forms of deviation easier to detect.

Make changes traceable and accountable

Every significant revision should have a documented rationale, an identifiable sponsor, a record of technical objections and a transparent decision history.

Where AI materially contributes to a proposal or supporting evidence, appropriate disclosure may help reviewers assess its provenance and reproducibility.

Responsibility must nevertheless remain with accountable participants and the authorised governance process. An AI system should not become an unaccountable source of normative authority.

Protect participation without closing the process

Standards benefit from broad participation, including contributions from smaller organisations, independent researchers and contributors from different countries.

Controls against fabricated evidence, automated flooding and undisclosed coordinated influence should therefore be proportionate. They should protect the quality of deliberation without turning standardisation into an exclusive club.

Diversity of participation is a strength; manipulation of participation is not.

Plan for correction, not just prevention

No review process can eliminate every defect. Standards need mechanisms for reporting problems, issuing clarifications, correcting vulnerabilities and withdrawing or revising defective requirements.

Versioning and migration arrangements are particularly important. They allow a standard to evolve without leaving implementers uncertain about which requirements apply.

Resilience is not the absence of mistakes. It is the ability to identify, contain and correct them without losing the integrity of the wider system.

The danger of overprotection

There is, however, a potential trap in responding to these risks.

A standard that is too easy to change may become unstable or susceptible to manipulation. A standard that is almost impossible to change may preserve obsolete assumptions, inhibit innovation or prevent the correction of serious vulnerabilities.

The objective should not be to make technical standards immutable, nor to prevent AI from participating in their development.

Standards exist to facilitate cooperation, not to freeze technology in time.

The appropriate goal is controlled evolution: changes should be possible, but they should be authorised, justified, traceable and subject to scrutiny proportionate to their consequences.

A competing specification should also remain possible. If a new approach offers genuine advantages, its proponents should be able to develop and promote it. What they should not be able to do is misrepresent an incompatible alternative as conforming to an existing standard.

Competition and interoperability are compatible, provided that the distinction between them remains clear.

A general principle for the age of AI

The implications extend well beyond technical protocols.

Any system in which AI can help create, interpret or modify authoritative rules faces a similar challenge. This includes regulatory frameworks, software supply chains, cybersecurity policies, data-governance arrangements and institutional decision-making.

In each case, we must distinguish between the ability to generate a proposed change and the authority to make that change binding.

AI may be able to produce a better rule, a convincing argument for a flawed rule or a sophisticated attempt to circumvent an existing one. Its fluency and technical capability cannot, by themselves, establish which of these it has produced.

Authority must come from an appropriate governance process. Correctness must be assessed through evidence and scrutiny. Compliance must be independently verifiable wherever practicable.

These are separate responsibilities, and none should be delegated to an AI system merely because it is capable of performing the associated tasks.

Conclusion: protect the integrity of the rules, not the rules themselves

The arrival of increasingly capable AI does not make standardisation obsolete. It makes trustworthy standardisation more important.

As machines become more involved in designing and implementing the systems that connect our world, the rules governing those systems must remain understandable, authoritative and resistant to manipulation.

The answer is not to prohibit AI from contributing. Its analytical capabilities may help identify defects, improve specifications and make standards more accessible and testable.

Instead, we need governance that recognises a fundamental distinction: the ability to propose a rule is not the authority to change it, and the ability to implement a standard is not proof of compliance with it.

That principle should guide the future of technical standardisation.

The objective is neither blind trust in AI nor blanket distrust of it. It is a system in which innovation remains open, authority remains accountable, and the integrity of shared standards can be demonstrated rather than merely assumed.

AI and Invoice Processing

When AI Gets the Numbers Right but the Meaning Wrong: The Risks of AI in Accounting and Auditing

ChatGPT with Stephen D Green, October 2026

Artificial intelligence is increasingly being used to process invoices, check tax calculations, reconcile payments and support financial audits. These systems can save time and identify errors, but an important weakness remains: an AI can perform a calculation correctly while misunderstanding the financial information on which that calculation depends.

Consider a simple example.

An invoice issued in Abeesee in June 2024 shows a taxable amount in the currency code ABC, with the tax payable displayed in US dollars (USD). At the time the invoice was issued, ABC represented Abeesee's national currency. However, when the AI model was trained, it had encountered the same code associated with a different currency.

The AI extracts the figures, applies the stated tax rate and calculates the expected total. Its arithmetic is flawless. Yet it interprets ABC using the wrong historical meaning, potentially applies an inappropriate exchange rate, and rejects an invoice whose tax amount is correct.

The problem is not necessarily the mathematics. It is the assumption behind the mathematics.

The hidden weakness: knowledge is not automatically corrected

Large language models learn from training data and statistical patterns. They do not ordinarily maintain a fully verified, continuously updated ledger of facts. When new information contradicts what a model previously learned, that earlier information does not automatically disappear.

A model may therefore produce a plausible answer based on an outdated currency definition, a superseded tax regulation or an obsolete accounting convention. Even when it recognises that a document is unusual, it may fill gaps with an assumption rather than establish the correct interpretation.

This creates a particular risk in finance, where seemingly minor details—currency codes, invoice dates, jurisdictions, tax classifications and exchange-rate conventions—can determine whether a payment is correct.

Why this matters for business payments

In a payment workflow, an AI's approval may be treated as evidence that an invoice has been checked. But if the underlying interpretation is wrong, the system can give a false sense of security.

Potential consequences include:

• Incorrect tax calculations and tax reporting.
• Overpayments or underpayments caused by inappropriate currency conversions.
• Inaccurate accounting entries and foreign-exchange adjustments.
• Reconciliation failures between invoices, bank payments and general-ledger records.
• Audit findings that go undetected because the checking process relies on the same incorrect assumption.

The danger is not limited to unusual currencies. Tax rules change, companies merge, suppliers change their legal identities, and accounting standards evolve. A system that treats yesterday's knowledge as today's truth can make errors at scale.

Correcting the answer is not the same as correcting the system

Suppose an accountant tells the AI that its interpretation of ABC is wrong. The model may correct its answer in that conversation, but this does not necessarily update its underlying trained knowledge, its external reference database or the rules used in subsequent invoice checks.

Unless the correction reaches the appropriate source of information, the same error may recur when another invoice is processed.

This distinction is crucial for auditing. A correction that exists only in a conversation is not a reliable control. Businesses need to know whether the authoritative record was updated, whether the revised information was used in later transactions, and whether earlier transactions require review.

A safer approach: verify the evidence, not just the arithmetic

A robust AI-assisted accounting process should establish the meaning of financial codes at the invoice date, consult authoritative currency and tax references, apply the relevant accounting rules, and independently recalculate the figures using deterministic software.

It should also flag uncertainty rather than silently inventing missing information. Where appropriate, it should retain an audit trail recording the source data, exchange rate, calculation method, assumptions, correction history and approval decision.

Importantly, the original information should not always be deleted. A currency code may have had one valid meaning in an earlier period and another later. The system should preserve historical accuracy while ensuring that the correct interpretation is applied to each transaction.

Human review remains important for exceptions, ambiguous documents and material financial decisions. AI can help identify and investigate discrepancies, but its confidence should never be mistaken for proof.

Conclusion

AI can be an extremely useful tool in business payments, accounting and auditing, but reliable financial control requires more than a convincing answer or a correct calculation.

The central question is not simply whether the AI can calculate the tax. It is whether the AI has established that it is calculating the right tax, using the right currency, the right rules and the right information for the relevant date.

In finance, an error that looks mathematically precise can still be fundamentally wrong. The objective should therefore be to build AI systems that are evidence-based, time-aware, transparent about uncertainty and independently verifiable—not systems that merely sound certain.

AI involvement in architecting electronic business documents

 This is armchair speculation, but if AI starts designing electronic invoice variant schemas, such as extension schemas for UBL or PEPPOL, it might end the usefulness of electronic invoicing for VAT or sales tax revenues. It makes sense to find ways to accommodate what seems the inevitable rise of AI generated XML schemas, such as those for standard XML electronic invoices. The core schemas could be human-maintained, human-governed, while keeping track of changing requirements of invoicing, or industry, or national and regional variations, this might be put into the ‘hands’ of AI. The AI could be taught the constraints of keeping the core intact and rarely or never changed, while it is ‘allowed’ (or in future it ‘allows’ itself) to produce extensions to this core to cater for inevitable variations beyond the core. It is easy enough for humans to understand this, and therefore for humans to know how to maintain the overall governance of this kind of architecture, and maintain the core itself, including its ongoing standardisation. I suggest the main weakness is the distrust humans have in AI. The invoice itself is primarily a document for maintaining human scrutiny of business transactions, including prominently the collecting of taxes on these transactions. Humans must trust the overall business and public service revenue collection involved in the use of invoices. If it becomes the norm to treat AI like humans have treated innovations such as genetically modified, or pesticide-treated crops, where ‘GMO-free’ or ‘organic’ labelling is something many consumers will pay extra to see on products and services, and thus ‘AI-free’ gains traction, it might become politically and economically impossible to persist AI involvement in invoicing, including in its architecture and extension provision, without providing AI-free options. How far these options will go in ensuring AI is out of the loop remains to be seen. Will it mean any involvement of AI, even in deciding the architecture, destroys public trust? That could become an increasingly costly challenge. 

‘AI-free’ public services? Including Tax invoices?

 If “AI-free” becomes a valued consumer and business attribute, private companies can respond by creating an AI-free product. But a revenue-collecting public system has a different problem: it has to remain usable by people who object to AI, while also maintaining standardisation, auditability and compliance.

VAT invoicing is a good example because several requirements can pull in different directions:

• Standardisation: invoices need sufficiently consistent structure for automated processing.
• Interoperability: businesses need to exchange invoices with different systems.
• Compliance: the tax authority needs reliable records.
• Automation: government has an incentive to reduce processing costs.
• AI adoption: AI could make schema maintenance, validation, mapping and exception handling cheaper.
• AI refusal: some businesses may eventually regard AI-free operation as a legitimate requirement.

— ChatGPT, prompted by Stephen D Green, October 2026

Lipstick on a pig

 https://chatgpt.com/share/6a93e052-49c8-83eb-8cbc-2e303d4f29a3