Why we don't build blackbox automation

I regularly sit in initial meetings where managing directors show me an automated process that another provider built years ago, and almost the same sentence always comes up: no one here still knows why the system decides the way it does at this point. The provider is gone, the documentation was never written, and there simply is no escalation path for exceptions. That is exactly what I call blackbox automation, and that is exactly what we don't build at NordFlux.

What exactly is blackbox automation?

Blackbox automation is an automated process whose decision logic can no longer be traced after the fact, which has no defined escalation paths for exceptions, and whose rules are documented nowhere. All three characteristics don't have to occur at the same time, but in practice they almost always travel together, because they stem from the same cause: a system was built for a quick go-live, not for the operation that follows.

This is not a niche problem. 38 percent of companies in Germany cite the lack of traceability of AI results as one of the biggest obstacles to using artificial intelligence, according to the Bitkom study "Artificial Intelligence 2025". So if you ever thought your inexplicable workflow was an isolated case: it isn't.

Why do blackboxes arise in the first place?

Blackboxes rarely arise from bad intent, but from time pressure and the wrong incentives. A proof of concept moves into production without further refactoring, an AI agent is trained on a narrow goal without anyone logging how it arrives at its decisions, and documentation is the first thing to go when a project is running late. In the end, there is a system that works as long as no one asks why.

Internationally, caution toward such systems has long been the majority opinion, not the exception: 61 percent of people worldwide are, according to the KPMG study "Trust in Artificial Intelligence", reluctant to trust AI systems, a figure closely linked to a lack of traceability. Any provider that doesn't respond to this mistrust with transparency only reinforces it.

How do we work at NordFlux instead?

We build every automation so that it lives on without us. In concrete terms, that means: every workflow runs in the customer's infrastructure, whether that's their own n8n instance, their own Microsoft tenant, or their own UiPath environment, with access credentials that belong to the customer, not to us. Anyone who ends the collaboration with us retains full access to their own processes.

Every project includes a Process Design Document that describes the decision logic, the data sources used, and the boundaries of the system in plain language, not just in the workflow diagram. For one customer from the public sector, this very document withstood an internal audit by IT security, because every branch in the process, along with its justification, was documented in writing rather than having to be explained verbally.

What exactly does an escalation model mean?

An escalation model defines in advance the point beyond which an automation or an AI agent no longer makes a decision on its own, but hands it over to a human. For an invoice review, for example, that means: amounts below a defined threshold with a clear assignment go through automatically; everything else, unclear assignment, unusually high sum, new supplier, goes to a case worker along with a justification. Without this model, the system makes some decision or other when in doubt, and no one notices until it's too late.

This expectation is also increasingly anchored in regulation: the EU AI Act explicitly requires, for high-risk AI systems, in Article 13, that their operation be designed to be sufficiently transparent so that operators can interpret and appropriately use the results, including traceable documentation of capabilities and limitations. Anyone who already builds this way today won't need to retrofit tomorrow.

Does transparency cost more time?

Honestly: yes, a little, at the beginning. Writing a Process Design Document and defining an escalation model takes longer than simply getting started. This extra effort pays off as soon as an employee changes, an auditor asks questions, or a process needs to be expanded, because then there is a foundation to build on instead of starting from zero. Control is therefore not a marketing promise, but simply cheaper over the course of the project.

You can read more about how we work on AI agents in our service AI Agents, and about the accompanying governance consulting in our service AI Consulting.

Frequently Asked Questions

What distinguishes a NordFlux automation from a blackbox?

A NordFlux automation is documented, runs in the customer's infrastructure, and hands off edge cases to a human according to a defined escalation model. A blackbox meets none of these three criteria.

Who owns the workflows and access credentials after the project ends?

The customer. We build in the customer's own n8n, Microsoft, or UiPath environment, not in our own, so that the collaboration can end at any time without dependency on us.

What happens if an AI agent is uncertain about a decision?

It doesn't make the decision itself. The predefined escalation model hands unclear or borderline cases, along with a justification, to a responsible person instead of outputting a guess as the result.

Does more transparency automatically cost more money?

A bit more effort during setup, yes, because documentation and the escalation model are planned in from the start. Over the course of a project, this is cheaper than a blackbox that has to be re-explained with every staff change or every audit.