Engineering Knowledge Is Everywhere but Nowhere

What building AI tools for plants taught me about answers you can actually trust

A while ago I was working through sensor data from a production line, with a growing list of questions for the process engineer. Some were about the process. Others were more basic than that.

Some variables had names but no clear definitions. The documentation described the process one way, while the data seemed to suggest something slightly different. And one of the connections we needed for the analysis wasn't captured digitally at all.

None of this was anyone's fault. It is simply what engineering knowledge looks like after years of real operation: some of it in systems, some in PDFs, some on paper and a lot of it in the heads of a few people. The only way forward was to sit down with the engineer and ask.

I've seen versions of the same picture in almost every project since.

Where the knowledge actually lives

In one project, the product knowledge for a particular product range was spread across hundreds of PDFs, many repeating the same information in slightly different forms. At another site, material usage was still captured on handwritten forms.

In another project, we found that some of the most useful operational knowledge wasn't documented at all. People knew which settings worked, which issues had happened before and how they had been solved, but that knowledge lived mostly in conversations and experience. When someone left the team or simply wasn't available, that knowledge became much harder to access.

Industrial automation teams describe the same thing with different nouns: Jira tickets, Bugzilla records, spreadsheets, maintenance notes. The effect is identical. Engineers end up re-investigating problems someone has already solved, and the same two or three experienced people get every question.

The first time I watched a model invent something

In one of our process optimization projects, an LLM read a plant's operating procedure and extracted the constraints the optimiser had to respect: maximum temperatures, pressure limits and so on.

The first version produced constraints that looked perfect. They had sensible names and sensible limits. Then training failed with “No input columns found”.

The model had named the constraints the way a human would describe them. The sensor dataset used a completely different naming convention. Nothing matched, so every constraint was silently ignored.

It was a harmless failure because the pipeline stopped. What stayed with me is how convincing the output was. If the column names had happened to be close enough, we might not have noticed until much later.

The fix wasn't a smarter model. We gave it the real list of column names, told it explicitly not to invent any and filtered its output against that list afterwards. The model became useful once it was constrained to the evidence in front of it.

That is, in one small bug, the whole case for grounded assistants.

What "grounded" means in practice

A grounded assistant retrieves content from approved sources first and only then writes an answer from what it found, with a link back to the original. The technical name is retrieval-augmented generation, or RAG.

The idea is simple. The details are where the work is. A few decisions from one knowledge assistant we built that turned out to matter:

• Filtering before searching. Documents were filtered by country and region before retrieval, so a correct answer from the wrong jurisdiction didn't reach the user.
• Every answer keeps its evidence. We stored which chunks each answer was built from, and users could leave feedback on it. When someone said "this is wrong", we could see why.
• Being explicit about what happens when nothing is found. If the retrieved context doesn't contain the answer, any response based on general knowledge has to be clearly identified as such. Otherwise, the assistant has to say it couldn't find enough information.

That last rule was a deliberate trade-off for that particular use case. For engineering questions about firmware versions, terminal assignments or interlocks, I would be stricter: no source, no answer.

“I don't know" is a feature

The behaviour I've come to value most in these systems is not how well they answer. It is how clearly they say they can't.

When we used an LLM to digitise handwritten material usage forms, the rule was that an unreadable field returns “null”. The model never guesses. Some of the extracted rows were flagged for a person to check. That isn't a weakness of the tool. Those rows are exactly where a human should look.

In another project, anything the assistant found on the web was labelled researched, not confirmed, until an engineer approved it. A value it had to assume was marked [ASSUMED: … needs validation] directly in the draft procedure.

On a sales analytics project, the team had this rule: a number that cannot be trusted is dropped, not approximated.

An engineering knowledge assistant should behave the same way. "I couldn't find an approved answer for this controller" is a good response. A confident firmware version that doesn't exist is not.

The part nobody puts in the demo

Most of the effort in these projects has little to do with the model.

• Sources change. Datasheets get revised, firmware gets updated, equipment gets modified. In one knowledge base we built, the rule was to regenerate from the updated documents, not patch the answers by hand.
• Access is real work. In one project, getting authentication to the document libraries right, and pulling documents from the correct libraries for each region, took several iterations before we could trust what was being indexed.
• Models change underneath you. We once moved to a newer model version and reverted it the same day. Today, when we migrate models at INSUS, we check that faithfulness doesn't drop and that hallucinations remain within an acceptable threshold before going live.
• Adoption depends on the workflow. If the assistant sits outside the tools people already use, or the answer can't be verified in one click, people go back to messaging the colleague who knows.

This is what separates an AI experiment from something a plant can rely on.

What engineers should be able to ask

Once the sources are governed, the questions become very practical:

- "Have we seen this PLC communication fault on a similar machine?"

- "Which firmware version was approved for this controller?"

- "Has this alarm appeared on another line, and how was it resolved?"

- "Which document describes this interface?"

A good answer shows the ticket, the document version and the passage it relied on. A good assistant also says clearly when none of those exist.

One industrial automation team we spoke to had already tried an in-house AI tool. It produced hallucinated technical details and was abandoned. Their scepticism is completely reasonable, and in my view it is the right starting point.

How we approach it at INSUS

This is why we built ExpertFlow AI

It brings approved SOPs, manuals, troubleshooting guides, tickets and maintenance records into one governed knowledge layer. Engineers ask questions in natural language and get answers linked to the source, restricted to the documentation the organisation has approved. When the evidence isn't there, it says so, and those gaps become a list of what should be documented next.

We usually start small: one team, a few repositories and a clearly defined set of questions. We make that work inside the existing systems, legacy IT and OT included, and only then extend it to other teams and sites.

If your engineers are still finding answers by asking the one person who remembers, we'd be glad to talk about what your knowledge landscape looks like.

Explore ExpertFlow AI or request a plant demo to discuss your engineering knowledge environment.

From AI pilots to reliable industrial deployment, INSUS helps turn fragmented engineering knowledge into a trusted operational capability.