AI in Software Development: Challenges, Quality, and Evolving Work

AI-driven software development has become faster, but quality and control do not follow automatically. Before AI or humans write a single line of code, it must be clear which problem is being solved, what the requirements are, and in what context it is being applied. The next step is to develop in a way that ensures the software remains testable, maintainable, and capable of evolving over the years. This article explores AI in software development: What can a language model actually achieve, and which expectations regarding speed, data, and finished products fail to hold up in complex industrial processes?

The goal of this article: After reading, you will be able to identify critical patterns in everyday business life: in reviews, status meetings, demos, and production. You will know where AI fails when context, requirements, and quality are missing. And you will have a benchmark for what it means in practice to clarify the problem and requirements first, and then develop in a way that keeps the software responsible.

Contents:

1. Five patterns where AI fails in software development
2. How language models really work in software development
3. Assumptions that become expensive with AI in software development
4. AI in software development: Building the right thing the right way
   
5. The benefits of AI in software development and when they fail to materialize
6. Challenges with AI in software development
7. Conclusion

1. Five patterns where AI fails in software development

Building the right thing the right way rarely fails because of the tool, but rather because of five recurring patterns. Do these points sound familiar?

Lack of an AI tool standard: Whether it's Copilot, ChatGPT, or Cursor, some reject it while others have clear preferences. AI is here to stay in software development, but the results produced are often questionable. The most common problem: there is no common standard for how software is developed with AI within the company.

Vibe-coding app in two days: With vibe-coding, you can quickly develop an interface that feels finished and looks good. However, what is often neglected in vibe-coding is architecture, strategy, and long-term planning. How does the software fit into existing processes and systems? Where does the ownership lie?

The wrong feature, just faster: Something often gets lost between the business department and the developers along the way. Communication happens too infrequently, leading to development based on assumptions rather than requirements. This is often only noticed once it is finished. With AI, this becomes more pronounced: a third actor enters the mix, filling in gaps with the most probable outcome. Good requirements engineering becomes even more important to ensure AI models don't waste tokens solving the wrong problem.

Loss of control: The code has been redeveloped by the service provider, the tests are passing, and yet you can no longer verify quality, traceability, or maintainability without clean code standards. Suddenly, you find yourself having to invest time and money into acquiring knowledge and avoiding dependencies instead of focusing on your core business.

"Can't we just do that ourselves with AI?" In theory, the answer is yes; in practice, the question is whether you have enough expertise to turn isolated experiments into decisions for specific problems. AI is not a quick fix without people who understand how to use it, prompt it correctly, provide the necessary context, and verify the quality.

Most of the people we talk to recognize themselves in at least one of these five points or have told us about these problems themselves.

2. How language models really work in software development

An LLM fills every gap you leave open with what is most probable, not what is right for you. The gap does not appear as a gap; it is filled with a fluent, convincing guess.

Let's start with what a language model actually does: an AI model is a probability model that constantly makes assumptions without necessarily having verifiable evidence for them.

A Large Language Model (LLM) does not understand. It searches.

Token by token, it determines the most likely continuation. This is based on the context you provide, plus the prior knowledge from its training. If context is missing, it cannot accurately determine the result and falls back on assumptions. This is exactly where the problems discussed in this blog post originate.

"Thinking" modes do not change anything fundamental about this: they give the search more computing time, but they do not generate insight. Without a clear plan, goal, and structure, the model consumes (expensive) tokens just to iterate toward a solution instead of hitting the right one from the start.

‍

This is precisely what makes AI in software development both powerful and risky. Powerful, because manual coding now accounts for an even smaller portion of the work. Risky, because the model does not indicate ambiguity. It delivers code at high speed that reads as if someone had actually thought it through.

Therefore: Good input increases the probability of landing a correct result that fits your requirements, your architecture, and your goals.

This is more sustainable in terms of team resources because less rework is required. And it is cost-efficient because fewer tokens are wasted on incorrect iterations. Requirements, architecture, domain rules, existing codebase, testable acceptance criteria: whatever is not in the context and not in the training data will be guessed.

Kent Beck, co-founder of Extreme Programming, sums up the shift perfectly:

"90 % of my skills are now worth $0 … but the other 10 % are worth 1000x." (Kent Beck)

Those 10 percent are intelligent judgment, strong architecture, and meaningful context.

What this means in concrete terms for an industrial company: AI in software development doesn't just mean "we now have a chat window in our IDE." It means that the actual work, which was previously hidden in the writing process, becomes visible: identifying the right thing to build and ensuring it is developed correctly.

3. Assumptions that become expensive when using AI in software development

In conversations with CTOs, heads of development, and business departments, the same assumptions about what AI can do in software development keep coming up. Often, the capabilities of AI are overestimated.

A vibe-coding prototype is not a product.

In the manufacturing industry, software must fit into real-world processes, systems, and operations. AI can provide support for complex workflows, but it doesn't "just solve" the challenges that come with them.

The bottleneck is often an understanding of business processes: data and systems must be integrated, often across different locations and countries. Where that connection is missing, a human has to step in. If a process is already fragile without AI, simply adding AI on top rarely makes a decisive difference in complex enterprise workflows.

An AI will not clean up your data foundation on its own. It can help with structuring, but the responsibility for the data foundation and quality control remains with humans.

Developers are not becoming obsolete. The State of DevOps Report 2025 (an annual study on the delivery performance of software teams) shows that individuals are completing more tasks and merging more pull requests. The time saved is often redirected into review and approval.

One developer put it this way:

"I'm babysitting the AI and checking what it does."

The workload isn't decreasing; it's just shifting.

Many companies have a list of open issues and are checking what can be solved with AI. The model won't work through this list on its own. It requires people to categorize the application, verify the results, and provide context. Otherwise, the search remains statistical rather than accurate.

And one more distinction, because it becomes expensive in practice: AI in software development is not the same as AI in the product. The former is about how a team develops software. The latter is about what the product can do: image processing, forecasting, assistance. Both can be related, but you shouldn't conflate the decisions.

4. AI in software development: Building the right thing the right way

This is the principle we at generic.de have based our work on for years. With AI in software development, it becomes even more important, because a wrong result can now be developed much faster.

This phrase encompasses the two most important framework conditions in software development.

The right thing means: solving a problem that is worth solving.

The right way means: developing software that is maintainable, testable, and capable of being evolved over many years.

To deliver on this philosophy, we develop software using AI in phases, leveraging a comprehensive knowledge graph and running discovery and actual coding in parallel through Dual Track Agile. To ensure quality even at high speeds, we utilize loops and quality gates.

4.1 Where value shifts

A development team has never just written code. It clarifies requirements, decides on architecture, plans, develops, tests, reviews, and deploys software. This is precisely where the real value in the value chain lies, especially now that AI plays a central role in software development.

This makes it clear that requirements engineering and review are becoming even more important today. Before AI or humans write a single line of code, it must be clear what is to be created and in what context. It is about building the right thing. Review means ensuring that the code written by agents can be accounted for by humans. Was the right thing implemented in the right way?

Those who neglect requirements engineering and review usually pay three times over: in solutions no one needs, in technical debt, and in software that no one understands anymore.

The latter point has had a name since 2026: Comprehension Debt (Addy Osmani).

Classic technical debt means: The code is bad. Comprehension debt means: The code might even be clean, but no one understands it.

‍

4.2 The four phases of software development

To build the right thing the right way, we think of the software development process in four phases: Invent, Explore, Develop, and Maintain.

‍

DESIGN (1–2 weeks)

In the first project phase, business problems are identified and prioritized to generate solution ideas and formulate a vision.

The focus is on investing in solving the right problem, creating decision-making certainty, and providing direction for the team and stakeholders.

This step also includes the initial assessment (requirements engineering). The most important requirements and specifications are bundled and summarized.

‍

DISCOVER (2–8 weeks)

In the second project phase, the solution concept is translated into a knowledge graph. In this machine-readable knowledge base, documents are structured and linked so that the AI can understand dependencies and take them into account during software development.

The knowledge graph should contain requirements, decisions, product versions, customer statements, proposal content, and relevant files, all in the highest possible quality. The graph is expanded throughout all phases, much like a wiki: change requests, new customer requirements, new stakeholders, technology decisions, as well as the history of decisions and their rationales.

In parallel, the project is set up, including the AI development environment. For this, we establish an individual agent harness.

An agent harness is the environment surrounding the language model. The LLM provides only a small part of the value (a benchmark of ~10%); the harness does the rest (~90%) by making the model usable, controllable, secure, and repeatable. It comprises layers from the inside out: framework (instructions, tools, orchestration, guardrails), developer interface, cloud infrastructure, and cross-cutting concerns like memory, testing, observability, and deployment. At its core is the defined information flow: specified inputs are processed into specified outputs.

Only when this foundation is in place is a project "dev-ready." Neither human nor agent writes code before it is clear what is to be created and under what conditions.

‍

DEVELOP (2–6 months)

This is where the actual software development takes place. Using a dual-track agile process, discovery and delivery run in parallel. AI agents accelerate routines, code, and testing, while our experts design, plan, and verify. New product increments can be tested earlier to continuously enrich the concept with user feedback. This reduces product risk, ensures the generated code remains the client's property, and keeps it maintainable and scalable thanks to human and AI quality gates.

‍

DEVOPS (continuous)

Responsibility does not end with the go-live: operations, hosting, support, security updates, and ongoing development ensure that cleanly developed code does not turn into a product burdened by "comprehension debt," but rather one that remains viable for five or ten years to come.

‍

4.3 Context Engineering: Making requirements machine-readable

Requirements engineering and the development of a knowledge graph have always been part of software development. What is new is that with AI in software development, an additional actor requires the same context that previously only the human developer needed: the model.

In practice, context engineering ensures that requirements and other data from the knowledge graph are actually readable and available to the AI.

It is about selection, not completeness. Loading the entire graph into every prompt would not be a solution, but a new problem. Beyond a certain volume of context, the AI's output quality drops rapidly.

Context engineering therefore means specifically determining and providing the relevant excerpt from the graph for each task, rather than supplying as much context as possible at once.

An analogy makes it clear what happens when this step is missing. An architect implicitly knows that windows and doors belong in a house and that people need to walk through them. This knowledge is not in any blueprint; it is in their head. If the AI lacks this exact usage context, it might place a door a meter too high or build a birdhouse-sized door in a human house – statistically plausible, but practically useless.

‍

4.4 How do we keep the AI in check?

The topic of Agent Harness was already touched upon in the four phases of AI in software development. Here, we dive a little deeper. In the past, developers kept these validation loops in their heads. Today, what was once in their minds must be made available to the AI.

Within the Agent Harness framework, quality gates, loops, and guardrails ensure that the AI does not deviate from the defined path.

Guardrails are essentially the traffic rules that the AI model must follow. Quality gates are the checkpoints where intermediate results are verified.

Together, they prevent the AI's statistical mediocrity and probabilistic assumptions from being implemented unnoticed.

Loops are the journeys (repeated cycles). They continue until the task requirements and the results align. An agent cannot complete a task without validated proof.

The human remains in the loop. Once the loop is complete, human approval is required. Agents can plan, implement, suggest tests, and check against acceptance criteria. They must not merge. Every agent-generated artifact remains a draft until a human reviews it, takes responsibility, and approves it.

‍

4.5 Clean Code as a Guardrail

Stefan Lieser, co-founder of the Clean Code Developer Initiative, draws an analogy:

"Nobody looks at the machine code generated by the compiler. The same will happen with AI: nobody will look at the code anymore once AI turns requirements into executable software." (Stefan Lieser)

If that were to happen, Clean Code would be redundant. That is not the case (yet) for three reasons.

1. ‍Existing code is context. Source code is part of the knowledge graph. The existing graph is the strongest input for the next feature. Clean data produces clean code.
2. ‍Humans still need to be able to read the code. In case they need to intervene manually, and to prevent comprehension debt. The code might look clean, but no one may be able to explain why it is the way it is.
3. ‍AI is not a compiler. A compiler translates deterministically. A language model does not. It does not always produce the same result. That is why the output requires the same standards as human-written code.

Clean Code used to be a toolkit. Today, Clean Code serves as a guardrail for software development with AI.

‍

‍

Our responsibility:

As a service provider, we aim to develop code that is understandable and accountable. The client who owns the code must be the one to take responsibility for it.

You don't have to write every line yourself. You must be able to take responsibility for every line.

5. The benefits of AI in software development, and when they fail to materialize

The benefits of using AI in software development are real, but they depend heavily on how AI is deployed. They primarily emerge when the initial phase is executed with precision.

Early tangible results: AI delivers a prototype in no time, serving as a basis for decision-making. The business department can experience how a solution feels and refine it accordingly. This way, false assumptions are eliminated before they tie up budget. The benefit fails to materialize if no one can classify and evaluate the proposal, because a prototype is not a finished product that meets enterprise requirements.

Faster time-to-increment: By testing product increments faster, user feedback can be gathered promptly. This reduces the risk of developing for a long time only to realize late in the game that it was the wrong approach. In a dual-track process, routine tasks, implementation, and testing can be accelerated so that increments reach real users up to three times faster, provided the context and gates are in place.

More value within the same budget: Less time spent on code development does not mean fewer engineers. Instead, capacity can shift to where enterprise software differs from standard software: requirements, domain modeling, architecture, and reviews. The benefit fails to materialize if the time saved on coding is booked as headcount reduction rather than as an investment in requirements engineering, context engineering, and quality reviews.

Investment protection: Industrial software often outlives the project that created it—frequently by five to fifteen years. What remains adaptable over this time keeps the client independent and agile. The source code remains a verifiable part of the knowledge graph. Quality gates apply equally to humans and AI. This was the economic core even before AI. With AI, it becomes a functional prerequisite: without this safeguard, every rapid iteration generates comprehension debt faster than a team can pay it down.

Control instead of black box: Traceable results make quality visible throughout the loop: test, evidence gate, review, and human approval. Agents are allowed to plan, develop, and test. Ownership remains with the human. Speed is thus not achieved at the expense of control, but in tandem with it. The benefit fails to materialize where reviews become mere "click-through" exercises: in that case, the black box has simply become faster.

Concurrent documentation: Documentation has long been a bottleneck. It was an afterthought that was rarely completed. With a knowledge graph and agent harness, it is created concurrently and partially automatically: requirements, decisions, and architectural justifications are captured while development is happening, not after. The benefit fails to materialize where documentation is still viewed as rework.

Increasing value across projects: The knowledge graph is not an artifact that becomes obsolete when a project ends. Every decision, every requirement, and every architectural justification remains available as context for the next feature and the next initiative. Traditional documentation loses value over time because it becomes outdated. The graph gains value the longer and more consistently a company maintains it. Knowledge is no longer just locked away in people's heads; it is available for the next generation of AI. This advantage is lost if the graph is not maintained.

The key is to shift a shared understanding of the problem, machine-readable context, quality assurance with verification, human approval, and capacity and resources toward code development.

‍

6. Challenges with AI in software development

‍

‍

The antidotes are unspectacular and precisely for that reason effective: smaller change sets · explain before accept · test first · plan as an artifact in advance, make the plan explicit · ask for counterarguments (that is the loop) · context hygiene: new task, new context, open a new agent if necessary.

‍

7. Conclusion

What a development team does all day has never been primarily writing code. It clarifies requirements, decides on architecture, plans, develops, tests, reviews, and puts software into operation. AI accelerates one of these steps. The value shifts to the phases before and after: requirements engineering at the beginning, and testing and approval at the end.

An LLM fills the gap you leave open with what is most probable, not what is right for you. The gap does not appear as a gap. Those who cut corners in these phases usually pay triple: in solutions no one needs, in technical debt, and in software that no one understands anymore. Since 2026, the latter has a name: Comprehension Debt.

A vibe-coding prototype is not the same as a production-ready product. The prototype shows that something is possible. It does not show that it can be operated: not whether it fits into processes, systems, and operations, and not who will be responsible for the code in five years.

Developing the right thing the right way is the benchmark that remains after this shift.

The right thing means: solving a problem that is worth solving.
The right way means: developing software that remains maintainable, testable, and capable of being evolved over years.

This requires machine-readable context, quality gates and loops, and clean code. Clean code used to be a toolkit. Today, clean code serves as a guardrail for software development with AI.

You don't have to write every line yourself. You do have to be able to take responsibility for every line.

‍

AI in software development – self-assessment

Two minutes of honesty is all it takes to see where your bottleneck lies.

Go through these points. Check off what applies to you:

0 to 1 checkmark: You are ahead of most teams. This is exactly the right time to take stock before your speed outpaces your standards.

‍2 to 3 checkmarks: This is the typical scenario. The bottleneck isn't the model; it's the context, requirements, and review process.

4 or more: You are accelerating coding while cutting corners on the phases that make AI accurate in the first place. Another language model won't solve that.

Are you cutting costs in the right places? Talk to us. In a consultation, we'll clarify where you stand and what the next logical step is. We'll also tell you if you can handle it internally without needing a project. Not every conversation leads to a contract.

Email us at team@generic.de or call us at +49 721 6190960.

‍

FAQ

1. Will AI replace software developers?
No. What a development team does all day has never been primarily about writing code. AI accelerates that one step. The work is shifting toward understanding, decision-making, reviewing, and taking responsibility. The 2025 State of DevOps Report shows the same trend: more tasks, more pull requests, and the time saved is being reinvested into review and approval.

2. Is a vibe-coding prototype a product?
No. It serves as a basis for decision-making. A product must be operational, integrated, and supported over many years. The model does not provide this unless it is explicitly required.

3. What is the difference between Copilot and agentic development?
Copilot accelerates coding within the IDE. Agentic development operates in an AI development environment: with an agent harness, knowledge graph, guardrails, quality gates, and human approval. The model is the smaller part. The environment determines whether the result is accurate and accountable.

4. Does AI make clean code obsolete?
No. The client who owns the code must be responsible for it. For that to happen, the code must remain understandable. Clean code is the benchmark for this, both for humans and models. The compiler analogy does not apply here: a compiler translates deterministically, a language model does not.

5. What is a knowledge graph?
A machine-readable knowledge database: requirements, decisions, customer statements, architecture, code, and tests, all linked together. Humans and AI work from the same baseline. The graph gains value across phases and projects if it is maintained.

6. What is context engineering?
Context engineering defines the relevant excerpt from the knowledge graph for every task. Too little context leads to assumptions. Too much context lowers quality. It is about selection, not completeness.

7. What is an agent harness?
The AI development environment (AI IDE) is the overall system. Within it lies the agent harness: the model, rules, skills, and the control layer surrounding them. The harness accesses the knowledge graph as long-term memory and connects to external systems via interfaces (such as MCP). The model provides only a small part of the value. The harness makes the deployment controllable, repeatable, and verifiable.

8. How realistic is "up to three times faster"?
It is realistic when problem understanding, context, and gates are in place. As a promise without these prerequisites, you are just three times faster at doing the wrong thing.

9. AI in software development versus AI in the product—what is the difference?
One is about how a team develops software. The other is about what the product can do, such as forecasting or assistance. Both can go together. You should not conflate the two decisions.

10. Which companies is this approach suitable for?
For projects where software is part of the core business: typically in industry, with long lifecycles and high integration and liability requirements. Many teams are already building small, clearly defined tools internally. As soon as processes, systems, and accountability come into play, vibe-coding is not enough.

11. Can't we do this ourselves with AI?
Yes. Many teams are already building small, clearly defined tools internally. The standard remains the same: a clear problem, machine-readable context, quality gates, human approval, and clean code as a guardrail. If you set this up with discipline, you don't need us. Where this standard is not yet established in a company, we help introduce it—not because you couldn't do it, but to ensure that speed does not outpace control.

12. How does this relate to Dual Track Agile?
Discovery clarifies what is right, Delivery develops it correctly, and the Knowledge Graph connects both. This allows understanding and development to run in parallel without agents turning unclear assumptions into code. More on this: Dual Track Agile.

‍

Glossary

Developing the right thing the right way
Two frameworks: solving a problem worth solving, and developing software that remains maintainable, testable, and evolvable over the years.

LLM (Large Language Model)
A probability model. It does not understand; it searches for the next continuation. It fills gaps with what is most probable, not with what is right for you.

Vibe Coding
Developing software using natural language without actually reading or understanding the generated code. A vibe coding prototype shows that something is possible. It is not equivalent to a production-ready product.

Comprehension Debt
The gap between generated code and understood code. The code may look clean, but no one can explain why it is the way it is. A risk for maintenance and liability.

Knowledge Graph
A machine-readable, linked knowledge database spanning from vision to testing. A single source of truth for both humans and agents. It is expanded across phases and projects. Some documents refer to it as a "Context Graph"; this means the same thing, not a second system.

Context Engineering
Providing the relevant portion of the graph for a specific task. Selection over completeness.

Context Window
The amount of context a model can process in a single pass. Beyond a certain volume, quality drops; simply adding more data does not help.

AI Development Environment (AI IDE)
The overall system in which agentic development takes place. It includes the agent harness, access to the Knowledge Graph, and integration with other systems.

Agent Harness
The part of the AI development environment surrounding the model: rules, skills, guardrails, tests, observability, and approvals. It accesses the knowledge graph as long-term memory. The model is the engine; the harness ensures the engine remains controllable.

Guardrails
The traffic rules for the model. They define the boundaries of what the AI is permitted to do before an output is processed further.

Quality Gate
The checkpoint. A verification point with proof. No proof, no progress.

Loop
Repeated cycles until the task requirements and the result align. An agent cannot complete a task without validated proof.

Human in the Loop
Human approval following the loop. Agents are permitted to plan, develop, and test, but they are not permitted to merge. Every agentic artifact remains a draft until a human takes responsibility.

Dual Track Agile
Discovery and delivery in parallel. A work package only moves to implementation once it has been sufficiently explored.

Clean Code
Once a toolkit, now the guardrail for software development with AI: the same standard for both human and model. Existing code provides context for the next feature. Understandable code prevents comprehension debt. You don't have to write every line yourself, but you must be able to take responsibility for every line.

Dev-ready
The point at which the vision, solution concept, graph, and development environment are in place. Only then do humans or agents begin writing code.

Review Fatigue
The reviewer becomes exhausted because more code must be checked in less time. This leads to errors or increments passing through unchecked.