A cautious approach to AI should lead to more code being written by AI, not less.
At S39, MK Tech’s team covering engineering, product, and design, we are deeply sceptical of AI-generated code. We are highly critical of “AI slop”: work that looks plausible but has not been properly thought through, understood, or checked. We believe we need to understand our own codebase and take responsibility for what it does because our products handle sensitive information, including health records and private personal information. When a failure can affect someone’s health, privacy, or life, understanding the software is part of taking that responsibility seriously. An agent’s confidence in its answer gives us very little to rely on by itself. It is precisely for this reason that we actually end up with even more AI-generated code.
An engineer might write the core implementation while AI writes the tests and supporting code around it. That supporting work can, and should, grow larger than the human-written core because establishing confidence in a solution takes substantial work of its own. Human-written code needs that scrutiny too. We want to understand what the tests establish, which assumptions they depend on, and whether they actually challenge the behaviour we care about. AI gives us the capacity to undertake more of that work, including testing that previously felt impractical.
The engineer stays closely involved because we need to take ownership of the code, the product it supports, and ultimately the business that depends on it.
This is why understanding our own codebase remains so important. We need to know what the product is supposed to do, how the solution fits into it, and what the consequences of getting it wrong might be. We use Codex as a sounding board when investigating problems and designing solutions, drawing on our shared product knowledge and documentation. The engineer stays closely involved because we need to take ownership of the code, the product it supports, and ultimately the business that depends on it. That ownership requires us to understand the decisions behind the implementation and take responsibility for their consequences.
Once we have established the direction, we let the agent work through implementation, testing, and revision, checking in to make sure it is still heading where we intended. A failing test might expose a problem in the implementation, or the test itself might need correcting. The agent can investigate, change either, run the tests again, and review its own work. That feedback cycle lets us work through more of the supporting detail before the engineer inspects the draft pull request.
We are equally sceptical of a passing test when the agent has changed both the implementation and the tests. Both can contain the same mistaken assumption, and a test can pass because it now accepts behaviour that should still be challenged. We have to understand whether the underlying problem was actually fixed. Generating more tests gives us more opportunities to examine the solution, but every test still needs to earn the confidence we place in it.
At the draft PR, we ask the agent to explain what it changed and why, and we inspect the actual code and test changes against the intended requirement. The explanation helps us follow its reasoning; the review is where we challenge whether that reasoning holds up. Engineering peers then review the work as well. We need to be able to explain and stand behind what we accept, including the supporting code written to help us check it.
Independent business acceptance matters for the same reason. The people who designed and built the solution bring an engineering perspective, while human user acceptance testing brings business and compliance/conformance perspectives. Those reviewers assess whether the product solves the business problem and meets the applicable requirements. Engineering review examines the implementation; business acceptance examines whether the result is suitable for its intended purpose. We believe both are necessary because understanding how something was built and judging whether it meets the business need require different perspectives.
Our conservatism gives AI more work to do because we ask for substantial evidence before relying on a solution. AI helps make more of that testing and supporting work feasible, and that is how we end up with so much AI-written code. We still have to understand what we accept and decide whether the evidence is good enough.