AI and the (possible) future of product development
What today's agentic AI coding tools tell us about the state of product development
Introduction
Readers of previous posts will have learned that I enjoy dabbling with AI tools, most recently agentic coding. In this space, I am no more than an enthusiastic hobbyist, exploring a tiny fraction of AI’s rapidly expanding surface area. That said, this experience can give some insights into how AI will change the way we work. In this post, I try to extrapolate the implications of AI for (tech) product development.
As in all my posts on this Substack, I rely solely on information that is in the public domain, and on technology that is available to the general public. As such, I won’t comment on (for example) unrestricted (and unreleased) versions of Anthropic’s Mythos model or other frontier LLMs. I tend to use paid-for closed model LLMs rather than open-weight models, so this may well skew my analysis and viewpoints. As always, these views are my own.
Approach
After building Weavify (see my previous post) I set out to use agentic coding to build two further apps. I used Claude Code to create Sensif.ai, an AI-augmented news and analysis site for AI and Defence Tech, in a possibly Sisyphean attempt to keep abreast of what’s happening in the fast-moving world of AI. To help build this, I created Claudify using OpenAI’s Codex tool. Claudify is a management app for Claude Code that allows me to design and shape workflows and keep track of key decisions, risks and where I am in the software development pipeline at any one time.
Learnings
Creating Weavify, a web bookmark and book manager showed how it was now practical to create genuinely useful apps with relatively little effort. But what would I surmise about where product development is heading from building these two other apps?
The real brilliance of Frontier Models is deceptive.
Frontier models such as Anthropic’s Fable 5 can undoubtedly create amazing experiences and apps from scratch to fully working outputs. Also, they are remarkably resourceful, if the latest AI security ‘breakouts’ are anything to go by. However, these admittedly amazing examples (Ethan Mollick’s social feed is particularly great) can give the wrong impression of their utility in real-world product development. In other words, their very brilliance can lead you to draw the wrong conclusions on their real-world utility.
Long-run agentic frontier AI systems can create something great and be really creative about how they approach the problem at hand. However, if you want to create something really distinctive that meets a specific need, then you really need to be hands-on in shaping the product. Think about it: if different competitors use generative AI systems to create products using similar generic top-level prompts or tasking and a similar input set, then you will see a proliferation of very similar, non-distinctive, and consequently, not very valuable products. As an example, Claude Code helped create a working prototype of Sensifai in a relatively small number of sprints. However, this was the tip of the iceberg. Fine-tuning the app to make it genuinely useful and meet my intended use cases, while in the meantime addressing usability issues took by far the greatest amount of my time and effort. In other words, there was an inverse Pareto experience at play. 20% of the effort was getting me 80% of the way there, though the final 20% was what made the difference between a nice demo and a product that is usable on a day-to-day basis.
Product Management is being re-invented and is at the heart of success.
In my previous post, I wrote about humans’ enduring role as AI Innovators - envisaging different futures enabled by the products they create, filling needs that may be poorly expressed, if at all. This practice of creativity is what lies at the heart of product management. AI models can augment and support this creativity in many ways. They can collate, synthesise, and help derive insight from essentially limitless amounts of data, be it held internally within your organisation or externally. Within a product development context, they can analyse your competitors’ pricing strategy, deduce their product strategies, and help analyse your customer usage data to identify opportunities for the taking.
However, for as long as humans remain the customers, users or purchasers of your products, in my opinion, product management will remain, at its heart, a human profession. There remains a gap between AI’s remarkable ability to propose options, and the very empathetic process of elucidating customer needs, both expressed and unexpressed. I suspect that product management will evolve to a model that is centred on the rapid iteration of working prototypes, rather than the manual creation of product requirements documents. Instead of being outnumbered by larger teams of engineers, product managers will likely work closely with a small number of technical experts to orchestrate rapid evolutions of the product. But the essence will remain unchanged. Product managers will continue to provide the insight and steer to a largely automated product development cycle.
Brainstorming for one? AI and the creative process.
While building Sensifai, one characteristic I really valued was having AI as ‘someone’ to challenge my ideas and propose some of its own. I have already written on how diversity of thought and viewpoints are at the heart of high-performing teams. (Matthew Syed’s book [1] is essential reading. AI can help bring that diversity into your thinking even if you are a team of one. In other words, they can help create single-person brainstorming and ideation. Given that even different agents rely on the same underlying model and training data, there is the risk of AI models producing simply the illusion of diverse inputs. However, recent research suggests that using distinct personas and careful prompting can create genuinely incremental diversity in the viewpoints being put forward. [2]
Let's consider a couple of examples. As I was building Sensif.ai with Claude Code, I asked ChatGPT 5.6 Sol to review the draft product from a usability perspective and to offer other ideas. Some suggestions were helpful. For example, the product could be made more useful by improving its visual analytics capability (which we called a ‘Radar’), using LLMs to analyse individual articles to then highlight trends, for example in AI research (which can be quite overwhelming) or in DefenceTech investments. Other ideas were very much mine - creating podcast-style summaries that I could listen to on my commute. However, some of ChatGPT’s suggestions misunderstood the product’s very premise. For example, as I told it I valued integrity and reliability of the product, it really doubled down on that, and it proposed a whole host of editorial tools, more suitable to a newsroom than to simplify how I get my AI news and updates.
Other proposals were sound in theory, but failed the real-life test. I built a sophisticated ‘clustering’ system to collapse multiple news items into single events in order to reduce multiple reporting of the same news. However, although the feature technically worked, it has been of limited value so far, as most of the news items were already unique, as they were sourced directly from research labs or commentators. An initial data analysis of the sources would have picked up this issue before implementation. Nevertheless, the relatively low cost (in time and money) of each development sprint meant that the opportunity costs of unsuccessful product concepts are small. Reducing the cost of failure directly increases innovation output.
Specificity really matters.
When creating responses to an input, transformer-based LLMs are carrying out a probabilistic sampling of possible outputs, meaning that for anything but the simplest task, no outputs are ever identical. Similarly, as LLMs are now able to operate over much longer runs, tasks are broken down into an ever-larger number of steps. This means that the number of possible approaches and outcomes grows exponentially. Taken together, these represent the power of LLMs. They produce outcomes that are not predetermined and come up with strategies that are not baked into their original training data. This is what makes LLMs so useful as generative tools.
Why does this matter? Well, there are parts of the product development process where you do want predetermined outcomes. Once you have a candidate product description for a given sprint, you’d like your agentic coding platform to implement what’s been described, and not to indulge in a bout of creativity. Even more importantly, if what you are building, be it software or hardware, needs to interface with another system, you want the system to be very prescriptive in how it behaves.
Similarly, when creating a workflow for how your product is built, tested, integrated, merged and deployed, you want an approach that is as deterministic and repeatable as possible. This is not a space for creativity. Creating predetermined or deterministic outcomes is achieved by putting in place clear, specific, and prescriptive key artefacts. While the code for the software (or hardware) being produced is the reference source information for what is being built, it does not necessarily capture the product’s intent. These are held in “anchor” or baseline documents, such as architecture specs, product requirement documents, workflow specifications, subagent definitions and so on.
As an example, I created a Claude Code workflow which described how to delegate different tasks to different specialised subagents. One subagent was the backend developer subagent, where typically I’d bring all the backend development together. In a particularly sloppy instruction, I asked Claude Code to follow the specified workflow and then to “delegate implementation to subagents.” It proceeded to start a separate subagent session for each of the six bugs I wanted fixing, rather than providing the six bugs to a single subagent. This resulted in an inordinate expense and particularly troublesome sprint, as these tasks could not easily be parallelised. The model had not failed to follow my instructions. I simply lacked the required specificity.
In a way, this need for specificity turns on its head the Agile Manifesto’s principle of “Working software over comprehensive documentation.” This was created in a world where creating software was the expensive part of the process. It optimised the value-adding stage of software creation. Today, success depends on clear sets of product and workflow instructions. Get those right, and the right software will flow.
The radical reshaping of dev teams in the age of AI
In setting up Claude Code, I set out to create teams of subagents to build my apps. Instinctively, my starting points were roles that we are familiar with in human teams: product managers, architects, back-end implementers and so on. My frame of reference was the traditional scrum team. I had used these to good effect in both software and mixed software-hardware development teams. It is also based on the same model as the two-pizza team used at Amazon, so it felt like a good place to start.
What became immediately apparent is that, like in human teams, different roles require different levels of skills and expertise. The most sophisticated models were best suited for the ‘more creative’ role, as they benefited from the broadest external context when shaping product concepts, requirements and trade-offs. Likewise, the architect had to deal with significant complexity, ensuring the design met the intended customer needs as well as non-functional requirements (such as security and scalability). It also played a crucial role in creating a development plan that built the product progressively from an architectural perspective - e.g. getting the data model right, validating it, enriching data sets and so on. Other roles such as the DevOps role, responsible for managing the workflow, gates, deployments and so on, were more mechanistic.
So far so good. We had an AI version of a typical human team. However, things become more interesting when you consider the human element. First, in traditional development teams, the constraining factor is the size of your software engineering team. Now, with the right project infrastructure in place and a sufficient token budget, this is no longer a limiting factor. You can delegate as much work to as many agents as can be parallelised. The constraint shifts towards the ability to shape “what” you want, and the ability to support the product, rather than your ability to create it.
Secondly, the role of the human within the development team is now very different, and can be divided into three. First, you have the product manager or intent authority who establishes intent and priorities, decides what matters, guiding, pivoting and correcting the development sprint-by-sprint, deciding when to pivot and so on. Secondly, you have the technical authority, who takes accountability for the technical architecture, establishes the “definition of done”, and defines the non-functional requirements. Both roles also act as final arbiters on items for escalation. Finally, you have the governance authority roles. In my case, it was about establishing the DevSecOps pipeline, determining the extent to which I was happy automating the process, which were the decisions that required human (i.e. my) intervention, and so on. Crucially, whilst the product manager and tech authority roles are team-specific, the governance roles are enterprise-wide and set the standards across all product teams.
Now of course, in enterprise settings, you will have many other roles, including ensuring that you have robust infosec assurance, that your training datasets and pipelines are robust, reliable and trustworthy, that infrastructure is operated and maintained, that legal compliance is assured and so on. However, for the core product development teams - the shape of roles, and the make-up of teams will change fundamentally. The traditional model of a scrum or agile team of 6-8 people, principally engineers, with a single product owner, a technical architect supporting several teams is likely not to persist for much longer. Instead, people will have oversight of much larger development programmes, executed primarily by AI agents. Technical skills will remain essential, but these will shift towards orchestration, planning, oversight, review and compliance. [5]
Final thoughts
So where does this leave us? A few things are clear - we are entering an explosion of product development capacity. Product organisations will no longer be constrained by how many humans can master the craft of designing engineering outputs, be they software, electronics or hardware. As long as they are able to remodel their workflows, organisation design and skills, they should be able to see an explosion of output. This will also have implications on how organisations are designed - the traditional scrum team model is no longer relevant to this way of working. We are looking at smaller teams with a much larger span of ownership. The ratio between people working on the ‘what’ and those working on the ‘how ’- which traditionally was skewed towards the latter - will likely rebalance.
Additionally, teams that master AI-human teamworking will be able to iterate much more quickly, respond to changes and pivot quickly. AI systems are well-suited to refactoring legacy architectures, so technical debt should become less of a burden. This means that the product development velocity can easily place a strain on organisations. The constraint is no longer “how long do we have to wait to build this,” but rather, how quickly can an organisation pivot and adapt to exploit the new capabilities. We are entering a world where high-velocity decision-making will be the difference between success and failure.
Notes on the use of AI in writing this blog
This is my personal blog. It represents my own thoughts and has been written by me. I have used ChatGPT 5.6 Sol as a research assistant, helping carry out online research, as as well as challenging and providing feedback. The title image is also clearly AI-generated. I have also used it to proofread the text. As always, opinions expressed are my own, and not of my employer or anyone else.
References and Further Reading
Syed M., “Rebel Ideas: The Power of Thinking Differently,” John Murray, 2021.
Wan Y., Kalman Y. M., “Diverse AI personas can mitigate the homogenization effect in human-AI collaborative ideation,” Computers in Human Behavior: Artificial Humans, Volume 8, April 2026.
Brown, T. B. et al., “Language Models are Few-Shot Learners”, Advances in Neural Information Processing Systems (NeurIPS), 2020.
Wei, J. et al., “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models”, Advances in Neural Information Processing Systems (NeurIPS), 2022.