Across organisations, AI is rapidly changing how data-driven teams operate, from how insights are generated to how decisions are made and executed. Whether it’s CRO, Engineering, Analytics, or Data Science, each function is experiencing a shift in responsibilities, required skillsets, and the speed at which value can be delivered.
In this series, I am working with current leaders in this space across the North of England and Midlands to explore how AI is impacting each of these core areas, highlighting the opportunities, challenges, and practical implications for teams looking to stay ahead. From automation and augmentation to entirely new ways of working, this collection breaks down what AI means in real terms for the people and functions at the heart of modern data organisations.
The first article is written by Richard Chapman, a CRO Leader and Co Founder at Co-Labs
AI is going to replace everyone’s jobs! Run for the hills! Not really…
Where does AI fit into the CRO and Experimentation world? Where does AI fit into the CRO and experimentation world? Realistically, it’s not going to replace everyone’s jobs. It can’t, yet. AI is an amazing tool, and it can certainly help us improve, become more efficient, and increase our output as teams. Laying staff off though? Not for me, that’s short-sighted and a short term profit grab by companies whose expectations of AI are beyond reality.
People feel fear with AI and their jobs, it’s natural because of its capability, we are overwhelmed in our jobs and lives because of the amount of tools and noise around AI. Knowing where to start in itself is tricky, it’s an ever-growing, rapidly expanding minefield.
With 12,000 AI tools and releases worldwide in the last year, it’s normal to feel this way.
So where does it fit into our world? And how do we, the CROs and Experimenters of the marketing world use it and get the best out of it. Get it to work for you and you have the advantage. How do we make it simple, actionable and timely so we can grow as people and businesses?
Data analysis
AI has arguably its biggest role to play here — and the impact is felt most acutely at the junior end of the team. In over 15 years of agency work, one of the most consistent challenges I’ve seen is less experienced team members struggling to move from raw data to clear, confident conclusions. Knowing what the numbers mean is one skill. Writing it up in a way that a CMO finds compelling and actionable is quite another — and that’s a craft that takes time and repetition to develop.
AI changes that dynamic significantly. It can surface patterns, anomalies and opportunities that might otherwise take a junior analyst days to find and present them clearly, quickly, and without the hesitation that comes from inexperience. For marketing leaders, that means faster insight and less time lost to manual reporting cycles and data analysis. This capability enables companies to pivot faster than ever before.
The honest question it raises, though, is this; if AI is doing the analytical heavy lifting, how does the next generation of CRO talent develop the instincts that come from wrestling with data themselves? The answer probably lies in how teams use AI as a starting point that gets interrogated and challenged, not a conclusion that gets accepted and forwarded. Junior team members learn through practised skills in writing and presenting for the appropriate audience, the risk is, will the learning be the same with AI in place? Or will a reliance on AI be the norm?
That’s a leadership question as much as a technology one.
Hypothesis creation
AI is useful here but not to the extent that a human is not needed. We (humans), will know a client’s business, the nuances, the competition, the history and the politics of an organisation. An AI would need to be fed a significant amount of information before it could emulate the human version of hypothesis creation, this takes time.
AI is good at hypothesis creation, but not 100%. From my experience AI at a top level (using ChatGPT hypothesis generator, Clarity Co Pilot, Opal website analyser etc) can generate low hanging fruit hypothesis, button colours, images, add missing elements like social proof etc. AI is capable of so much more but, you need to feed it and this is where humans take charge. AI can only be effective based on what we give it. The same thing data science has taught us for decades still applies, bad data in, bad data out.
Prioritisation
Once we have our hypotheses, deciding which tests to prioritise has always been one of the most politically charged moments in the CRO process. Stakeholder pressure — ‘I want my test done first’ — will be familiar to most CROs and experimentation practitioners, and one that frameworks like ICE, PIE and RICE were designed to solve. By scoring tests objectively against criteria like impact, confidence and ease of implementation, these frameworks take the favouritism and bias out of the workflow. The problem is that scoring tests manually, aligning teams on the inputs, and managing the meetings and sign-off around it all takes significant time.
This is where AI earns its place on the roadmap. Feed it the right inputs – page location, traffic volume, the target behaviour change and the underlying hypothesis – and it can score and sequence your test backlog consistently, without the rounds of stakeholder negotiation. Your team still provides the strategic input that matters. What AI removes is the administrative drag around it. No more scoring spreadsheets, meeting scheduling, the back-and-forth and so on. The same roadmap gets produced in less time, with fewer meetings, and with the politics largely taken off the table.
Test design and experience building
This still requires human judgement in ways AI can’t replicate well, AI’s coding experiments are well on the way but they are not perfect… yet. Developers can breathe for now but with a look over their shoulder. Deciding what change to make, why it might matter to a real person, whether it’s on-brand or whether it creates unintended friction elsewhere – that’s design thinking, not data processing. AI can generate variant copy or layout suggestions, but someone still needs to look at it and say “yes, that’s what our customers actually need.” or “That will affect our business goals.”
Test reporting and stakeholder communication
Data analysis and post-test reporting share a lot of common ground, and AI plays a similar supporting role in both, helping to process results, identify patterns and structure initial findings quickly. That part of the process benefits from everything we described earlier.
But what happens next is irreducibly human, and that isn’t going to change any time soon.
Presenting test results to senior stakeholders is rarely just a data exercise. It involves reading the room before you’ve even opened the deck. It means knowing how to frame a result that didn’t go the way the business hoped — explaining why a test that ‘should have won’ didn’t, without undermining confidence in the programme. It means managing the moment when a pet project, one a director has championed for months, gets killed by the data. And it means doing all of that in a way that keeps people invested in experimentation rather than retreating from it.
That takes persuasion, empathy, political awareness and experience. It takes someone who understands not just what the results mean, but what this particular group of people need to hear, and how they need to hear it. No AI agent is going to pick up on the body language or mood in the room, sense that the CFO is sceptical, or know when to let a silence sit.
For marketing leaders, this is an important point. The human skills that make experimentation programmes succeed internally include the ability to build a culture of testing, to keep stakeholders engaged through losses as well as wins, to make the case for the next round of investment. Those skills become more valuable as AI takes on more of the technical workload, not less. Your experienced practitioners aren’t freed up to do less. They’re freed up to do more of what only they can do.
AI can draft the slide deck. It can’t read the room.
Where humans remain firmly in charge
For all that AI changes about the mechanics of CRO and experimentation, the things that matter most remain stubbornly, reassuringly human.
Finding the big picture is still a person’s job. Spotting that your test results aren’t just telling you about a hypothesis but about a fundamental shift in how your customers think about value — that kind of strategic pattern recognition requires someone who understands the business, the market and the moment. AI surfaces the data. Humans decide what it means and where to go next.
Overall strategy remains human territory too. Which pages matter most right now. Which customer segments deserve focus. How experimentation fits into a broader growth story that needs to be told to a board, a leadership team, or an investor. These are judgements that require context AI simply doesn’t have access to, the water-cooler conversations, the commercial pressures of the quarter, the strategic moves the business is making and external factors such as economic and technology changes. The ability to look up from the data, read what’s happening and adjust course is one of the most valuable things an experienced CRO and Data Science team brings. That’s human value.
What AI gives you is the bandwidth to do all of this better. When your team isn’t buried in data processing, scoring spreadsheets and formatting slide decks, they have the headspace to think at the level your business actually needs them to think at, this is a huge benefit we are often overlooking.
The teams already embracing this are moving faster, learning more and making smarter decisions with the same headcount they had before. The gap between them and those still doing it the old way is already opening up.
Hiring for AI? Let’s Make It Simple
Download our AI Hiring Guide – a practical resource to help you identify which roles to hire and when.
Or book a consultation with an AI recruitment expert who understands your industry.