I have been to a fair few conferences this year. And if there is one phrase I keep hearing, it is "humans in the loop." Every platform, every panel, every presentation. Everyone has humans in the loop.
Which made me start asking a question nobody seems to be asking… what does that actually mean?
Because if it means a researcher signs off on an AI output before it goes to the client, that is quality control, not human judgment. If it means someone checks the discussion guide before the AI moderator runs it, that is a useful step, but it is not the same as real research expertise shaping the work.
The phrase has become so common that it has started to lose its meaning. And in research, that matters.
What the phrase is trying to signal
The concern underneath it is legitimate. AI moderation changes the nature of qualitative research in ways that are mostly positive but not without risk. An AI moderator that probes without context, or surfaces patterns without understanding what they mean for a specific brand in a specific category, can produce data that looks rich and feels convincing but misses the point entirely.
The instinct to keep humans involved is right. The question is what kind of involvement, at what stage and with what expertise.
The scale of adoption makes this question urgent. The 2025 GRIT Insights Practice Report found that 67% of suppliers now embed generative AI into client deliverables.
That is a significant shift in a short space of time. But it also means 67% of suppliers now need a credible answer to what their human expertise is actually contributing. Right now, not all of them have one.
Where AI does its best work
When used correctly, AI moderation can be transformative in specific and important ways. It removes the social performance dynamic that causes respondents to give you the calibrated, socially acceptable version of their opinion rather than the honest one. It scales conversations in ways no human moderator team ever could. And it surfaces patterns across large datasets at a speed that changes what is possible.
But there is a clear boundary to what AI does well. It finds the patterns. It does not know which ones matter.
Where human judgment is irreplaceable
In the 2026 GRIT Insights Practice Report, Leonard Murphy, Chief Advisor for Insights and Development at Greenbook, writes:
"AI policy governance is the only formal decision role that fails to crack the top three in any segment, even among the heaviest agentic users. We have built an industry that is governing supplier choice, technology acquisition, and outsourcing while quietly leaving AI policy to be set somewhere else."
That observation cuts to the heart of the problem. The industry is adopting AI at pace while leaving the question of human oversight largely unresolved.
I have spent my career across Kantar, Streetbees and now Bolt Insight. The thing that has stayed consistent across all of it is this. The difference between insight and information is interpretation. And interpretation requires context that AI does not have.
When a pattern surfaces in a qualitative dataset, the meaningful question is not just what it says but what it means for this brand, in this category, at this moment. That requires understanding the business question behind the research. It requires knowing what has been tried before and why it did or did not work. It requires cultural fluency, category expertise and the kind of judgment that comes from having sat in rooms where these decisions get made.
Humans in the loop means researchers with that kind of expertise involved at every stage, from study design through to the framework that makes findings actionable. Not a sign-off. Not a checkbox.
The risk of getting this wrong
The danger of humans in the loop becoming a marketing claim rather than a methodology principle is real. Research that is AI-generated and human-approved can still miss the point entirely if the approval is superficial or if the person approving it does not have the depth to know what they are looking at.
I have seen this happen. A technically proficient research output that answered the question as asked but not the question the client actually needed answered. The findings were accurate. The insight was not there.
Getting it right means being specific about what the humans bring, where they are involved and what difference their involvement makes to the quality of the output.
The methodology needs to be able to withstand the question… what would this research have missed without you?
The bottom line
The next time you hear humans in the loop, ask the follow up question. Which humans? With what expertise? At what point in the process?
Those are the questions that separate a methodology claim from a methodology. And in research, that distinction is exactly what matters.
