“Just because it looks pretty on the screen, don’t assume that it’s right,” said Michael DeHaven of Removery, after small glitches slipped past his tests. The same discipline tells him where AI stops. The last thing to hand a non-human agent, he said, is “true care for a customer.”

Summary

At Birdeye View 2026, Michael DeHaven, Chief Systems and Digital Officer at Removery (165+ studios), and moderator Sanjay Kini, Birdeye’s Chief Customer Officer, discussed what AI should and should not own. Removery has AI agents test each other’s work before humans review it, uses AI to respond to positive reviews and route problems, and keeps clinical data and customer care in human hands. This post covers each guardrail.

What did Removery learn from trusting AI output too quickly?

“I wasn’t doing enough QA,” DeHaven said of the early phase. He ran unit tests, saw them pass, and moved on. Then small glitches started showing up in work he had thought was perfect. Coding quality from AI agents is “night and day” better than a year ago, he said, but it still makes mistakes, and he does not expect it ever to be perfect.

Kini hit the same wall in Birdeye’s own support team. His head of technical support ran case data through Claude and got a polished analysis quickly. Something looked wrong, so he asked whether it was right or whether Claude was guessing. The reply was “oh, I guess.” Kini remembered Claude wasn’t sure, but nothing in the polished answer suggested that. A confident tone tells you nothing about whether the answer is reliable.

How does Removery verify what AI builds?

DeHaven rebuilt his process around a requirement: multiple rounds of QA before anything is pushed to GitHub.

One evening, a request landed on his desk that was not on his priority list. Before leaving, he gave an AI agent the problem and had it research three code bases. It came back with a dev story and acceptance criteria. He approved it, and by morning the agent had built the whole thing, with pieces already deployed. Four agents ran QA at the same time. They found errors, the fixes went in, and the whole loop ran three times before the work was cleared to push.

For another project, so complex that eight QA agents were working against it, the agents were “antagonizing” the code base and trying to break it in every possible way. Human QA still has a job to do as well. The right number of agents depends on the project, but the principle holds: the agent that builds something should not be the only one vouching for it.

Where does Removery draw the line with customers?

For reviews, the rule is simple. When someone writes a four- or five-star review, having AI react in a relevant way “can be absolutely amazing,” DeHaven said. A quick “thanks for writing that review, you’re awesome” is something he is happy to let a bot handle.

Constructive reviews, and sometimes negative ones, route to people, so the response is meaningful and someone takes action. Sometimes that means identifying the client and reaching out personally to make sure they are taken care of. “If somebody’s having a problem,” he said, “don’t lose the personal touch.” PennyMac draws a similar line, which we cover in our post on regulated brands.

AI still does the routing. A studio is not always the right place to send an issue, so Removery wants to connect each customer with whoever can solve it fastest and best. AI can make that call instead of fixed procedural logic. Store teams still learn from the feedback, and DeHaven expects the right people to have more time to do excellent work instead of moving each customer through.

What data does Removery keep away from AI?

Removery’s work is considered a medical service in many states and countries, DeHaven said, so the company separates customer data from clinical data. Most of the time, clinical data stays private and available only to humans. Other customer data can be queried by chatbots, people, and dashboards alike.

His goal is for humans and AI agents to “coexist” in the same environment and complement each other. Data has to be accessible, he said, but at the right moment and with privacy controls. Legal questions work the same way. He often asks Claude or Gemini what a state law change means, then decides whether it needs the humans in legal for an official opinion.

How does Birdeye support a human-in-the-loop review workflow?

DeHaven credits a Birdeye feature built for exactly this split. Birdeye Reviews, powered by Jay, the marketing AI coworker, lets teams decide which reviews AI answers and which go to a person.

  • Answer routine reviews: AI replies to positive reviews in the brand’s voice.
  • Route the rest: Constructive and negative reviews go to a person who can respond and act.
  • Choose the setting: Responses can post automatically or wait for approval.
Jay- Birdeye Marketing AI Coworker

Frequently asked questions

What is human-in-the-loop AI?

It is an approach where AI does or supports parts of a workflow while people stay involved in checking the work, making decisions, escalating issues, and caring for customers. At Removery, that means AI agents test each other’s work, humans QA as well, and people handle customer issues.

How many AI QA agents does a project need?

There is no set number. Removery used four QA agents on one project and eight on a more complex one, and the right number depends on the project’s complexity and style. The principle is that the agent building the work should not be its only check.

Should AI agents have access to all company data?

No. Removery keeps clinical data private and accessible only to humans, while chatbots, people, and dashboards can query other customer data. Data should be accessible at the right moment, with privacy controls.

What should leaders do first?

List the customer moments in your workflow and mark each one as AI alone, AI then human, or human only. Start with the moments where a mistake would cost a customer’s trust.