Adeline Yaw

AI Integration Specialist (Happiness Engineer) @ Automattic

  • Most people have been stuck with a support bot that won’t let them through. I work on the opposite problem: teaching ours to notice when it should step aside.

    I wrote a post on the WordPress.com blog about how it makes that call. Here’s the short version.

    Before the assistant answers anything, it runs a few checks. Have you already asked for a person? Have you asked this same question before? Have you contacted support about it recently? Is this the kind of topic a bot shouldn’t handle on its own? If the answer to any of those is yes, it starts moving you toward a human instead of trying again.

    It also offers a person without being asked. That happens when a problem needs someone to look into an account properly, when the assistant isn’t confident in its own answer, or when what it already suggested didn’t work.

    And if you ask for a person outright, it doesn’t argue with you. It might ask one or two quick questions in case it can fix the problem on the spot, but the offer to hand over stays on the table. You choose.

    The part I care about most is what happens after that. When it hands you over, the whole conversation goes with you — what you asked, what it tried, what didn’t work. Nobody has to explain their problem twice.

    The number that surprised me: about one in four conversations ends up with a person, and in about half of those, the assistant is the one who suggested it. It offers first more often than people expect.

    That’s the goal, really. Solve what it can. When it can’t, get out of the way quickly.

    Read the full post on the WordPress.com blog →

    + ,
  • > grep –lessons-learned ./experience/*

    It’s been about nine months since I took on the AI Integration Specialist role, and I wanted to share some of the things I’ve learned along the way. These aren’t polished frameworks, they’re real observations from doing the work every day.


    Accuracy is trust

    When an AI bot gives a wrong answer, it doesn’t just fail to help, it actively erodes trust. Users who receive incorrect information from our Bots are less likely to engage with the bot next time. That’s why we treat accuracy as the single most important metric. Getting it right matters more than responding fast.


    Review everything, assume nothing

    One of the biggest shifts in my thinking has been around evaluation. You can’t improve what you don’t measure, and you can’t measure what you don’t look at. Our regular feedback reviews are the foundation of everything we do. Every conversation the bot has is an opportunity to learn.


    Cross-team alignment is non-negotiable

    AI work touches almost every part of our support workflow. If the AI Squad operated in isolation, we’d quickly drift out of sync with what the business actually needs. Regular check-ins and close collaboration with various stakeholders keep us grounded and focused.


    The human side of AI leadership

    Leading a squad isn’t just about data and strategy. It’s about people. Providing constructive feedback, supporting professional growth, and making sure everyone on the team feels valued and heard, that’s where the real impact happens. A motivated, well-supported team does better work, and that shows up in the bot’s performance.


    Still learning

    Nine months in, I’m more convinced than ever that the intersection of AI and customer experience is where some of the most meaningful work is happening. I don’t have all the answers, but I’m learning something new every day. And that’s exactly where I want to be.

    + , ,
  • > forecast –ai-support-2026

    After nearly a year of working full-time on AI in customer support at WordPress.com, I’ve been thinking a lot about where this is all heading. Not in a hype-cycle, buzzword kind of way, but based on what I’m actually seeing in our data, our conversations, and our day-to-day work.


    AI is getting better at knowing what it doesn’t know

    One of the most important developments I’ve seen is AI getting better at recognizing its own limitations. Early on, bots would confidently give wrong answers. Now, the best systems know when to say “I’m not sure about this, let me connect you with a human.” That self-awareness is a game-changer for user trust.


    The human-AI handoff is the new frontier

    The moment a conversation moves from bot to human support is critical. Do it too early, and you lose the efficiency benefits of AI. Do it too late, and the user is frustrated. Getting that transition smooth and seamless is one of the biggest areas of focus for teams like ours, and I expect it to be a major differentiator for support organizations in the coming years.


    Quality evaluation needs to evolve

    How we evaluate AI support quality today is still largely manual. Our AI squad reviews conversations, scores them against criteria, and identifies patterns. That’s valuable, but as volume scales, we’ll need smarter ways to evaluate at scale. Think automated quality scoring, anomaly detection, and real-time accuracy monitoring. The teams that figure this out first will have a serious advantage.


    AI doesn’t replace empathy

    No matter how good AI gets, customer support will always need a human heart. AI can handle the “what” and the “how” but the “I understand how frustrating this must be” still needs to come from a real person. The future isn’t AI vs. humans. It’s AI and humans working together, each doing what they do best.


    What I’m excited about

    I’m genuinely optimistic about where we’re heading at WordPress.com. The work we’re doing with our Bots, the investments in AI quality, and the cross-team collaboration across divisions and teams, it all points toward a support experience that’s faster, smarter, and more caring. And I’m proud to be part of building that future.

    + ,
  • > cat ai-squad-overview.md

    One of the most common questions I get from colleagues is: “What exactly does the AI Squad do?” So I thought I’d write a proper overview of how our team works, what we focus on, and why it matters.


    The mission

    The AI Squad exists to make WordPress.com’s AI support bot as accurate, helpful, and ‘human-feeling’ as possible. Our bot handles thousands of support conversations, and our job is to make sure those conversations are genuinely useful for the people on the other end.


    How we work

    Our process revolves around a few key activities:

    // feedback reviews — Squad members regularly review our bot’s responses to evaluate quality. We look at accuracy, tone, completeness, and whether the bot genuinely resolved the user’s issue. These reviews feed directly into improvements.

    // data analysis — I track bot statistics, accuracy rates, CSAT scores, and escalation patterns. When something trends in the wrong direction, we investigate and act.

    // cross-team collaboration — We don’t work in a silo. We partners closely with teams to ensure our priorities align with the broader division strategy.

    // weekly reporting — Every week we publish detailed updates. This keeps everyone informed and creates a transparent record of our progress.


    Why it matters

    AI in customer support isn’t about replacing human support. It’s about handling routine questions quickly and accurately so that our Happiness Engineers can focus on complex, high-value interactions. When our bot gets it right, users get faster help and our Happiness Engineers can spend their time where it counts most.

    That’s the balance we’re always working towards.

    + , ,