Claude Is My Beta Tester Now

A really fun software project just landed on my desk. I'm not going to tell you what it is yet. Details will emerge, and until they do you'll have to be satisfied with the vague version. What I can tell you is how it's getting built, because something changed in how I work and I think it's worth writing down while it's fresh.

Short version: Claude is building the app, and Claude is testing the app, on my actual desktop, clicking the actual buttons. I'm still the architect. But I went to the gym in the middle of it and the work kept going. That's new.

I still believe in the human in the loop

Let me get my position on the record first, because I could eat these words later. When I build software I like to be in the loop. I think you're going to need a human in the loop for the very foreseeable future, and anyone telling you otherwise is selling something. That hasn't changed.

What has changed is the length of the leash. The models are improving fast, and they can execute tasks far more autonomously than they could even six months ago. You still have to watch them. I learned that the hard way when one of my agents decided the fix for a networking problem was to expose my Mac mini to the public internet. But "watch them" is a different job than "do it yourself," and the gap between those two is where I'm getting my time back.

Who does what

Here's the split on this project, and I think it's the right one for right now.

I made every high-level architecture call. How the thing is laid out, what the data model looks like, what talks to what, what we're building first and what can wait. I wrote the product roadmap. I fed in a few transcripts from my business partner, who had features he wanted and features he thought would be cool, which is its own kind of input and a valuable one. That's my lane: strategy, architecture, taste.

Claude's lane is the loop. Build the feature. Test it. Fix what broke. Report back. Then I review the report, correct what needs correcting, and send it around again. I'm not writing the code by hand and I'm not clicking through the app by hand. I'm the one deciding what "done" means and checking that it got there.

I gave it my mouse

Here's the part I was hesitant about for a long time. On my Mac, I gave Claude permission to use AppleScript. Which means that instead of testing the app in some headless browser I can't see, it can actually click on my desktop. Open the app. Press the button. Type in the field. Look at what happened.

I resisted this for a while and I think the hesitation was healthy. Handing an AI control of your actual screen is a real decision. But the more I use this technology the better I understand its limits, its faults, what it's genuinely good at, and the shape of the thing I'm trusting it with. End-to-end testing turns out to be squarely in the "good at" column. It's precise, it's patient, it does not get bored on the fortieth click, and if the login prompt needs a human, it stops and asks me and I log in myself. Which is exactly the loop I want.

The tedious part is the whole part

Something I'm learning about app development specifically: the amount of end-to-end testing is enormous, and most of it is tedious. Not hard. Tedious. Clicking through the same flows, over and over, looking for the one thing that broke.

On this project it broke into three buckets, and Claude ran all three while I was training legs:

• Integrations. Does the calendar actually sync? Does it pull all the data correctly? Are the APIs responding the way the docs claim? Is the OAuth flow right, meaning when a user connects their account, does the handshake complete and does the token come back and get stored? I'll write a whole piece on testing integrations soon, because it deserves one.
• The UI. Does the button do what it's supposed to do? Does clicking this open that? When a user takes an action, is the event actually landing in the database, or does it look like it worked while nothing was saved? That last one is the quiet killer, and it's exactly the kind of thing a patient tester catches and a tired human doesn't.
• Performance. There's a tool called Lighthouse that scores how fast your app loads and how well it's built. Faster app, higher conversion, because every second you make someone wait is a chance for them to leave. Claude ran the audit, read the scores, and fixed the low ones.

I wrote last night that your business should free you. This is what that looked like the next morning. I said, I'm going to the gym, I want these three things done and these things tested. Then I left. When I got back there was a report waiting, with what passed, what failed, and what it fixed, and I reviewed it the way I'd review a pull request from a developer.

AI is becoming the beta tester

So here's the bigger point, and it's the one I'd bet on.

In product development, AI is becoming the beta tester. Not the architect, not the product owner, not the person with taste. The one who clicks everything, tries everything, and tells you what broke before a customer does.

I wrote about failing loudly and why watching the rocket explode is a gift. That's still true. Your rockets are still going to explode. Nothing about this changes that, and anyone who says AI means bug-free software is a type one charlatan. What it changes is where the explosion happens. More of them happen in the hangar, on my desktop, at 2pm on a Wednesday, in front of nobody. Fewer of them happen in front of users. The failures are just as loud. They're a lot less embarrassing.

The takeaway

Keep the human in the loop. Keep the architecture, the roadmap, the taste, the final review. Those are yours and I don't see that changing soon.

But the loop underneath you, build, test, fix, report, is something a model can run now, on your real screen, with your real app, while you're somewhere else. The tedious part of shipping software was always the biggest part. It just stopped being my part.

More on the project when there's more to say. More on testing integrations soon. For now, I'm going to go check what broke while I wrote this.

Want software that's been clicked through before you see it?

That's how we ship at HyppoAI now. Built, tested, and reported on before it ever reaches a user.

Let's build it