Agents

There’s an agent design pattern with an uncomfortable premise: the agent is the orchestrator, and you are one of its tools. It goes by a few names — human-as-a-tool, human-as-tool, sometimes lumped under “human-in-the-loop.” But it’s worth separating the two, because the inversion is the whole point. Human-in-the-loop vs. human-as-a-tool Human-in-the-loop keeps the human in charge. The agent proposes, the human approves. You’re the supervisor signing off on the intern’s work before it ships....

When you run a team, you don’t decide who to keep based on their resume from two years ago. You watch what they actually ship on your work, and you promote the ones who deliver and let go of the ones who don’t. I want the same thing for the models I use as agents. And public leaderboards don’t give it to me. Why public benchmarks don’t tell you what you need A leaderboard tells you how a model did on someone else’s tasks, scored on someone else’s rubric, often on problems that leaked into training data....