It’s fantasy football season! There’s nothing like obsessively watching the waiver wire during the week, then posting up in front of the TV to root for dumb outcomes that have nothing to do with the actual game being played.
Sadly, as a person with two small children, a business to run and AI to tinker with, I don’t really have time for any of that. But it’d be rude to leave my leagues with an odd number of players, and what is fantasy football if not a perfect eval?
This year I’m in two leagues, one with some college friends that is decidedly unserious and will inevitably see at least two participants stop paying attention entirely by week two, and another where folks will actually watch all the games and actively manage their rosters.
Since it’s likely that at least some readers are unfamiliar with fantasy, an extremely quick primer to start. The core of the game is that you draft a team consisting of players at different offensive positions in the NFL, and each time they do something good (catch the ball, run forward with the ball, score touchdowns, etc.), you get points.
Each week you’re matched up against someone else in the league, and whoever scores more points that week wins. The teams with the most victories at the end of the season have a brief single-elimination tournament to decide the winner. You can change your players during the season either by trading with other teams or picking up players that nobody owns.
All of this makes managing a team a pretty interesting task for AI, because you have to constantly be evaluating your players, the players available to trade for/pick up, the NFL schedule (since a player who is going against a team with a bad defense is likely to score more points) and the state of your league. Lots of weekly research, analysis and decision-making to be done. But first things first — you have to draft your team.
Before draft day, I had two major tasks to complete:
Build out the technical infrastructure to allow Claude to draft and manage my teams
Create a strategy document that would guide its decision making
For the first one, I had two different platforms to deal with — one league uses ESPN’s fantasy app, and the other uses Yahoo’s.
These days I tend to use ChatGPT for planning these kinds of technical projects because Claude is bad at research. It gave me a quick rundown of the tools available.
Yahoo has documentation of an official API with all of the access needed to manage your team during the season but no API to use during the draft. Unfortunately, that appears to be out of date, since the API is now application-gated, and the application specifies that it’s read-only. We’ll investigate that more later, but since the imminent problem was the draft, which was unambiguously not supported by the API, it was browser use to start.
ESPN doesn’t have an official API. It turns out, though, that there is an API that its frontend uses, which has been thoroughly documented by fantasy football tinkerers. That one does have functionality for the draft, but ChatGPT decided it was too much of a risk to use given that it’s a one-time event. So, as with Yahoo, we will explore the APIs for managing the team later, but it’s browser use for the draft.
Next up — the actual strategy. The two leagues run on different rulesets that require different strategies. The unserious league uses a snake draft, in which the eight participants get randomly seeded 1-8. They pick in that order, then at the end of the first eight picks the order reverses (so person 8 picks twice in a row). The serious league uses an auction draft, in which everyone takes turns nominating a player, then everyone can bid on him, and the highest bid wins. You get $100 to spend, which you can either YOLO on a couple of top players or spread out more evenly.
The snake draft doesn’t require that much strategy — you take what you believe to be the best player available on your turn, making sure that you have the right mix of positions. For that one, I had ChatGPT take a pass at doing its own research and valuing the top 100 players, but it came up with a similar ranking to the ESPN default. That’s not surprising; it’s very clear who the best players in the NFL are.
You can get more value during a snake draft by picking up sleepers at the end. You get these more based on news than historical stats, like the rookie receiver who’s seeing a surprising number of passes thrown his way during the preseason or the backup to a starting running back with an injury of ambiguous severity. I had ChatGPT do the extensive reading of NFL news that I’d normally do myself to create this list, and spot checking showed that it came up with exactly the type of folks I’d want it to.
The same list was useful for the auction draft, but Claude asked some good clarifying questions related to the format. Did I want to get top stars early no matter what or wait until there was good value to be had? Any particular positions or players that I wanted to prioritize?
I let it know that the people in this league tend to overpay for the big stars early, and unfortunately you kind of have to do that too in order to not get shut out of the key players. I told it to watch the first five picks to get a sense of what the premium is, then take a couple of top players at acceptable prices relative to that premium. After that, go for good value (not premium-adjusted) for mid-tier players and pick up some sleepers at the very end.
With that in mind, I let it do a bunch of mock drafts, all of which went fine, so we decided we were ready for draft day.
Both leagues drafted on Labor Day, with the unserious one at 8am and the serious one at 8pm.
ESPN lets you enter the draft room an hour before it starts, so Claude tried at 7am and promptly hit a 404. It figured out that my member ID had to be in the URL (apparently not necessary in mock drafts) and got in with plenty of time to spare.
Before the draft started, I had it give me its top 20 picks. For a snake draft, your first two picks will each be the best player available from your list when it’s your turn, so I’d be able to tell immediately if something went wrong.
On my first pick, something went wrong. The top available player was Jonathan Taylor, but Claude picked Jaxon Smith-Njigba. Claude explains:
What the bot was supposed to do
Before clicking anything, it runs an identity check: find the Draft button carrying the ESPN player id, walk up the DOM to the surrounding table row, then read that row’s name, team, and position and confirm all three match the player it decided on. Only then does it click. This exists so a re-sorted or re-rendered player table can never make it draft the wrong guy.
What actually happened at pick 6
The bot also pre-loads its top targets into ESPN’s draft queue panel — at 8:01 it queued Nacua and Taylor. Nacua went at 1.4, so by our turn Taylor was the top choice (score 484.0 vs JSN’s 475.4).
The problem: the queue panel renders its own Draft button for each queued player, carrying the same data-player-id. And it sits earlier in the DOM than the players table. So the search for “the visible Draft button for id 4242335” found the queue panel’s copy first.
That button has no players-table row above it. The walk-up-to-the-row step found nothing, sat there until the 1.5-second timeout, and threw:
draft attempt failed for Jonathan Taylor: Locator.inner_text: Timeout 1500ms exceeded.
- waiting for ...ancestor::*[contains(@class,’public_fixedDataTable_bodyRow’)][1]...
The identity check failed, so the loop did the safe thing: it dropped Taylor and moved to the next name on the board. JSN, verified, clicked, drafted. Taylor went one pick later at 1.7.
So it wasn’t a wrong pick — it was a skipped one. The guardrail worked exactly as designed; it just fired on a false negative.
Why it never showed up in six practice drafts
Practice runs used a smaller queue or the queued player was gone by our turn. Round 1 live was the first time the #1 board choice was also sitting in the queue panel at the moment we picked. That’s the exact collision required.
The fix
Instead of taking the first matching Draft button, loop over all of them and keep the first one that actually has a players-table row as an ancestor. If none does, fail with a clear message (“only outside the players table (queue panel?)”) rather than a mystery timeout.
Not great! But JSN is still a top player, so not devastating. The rest of the draft went according to plan, and my team is solid.
The evening draft was going to be higher stakes, not only because the buy-in for the league is higher, but also because my daughter’s crib is in the office where Claude would be using my computer. Her bedtime is before 8, so I wouldn’t be able to keep an eye on it. I had the Claude Code session running on my phone via remote control and the Yahoo Fantasy app to take over if needed, but I was slightly nervous about the thing nonetheless.
When the draft opened, I pulled it up on my phone so I could follow along, then got kicked out shortly afterwards with a message that I had logged in from another device. Sure enough, Claude had been kicked off when I joined and had then rejoined, booting me. I was only going to be able to watch via Claude’s own commentary. Fingers crossed.
Luckily, things started off well. It watched the first five picks, saw that people were bidding about 20% over our assigned values for each player, then grabbed a couple of top players at ~15% premiums, so good relative value and correct execution of the plan.
Mid-draft is where it hit the first snag. It had picked up a quarterback early on, which is an atypical thing to do, but someone had nominated one as a bit of mischief, and because nobody wanted to bid up a QB early, Claude secured him for under our value. That’s unusual for how these things go but well handled.
When someone else nominated another quarterback shortly thereafter, a similar situation occurred — nobody wanted to bid, so Claude saw an opportunity to get a player below our value. While the valuation logic was right, it didn’t matter because there was no reason to get another QB at that point.
Claude was evaluating the state of affairs after each turn and immediately realized its own error.
At that point, it was getting late and looking like things wouldn’t go catastrophically off the rails, so I went to sleep. When I checked my team the next morning, it was pretty good (even got a cheap, late-round Travis Kelce pick — hopefully now that the wedding is over he can focus on the game), except for two glaring issues. First, a kicker who isn’t active this week, and second, a player who had been cut by his team the week before.
Oops! But hey, at least Claude handled it like a true fantasy football manager and took no accountability whatsoever for his failings.
The upside is that kicker is the most fungible position, so easy enough to swap out for a new one, and the other guy was a late-round pick that didn’t cost anything anyway.
Overall, I feel like this went well! The failures weren’t too bad, and I’m happy with both of my teams. Last year I was using ChatGPT for most of my research and decision-making but pressing all of the buttons myself, so it’s a good reminder of how much browser use has improved in the last 12 months that I could leave this one with Claude and go to sleep.
Now we see how Claude manages the season, where the choices are meaningfully more complex. I had it take a first pass at setting up a weekly routine of research, lineup setting and waiver claims, but unfortunately its initial proposals in my auction league were to unnecessarily spend 1/3 of my budget for the entire season before a single game is played. So we shall stay human in the loop on big moves for now, but I’ll post an update later in the season when I’ve (hopefully) got Claude managing the whole thing with no input from me.