The Problem of Evaluating Coaches

Is Bob Bowman a good coach? Probably.

Yesterday, at the Golden Goggles, Bob Bowman was awarded the title of "Coach of The Year". Earlier this week, I wrote about the departure of Paulus Wildeboer from Denmark, almost universally acclaimed as that country's best ever national team coach.  Yet, if you were to anonymously poll American swim coaches about Bob Bowman's success, I think a significant amount of them would say that Bowman was nothing more than a jockey to Michael Phelps' horse for these past twelve years, while just weeks ago prominent members of the Danish swimming community were calling for Wildeboer's head. All of which begs the question: how do we know if a coach is any good?


It's a vexing question in a sport full of definitive answers. Races are swum and the fastest time wins. We know who the fastest male 200 freestyler in the world is (Yannick Agnel), especially with how little doubt he left in London and since. Coaches are much harder to evaluate. They are a layer removed from the times swum in the pool. The rules for their competition are far less clearly defined.

The most common shorthand is how fast a swimmer you coach. By this criteria, Bob Bowman is definitively the best coach of the last decade because Michael Phelps has been so dominant over that time period. Todd Schmitz has seized that baton on the women's side for the time being as Missy Franklin's coach. In Omaha this summer, wild rumors flew that Schmitz was demanding an assistant coaching position at whatever school Missy committed too. This turned out to be pure fallacy, but the rumors held weight only because Schmitz coaching mojo will undeniably take a hit when Franklin leaves his program, which is incredibly silly when you think about it. Will he somehow become far worse at coaching?

People who want to look a layer deeper will look at overall team success. This too, has some serious drawbacks. The most successful club teams in America all draw from gigantic club numbers. Sheer probability predicts that they will have successful swimmers in the higher ranks. Basic arithmetic tells you that the coach of a 70 swimmer club will never best a 1200 swimmer club in any meaningful club championship meet. In college, the coaches of the day are the duo at Cal: Teri McKeever and Dave Durden. In recent memory, Durden led the University of Maryland Swimming and Diving team (RIP) to middle of the ACC results- and yet now he seems to be building and NCAA dynasty. I bring up Maryland not as a knock on Durden, but more to point out that the list of coaches who could possibly build an NCAA dynasty at Cal is considerably longer than the list that could do so at Maryland. McKeever? I'm such an admirer of her results I can't begin to say anything objective about her.

The last metric, for the hardest core, is improvement, and yet this still comes up short. When I was a collegiate swimmer in Division 3, I admired a lot of coaches for their ability to get huge improvement from swimmers at that level. Those raw time improvements happen far less at higher levels. Does this mean that Division 3 coaches are better than Division 1? No, although I do think that the results of Division 3 coaches are dismissed far more than they should be. Slower swimmers are, on average, easier to improve.

Perhaps the biggest problem in evaluating coaches is the truth that coaches desperately have to avoid: they don't determine a majority of why a swimmer performs well or not. In fact, they may be only contributing a very small piece. The athlete's themselves, their growth, their maturity, their determination play the biggest part, and you can only augment these characteristics so much. I say they have to avoid thinking about this, because it's pretty demoralizing to show up to work everyday telling yourself "I'm not going to make much of a difference today!". As a coach, you have to believe irrationally that you can, through your own singular efforts, make someone better.

All that to say, that if you were to combine all the metrics above, assigning proper weight to each, you might begin to understand which coaches are better than others. It might be akin to lighting a match in a pitch black room but it's better than wandering around in the dark.