Initially, I measured the number of seasons each of the players had with a WAR over 10, 8, 5 and 3. I used these values because they seem like good cutoff points. Why, though? I'm not sure, really, and as I re-read my post, they just seemed too... arbitrary. Of course, the whole process is a bit arbitrary. Drawing the line from what makes a Hall of Famer and what doesn't is ENTIRELY the preference of the person making that choice. If you're a "small Hall of Fame" voter, and believes that the Hall should only be for the Ruth/Mays/Aaron/Williams types, and you vote that way consistently, that's fine. As long as someone puts thought and effort into their methodology and is consistent with their standards, then that's all I can ask for.
However, "consistent" shouldn't mean "unchanging" in this case. As new information becomes available, of course it should be considered. The case I've used a few times this year is regarding Mike Fast's work at Baseball Prospectus regarding catcher defense. If we discover new information, we should use it. At the same time, if a methodology is developed for rating players, and a flaw is found in that, it's ok - in fact, it's necessary - to change it.
So I went back through my study, and realized these WAR season scores were just TOO arbitrary. Why was I drawing the lines where I was? No reason - they just seemed like good places to draw lines. However, with only one season of a first baseman having a 10+ WAR in our study, maybe that's too arbitrary? So I decided to put every season of our 25 players back into a spreadsheet, and instead of WARs of 10, 8. 5 and 3 as dividers, I'd use percentiles - the 99th, 90th, 75th and 50th.
This isn't perfect. It includes seasons at the beginning and end of careers when players weren't regulars or ended up stopping play. Still, it is BETTER. "Meliora" - that's what we're shooting for, right Yellow Jackets? If we can do better, then we should, even if better still isn't perfect.
Once completed, we're left with 396, sorted by WAR. the 99th percentile is 8.8, the 90th is 5.9, the 75th is 4.1 and the 50th is 2.0. These are still technically arbitrary cutoffs, of course - they're just drawn where they are for a reason.
Compare this to our chart in our original post - Frank Thomas, who had no 8+ WAR seasons, had six where he's in the 90th percentile - he now gets credit for that. To me, this is just a more sensible measurement.
This changes our final results so slightly that my conclusions are the same. Thomas, Bagwell and Thome are still the definite choices, arguments can be made for Palmeiro and Helton, and McGwire and Giambi are, again too far off:
Who knows - I may change this even more as I move on. But for now, I feel like it's an improvement over my previous results.