Le Mans Driver Ratings

by Sean Wrona

Even though I haven't written a column in nearly two months, I have still been bursting with numerous new ideas for the past several months, some of which I do intend to discuss in future columns this month. I know I haven't had a great track record of writing the column I say I'm going to write in each previous column. For instance, I know I stated that I was going to recap this year's 24 Hours of Daytona (now over five months ago) shortly after my Last Year's Rolex column and I didn't, and in my IndyCar column I suggested I would likely do Formula One soon (not to mention there have been seven IndyCar races including the record-setting Fontana race, which I know would be exhausting to calculate the passing scores. However, my interests are evolving and I'm much more interested in inventing new statistics than going through the laborious and time-consuming process of calculating race scores for every major-league series as I previously implied I'd do, especially because I do it more or less manually. Sure, I use a spreadsheet and I'm quite quick at it, but I have not yet undertaken the task of writing a computer program to spot all the lead changes instantly, which is easily the most time-consuming part of these analyses, so after this, I'm leaning towards more new and simplified methods of analysis. Although I still intend to use my current formula for future races, it won't be my main focus.

If I wrote a column on a 24 Hours of Daytona that at that point wasn't even current (as I did) writing a Le Mans column is obligatory considering it is probably the most prestigious race of the season in any series. I was planning on doing the same thing I did for the Rolex by reviewing last year's Le Mans and writing a column shortly before this one started, but because of how time-consuming calculating all this is in a 24 hour race, I was unable to finish either column before the race started. This time, I just scrapped my work on the 2014 driver ratings and started a more current column, which would no doubt be of much greater interest to the motorsports community. While I continue to calculate average running position and natural peak the same way I always have, I've made my definitions for average speed and passing less nebulous. As I did in my IndyCar column, I now base a driver's average speed based on all laps within 5% of the driver's fastest lap, which accurately measures pure pace and thereby ignores pit stops, running out of fuel, mechanical problems, and the like. Following the ideas in my "How the Races Were Won" series, I initially wanted to look for video footage for every single pass where the passer and driver passed had significantly different lap times, but many position changes will not be televised at all, and I don't get Fox Sports 2 so it would be cumbersome for me to attempt something like that at all for the portions of the race (I haven't even watched the FS1 portions of the race yet, which I have recorded). I decided a Potter Stewart-esque "I know it when I see it" approach to determining which passes "count" is not good enough. There was already too much of that in "How the Races Were Won". Mind you, usually passes are cut and dry, but I wanted a specific formula for which passes count. I've decided a pass 'counts' if the driver passed has a lap time within 3% of the passer. Anything more than that in most cases in most series would likely be a case of inheriting the lead, so I counted all passes where there was a 3% time difference or less and only those.

As I mentioned in my Rolex column, my attempt to create a one size fits all metric for measuring race performance, while interesting, is definitely a failure in sports car series with multiple drivers sharing the car. The problem quite simply is that there isn't enough passing. In the entire 24 Hours of Le Mans in all classes there were 111 green-flag passes for position (38 in GTE Am, 33 in LMP2, and 20 in GTE Pro and LMP1...note that I only count passes for position and do not count passing lapped cars, which would be much more frequent). Furthermore, only eight of those passes in all classes happened after lap 100 (and none of those in the GTE Pro class, which is fairly understandable since the class only had eight cars in it). This means that a large percentage of drivers in all four classes neither passed anyone nor were passed by anyone, so the passing metric is fairly screwed up. For the drivers that made passes, it's very effective, but for the drivers who didn't, it's not. Some drivers may not have made any passes because they took over the car when they were already laps down and nowhere near anyone else in terms of track position, some drivers were passed but were more than 3% away in terms of lap time, and some drivers even competitive in the race may not have made any passes because the field was so spread out during the long periods between safety cars that even cars on the lead lap are separated by many seconds. This induces a bias towards whichever driver started the car, because the starting driver has many more opportunities to pass when the field is bunched up, while the drivers who take over the car later (especially the third driver) likely have little chance. Obviously, passing goes both ways, so the drivers with more extreme passer ratings invariably started earlier, so does this really reflect passing in a sports car race or just team strategy (which driver enters the car first)? Furthermore, this directly affects natural peak because natural peak is defined as the highest position obtained on-track via a natural pass or the position on the first lap, whichever is higher. There were so few drivers who had a natural pass at all that natural peak ended up being determined primarily by initial lap position for most drivers, which often has little to do with the driver and more to do with pit strategy and attrition. If the qualifying driver did not start, or if the driver was the second or third driver to enter the car, the initial position has little if anything to do with what the driver actually did. In single-driver series it works because at least the driver qualified the car, but in multi-driver series, it doesn't, and considering most drivers did not pass anyone, both the passing metric (where most drivers scored 0 including some very fast ones AND some very slow ones, rendering a measure of passing meaningless at least in terms of comparing drivers who made no passes) and especially natural peak, which has even less meaning. For the drivers who did make passes (and most of the drivers on the teams fighting for class wins did) the rankings have value, but for those in the middle of the pack, they don't. Considering how much time I spent counting passes, it simply is not worth it for sports car series. Using metrics designed for single-driver series doesn't work. It works to some degree, but not well enough. Ultimately, as I suggested in my Rolex column, I think comparing average speeds is the only thing that matters for sports car racing, because passing is dependent on what part of the race the driver is running, and the final result is usually determined by pit strategy and/or attrition rather than on-track passes because the field is so spread out, so that's hardly dependent on the drivers either. Well, at least in my opinion. I usually am disinclined to blame drivers for mechanical failures, which are most of the attrition in endurance races. I know passing is not what sports car racing is about. It's about innovative cars, but it does make metrics like these simultaneously complex and ineffective. I will continue to use this formula for single-driver series but I think I'll just stick to work based on lap times for sports car series now. There's a lot that can be done simply based on that. Teammate-to-teammate comparisons become way easier when drivers are sharing the same car, for instance, as opposed to teammates in other series, where many fans in Formula One argued that the #2 Ferrari wasn't REALLY as fast as the #1 Ferrari or in NASCAR that the #25 Hendrick didn't have the same speed as the #24 or #48. Having said all that, I still undertook essentially the same analysis I did for the 2014 Rolex and I'm not dumping it just because I don't think it worked. I am going to say for sure now I'm not going to replicate it in this way for any other sports car series, including the 2015 Rolex, which I didn't end up analyzing.

In the premier LMP1 class, despite Porsche breaking the traditional Audi monopoly on the competition this year, Audi still had the best driver. André Lotterer may have lost the race this time but he put on a passing clinic that was out of this world. His passing score of 184 was nearly four times greater than that of the second-best passer and winner Nico Hülkenberg. Lotterer was one of the only three drivers to pass Hülkenberg in the race, and his 6-0 passing record also included two passes each of Loïc Duval and Timo Bernhard and a pass of Neel Jani, with no one passing him at any point in the race. Hülkenberg despite being a Le Mans rookie was the only driver on the winning team to make a pass in the race, but that's because he started the race when the field was not so spread out, and it probably would have been little different if Nick Tandy or Earl Bamber had started the race instead (in fact, they both averaged faster lap times than did Hülkenberg). Timo Bernhard, by contrast, led in natural peak as he held the lead for 18 laps after a natural pass to Lotterer's 9. Even though Porsche beat Audi in the race, it does appear that the Audis remained faster based on the average lap speeds, so I would still probably predict Audi to beat Porsche next year, since I think speed is still the best predictor for the next race. It appears the #19 Porsche primarily won in the pits since none of the drivers led the race naturally, but there's nothing wrong with an off-track win especially when it looks like the Porsches were as fast and you consider how little passing there was in the race in general.

In LMP2, not much happened. The #47 Oreca Nissan of Richard Bradley, Nicolas Lapierre, and Matthew Howson led almost start-to-finish. Bradley had the highest-scoring performance since he made a pass for the lead early in the race while his teammates maintained it, but given that car's steadily consistent performance, Lapierre certainly would have been able to do it since he was over two seconds faster than Bradley, although he didn't need to. A two-second difference is so massive that I would say Lapierre was the true linchpin and MVP of the class (even though he was slightly slower than Oliver Turvey). There wasn't much passing for the lead in the race, so the passing list is wildly different. Bradley places highly in fourth based on his two passes of Sam Bird and Tristan Gommendy, but Gommendy, who had made several more passes earlier in the race, was second with a passing score of 158 to Laurens Vanthoor's 182. Both drivers were also among the fastest 15 in the race (Vanthoor was even faster than Bradley), but a major level shout-out goes to Jon Fogarty, the former perennial Grand-Am Daytona Prototype championship contender, who placed 3rd in passing with a passing score of 102 points based on a 4-0 passing record. What makes this incredible is that Fogarty was only 39th fastest of the 56 drivers in speed. He passed Aleshin twice despite Aleshin being on average a second faster and he passed Roman Rusinov who was THREE SECONDS faster, despite never being passed by anyone in the race. Perhaps the Extreme Speed team had a mechanical problem and he was faster earlier in the race, but it's still remarkable since none of his teammates in either car were doing much passing at all. It's nice to see he's found his bearings after the GAINSCO team closed down. Oliver Webb also did a nice job with a relatively slow car and was sixth in passing.

In the GTE Pro class, the #64 Chevy Corvette drivers Oliver Gavin, Tommy Milner, and Jordan Taylor may have inherited the win when the #51 Ferrari had gearbox issues, but the Corvette drivers were doing most of the passing. Milner led the way in natural peak leading for 22 laps after obtaining the lead (all other lead changes between the #51 and #64 after that were in the pits). Neither Gavin nor Taylor took the lead on track but they both made passes for position regardless of the fact that there were only eight drivers in the class. This was the only winning entry where all three drivers made a pass for position. Taylor was the fastest of the three Corvette drivers despite being somewhat less experienced at Le Mans, but he was only fourth overall as Nicki Thiim led the category in both speed and passing. Taylor also had the highest average running position of any driver, but despite that category counting 30%, Milner's passes of Darren Turner and Davide Rigon on lap 49 were the moves of the race. Gianmaria Bruni in the #51 also made a pass for the lead on Richie Stanaway on the opening lap, but lost it the following lap. The other two Ferrari drivers did not make a pass for position. This category was in some ways more competitive than the others as four drivers led naturally (Milner, Stanaway, Bruni, and Rigon) even though the class was the smallest of all four classes, even if there weren't any passes after Rob Bell's pass of Alex MacDowall on lap 96.

In GTE Am, it was essentially a battle of two cars. The #98 Aston Martin of Paul Dalla Lana, Pedro Lamy, and Mathias Lauda led most of the middle sections. Lamy and Lauda both left the car after making on-track passes for the lead and returned to the car leading. In these cases, I counted all consecutive led laps for them towards their totals for natural peak. Even though Dalla Lana led an entire stint himself in between, I did not do the same for him because he was not leading at the end of his previous stint, so he inherited the lead from his teammate pitting rather than taking it on-track himself. In the end, however, the #72 SMP Ferrari of Russians Viktor Shaitar and Aleksey Basov and Italian Andrea Bertolini reigned triumphant. Bertolini had the fastest average speed in the class, while Shaytar was third. Shaytar however was the MVP in the class since he was 2nd in passing to Matt Griffin, was the only driver besides Lamy and Lauda to lead the race naturally, and he had the best average running position on the team, even though he was behind the three #98 drivers in this category. Once the #98 car went out, Shaytar and Bertolini were obviously the strongest contenders left. They led the team with Basov nearly seven seconds closer, but in a pro-am class, such distinctions are commonplace.

Even though it took me most of back-to-back off days to complete this, it was fun, although I don't really feel up to calculating this much semi-manually for a 24 hour race ever again. As I said, I don't think the overall results are meaningful enough to justify doing these calculations on a regular basis, but I didn't want to do the work and not publish it anywhere, and I do think the category leaders are somewhat meaningful. Now in another couple weeks or so, maybe I'll actually watch the race!

Below are the top ten drivers in each category in each class (I have omitted the finishing positions, which you can find at http://www.race-database.com/results/results.php?year=2015&race=1&series_id=8.

Below are the overall rankings for each class with each driver's score in each category (based on the percentage of other drivers beaten in that category, with a maximum of 30 points for average running position, 20 points for finish, 10 for speed, 20 for passing, and 20 for natural peak, but obviously I am going to rethink that since I now think I have seriously undervalued speed, which one could easily argue is the most important):

Sean Wrona is the Managing Editor of racermetrics.com, the Webmaster of race-database.com, the winner of the 2010 Ultimate Typing Championship at the SXSW Interactive Conference in Austin, and the ratings compiler and statistician for the Mensa Scrabble-by-Mail SIG. He earned a master's in applied statistics from Cornell University in 2008 and previously digitized several seasons of NBA box scores on basketball-reference.com. You may contact him at sean@racermetrics.com.

---