Monday, November 27, 2023

WSC 2023 Cruiser Wrapup

Another week, another blogpost. This time: Cruisers

Like we'd speculated in the Day 4 post, because everyone failed to finish the leg to Coober Pedy, only the first two days of the event (the Darwin to Tennant Creek stage) "counted" in the team's scores. Some teams just kept on truckin' anyway - Minnesota, Solaride, Ascend, and Apollo drove most or all of the distance after Coober Pedy anyway even though nothing mattered anymore (good for them), while teams like Sunswift, Flinders, and Sun Shuttle packed the car in the trailer all the way to Port Augusta before driving just the last leg.

Ultimately, Sunswift came out on top. With an efficiency score of 109.44, a practicality score of 83.3%, and no time penalty, their final score was 91.17.

Their nearest competitor was Minnesota with an efficiency score of 51.98 and a practicality score of 72.6%, good for a final score of 37.74 - except with an 0.59 penalty multiplier for being 52 minutes late at TC, bringing their score down to 22.38.

In third was Solaride with an efficiency score of 39.44, practicality score of 71.8%, un-penalized score of 28.32, but an 0.52 penalty multiplier due to being 65 minutes late at TC for a final score of 14.73.

I've been waffling about how much of a rant I should post here: how many feathers do I want to ruffle and how much I should censor myself, but I've just gotta get it out... IMHO the Cruiser Class has a ton of problems. Warning: Some saltiness ahead.

How the Rules Define The Race, or: The Failure Machine

Background: Currently, the rules define the score as "Efficiency * Practicality * Penalty", where "Efficiency" = Person-KM/External_Energy, "Practicality" is a value between 100% and 0% that is subjectively judged at the finish line, and "Penalty" = 0.99^Minutes_Late. Each stage (Darwin to Tennant Creek, Tennant Creek to Coober Pedy, and Cooper Pedy to Adelaide) has a target finish time and teams start accruing penalty minutes for missing those target times.

I sort of understand how WSC got to where they did with the scoring equation. The 2013/2015 sum-of-factors was kind of a mess: it was very unclear (even to me!) how a team should balance the various goals (minimizing energy usage, maximizing person-km, and going fast) to score as well as possible on the on-road portion, even though all of these boil down to "build a more efficient car". Locking at least one of those numbers would allow for a clearer on-road performance metric. The Cruiser teams have repeatedly expressed that they want to have freedom with the grid energy charging and number of occupants, so WSC drew a line in the sand with speed and gave the teams a fixed target time. But my opinion is that the fixed target speed/time turns the event into a machine that encourages teams to fail.

The Challenger class goal is to go as fast as possible, which places them as far away from failing to finish the event as they can get. If a team has unexpected issues, they fall back and don't finish as well as they'd like, but they have been incentivized for the entire race to build as much margin as possible ahead of the failure line. Meanwhile, the Cruiser class goal is to minimize energy usage within the target time, and that means driving as slowly as allowed. And there's not very much margin between the target time and the failure time: three hours at Tennant Creek, two and a half hours at Adelaide, and only half an hour of margin at Coober Pedy - after the longest and most challenging leg of the race for the Cruisers. That's not a lot of wiggle room between "ideal" strategy and failing out of the event!

If it's a rough year for weather (like this year, with smoke and headwinds in the middle), maybe fewer Challengers finish but at least we still have a full podium (and then some). But with the Cruisers, all the teams were planning to toe the failure line, and instead trip and fall over it together.

"But they should just have planned to drive faster in the first place to build margin for themselves!" OK but A) Going faster uses more energy. For bigger, heavier, less aerodynamic cars like the Cruisers, a lot more energy. That means more battery, which is more weight, which needs a stronger chassis to carry the loads, which is even more weight, and now the car needs heavier suspension to support all of that, and needs more battery to power the heavier car, and oops now you're in a weight spiral. A team that designs with energy margin for a year with bad weather is going to score poorly on efficiency on a year with good weather because they're going to expend more energy lugging all that extra weight around ("But they should just design a more efficient car instead of piling in more batteries!" We'll... get to that). And B), even if the teams build appreciable time margin into their strategy... if it looks like the efficiency scores are going to be close, the teams are going to want to pay off that margin and slow down to reduce unnecessary energy expenditure in order to improve their scores. Two teams can end up playing chicken with one another until they both run off a cliff. Even if a game of chicken doesn't happen, teams that build schedule margin early are still going to be tempted to slow down and start reducing their margin in order to increase their efficiency scores as they near the end of the stage. And if they break down at the wrong time... Tokai had to spend two hours on the side of the road with a suspension issue this year, and they still finished well. If they'd have been a Cruiser team? That's it, they'd have been out.

Let's look at how fast the Cruisers are expected to drive: in 2013 and 2015, teams could at least finish the Cruiser class at a ~60kph pace, and the target speed in 2017 was ~65kph. But in 2019 and 2023, the target speed on the TC to CP leg is 75kph and the fail-out-of-the-event speed on that leg is 72kph - that's the overall pace that Top Dutch set this year in 6th place in the Challenger class, so that's a pretty high bar to set for the cruiser teams! And in fact, if we look at the Challenger teams interval times on the TC to CP leg, that's where all the Challengers drove the slowest this year due to the headwinds and smoky skies. If we'd held the Challengers to the same schedule as the Cruisers, we'd have had 4 finishers this year instead of 12: only Innoptus, Twente, Brunel, and Michigan would have completed the second stage. Sonnenwagen's interval time is 37 minutes longer than the Cruiser cutoff time, Top Dutch's is an hour and seven minutes too long, Tokai would have shown up two hours and thirteen minutes past the cutoff time (due to their aforementioned suspension issues making them park on the side of the road for 2+ hours), and every other team behind was far too slow. That's rough!

This exposes the difficulty of running a staged event simultaneously with an un-staged event, while holding the staged cars to the same overall time limit as the un-staged cars. The Challengers have flexibility to allow their speed to wax and wane over the entire length of the race; it doesn't matter as much if the entire field has a slow day or two working through a weather system. But the Cruiser class is a series of 2-day sprints instead of a 6-day marathon, and failing a single one of those sprints causes a team to fail out of the event. Even if the Cruiser class became an elapsed time event while sticking to the current staged format, it would still be a harder event to complete than the Challenger class. The Cruisers would never be given the opportunity to build much margin past the failure line: they park for the day at Tennant Creek and Coober Pedy no matter how early they arrive, and any margin they've built up over the stage goes away.

Watching the Event

The target time is no fun as a spectator. In the ideal scenario, all the Cruiser teams are at essentially the same position on the road, and there's no way to tell from road position how they're doing on the things they're actually scored on. The only thing you can tell during the race day from road position is a binary "are they failing out of the event or not". Spectating the road position of the Challengers is fun and exciting, we're cheering as our favorite teams charge ahead and groaning as they fall back, and eagerly watching tight contests. But when I look at the road position of the Cruisers, there's nothng to cheer about; I'm just peeking out from between my fingers with dread.

It's also impossible to write about or explain to people how the teams are doing over the course of the event - even to technically-minded solar car alums! Absent road position mattering, we only really get a snapshot of how the teams are doing every two days, and it's "low resolution" summary information. For instance, as far as I can tell there's no publicly-accessible record of how many people the teams carried over each control stop interval, the results page is only set up to show us the total energy used rather than the amount charged at each stage stop, etc. The scoring is even difficult to explain in a useful way after the fact - witness ScientificGem's variety of different attempts over the years.

The Teams and the Cars

As a cranky old solar car designer with a competitive streak, when I survey the field of competitors, it feels like the teams are more confused by the rules than anything else. The cars don't seem to be designed to actually do well under the rules of the competition, and that's intensely frustrating.

I would like to start by pointing out: this year the Cruiser cars finished exactly in order of "efficiency". We could eliminate practicality, and even the minutes-late penalty time, and the 6 cars that completed the first stage are ranked in exactly the same order. And this isn't a fluke: The three cars that finished the event in 2019 ended up ranked in order of efficiency, and ditto for the three cars that finished in 2017. As long as we've had the efficiency metric and the practicality metric, the spread in efficiency scores has always been large enough that the spread in practicality scores was not large enough to alter the placement of the cars! If you want some numbers: the spread in efficiency this year between the 6 cars that made it to Tennant Creek was 6.28x. The efficiency spread between the top two cars alone was 2.11x! That's about as big as the total spread in practicality scores has ever gotten: 1.48x in 2023, 1.74x in 2019, 2.38x in 2017, 2.36x in 2015, and 1.76x in 2013.

It sure seems to me there is a ton more room right now for the teams in the Cruiser class to compete on efficiency rather than practicality! Yeah practicality being a score multiplier that can technically be zero in 2019 and 2023 is big and scary, but efficiency is a multiplier as well in the scoring equation! In practice, practicality varies between about 0.5 and 0.9 rather than between 0.0 and 1.0, but the efficiency score sure as shit can and does go to zero if your inefficient car fails to finish the event! So why does every single cruiser car nowadays look like it was designed with the practicality portion of the score as the primary concern and efficiency as a secondary concern? Why are they all trying to copy "normal car" styling cues to the detriment of efficiency?

There is something about how the scoring is set up that seems to be obliterating "solar car thought" from the teams that are entered in the Cruiser class. Post-event this year, some WSC officials brought up that they really aren't stoked by cars with 40, 50, and even 60kWh batteries in the Cruiser class nowadays (at this point it's basically an EV race with fig leafs symbolic solar cells on top of the cars), and proposed bumping the Cruiser array size up from 5m² to maybe as high as 8m². The response from some team members was "no, we have enough trouble fitting 5m² on the car as is". What??? That ain't solar car thinking! They should be absolutely stoked at the idea of getting more solar generation on the car, getting heavy batteries out of the car, and getting into a virtuous weight cycle in their designs. Teams had to fit 6m² of array on 4.5x1.8m Cruiser in 2013, they should be able to figure out how to fit 8m² on the current 5x2m cars. Another comment was "it's just too much to expect us to be able to go 1200km from Tennant Creek to Coober Pedy without recharging, we need to add back the third grid charge location in Alice Springs". Give me a #$%@ing break, the Challenger cars do Darwin to Adelaide without charging off the grid! They do it with comparatively eensy-weensy batteries, too. And y'all know it, you've seen it! You were at the same event! You're sitting there with them at the finish line and saying shit like this!

To refresh all of our memories about what the Cruiser teams can do (when they want to), I went back and looked at the 2013 and 2015 Cruiser average speeds, when elapsed time mattered in their scores. In 2013, the teams had ~16kWh batteries and could charge in TC, AS, and CP. That's a total of 64kWh available - well less than the 100kWh++ that teams were on track to use this year, had they finished - and three teams finished with average speeds >>80kph over the course of the whole event. But OK, they had three charge locations rather than two. So let's look at 2015, when the teams were limited to ~15kWh batteries and they were only allowed to charge off the grid once: in Alice Springs. That's only ~30kWh available over the entire race; less than any of the teams this year had in their battery packs on the start line! Kogakuin averaged 90.59kph over the whole event. They used less grid energy to go 3022km in 2015 than any team used to go 985km this year, and they were going a solid ~30% faster! Teams are complaining that Tennant Creek to Coober Pedy is too far to go without recharging? In 2015 they had to go 25% farther than that - twice! If your response is "Kogakuin's 2015 car was basically a cheater-challenger car disguised as a cruiser, they only had a single person in the car the whole way and it got a bad practicality score", well: Eindhoven averaged 86.62kph - just a smidge slower than Kogakuin - on the same amount of external energy, had two people in the car the whole way, and got the highest practicality score that year! It is totally possible to build a multi-seat solar car that goes fast and has a small battery and doesn't charge off the grid frequently.

Ok ok Eindhoven may be an outlier team, they've dominated every year, so let's look at Kogakuin again. Even if you don't like Kogakuin's strategy of favoring efficiency and basically neglecting practicality in their design, it was a good one that netted them 2nd place overall in 2015. And practicality becoming a multiplier in the 2019/23 score equation doesn't actually wreck this car; a highly-efficient car like that would still place extremely well under the current rules! Let's look at how it would do in 2023: If we took that car with its small 15kWh battery, gave it the same practicality score from 2015 (which would be the worst by a decent margin in 2023!), filled both seats (they only put one person inside in 2015, but we're going far slower + less distance between charge locations), and it only finished the first stage like all the other cars this year, its efficiency score would be high enough to offset the low practicality such that it would still finish 2nd overall in 2023. And by a pretty wide margin over the next car! Sunswift's Efficiency*Practicality score was 91.17, Kogakuin's theoretical 2015-car-in-2023 would have scored 67.97, and Minnesota's score was 22.38 (if un-minutes-late-penalized, it would have been 37.74).

And if Kogakuin actually finished the event, which that car could totally do at a 75kph pace even with the weather this year... It would have, y'know, won the 2023 Cruiser class, regardless of how far WSC nuked its practicality score?


Author photos of Kogakuin at WSC 2015

It would have done extremely well in 2017 and 2019 too, in years where it would not be the only finisher! Remember its performance in 2015: 90kph, one person, 30kWh. In 2017, if they stuck two people in the car and kept their total energy to under 24kWh - plausible considering that they'd only need to drive 65kph to meet the target time that year - they would have won! (it's also they likely didn't use a full 30kWh in 2015; that year charging was not metered and the score just assumed a full pack every time the car was connected to the grid. It's unlikely their battery was COMPLETELY empty when they charged in Alice Springs. Of course, charging was not metered in 2017 either, so they would have needed to shrink from a 15kWh pack to a 12kWh pack). In 2019, they would have needed two people in the car the whole way and to keep external energy under 30kWh to win, maybe more of a stretch at 75kph - but they'd have had the advantage of net metering that year, so it still seems plausible? And even if they couldn't pull off the win in either 2017 or 2019, the chasm in scores between 1st and 2nd in those two years was such that they'd have had 2nd place completely locked up.

Any team can look back at Kogakin's performance in 2015 and game out how they'd do under the current regs. So why aren't many (any?) teams building hyper-efficiency-focused Cruiser cars? Knowing everything we do about how the efficiency scores have played out vs the practicality scores, why are all the teams showing up with cars that appear to be emphasizing (imagined) practicality over (real) efficiency??? It's just inexplicable to me, it makes me want to tear my hair out.

Every now and then we get to see something good. For instance, Onda Solare's Emilia 4 at ASC 2018 was far more efficient than everything else that showed up, and ran away with the event. That car was extremely lightweight and had a clear focus on aerodynamics.

Emilia 4, via Facebook

See also Polytech Montreal's Esteban X at ASC 2022 - this car was extremely lightweight, it absolutely clowned on everyone's efficiency scores, and only lost due to getting DQ'd on a technicality (they got lost and took a longcut, inadvertently skiping an eensy portion of the route in the process). This car only had a 9.2kWh battery, and only slurped 3.3kWh off the grid over the course of the event. That's a real solar car!

Esteban X. Sharp trailing edges? On a Cruiser car? Yes, it's possible!
image source

I was also really interested by ATN's Pricilla at WSC 2023. Unfortunately suspension issues prevented them from finishing the first stage so we never got to see performance numbers, but it's still recognizably a solar car: Prioritizing flat array area over cosmetic curvaceousness, skinny wheel fairings, relatively slim trailing edges... This is the kind of car that I expect to see more of, but don't.

via Facebook

via Facebook

And of course, Eindhoven has dominated every prior WSC Cruiser event due to being wildly more efficient than everyone else. Yes, they've always topped the practicality scores, but never by very much - the vast chasm in the final scores between them and 2nd place has always been due to the difference in efficiency.

But this year, man... we really didn't get any super efficiency-focused cars in the Cruiser class. The batteries were all larger than before, the cars were all h e a v y, and they all had giant blunt rear ends that I'm still scratching my head over. A member of one team tried to argue with me that it was actually more aerodynamic, and, c'mon buddy. Sure, a Kammback is better than some other blunt shapes but it's still worse than a nice sharp airfoil; you don't see any Challenger teams making Kammbacks. Cessna or Cirrus aren't designing light aircraft with Kammbacks.

The only explanation I have for why the Cruiser teams seem to be lacking a focus on efficiency is a combination of A) teams look at the practicality multiplier in the 2019/2023 score and worry that WSC will put their finger on the scale at the finish line practicality judging to nuke their score to zero if WSC feels like their car wasn't in the "spirit" of the Cruiser event, so they focus on "soft" practicality considerations like "normal-car-like-aesthetics" more that the historic spread in Efficiency and Practicality scores implies that they should, and B) "you should use less energy than everyone else" is somehow a worse incentive to design an efficient car than "you need to go faster than everyone else".

Where do we go from here

Man, I dunno. I'm just monday-morning quarterbacking here, I fully admit that it's a lot easier to critique an event than it is to design a better one. And I don't have a lot of good answers here that won't make everyone mad in some way. The teams have repeatedly expressed the desire to keep the design space wide open in terms of batteries and number of occupants; I think this is one of those situations where people just don't know what's good for them. So if I was king:

My strongest opinion is that it should go back to an elapsed time event where the goal is to go faster: it's better for the event to encourage the teams to get as far away from failing as possible, it makes the event way more fun to watch and easier to understand as a spectator, and (I think) it actually made the teams care about building efficient cars in the early years of the Cruiser class in a way that the current target time fails to do. But if we re-introduce elapsed time variability, at least one of the other on-road factors has to go.

I'd scrap person-km altogether; just fix the number of people in the car and declare all seats have to be occupied at all times. The "allow teams to pick the number of seats" has never really been interesting to me from a competition design perspective. The team with the most seats occupied has always won the Cruiser class, the scoring equation for the past three events has very clearly favored "as many seats as possible", I don't really see much point in continuing to allow variability here going forward. And hey - if you start off with a full car in Darwin, but can't carry every person in the car all the way to Adelaide, that doesn't seem practical to me? You shouldn't require some of your passengers to hitchhike part of the way? The troll in me wants to say we should require three seats, because it's not immediately obvious what's the best way to arrange the seats and we'd probably get a fun variety of solutions.

Let's give them 8m² of array to play with, so they have a chance to be solar cars again. But remember: cars with 6m² of array (and one or two occupants) in 2015 were "too fast", so we'll need to be careful with how much external energy to allow.

Speaking of external energy, I'd really prefer to lock that down as well: no grid charging, no battery size variability, lock them into a fairly small battery, and the score is just something like "80*normalized elapsed time + 20*normalized practicality" or "normalized elapsed time * practicality, where practicality ranges from 1 to 0.5", or something similar. That would also let us eliminate the staged nature of the Cruiser event to give them more flexibility with their speeds over the course of the entire event. But, this might result in Cruisers beating Challengers to Adelaide, and I've been told that's a big no-no.

So one option would be to leave the battery size open and give 'em one grid charge location in Alice Springs: staging all the Cruisers up there on the third night means they finish about a day behind the lead Challengers no matter how fast the Cruisers manage to go, and I think teams have a better chance of success with two 3-day stages instead of three 2-day stages (the "fail out of the event" speeds are 60.9kph for 3 days from Darwin to Alice Springs and 61.1kph for 3 days from Alice Springs to Adelaide; far more forgiving than the current 72.4kph minimum speed for 2 days from Tennant Creek to Coober Pedy). In this case you would need some sort of exponential term applied to the energy usage part of the scoring equation to make burning energy to go faster have diminishing or even negative returns past a certain point; imho you'd want the "sweet spot" in the score to be an elapsed time around 75-85kph average speed. You don't want the ideal strategy to be to toe the speed limit regardless of energy usage (kind of like it almost was in 2015). But this may be a hard target to hit in the rules...

Or, just don't worry about that at all? Peg the battery size small enough that even with a grid recharge in Alice Springs, it's unlikely that anyone can sustain speeds that are toeing the speed limit? Eliminate "external energy" from the scoring entirely - like with the Challengers, just assume that it's in every team's best interests to max out the (small) battery size and also in their best interests to use the charge opportunity in Alice Springs to the fullest (This is what happened in 2013 and 2015: everyone's energy score was the same, ~modulo some eensy variations in pack construction). If a team messes up their strategy such that they haven't gotten close to the bottom of their pack by Alice Springs, or if they have electrical gremlins and can't charge all the way up to a full pack, well, that's their problem. Now the team's elapsed time to Adelaide is essentially a direct measure of the "efficiency" of the car and team. This option would be my personal preference.

Or if we must judge the cars on a variable amount of external energy, another fun idea would be to allow grid charging at multiple control stop locations... but grid charging can only be done on the clock between 8am and 5pm; no grid charging in the morning and evening. If you want to grid charge, you pull out of the control stop after serving your 30-minute time and pull into a metered grid charge area. This would force some hard decisions from teams about if time or energy is more valuable... but could also be confusing for spectators, confusing for team strategy, and could lead to some drama for officials if the grid charge equipment doesn't provide the charge rates that was promised to teams... Maybe table this idea for now.

Alright, I think I've typed too much about this already. See y'all when the WSC 2025 season starts heating up.

No comments:

Post a Comment