There has been some debate about the jittery gauge chart we used in our live election forecast. Rather than replying to dozens of tweets I decided to wrap this up in an old-fashioned blog post™. Feel free to add your thoughts to the comments.
straight up: the NYT needle jitter is irresponsible design at best and unethical design at worst and you should stop looking at it
— Richard Porczak (@tsiro) November 9, 2016
Why did we let the gauge needles jitter?
We added movement in our gauges for two reasons.
First, we wanted to convey the reality that our forecast was “live” – connected to a live data feed and updating continuously, without any need for a user to refresh his or her browser. Movement helps convey that reality. This technique is common across the web, whether it’s on a site like chartbeat or a live gamecast of a baseball game. A “live” web page where nothing moves feels quite dead to the user.
Second, we thought (and still think!) this movement actually helped demonstrate the uncertainty around our forecast, conveying the relative precision of our estimates. In our opinion, having the dial fluctuate a small amount – bound by the 25th and 75 percentile of simulated outcomes – was more successful at conveying the uncertainty around our forecast than simply listing what those percentiles were. As the night went on, the gauges fluctuated less and less as our forecast became more precise. By the end of the night, the gauges barely moved.
Why didn’t you just show the chance of winning?
The chance of winning is an important indicator that includes the uncertainty of the forecast. So why was that not enough? Well, the problem is that most people aren’t very good at interpreting a probability scale. While it is relatively easy to understand that a 50:50 chance is pretty high, and that a one in a million chance is pretty low, it is very hard to make sense of probabilities in between. How good is a 75% chance of winning? How much better is 85%? So rather than displaying just the chance of winning we wanted to show likely outcomes both in vote margin and electoral votes.
Why didn’t you just display the median outcome?
Because we thought that most people would read the median outcome as the one most likely outcome, as opposed to one of many very likely outcomes. Seeing the needle actually walk to the lower and higher end of the projected range makes us aware that these are real possibilities.
Was it really just random jitter?
The jittering was random, but the jitter range was not. The range was fixed between the 25th and 75th percentile of the simulated outcomes. Early in the night, the uncertainty was higher and the ranges wider. Towards the end of the night they got more narrow. The range was displayed as pie chart slice on the gauge, along with the 5-95th range.
Why did the needle movements look almost real?
To make the movements of the needle look more real we used Perlin noise, an algorithm that originates in computer graphics to simulate natural textures like clouds. We used the implementation from processing.js.
Why was the jittering stressing out a lot of people?
Part of the answer is that we simply don’t feel comfortable being left in uncertainty about something as important as this election. We like to feel safe and cozy, being told our favorite candidate has a “narrow but persistent” lead, but we don’t like being confronted with likely scenarios in which the candidate might actually lose. Of course, the bigger reason for the stress was probably the outcome of the election itself.
Update: here’s another take on this subject by Andy Kirk