That's right — if your app needs 80 dynos with an intelligent router, it needs 4,000 with a random router.
If that seems like an outrageous figure, recall the birthday paradox, which shows that you only need to get 23 people in a room before there’s a 50% chance that two of them will share a birthday. The birthday paradox is called a paradox because nobody quite believes the number’s going to be that small; our minds vastly underestimate the probability of hash collisions.
See also the below graph, which plots the probability of all dynos being open (i.e., no collisions/queuing) given a certain number of simultaneous requests. This is with a naive router. (With an intelligent router, of course, this would look like a flat line at 1.0, since you’d never having any queuing, no matter how many requests you had.)
Here’s another “intuition pump.” Imagine you’re playing a game of Russian roulette. You’re holding a revolver with n chambers. All but one are loaded.
Before you put the barrel to your temple and pull the trigger, you’re given a choice between two methods for spinning the cylinder: (A) spin until you reach the first empty chamber, or (B) spin randomly. Which do you choose?
Unless you have a death wish, you choose A. Of course you choose A. And the larger n gets, the more you want to choose A, because as n grows, so too does the probability that a random spin will land on a bullet.
The analogy back to Heroku’s two routing regimes should be clear.
To help improve the meaning of these lyrics, visit "Heroku's Ugly Secret" by James Somers (Ft. Andrew Warner, ATodd, Chrissy & LEMON) and leave a comment on the lyrics box