Websites across Europe and North America went down yesterday, apparently thanks to a problem with Juniper routers.
Dozens of websites were offline for hours following a bug in a Border Gateway Protocol firmware update for the routers that caused a network failure at backbone provider Level 3 Communications – one of the key internet routes in the US.
The outage, which hit UK internet service providers and RIM’s BlackBerry services, amongst others, caused devices to core dump and reload. The company’s since issued a fix.
“Shortly after 9 a.m. ET today, Level 3’s network experienced several outages across North America and Europe relating to some of the routers on our network,” says Level 3.
“Our technicians worked quickly to bring systems back online. At this time, all connection issues have been resolved, and we are working hard with our equipment vendors to determine the exact cause of the outage and ensure all systems are stable.”
An advisory posted anonymously on Pastebin says Juniper’s MX series of routers was affected.
“Junos versions affected include 10.0, 10.1, 10.2, 10.3, 10.4 prior to 10.4R6, and 11.1 prior to 11.1R4. The trigger for the MPC crash was determined to be a valid BGP UPDATE received from a registered network service provider, although this one UPDATE was determined to not be solely responsible for the crashes,” says the post.
“A complex sequence of preconditions is required to trigger this crash. Both IPv4 and IPv6 routing prefix updates can trigger this MPC crash. nThere is no indication that this issue was triggered maliciously.”