Case study
Failure simulation and a metric plan for a busy backbone
Summary
The problem
At busy hour one long-distance link ran close to full while parallel paths had room, and there was no reliable way to know what would happen if a particular cable or router failed.
What we did
Rebuilt the IGP topology from a read-only inventory of every backbone router and estimated the traffic matrix from interface counters. Simulated every single-cable, router and LAG-member failure at busy hour, then searched for OSPF metric changes that moved traffic off the hot link without creating a new one.
Outcome
Metric changes ranked and checked against link and router failures, then applied to the network.
In more detail
- New route from A to C
- Backbone
A traffic matrix from what you already measure
Most networks don’t have a router-to-router traffic matrix, and buying a tool to get one is a project in itself. Interface counters are enough to estimate one well: the method fits the demand between every pair of routers to the load measured on every link, and the fit can be checked against links that were left out.
Asking what breaks
With the topology and the matrix in one model, a failure becomes a question you can ask in seconds: take this cable away at busy hour, recompute the shortest paths with ECMP, and see which links fill up. We ran every single cable, router and LAG-member failure and kept the ones that pushed a link past a safe load.
Changing one metric at a time
The optimiser proposes metric changes, but only the ones that improve the busy-hour picture and don’t make any simulated failure worse survive. Each change is small enough to apply on its own and roll back if the real network disagrees with the model.
The same approach is built into Corepath, our traffic-engineering platform.