Sense
Detect gray failures
Active path probes between leaves; a path is marked bad only when several leaves agree. Interface counters are a second, slower signal that is logged.
Sense the fabric, route around failure.
See the demoIllustrative simulation based on lab runs. Not a live measurement.
Click a link to cut it, or click a spine to make it silently drop traffic for one destination (10.0.1.0/24). The times, outage and failed-request counts replay measured runs from our emulated lab: veth links, emulated GPUs, FRR 10.4.5, BFD 50 ms × 3, 4 leaves and 4 spines. The lab cut was on leaf3–spine2 and the gray failure on spine2; the simulation reuses those runs for whichever link or spine you pick.
flow rerouted flow flow on the failed path
The simulated clock runs slower than real time for the sub-second outages and faster for the 9 s one; it stops at the outage the lab measured. The outage ends at the first leaf route change that moves traffic off the failed path, or, if traffic is never rerouted, when probes see the path clean again after the lab removes the fault. BFD and BGP run with the controller ON and OFF; OFF means it observes only.
When a link goes dark, BFD notices within about 100–150 ms in our lab (50 ms × 3), BGP withdraws the route and ECMP sends traffic elsewhere. That part of the fabric already works well.
A path that silently drops traffic, for example a corrupted forwarding entry on a spine, keeps its BFD session and BGP routes while links stay up. ECMP keeps hashing flows onto it. Training jobs can stall on retransmits and timeouts, and nothing in the control plane says why.
Product: Bixder Fabrisent. Sense the fabric, route around failure. Bixder Fabrisent has two halves: Sense and Act. NIP is a separate product, in development. Lab results on this page cover detection and failover only.
Detect gray failures
Active path probes between leaves; a path is marked bad only when several leaves agree. Interface counters are a second, slower signal that is logged.
Drain and reroute
Traffic is moved off the bad path with an ordinary BGP inbound policy change. Every step is timestamped. Once the fault clears and probes show the path clean for a short hold (3 s in the lab), the policy is withdrawn and the path rejoins ECMP.
Separate product. In development, not yet available
A read-only review of a network before a change.
Active path probes between leaves; a path is marked bad only when several leaves agree. Interface counters are a second, slower signal that is logged.
Traffic is moved off the bad path with an ordinary BGP policy change. No new protocol and no special switch hardware.
Every step is timestamped. Once the fault clears and the path checks out, the policy is withdrawn and the path rejoins ECMP.
Headline metric: network outage time, from fault injection to the first kernel route change that moves traffic off the failed path1. Ranges come from repeated runs. Single runs are examples. ON means Bixder Fabrisent acts. OFF means it only observes. BFD and BGP run in both.
A spine silently drops traffic for one destination; links, BFD and BGP stay up.
With it OFF, BFD and BGP never react. Traffic keeps hitting the bad path until the fault is removed at 8 s (the outage runs about 1 s past that). This is where Bixder Fabrisent makes a difference.
Example runs: ON 250.7 ms (the slowest of the 18), OFF 9039.6 ms2.
A silent cut: the link stops carrying traffic.
No measured advantage. The ranges overlap. In the controller-OFF runs BFD and BGP did the rerouting, and the outage was about the same as with it ON (OFF n=3, small sample).
Example run: ON 207.2 ms2. One more ON run was left out of the range3.
tools/outage.py. It is a lower bound and does not include client timeouts. If traffic is never rerouted, the outage ends when probes see the path clean again after the fault is removed (8 s), so the OFF figure reflects the lab's fault duration; it means 'not rerouted', not a measured time to recover. Lab runs on 29 Oct 2026. ↩metrics.json for each run. Run ids and hashes are in the lab's run archive. ↩A two-minute walkthrough of the lab: a fibre cut and a gray failure with the controller acting, a controller-off comparison for each, and the result that a plain cut is rerouted by BFD/BGP either way. The video was recorded under the earlier working name Fabric IQ; the product is now Fabrisent.
Screen recording of the lab dashboard (2 min 6 s, 1280×720) with narration.
Network teams running AI data centers and neoclouds on commodity BGP fabrics (the lab runs FRR; SONiC is not tested), who want gray failures found and routed around without changing hardware or protocols.
BGP / FRR Leaf–spine, multi-rail