ON THIS PAGE
 

EVPN-MPLS Type 5 Unequal Load Balancing

EVPN-MPLS Type 5 UCMP balances traffic using generalized weights from the EVPN link bandwidth extended community. You can tune these weights with export policies.

EVPN-MPLS unequal-cost multipath (UCMP) load balancing enhances network efficiency by enabling traffic distribution based on a generalized weight, which is determined by the number of gateways in active forwarding routes, or based on the layer 3 (L3) link bandwidth between the PE and CE devices, where the bandwidth is the aggregate of all L3 gateway link bandwidths in the active forwarding routes.

This mechanism uses the EVPN Link Bandwidth Extended Community to manage traffic according to the varying capacities of devices, preventing congestion. This feature is compatible with platforms running Junos OS and Junos OS Evolved. Features such as graceful Routing Engine switchover (GRES) help maintain high availability and minimize disruption during system updates or failures. However, nonstop active routing (NSR) is not supported for EVPN-MPLS.

With generalized weight-based EVPN Type 5 unequal-load balancing, we can fine-tune the weight parameter using policy options and thereby fine-tune traffic distribution. In contrast, with L3 link bandwidth-based EVPN Type 5 unequal-load balancing, we cannot fine-tune the traffic distribution because it is based on the actual link capacity associated with the L3 gateway links in the active forwarding routes. The traffic distribution ratio is therefore determined directly by the advertised link capacities.

Benefits of EVPN-MPLS Type-5 UCMP Load Balancing

  • Optimizes network performance by distributing traffic based on device capacity, preventing congestion and ensuring efficient use of resources.

  • Provides flexible traffic management through configurable generalized weights, allowing adjustment according to specific network needs.

  • Maintains high availability with support for features like GRES, ensuring continuous network operation even during maintenance or failover events.

  • Compatible with devices running Junos OS or Junos OS Evolved, providing flexibility in network design and deployment.

Overview of Unequal Load Balancing for EVPN-MPLS Type 5 Routing

Unequal load balancing for EVPN-MPLS Type 5 routing can be achieved using two methods:

  • Generalized weight

  • Link bandwidth-based

Generalized weight unequal load balancing is achieved by using the EVPN Link Bandwidth Extended Community to distribute traffic across various paths based on their calculated weight. You implement this functionality by configuring policy statements that define the aggregate weight of each path. This weight is derived from the number of gateways in active forwarding routes, ensuring proportional traffic allocation according to the capacity of each path. You can modify the calculated weight by adding an optional multiplier statement to the aggregate-weight statement. You can use these configurations to achieve optimal traffic flow and reduce congestion on less capable paths, thus maximizing network efficiency. Configure the generalized weight with an export policy and specify routes that should advertise the community to enable granular control. This is useful for tuning distribution according to your requirements.

Configure EVPN-MPLS UCMP by applying weight-based policy statements and monitoring the results, as follows:

These policies define the policy framework for EVPN-MPLS UCMP so that remote PE devices can balance traffic accurately. The aggregate-weight statement, with an optional multiplier, provides flexibility for weight adjustments. The aggregate-weight statement enables a dynamic response to changes in network topology or capacity. The export policy integrates weight-based load balancing into your routing policy.

Link bandwidth–based EVPN Type 5 unequal-load balancing distributes traffic in proportion to the actual bandwidth of the active CE–PE links. It uses protocol-derived weighted load balancing on the local PE and advertises aggregate gateway bandwidth in EVPN Type 5 routes so that remote PEs can make proportional forwarding decisions.

When BGP, OSPF, or IS-IS runs between the CE and PE, the PE derives the bandwidth of each CE–PE link and calculates forwarding weights. The PE then advertises the aggregate bandwidth of all active CE–PE gateway links in the EVPN Type 5 route by using the EVPN Link Bandwidth extended community. Remote PEs use the advertised aggregate bandwidth to distribute traffic proportionally across the available gateways.

To enable bandwidth-aware unequal-cost load balancing on CE–PE links and advertise total active gateway bandwidth to remote PEs, enable EVPN Type 5 aggregate-bandwidth advertisement on the PEs and configure the applicable CE–PE routing protocol:

  • BGP—Enable BGP weighted load balancing on the CE–PE links.

Note: By default, the EVPN Type 5 route can carry both the BGP Link Bandwidth extended community and the EVPN Link Bandwidth Extended Community. If needed, apply a policy to remove the BGP Link Bandwidth extended community from the EVPN Type 5 route advertisement.

  • OSPF—Configure OSPF bandwidth-based weighted load balancing on the CE–PE interfaces.

  • IS-IS—Configure IS-IS bandwidth-based weighted load balancing on the CE–PE interfaces.

Unlike generalized weight-based EVPN Type 5 unequal-load balancing (which can use a policy to tune weights), link bandwidth–based unequal-load balancing derives the traffic distribution directly from the bandwidth of the active CE–PE gateway links. As a result, the traffic distribution ratio automatically reflects the relative link capacities of the active forwarding paths.

EVPN Link Bandwidth Extended Community

The EVPN Link Bandwidth Extended Community enables UCMP or WECMP load balancing for EVPN Type 5 routes by advertising the per-link capacity associated with each route. With bandwidth information attached to the route, the remote PE device can distribute traffic proportionally across EVPN-MPLS next hops that have different capacities, improving load sharing and reducing congestion. The EVPN Link Bandwidth Extended Community is transitive, so the bandwidth signal propagates across the EVPN domain.

Each EVPN Type 5 route carries the EVPN Link Bandwidth Extended Community. The community’s format defines two value indicators: Value-Units and Value-Weight. The Value-Units field indicates whether Value-Weight represents Mbps or a generalized weight.

The Value-Units field is encoded as follows:

  • 0x00—Value-Weight is expressed in Mbps (default).

  • 0x01—Value-Weight is a generalized weight expressed in units other than Mbps.

The Value-Weight is a 5-octet unsigned integer. The generalized weight units are intentionally unspecified to allow flexibility across different applications. Please see EVPN Link Bandwidth Extended Community for more information.

Note:

The EVPN-MPLS Type 5 unequal load-balancing feature supports the Value-Units "0x01", which expresses the Value-Weight in a generalized weight value.

By default, a remote PE device shares traffic equally across multihomed devices. If an L3 CE-to-device link has lower capacity than the traffic delivered to it, equal sharing can overload that path. To avoid overloading a path, each multihomed device advertises the weight of its CE-facing L3 link using the EVPN Link Bandwidth Extended Community.

For EVPN-MPLS Type 5 route-based UCMP, the remote PE device:

  • Associates each destination’s Type 5 route with its advertised weight.

  • Computes a balance factor for each EVPN-MPLS next hop.

  • Distributes traffic in proportion to the ratio of the CE-to-device link weights.

If an L3 link between a multihomed device and the CE fails or its capacity changes, the remote PE device re-evaluates the advertised weight ratios and adjusts forwarding toward the remaining paths, maintaining proportional distribution as the topology or link capacities evolve.

How EVPN-MPLS Unequal Load Balancing Works

With ECMP routing, the border leaf device distributes traffic equally across all leaf devices. This can result in traffic congestion when a lower-weight leaf device receives the same amount of traffic as a higher-weight leaf device from the border leaf via the EVPN overlay. The weight for the route is determined by the number of L3 gateways on the forwarding routes for that device.

UCMP addresses this issue by using the EVPN link bandwidth extended community with a Value-Units field of 0x01 to advertise a weight for each leaf device.

The UCMP feature is especially useful when multiple customer network devices advertise their anycast address to connected leaf devices. Each leaf device learns the anycast address from several connected customer network devices, calculates a weight, and advertises the anycast address to the border leaf device as an EVPN Type 5 route. The weight is calculated based on the number of L3 gateways in forwarding routes. If the number of customer network devices changes, the weight is adjusted, triggering a Type 5 route update to be sent to the border leaf device. The border leaf device recalculates the balance factor and distributes traffic according to the updated ratio.

The border leaf device ensures that the Type 5 route with the EVPN Link Bandwidth Extended Community is received from all leaf devices and verifies the consistency of the Value-Units field. If the EVPN Link Bandwidth Extended Community is missing from the Type 5 route of any leaf device, or if there is an inconsistency in the Value-Units field, the border leaf device uses standard ECMP forwarding to the leaf devices.

Unequal Load Balancing Topology Example

In the following illustration, each compute node is a customer device connected to a leaf device. Each leaf device can have one or more compute nodes, with an EBGP session for each compute node. The compute nodes advertise a common anycast address to the leaf devices. If the leaf device receives the anycast address from multiple compute nodes and multipath is configured, the active route is a multipath route.

Unequal Load Balancing Topology Example

In this scenario:

• Leaf 1 and Leaf 2 connect to the same compute node through EBGP sessions. Therefore, each leaf learns the anycast address with a single L3 gateway.

• Leaf 3 connects to three different compute nodes through three EBGP sessions that advertise the anycast route. Therefore, Leaf 3 has a multipath route with three L3 gateways.

• Leaf 4 connects to a single compute node through an EBGP session. Therefore, Leaf 4 learns the anycast route with a single L3 gateway.

Each leaf device calculates a weight for its route by checking the active route’s L3 gateway or gateways and uses an export policy to attach the EVPN Link Bandwidth Extended Community to the EVPN Type 5 advertisement. Each EBGP session has a weight of one. You can adjust the default weight by configuring a multiplier for the aggregate-weight in the export policy. This configuration increases the value and enables weight tuning on the leaf device.

In this example, the weight of the Leaf 1, Leaf 2, and Leaf 4 anycast addresses is one, while the weight of the Leaf 3 anycast address is three. Each leaf advertises its weight to the Border Leaf. The Border Leaf maintains a list of next-hops to each leaf device with a traffic-distribution factor based on a 1:1:3:1 ratio. This ratio reflects the number of compute nodes connected to each leaf device for the destination address.

Normal ECMP Example

Normal ECMP Example

The standard ECMP distribution from the border leaf device divides the traffic flows evenly between the leaf devices. In this example, Leaf 1 has four compute nodes, while Leaf 2 has only two compute nodes. However, the border leaf device divides the six flows of traffic evenly between the leaf devices, sending three traffic flows to each device. In this example, the standard ECMP distribution results in traffic congestion on the Leaf 2.

Generalized Weight-based UCMP Example

Generalized Weight-based UCMP Example

When you configure UCMP, each leaf device calculates a weight and advertises the weight to the Border Leaf device using the EVPN Link Bandwidth Extended Community in an EVPN Type 5 route. The Border Leaf device distributes the traffic to the leaf devices based on their advertised weights. In this example, Leaf 1 advertises a weight of four, and Leaf 2 advertises a weight of two. Therefore, the Border Leaf device sends four flows to Leaf 1 and two flows to leaf 2.

Note: If the EVPN Link Bandwidth Extended Community is missing from the Type 5 route of any leaf device, or if there is an inconsistency in the Value-Units field, the Border Leaf device generates a debug log and uses standard ECMP forwarding to the leaf devices.

Single EBGP Session over a Bundled Interface

With UCMP, each leaf device calculates a route’s weight by checking the number of L3 gateways. Each EBGP session has a weight of one, even if that session connects over a link aggregation group (LAG) that consists of several physical links.

Single EBGP Session over a Bundled Interface

In this example:

  • Leaf 1 connects to the CE device through a LAG consisting of four physical links.

  • Leaf 2 connects to the CE device through a single physical link.

  • Each leaf device connects to the CE device through a single EBGP session.

As a result, each leaf device advertises the anycast route for the CE device to its remote peers with a weight of one, even though Leaf 1 has multiple physical links.

Configure and Manage EVPN-MPLS Unequal Load Balancing

To manage weight-based load balancing in EVPN-MPLS Type 5 networks, ensure that the EVPN Link Bandwidth Extended Community is applied consistently across all devices. If discrepancies occur, the system defaults to ECMP forwarding.

Use CLI commands to configure weight-based load balancing and verify the settings. Define a weight-based policy in a policy statement, then export it to EVPN Type 5 routes.

Use the show route and show evpn ip-prefix-database extensive commands to verify distribution and view bandwidth allocation and balance factors. Check system log messages for missing or inconsistent bandwidth information that can trigger an ECMP fallback.

Configure EVPN Link Bandwidth Policies

As part of the EVPN Type 5 route advertisement, each device uses an export policy with the aggregate-weight statement with an optional multiplier statement, to calculate the weight and advertise it using the EVPN Link Bandwidth Extended Community.

The following example shows a minimal export policy configuring the aggregate-weight with the optional multiplier. You can customize this policy using match conditions to specify protocol or route-filter criteria.

Remove BGP Link Bandwidth Extended Community Advertisements

The BGP multipath configuration uses the BGP Link Bandwidth extended community to provide WECMP for unequal load balancing. When you configure an export policy with the aggregate-weight statement, EVPN adds the BGP Link Bandwidth extended community to the Type 5 route advertisement. As a result, the Type 5 route contains both the BGP Link Bandwidth extended community and the EVPN Link Bandwidth Extended Community by default.

You can configure a policy to remove the BGP Link Bandwidth extended community from the Type 5 route advertisement if you don't want both advertised.

In the output from the following show route extensive command, you can see both the BGP Link Bandwidth extended community and the EVPN Link Bandwidth Extended Community values were added to the route.

If you don't want both extended communities advertised with the Type 5 route, you can use a policy to remove the BGP Link Bandwidth extended community from the advertisement.

The following example policy removes the BGP Link Bandwidth extended community:

When you run the show route extensive command with the configuration that removes the BGP Link Bandwidth extended community, you see only the EVPN Link-Bandwidth Extended Community values in the EVPN Type 5 route:

Verify and Troubleshoot EVPN-MPLS Unequal Load Balancing

You can verify the bandwidth distribution with the show evpn ip-prefix-database extensive command. This command provides detailed insights into bandwidth allocation and balance factors. It will also indicate the Unequal ECMP status.

For example, in the first output below, the device is receiving consistent EVPN link bandwidth information from all the peer devices. So the Unequal ecmp state: is Enabled.

However, in the second and third outputs, EVPN link bandwidth information is not consistent, so the Unequal ecmp state: is Disabled. In this case the system defaults to ECMP routing to maintain network stability.

Note:

The show evpn ip-prefix-database extensive outputs only display the EVPN bandwidth information when the feature is enabled.

In the event of an error, such as a PE router not advertising bandwidth information, or an inconsistent Value-Units in the EVPN Link Bandwidth Extended Community, the output displays Unequal ecmp state: Disabled.

The outputs below illustrate what might be seen on a device when looking at the information received from two gateway devices.

These outputs show the device receiving EVPN link bandwidth data from both gateway 10.255.1.1 and gateway 10.255.1.2.

This output shows the remote device receiving EVPN link bandwidth data from gateway 10.255.1.1, but not receiving it from gateway 10.255.1.2. So the Unequal ecmp state: is Disabled.

This output shows the remote device receiving EVPN link bandwidth data from gateway 10.255.1.1, but gateway 10.255.1.2 is advertising the Value-Weight field encoded as 0x00 (Mbps). This mismatch of the EVPN link bandwidth community type causes the Unequal ecmp state: to be Disabled.

Related Documentation