Sample Header Ad - 728x90

LACP bond seems to be favouring one interface when multiple concurrent processes are transferring data

1 vote
1 answer
1557 views
This is likely a really easy one (ie I've fundamentally missed something and its just my fault / lack of knowledge / assumptions). So I have a machine with 2x 25GbE fibres bonded into bond0 via LACP (cisco switches) with two VLANs. When I start 5 or 6 concurrent rsyncs transferring different data from one source to a destination path I was slightly surprised to see the data basically favouring one physical interface almost exclusively (~900MiB/s). I was under the assumption that the load would have been somewhat split between the two interfaces that constitute the bond. image of Grafana graph showing network traffic per interface I am fully aware that packets are NOT split across interfaces for a single stream, but as my rsync's are all separate processes I would have expected at least one, or two to use the 2nd physical interface.... for reference a 'rough' outline (ie removed info I think is sensitive) of the netplan config in use:
network:
 version: 2
 renderer: networkd
 ethernets:
  eno1:
   dhcp4: false
   dhcp6: false
   optional: true
   link-local: []
  ens5f0np0:
   dhcp4: false
   dhcp6: false
   optional: true
  ens5f1np1:
   dhcp4: false
   dhcp6: false
   optional: true	
 bonds:
  bond0:
   dhcp4: false
   dhcp6: false
   interfaces: [ens5f0np0, ens5f1np1]
   mtu: 9000
   parameters:
    mode: 802.3ad
    lacp-rate: fast
    mii-monitor-interval: 100
 vlans:
  bond0.xxx:
   id: xxx
   link: bond0
   addresses: [ip]
   gateway4: ip
   mtu: 1500
   nameservers:
    search: [domains]
    addresses: [ips]
  bond0.xxx:
   id: xxx
   link: bond0
   addresses: [ip]
   mtu: 9000
   routes:
   - to: random subnet
     via: local subnet ip
   - to: random subnet
     via: local subnet ip
   - to: random subnet
     via: local subnet ip
Is the issue that although the rsync are different processes, the source and destination IPs are the same (each rsync is reading a large sub-folder in one location, and copying to a common location)and the hashing being done at the bond basically means it sees it all as the same traffic? The source data lives on a server in 1 VLAN, and the destination server is on the other. If it is my fault / improper assumptions would still like to learn all the same as I would have thought the different rsyncs would constitute a different 'stream' of data.
Asked by user279851
Jul 16, 2022, 07:30 PM
Last activity: Jul 16, 2022, 09:09 PM