LACP bond seems to be favouring one interface when multiple concurrent processes are transferring data
1
vote
1
answer
1557
views
This is likely a really easy one (ie I've fundamentally missed something and its just my fault / lack of knowledge / assumptions).
So I have a machine with 2x 25GbE fibres bonded into bond0 via LACP (cisco switches) with two VLANs.
When I start 5 or 6 concurrent rsyncs transferring different data from one source to a destination path I was slightly surprised to see the data basically favouring one physical interface almost exclusively (~900MiB/s). I was under the assumption that the load would have been somewhat split between the two interfaces that constitute the bond.
I am fully aware that packets are NOT split across interfaces for a single stream, but as my rsync's are all separate processes I would have expected at least one, or two to use the 2nd physical interface....
for reference a 'rough' outline (ie removed info I think is sensitive) of the netplan config in use:

network:
version: 2
renderer: networkd
ethernets:
eno1:
dhcp4: false
dhcp6: false
optional: true
link-local: []
ens5f0np0:
dhcp4: false
dhcp6: false
optional: true
ens5f1np1:
dhcp4: false
dhcp6: false
optional: true
bonds:
bond0:
dhcp4: false
dhcp6: false
interfaces: [ens5f0np0, ens5f1np1]
mtu: 9000
parameters:
mode: 802.3ad
lacp-rate: fast
mii-monitor-interval: 100
vlans:
bond0.xxx:
id: xxx
link: bond0
addresses: [ip]
gateway4: ip
mtu: 1500
nameservers:
search: [domains]
addresses: [ips]
bond0.xxx:
id: xxx
link: bond0
addresses: [ip]
mtu: 9000
routes:
- to: random subnet
via: local subnet ip
- to: random subnet
via: local subnet ip
- to: random subnet
via: local subnet ip
Is the issue that although the rsync are different processes, the source and destination IPs are the same (each rsync is reading a large sub-folder in one location, and copying to a common location)and the hashing being done at the bond basically means it sees it all as the same traffic? The source data lives on a server in 1 VLAN, and the destination server is on the other.
If it is my fault / improper assumptions would still like to learn all the same as I would have thought the different rsyncs would constitute a different 'stream' of data.
Asked by user279851
Jul 16, 2022, 07:30 PM
Last activity: Jul 16, 2022, 09:09 PM
Last activity: Jul 16, 2022, 09:09 PM