tc traffic shaping with HTB and CQB causes packet transmission gap inconsistencies
0
votes
0
answers
557
views
I am sorry if this is duplicate of https://serverfault.com/q/1076769/822163 .
I created that first and then realized the Linux and Unix stack exchange is the right place.
Problem: When the tc HTB or CQB is used to do traffic shaping, first two packet that are sent after some time gap are sent back to back as recorded in the pcap log.
I have a intermediate computer with ubuntu 18.4 with network forwarding enabled. I run the tc with HTB to shape the traffic to have constant bitrate output on egress port.
Client requests the chunks with variable sizes from the sever. Server sends the chunk transfer encoded data with gap of 200ms between each chunk to client. With my setup having the intermediate computer, These packets are passed through the traffic shaper on intermediate computer to obtain fixed bitrate of 500kbps.
As I disable offload (TSO and GRO) each n bytes are split into frames by pcap.
Most of 1448B packets have time gap close to 24.224ms which is expected at 500kbps
Issue: Although the frames arrive in the sequence, their time gap of arrival is not consistent.
Second large packet (1448B) after gap of 200ms always comes almost back to back with first packet.
Last packet in chunk ( 654B) arrives with delay (24.224ms instead of 10.464ms in example in the picture attached)
Screen shot of the timings
TC command with HTB:
tc qdisc del dev eno1 root 2> /dev/null > /dev/null
tc qdisc add dev eno1 root handle 1:0 htb default 2
tc class add dev eno1 parent 1:1 classid 1:2 htb rate 500kbit ceil 500kbit burst 10 cburst 10 prio 2
tc filter add dev eno1 protocol ip parent 1:0 u32 match ip dst 192.168.2.103 flowid 1:2
If I am not doing any mistake in calculation I think the issue could be due to the token handling in tc that I am using for trafic shaping.
I think the tokens are accumulated in the idle time and when the next packet is received it sends the two packets back to back.
from third packet token consumtion rate settles down.
If this is what is happenning, I would like to know if there is a way to avoid using the accumulated tokens for second packet in the chunk.
I tried various options in tc command
I also tried using CQB - command below
Reference : https://lartc.org/lartc.html#AEN2233
Observation:
reducing the burst = 10 slightly increases the gap between first and second packet.
tc With CQB:
tc qdisc del dev eno1 root 2> /dev/null > /dev/null
tc qdisc add dev eno1 root handle 1: cbq avpkt 5000 bandwidth 10mbit
tc class add dev eno1 parent 1: classid 1:1 cbq rate 500kbit allot 5000 prio 5 bounded isolated
tc class add dev eno1 parent 1:1 classid 1:10 cbq rate 500kbit allot 5000 prio 1 avpkt 5000 bounded
tc class add dev eno1 parent 1:1 classid 1:20 cbq rate 500kbit allot 5000 avpkt 5000 prio 2
tc filter add dev eno1 protocol ip parent 1:0 u32 match ip dst 192.168.2.103 flowid 1:10
tc filter add dev eno1 parent 1: protocol ip prio 13 u32 match ip dst 0.0.0.0/0 flowid 1:20
Further:
As per suggestion from http://linux-ip.net/articles/hfsc.en/ I tried HFSC (referred ). I need help with HFSC here.
Here is the script that I used
tc qdisc del dev eno1 root 2> /dev/null > /dev/null
tc qdisc add dev eno1 root handle 1: hfsc
tc class add dev eno1 parent 1: classid 1:1 hfsc sc rate 1000kbit ul rate 1000kbit
tc class add dev eno1 parent 1:1 classid 1:10 hfsc sc rate 1000kbit ul rate 1000kbit
tc class add dev eno1 parent 1:1 classid 1:20 hfsc sc rate 10000kbit ul rate 10000kbit
tc class add dev eno1 parent 1:10 classid 1:11 hfsc sc umax 1480b dmax 53ms rate 400kbit ul rate 1000kbit
tc class add dev eno1 parent 1:10 classid 1:12 hfsc sc umax 1480b dmax 30ms rate 100kbit ul rate 1000kbit
tc filter add dev eno1 protocol ip parent 1:0 u32 match ip dst 192.168.2.103 flowid 1:11
output of my
tc class show eno1
Output:
class hfsc 1:11 parent 1:10 sc m1 0bit d 23.4ms m2 400Kbit ul m1 0bit d 0us m2 1Mbit
class hfsc 1: root
class hfsc 1:1 parent 1: sc m1 0bit d 0us m2 1Mbit ul m1 0bit d 0us m2 1Mbit
class hfsc 1:10 parent 1:1 sc m1 0bit d 0us m2 1Mbit ul m1 0bit d 0us m2 1Mbit
class hfsc 1:20 parent 1:1 sc m1 0bit d 0us m2 10Mbit ul m1 0bit d 0us m2 10Mbit
class hfsc 1:12 parent 1:10 sc m1 394672bit d 30.0ms m2 100Kbit ul m1 0bit d 0us m2 1Mbit
I am not sure what does it mean by
> ul m1 0bit d 0us
where as In my tc command I have
> sc umax 1480b dmax 53ms
After running this script I try to ping 192.168.1.102. I get few ping responses and then the ARP
> who has 192.168.2.100
kicks in where 192.168.2.100 is ip address of ip forwarding port where I am running tc.
The command is mostly copied from http://linux-ip.net/articles/hfsc.en/
I have just added route
> tc filter add dev eno1 protocol ip parent 1:0 u32 match ip dst 192.168.2.103 flowid 1:11
It would be great if someone could help to fix the umax and dmax issue.

Asked by Chinmaey Shende
(1 rep)
Sep 6, 2021, 05:38 PM
Last activity: Sep 8, 2021, 03:56 PM
Last activity: Sep 8, 2021, 03:56 PM