Limit bandwidth for outgoing/incoming with latency for specific port using TC

I'm trying to limit the incoming and outgoing bandwidth with latency through a specific port using TC but I can't get it to work 100%. I'm not sure if I've done it correctly. When I ping google, the latency is added. I have used iperf to test the bandwidth through port 5001. The bandwidth throttle seems to work on the client where I have applied the settings below (client as iperf -s, incoming), however if I use this client (outgoing) to connect to another one, the bandwidth is restricted to 1-2 mbit instead of the expected 5mbit. What I would like to, for example, is to have the latency set to 100ms, bandwidth outgoing/incoming throttled to 5mbit, all these rules applied to port 5001. Am I on the correct path or have I misunderstood the concept of tc?

Ip link add name ifb0 type ifb 2>/dev/null || :
Ip link set dev ifb0 up
Tc qdisc add dev ifb0 root handle 1: htb
Tc class dev ifb0 parent 1: classid 1:20 htb rate 5mbit
Tc qdisc add dev ifb0 parent 1:20 handle 20: sfq perturb 10
Tc filter add dev ifb0 parent 1: protocol ip prio 1 basic match ‘cmp(u16 at 2 layer transport eq 5001)’ flowid 1:20
Tc qdisc add dev ens192 root netem delay 200ms
Tc qdisc add dev ens192 ingress
Tc filter add add dev ens192 ingress protocol ip basic match ‘cmp(u16 at 2 layer transport eq 5001)’ action mirred egress redirect dev ifb0