configuring IP over infiband (IPoIB) in RHEL 7, scp speed

I am giving myself a crash course on infiniband in RHEL 7.9. I have server(s) each having an HDR ConnectX-6 card installed; this is in my lab and it's on a simple LAN where each server is connected to a Q8700 mlnx *switch* and for starters we just want basic network functionality over the infiniband so we can ssh, scp and create a basic nfs mount between servers. - I have 5 servers, named c1 to c5 - Each is running RHEL 7.9 x86-64 - I did mlnxofedinstall from the MLNX_OFED_LINUX-5.2-1.0.4.0-rhel7.9-x86_64.iso on each - I did systemctl enable openibd on each followed by a reboot - after that I manually /etc/init.d/opensmd start but only on c1 - using the NetworkManager gui in RHEL 7 I configured the ib0 interface to be - 192.168.2.1 to 192.168.2.5 for c1..c5 respectively - subnet mask is 255.255.255.0 for each with no gateway specified. - if i do ip ib0 it reports link speed of 100000 ----- - is this all there is to it or am I missing a whole bunch of other stuff ? - in doing scp over my copper 1gbps network, I always see a solid 112MB/sec; my take is 1000 mbps / 8 = 125 MBps then at 89.6% efficiency is 112 MBps. If I assume 100000 mb/s on HDR infiniband datagram IPoIB being 100x faster I should see 11200 MB/sec ? - I am getting 262 MB/sec currently (scp'ing a 30gb tar file) having done only what's mentioned above. Should I be getting better if so how much?