Is there benefit of MLNX OFED install vs default Redhat InfiniBand Support
1
vote
1
answer
1068
views
When using RHEL 8.8, there is a choice to be made for users with Mellanox InfiniBand hardware. They can either utilize Red Hat's included
InfiniBand Support
group installation (which I choose to do during the installation process) in order to enable InfiniBand networking, or they can opt for MLNX_OFED_LINUX-23.04-1.1.3.0-rhel8.8-x86_64.iso
and execute their mlnxofedinstall
script, which replaces Red Hat packages with their own.
Aside from firmware updates, does anyone know of any advantages of using the MLNX path as opposed to the Red Hat path for setting up InfiniBand?
The MLNX approach generally involves more setup and configuration, so I am interested in any performance justifications for using the MLNX method (considering that I am using their hardware) instead of what Red Hat provides.
For reference:
Usage: ./mlnxofedinstall [OPTIONS]
Installation control:
--force Force installation
--tmpdir Change tmp directory. Default: /tmp
-k|--kernel
Default on this system: 3.10.0-1160.92.1.el7.x86_64
-s|--kernel-sources
Default on this system: /lib/modules/3.10.0-1160.92.1.el7.x86_64/build
-U|--update Update installed version
--with-32bit Install 32-bit libraries. Relevant for x86_64 platforms
--without-32bit Skip 32-bit libraries installation (Default). Relevant for x86_64 platforms
--distro Set Distro name for the running OS (e.g: rhel6.5, sles12sp3). Default: Use auto-detection
--skip-distro-check Do not check MLNX_OFED_LINUX vs Distro matching
--without-depcheck Run the installation without verifying that all required Distro's packages are installed
--check-deps-only Check for missing required Distro's packages and exit
--print-distro Print distribution name and exit
--add-kernel-support
Add kernel support (Run mlnx_add_kernel_support.sh) and install it
--add-kernel-support-build-only
Add kernel support (Run mlnx_add_kernel_support.sh) and exit
--kernel-extra-args ''
pass to kernel configure script (single paraeter, space separated)
- Pass it along with --add-kernel-support
--kmp Install kernel RPMs with KMP support if available
- Enabled by default with official MLNX_OFED_LINUX releases
- Disabled by default if new MLNX_OFED_LINUX was created with mlnx_add_kernel_support.sh
- Passing it along with --add-kernel-support will pass --kmp to mlnx_add_kernel_support.sh
--disable-kmp Install kernel RPMs without KMP support if available
- Enabled by default if new MLNX_OFED_LINUX was created with mlnx_add_kernel_support.sh
--skip-kmp-verify Do not check if the KMP rpms are compatiable with the target kernel
--speed-up-kmp Speed up KMP RPMs installation by skipping %post section and running the required operations once for all KMPs
--umad-dev-rw Grant non root users read/write permission for umad devices instead of default
--umad-dev-na Prevent from non root users read/write access for umad devices. Overrides '--umad-dev-rw'
--enable-mlnx_tune Enable Running the mlnx_tune utility
--enable-affinity Run mlnx_affinity script upon boot
--disable-affinity Disable mlnx_affinity script (Default)
--skip-unsupported-devices-check
Don't abort if system has an older, unsupported card
--enable-opensm Run opensm upon boot
--package-install-options
RPM install options to use when installing RPM packages (comma separated list)
--pre-install-
Run given script before given package's install
--post-install-
Run given script after given package's install
Firmware update control:
--without-fw-update Skip firmware update
--fw-update-only Update firmware. Skip driver installation
--force-fw-update Force firmware update
--fw-image-dir Firmware images directory to use instead of default package content
Package selection:
-c|--config
Example of the config file can be found under docs
--all Install all available packages
--bluefield Install BlueField packages
--hpc Install minimum packages required for HPC
--basic Install minimum packages for basic functionality
--dpdk Install minimum packages required for DPDK
--ovs-dpdk Install DPDK and OVS packages
--vma Install minimum packages required for VMA
--xlio Install minimum packages required for XLIO
--guest Install minimum packages required by guest OS
--hypervisor Install minimum packages required by hypervisor OS
--with-fabric-collector
Enable installing fabric-collector package
--with-nvmf Enable NVMEoF support
--with-nfsrdma Enable NFSoRDMA support
--without-nfsrdma Disable NFSoRDMA support (Default)
Extra package filtering:
--kernel-only Install kernel space packages only
--user-space-only Filter selected packages and install only User Space packages
--without- Do not install package
--with- Force installing package
Miscellaneous:
-h|--help Display this help message and exit
--post-start-delay
Set openibd POST_START_DELAY parameter in seconds. (Default 0)
-p|--print-available Print available packages for current platform
And create corresponding ofed.conf file
--copy-ifnames-udev Copy compatibility udev rules for interface names
Output control:
-v|-vv|-vvv Set verbosity level
-q Set quiet - no messages will be printed
Asked by ron
(8647 rep)
Jun 27, 2023, 02:45 PM
Last activity: Jun 27, 2023, 04:37 PM
Last activity: Jun 27, 2023, 04:37 PM