ncurses applications not working in iTerm2

I cannot get commandline tools that use ncurses to work in iTerm2. They either fail or fall-back to some other UI if they have it (e.g. text only). Example for VLC (installed via Homebrew):

% /Applications/VLC.app/Contents/MacOS/VLC -I ncurses
VLC media player 3.0.16 Vetinari (revision 3.0.16-0-g5e70837d8d)
Error opening terminal: xterm-256color.

I have tried with different values for the TERM (xterm, rxvt, vt100) env variable and got the same error. More details about the iTerm2 session settings:

% env | grep -i term
LC_TERMINAL_VERSION=3.4.9beta1
ITERM_PROFILE=Default
__CFBundleIdentifier=com.googlecode.iterm2
TERM_PROGRAM_VERSION=3.4.9beta1
TERM_PROGRAM=iTerm.app
LC_TERMINAL=iTerm2
COLORTERM=truecolor
TERM=xterm-256color

Worth mentioning that this is only happening in an M1 MacBook Pro running macOS BigSur 11.5. With the same iTerm settings ncurses apps work fine in an Intel Macbook Pro running the same OS version.