quick-swap TUI mode while using vi keybindings in GDB

In the default mode of gdb, pressing ctrl+x a turns on and off the UI, and ctrl+x 2 cycles through different split views. There are a number of similar useful shortcuts. I can turn on vi emulation either with esc ctrl+j, or by adding set editing-mode vi into ~/.inputrc. When vi emulation is enabled, ctrl+x now puts the characters ^X into my prompt, and the shortcuts cease to function. How do I fix this? Cheers