Byobu (tmux backend) after copying text in scrollback mode automatically quits scrollback mode

To clarify, "automatically quits scrollback" mode means byobu goes into normal mode and jumps to the bottom, and I can't just go back to the point I previously scrolled up to. I am also using iterm2 and use mouse (set in .tmux.conf) to enter scrollback mode, not sure how it affects the configurations.