Copy text in byobu when the mouse is "off"

I recently started using byobu with tmux as a backend. I have learned that AltF12 toggles "mouse mode" off and on. When "mouse mode" is on, I can select text with my mouse and paste it with the middle button as I would normally do. However, I cannot use my wheel to scroll up the pane's buffer. If I turn the "mouse mode" off, I can scroll up normally with the wheel, but can no longer use the mouse to select. Since I often want to scroll up and copy text, this is a problem. The AltPgUp shortcut which works in regular shells is captured by my terminal emulator instead of byobu so that doesn't help me scroll back either. The only workaround I've found is to enter scrollback mode with F. This lets me use Up,Down and PgUp,PgDn to scroll and I can indeed use the mouse to select, but I would still rather use the scroll wheel for this instead. So, how can I use the mouse both to scroll up to see previous lines _and_ for copying them?