Is there any OS-level caching for bare block devices? If so, how do I bypass it?

If I read and write directly to a block device (e.g. /dev/sda1), is there any OS-level caching involved on Linux? If so, how do I bypass it, is opening with O_DIRECT enough? I'm writing a simple benchmark script to characterize the behavior of a shingled magnetic recording (SMR) drive I have, so I don't want to bypass any drive-level caching or reordering, only anything the OS is doing. Searching for the relevant terms gives lots of results that do not address this specific question, though I did learn that Solaris and FreeBSD have both block and character devices for disks, with the block devices being buffered. On my Linux I only see block devices for disks.