Sample Header Ad - 728x90

How can I display graphics on the screen from the kernel over the top of my X11/Wayland session?

3 votes
0 answers
1168 views
I want to draw simple (2D bitmapped) graphics onto my screen (in response to (simple) external inputs) with the lowest latency possible (the order of tens of milliseconds) so I can empirically test the results of drawing to the screen a) **in realtime**, b) with the least overhead possible, c) with page flipping completely disabled (tearing is fine). Then I can compare this with drawing to the screen in various stereotypical (and arguably pathological :) ) scenarios (eg X11, Wayland, Wayland+XWayland, Wayland+XWayland+xcompmgr, etc). To this end, how might I modify Linux so that I can draw **over the top of** my existing X11/Wayland session? Worded differently, yes, I want to fiddle with DRM (Direct Rendering Manager), a. *from inside the kernel*, b. *while X11 owns the DRM master*. :) I suspect that a hardware overlay would be the easiest (and most hardware-accelerated!) way to keep X11/Wayland from scribbling over what I'm drawing. (I'm imagining alternatives involving implementing a shadow/write-through framebuffer cache, to avoid read-back... no thanks!) I really, really want to confine my madness to a kernel module, and I don't want to have to fiddle with my graphics driver. :S (I'm incidentally using Intel graphics, albeit on an i915) So, reading through https://dri.freedesktop.org/docs/drm/ , I get the idea that *maybe* I want to try and hack together something that lets me create a dumb framebuffer, create a plane looking at that dumb framebuffer, and then *double-maybe* set up some kind of DMA-BUF... no, wait, if I've got a plane pointing at the framebuffer, it's already on the screen... I think? My main question is, how do I play fast and loose with DRM such that it basically tells X it's the only thing talking to the screen, while behind the scenes I'm managing an extra plane? Answers very greatly appreciated! --- My current understanding of Wayland is that it is fundamentally based around compositing, and by design cannot function without buffering one or more entire video frames before releasing it/them to the video card. While X11 does not have this restriction - COMPOSITE is an optional extension - it uses a stream-based drawing protocol, and I am not aware of any method to draw directly into a window. The closest thing I know of is the use of the MIT-SHM extension, and while this does allow the use of shared memory, that involves at least two memory copies (me->kernel, kernel->X11), *and then* the poking of an XShmPutImage down the X11 pipe to tell X11 to please flip. This means that, even if I were to make my process run in realtime... well, I'm too chicken to run X in realtime as well, so I'd still have to wait for X to be scheduled, decode and reach my request in its command queue, and finally flip. Hence my throwing my hands up in the air and trying to see if I can just shove my graphics drawing code straight into the kernel, and hopefully make it all coexist somehow. I can imagine all of this additional overhead really adds up, and I want to quantify exactly *how* - or, alternatively, concretely establish that my mental models are completely incorrect and that the speed of contemporary hardware makes these concerns moot. I am ***extremely*** curious to see what the impact would be if I eliminated all the bottlenecks, and also what the difference would be like comparing older systems versus more modern hardware. Incidentally, this question is related to another question I asked [over here](https://unix.stackexchange.com/questions/500167/how-do-i-find-the-video-memory-regions-representing-whats-on-my-screen-from) and [over here](https://stackoverflow.com/questions/42748390/directly-accessing-video-memory-within-the-linux-kernel-in-a-driver-agnostic-man) , both of which sadly still have no answers, and which I am no further forward on.
Asked by i336_ (1077 rep)
Aug 15, 2019, 03:10 PM
Last activity: Jun 26, 2020, 05:45 PM