I am wondering why, when using vaapi with ffmpeg, if I use x11grab I must use -vf format=nv12,hwupload but with kmsgrab I must use -vf scale_vaapi=format=nv12 instead. In particular I want to know 2 things. - Why is it this way? - Is there any way to use normal filters (like eq=brightness=50 or zscale) with vaapi when using kmsgrab? Because the dilemma that's causing me to ask these questions is that I need to do some color corrections and filters seem like the best way to do that, but I'm unable to use filters when I'm using kmsgrab because scale_vaapi breaks other filters, and I even tried using some obscure filters like procamp_vaapi which shuold allow me to do the same things as eq, but I couldn't get it to work. my kmsgrab command: ffmpeg -y -device /dev/dri/card1 -f kmsgrab -i - -vf 'hwmap=derive_device=vaapi,scale_vaapi=format=nv12' -c:v h264_vaapi -qp 24 output.mkv

I'm trying to enable hardware acceleration on my debian wayland machine with an integrated GPU Intel Graphics. I set the LIBVA_DRIVER_NAME variable to iHD, and I got this error : $ vainfo libva info: VA-API version 1.17.0 libva info: User environment variable requested driver 'iHD' libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so libva info: Found init function __vaDriverInit_1_17 libva error: /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so init failed libva info: va_openDriver() returns 2 vaInitialize failed with error code 2 (resource allocation failed),exit I did some research, and when I unset the XDG_RUNTIME_DIR variable, the command works (even if it complains about it) : $ XDG_RUNTIME_DIR= vainfo error: XDG_RUNTIME_DIR is invalid or not set in the environment. libva info: VA-API version 1.17.0 libva info: User environment variable requested driver 'iHD' libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so libva info: Found init function __vaDriverInit_1_17 libva info: va_openDriver() returns 0 vainfo: VA-API version: 1.17 (libva 2.12.0) vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 23.1.1 () vainfo: Supported profile and entrypoints VAProfileNone : VAEntrypointVideoProc VAProfileMPEG2Simple : VAEntrypointVLD VAProfileMPEG2Main : VAEntrypointVLD VAProfileH264Main : VAEntrypointVLD VAProfileH264Main : VAEntrypointEncSliceLP VAProfileH264High : VAEntrypointVLD VAProfileH264High : VAEntrypointEncSliceLP VAProfileJPEGBaseline : VAEntrypointVLD VAProfileJPEGBaseline : VAEntrypointEncPicture VAProfileH264ConstrainedBaseline: VAEntrypointVLD VAProfileH264ConstrainedBaseline: VAEntrypointEncSliceLP VAProfileVP8Version0_3 : VAEntrypointVLD VAProfileHEVCMain : VAEntrypointVLD VAProfileHEVCMain : VAEntrypointEncSliceLP VAProfileHEVCMain10 : VAEntrypointVLD VAProfileHEVCMain10 : VAEntrypointEncSliceLP VAProfileVP9Profile0 : VAEntrypointVLD VAProfileVP9Profile0 : VAEntrypointEncSliceLP VAProfileVP9Profile1 : VAEntrypointVLD VAProfileVP9Profile1 : VAEntrypointEncSliceLP VAProfileVP9Profile2 : VAEntrypointVLD VAProfileVP9Profile2 : VAEntrypointEncSliceLP VAProfileVP9Profile3 : VAEntrypointVLD VAProfileVP9Profile3 : VAEntrypointEncSliceLP VAProfileHEVCMain12 : VAEntrypointVLD VAProfileHEVCMain422_10 : VAEntrypointVLD VAProfileHEVCMain422_12 : VAEntrypointVLD VAProfileHEVCMain444 : VAEntrypointVLD VAProfileHEVCMain444 : VAEntrypointEncSliceLP VAProfileHEVCMain444_10 : VAEntrypointVLD VAProfileHEVCMain444_10 : VAEntrypointEncSliceLP VAProfileHEVCMain444_12 : VAEntrypointVLD VAProfileHEVCSccMain : VAEntrypointVLD VAProfileHEVCSccMain : VAEntrypointEncSliceLP VAProfileHEVCSccMain10 : VAEntrypointVLD VAProfileHEVCSccMain10 : VAEntrypointEncSliceLP VAProfileHEVCSccMain444 : VAEntrypointVLD VAProfileHEVCSccMain444 : VAEntrypointEncSliceLP VAProfileAV1Profile0 : VAEntrypointVLD VAProfileAV1Profile0 : VAEntrypointEncSliceLP VAProfileHEVCSccMain444_10 : VAEntrypointVLD VAProfileHEVCSccMain444_10 : VAEntrypointEncSliceLP Why does that happen ? I'm not sure I understand completely what XDG_RUNTIME_DIR does, and I suspect it is not really the root cause of the problem, but I can't find any other clues as to what I can try next. I'm happy to update my question if more system info is needed.

I know that the most basic syntax to horizontally stack two videos side-by-side is using the following syntax: ffmpeg -i left.mp4 -i right.mp4 -filter_complex hstack sideByside.mp4 How, if possible, to do this using existing hardware to accelerate the process? In other word, how to achieve this horizontal stack using h264_vaapi encoder like this: ffmpeg -i left.mp4 -i right.mp4 -filter_complex hstack -c:v h264_vaapi sideBySide.mp4 thanks in advance

this my ffmpeg call in script:

ffmpeg -hwaccel vaapi -hwaccel_output_format vaapi -vaapi_device /dev/dri/renderD128 \
	-i "_$v" -i /home/john/Web/blackblog/_meta/graphics/watermark.png \
	-filter_complex "[0:v][1:v] overlay_vaapi=25:25" -pix_fmt yuv420p \
	-hide_banner -vcodec h264 -acodec aac "$filename.mp4"

and it throws this error:

[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5f654b2a1a00] All samples in data stream index:id [3:4] have zero duration, stream set to be discarded by default. Override using AVStream->discard or -discard for ffmpeg command.
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'vid.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 131072
    compatible_brands: isomiso2mp41
    creation_time   : 2024-02-19T19:48:49.000000Z
    com.android.manufacturer: Google
    com.android.model: Pixel 6
    com.android.capture.fps: 30.000000
  Duration: 00:02:03.55, start: 0.000000, bitrate: 19964 kb/s
  Stream #0:0[0x1](eng): Data: none (mett / 0x7474656D), 29 kb/s (default)
    Metadata:
      creation_time   : 2024-02-19T19:48:49.000000Z
      handler_name    : MetaHandle
  Stream #0:1[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 191 kb/s (default)
    Metadata:
      creation_time   : 2024-02-19T19:48:49.000000Z
      handler_name    : SoundHandle
      vendor_id       : 
  Stream #0:2[0x3](eng): Video: hevc (Main) (hvc1 / 0x31637668), yuvj420p(pc, bt709), 1920x1080, 19736 kb/s, SAR 1:1 DAR 16:9, 29.97 fps, 29.97 tbr, 90k tbn (default)
    Metadata:
      creation_time   : 2024-02-19T19:48:49.000000Z
      handler_name    : VideoHandle
      vendor_id       : 
  Stream #0:3[0x4](eng): Data: none (mett / 0x7474656D) (default)
    Metadata:
      creation_time   : 2024-02-19T19:48:49.000000Z
      handler_name    : MetaHandle
Input #1, png_pipe, from 'logo.png':
  Duration: N/A, bitrate: N/A
  Stream #1:0: Video: png, rgba(pc, gbr/unknown/unknown), 200x44 [SAR 11811:11811 DAR 50:11], 25 fps, 25 tbr, 25 tbn
Stream mapping:
  Stream #0:2 (hevc) -> overlay_vaapi (graph 0)
  Stream #1:0 (png) -> overlay_vaapi (graph 0)
  overlay_vaapi:default (graph 0) -> Stream #0:0 (libx264)
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
Impossible to convert between the formats supported by the filter 'graph 0 input from stream 1:0' and the filter 'auto_scale_0'
[fc#0 @ 0x5f654b220b40] Error reinitializing filters!
Failed to inject frame into filter network: Function not implemented
Error while filtering: Function not implemented
Impossible to convert between the formats supported by the filter 'graph 0 input from stream 1:0' and the filter 'auto_scale_0'
[fc#0 @ 0x5f654b220b40] Error reinitializing filters!
Failed to inject frame into filter network: Function not implemented
[out#0/mp4 @ 0x5f654c418a40] Nothing was written into output file, because at least one of its streams received no packets.
frame=    0 fps=0.0 q=0.0 Lsize=       0kB time=N/A bitrate=N/A speed=N/A
Conversion failed!

When I remove all arguments from first line it works, but is super slow. When I turn hardware acceleration on, it gives me this error. I am running it on Arch linux and this is output of ffmpeg -version:

ffmpeg version n6.1.1 Copyright (c) 2000-2023 the FFmpeg developers
built with gcc 13.2.1 (GCC) 20230801
configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-frei0r --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libharfbuzz --enable-libiec61883 --enable-libjack --enable-libjxl --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libplacebo --enable-libpulse --enable-librav1e --enable-librsvg --enable-librubberband --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpl --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-nvdec --enable-nvenc --enable-opencl --enable-opengl --enable-shared --enable-vapoursynth --enable-version3 --enable-vulkan
libavutil      58. 29.100 / 58. 29.100
libavcodec     60. 31.102 / 60. 31.102
libavformat    60. 16.100 / 60. 16.100
libavdevice    60.  3.100 / 60.  3.100
libavfilter     9. 12.100 /  9. 12.100
libswscale      7.  5.100 /  7.  5.100
libswresample   4. 12.100 /  4. 12.100
libpostproc    57.  3.100 / 57.  3.100

Is there a standard (or de facto standard) pixel format in the va api that both software and drivers needs to implement as a minimum? I currently trying to debug va-api implementation between software that seems to hard code it to RGBX_8888/XRGB and driver that seems to only accept YUV_420_BIPLANAR/NV12. This will obviously not work. So, what is the "de facto standard" or "expectations"? I don't want to write "requirements" because it is after all open source projects and not a commercial product, but both sides are projects coming from large major companies so I do expect some kind of platform standard, minimum implementation so things can work or it would not be much use of a standard va-api. Should the driver support more than one pixel format or should the software? And if so which one?

I'm looking for an alternative to Handbrake. Sadly, it doesn't support VA-API, so you have to install separate support for QSV/VCN/NVENC. While I don't know how bad the situation with Intel and NVIDIA is, the situation for AMD is abysmal. To get hardware encoding for Handbrake, you need to install the AMDGPU-Pro Stack and the separate AMF module. AMDGPU support for various Linux distributions is slim at best, so that's basically no option for many. On the other hand, it's a really great toolbox. Sure, you can simply use FFMPEG, but that's not really a practical option as you have to do too much googling to find anything. Does anybody know any alternative, like Handbrake with added VA-API support, or a similarly good GUI for FFMPEG?

I am using Debian Sid, kernel 5.19.0-1-amd64. My machine is a ThinkPad T14 Gen 3 with an integrated graphics controller and an NVIDIA GPU; here is the result of lshw -c display:

*-display                 
       description: VGA compatible controller
       product: Alder Lake-P Integrated Graphics Controller
       vendor: Intel Corporation
       physical id: 2
       bus info: pci@0000:00:02.0
       logical name: /dev/fb0
       version: 0c
       width: 64 bits
       clock: 33MHz
       capabilities: pciexpress msi pm vga_controller bus_master cap_list rom fb
       configuration: depth=32 driver=i915 latency=0 mode=3840x2160 resolution=3840,2400 visual=truecolor xres=3840 yres=2160
       resources: iomemory:600-5ff iomemory:400-3ff irq:178 memory:6052000000-6052ffffff memory:4000000000-400fffffff ioport:3000(size=64) memory:c0000-dffff memory:4010000000-4016ffffff memory:4020000000-40ffffffff
  *-display
       description: 3D controller
       product: TU117M [GeForce MX550]
       vendor: NVIDIA Corporation
       physical id: 0
       bus info: pci@0000:03:00.0
       version: a1
       width: 64 bits
       clock: 33MHz
       capabilities: pm msi pciexpress bus_master cap_list rom
       configuration: driver=nvidia latency=0
       resources: iomemory:600-5ff iomemory:600-5ff irq:176 memory:bd000000-bdffffff memory:6040000000-604fffffff memory:6050000000-6051ffffff ioport:2000(size=128)

I'm trying to get hardware acceleration to work (mostly within Chrome, so Google Meet & Zoom Web run smoothly). The problem seems to be that VA-API (version 1.15.0) doesn't seem to be connecting to either card, regardless of what environment parameter I use.

root@cyclone:/# vainfo
libva info: VA-API version 1.15.0
libva error: vaGetDriverNameByIndex() failed with unknown libva error, driver_name = (null)
vaInitialize failed with error code -1 (unknown libva error),exit

root@cyclone:/# LIBVA_DRIVER_NAME=iHD vainfo
libva info: VA-API version 1.15.0
libva info: User environment variable requested driver 'iHD'
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_15
DRM_IOCTL_I915_GEM_APERTURE failed: Invalid argument
Assuming 131072kB available aperture size.
May lead to reduced performance or incorrect rendering.
get chip id failed: -1 
param: 4, val: 0
libva error: /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so init failed
libva info: va_openDriver() returns 18
vaInitialize failed with error code 18 (invalid parameter),exit

root@cyclone:/# LIBVA_DRIVER_NAME=nvidia vainfo
libva info: VA-API version 1.15.0
libva info: User environment variable requested driver 'nvidia'
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/nvidia_drv_video.so
libva info: Found init function __vaDriverInit_1_0
libva error: /usr/lib/x86_64-linux-gnu/dri/nvidia_drv_video.so init failed
libva info: va_openDriver() returns 1
vaInitialize failed with error code 1 (operation failed),exit

The problem is the same regardless of whether I install the free driver for the Intel hardware (package intel-media-va-driver) or the non-free one (intel-media-va-driver-non-free). For NVIDIA drivers, I have tried installing the package recommended by nvidia-detect (which was simply nvidia-driver) as well as the other suggested one (nvidia-testla-470-driver), both of them fail in the same way. I also tried to modify Xorg config to [only use the NVIDIA GPU](https://wiki.archlinux.org/title/NVIDIA_Optimus#Use_NVIDIA_graphics_only) , but this did not change anything. I don't know if it's related, but intel_gpu_top seems to fail with an obscure error message as well:

root@cyclone:/# intel_gpu_top
intel_gpu_top: ../tools/intel_gpu_top.c:1910: init_engine_classes: Assertion `max >= 0' failed.
Aborted

What should my next steps be here?

I have Intel Core-i7 8650U processor. I transcode video with ffmpeg. I take attempts to accelerate it. Now I use VAAPI codecs in ffmpeg (h264_vaapi) and vaapi rescaling (scale_vaapi) in filters. And when I use them my CPU load is not on maximum, but framerate doesn't insrease more. So I think that VAAPI's resources are over. **So I have to monitor the usage of Intel graphics hardware that is in my CPU. Is there any API or tool that could show usage of this hardware?** (It would be cool to know both tool(like **htop**) and opportunity to request it at moment from any script (like **ps aux**) or at least one of them). For CPU it's htop For NVIDIA GPUs it's nvidia-smi **For VAAPI it's ...?**

On my Lenovo Ideapad S110 Debian Stretch is just working fine, but HD videos don't run smoothly, at least not in full screen. i965-va-driver is installed and up to date, vainfo returns sudo vainfo error: XDG_RUNTIME_DIR not set in the environment. libva info: VA-API version 0.39.4 libva info: va_getDriverName() returns -1 libva error: va_getDriverName() failed with unknown libva error,driver_name=(null) vaInitialize failed with error code -1 (unknown libva error),exit Maybe anybody can help. Thanks!

I am using archlinux trying to connect to an RTSP stream on my local network (local IP camera) and am getting very hit-and-miss performance from VLC. Most of the time the connection runs just fine, but sometimes the connection _freezes_ and I have to close and reopen to resume. Running from the terminal shows the following: VLC media player 2.2.4 Weatherwax (revision 2.2.3-37-g888b7e89) [0000000000bbf1f8] dummy interface: using the dummy interface module... Failed to open VDPAU backend libvdpau_nvidia.so: cannot open shared object file: No such file or directory Failed to open VDPAU backend libvdpau_nvidia.so: cannot open shared object file: No such file or directory [00007fa54800c618] avcodec decoder error: more than 5 seconds of late video -> dropping frame (computer too slow ?) ... ... [00007fa54800c618] avcodec decoder error: more than 5 seconds of late video -> dropping frame (computer too slow ?) This is obviously a bit off as this is an intel laptop with no nvidia components. I cannot figure out where/why it is looking for nvidia files (or if it is even relevant to the problem) Running grep -iE 'vdpau|dri driver' ~/.local/share/xorg/Xorg.0.log yields: [ 99171.065] (II) intel(0): [DRI2] DRI driver: i965 [ 99171.065] (II) intel(0): [DRI2] VDPAU driver: va_gl Running vainfo yields: libva info: VA-API version 0.39.4 libva info: va_getDriverName() returns 0 libva info: Trying to open /usr/lib/dri/i965_drv_video.so libva info: Found init function __vaDriverInit_0_39 libva info: va_openDriver() returns 0 vainfo: VA-API version: 0.39 (libva 1.7.3) vainfo: Driver version: Intel i965 driver for Intel(R) Ivybridge Mobile - 1.7.3 vainfo: Supported profile and entrypoints VAProfileMPEG2Simple : VAEntrypointVLD VAProfileMPEG2Simple : VAEntrypointEncSlice VAProfileMPEG2Main : VAEntrypointVLD VAProfileMPEG2Main : VAEntrypointEncSlice VAProfileH264ConstrainedBaseline: VAEntrypointVLD VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice VAProfileH264Main : VAEntrypointVLD VAProfileH264Main : VAEntrypointEncSlice VAProfileH264High : VAEntrypointVLD VAProfileH264High : VAEntrypointEncSlice VAProfileH264StereoHigh : VAEntrypointVLD VAProfileVC1Simple : VAEntrypointVLD VAProfileVC1Main : VAEntrypointVLD VAProfileVC1Advanced : VAEntrypointVLD VAProfileNone : VAEntrypointVideoProc VAProfileJPEGBaseline : VAEntrypointVLD I've read the relevant arch wiki articles (I believe), but cannot get a stable connection. I have extra/libva, extra/libva-intel-driver, and extra/libva-vdpau-driver installed and this seems like it should do it... what am I missing?

I own a laptop with Intel HD graphics (i915 kernel driver) and I'm trying to get hardware-accelerated video playback under Debian Jessie. I have installed i965-va-driver and libva-intel-vaapi-driver packages, and vainfo gives me a list of supported codecs: $ vainfo libva info: VA-API version 0.36.0 libva info: va_getDriverName() returns 0 libva info: Trying to open /usr/lib/i386-linux-gnu/dri/i965_drv_video.so libva info: Found init function __vaDriverInit_0_36 libva info: va_openDriver() returns 0 vainfo: VA-API version: 0.36 (libva 1.4.1) vainfo: Driver version: Intel i965 driver for Intel(R) CherryView - 1.4.1 vainfo: Supported profile and entrypoints VAProfileMPEG2Simple : VAEntrypointVLD VAProfileMPEG2Simple : VAEntrypointEncSlice VAProfileMPEG2Main : VAEntrypointVLD VAProfileMPEG2Main : VAEntrypointEncSlice ... Yet when I try to play a video with mpv --hwdec=vaapi filename, I get a totally broken video output: Now, the picture does change with the rhythm which matches the audio, suggesting the decoding does happen, but the result is completely unusable. Does anyone happen to have experience with accelerated video on Intel hardware? Any ideas of what I could be doing wrong? Here's what my /var/log/Xorg.0.log says: [ 11.754] (II) intel(0): SNA initialized with Cherryview (gen8) backend [ 11.755] (==) intel(0): Backing store enabled [ 11.755] (==) intel(0): Silken mouse enabled [ 11.755] (II) intel(0): HW Cursor enabled [ 11.755] (II) intel(0): RandR 1.2 enabled, ignore the following RandR disabled message. [ 11.756] (==) intel(0): DPMS enabled [ 11.757] (==) intel(0): Display hotplug detection enabled [ 11.757] (II) intel(0): [DRI2] Setup complete [ 11.757] (II) intel(0): [DRI2] DRI driver: i965 [ 11.757] (II) intel(0): [DRI2] VDPAU driver: va_gl [ 11.757] (II) intel(0): direct rendering: DRI2 enabled [ 11.757] (II) intel(0): hardware support for Present enabled