DRM initializing very late

I'm porting an embedded Linux device to mainline from a vendor kernel. I am almost there but there is one issue I can't seem to get rid of, which is DRM initializing very slowly. Here's the setup: SOC: imx8mp Kernel: 6.12 OS: Debian 12 bookworm The gpu seems to initialize at a reasonable time:

[    4.984644] etnaviv etnaviv: bound 38000000.gpu (ops gpu_ops [etnaviv])
[    5.005189] etnaviv etnaviv: bound 38008000.gpu (ops gpu_ops [etnaviv])
[    5.038057] etnaviv etnaviv: bound 38500000.npu (ops gpu_ops [etnaviv])
[    5.048542] etnaviv-gpu 38000000.gpu: model: GC7000, revision: 6204
[    5.060472] etnaviv-gpu 38008000.gpu: model: GC520, revision: 5341
[    5.087467] etnaviv-gpu 38500000.npu: model: GC8000, revision: 8002
[    5.101070] etnaviv-gpu 38500000.npu: etnaviv has been instantiated on a NPU, for which the UAPI is still experimental
[    5.121842] [drm] Initialized etnaviv 1.4.0 for etnaviv on minor 0

systemd takes over at about:

[    2.318927] systemd: systemd 252.33-1~deb12u1 running in system mode (+PAM +AUDIT
+SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL
+ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT
+QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT
default-hierarchy=unified)

But then there is a second round of DRM initialization which actually enables the display stream:

[    6.904837] imx-dwmac 30bf0000.ethernet end0: Register MEM_TYPE_PAGE_POOL RxQ-0
[    6.971593] imx-dwmac 30bf0000.ethernet end0: PHY [stmmac-0:00] driver [SMSC LAN8710/LAN8720] (irq=147)
[    6.987060] imx-dwmac 30bf0000.ethernet end0: No Safety Features support found
[    6.994331] imx-dwmac 30bf0000.ethernet end0: IEEE 1588-2008 Advanced Timestamp supported
[    7.002765] imx-dwmac 30bf0000.ethernet end0: registered PTP clock
[    7.009583] imx-dwmac 30bf0000.ethernet end0: configuring for phy/rmii link mode
[    8.601862] imx-dwmac 30bf0000.ethernet end0: Link is Up - 100Mbps/Full - flow control rx/tx
[   15.334021] samsung-dsim 32e60000.dsi: supply vddcore not found, using dummy regulator
[   15.343694] samsung-dsim 32e60000.dsi: supply vddio not found, using dummy regulator
[   15.365379] samsung-dsim 32e60000.dsi: [drm:samsung_dsim_host_attach [samsung_dsim]] Attached sn65dsi83 device (lanes:4 bpp:24 mode-flags:0x2e3)
[   15.380621] [drm] Initialized imx-lcdif 1.0.0 for 32e80000.display-controller on minor 1
[   15.522131] Console: switching to colour frame buffer device 240x45
[   15.555056] imx-lcdif 32e80000.display-controller: [drm] fb0: imx-lcdifdrmfb frame buffer device
[   15.592315] dw100 32e30000.dwe: dw100 v4l2 m2m registered as /dev/video0
[   15.601817] hantro-vpu 38300000.video-codec: registered nxp,imx8mm-vpu-g1-dec as /dev/video1
[   15.611540] hantro-vpu 38310000.video-codec: registered nxp,imx8mq-vpu-g2-dec as /dev/video2

There is just a random gap of 10 seconds between these two rounds of initialization and I don't understand why. systemd-analyze:

systemd-analyze
Startup finished in 2.034s (kernel) + 4.558s (userspace) = 6.592s 
graphical.target reached after 4.450s in userspace.

Thanks in advance for any help!