Ignore vlan tagged packets on primary interface

I have an Ubuntu VM on ESXi with a single network interface which is a port group tagged as VLAN 4095 so it has access to all tagged VLANs. The native VLAN is 37. I've created several vlan interfaces using Netplan like this:

network:
  ethernets:
    ens160:
      dhcp4: true
  version: 2
  vlans:
    vlan98:
      id: 98
      link: ens160
      dhcp4: true
      optional: true
      link-local: []

I have a service receiving multicast traffic on all interfaces, including the primary ens160, and I want the traffic to remain separate for each interface. The problem I'm facing is: ens160 receives *all* packets, including tagged ones. Is there a way to make this primary interface *only* see untagged packets?

# ip -d link show
2: ens160:  mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
    link/ether 00:0c:29:07:84:53 brd ff:ff:ff:ff:ff:ff promiscuity 13 minmtu 60 maxmtu 9000 addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 
25: vlan98@ens160:  mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 00:0c:29:07:84:53 brd ff:ff:ff:ff:ff:ff promiscuity 1 minmtu 0 maxmtu 65535 
    vlan protocol 802.1Q id 98  addrgenmode none numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535

Example showing the packet I would like to exclude:

# tcpdump -ni ens160 -e host 10.37.154.4
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens160, link-type EN10MB (Ethernet), capture size 262144 bytes
15:55:00.023677 00:60:74:fb:d0:87 > 01:00:5e:00:01:81, ethertype 802.1Q (0x8100), length 90: vlan 98, p 0, ethertype IPv4, 10.37.154.4.319 > 224.0.1.129.319: UDP, length 44

I tried disabling dhcp on ens160 and then adding an additional interface for vlan37 but that brings networking down entirely.