Google/Android Device Backup taking up a lot of space on my Google cloud storage. How do I shrink it?

I was at 99% of my Google drive storage the other day, so I went into my Google storage and deleted an Android phone backup for an old phone stored on Google storage, that took up about 3gb, with my current phone backup taking about 3.5 GB. Fast forward a few days, I get that annoying red notice on my profile again, and now I check and my phone back up is now taking up 6 GB! It seems like it will take up what ever storage it is able to? I don't want to delete my phone backups outright, but I also want a couple gigs of storage for things I need, and to get rid of that red notice. Is there a way to choose what is backed up to this category (I have photos backup turned off) or to cap the size of the backup? All I really want backed up are device preferences and text messages. I definitely don't need 6+ GB of my app data and device preferences backed up.