iptables: drop any ip not on whitelist, short circuiting chain

I have a whitelist of ip addresses I'm storing in a ipset. I want to craft an iptables rule for my input chain where any IP NOT on the whitelist gets dropped immediately and no rules further down the chain get considered. If a ip matches an address on the whitelist then it continues down the Chain, checking other rules. If I just put a default policy of DROP and an ALLOW rule based on the whitelist, ip addresses not on the whitelist might be compared against other rules in the chain and allowed through based on those criteria, which I do not want. I also don't want to immediately let through traffic matching the whitelist rule (I guess whitelist is a but of a misnomer here) but, rather, apply the traffic to further scrutiny. Does iptables support this "DROP on not match" logic?