Prevent network namespaces / lxd-bridges from talking to each other, but keep internet access?
1
vote
0
answers
192
views
Following goal:
- I have lxd containers
- each set of containers should have their dedicated isolated network
- each should still be able to connect to the internet (e.g. apt update or curl), but not leak ports (e.g. a webserver) to the internet, unless I lxd proxy them
I have tried to deny same subnet communication with 10.0.0.0/8 and alikes, but that would prevent it from getting an IP from LXD-dhcp or the internet.
To allow internet access to LXD currently I do:
sudo ufw allow in on
sudo ufw route allow in on
I have now attempted to create network-namespaces to control the flow manually, but as soon as the namespaces are connected to their respective bridge, they can cross-talk, and I'm at my starting point of trying to keep them separate.
I have come across many solutions that just suggest to e.g. deny bridge1 to bridge2, but that scales horribly, the more networks I'd have, the more rules going both ways I'd have to setup, even scripting that would be cumbersome and spam iptables with hundreds of rules.
Is there a way to achieve the above goal, but without having to add hundreds of rules to keep cross-talk away, some kind of default-off, opt-in communication? since maybe I would want some networks to communicate to each other, but by default I would want them all to keep to their own namespace/bridge.
The following diagram shows it for my current attempt of using network-namespaces:
(orange lines mark optional communication, I could live without it [and can currently toggle it via lxd port security], if it meant there's an easy solution for the rest, but would prefer to have the option to allow those communications if the need arises)
it has been bootstrapped by the following code:
#!/usr/bin/env bash
NS1="ns1"
VETH1="veth1"
VPEER1="vpeer1"
NS2="ns2"
VETH2="veth2"
VPEER2="vpeer2"
# clean up previous
ip netns del ${NS1} >/dev/null 2>&1
ip netns del ${NS2} >/dev/null 2>&1
ip link delete ${VETH1} >/dev/null 2>&1
ip link delete ${VETH2} >/dev/null 2>&1
ip link delete ${VETH1} >/dev/null 2>&1
ip link delete ${VETH2} >/dev/null 2>&1
# create namespace
ip netns add $NS1
ip netns add $NS2
# create veth link
ip link add ${VETH1} type veth peer name ${VPEER1}
ip link add ${VETH2} type veth peer name ${VPEER2}
# setup veth link
ip link set ${VETH1} up
ip link set ${VETH2} up
# add peers to ns
ip link set ${VPEER1} netns ${NS1}
ip link set ${VPEER2} netns ${NS2}
# setup loopback interface
ip netns exec ${NS1} ip link set lo up
ip netns exec ${NS2} ip link set lo up
# setup peer ns interface
ip netns exec ${NS1} ip link set ${VPEER1} up
ip netns exec ${NS2} ip link set ${VPEER2} up
# assign ip address to ns interfaces
VPEER_ADDR1="10.10.0.10"
VPEER_ADDR2="10.20.0.10"
ip netns exec ${NS1} ip addr add ${VPEER_ADDR1}/16 dev ${VPEER1}
ip netns exec ${NS2} ip addr add ${VPEER_ADDR2}/16 dev ${VPEER2}
setup_bridge() {
BR_ADDR="$1"
BR_DEV="$2"
NAMESPACE="$3"
VETH="$4"
# delete old bridge
ip link delete ${BR_DEV} type bridge >/dev/null 2>&1
# setup bridge
ip link add ${BR_DEV} type bridge
ip link set ${BR_DEV} up
# assign veth pairs to bridge
ip link set ${VETH} master ${BR_DEV}
# setup bridge ip
ip addr add ${BR_ADDR}/16 dev ${BR_DEV}
# add default routes for ns
ip netns exec ${NAMESPACE} ip route add default via ${BR_ADDR}
# enable ip forwarding
bash -c 'echo 1 > /proc/sys/net/ipv4/ip_forward'
# masquerade (internet => bridge)
iptables -t nat -A POSTROUTING -s ${BR_ADDR}/16 ! -o ${BR_DEV} -j MASQUERADE
}
# clear out postrouting
iptables -t nat -F
BR_IP="10.10.0.1"
BR_DEV="br0"
setup_bridge $BR_IP $BR_DEV $NS1 $VETH1
BR_IP="10.20.0.1"
BR_DEV="br1"
setup_bridge $BR_IP $BR_DEV $NS2 $VETH2
Thanks!

Asked by stackbox
(11 rep)
Aug 4, 2022, 03:09 AM