[arch-general] Bridge interface with netctl
Dear list, I am messing with assigning a static IP on a systemd-nspawn container, on both sides (host,container). My idea is to first create a bridge (br0) on the host. My config on host : systemd-git netctl to configure network ******************** $ ifocnfig enp7s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.1.87 netmask 255.255.255.0 broadcast 192.168.1.255 lo... virbr0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500 <<< libvirt inet 192.168.122.1 netmask 255.255.255.0 broadcast 192.168.122.255 ************** % netctl list bridge-hortensia ************* % systemctl --failed UNIT LOAD ACTIVE SUB DESCRIPTION systemd-back...klight:eeepc-wmi.service loaded failed failed Load/Save Screen Backlight Brightness o <<<< not important . **************** % systemctl status dhcpcd.service dhcpcd.service - dhcpcd on all interfaces Loaded: loaded (/usr/lib/systemd/system/dhcpcd.service; enabled) Active: active (running) since Tue 2014-02-25 12:14:04 CET; 21min ago Process: 320 ExecStart=/bin/dhcpcd -q -b (code=exited, status=0/SUCCESS) Main PID: 329 (dhcpcd) CGroup: /system.slice/dhcpcd.service └─329 /bin/dhcpcd -q -b Feb 25 12:22:30 hortensia dhcpcd[329]: br0: removing interface Feb 25 12:22:30 hortensia dhcpcd[329]: enp7s0: carrier lost Feb 25 12:22:31 hortensia dhcpcd[329]: enp7s0: carrier acquired Feb 25 12:22:31 hortensia dhcpcd[329]: enp7s0: soliciting an IPv6 router Feb 25 12:22:31 hortensia dhcpcd[329]: enp7s0: rebinding lease of 192.168.1.87 Feb 25 12:22:40 hortensia dhcpcd[329]: enp7s0: leased 192.168.1.87 for 86400 seconds Feb 25 12:22:40 hortensia dhcpcd[329]: enp7s0: adding host route to 192.168.1.87 via 127.0.0.1 Feb 25 12:22:40 hortensia dhcpcd[329]: enp7s0: adding route to 192.168.1.0/24 Feb 25 12:22:40 hortensia dhcpcd[329]: enp7s0: adding default route via 192.168.1.254 Feb 25 12:22:45 hortensia dhcpcd[329]: enp7s0: no IPv6 Routers available ********************** gabx@hortensia ➤➤ ~ % cat /etc/netctl/bridge-hortensia Description="Example Bridge connection" Interface=br0 Connection=bridge BindsToInterfaces=enp7s0 IP=no ********************* Now, I want to start the bridge: # netctl start bridge-hortensia % ping -c 3 8.8.8.8 connect: Network is unreachable Same result if I change IP=dhcp I can't see where does the issue come from. Shall I enable a "ethernet-dhcp" netctl profile and disable dhcpcd.service ? Thank you for help.
Now, I want to start the bridge: # netctl start bridge-hortensia % ping -c 3 8.8.8.8 connect: Network is unreachable
To be more precise, here is the output of ip addr when starting the bridge profile. As you can see, there is no more IP adress assigned on enp7s0 neither on br0 % ip addr 2: enp7s0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br0 state UP group default qlen 1000 link/ether 14:da:e9:b5:7a:88 brd ff:ff:ff:ff:ff:ff inet6 fe80::16da:e9ff:feb5:7a88/64 scope link valid_lft forever preferred_lft forever 7: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 14:da:e9:b5:7a:88 brd ff:ff:ff:ff:ff:ff inet6 fe80::31:82ff:fe4f:3e08/64 scope link valid_lft forever preferred_lft forever
On Tue, Feb 25, 2014 at 1:43 PM, arnaud gaboury <arnaud.gaboury@gmail.com> wrote:
Now, I want to start the bridge: # netctl start bridge-hortensia % ping -c 3 8.8.8.8 connect: Network is unreachable
I realized that starting the bridge profile empty the /etc/resolv.conf. Even if I modified the bridge profile, following the man page: Description="Example Bridge connection" Interface=br0 Connection=bridge BindsToInterfaces=(enp7s0) IP=no #Adress=('192.168.1.94/24') #Gateway='192.168.1.254' #DNS=('212.147.10.180' '212.147.10.162') DNSDomain=lan DNSSearch=('192.168.1.254') <<< these two last lines are supposed to pass these settings to /etc/resolv.conf but nada :-(
On 25.02.14 at 16:33, arnaud gaboury wrote:
On Tue, Feb 25, 2014 at 1:43 PM, arnaud gaboury <arnaud.gaboury@gmail.com> wrote:
Now, I want to start the bridge: # netctl start bridge-hortensia % ping -c 3 8.8.8.8 connect: Network is unreachable
I realized that starting the bridge profile empty the /etc/resolv.conf.
Even if I modified the bridge profile, following the man page:
Description="Example Bridge connection" Interface=br0 Connection=bridge BindsToInterfaces=(enp7s0) IP=no #Adress=('192.168.1.94/24') #Gateway='192.168.1.254' #DNS=('212.147.10.180' '212.147.10.162') DNSDomain=lan DNSSearch=('192.168.1.254') <<< these two last lines are supposed to pass these settings to /etc/resolv.conf
but nada :-(
DNS is just a cherry on the cake, first you need to establish the network connection. I'm not using netctl for some time, but I'm pretty sure it does not launch any DHCP server. If you want to use DHCP, you will need to start the server on your own. I have written several scripts [1-4] to manage a bridge interface for QEMU VMs, see if you can reuse some of it. Especially [2] might be useful for you, though you probably do not need to create the tap interface. Note that the original author of [3] and [4] is Xyne, see his excellent tutorial [5]. [1]: https://github.com/lahwaacz/archlinux-dotfiles/blob/master/Scripts/qemu-laun... [2]: https://github.com/lahwaacz/archlinux-dotfiles/blob/master/Scripts/qemu-tap-... [3]: https://github.com/lahwaacz/archlinux-dotfiles/blob/master/Scripts/nat-launc... [4]: https://github.com/lahwaacz/archlinux-dotfiles/blob/master/Scripts/nat-launc... [5]: http://xyne.archlinux.ca/notes/network/dhcp_with_dns.html Regards, -- jlk
Note that the original
author of [3] and [4] is Xyne, see his excellent tutorial [5].
[5]: http://xyne.archlinux.ca/notes/network/dhcp_with_dns.html
Regards,
Thank you for this very interesting link
On Tue, 25 Feb 2014 12:47:02 +0100 arnaud gaboury <arnaud.gaboury@gmail.com> wrote:
Dear list,
I am messing with assigning a static IP on a systemd-nspawn container, on both sides (host,container).
You assign an IP address only inside the container.
My idea is to first create a bridge (br0) on the host. [...] gabx@hortensia ➤➤ ~ % cat /etc/netctl/bridge-hortensia Description="Example Bridge connection" Interface=br0 Connection=bridge BindsToInterfaces=enp7s0 IP=no
This profile is wrong. Here is the right one: --- $ cat /etc/netctl/lxc_lan_bridge Description="LAN bridge for LXC containers" Connection=bridge Interface=br0 SkipNoCarrier="yes" BindsToInterfaces=() IP=static Address=(10.137.0.1/24) --- Also, since you are running systemd >= 209, you can use networkd. Here are the config files: --- $ cat /etc/systemd/network/lxc_bridge.netdev [NetDev] Name=br0 Kind=bridge $ cat /etc/systemd/network/lxc_bridge.network [Match] Name=br0 [Network] Description=LAN bridge for LXC containers DHCP=false [Address] Address=10.137.0.1/24 --- Now: * Populate the iptables FORWARD chain to route traffic from your physical interface to the bridge and back. * Inside the container, get rid of dhcpcd, and use either networkd or netctl to assign a static IP like you would do on a physical host. * The simplest way to take care of DNS (that is without any additional host configuration) is to use a static resolv.conf inside the container with a public DNS service, e.g. --- $ cat /var/lib/lxc/appserver1/rootfs/etc/resolv.conf nameserver 8.8.8.8 nameserver 8.8.4.4 --- Cheers, -- Leonid Isaev GPG key fingerprint: C0DF 20D0 C075 C3F1 E1BE 775A A7AE F6CB 164B 5A6D
Op 25 feb. 2014 20:56 schreef "Leonid Isaev" <lisaev@umail.iu.edu> het volgende:
[...]
* The simplest way to take care of DNS (that is without any additional host configuration) is to use a static resolv.conf inside the container with a public DNS service, e.g. --- $ cat /var/lib/lxc/appserver1/rootfs/etc/resolv.conf nameserver 8.8.8.8 nameserver 8.8.4.4
This seems a bit weird; personally i would prefer something like dnsmasq, or at least some internal dns server. Unless the container is only used for browsing/serving on the Internet. Using public DNS servers, you won't be able to resolv local addresses. mvg, Guus
This profile is wrong. Here is the right one: --- $ cat /etc/netctl/lxc_lan_bridge Description="LAN bridge for LXC containers" Connection=bridge Interface=br0 SkipNoCarrier="yes" BindsToInterfaces=() IP=static Address=(10.137.0.1/24) --- Also, since you are running systemd >= 209, you can use networkd. Here are the config files: --- $ cat /etc/systemd/network/lxc_bridge.netdev [NetDev] Name=br0 Kind=bridge $ cat /etc/systemd/network/lxc_bridge.network [Match] Name=br0
[Network] Description=LAN bridge for LXC containers DHCP=false
[Address] Address=10.137.0.1/24 ---
For now, I have a working setup, but I am not satisfied and I think I can improve it. ******* % cat /etc/netctl/dhcp-hortensia Description='A basic dhcp ethernet connection' Interface=enp7s0 Connection=ethernet IP=dhcp ********* This profile is enable and start at boot. Then I manually # start bridge-hortensia *********** % cat /etc/netctl/bridge-hortensia Description="Example Bridge connection" Interface=br0 Connection=bridge BindsToInterfaces=(enp7s0) IP=dhcp *************** What puzzles me is that IF I enable the bridge profile, my system boots with a borken network with an empty /etc/resolv.conf. I would like to overcome this issue. Shall I go static ? Shall I start a specific profile before the other one? Why my resolv.conf is left empty when enabling both profiles ? then my systemd-networkd : ****** % cat /etc/systemd/network/70-dahlia.netdev [Match] #Host=dahlia Virtualization=container [NetDev] Name=br0 Kind=bridge ******************* gabx@hortensia ➤➤ ~ % cat /etc/systemd/network/80-dahlia.network [Match] Virtualization=container MACAddress=14:da:e9:b5:7a:88 [Network] DHCP=yes [Address] Address=192.168.1.94 [Route] Gateway=192.168.1.254 ********************** Nothing on the container side, no netctl profile. This set up leave me with a working network. I can for example http://my_public_ip and then be on the nginx welcome page. But again this set up doesn't sound very academic neither solid to me. last: % ip addr 2: enp7s0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br0 state UP group default qlen 1000 link/ether 14:da:e9:b5:7a:88 brd ff:ff:ff:ff:ff:ff inet6 fe80::16da:e9ff:feb5:7a88/64 scope link valid_lft forever preferred_lft forever 4: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 14:da:e9:b5:7a:88 brd ff:ff:ff:ff:ff:ff inet 192.168.1.94/24 brd 192.168.1.255 scope global br0 valid_lft forever preferred_lft forever inet6 fe80::16da:e9ff:feb5:7a88/64 scope link valid_lft forever preferred_lft forever As you can see, 192.168.1.94/24 is attached to br0, but no IP for my eth interface. Thank you for your help fine tuning this set up. It took me lots of reading and work (yes) to find a way to setup correctly the container network (and other). Documentation on container administered by systemd-nspawn are spare if non existent. I am left with the systemd man page and systemd-dev mailing list for lonely friends.
--
Now: * Populate the iptables FORWARD chain to route traffic from your physical interface to the bridge and back.
I missed totally this part of the setup. I must admit this topic is a little bit new to me. Will try to go this way. -- google.com/+arnaudgabourygabx
On Wed, Feb 26, 2014 at 1:37 PM, arnaud gaboury <arnaud.gaboury@gmail.com> wrote:
--
Now: * Populate the iptables FORWARD chain to route traffic from your physical interface to the bridge and back.
I missed totally this part of the setup. I must admit this topic is a little bit new to me. Will try to go this way.
The more I read and try with various set up, the less I understand and the more I break my container :-( I first managed to solve this empty /etc/resolve.conf by using /etc/resolveconf.conf facility. But now, on the container, with the netctl and network files cited before, I can not connect to network anymore. *The weird part is that inside the container, the "$ ip addr " command does not return br0, but only lo. No idea why. * Then, when testing various kind of netctl profiles, I remarked using a static IP in my bridge profile breaks immediately the connection to network on host. At first, I thought it had to do with my empty /etc/resolve.conf, but nada. This file stays now correct. So I am now with 24 hours of more work and a broken network on container! Nice job.
On Wed, Feb 26, 2014 at 2:09 PM, arnaud gaboury <arnaud.gaboury@gmail.com> wrote:
On Wed, Feb 26, 2014 at 1:37 PM, arnaud gaboury <arnaud.gaboury@gmail.com> wrote:
--
Now: * Populate the iptables FORWARD chain to route traffic from your physical interface to the bridge and back.
I missed totally this part of the setup. I must admit this topic is a little bit new to me. Will try to go this way.
After more test, I do think I start to understand where do my issues come from. First of all, my kernel is a custom (need user space set) 3.13.5-1. I realized that no iptable modules were loaded. Then, starting with a first basic dhcp netctl profile (systemd-networkd disabled, dhcpcd.service disabled), my network is working. ************** gabx@hortensia ➤➤ ~ % ip addr 2: enp7s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 14:da:e9:b5:7a:88 brd ff:ff:ff:ff:ff:ff inet 192.168.1.87/24 brd 192.168.1.255 scope global enp7s0 valid_lft forever preferred_lft forever **************** gabx@hortensia ➤➤ ~ % route Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface default dsldevice.lan 0.0.0.0 UG 202 0 0 enp7s0 192.168.1.0 * 255.255.255.0 U 202 0 0 enp7s0 hortensia.lan localhost.local 255.255.255.255 UGH 202 0 0 lo ************** Then, when starting a seond netctl profile, a bridge br0 with static IP, my network is down. gabx@hortensia ➤➤ ~ % ip addr 2: enp7s0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br0 state UP group default qlen 1000 link/ether 14:da:e9:b5:7a:88 brd ff:ff:ff:ff:ff:ff 8: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 14:da:e9:b5:7a:88 brd ff:ff:ff:ff:ff:ff *************** gabx@hortensia ➤➤ ~ % route Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface **************************** So now let's read about iptable or even nftables. IF I understand : < * Populate the iptables FORWARD chain to route traffic from your physical interface to the bridge and back.> , I am far from being comfortable doing this.
Le 25/02/2014 12:47, arnaud gaboury a écrit :
Now, I want to start the bridge: # netctl start bridge-hortensia % ping -c 3 8.8.8.8 connect: Network is unreachable
To ping 8.8.8.8 you must have a default route or a gateway. What is the result of "ip route" or "route"? Regards, BARA Ewen
participants (5)
-
arnaud gaboury
-
Ewen Bara
-
Guus Snijders
-
Jakub Klinkovský
-
Leonid Isaev