diff options
Diffstat (limited to 'content/devops/containers_using_systemd-nspawn.rst')
| -rw-r--r-- | content/devops/containers_using_systemd-nspawn.rst | 324 |
1 files changed, 324 insertions, 0 deletions
diff --git a/content/devops/containers_using_systemd-nspawn.rst b/content/devops/containers_using_systemd-nspawn.rst new file mode 100644 index 0000000..f5a71fe --- /dev/null +++ b/content/devops/containers_using_systemd-nspawn.rst @@ -0,0 +1,324 @@ +Taming systemd-nspawn for running containers +############################################ + +:date: 2015-11-09 12:52 +:slug: taming_systemd_nsapwn +:tags: systemd-nspawn, containers, debian +:author: copyninja +:summary: Brief notes on maintaining container using systemd-nspawn + +I've been trying to run containers using systemd-nspawn for quite some +time. I was always bumping to one or other dead end. This is not +systemd-nspawn's fault, rather my impatience stopping me from reading +manual pages properly lack of good tutorial like article available +online. Compared to this LXC has a quite a lot of good tutorials and +howto's available online. + +This article is my effort to create a notes putting all required +information in one place. + + +Creating a Debian Base Install +============================== + +First step is to have a minimal Debian system some where on your hard +disk. This can be easily done using *debootsrap*. I wrote a custom +script to avoid reading manual every time I want to run +debootstrap. Parts of this script (mostly packages and the root +password generation) is stolen from *lxc-debian* template provided by +lxc package. + +.. code-block:: shell + + #!/bin/sh + + set -e + set -x + + usage () { + echo "${0##/*} [options] <suite> <target> [<mirror>]" + echo "Bootstrap rootfs for Debian" + echo + cat <<EOF + --arch set the architecture to install + --root-passwd set the root password for bootstrapped rootfs + EOF + } + + # copied from the lxc-debian template + packages=ifupdown,\ + locales,\ + libui-dialog-perl,\ + dialog,\ + isc-dhcp-client,\ + netbase,\ + net-tools,\ + iproute,\ + openssh-server,\ + dbus + + if [ $(id -u) -ne 0 ]; then + echo "You must be root to execute this command" + exit 2 + fi + + if [ $# -lt 2 ]; then + usage $0 + fi + + while true; do + case "$1" in + --root-passwd|--root-passwd=?*) + if [ "$1" = "--root-passwd" -a -n "$2" ]; then + ROOT_PASSWD="$2" + shift 2 + elif [ "$1" != "${1#--root-passwd=}" ]; then + ROOT_PASSWD="${1#--root-passwd=}" + shift 1 + else + # copied from lxc-debian template + ROOT_PASSWD="$(dd if=/dev/urandom bs=6 count=1 2>/dev/null|base64)" + ECHO_PASSWD="yes" + fi + ;; + --arch|--arch=?*) + if [ "$1" = "--arch" -a -n "$2" ]; then + ARCHITECTURE="$2" + shift 2 + elif [ "$1" != "${1#--arch=}" ]; then + ARCHITECTURE="${1#--arch=}" + shift 1 + else + ARCHITECTURE="$(dpkg-architecture -q DEB_HOST_ARCH)" + fi + ;; + *) + break + ;; + esac + done + + + + release="$1" + target="$2" + + if [ -z "$1" ] || [ -z "$2" ]; then + echo "You must specify suite and target" + exit 1 + fi + + if [ -n "$3" ]; then + MIRROR="$3" + fi + + MIRROR=${MIRROR:-http://httpredir.debian.org/debian} + + echo "Downloading Debian $release ..." + debootstrap --verbose --variant=minbase --arch=$ARCHITECTURE \ + --include=$packages \ + "$release" "$target" "$MIRROR" + + if [ -n "$ROOT_PASSWD" ]; then + echo "root:$ROOT_PASSWD" | chroot "$target" chpasswd + echo "Root password is '$ROOT_PASSWRD', please change!" + fi + +It just gets my needs done, if you don't like it feel free to modify +or use debootstrap directly. + +**!NB Please install dbus package in the minimal base install, +otherwise you will not be able to control the container using +machinectl** + +Manually Running Container and then persisting it +================================================= + +Next we need to run the container manually. This is done by using +following command. + +.. code-block:: shell + + systemd-nspawn -bD /path/to/container --network-veth \ + --network-bridge=natbr0 --machine=Machinename + +*--machine* option is not mandatory, if not specified systemd-nspawn +will take the directory name as machine name, and if you have +characters like - in the directory name it translates to hexcode \x2d +and controlling container with name becomes difficult. + +*--network-veth* specifies the systemd-nspawn to enable virtual +ethernet based networking and *--network-bridge* tells the bridge +interface on host system to be used by systemd-nspawn. These options +together constitutes *private networking* for container. If not +specified container can use host systems interface there by removing +network isolation of container. + +Once you run this command container comes up. You can now run +machinectl to control the container. Container can be persisted using +following command + +.. code-block:: shell + + machinectl enable container-name + +This will create a symbolic link of +*/lib/systemd/system/systemd-nspawn@service* to +*/etc/systemd/system/machine.target.wants/*. This allows you to start +or stop container using *machinectl* or *systemctl* command. Only +catch here is your base install should be in +*/var/lib/machines/*. What I do in my case is create a symbolic link +from my base container to */var/lib/machines/container-name*. + +**!NB Note that symbolic link name under /var/lib/machines should be +same as the container name you gave using --machine switch or the +directory name if you didn't specify --machine** + +Persisting Container Networking +=============================== + +We did persist the container in above step, but this doesn't persist +the networking options we provided in command +line. systemd-nspawn@.service provides following command to invoke +container. + +.. code-block:: ini + + ExecStart=/usr/bin/systemd-nspawn --quiet --keep-unit --boot --link-journal=try-guest --network-veth --settings=override --machine=%I + +To persist the bridge networking configuration we did in command line, +we need the help of systemd-networkd. So first we need to enable the +systemd-networkd.service on both container and the host system. + +.. code-block:: shell + + systemctl enable systemd-networkd.service + +Now inside the container, interfaces will be named as +*hostN*. Depending on how many interfaces we have N increments. In our +example case we had single interface, hence it will named as +*host0*. By default network interfaces will be down inside container, +hence systemd-networkd is needed to put it up. + +We put the following in /etc/systemd/network/host0.network file inside +the container. + +.. code-block:: ini + + [Match] + Name=host0 + + [Network] + Description=Container wired interface host0 + DHCP=yes + +And in the host system we just configure the bridge interface using +systemd-nspawn. I put following in *natbr0.netdev* in +/etc/systemd/network/ + +.. code-block:: ini + + [NetDev] + Description=Bridge natbr0 + Name=natbr0 + Kind=bridge + +In my case I already had configured the bridge using +*/etc/network/interfaces* file for lxc. I think its not really needed +to use systemd-networkd in this case. Since systemd-networkd doesn't +do anything if network / virtual device is already present I safely +put above configuration and enabled systemd-networkd. + +Just for the notes here is my natbr0 configuration in *interfaces* +file. + +.. code-block:: interfaces + + auto natbr0 + iface natbr0 inet static + address 172.16.10.1 + netmask 255.255.255.0 + pre-up brctl addbr natbr0 + post-down brctl delbr natbr0 + post-down sysctl net.ipv4.ip_forward=0 + post-down sysctl net.ipv6.conf.all.forwarding=0 + post-up sysctl net.ipv4.ip_forward=1 + post-up sysctl net.ipv6.conf.all.forwarding=1 + post-up iptables -A POSTROUTING -t mangle -p udp --dport bootpc -s 172.16.0.0/16 -j CHECKSUM --checksum-fill + pre-down iptables -D POSTROUTING -t mangle -p udp --dport bootpc -s 172.16.0.0/16 -j CHECKSUM --checksum-fill + +Once this is done just reload the *systemd-networkd* and make sure you +have dnsmasq or any other DHCP server running in your system. + +Now the last part is to tell systemd-nspawn to use the bridge +networking interface we have defined. This is done using +*container-name.nspawn* file. Put this file under +*/etc/systemd/nspawn* folder. + +.. code-block:: ini + + [Exec] + Boot=on + + [Files] + Bind=/home/vasudev/Documents/Python/upstream/apt-offline/ + + [Network] + VirtualEthernet=yes + Bridge=natbr0 + +Here you can specify networking, and files mounting section of the +container. For full list please refer the *systemd.nspawn* manual +page. + + +Now all this is done you can happily do + +.. code-block:: shell + + machinectl start container-name + #or + systemctl start systemd-nspawn@container-name + +Resource Control +================ + +Now all things said and done, one last part remains. Yes what is the +point if we can't control how much resource does the container +use. Atleast it is more important for me, because I use old and bit +low powered laptop. + +Systemd provides way to control the resource using Control +interface. To see all the the interfaces exposed by systemd please +refer *systemd.resource-control* manual page. + +The way to control the resource is using *systemctl*. Once container +starts running we can run following command. + +.. code-block:: shell + + systemctl set-property container-name CPUShares=200 CPUQuota=30% MemoryLimit=500M + +The manual page does say that these things can be put under [Slice] +section of unit files. Now I don't have clear idea if this can be put +under .nspawn files or not. For the sake of persisting the container I +manually wrote the service file for container by copying +systemd-nspawn@.service and adding [Slice] section. But if I don't +know how to find out if this had any effect or not. + +If some one knows about this please share your suggestions to me and I +will update this section with your provided information. + +Conclussion +=========== + +All in all I like systemd-nspawn a lot. I use it to run container for +development of *apt-offline*. I previously used lxc where all can be +controlled using a single config file. But I feel systemd-nspawn is +more tightly integrated with system than lxc. + +There is definitely more in systemd-nspawn than I've currently figured +out. Only thing is its not as popular as other alternatives and +definitely lacks good howto documentation.For now only way out is dig +the manual pages, scratch your head, pull your hair out and figure out +new possibilities in systemd-nspawn. ;-) |