Creating containers in Linux post 2 of several

Linux containers

In blog post 1, I defined containers and how they are distinguished from virtualization.
This post focuses on Linux containers.
LXC is an acronym for LinuXContainer.  LXC is not a container, but a set of tools that interfaces with kernel namespaces, cgroups etc to create and manage containers.
There are two types of containers, privileged, run lxc commands as root, and non-privileged, run lxc commands as a non-root user.

Container concepts

  1. cgroups
  2. Namespaces
  3. Templates
  4. Networking

Cgroups

Linux control groups, controls how resources are used in a container: Memory, disk and network i/o, guarantee CPU time, lock a container to a specific CPU and provide resource accounting.

Namespaces

This allows containers to be isolated from one another. Groups of processes are separated and cannot “see” resources in other groups. There are several namespaces:

  1. The PID namespace isolates process identifiers and their details.
  2. Network namespace isolates physical or virtual NICs, firewall rules, routing tables etc.
  3. “UTS” namespace allows different hostnames
  4. The Mount namespace allows different file system layout, or specific mount points read-only.
  5. User namespace isolates user IDs between namespaces.

Templates

Templates define:

  • root filesystem
  • selinux on or off
  • hostname
  • networking
  • root password
  • services – define some; remove others that are superfluous
  • etc

Templates are available per Linux distribution and those shipped with lxc are found here: /usr/share/lxc/templates.

Creating and networking containers

jonathan@ubuntu:~$ ls /usr/share/lxc/templates
lxc-alpine lxc-archlinux lxc-centos lxc-debian lxc-fedora lxc-openmandriva lxc-oracle lxc-sshd lxc-ubuntu-cloud lxc-altlinux lxc-busybox lxc-cirros lxc-download lxc-gentoo lxc-opensuse lxc-plamo lxc-ubuntu

Let’s see cirros (or any other template) details.

more /usr/share/lxc/templates/lxc-cirros

This command creates a cirros container

sudo lxc-create --template /usr/share/lxc/templates/lxc-cirros --name c2

Another alternative is to download a template from a server.

sudo lxc-create --template download --name c1
This provides a list of images to choose from and I chose:
  • Distribution: ubuntu
  • Release: trusty
  • Architecture: amd64

Now I have two containers, c1 and c2

$ sudo lxc-ls -f
NAME  STATE    IPV4  IPV6  AUTOSTART  
------------------------------------
c1    STOPPED  -     -     NO         
c2    STOPPED  -     -     NO

Where are the containers c1 and c2 stored on the host?

$sudo ls -alt /var/lib/lxc
drwx------  4 root root 4096 Jul  1 10:59 .
drwxrwx---  3 root root 4096 Jul  1 10:58 c2
drwxrwx---  3 root root 4096 Jul  1 10:20 c1
drwxr-xr-x 41 root root 4096 Jul  1 09:56 ..

Networking

LXC creates a private network namespace for each container, which includes a layer 2 networking stack. A container can exist without a private network namespace, and will only have access to the host network.

Physical NICs: Containers connect to the outside world by a physical NIC or veth tunnel endpoint connected to  the container. But a NIC can only exist in one namespace at a time, so a physical NIC cannot simultaneously connect to the host and a container.

Bridge: LXC creates a bridge, lxcbr0, at host startup.

Containers created using the default configuration will have one veth NIC with the remote end connected to the lxcbr0 bridge.

Thus my cirros container has an IP 10.0.2.120 which is only accessible from the host ‘ubuntu’,

jonathan@ubuntu:~$ ping 10.0.2.120
PING 10.0.2.120 (10.0.2.120) 56(84) bytes of data.
64 bytes from 10.0.2.120: icmp_seq=1 ttl=64 time=0.060 ms

not other machines.

us-jonathan-mac:~ jonathan$ ping 10.0.2.120
PING 10.0.2.120 (10.0.2.120): 56 data bytes
Request timeout for icmp_seq 0

While it can resolve remote servers, it cannot reach them

$ ip addr show
12: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
 link/ether 00:16:3e:aa:df:dd brd ff:ff:ff:ff:ff:ff
 inet 10.0.2.120/24 brd 10.0.2.255 scope global eth0
 valid_lft forever preferred_lft forever
 inet6 fe80::216:3eff:feaa:dfdd/64 scope link 
 valid_lft forever preferred_lft forever
$ route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default 10.0.2.1 0.0.0.0 UG 0 0 0 eth0
10.0.2.0 * 255.255.255.0 U 0 0 0 eth0
$ ping cnn.com
PING cnn.com (157.166.226.26): 56 data bytes

The container config file tells me that it is currently set to bridge to the host only. Network type veth means virtual ethernet and it is linked to the host by br0

$ sudo more /var/lib/lxc/c2/config
<snip>
# Network configuration
lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = br0
lxc.network.hwaddr = 00:16:3e:13:38:b8

So I need to setup a bridge, br0, between the host and the container. So on the host I setup:

$ more /etc/network/interfaces
# loopback
auto lo
iface lo inet loopback
# primary
auto eth0
iface eth0 inet static
address 10.0.1.16
netmask 255.255.255.0
gateway 10.0.1.1
#bridge
auto br0
iface br0 inet dhcp
bridge_ports eth0

So now from my Cirros container, I can ping the outside world since it picked up a dhcp address.

login as 'cirros' user. default password: 'cubswin:)'. use 'sudo' for root.cirros login: cirros
Password: 
$ ip addr show
eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000 link/ether 00:16:3e:aa:df:dd brd ff:ff:ff:ff:ff:ff net 10.0.1.16/24 brd 10.0.1.255 scope ...
$ ping cnn.com
PING cnn.com (157.166.226.25): 56 data bytes
64 bytes from 157.166.226.25: seq=0 ttl=112 time=88.989 ms

Check container configuration

# lxc-checkconfig
Kernel configuration not found at /proc/config.gz; searching...
Kernel configuration found at /boot/config-3.10.0-229.el7.x86_64
--- Namespaces ---
Namespaces: enabled
Utsname namespace: enabled
Ipc namespace: enabled
Pid namespace: enabled
User namespace: enabled
Network namespace: enabled
Multiple /dev/pts instances: enabled
--- Control groups ---
Cgroup: enabled
Cgroup clone_children flag: enabled
Cgroup device: enabled
Cgroup sched: enabled
Cgroup cpu account: enabled
Cgroup memory controller: enabled
Cgroup cpuset: enabled
--- Misc ---
Veth pair device: enabled
Macvlan: enabled
Vlan: enabled
File capabilities: enabled

I can show the bridge info

$ brctl show
bridge name bridge id STP enabled interfaces
br0 8000.000c290a1957 no eth0
vethY0OLAB
lxcbr08000.000000000000no

And get info on the running container to see how it is connected to the bridge.

jonathan@ubuntu:~$ sudo lxc-info --name c2
Name:           c2
State:          RUNNING
PID:            1717
IP:             10.0.1.16
IP:             2601:647:4b00:c3:216:3eff:feaa:dfdd
CPU use:        0.15 seconds
BlkIO use:      3.98 MiB
Memory use:     2.62 MiB
KMem use:       0 bytes
Link:           vethY0OLAB
TX bytes:      4.41 KiB
RX bytes:      348.96 KiB
Total bytes:   353.37 KiB

Next post… containers in public clouds, AWS and Google