Creating containers in Linux post 2 of several

Linux containers

In blog post 1, I defined containers and how they are distinguished from virtualization.
This post focuses on Linux containers.
LXC is an acronym for LinuXContainer.  LXC is not a container, but a set of tools that interfaces with kernel namespaces, cgroups etc to create and manage containers.
There are two types of containers, privileged, run lxc commands as root, and non-privileged, run lxc commands as a non-root user.

Container concepts

  1. cgroups
  2. Namespaces
  3. Templates
  4. Networking


Linux control groups, controls how resources are used in a container: Memory, disk and network i/o, guarantee CPU time, lock a container to a specific CPU and provide resource accounting.


This allows containers to be isolated from one another. Groups of processes are separated and cannot “see” resources in other groups. There are several namespaces:

  1. The PID namespace isolates process identifiers and their details.
  2. Network namespace isolates physical or virtual NICs, firewall rules, routing tables etc.
  3. “UTS” namespace allows different hostnames
  4. The Mount namespace allows different file system layout, or specific mount points read-only.
  5. User namespace isolates user IDs between namespaces.


Templates define:

  • root filesystem
  • selinux on or off
  • hostname
  • networking
  • root password
  • services – define some; remove others that are superfluous
  • etc

Templates are available per Linux distribution and those shipped with lxc are found here: /usr/share/lxc/templates.

Creating and networking containers

jonathan@ubuntu:~$ ls /usr/share/lxc/templates
lxc-alpine lxc-archlinux lxc-centos lxc-debian lxc-fedora lxc-openmandriva lxc-oracle lxc-sshd lxc-ubuntu-cloud lxc-altlinux lxc-busybox lxc-cirros lxc-download lxc-gentoo lxc-opensuse lxc-plamo lxc-ubuntu

Let’s see cirros (or any other template) details.

more /usr/share/lxc/templates/lxc-cirros

This command creates a cirros container

sudo lxc-create --template /usr/share/lxc/templates/lxc-cirros --name c2

Another alternative is to download a template from a server.

sudo lxc-create --template download --name c1
This provides a list of images to choose from and I chose:
  • Distribution: ubuntu
  • Release: trusty
  • Architecture: amd64

Now I have two containers, c1 and c2

$ sudo lxc-ls -f
c1    STOPPED  -     -     NO         
c2    STOPPED  -     -     NO

Where are the containers c1 and c2 stored on the host?

$sudo ls -alt /var/lib/lxc
drwx------  4 root root 4096 Jul  1 10:59 .
drwxrwx---  3 root root 4096 Jul  1 10:58 c2
drwxrwx---  3 root root 4096 Jul  1 10:20 c1
drwxr-xr-x 41 root root 4096 Jul  1 09:56 ..


LXC creates a private network namespace for each container, which includes a layer 2 networking stack. A container can exist without a private network namespace, and will only have access to the host network.

Physical NICs: Containers connect to the outside world by a physical NIC or veth tunnel endpoint connected to  the container. But a NIC can only exist in one namespace at a time, so a physical NIC cannot simultaneously connect to the host and a container.

Bridge: LXC creates a bridge, lxcbr0, at host startup.

Containers created using the default configuration will have one veth NIC with the remote end connected to the lxcbr0 bridge.

Thus my cirros container has an IP which is only accessible from the host ‘ubuntu’,

jonathan@ubuntu:~$ ping
PING ( 56(84) bytes of data.
64 bytes from icmp_seq=1 ttl=64 time=0.060 ms

not other machines.

us-jonathan-mac:~ jonathan$ ping
PING ( 56 data bytes
Request timeout for icmp_seq 0

While it can resolve remote servers, it cannot reach them

$ ip addr show
12: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
 link/ether 00:16:3e:aa:df:dd brd ff:ff:ff:ff:ff:ff
 inet brd scope global eth0
 valid_lft forever preferred_lft forever
 inet6 fe80::216:3eff:feaa:dfdd/64 scope link 
 valid_lft forever preferred_lft forever
$ route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default UG 0 0 0 eth0 * U 0 0 0 eth0
$ ping
PING ( 56 data bytes

The container config file tells me that it is currently set to bridge to the host only. Network type veth means virtual ethernet and it is linked to the host by br0

$ sudo more /var/lib/lxc/c2/config
# Network configuration = veth = up = br0 = 00:16:3e:13:38:b8

So I need to setup a bridge, br0, between the host and the container. So on the host I setup:

$ more /etc/network/interfaces
# loopback
auto lo
iface lo inet loopback
# primary
auto eth0
iface eth0 inet static
auto br0
iface br0 inet dhcp
bridge_ports eth0

So now from my Cirros container, I can ping the outside world since it picked up a dhcp address.

login as 'cirros' user. default password: 'cubswin:)'. use 'sudo' for root.cirros login: cirros
$ ip addr show
eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000 link/ether 00:16:3e:aa:df:dd brd ff:ff:ff:ff:ff:ff net brd scope ...
$ ping
PING ( 56 data bytes
64 bytes from seq=0 ttl=112 time=88.989 ms

Check container configuration

# lxc-checkconfig
Kernel configuration not found at /proc/config.gz; searching...
Kernel configuration found at /boot/config-3.10.0-229.el7.x86_64
--- Namespaces ---
Namespaces: enabled
Utsname namespace: enabled
Ipc namespace: enabled
Pid namespace: enabled
User namespace: enabled
Network namespace: enabled
Multiple /dev/pts instances: enabled
--- Control groups ---
Cgroup: enabled
Cgroup clone_children flag: enabled
Cgroup device: enabled
Cgroup sched: enabled
Cgroup cpu account: enabled
Cgroup memory controller: enabled
Cgroup cpuset: enabled
--- Misc ---
Veth pair device: enabled
Macvlan: enabled
Vlan: enabled
File capabilities: enabled

I can show the bridge info

$ brctl show
bridge name bridge id STP enabled interfaces
br0 8000.000c290a1957 no eth0

And get info on the running container to see how it is connected to the bridge.

jonathan@ubuntu:~$ sudo lxc-info --name c2
Name:           c2
State:          RUNNING
PID:            1717
IP:             2601:647:4b00:c3:216:3eff:feaa:dfdd
CPU use:        0.15 seconds
BlkIO use:      3.98 MiB
Memory use:     2.62 MiB
KMem use:       0 bytes
Link:           vethY0OLAB
TX bytes:      4.41 KiB
RX bytes:      348.96 KiB
Total bytes:   353.37 KiB

Next post… containers in public clouds, AWS and Google