Use cases for nested Docker containers
Running Docker within a container is generally not recommended for most production environments, although there are scenarios in which nested containers can be beneficial or even necessary.
Development and testing
Applications, scripts, and tools often run in Docker containers within isolated environments. This approach is hugely popular especially in testing and development. Docker containers allow Dockerfile testing in a method that does not impact the system it is hosted on. They are also valuable in areas such as training areas or labs where complex network configurations or systems need to be simulated.
Continuous Integration/Continuous Deployment (CI/CD)
Nested containers, within CI/CD pipelines such as Jenkins, GitLab CI or GitHub Actions, are capable of creating reproducible testing or building environments that are also isolated. Running Docker within Docker enables the building, testing and packaging of containerized applications as part of a CI/CD workflow.
Isolation and security
Nested containers are an effective way to create an additional layer of isolation between applications, which can be particularly useful in high-security situations. By running security tools and services within a nested container, they can be effectively isolated from the host system and other containers.
In this post, we’ll delve into the idea of Docker-in-Docker (dind), its possible uses, and how to execute Docker within a Docker container. The reasons to avoid the use of nested docker containers will also be covered, along with some superior alternatives.
Docker socket (Dood)
This technique is often referred to as DooD (Docker outside Docker) as it employs the Docker daemon from the host machine within a Docker container. This is achieved by mounting the host environment’s socket (/var/run/docker.sock) inside a container. docker.sock is a Unix socket file that the Docker daemon utilizes to listen for connections from the Docker CLI and other Docker APIs. This acts as a channel of communication between the Docker daemon and the clients, facilitating the exchange of commands and data.
The command below sends an HTTP request to the Docker daemon via the Unix socket at /var/run/docker.sock to retrieve version information about the Docker installation.
curl --unix-socket /var/run/docker.sock http://localhost/version
Retrieving the Docker version using a socket
Now, use docker.sock to run Docker inside a Docker container.
docker run -v /var/run/docker.sock:/var/run/docker.sock -ti docker:18.06
This command pulls the latest Docker image from the Docker Hub registry and creates a container. After you execute the command, you will find yourself inside the container with an interactive command prompt from which you can run commands.
Creating a Docker container using the docker.sock method
Here is a more detailed explanation of the above command:
- -v /var/run/docker.sock:/var/run/docker.sock: This positions the Docker daemon socket from your system host (/var/run/docker.sock positioned before the colon) inside the container along the identical path (/var/run/docker.sock positioned following the colon). The -v parameter is used to mount this.
- -ti: In a Docker command, the -ti flag is a fusion of -t and -i. The -t allocates a pseudo-TTY (terminal), while -i keeps STDIN open (interactive), facilitating interaction with the container via the terminal.
Our first step will be to import the most updated version of the httpd (Apache HTTP Server) image sourced from the Docker Hub repository directly to your machine.
docker pull httpd
Note: There was an image here showing the downloading of the Apache image inside the container, but it has been removed.
Verify that the Apache image has been pulled from the Docker repository.
Docker images
Now, create a container from the Apache image.
docker container run -dit --name=apache-container httpd
Next, verify that the Apache container has been created.
docker ps
Verifying the status of the Apache container
Verifying the status of the Apache Container
Here we have an Apache container functioning within a Docker container, essentially forming a nested Docker container.
dind tag
The approach described here employs a Docker image tagged with ‘dind’. In Docker parlance, dind stands for “Docker in Docker”. This tag is assigned to Docker images especially configured to facilitate the operation of the Docker daemon within a Docker container. The ‘dind’ image incorporates all necessary dependencies and configurations needed to launch a Docker daemon inside its container.
The use of the –privileged option is a requirement when working with the dind tag, as the Docker-in-Docker workings need a higher level of system access compared to normal containerized processes. In essence, we provide the nested Docker container with the same privileges as the Docker daemon running on the host.
This means that the nested Docker daemon within the container can interact with the host system almost as if it were running directly on the host, allowing it to manage containers, configure networking, access devices, and perform other system-level operations that the Docker daemon typically requires.
However, it’s important to note that this configuration can pose security risks, as it breaks down the isolation between the container and the host system, potentially exposing the host to malicious actions or misconfigurations from within the container.
The command below creates and starts a new detached Docker container named dind-container with elevated privileges, using the second image to facilitate Docker-in-Docker operations. A detached Docker container is one that runs in the background of your shell session instead of being attached to the terminal’s input/output.
docker run --privileged --name dind-container -d docker:stable-dind
If you now try to run a container inside this container, you might run into this error message:
docker: Error response from daemon: cgroups: cgroup mountpoint does not exist: unknown.
To avoid this issue, you have to mount a cgroup filesystem, specifically for the systemd controller. Systemd manages services and their associated cgroups. A cgroup (control group) is a Linux kernel feature that allows allocating resources for tasks and their children by partitioning processes into hierarchical groups. Docker provides process-level isolation by making use of cgroups.
In a scenario where Docker-in-Docker is employed, and the intention is to run a systemd-based container within another Docker container, the systemd process in the inner container might necessitate access and management of cgroups. Given that the outer Docker container is already crafting its own cgroup hierarchy, the inner container (which houses systemd) may not locate the default configuration of cgroup it assumes.
The set of command lines provided below achieve the mounting of systemd cgroup subsystem. This grants the systemd the capability to function correctly within the setting of the internal container by obtaining access into /sys/fs/cgroup/systemd.
sudo mkdir /sys/fs/cgroup/systemd
sudo mount -t cgroup -o none,name=systemd cgroup /sys/fs/cgroup/systemd
The next step would be for you to establish a connection with the dind-container using the docker exec command.
docker exec -it dind-container /bin/sh
After the successful connection to the dind-container, the next step involves the creation and mounting of the cgroup once again inside this container:
mkdir /sys/fs/cgroup/systemdmount -t cgroup -o none,name=systemd cgroup /sys/fs/cgroup/systemd
The subsequent step is to create an Nginx container within the nested container. This can be achieved by using the command as shown below:
docker container run -dit --name=nginx-container nginx:latest
Now, verify that the Nginx container is running using the docker ps command.
docker ps
Verifying Nginx container status
Sysbox runtime
Sysbox, developed by Nestbox, is a unique container runtime that enables system-level tasks to be executed inside containers, effectively turning Docker and OCI-based containers into virtual hosts. Unlike conventional container runtimes, Sysbox permits the use of low-level system applications as well as traditional software inside containers.
To begin, terminate and eliminate any current containers in operation using the instructions given below. When installing Sysbox without taking these precautions, you may encounter the following error:
dpkg: error processing package sysbox-ce (–configure):
installed sysbox-ce package post-installation script subprocess returned error exit status 1
This command stops all containers:
docker stop (docker ps -q)
Make sure that you no longer need the containers before you remove them all from your system.
docker rm -f $(docker ps -a -q)
Next, download the Sysbox package file using the wget command.
wget https://downloads.nestybox.com/sysbox/releases/v0.6.2/sysbox-ce_0.6.2-0.linux_amd64.deb rel="nofollow" target="_blank"
Now install the downloaded package file using the apt command.
sudo apt install ./sysbox-ce_0.6.2-0.linux_amd64.deb
Once you have Sysbox runtime installed on your system, create a Docker container with a Sysbox runtime flag.
docker run --runtime=sysbox-runc --name sysbox-container -d docker:dind
Creating a Docker container using the Sysbox method
Next, connect to the Sysbox-container.
docker exec -it sysbox-container /bin/sh
You can now run any Docker command inside the container. For example, to check which Docker version is installed inside the Docker container, run the following command in the container:
docker info
You will see something like this:
Showing Docker version information
Comparing nested Docker solutions
Docker Socket Mounting (DooD)
Advantages
Simplicity: The setup is easier when compared to true Docker-in-Docker. The ability to bind-mount the Docker socket into a container allows that particular container to directly communicate with the Docker daemon of the host.
Performance: With the absence of a nested Docker daemon layer, containers kickstarted by the inner Docker client have the ability to run directly on the Docker daemon of the host.
Compatibility: Majority of the time, functions efficiently with existing Docker tooling and calls for negligible modifications.
Disadvantages
Security: This poses as a significant concern. The container is granted total access to the Docker daemon, an element that carries potential risks such as undesired or harmful operations, inclusive of altering or accessing other containers.
State persistence: Modifications to Docker from within the container (like image pulls) have implications on the host system and remain even post the removal of the container.
Docker-in-Docker (dind tag)
Advantages
Isolation: The nested Docker environment offers separation from the host Docker. This separation provides a layer of security, ensuring host Docker operations aren’t disturbed by what’s happening in the nested environment.
Clean environment: This feature proves crucial, especially in CI/CD pipelines where an unpolluted Docker environment is desirable for each build or test run.
Disadvantages
Complexity: Introduces additional layers of networking, storage, etc., which can be challenging to configure and manage.
Performance: Running an entire Docker daemon inside a container can have overhead.
Storage: The inner Docker daemon has its own set of image layers and containers, which can consume more disk space.
Sysbox Runtime
Benefits
Safety: Created to offer firm separation between the container and the host, even in instances where system-level workloads are managed inside the container.
Versatility: Has the capacity to handle a range of system-level workloads within containers devoid of any distinctive setup (such as system services, Docker, Kubernetes, etc.).
Efficiency: Tailored for system-level workloads, possibly extending superior performance compared to the customary Docker-in-Docker.
Disadvantages
Maturity: Sysbox is newer in comparison to traditional Docker setups, so there might be concerns regarding its maturity or community support.
Compatibility: Requires the Sysbox runtime, so there might be considerations when integrating with existing systems or tools that expect the default Docker runtime.
In conclusion, the method you choose will largely depend on your specific needs, security concerns, and the environment in which you’re operating. Each method has its unique strengths and weaknesses, so it’s crucial to evaluate based on the criteria most important to your use case.
Downside of nested Docker containers
Nested Docker containers can lead to issues. Here are some reasons why you should avoid this approach: