How to run Docker inside of Docker easily

There are some cases in the container world that you might run into docker in docker phenomena. Even though the docker in docker is a perfectly valid scenario, it should be avoided as much as possible. But if you cannot, then the only way is to make it work.
But before jumping into the solution, it is vital to have a clear understanding of the problem statement. For that purpose, I have prepared a detailed example as follow:

Assume that there is a Jenkins docker image that is running in a barebone EC2 instance, let’s call this docker Jenkins. This instance, needs to run some integration tests which rely on test containers, let say Elasticsearch container. So in this scenario, our Jenkins run a test suite which bootstraps an Elasticsearch docker. This is exactly the scenario known as the docker in docker.

In first glance, it seems that Elasticsearch container is a child of the Jenkins container. Well, that is not true at all. To know why we first need to know how Jenkins docker works underneath.

Basically, the so-called parent container, Jenkins in this example, cannot have a separate independent docker environment by itself. It inevitably relies on host docker. Means that both Jenkins and Elasticsearch containers must run on the same docker environment which is the host machine docker. And since both are running on the same docker they are more or less are siblings to each other.

But why should we care about this at all? Well, we shouldn’t care about it per se but without knowing the internal of the docker, we can’t solve the problem.

Let’s circle back to the example. To solve the problem, we need to give host docker access to the Jenkins container. For that, the Jenkins container needs to have the same group id and be in the same user groupas the host machine. To achieve that, we need to add the following script to the Jenkins entrypoint.shfile.

Now the Jenkins container can access to the host docker since they are in the same user group name with the same group id.

However, our problem has not solved yet. That is because Elasticsearch is not able to run under the rootuser which is the default user in our Jenkins container. To solve the problem, we need to create another user inside of the Jenkins container. So we easily can modify the script to accommodate that. The final script is like below:

If you like to be more adventurous you can even have a docker network between the containers. To know more about the network, have a look at this article.