DOCKER

 VIRTUAL MACHINES

Virtual machines (VMs) are an abstraction of physical hardware turning one server into many servers. The hypervisor allows multiple VMs to run on a single machine. Each VM includes a full copy of an operating system, the application, necessary binaries and libraries - taking up tens of GBs. VMs can also be slow to boot.

CONTAINERS

Containers are an abstraction at the app layer that packages code and dependencies together. Multiple containers can run on the same machine and share the OS kernel with other containers, each running as isolated processes in user space. Containers take up less space than VMs (container images are typically tens of MBs in size), can handle more applications and require fewer VMs and Operating systems.

Types of Containers

Linux Containers (LXC) — The original Linux container technology is Linux Containers, commonly known as LXC. LXC is a Linux operating system level virtualization method for running multiple isolated Linux systems on a single host.

Docker — Docker started as a project to build single-application LXC containers, introducing several changes to LXC that make containers more portable and flexible to use. It later morphed into its own container runtime environment. At a high level, Docker is a Linux utility that can efficiently create, ship, and run containers.

Benefits of Containers

• Reduced IT management resources
• Reduced size of snapshots
• Quicker spinning up apps
• Reduced & simplified security updates
• Less code to transfer, migrate, upload workloads

Popular Container Providers

1) Linux Containers

• LXC
• LXD
• CGManager

2) Docker
3) Windows Server Containers

Differences between Virtual Machines and Containers

Virtual Machines	Containers
Heavy weight Limited performance Each VM runs in its own OS Hardware-level virtualization Startup time in minutes Allocates required memory Fully Isolated and hence more secure	Light weight Native Performance All containers share the host OS OS Virtualization Startup time in Milliseconds Requires less memory space Process-level isolation, possibly less secure

DOCKER ARCHITECTURE

Docker uses a client-server architecture. The Docker client talks to the Docker daemon, which does the heavy lifting of building, running, and distributing your Docker containers. The Docker client and daemon can run on the same system, or you can connect a Docker client to a remote Docker daemon. The Docker client and daemon communicate using a REST API, over UNIX sockets or a network interface.

The Docker daemon

The Docker daemon (dockerd) listens for Docker API requests and manages Docker objects such as images, containers, networks, and volumes. A daemon can also communicate with other daemons to manage Docker services.

The Docker client

The Docker client (docker) is the primary way that many Docker users interact with Docker. When you use commands such as docker run, the client sends these commands to dockerd, which carries them out. The docker command uses the Docker API. The Docker client can communicate with more than one daemon.

Docker registries

A Docker registry stores Docker images. Docker Hub is a public registry that anyone can use, and Docker is configured to look for images on Docker Hub by default. You can even run your own private registry.

When you use the docker pull or docker run commands, the required images are pulled from your configured registry. When you use the docker push command, your image is pushed to your configured registry.

Docker objects

When you use Docker, you are creating and using images, containers, networks, volumes, plugins, and other objects. This section is a brief overview of some of those objects.

IMAGES

An image is a read-only template with instructions for creating a Docker container. Often, an image is based on another image, with some additional customization. For example, you may build an image which is based on the ubuntu image, but installs the Apache web server and your application, as well as the configuration details needed to make your application run.

You might create your own images or you might only use those created by others and published in a registry. To build your own image, you create a Dockerfile with a simple syntax for defining the steps needed to create the image and run it. Each instruction in a Dockerfile creates a layer in the image. When you change the Dockerfile and rebuild the image, only those layers which have changed are rebuilt. This is part of what makes images so lightweight, small, and fast, when compared to other virtualization technologies.

CONTAINERS

A container is a runnable instance of an image. You can create, start, stop, move, or delete a container using the Docker API or CLI. You can connect a container to one or more networks, attach storage to it, or even create a new image based on its current state.

By default, a container is relatively well isolated from other containers and its host machine. You can control how isolated a container’s network, storage, or other underlying subsystems are from other containers or from the host machine.

A container is defined by its image as well as any configuration options you provide to it when you create or start it. When a container is removed, any changes to its state that are not stored in persistent storage disappear.

Docker Installation :

Install docker using the following commands

$ curl -fsSL https://get.docker.com -o get-docker.sh
$ sudo sh get-docker.sh

Check the docker version using the following command :

$ sudo docker version

Run a sample application using the following command

$ sudo docker run hello-world

vagrant@vagrant-ubuntu-trusty-64:~$ sudo docker run hello-world
docker: Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:297: copying bootstrap data to pipe caused \"write init-p: broken pipe\"": unknown.

If you get the above error , then run this command to update the docker version :  sudo apt-get install docker-ce=18.06.1~ce~3-0~ubuntu then run sudo docker run hello-world. So we are basically downgrading docker version to 18.06.1-ce

Verify if the image is created by running the following command

$ sudo docker images

Docker commands:

docker run [OPTIONS] IMAGE [COMMAND] [ARG...]

The docker run command first creates a writeable container layer over the specified image, and then starts it using the specified command. That is, docker run is equivalent to the API /containers/create then /containers/(id)/start. A stopped container can be restarted with all its previous changes intact using docker start. See docker ps -a to view a list of all containers.

Examples:

docker run image   [ This command pulls the image from docker hub repository if not present in local repository and runs the image ]

docker run --name <name> image [ This command assigns a name to the docker image and runs it ]

More examples on docker run command : 

docker run -i -t ubuntu /bin/bash

When you run this command, the following happens (assuming you are using the default registry configuration):

1. If you do not have the ubuntu image locally, Docker pulls it from your configured registry, as though you had run docker pull ubuntu manually.
2. Docker creates a new container, as though you had run a docker container create command manually.
3. Docker allocates a read-write filesystem to the container, as its final layer. This allows a running container to create or modify files and directories in its local filesystem.
4. Docker creates a network interface to connect the container to the default network, since you did not specify any networking options. This includes assigning an IP address to the container. By default, containers can connect to external networks using the host machine’s network connection.
5. Docker starts the container and executes /bin/bash. Because the container is running interactively and attached to your terminal (due to the -i and -t flags), you can provide input using your keyboard while the output is logged to your terminal.
6. When you type exit to terminate the /bin/bash command, the container stops but is not removed. You can start it again or remove it.

docker run nginx

This command pulls nginx image from docker registry and runs it. 

docker ps

This command lists all the running containers information.

docker ps -a

This command lists all the running and previously exited container information

docker stop [OPTIONS] CONTAINER [CONTAINERS...]

This will stop one or more running containers. The main process inside the container will receive SIGTERM, and after a grace period, SIGKILL. Provide CONAINER ID or NAME to the docker stop command . 

docker stop -t CONTAINER [or] docker stop --time CONTAINER

--time or -t option is used for Seconds to wait for stop before killing it 

docker stop dreamy_albattani

docker ps command shows that the container is not running and docker ps -a command shows that the container exited.

docker rm  [OPTIONS] CONTAINER [CONTAINER...]

This will remove stopped or exited container permanently.

Options:

--force or -f  : Force the removal of running container (uses SIGKILL)

--link or -l    : Remove  the specific link

--volumes or -v : Removes anonymous volumes associated with the container

EXAMPLES

Remove a container:

docker rm /redis

This removes the container referenced under the link /redis.

Remove a link specified with --link on the default bridge network:

docker rm --link /webapp/redis

This removes the underlying link between /webapp and the /redis containers on the default bridge network, removing all network communication between the two containers. This does not apply when --link is used with user-specified networks.

Force-remove a running container:

docker rm --force redis

This command force-removes a running container.

The main process inside the container referenced under the link redis will receive SIGKILL, then the container will be removed.

Remove all stopped containers:

docker rm $(docker ps -a -q)

This command deletes all stopped containers. The command docker ps -a -q above returns all existing container IDs and passes them to the rm command which deletes them. Running containers are not deleted.

Remove a container and its volumes:

docker rm -v redis

This command removes the container and any volumes associated with it. Note that if a volume was specified with a name, it will not be removed.

Remove a container and selectively remove volumes:

docker create -v awesome:/foo -v /bar --name hello redis

docker rm -v hello

In this example, the volume for /foo remains intact, but the volume for /bar is removed. The same behavior holds for volumes inherited with --volumes-from.

docker images [OPTIONS] [REPOSITORY[:TAG]]

This command will show all top level images, their repository and tags, and their size

Docker images have intermediate layers that increase reusability, decrease disk usage, and speed up docker build by allowing each step to be cached. These intermediate layers are not shown by default.

The SIZE is the cumulative space taken up by the image and all its parent images. This is also the disk space used by the contents of the Tar file created when you docker save an image.

An image will be listed more than once if it has multiple repository names or tags. This single image (identifiable by its matching IMAGE ID) uses up the SIZE listed only once.

OPTIONS

-all or -a            : Show all images (default hides intermediate images)

--digests            : Show digests

--filter or -f       : Filter output based on conditions provided

--format            : Pretty-print images using a Go template

--no-trunc         : Don't truncate output

--quiet or -q      : Only show image IDs

Examples

List the most recent images

docker images

List images by name and tag

docker images [REPOSITORY]

If you specify repository but no TAG , the docker images command lists all images in the given repository

docker images [REPOSITORY[:TAG]]

If both REPOSITORY and TAG are provided, only images matching that repository are listed. If nothing matches the list is empty.

List full length image IDs

docker images --no-trunc

This will list images with full length IDs and don't truncate the output.

List image digests

docker images --digests

Images that use the v2 or later format have a content-addressable identifier called a digest. As long as the input used to generate the image is unchanged, the digest value is predictable. To list image digest values, use the --digests flag

When pushing or pulling to a 2.0 registry, the push or pull command output includes the image digest. You can pull using a digest value. You can also reference by digest in create, run, and rmi commands, as well as the FROM image reference in a Dockerfile.

Filtering

The filtering flag (-f or --filter) format is of “key=value”. If there is more than one filter, then pass multiple flags (e.g., --filter "foo=bar" --filter "bif=baz")

The currently supported filters are:

dangling (boolean - true or false)

label (label=<key> or label=<key>=<value>)

before (<image-name>[:<tag>], <image id> or <image@digest>) - filter images created before given id or references

since (<image-name>[:<tag>], <image id> or <image@digest>) - filter images created since given id or references

reference (pattern of an image reference) - filter images whose reference matches the specified pattern

docker images --filter "dangling=true"

This will display untagged images that are the leaves of the images tree (not intermediary layers). These images occur when a new build of an image takes the repo:tag away from the image ID, leaving it as <none>:<none> or untagged. A warning will be issued if trying to remove an image when a container is presently using it. By having this flag it allows for batch cleanup.

You can use this in conjunction with docker rmi 

docker rmi $(docker images -f "dangling=true" -q)

Docker warns you if any containers exist that are using these untagged images.

Format the output

The formatting option (--format) will pretty print container output using a Go template

.ID                 Image ID

.Repository Image repository

.Tag                 Image tag

.Digest         Image digest

.CreatedSince Elapsed time since the image was created

.CreatedAt Time when the image was created

.Size         Image disk size

Examples

docker images --format "{{.ID}}: {{.Repository}}"

docker images --format "table {{.ID}}\t{{.Repository}}\t{{.Tag}}"

   #docker images 

    #docker images nginx

    #docker images --no-trunc

    #docker images --digests

docker rmi

docker rmi [OPTIONS] IMAGE [IMAGES...]

OPTIONS

--force or -f    : Force removal of the image

--no-prune      : Do not delete untagged parents

Removes (and un-tags) one or more images from the host node. If an image has multiple tags, using this command with the tag as a parameter only removes the tag. If the tag is the only one for the image, both the image and the tag are removed.

This does not remove images from a registry. You cannot remove an image of a running container unless you use the -f option. To see all images on a host use the docker image ls command.

Examples:

You can remove an image using its short or long ID, its tag, or its digest. If an image has one or more tags referencing it, you must remove all of them before the image is removed. Digest references are removed automatically when an image is removed by tag.

 root@vagrant-ubuntu-trusty-64:~# docker pull nginx:1

root@vagrant-ubuntu-trusty-64:~# docker images

root@vagrant-ubuntu-trusty-64:~# docker rmi f6d0b4767a6c

Error response from daemon: conflict: unable to delete f6d0b4767a6c (must be forced) - image is referenced in multiple repositories

This image can't be removed because it is tagged in multiple repositories. Use -f option to force. 

I have removed the tag by running the following command.

root@vagrant-ubuntu-trusty-64:~# docker rmi nginx:1

root@vagrant-ubuntu-trusty-64:~# docker images

root@vagrant-ubuntu-trusty-64:~# docker rmi f6d0b4767a6c

command has been executed successfully after removing the tag.

root@vagrant-ubuntu-trusty-64:~# docker rmi -f f6d0b4767a6c

If you use the -f flag and specify the image’s short or long ID, then this command untags and removes all images that match the specified ID.

docker pull

docker pull [OPTIONS] NAME[:TAG|@DIGEST]

OPTIONS

--all-tags or -a              : Download all tagged images in the repository

--disable-content-trust : Skip image verification

--platform                    : Set platform if server is multi-platform capable

--quiet or -q                 : Suppress verbose output

Docker Hub contains many pre-built images that can pull and try without needing to define and configure your own. 

If you are behind an HTTP proxy server, for example in corporate settings, before open a connect to registry, you may need to configure the Docker daemon’s proxy settings, using the HTTP_PROXY, HTTPS_PROXY, and NO_PROXY environment variables. To set these environment variables on a host using systemd, refer to the control and configure Docker with systemd for variables configuration.

By default the Docker daemon will pull three layers of an image at a time. If you are on a low bandwidth connection this may cause timeout issues and you may want to lower this via the --max-concurrent-downloads daemon option.

Examples:

Pull an image from Docker Hub

To download a particular image, or set of images (i.e., a repository), use docker pull. If no tag is provided, Docker Engine uses the :latest tag as a default.

root@vagrant-ubuntu-trusty-64:~# docker pull nginx

Docker images can consist of multiple layers. In the example above, the image consists of 5 layers. These layers can be reused by images. 

Layer 1: a076a628af6f

Layer 2: 0732ab25fa22

Layer 3: d7f36f6fe38f

Layer 4: f72584a26f32

Layer 5: 7125e4df9063

root@vagrant-ubuntu-trusty-64:~# docker pull nginx:1

Pulling nginx:1 image only pulls its metadata, but not its layers, because layers are already present locally.

Pull an image by digest (immutable identifier)

In some cases you don’t want images to be updated to newer versions, but prefer to use a fixed version of an image. Docker enables you to pull an image by its digest. When pulling an image by digest, you specify exactly which version of an image to pull. Doing so, allows you to “pin” an image to that version, and guarantee that the image you’re using is always the same.

root@vagrant-ubuntu-trusty-64:~# docker pull ubuntu:14.04

root@vagrant-ubuntu-trusty-64:~# docker pull ubuntu@sha256:63fce984528cec8714c365919882f8fb64c8a3edf23fdfa0b218a2756125456f

It says image is up to date as we have already pulled the same version earlier.

Pull from a different registry

By default, docker pull pulls images from Docker Hub. It is also possible to manually specify the path of a registry to pull from. For example, if you have set up a local registry, you can specify its path to pull from it. A registry path is similar to a URL, but does not contain a protocol specifier (https://).

docker pull myregistry.local:5000/testing/test-image

Pull a repository with multiple images

By default, docker pull pulls a single image from the registry. A repository can contain multiple images. To pull all images from a repository, provide the -a (or --all-tags) option when using docker pull.

root@vagrant-ubuntu-trusty-64:~# docker pull --all-tags fedora

This command pulls all images from the fedora repository:

docker exec

docker exec [OPTIONS] CONTAINER COMMAND [ARG...]

The docker exec command runs a new command in a running container.

The command started using docker exec only runs while the container’s primary process (PID 1) is running, and it is not restarted if the container is restarted.

COMMAND will run in the default directory of the container. If the underlying image has a custom directory specified with the WORKDIR directive in its Dockerfile, this will be used instead.

COMMAND should be an executable, a chained or a quoted command will not work. Example: docker exec -ti my_container "echo a && echo b" will not work, but docker exec -ti my_container sh -c "echo a && echo b" will.

OPTIONS:

--detach or -d        : Detached mode: run command in the background

--detach-keys         : Override the key sequence for detaching a container

--env or -e           : Set environment variables

--env-file            : Read in a file of environment

--interactive or -i   : Keep STDIN open even if not attached

--privileged          : Give extended privileges to the command

--tty or -t           : Allocate a pseudo-TTY

--user or -u          : username or UID

--workdir or -w       : Working directory inside the container

root@vagrant-ubuntu-trusty-64:~# docker run --name ubuntu_bash --rm -i -t ubuntu bash

This command created a new ubuntu bash session. This is highlighted in yellow. 

root@vagrant-ubuntu-trusty-64:~#  docker exec -d ubuntu_bash touch /tmp/execWorks

This will create a new file /tmp/execWorks inside the running container ubuntu_bash, in the background.

root@vagrant-ubuntu-trusty-64:~# docker exec -it -e VAR=1 ubuntu_bash bash

This will create a new Bash session in the container ubuntu_bash with environment variable $VAR set to “1”. Note that this environment variable will only be valid on the current Bash session.

root@vagrant-ubuntu-trusty-64:~# docker exec -it -w /root ubuntu_bash pwd

By default docker exec command runs in the same working directory set when container was created. You can select working directory for the command using -w option