Docker

Docker est un logiciel libre qui automatise le déploiement d'applications Linux dans des conteneurs logiciels. Il offre une solution légère pour la virtualisation de machines d'exécution Linux, en comparaison des machines virtuelles systèmes comme Xen, VMVare, Hyper-V.

https://www.docker.io/

Voir

• Panamax
• Vagrant
• Kubernetes
• Moby
• Bocker

Installation de Docker et Docker Compose

Sur MacOS

TODO

Sur Debian/Ubuntu

# Docker

sudo apt-get remove docker docker-engine docker.io
sudo apt-get update
sudo apt-get install -y \
    linux-image-extra-$(uname -r) \
    linux-image-extra-virtual

sudo apt-get install -y \
    apt-transport-https \
    ca-certificates \
    curl \
    software-properties-common

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

sudo apt-key fingerprint 0EBFCD88

sudo add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"

sudo apt-get update

sudo apt-get install -y docker-ce

sudo groupadd docker

sudo usermod -aG docker $USER

docker --version 

exit


# Docker Machine
curl -L https://github.com/docker/machine/releases/download/v0.12.0/docker-machine-`uname -s`-`uname -m` >/tmp/docker-machine && \
chmod +x /tmp/docker-machine && \\
sudo cp /tmp/docker-machine /usr/local/bin/docker-machine

docker-machine --version

curl -L https://github.com/docker/compose/releases/download/1.6.2/docker-compose-`uname -s`-`uname -m` > docker-compose-`uname -s`-`uname -m`
sudo cp docker-compose-`uname -s`-`uname -m` /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose

Getting started

                        ##         .
                  ## ## ##        ==
               ## ## ## ## ##    ===
           /"""""""""""""""""\___/ ===
      ~~~ {~~ ~~~~ ~~~ ~~~~ ~~~ ~ /  ===- ~~~
           \______ o           __/
             \    \         __/
              \____\_______/

> docker
Usage: docker [OPTIONS] COMMAND [arg...]
       docker [ --help | -v | --version ]

A self-sufficient runtime for containers.

Options:

  --config=~/.docker              Location of client config files
  -D, --debug                     Enable debug mode
  -H, --host=[]                   Daemon socket(s) to connect to
  -h, --help                      Print usage
  -l, --log-level=info            Set the logging level
  --tls                           Use TLS; implied by --tlsverify
  --tlscacert=~/.docker/ca.pem    Trust certs signed only by this CA
  --tlscert=~/.docker/cert.pem    Path to TLS certificate file
  --tlskey=~/.docker/key.pem      Path to TLS key file
  --tlsverify                     Use TLS and verify the remote
  -v, --version                   Print version information and quit

Commands:
    attach    Attach to a running container
    build     Build an image from a Dockerfile
    commit    Create a new image from a container's changes
    cp        Copy files/folders between a container and the local filesystem
    create    Create a new container
    diff      Inspect changes on a container's filesystem
    events    Get real time events from the server
    exec      Run a command in a running container
    export    Export a container's filesystem as a tar archive
    history   Show the history of an image
    images    List images
    import    Import the contents from a tarball to create a filesystem image
    info      Display system-wide information
    inspect   Return low-level information on a container or image
    kill      Kill a running container
    load      Load an image from a tar archive or STDIN
    login     Register or log in to a Docker registry
    logout    Log out from a Docker registry
    logs      Fetch the logs of a container
    network   Manage Docker networks
    pause     Pause all processes within a container
    port      List port mappings or a specific mapping for the CONTAINER
    ps        List containers
    pull      Pull an image or a repository from a registry
    push      Push an image or a repository to a registry
    rename    Rename a container
    restart   Restart a container
    rm        Remove one or more containers
    rmi       Remove one or more images
    run       Run a command in a new container
    save      Save an image(s) to a tar archive
    search    Search the Docker Hub for images
    start     Start one or more stopped containers
    stats     Display a live stream of container(s) resource usage statistics
    stop      Stop a running container
    tag       Tag an image into a repository
    top       Display the running processes of a container
    unpause   Unpause all processes within a container
    update    Update resources of one or more containers
    version   Show the Docker version information
    volume    Manage Docker volumes
    wait      Block until a container stops, then print its exit code

Run 'docker COMMAND --help' for more information on a command.

docker-compose --help
Define and run multi-container applications with Docker.

Usage:
  docker-compose [options] [COMMAND] [ARGS...]
  docker-compose -h|--help

Options:
  -f, --file FILE           Specify an alternate compose file (default: docker-compose.yml)
  -p, --project-name NAME   Specify an alternate project name (default: directory name)
  --verbose                 Show more output
  -v, --version             Print version and exit

Commands:
  build              Build or rebuild services
  help               Get help on a command
  kill               Kill containers
  logs               View output from containers
  port               Print the public port for a port binding
  ps                 List containers
  pull               Pulls service images
  restart            Restart services
  rm                 Remove stopped containers
  run                Run a one-off command
  scale              Set number of containers for a service
  start              Start services
  stop               Stop services
  up                 Create and start containers
  migrate-to-labels  Recreate containers to add labels

Usage: docker-machine [OPTIONS] COMMAND [arg...]

Create and manage machines running Docker.

Version: 0.6.0, build e27fb87

Author:
  Docker Machine Contributors - <https://github.com/docker/machine>

Options:
  --debug, -D						Enable debug mode
  -s, --storage-path "/Users/donsez/.docker/machine"	Configures storage path [$MACHINE_STORAGE_PATH]
  --tls-ca-cert 					CA to verify remotes against [$MACHINE_TLS_CA_CERT]
  --tls-ca-key 						Private key to generate certificates [$MACHINE_TLS_CA_KEY]
  --tls-client-cert 					Client cert to use for TLS [$MACHINE_TLS_CLIENT_CERT]
  --tls-client-key 					Private key used in client TLS auth [$MACHINE_TLS_CLIENT_KEY]
  --github-api-token 					Token to use for requests to the Github API [$MACHINE_GITHUB_API_TOKEN]
  --native-ssh						Use the native (Go-based) SSH implementation. [$MACHINE_NATIVE_SSH]
  --bugsnag-api-token 					BugSnag API token for crash reporting [$MACHINE_BUGSNAG_API_TOKEN]
  --help, -h						show help
  --version, -v						print the version
  
Commands:
  active		Print which machine is active
  config		Print the connection config for machine
  create		Create a machine
  env			Display the commands to set up the environment for the Docker client
  inspect		Inspect information about a machine
  ip			Get the IP address of a machine
  kill			Kill a machine
  ls			List machines
  provision		Re-provision existing machines
  regenerate-certs	Regenerate TLS Certificates for a machine
  restart		Restart a machine
  rm			Remove a machine
  ssh			Log into or run a command on a machine with SSH.
  scp			Copy files between machines
  start			Start a machine
  status		Get the status of a machine
  stop			Stop a machine
  upgrade		Upgrade a machine to the latest version of Docker
  url			Get the URL of a machine
  version		Show the Docker Machine version or a machine docker version
  help			Shows a list of commands or help for one command
  
Run 'docker-machine COMMAND --help' for more information on a command.

Déploiement d'un conteneur

docker pull mesoscloud/zookeeper

docker pull ansi/mosquitto

Run 2 containers (-d for detached, -p for NAT ports)

export CID=my-mosquitto
docker run --name $CID -d ansi/mosquitto
docker run --name my-second-mosquitto -p 1883:1887 -d ansi/mosquitto

Translation de ports série

docker run --device /dev/ttyAMA0:/dev/ttyAMA0 --device /dev/mem:/dev/mem --privileged -ti acencini/rpi-python-serial-wiringpi /bin/bash

Pass env vars

docker run --name my-mysql -e MYSQL_ROOT_PASSWORD=root+1 -d mysql

List containers

docker ps -a

List images

docker images

List dangling images

docker images -q -f dangling=true

Exec commands

docker exec $CID ps
docker exec $CID ls /

Top command into the container

docker top $CID

Log as root

docker exec -it $CID bash

List ports

docker port $CID

Stop container

docker stop $CID

Remove container

docker rm $CID

Remove image

docker rmi hello-world

Remove dangling images

docker rmi $(docker images -q -f dangling=true)

Remarque : Docker can't connect to docker daemon

Définition d'un Dockerfile

• Dockerfile Reference https://docs.docker.com/engine/reference/builder/
• Best Practices https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/

Voir https://nodejs.org/en/docs/guides/nodejs-docker-webapp/

TODO bridge Node.js MQTT --> InfluxDB

docker login
docker push donsez/mqtt2influxdb:0.1

Docker Compose

Déploiement d'une composition

Créer le descripteur suivant pour cet stack IoT : docker-compose.yml

mongodb:
  build: ./build/mongodb
  volumes:
    - /var/lib/docker/mongo/mongodb:/data/db
  command: mongod --smallfiles
  ports:
    - "27017:27017"

mosquitto:
  image: ansi/mosquitto
  ports:
    - "1883:1883"

nodered:
  image: cpswan/node-red
  volumes:
    - /var/lib/docker/node-red/:/root/.node-red/
  ports:
    - "1880:1880"
  links:
    - mongodb
    - mosquitto

Exercice: Ajouter InfluxDB et Grafana à la composition

Exécuter

docker-compose build
docker-compose up

Depuis un autre terminal

docker-compose ps
docker-compose logs
^C
docker-compose stop
docker-compose ps
docker-compose logs
^C
docker-compose start
docker-compose ps
docker-compose logs
^C

Test de la composition

Depuis un autre terminal de la machine qui exécute le conteneur Docker (ou depuis une autre machine)

docker ps -a
docker images
python -m webbrowser -t "http://localhost:1880"

Ajouter le flow suivant via Menu > Import > Clipboard

[{"id":"6f440ee6.a1716","type":"mqtt-broker","z":"8072fbb2.b48e6","broker":"test.mosquitto.org","port":"1883","clientid":"","usetls":false,"verifyservercert":true,"compatmode":true,"keepalive":"60","cleansession":true,"willTopic":"","willQos":"0","willRetain":null,"willPayload":"","birthTopic":"","birthQos":"0","birthRetain":null,"birthPayload":""},{"id":"96a17df1.8ff65","type":"mqtt in","z":"8072fbb2.b48e6","name":"IoT","topic":"fr/imag/air/iot","broker":"6f440ee6.a1716","x":450,"y":240,"wires":[["7947a469.896f8c","f8335f1d.a01108"]]},{"id":"7947a469.896f8c","type":"file","z":"8072fbb2.b48e6","name":"","filename":"sensor.txt","appendNewline":true,"createDir":false,"overwriteFile":"false","x":850,"y":240,"wires":[]},{"id":"f8335f1d.a01108","type":"debug","z":"8072fbb2.b48e6","name":"","active":true,"console":"false","complete":"false","x":850,"y":320,"wires":[]}]

Depuis une autre machine:

mosquitto_pub -h test.mosquitto.org -t fr/imag/air/iot -m "sensor deveui=1234567 temp=20.10;hum=56.0;pres=1013.25;wind=5.0;rainlast10min=5;rainlasthour=10"

Vérifier la récupération du message dans l'onglet Debug de Node-RED.

Affichage des logs des containers

docker-compose logs

Arrêt d'une composition

docker-compose stop

Docker Machine

https://docs.docker.com/machine/get-started-cloud/

En local avec VirtualBox

docker-machine create -d virtualbox qa

docker-machine ls

docker-machine start qa
docker-machine ls

docker-machine env qa

docker-machine stop qa

Sur un cloud public

export AWS_ACCESS_KEY=AKI*******
export AWS_SECRET_KEY=8T93C*******
docker-machine create --driver amazonec2 --amazonec2-access-key $AWS_ACCESS_KEY --amazonec2-secret-key $AWS_SECRET_KEY aws-sandbox

Sur un cluster

docker-machine create --url=tcp://50.134.234.20:2376 custombox

Docker Swarm

https://docs.docker.com/swarm/overview/ Docker Swarm is native clustering for Docker. It turns a pool of Docker hosts into a single, virtual Docker host.

Installation

docker run swarm:latest

docker-machine create \
    -d virtualbox \
    --swarm \
    --swarm-master \
    --swarm-discovery token://SWARM_CLUSTER_TOKEN \
    swarm-master

docker-machine create \
    -d virtualbox \
    --swarm \
    --swarm-discovery token://SWARM_CLUSTER_TOKEN \
    HOST_NODE_NAME

Découverte

https://docs.docker.com/swarm/discovery/

La découverte utilise au choix Consul, Zookeeper ou Etcd.

Haute-disponibilité

https://docs.docker.com/swarm/multi-manager-setup/

GPU CUDA

NVidia Docker

Krane

https://github.com/krane-io/krane Krane is an open-source platform that enables developers and sysadmins to manage Docker instances across multiple clouds. Krane is built on the Docker code base, so supports the existing Docker command-line in a multi-cloud environment. This allows Docker users to use their existing workflows to launch apps transparently in multiple clouds, saving time and further reducing the friction of moving workloads between development, QA and production.

Container-as-a-Service : Docker Hosting

See https://dzone.com/articles/the-shortlist-of-docker-hosting

• Docker Machine
• Digital Ocean Droplet
• Rackspace Carina http://getcarina.com
• Giant Swarm
• Google Compute Engine (GCE)
• Microsoft Azure
• Heroku
• Quay https://quay.io/
• Sloppy.io
• AWS EC2 container service (ECS)
• Deis Workflow
• Docker Cloud
• ContainerShip

CloudFoundry

TBD

https://docs.cloudfoundry.org/concepts/docker.html

Gradle Docker Plugin

See

• https://dzone.com/articles/docker-containers-with-gradle-application-plugin
• https://github.com/Transmode/gradle-docker

Monitoring

Lire https://dzone.com/refcardz/intro-to-docker-monitoring

Les commandes sont ci-dessous.

Docker Stat API

CONTAINER_NAME=mosquitto
curl --unix-socket /var/run/docker.sock http:/containers/${CONTAINER_NAME}/stats?stream=false
curl --unix-socket /var/run/docker.sock http:/containers/${CONTAINER_NAME}/stats

cAdvisor

sudo docker run \
 --volume=/:/rootfs:ro \
 --volume=/var/run:/var/run:rw \
 --volume=/sys:/sys:ro \
 --volume=/var/lib/docker/:/var/lib/docker:ro \
 --publish=8080:8080 \
 --detach=true \
 --name=cadvisor \
 google/cadvisor:latest

Browse http://localhost:8080

Ajouter des plugins à cAdvisor [1] et [2]

Sysdig

Installation

sudo apt install -y sysdig

CONTAINER_NAME=mosquitto
sudo sysdig -c topcontainers_cpu
sudo sysdig -pc -c topprocs_cpu
sudo sysdig -pc -c topprocs_cpu container.name=$CONTAINER_NAME
sudo sysdig -pc -c topprocs_cpu container.name contains mosqui
sudo sysdig -pc -c topprocs_net
sudo sysdig -pc -c topconns
sudo sysdig -pc -c topfiles_bytes

Depuis un terminal

CONTAINER_NAME=sshd
sudo sysdig -pc -c spy_users container.name=$CONTAINER_NAME

Depuis un autre terminal

ssh -i ubuntu.pem ubuntu@localhost -p 2022

sudo csysdig

Appuyez sur F2 et sélectionnez Containers avec les flèches haut et bas. Sélectionnez le container et appuyez sur Enter. Appuyez sur p pour faire une pause et pour reprendre le monitoring.

Autres exemples : https://www.sysdig.org/wiki/sysdig-examples/#containers

Voir Dagda : a tool to perform static analysis of known vulnerabilities in docker images/containers and to monitor running docker containers for detecting anomalous activities.

• https://github.com/eliasgranderubio/dagda
• http://www.sysdig.org/falco/

Livres

• Docker in Action, http://www.manning.com/nickoloff/DockerinAction_MEAP_ch01.pdf

Articles

Performances

• http://stackoverflow.com/questions/21889053/what-is-the-runtime-performance-cost-of-a-docker-container
• http://domino.research.ibm.com/library/cyberdig.nsf/papers/0929052195DD819C85257D2300681E7B/$File/rc25482.pdf

Docker @ AIR

• IaaS collaboratif avec Docker (RICM5 en 2015-2016)
• PM2M (M2PGI en 2015-2016)
• Projet eCom (RICM5)