Docker

Docker est un logiciel libre qui automatise le déploiement d'applications Linux dans des conteneurs logiciels. Il offre une solution légère pour la virtualisation de machines d'exécution Linux, en comparaison des machines virtuelles systèmes comme Xen, VMVare, Hyper-V.

https://www.docker.io/

Voir
 * Panamax
 * Vagrant
 * Kubernetes
 * Moby
 * Bocker

=Installation de Docker et Docker Compose=

Sur MacOS
TODO

Sur Debian/Ubuntu

 * 1) Docker

sudo apt-get remove docker docker-engine docker.io sudo apt-get update sudo apt-get install -y \ linux-image-extra-$(uname -r) \ linux-image-extra-virtual

sudo apt-get install -y \ apt-transport-https \ ca-certificates \ curl \ software-properties-common

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

sudo apt-key fingerprint 0EBFCD88

sudo add-apt-repository \ "deb [arch=amd64] https://download.docker.com/linux/ubuntu \  $(lsb_release -cs) \   stable"

sudo apt-get update

sudo apt-get install -y docker-ce

sudo groupadd docker

sudo usermod -aG docker $USER

docker --version

exit

curl -L https://github.com/docker/machine/releases/download/v0.12.0/docker-machine-`uname -s`-`uname -m` >/tmp/docker-machine && \ chmod +x /tmp/docker-machine && \\ sudo cp /tmp/docker-machine /usr/local/bin/docker-machine
 * 1) Docker Machine

docker-machine --version

curl -L https://github.com/docker/compose/releases/download/1.6.2/docker-compose-`uname -s`-`uname -m` > docker-compose-`uname -s`-`uname -m` sudo cp docker-compose-`uname -s`-`uname -m` /usr/local/bin/docker-compose sudo chmod +x /usr/local/bin/docker-compose

=Getting started=

##        .                  ## ## ##        ==               ## ## ## ## ##    ===           /"""""""""""""""""\___/ ===       { ~  ~  ~ /  ===-            \______ o           __/             \    \         __/              \____\_______/

> docker Usage: docker [OPTIONS] COMMAND [arg...] docker [ --help | -v | --version ]

A self-sufficient runtime for containers.

Options:

--config=~/.docker             Location of client config files -D, --debug                    Enable debug mode -H, --host=[]                  Daemon socket(s) to connect to  -h, --help                      Print usage -l, --log-level=info           Set the logging level --tls                          Use TLS; implied by --tlsverify --tlscacert=~/.docker/ca.pem   Trust certs signed only by this CA  --tlscert=~/.docker/cert.pem    Path to TLS certificate file --tlskey=~/.docker/key.pem     Path to TLS key file --tlsverify                    Use TLS and verify the remote -v, --version                  Print version information and quit

Commands: attach   Attach to a running container build    Build an image from a Dockerfile commit   Create a new image from a container's changes cp       Copy files/folders between a container and the local filesystem create   Create a new container diff     Inspect changes on a container's filesystem events   Get real time events from the server exec     Run a command in a running container export   Export a container's filesystem as a tar archive history  Show the history of an image images   List images import   Import the contents from a tarball to create a filesystem image info     Display system-wide information inspect  Return low-level information on a container or image kill     Kill a running container load     Load an image from a tar archive or STDIN login    Register or log in to a Docker registry logout   Log out from a Docker registry logs     Fetch the logs of a container network  Manage Docker networks pause    Pause all processes within a container port     List port mappings or a specific mapping for the CONTAINER ps       List containers pull     Pull an image or a repository from a registry push     Push an image or a repository to a registry rename   Rename a container restart  Restart a container rm       Remove one or more containers rmi      Remove one or more images run      Run a command in a new container save     Save an image(s) to a tar archive search   Search the Docker Hub for images start    Start one or more stopped containers stats    Display a live stream of container(s) resource usage statistics stop     Stop a running container tag      Tag an image into a repository top      Display the running processes of a container unpause  Unpause all processes within a container update   Update resources of one or more containers version  Show the Docker version information volume   Manage Docker volumes wait     Block until a container stops, then print its exit code

Run 'docker COMMAND --help' for more information on a command.

docker-compose --help Define and run multi-container applications with Docker.

Usage: docker-compose [options] [COMMAND] [ARGS...] docker-compose -h|--help

Options: -f, --file FILE          Specify an alternate compose file (default: docker-compose.yml) -p, --project-name NAME  Specify an alternate project name (default: directory name) --verbose                Show more output -v, --version            Print version and exit

Commands: build             Build or rebuild services help              Get help on a command kill              Kill containers logs              View output from containers port              Print the public port for a port binding ps                List containers pull              Pulls service images restart           Restart services rm                Remove stopped containers run               Run a one-off command scale             Set number of containers for a service start             Start services stop              Stop services up                Create and start containers migrate-to-labels Recreate containers to add labels

Usage: docker-machine [OPTIONS] COMMAND [arg...]

Create and manage machines running Docker.

Version: 0.6.0, build e27fb87

Author: Docker Machine Contributors - 

Options: --debug, -D						Enable debug mode -s, --storage-path "/Users/donsez/.docker/machine"	Configures storage path [$MACHINE_STORAGE_PATH] --tls-ca-cert 					CA to verify remotes against [$MACHINE_TLS_CA_CERT] --tls-ca-key 						Private key to generate certificates [$MACHINE_TLS_CA_KEY] --tls-client-cert 					Client cert to use for TLS [$MACHINE_TLS_CLIENT_CERT] --tls-client-key 					Private key used in client TLS auth [$MACHINE_TLS_CLIENT_KEY] --github-api-token 					Token to use for requests to the Github API [$MACHINE_GITHUB_API_TOKEN] --native-ssh						Use the native (Go-based) SSH implementation. [$MACHINE_NATIVE_SSH] --bugsnag-api-token 					BugSnag API token for crash reporting [$MACHINE_BUGSNAG_API_TOKEN] --help, -h						show help --version, -v						print the version Commands: active		Print which machine is active config		Print the connection config for machine create		Create a machine env			Display the commands to set up the environment for the Docker client inspect		Inspect information about a machine ip			Get the IP address of a machine kill			Kill a machine ls			List machines provision		Re-provision existing machines regenerate-certs	Regenerate TLS Certificates for a machine restart		Restart a machine rm			Remove a machine ssh			Log into or run a command on a machine with SSH. scp			Copy files between machines start			Start a machine status		Get the status of a machine stop			Stop a machine upgrade		Upgrade a machine to the latest version of Docker url			Get the URL of a machine version		Show the Docker Machine version or a machine docker version help			Shows a list of commands or help for one command Run 'docker-machine COMMAND --help' for more information on a command.

=Déploiement d'un conteneur= docker pull mesoscloud/zookeeper

docker pull ansi/mosquitto

Run 2 containers (-d for detached, -p for NAT ports) export CID=my-mosquitto docker run --name $CID -d ansi/mosquitto docker run --name my-second-mosquitto -p 1883:1887 -d ansi/mosquitto

Translation de ports série docker run --device /dev/ttyAMA0:/dev/ttyAMA0 --device /dev/mem:/dev/mem --privileged -ti acencini/rpi-python-serial-wiringpi /bin/bash

Pass env vars docker run --name my-mysql -e MYSQL_ROOT_PASSWORD=root+1 -d mysql

List containers docker ps -a

List images docker images

List dangling images docker images -q -f dangling=true

Exec commands docker exec $CID ps docker exec $CID ls /

Top command into the container docker top $CID

Log as root docker exec -it $CID bash

List ports docker port $CID

Stop container docker stop $CID

Remove container docker rm $CID

Remove image docker rmi hello-world

Remove dangling images docker rmi $(docker images -q -f dangling=true)

Remarque : Docker can't connect to docker daemon

=Définition d'un Dockerfile=


 * Dockerfile Reference https://docs.docker.com/engine/reference/builder/
 * Best Practices https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/

Voir https://nodejs.org/en/docs/guides/nodejs-docker-webapp/

TODO bridge Node.js MQTT --> InfluxDB

docker login docker push donsez/mqtt2influxdb:0.1

=Docker Compose=

Déploiement d'une composition
Créer le descripteur suivant pour cet stack IoT : docker-compose.yml mongodb: build: ./build/mongodb volumes: - /var/lib/docker/mongo/mongodb:/data/db command: mongod --smallfiles ports: - "27017:27017"

mosquitto: image: ansi/mosquitto ports: - "1883:1883"

nodered: image: cpswan/node-red volumes: - /var/lib/docker/node-red/:/root/.node-red/ ports: - "1880:1880" links: - mongodb - mosquitto

Exercice: Ajouter InfluxDB et Grafana à la composition

Exécuter docker-compose build docker-compose up

Depuis un autre terminal docker-compose ps docker-compose logs ^C docker-compose stop docker-compose ps docker-compose logs ^C docker-compose start docker-compose ps docker-compose logs ^C

Test de la composition
Depuis un autre terminal de la machine qui exécute le conteneur Docker (ou depuis une autre machine) docker ps -a docker images python -m webbrowser -t "http://localhost:1880"

Ajouter le flow suivant via Menu > Import > Clipboard [{"id":"6f440ee6.a1716","type":"mqtt-broker","z":"8072fbb2.b48e6","broker":"test.mosquitto.org","port":"1883","clientid":"","usetls":false,"verifyservercert":true,"compatmode":true,"keepalive":"60","cleansession":true,"willTopic":"","willQos":"0","willRetain":null,"willPayload":"","birthTopic":"","birthQos":"0","birthRetain":null,"birthPayload":""},{"id":"96a17df1.8ff65","type":"mqtt in","z":"8072fbb2.b48e6","name":"IoT","topic":"fr/imag/air/iot","broker":"6f440ee6.a1716","x":450,"y":240,"wires":"7947a469.896f8c","f8335f1d.a01108"},{"id":"7947a469.896f8c","type":"file","z":"8072fbb2.b48e6","name":"","filename":"sensor.txt","appendNewline":true,"createDir":false,"overwriteFile":"false","x":850,"y":240,"wires":[]},{"id":"f8335f1d.a01108","type":"debug","z":"8072fbb2.b48e6","name":"","active":true,"console":"false","complete":"false","x":850,"y":320,"wires":[]}]

Depuis une autre machine: mosquitto_pub -h test.mosquitto.org -t fr/imag/air/iot -m "sensor deveui=1234567 temp=20.10;hum=56.0;pres=1013.25;wind=5.0;rainlast10min=5;rainlasthour=10"

Vérifier la récupération du message dans l'onglet Debug de Node-RED.

Affichage des logs des containers
docker-compose logs

Arrêt d'une composition
docker-compose stop

=Docker Machine= https://docs.docker.com/machine/get-started-cloud/

En local avec VirtualBox
docker-machine create -d virtualbox qa

docker-machine ls

docker-machine start qa docker-machine ls

docker-machine env qa

docker-machine stop qa

Sur un cloud public
export AWS_ACCESS_KEY=AKI******* export AWS_SECRET_KEY=8T93C******* docker-machine create --driver amazonec2 --amazonec2-access-key $AWS_ACCESS_KEY --amazonec2-secret-key $AWS_SECRET_KEY aws-sandbox

Sur un cluster
docker-machine create --url=tcp://50.134.234.20:2376 custombox

Sur Docker Swram
=Docker Swarm= https://docs.docker.com/swarm/overview/ ''Docker Swarm is native clustering for Docker. It turns a pool of Docker hosts into a single, virtual Docker host.''

Installation
docker run swarm:latest

docker-machine create \ -d virtualbox \ --swarm \ --swarm-master \ --swarm-discovery token://SWARM_CLUSTER_TOKEN \ swarm-master

docker-machine create \ -d virtualbox \ --swarm \ --swarm-discovery token://SWARM_CLUSTER_TOKEN \ HOST_NODE_NAME

Découverte
https://docs.docker.com/swarm/discovery/

La découverte utilise au choix Consul, Zookeeper ou Etcd.

Haute-disponibilité
https://docs.docker.com/swarm/multi-manager-setup/

GPU CUDA
NVidia Docker

=Krane= https://github.com/krane-io/krane ''Krane is an open-source platform that enables developers and sysadmins to manage Docker instances across multiple clouds. Krane is built on the Docker code base, so supports the existing Docker command-line in a multi-cloud environment. This allows Docker users to use their existing workflows to launch apps transparently in multiple clouds, saving time and further reducing the friction of moving workloads between development, QA and production.''

=Container-as-a-Service : Docker Hosting= See https://dzone.com/articles/the-shortlist-of-docker-hosting


 * Docker Machine
 * Digital Ocean Droplet
 * Rackspace Carina http://getcarina.com
 * Giant Swarm
 * Google Compute Engine (GCE)
 * Microsoft Azure
 * Heroku
 * Quay https://quay.io/
 * Sloppy.io
 * AWS EC2 container service (ECS)
 * Deis Workflow
 * Docker Cloud
 * ContainerShip

=CloudFoundry= TBD

https://docs.cloudfoundry.org/concepts/docker.html

=Gradle Docker Plugin= See
 * https://dzone.com/articles/docker-containers-with-gradle-application-plugin
 * https://github.com/Transmode/gradle-docker

=Monitoring=

Lire https://dzone.com/refcardz/intro-to-docker-monitoring

Les commandes sont ci-dessous.

Docker Stat API
CONTAINER_NAME=mosquitto curl --unix-socket /var/run/docker.sock http:/containers/${CONTAINER_NAME}/stats?stream=false curl --unix-socket /var/run/docker.sock http:/containers/${CONTAINER_NAME}/stats

cAdvisor
sudo docker run \ --volume=/:/rootfs:ro \ --volume=/var/run:/var/run:rw \ --volume=/sys:/sys:ro \ --volume=/var/lib/docker/:/var/lib/docker:ro \ --publish=8080:8080 \ --detach=true \ --name=cadvisor \ google/cadvisor:latest

Browse http://localhost:8080

Ajouter des plugins à cAdvisor |lien et |lien

Sysdig
Installation sudo apt install -y sysdig

CONTAINER_NAME=mosquitto sudo sysdig -c topcontainers_cpu sudo sysdig -pc -c topprocs_cpu sudo sysdig -pc -c topprocs_cpu container.name=$CONTAINER_NAME sudo sysdig -pc -c topprocs_cpu container.name contains mosqui sudo sysdig -pc -c topprocs_net sudo sysdig -pc -c topconns sudo sysdig -pc -c topfiles_bytes

Depuis un terminal CONTAINER_NAME=sshd sudo sysdig -pc -c spy_users container.name=$CONTAINER_NAME

Depuis un autre terminal ssh -i ubuntu.pem ubuntu@localhost -p 2022

sudo csysdig

Appuyez sur F2 et sélectionnez Containers avec les flèches haut et bas. Sélectionnez le container et appuyez sur Enter. Appuyez sur p pour faire une pause et pour reprendre le monitoring.

Autres exemples : https://www.sysdig.org/wiki/sysdig-examples/#containers

Voir Dagda : a tool to perform static analysis of known vulnerabilities in docker images/containers and to monitor running docker containers for detecting anomalous activities.
 * https://github.com/eliasgranderubio/dagda
 * http://www.sysdig.org/falco/

=Livres=
 * Docker in Action, http://www.manning.com/nickoloff/DockerinAction_MEAP_ch01.pdf

=Articles=

Performances

 * http://stackoverflow.com/questions/21889053/what-is-the-runtime-performance-cost-of-a-docker-container
 * http://domino.research.ibm.com/library/cyberdig.nsf/papers/0929052195DD819C85257D2300681E7B/$File/rc25482.pdf

=Docker @ AIR=
 * IaaS collaboratif avec Docker (RICM5 en 2015-2016)
 * PM2M (M2PGI en 2015-2016)
 * Projet eCom (RICM5)