Monthly Archives: June 2020

Docker

Install docker on mac:

  1. brew install docker
  2. Download docker.dmg and install

Run jenkins in background
docker run -d -p 8080:8080 -p 50000:50000 jenkins

-d makes it to run background. Just like nohup

Run centos in docker
docker run -it centos /bin/bash

-it makes it run interactive mode,

Enter docker container.

For example if container id 86289a1dad7c is centOS, this will make it into centos command line:

docker attach 86289a1dad7c

Create overlay network

docker network create -d overlay –attachable kafka-net

Install Kafka by docker:

  1. create a zk(zookeeper)
    docker service create –network kafka-net –name=zookeeper –publish 2181:2181 qnib/plain-zookeeper:2018-04-25
  2. create a zk UI
    docker service create –network kafka-net –name=zkui –publish 9090:9090 qnib/plain-zkui@sha256:30c4aa1236ee90e4274a9059a5fa87de2ee778d9bfa3cb48c4c9aafe7cfa1a13
  3. Create a broker
    docker service create –network kafka-net –name broker –hostname=”{{.Service.Name}}.{{.Task.Slot}}.{{.Task.ID}}” -e KAFKA_BROKER_ID={{.Task.Slot}} -e ZK_SERVERS=tasks.zookeeper qnib/plain-kafka –publish 9092:9092
  4. Scale up brokers
    docker service update –replicas=3 broker
  5. Create 3 brokers, with port mapping
    docker service create –network kafka-net –name broker –publish 9092:9092 –replicas 3 –hostname=”{{.Service.Name}}.{{.Task.Slot}}.{{.Task.ID}}” -e KAFKA_BROKER_ID={{.Task.Slot}} -e ZK_SERVERS=tasks.zookeeper qnib/plain-kafka

Confluent connect to Kafka
kafka-console-producer -broker-list broker_ip:port –topic test
kafka-console-consumer -bootstrap-server broker_ip:port –topic test

Docker network

Bridge, normal one. It has a gateway, each container has its own IP.
host, each docker host can have only 1 container.
none, an isolation network.

docker_network

Overlay

The swarm has to live on overlay network. Such as this command “docker service create –network overlay_network –name=centos –replicas 2 centos”

Each container has same port number, so overlay network has an internal load balance to forward traffic.

docker_overlay

 

https://success.docker.com/article/getting-started-with-kafka