Shan

Docker跨主机网络[1]-Overlay(Etcd)
一、环境说明 虚拟环境:VMware Workstation 14 Pro 系统环境:CentOS7.5 主机地址...
扫描右侧二维码阅读全文
30
2018/10

Docker跨主机网络[1]-Overlay(Etcd)

一、环境说明

虚拟环境:VMware Workstation 14 Pro
系统环境:CentOS7.5
主机地址:host1 192.168.5.129
host2 192.168.5.130

此次部署使用Docker Machine实现多主机管理创建的环境。

二、overlay概述

为支持容器跨主机通信,Docker提供了overlay driver,使用户可以创建基于VxLAN的overlay网络。VxLAN可将二层数据封装到UDP进行传输,VxLAN提供与VLAN相同的以太网二层服务,但是拥有更强的扩展性和灵活性。overlay网络用于连接不同宿主机上的docker容器,允许不同宿主机上的容器相互通信,同时支持对消息进行加密。
由于overlay网络需要一个key-value数据库来保存网络状态信息,所以此处选择使用etcd来实现状态信息的存储。

三、配置overlay网络

1、安装etcd

下载并解压etcd
# wget https://github.com/etcd-io/etcd/releases/download/v3.3.12/etcd-v3.3.12-linux-amd64.tar.gz
# tar zxvf etcd-v3.3.12-linux-amd64.tar.gz
主要文件有:Documentation(文档目录)  etcd(服务端二进制文件)  etcdctl(客户端二进制文件)

复制二进制文件到系统PATH路径下
# cp etcd-v3.3.12-linux-amd64/etcd* /usr/bin

2、配置etcd

host1:
nohup etcd --name docker-host1 --initial-advertise-peer-urls http://192.168.5.129:2380 \
--listen-peer-urls http://192.168.5.129:2380 \
--listen-client-urls http://192.168.5.129:2379,http://127.0.0.1:2379 \
--advertise-client-urls http://192.168.5.129:2379 \
--initial-cluster-token etcd-cluster \
--initial-cluster docker-host1=http://192.168.5.129:2380,docker-host2=http://192.168.5.130:2380 \
--data-dir ./etcd_data \
--initial-cluster-state new &

host2:
nohup etcd --name docker-host2 --initial-advertise-peer-urls http://192.168.5.130:2380 \
--listen-peer-urls http://192.168.5.130:2380 \
--listen-client-urls http://192.168.5.130:2379,http://127.0.0.1:2379 \
--advertise-client-urls http://192.168.5.130:2379 \
--initial-cluster-token etcd-cluster \
--initial-cluster docker-host1=http://192.168.5.129:2380,docker-host2=http://192.168.5.130:2380 \
--data-dir ./etcd_data \
--initial-cluster-state new &


选项解析:

  • --initial-advertise-peer-urls
    表示节点监听其他节点同步信号的地址,默认端口:2380,7001(已废弃)
  • --listen-peer-urls
    节点与节点之间数据交换, 需要集群内的其他机器访问本机,因此监听在本机的IP地址上,默认端口:2380,7001(已废弃)
  • --listen-client-urls
    对外提供服务的地址,客户端会连接到这里与etcd交互,默认端口为: 2379 & 4001(已废弃)
  • --advertise-client-urls
    集群内同伴需要监听的地址,会告诉集群中其他节点自己的地址,默认端口:2380,7001(已废弃)
  • --initial-cluster-token
    创建集群的token,每个集群都不一样
  • --initial-cluster
    集群内所有节点的信息
  • --data-dir
    指定存放数据的目录
  • --initial-cluster-state
    新建集群的话,这个值为new;已存在集群,这个值为existing

3、查看集群健康状态

# etcdctl cluster-health
member a90e830c04d4df5f is healthy: got healthy result from http://192.168.5.129:2379
member addd922174385726 is healthy: got healthy result from http://192.168.5.130:2379
cluster is healthy

4、编辑docker-daemon的配置文件

两台主机都需要修改
# vim /etc/systemd/system/docker.service.d/10-machine.conf
在ExecStart处添加下面内容:
--cluster-store=etcd://(任意一个主机的IP):2379 --cluster-advertise=(本机IP):2375

注:由于etcd组成一个集群,所以在cluster-store处使用任意集群内主机都可以访问到数据。此处使用本机地址。

编辑完成后如下图所示:
overlay-1.png
重载并重启Docker Daemon:

# systemctl daemon-reload;systemctl restart docker

四、创建overlay网络并测试

1、创建overlay网络

在host1上:

# docker network create -d overlay test
dbb4831869b0cb93fc9bdca81b0886fa1726176cc08889b53dc51729b7d89457

2、查看当前网络

在host2上:

# docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
53f60200da45        bridge              bridge              local
8e9357b29363        host                host                local
13b2c1536acc        none                null                local
dbb4831869b0        test                overlay             global

我们能看到,test网络自动添加到了host2上面。这就是etcd的功劳了,可以通过etcdctl查看存储的数据信息

[root@host2 ~]# etcdctl ls /docker
/docker/network
/docker/nodes
[root@host2 ~]# etcdctl ls /docker/nodes
/docker/nodes/192.168.5.129:2375
/docker/nodes/192.168.5.130:2375
[root@host2 ~]# etcdctl ls /docker/network
/docker/network/v1.0
[root@host2 ~]# etcdctl ls /docker/network/v1.0
/docker/network/v1.0/network
/docker/network/v1.0/endpoint_count
/docker/network/v1.0/endpoint
/docker/network/v1.0/ipam
/docker/network/v1.0/idm
/docker/network/v1.0/overlay
[root@host2 ~]# etcdctl ls /docker/network/v1.0/network
/docker/network/v1.0/network/dbb4831869b0cb93fc9bdca81b0886fa1726176cc08889b53dc51729b7d89457
[root@host2 ~]# etcdctl get /docker/network/v1.0/network/dbb4831869b0cb93fc9bdca81b0886fa1726176cc08889b53dc51729b7d89457
{"addrSpace":"GlobalDefault","attachable":false,"configFrom":"","configOnly":false,"created":"2019-02-12T15:45:23.186896611+08:00","enableIPv6":false,"generic":{"com.docker.network.enable_ipv6":false,"com.docker.network.generic":{}},"id":"dbb4831869b0cb93fc9bdca81b0886fa1726176cc08889b53dc51729b7d89457","inDelete":false,"ingress":false,"internal":false,"ipamOptions":{},"ipamType":"default","ipamV4Config":"[{\"PreferredPool\":\"\",\"SubPool\":\"\",\"Gateway\":\"\",\"AuxAddresses\":null}]","ipamV4Info":"[{\"IPAMData\":\"{\\\"AddressSpace\\\":\\\"GlobalDefault\\\",\\\"Gateway\\\":\\\"10.0.0.1/24\\\",\\\"Pool\\\":\\\"10.0.0.0/24\\\"}\",\"PoolID\":\"GlobalDefault/10.0.0.0/24\"}]","labels":{},"loadBalancerIP":"","loadBalancerMode":"NAT","name":"test","networkType":"overlay","persist":true,"postIPv6":false,"scope":"global"}

3、启动容器测试连通性

在host1上:

创建容器
# docker run -d --name test_host1 --net test busybox sh -c "while true;do sleep 3600;done"

查看网络信息
# docker exec test_host1 ifconfig
eth0      Link encap:Ethernet  HWaddr 02:42:0A:00:00:02  
          inet addr:10.0.0.2  Bcast:10.0.0.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1450  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

eth1      Link encap:Ethernet  HWaddr 02:42:AC:12:00:02  
          inet addr:172.18.0.2  Bcast:172.18.255.255  Mask:255.255.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:8 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:656 (656.0 B)  TX bytes:0 (0.0 B)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

在host2上:

创建容器
# docker run -d --name test_host2 --net test busybox sh -c "while true;do sleep 3600;done"

查看网络信息
# docker exec test_host2 ifconfig
eth0      Link encap:Ethernet  HWaddr 02:42:0A:00:00:04  
          inet addr:10.0.0.4  Bcast:10.0.0.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1450  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

eth1      Link encap:Ethernet  HWaddr 02:42:AC:12:00:03  
          inet addr:172.18.0.3  Bcast:172.18.255.255  Mask:255.255.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:8 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:656 (656.0 B)  TX bytes:0 (0.0 B)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

测试连通性
# docker exec test_host2 sh -c 'ping -c 3 10.0.0.2'
PING 10.0.0.2 (10.0.0.2): 56 data bytes
64 bytes from 10.0.0.2: seq=0 ttl=64 time=4.907 ms
64 bytes from 10.0.0.2: seq=1 ttl=64 time=1.367 ms
64 bytes from 10.0.0.2: seq=2 ttl=64 time=0.684 ms

--- 10.0.0.2 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.684/2.319/4.907 ms

能够ping通host1上面的test_host1容器,实现了跨主机网络overlay。

五、小结

key-value数据库不止etcd一种,Consul、Zookeeper都是docker所支持的。
etcd与consul相比:
consul:官方提供了web管理界面,节点健康检查,节点动态增减,提供DNS服务
etcd:没有web管理界面,节点健康检查,实现动态新增节点需依赖第三方组件,提供DNS服务
虽然consul是docker官方推荐的key-value软件,但是我们还是需要根据业务的具体需求来选择使用。

Last modification:February 13th, 2019 at 02:15 pm

Leave a Comment