01-ETCD集群部署

[TOC]

部署ETCD集群

1、概述

- 大意：
    etcd 是兼具一致性和高可用性的键值数据库，可以作为保存 Kubernetes 所有集群数据的后台数据库。
- 官方网址：
    https://etcd.io/docs/

2、服务器配置

3、准备cfssl证书生成工具

    cfssl是一个开源的证书管理工具，使用json文件生成证书.
    

• 在任意一台服务器上操作，这里选择k8s01

wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64
wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64
chmod +x cfssl_linux-amd64 cfssljson_linux-amd64 cfssl-certinfo_linux-amd64
mv cfssl_linux-amd64 /usr/local/bin/cfssl
mv cfssljson_linux-amd64 /usr/local/bin/cfssljson
mv cfssl-certinfo_linux-amd64 /usr/bin/cfssl-certinfo

4、创建etcd相关目录

$ mkdir -pv /opt/kubernetes/etcd/{bin,cfg,ssl,date}

5、创建证书

5.1、创建ca证书json文件

$ cd /opt/kubernetes/etcd/ssl
# 创建ca-config
$ vim ca-config.json
{
  "signing": {
    "default": {
      "expiry": "87600h"
    },
    "profiles": {
      "www": {
         "expiry": "87600h",
         "usages": [
            "signing",
            "key encipherment",
            "server auth",
            "client auth"
        ]
      }
    }
  }
}
# 创建ca-csr
$ vim ca-csr.json 
{
    "CN": "etcd CA",
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "Beijing",
            "ST": "Beijing"
        }
    ]
}

5.2、生成ca证书

$ cfssl gencert -initca ca-csr.json | cfssljson -bare ca -

5.3、使用自签ca签发etcd证书

• 创建证书申请文件：

$ cd /opt/kubernetes/etcd/ssl
# 注意hosts内容，etcd集群内的ip都要写上，可以预留几个，为以后扩容使用
$ vim server-csr.json
{
    "CN": "etcd",
    "hosts": [
    "192.168.1.234",
    "192.168.1.235",
    "192.168.1.236"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "BeiJing",
            "ST": "BeiJing"
        }
    ]
}

• 生成证书

$ cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json | cfssljson -bare server
# 会生成server.pem和server-key.pem文件 
$ ll |grep server
-rw-r--r-- 1 root root 1013 Sep 14 15:06 server.csr
-rw-r--r-- 1 root root  290 Sep 14 15:05 server-csr.json
-rw------- 1 root root 1679 Sep 14 15:06 server-key.pem
-rw-r--r-- 1 root root 1338 Sep 14 15:06 server.pem

6、下载etcd二进制文件

• 下载地址

      https://github.com/etcd-io/etcd/releases/download/v3.5.0/etcd-v3.5.0-linux-amd64.tar.gz
      

7、部署ETCD集群

7.1、复制二进制文件到指定文件

$ tar xf etcd-v3.5.0-linux-amd64.tar.gz
$ cp etcd-v3.5.0-linux-amd64/{etcd,etcdctl,etcdutl} /opt/kubernetes/etcd/bin

7.2、创建etcd配置文件

$ vim /opt/kubernetes/etcd/cfg/etcd.conf
#[Member]
ETCD_NAME="etcd-1"                                            # k8s01为etcd-1，k8s02为etcd-2。。。每个节点唯一标识符
ETCD_DATA_DIR="/opt/kubernetes/etcd/data/default.etcd"
ETCD_LISTEN_PEER_URLS="https://192.168.1.234:2380"            # 修改对应ip，k8s01为234，k8s02为235...
ETCD_LISTEN_CLIENT_URLS="https://192.168.1.234:2379"        # 修改对应ip，k8s01为234，k8s02为235...

#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.1.234:2380"        # 修改对应ip，k8s01为234，k8s02为235...
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.1.234:2379"                # 修改对应ip，k8s01为234，k8s02为235...
ETCD_INITIAL_CLUSTER="etcd-1=https://192.168.1.234:2380,etcd-2=https://192.168.1.235:2380,etcd-3=https://192.168.1.236:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"

• 注释

ETCD_NAME：节点名称，集群中唯一
ETCD_DATA_DIR：数据目录
ETCD_LISTEN_PEER_URLS：集群通信监听地址
ETCD_LISTEN_CLIENT_URLS：客户端访问监听地址
ETCD_INITIAL_ADVERTISE_PEERURLS：集群通告地址
ETCD_ADVERTISE_CLIENT_URLS：客户端通告地址
ETCD_INITIAL_CLUSTER：集群节点地址
ETCD_INITIALCLUSTER_TOKEN：集群Token
ETCD_INITIALCLUSTER_STATE：加入集群的当前状态，new是新集群，existing表示加入已有集群

7.3、创建systemd文件

$ vim /etc/systemd/system/etcd.service
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
EnvironmentFile=/opt/kubernetes/etcd/cfg/etcd.conf
ExecStart=/opt/kubernetes/etcd/bin/etcd \
--cert-file=/opt/kubernetes/etcd/ssl/server.pem \
--key-file=/opt/kubernetes/etcd/ssl/server-key.pem \
--peer-cert-file=/opt/kubernetes/etcd/ssl/server.pem \
--peer-key-file=/opt/kubernetes/etcd/ssl/server-key.pem \
--trusted-ca-file=/opt/kubernetes/etcd/ssl/ca.pem \
--peer-trusted-ca-file=/opt/kubernetes/etcd/ssl/ca.pem \
--logger=zap
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

8、分发etcd文件

• k8s02

$ scp -r /opt/kubernetes k8s02:/opt/
$ scp /etc/systemd/system/etcd.service k8s02:/etc/systemd/system/
# 记得修改etcd配置文件

• k8s03

$ scp -r /opt/kubernetes k8s03:/opt/
$ scp /etc/systemd/system/etcd.service k8s03:/etc/systemd/system/
# 记得修改etcd配置文件

9、分别启动etcd服务

• k8s01

$ systemctl start etcd.service

• k8s02

$ systemctl start etcd.service

• k8s03

$ systemctl start etcd.service

10、查看集群状态

$ cd /opt/kubernetes/etcd
$ ETCDCTL_API=3 ./bin/etcdctl --cacert=/opt/kubernetes/etcd/ssl/ca.pem --cert=/opt/kubernetes/etcd/ssl/server.pem --key=/opt/kubernetes/etcd/ssl/server-key.pem --endpoints="https://192.168.1.234:2379,https://192.168.1.235:2379,https://192.168.1.236:2379" endpoint health --write-out=table
+----------------------------+--------+-------------+-------+
|          ENDPOINT          | HEALTH |    TOOK     | ERROR |
+----------------------------+--------+-------------+-------+
| https://192.168.1.234:2379 |   true |  8.913068ms |       |
| https://192.168.1.236:2379 |   true |  9.757387ms |       |
| https://192.168.1.235:2379 |   true | 12.405075ms |       |
+----------------------------+--------+-------------+-------+

以上内容均为学习笔记，仅供参考。