前言

Kubernetes是用于自动部署、扩展和管理“容器化应用程序”的开源系统。该系统由Google设计并捐赠给Cloud Native Computing Foundation来使用。 它旨在提供“跨主机集群的自动部署、扩展以及运行应用程序容器的平台”。 它支持一系列容器工具，包括Docker等。

前置条件

不少于2台的拥有公网ip的云服务器(强烈建议使用海外服务器)，也可使用VMware虚拟化服务器，需要两块网卡，NAT网卡及仅主机模式网卡，服务器最低配置：CPU2核心，内存2G，硬盘20G。

NAT网卡为DHCP自动分配IP

仅主机模式网卡：

k8s-master 192.168.192.10

k8s-worker1 192.168.192.11

k8s-worker2 192.168.192.12

系统为CentOS 7.9

准备工作

以下准备工作均在所有节点上执行

关闭防火墙

1
2
systemctl stop firewalld
systemctl disable firewalld

关闭SElinux

1
2
setenforce 0
sed -i --follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux

关闭swap

1
2
3
swapoff /dev/mapper/centos-swap # 关闭swap
rm /dev/mapper/centos-swap # 删除swap
/dev/mapper/centos-swap swap swap default 0 0 # 请在/etc/fstab 注释或删除此行

配置时间同步

1
2
3
4
vim /etc/chrony.conf
# 修改时间同步服务器为 ntp.aliyun.com
systemctl restart chronyd
chronyc sources -v

修改主机名

1
2
3
4
5
6
7
8
9
# master节点
hostnamectl set-hostname k8s-master
# worker1节点
hostnamectl set-hostname k8s-worker1
# worker2节点
hostnamectl set-hostname k8s-worker2
.....
# worker*节点
hostnamectl set-hostname k8s-worker*

配置hosts文件

1
2
3
4
5
vim /etc/hosts

192.168.192.10 k8s-master
192.168.192.11 k8s-worker1
192.168.192.12 k8s-worker2

配置免密登录

1
2
3
4
# 仅在master节点上配置
ssh-keygen
ssh-copy-id root@k8s-worker1
ssh-copy-id root@k8s-worker2

安装docker

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
# 安装常用工具
yum install -y yum-utils device-mapper-persistent-data lvm2
# 添加docker软件源信息
yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
sed -i 's+download.docker.com+mirrors.aliyun.com/docker-ce+' /etc/yum.repos.d/docker-ce.repo
yum makecache fast
# 安装docker-ce
yum -y install docker-ce
# (可选) 安装指定版本的Docker-CE
yum list docker-ce.x86_64 --showduplicates | sort -r
# 开启docker服务
systemctl start docker
systemctl enable docker
# 配置镜像加速器
mkdir -p /etc/docker
tee /etc/docker/daemon.json <<-'EOF'
{
  "registry-mirrors": ["https://7c4t92zk.mirror.aliyuncs.com"]
}
EOF
# 配置cgroup
vim /etc/docker/daemon.json
{
  "registry-mirrors": ["https://7c4t92zk.mirror.aliyuncs.com"],
        "exec-opts": ["native.cgroupdriver=systemd"],
        "log-driver": "json-file",
        "log-opts": {
                "max-size": "100m"
        }
}
systemctl daemon-reload
systemctl restart docker

允许iptables 检查桥接流量

1
2
3
4
5
6
7
8
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
br_netfilter
EOF

cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF

开始安装K8s

添加K8s软件源(所有节点)

1
2
3
4
5
6
7
8
9
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

安装必需组件(所有节点)

1
2
3
yum install -y kubelet-1.23.0 kubectl-1.23.0 kubeadm-1.23.0
systemctl start kubelet
systemctl enable kubelet

初始化K8s集群(仅master节点)

1
2
3
4
5
6
# 初始化集群
kubeadm init --apiserver-advertise-address 192.168.192.11 --image-repository registry.aliyuncs.com/google_containers --pod-network-cidr 172.25.0.0/16 --service-cidr 10.88.0.0/12 --v=6
# 创建必需文件夹
mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config

安装网络插件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
# github安装
wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
kubectl apply -f kube-flannel.yml
# 托管安装
wget https://cloud.18db.top/d/aliyunpan/get/k8s/kube-flannel.yml
kubectl apply -f kube-flannel.yml

# 查看node状态
kubectl get nodes
---
NAME          STATUS     ROLES                  AGE   VERSION
k8s-master    Ready      control-plane,master   9h    v1.23.0
---

# 异常状态
---
NAME          STATUS     ROLES                  AGE   VERSION
k8s-master    NotReady      control-plane,master   9h    v1.23.0
---

# 若长时间仍处于NotReady状态,查看工作日志
kubectl describe node k8s-master

# 显示以下内容,CNI不正常工作
----
untime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
----

# 在每个不正常的节点进行手动配置CNI
docker pull quay.io/coreos/flannel:v0.9.1-amd64

# 查看是否有10-flannel.conflist,如果有
mkdir -p /etc/cni/net.d/
ll /etc/cni/net.d/
mv 10-flannel.conflist 10-flannel.conf

# 如果没有，手动创建
cat <<EOF> /etc/cni/net.d/10-flannel.conf
{
  "name": "cbr0",
  "cniVersion": "0.3.1",
  "plugins": [
    {
      "type": "flannel",
      "delegate": {
        "hairpinMode": true,
        "isDefaultGateway": true
      }
    },
    {
      "type": "portmap",
      "capabilities": {
        "portMappings": true
      }
    }
  ]
}
EOF

mkdir /usr/share/oci-umount/oci-umount.d -p
mkdir /run/flannel/

# 注意:三个node节点的FLANNEL_NETWORK和FLANNEL_SUBNET均不同
cat <<EOF> /run/flannel/subnet.env
FLANNEL_NETWORK=172.100.0.0/16
FLANNEL_SUBNET=172.100.1.0/24
FLANNEL_MTU=1450
FLANNEL_IPMASQ=true
EOF

# 最后查看cni0网卡是否正常
ip a | grep cni0

# 再次查看node状态,节点状态正常
NAME          STATUS     ROLES                  AGE   VERSION
k8s-master    Ready      control-plane,master   9h    v1.23.0

将Worker节点加入K8s集群

1
2
# k8s集群初始化后显示
kubeadm join [master节点ip] --token xxx --discovery-token-ca-cert-hash xxx

重置K8s集群

1
2
3
4
5
6
# 重置k8s集群(master节点)
kubeadm reset
# 删除必需目录(master节点)
rm -rf $HOME/.kube
# 重置k8s(worker节点),worker节点也必须做重置操作，否则无法重新添加k8s集群
kubeadm reset

使用 Deployment 运行一个无状态应用

1
2
3
4
5
6
7
8
9
10
# Kubernetes Deployment 对象来运行nginx:1.14.2
kubectl apply -f https://k8s.io/examples/application/deployment.yaml
# 查看Deployment 的相关信息
kubectl describe deployment nginx-deployment
# 列出该 Deployment 创建的 Pod
kubectl get pods -l app=nginx
# 展示某一个 Pod 信息
kubectl describe pod <pod-name>
# 删除deployment
kubectl delete deployment nginx-deployment

调试Pod

https://kubernetes.io/zh-cn/docs/tasks/debug/debug-application/debug-pods/