重庆分公司,新征程启航
为企业提供网站建设、域名注册、服务器等服务
由于众所周知的原因,在国内无法直接访问Google的服务。二进制包由于其下载方便、灵活定制而深受广大kubernetes使用者喜爱,成为企业部署生产环境比较流行的方式之一,Kubernetes v1.13.2是目前的最新版本。安装部署过程可能比较复杂、繁琐,因此在安装过程中尽可能将操作步骤脚本话。文中涉及到的脚本已经通过本人测试。
为五河等地区用户提供了全套网页设计制作服务,及五河网站建设行业解决方案。主营业务为做网站、网站建设、五河网站设计,以传统方式定制建设网站,并提供域名空间备案等一条龙服务,秉承以专业、用心的态度为用户提供真诚的服务。我们深信只要达到每一位用户的要求,就会得到认可,从而选择与我们长期合作。这样,我们也可以走得更远!
OS(最小化安装版):
cat /etc/centos-release
CentOS Linux release 7.6.1810 (Core)
Docker Engine:
docker version
Client:
Version: 18.06.0-ce
API version: 1.38
Go version: go1.10.3
Git commit: 0ffa825
Built: Wed Jul 18 19:08:18 2018
OS/Arch: linux/amd64
Experimental: false
Server:
Engine:
Version: 18.06.0-ce
API version: 1.38 (minimum version 1.12)
Go version: go1.10.3
Git commit: 0ffa825
Built: Wed Jul 18 19:10:42 2018
OS/Arch: linux/amd64
Experimental: false
Kubenetes:
kubectl version
Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.2", GitCommit:"cff46ab41ff0bb44d8584413b598ad8360ec1def", GitTreeState:"clean", BuildDate:"2019-01-10T23:35:51Z", GoVersion:"go1.11.4", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.2", GitCommit:"cff46ab41ff0bb44d8584413b598ad8360ec1def", GitTreeState:"clean", BuildDate:"2019-01-10T23:28:14Z", GoVersion:"go1.11.4", Compiler:"gc", Platform:"linux/amd64"}
ETCD:
etcd --version
etcd Version: 3.3.11
Git SHA: 2cf9e51d2
Go Version: go1.10.7
Go OS/Arch: linux/amd64
Flannel:
flanneld -version
v0.11.0
IP | 主机名(Hostname) | 角色(Role) | 组件(Component) |
---|---|---|---|
172.31.2.11 | gysl-master | Master&Node | kube-apiserver,kube-controller-manager,kube-scheduler,etcd,(kubectl),kubelet,kube-proxy,docker,flannel |
172.31.2.12 | gysl-node1 | Node | kubelet,kube-proxy,docker,flannel,etcd |
172.31.2.13 | gysl-node2 | Node | kubelet,kube-proxy,docker,flannel,etcd |
注:加粗部分是Master节点必须安装的组件,etcd可以部署在其他节点,也可以部署在Master节点,kubectl是管理kubernetes的命令行工具。其余部分是Node节点必选组件。
Master节点:
Master节点上面主要由四个模块组成,apiserver,schedule,controller-manager,etcd。
apiserver: 负责对外提供RESTful的kubernetes API 的服务,它是系统管理指令的统一接口,任何对资源的增删该查都要交给apiserver处理后再交给etcd。kubectl(kubernetes提供的客户端工具,该工具内部是对kubernetes API的调用)是直接和apiserver交互的。
schedule: 负责调度Pod到合适的Node上,如果把scheduler看成一个黑匣子,那么它的输入是pod和由多个Node组成的列表,输出是Pod和一个Node的绑定。kubernetes目前提供了调度算法,同样也保留了接口。用户根据自己的需求定义自己的调度算法。
controller-manager: 如果apiserver做的是前台的工作的话,那么controller-manager就是负责后台的。每一个资源都对应一个控制器。而control manager就是负责管理这些控制器的,比如我们通过APIServer创建了一个Pod,当这个Pod创建成功后,apiserver的任务就算完成了。
etcd:etcd是一个高可用的键值存储系统,kubernetes使用它来存储各个资源的状态,从而实现了Restful的API。
Node节点:
每个Node节点主要由二个模块组成:kublet, kube-proxy。
kube-proxy: 该模块实现了kubernetes中的服务发现和反向代理功能。kube-proxy支持TCP和UDP连接转发,默认基Round Robin算法将客户端流量转发到与service对应的一组后端pod。服务发现方面,kube-proxy使用etcd的watch机制监控集群中service和endpoint对象数据的动态变化,并且维护一个service到endpoint的映射关系,从而保证了后端pod的IP变化不会对访问者造成影响,另外,kube-proxy还支持session affinity。
kublet:kublet是Master在每个Node节点上面的agent,是Node节点上面最重要的模块,它负责维护和管理该Node上的所有容器,但是如果容器不是通过kubernetes创建的,它并不会管理。本质上,它负责使Pod的运行状态与期望的状态一致。
在所有主机上执行脚本KubernetesInstall-01.sh,以Master节点为例。
[root@gysl-master ~]# sh KubernetesInstall-01.sh
脚本内容如下:
#!/bin/bash
# Initialize the machine. This needs to be executed on every machine.
# Add host domain name.
cat>>/etc/hosts</etc/sysctl.d/kubernetes.conf<&/dev/null
# Turn off and disable the firewalld.
systemctl stop firewalld
systemctl disable firewalld
# Disable the SELinux.
sed -i.bak 's/=enforcing/=disabled/' /etc/selinux/config
# Disable the swap .
sed -i.bak 's/^.*swap/#&/g' /etc/fstab
# Reboot the machine.
reboot
在所有主机上执行脚本KubernetesInstall-02.sh,以Master节点为例。
[root@gysl-master ~]# sh KubernetesInstall-02.sh
脚本内容如下:
#!/bin/bash
# Install the Docker engine. This needs to be executed on every machine.
curl http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -o /etc/yum.repos.d/docker-ce.repo>&/dev/null
if [ $? -eq 0 ] ;
then
yum remove docker \
docker-client \
docker-client-latest \
docker-common \
docker-latest \
docker-latest-logrotate \
docker-logrotate \
docker-selinux \
docker-engine-selinux \
docker-engine>&/dev/null
yum list docker-ce --showduplicates|grep "^doc"|sort -r
yum -y install docker-ce-18.06.0.ce-3.el7
rm -f /etc/yum.repos.d/docker-ce.repo
systemctl enable docker && systemctl start docker && systemctl status docker
else
echo "Install failed! Please try again! ";
exit 110
fi
注意:以上步骤需要在每一个节点上执行。如果启用了swap,那么是需要禁用的(脚本KubernetesInstall-01.sh已有涉及),具体可以通过 free 命令查看详情。另外,还需要关注各个节点上的时间同步情况。
在Master执行脚本KubernetesInstall-03.sh即可进行下载。
[root@gysl-master ~]# sh KubernetesInstall-03.sh
脚本内容如下:
#!/bin/bash
# Download relevant softwares. Please verify sha512 yourself.
while true;
do
echo "Downloading, please wait a moment." &&\
curl -L -C - -O https://dl.k8s.io/v1.13.2/kubernetes-server-linux-amd64.tar.gz && \
curl -L -C - -O https://github.com/etcd-io/etcd/releases/download/v3.2.26/etcd-v3.2.26-linux-amd64.tar.gz && \
curl -L -C - -O https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 && \
curl -L -C - -O https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 && \
curl -L -C - -O https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64 \
curl -L -C - -O https://github.com/coreos/flannel/releases/download/v0.11.0/flannel-v0.11.0-linux-amd64.tar.gz
if [ $? -eq 0 ];
then
echo "Congratulations! All software packages have been downloaded."
break
fi
done
kubernetes-server-linux-amd64.tar.gz包括了kubernetes的主要组件,无需下载其他包。etcd-v3.2.26-linux-amd64.tar.gz是部署etcd需要用到的包。其余的是cfssl相关的软件,暂不深究。网络原因,只能写脚本来下载了,这个过程可能需要一会儿。
在Master执行脚本KubernetesInstall-04.sh。
[root@gysl-master ~]# sh KubernetesInstall-04.sh
2019/01/28 16:29:47 [INFO] generating a new CA key and certificate from CSR
2019/01/28 16:29:47 [INFO] generate received request
2019/01/28 16:29:47 [INFO] received CSR
2019/01/28 16:29:47 [INFO] generating key: rsa-2048
2019/01/28 16:29:47 [INFO] encoded CSR
2019/01/28 16:29:47 [INFO] signed certificate with serial number 368034386524991671795323408390048460617296625670
2019/01/28 16:29:47 [INFO] generate received request
2019/01/28 16:29:47 [INFO] received CSR
2019/01/28 16:29:47 [INFO] generating key: rsa-2048
2019/01/28 16:29:48 [INFO] encoded CSR
2019/01/28 16:29:48 [INFO] signed certificate with serial number 714486490152688826461700674622674548864494534798
2019/01/28 16:29:48 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
/etc/etcd/ssl/ca-key.pem /etc/etcd/ssl/ca.pem /etc/etcd/ssl/server-key.pem /etc/etcd/ssl/server.pem
脚本内容如下:
#!/bin/bash
mv cfssl* /usr/local/bin/
chmod +x /usr/local/bin/cfssl*
ETCD_SSL=/etc/etcd/ssl
mkdir -p $ETCD_SSL
# Create some CA certificates for etcd cluster.
cat<$ETCD_SSL/ca-config.json
{
"signing": {
"default": {
"expiry": "87600h"
},
"profiles": {
"www": {
"expiry": "87600h",
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
]
}
}
}
}
EOF
cat<$ETCD_SSL/ca-csr.json
{
"CN": "etcd CA",
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"L": "Beijing",
"ST": "Beijing"
}
]
}
EOF
cat<$ETCD_SSL/server-csr.json
{
"CN": "etcd",
"hosts": [
"172.31.2.11",
"172.31.2.12",
"172.31.2.13"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"L": "Beijing",
"ST": "Beijing"
}
]
}
EOF
cd $ETCD_SSL
cfssl_linux-amd64 gencert -initca ca-csr.json | cfssljson_linux-amd64 -bare ca -
cfssl_linux-amd64 gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json | cfssljson_linux-amd64 -bare server
cd ~
# ca-key.pem ca.pem server-key.pem server.pem
ls $ETCD_SSL/*.pem
在Master执行脚本KubernetesInstall-05.sh。
[root@gysl-master ~]# sh KubernetesInstall-05.sh
脚本内容如下:
#!/bin/bash
# Deploy and configurate etcd service on the master node.
ETCD_CONF=/etc/etcd/etcd.conf
ETCD_SSL=/etc/etcd/ssl
ETCD_SERVICE=/usr/lib/systemd/system/etcd.service
tar -xzf etcd-v3.3.11-linux-amd64.tar.gz
cp -p etcd-v3.3.11-linux-amd64/etc* /usr/local/bin/
# The etcd configuration file.
cat>$ETCD_CONF<$ETCD_SERVICE<
在Node1执行脚本KubernetesInstall-06.sh。
[root@gysl-master ~]# sh KubernetesInstall-06.sh
脚本内容如下:
#!/bin/bash
# Deploy etcd on the node1.
ETCD_SSL=/etc/etcd/ssl
mkdir -p $ETCD_SSL
scp gysl-master:~/etcd-v3.3.11-linux-amd64.tar.gz .
scp gysl-master:$ETCD_SSL/{ca*pem,server*pem} $ETCD_SSL/
scp gysl-master:/etc/etcd/etcd.conf /etc/etcd/
scp gysl-master:/usr/lib/systemd/system/etcd.service /usr/lib/systemd/system/
tar -xvzf etcd-v3.3.11-linux-amd64.tar.gz
mv ~/etcd-v3.3.11-linux-amd64/etcd* /usr/local/bin/
sed -i '/ETCD_NAME/{s/etcd-01/etcd-02/g}' /etc/etcd/etcd.conf
sed -i '/ETCD_LISTEN_PEER_URLS/{s/2.11/2.12/g}' /etc/etcd/etcd.conf
sed -i '/ETCD_LISTEN_CLIENT_URLS/{s/2.11/2.12/g}' /etc/etcd/etcd.conf
sed -i '/ETCD_INITIAL_ADVERTISE_PEER_URLS/{s/2.11/2.12/g}' /etc/etcd/etcd.conf
sed -i '/ETCD_ADVERTISE_CLIENT_URLS/{s/2.11/2.12/g}' /etc/etcd/etcd.conf
rm -rf ~/etcd-v3.3.11-linux-amd64*
systemctl daemon-reload
systemctl enable etcd.service --now
systemctl status etcd
在Node2执行脚本KubernetesInstall-07.sh。
[root@gysl-master ~]# sh KubernetesInstall-07.sh
脚本内容如下:
#!/bin/bash
# Deploy etcd on the node2.
ETCD_SSL=/etc/etcd/ssl
mkdir -p $ETCD_SSL
scp gysl-master:~/etcd-v3.3.11-linux-amd64.tar.gz .
scp gysl-master:$ETCD_SSL/{ca*pem,server*pem} $ETCD_SSL/
scp gysl-master:/etc/etcd/etcd.conf /etc/etcd/
scp gysl-master:/usr/lib/systemd/system/etcd.service /usr/lib/systemd/system/
tar -xvzf etcd-v3.3.11-linux-amd64.tar.gz
mv ~/etcd-v3.3.11-linux-amd64/etcd* /usr/local/bin/
sed -i '/ETCD_NAME/{s/etcd-01/etcd-03/g}' /etc/etcd/etcd.conf
sed -i '/ETCD_LISTEN_PEER_URLS/{s/2.11/2.13/g}' /etc/etcd/etcd.conf
sed -i '/ETCD_LISTEN_CLIENT_URLS/{s/2.11/2.13/g}' /etc/etcd/etcd.conf
sed -i '/ETCD_INITIAL_ADVERTISE_PEER_URLS/{s/2.11/2.13/g}' /etc/etcd/etcd.conf
sed -i '/ETCD_ADVERTISE_CLIENT_URLS/{s/2.11/2.13/g}' /etc/etcd/etcd.conf
rm -rf ~/etcd-v3.3.11-linux-amd64*
systemctl daemon-reload
systemctl enable etcd.service --now
systemctl status etcd
几个节点上的安装过程大同小异,唯一不同的是etcd配置文件中的服务器IP要写当前节点的IP。主要参数:
执行以下命令:
[root@gysl-master ~]# etcdctl \
--ca-file=/etc/etcd/ssl/ca.pem \
--cert-file=/etc/etcd/ssl/server.pem \
--key-file=/etc/etcd/ssl/server-key.pem \
--endpoints="https://172.31.2.11:2379,https://172.31.2.12:2379,https://172.31.2.13:2379" cluster-health
member 82184ce461853bed is healthy: got healthy result from https://172.31.2.12:2379
member d85d48cef1ccfeaf is healthy: got healthy result from https://172.31.2.13:2379
member fe6e7c664377ad3b is healthy: got healthy result from https://172.31.2.11:2379
cluster is healthy
"cluster is healthy"说明etcd集群部署成功!如果存在问题,那么首先看日志:/var/log/message 或 journalctl -u etcd,找到问题,逐一解决。命令看起来不是那么直观,可以直接复制下面的命令来进行检验:
etcdctl \
--ca-file=/etc/etcd/ssl/ca.pem \
--cert-file=/etc/etcd/ssl/server.pem \
--key-file=/etc/etcd/ssl/server-key.pem \
--endpoints="https://172.31.2.11:2379,https://172.31.2.12:2379,https://172.31.2.13:2379" cluster-health
由于Flannel需要使用etcd存储自身的一个子网信息,所以要保证能成功连接Etcd,写入预定义子网段。写入的Pod网段${CLUSTER_CIDR}必须是/16段地址,必须与kube-controller-manager的–-cluster-cidr参数值一致。一般情况下,在每一个Node节点都需要进行配置,执行脚本KubernetesInstall-08.sh。
[root@gysl-master ~]# sh KubernetesInstall-08.sh
脚本内容如下:
#!/bin/bash
KUBE_CONF=/etc/kubernetes
FLANNEL_CONF=$KUBE_CONF/flannel.conf
mkdir $KUBE_CONF
tar -xvzf flannel-v0.11.0-linux-amd64.tar.gz
mv {flanneld,mk-docker-opts.sh} /usr/local/bin/
# Check whether etcd cluster is healthy.
etcdctl \
--ca-file=/etc/etcd/ssl/ca.pem \
--cert-file=/etc/etcd/ssl/server.pem \
--key-file=/etc/etcd/ssl/server-key.pem \
--endpoints="https://172.31.2.11:2379,\
https://172.31.2.12:2379,\
https://172.31.2.13:2379" cluster-health
# Writing into a predetermined subnetwork.
cd /etc/etcd/ssl
etcdctl \
--ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem \
--endpoints="https://172.31.2.11:2379,https://172.31.2.12:2379,https://172.31.2.13:2379" \
set /coreos.com/network/config '{ "Network": "172.17.0.0/16", "Backend": {"Type": "vxlan"}}'
cd ~
# Configuration the flannel service.
cat>$FLANNEL_CONF</usr/lib/systemd/system/flanneld.service<
在脚本执行之前需要把Flannel安装包拷贝到用户的HOME目录。脚本执行完毕之后需要检查各服务的状态,确保docker0和flannel.1在同一网段。
这一步中创建了kube-apiserver和kube-proxy相关的CA证书,在Master节点执行脚本KubernetesInstall-09.sh。
[root@gysl-master ~]# sh KubernetesInstall-09.sh
脚本内容如下:
#!/bin/bash
# Deploy the master node.
KUBE_SSL=/etc/kubernetes/ssl
mkdir $KUBE_SSL
# Create CA.
cat>$KUBE_SSL/ca-config.json<$KUBE_SSL/ca-csr.json<$KUBE_SSL/server-csr.json<$KUBE_SSL/kube-proxy-csr.json<
执行完毕之后应该看到以下文件:
/etc/kubernetes/ssl/ca-key.pem /etc/kubernetes/ssl/kube-proxy-key.pem /etc/kubernetes/ssl/server-key.pem
/etc/kubernetes/ssl/ca.pem /etc/kubernetes/ssl/kube-proxy.pem /etc/kubernetes/ssl/server.pem
将备好的安装包解压,并移动到相关目录,进行相关配置,执行脚本KubernetesInstall-10.sh。
[root@gysl-master ~]# sh KubernetesInstall-10.sh
脚本内容如下:
#!/bin/bash
KUBE_ETC=/etc/kubernetes
KUBE_API_CONF=/etc/kubernetes/apiserver.conf
tar -xvzf kubernetes-server-linux-amd64.tar.gz
mv kubernetes/server/bin/{kube-apiserver,kube-scheduler,kube-controller-manager} /usr/local/bin/
# Create a token file.
cat>$KUBE_ETC/token.csv<$KUBE_API_CONF</usr/lib/systemd/system/kube-apiserver.service<
参数说明:
之前已经将kube-scheduler相关的二进制文件移动到了相关目录,直接执行脚本KubernetesInstall-11.sh。
[root@gysl-master ~]# sh KubernetesInstall-11.sh
脚本内容如下:
#!/bin/bash
# Deploy the scheduler service.
KUBE_ETC=/etc/kubernetes
KUBE_SCHEDULER_CONF=$KUBE_ETC/kube-scheduler.conf
cat>$KUBE_SCHEDULER_CONF</usr/lib/systemd/system/kube-scheduler.service<
参数说明:
之前已经将kube-scheduler相关的二进制文件移动到了相关目录,直接执行脚本KubernetesInstall-12.sh。
[root@gysl-master ~]# sh KubernetesInstall-12.sh
脚本内容如下:
#!/bin/bash
# Deploy the controller-manager service.
KUBE_CONTROLLER_CONF=/etc/kubernetes/kube-controller-manager.conf
cat>$KUBE_CONTROLLER_CONF</usr/lib/systemd/system/kube-controller-manager.service<
直接执行脚本KubernetesInstall-13.sh。
[root@gysl-master ~]# sh KubernetesInstall-13.sh
脚本内容如下:
#!/bin/bash
# Check the service.
mv kubernetes/server/bin/kubectl /usr/local/bin/
kubectl get cs
如果部署成功的话,将看到如下结果:
[root@gysl-master ~]# kubectl get cs
NAME STATUS MESSAGE ERROR
controller-manager Healthy ok
scheduler Healthy ok
etcd-2 Healthy {"health":"true"}
etcd-0 Healthy {"health":"true"}
etcd-1 Healthy {"health":"true"}
Master apiserver启用TLS认证后,Node节点kubelet组件想要加入集群,必须使用CA签发的有效证书才能与apiserver通信,当Node节点很多时,签署证书是一件很繁琐的事情,因此有了TLS Bootstrapping机制,kubelet会以一个低权限用户自动向apiserver申请证书,kubelet的证书由apiserver动态签署。在前面创建的token文件在这一步派上了用场,在Master节点上执行脚本KubernetesInstall-14.sh创建bootstrap.kubeconfig和kube-proxy.kubeconfig。
[root@gysl-master ~]# sh KubernetesInstall-14.sh
脚本内容如下:
#!/bin/bash
BOOTSTRAP_TOKEN=$(awk -F "," '{print $1}' /etc/kubernetes/token.csv)
KUBE_SSL=/etc/kubernetes/ssl/
KUBE_APISERVER="https://172.31.2.11:6443"
cd $KUBE_SSL
# Set cluster parameters.
kubectl config set-cluster kubernetes \
--certificate-authority=./ca.pem \
--embed-certs=true \
--server=${KUBE_APISERVER} \
--kubeconfig=bootstrap.kubeconfig
# Set client parameters.
kubectl config set-credentials kubelet-bootstrap \
--token=${BOOTSTRAP_TOKEN} \
--kubeconfig=bootstrap.kubeconfig
# Set context parameters.
kubectl config set-context default \
--cluster=kubernetes \
--user=kubelet-bootstrap \
--kubeconfig=bootstrap.kubeconfig
# Set context.
kubectl config use-context default --kubeconfig=bootstrap.kubeconfig
# Create kube-proxy kubeconfig file.
kubectl config set-cluster kubernetes \
--certificate-authority=./ca.pem \
--embed-certs=true \
--server=${KUBE_APISERVER} \
--kubeconfig=kube-proxy.kubeconfig
kubectl config set-credentials kube-proxy \
--client-certificate=./kube-proxy.pem \
--client-key=./kube-proxy-key.pem \
--embed-certs=true \
--kubeconfig=kube-proxy.kubeconfig
kubectl config set-context default \
--cluster=kubernetes \
--user=kube-proxy \
--kubeconfig=kube-proxy.kubeconfig
kubectl config use-context default --kubeconfig=kube-proxy.kubeconfig
cd ~
# Bind kubelet-bootstrap user to system cluster roles.
kubectl create clusterrolebinding kubelet-bootstrap \
--clusterrole=system:node-bootstrapper \
--user=kubelet-bootstrap
因为kubernetes-server-linux-amd64.tar.gz已经在Master节点的HOME目录解压,所以可以在各节点上执行脚本KubernetesInstall-15.sh。
[root@gysl-node1 ~]# sh KubernetesInstall-15.sh
脚本内容如下:
#!/bin/bash
KUBE_CONF=/etc/kubernetes
KUBE_SSL=$KUBE_CONF/ssl
IP=172.31.2.13
mkdir $KUBE_SSL
scp gysl-master:~/kubernetes/server/bin/{kube-proxy,kubelet} /usr/local/bin/
scp gysl-master:$KUBE_CONF/ssl/{bootstrap.kubeconfig,kube-proxy.kubeconfig} $KUBE_CONF
cat>$KUBE_CONF/kube-proxy.conf</usr/lib/systemd/system/kube-proxy.service<$KUBE_CONF/kubelet.yaml<$KUBE_CONF/kubelet.conf</usr/lib/systemd/system/kubelet.service<
以上脚本有多少个Node节点就在相应的Node节点上执行多少次,每次执行只需修改IP的值即可。
参数说明:
可以手动或自动approve CSR请求。推荐使用自动的方式,因为从 v1.8 版本开始,可以自动轮转approve csr后生成的证书。未approve之前如下:
[root@gysl-master ~]# kubectl get csr
NAME AGE REQUESTOR CONDITION
node-csr-FpTP2sCI0SiYDCxaIHa1SRukS_5u9BQN10BsTd6RU1Y 20m kubelet-bootstrap Pending
node-csr-YYfnPwAws2LxJzV-OgYjJ22zy_z9XQM8PT0MnqZN910 24m kubelet-bootstrap Pending
在Master节点上执行脚本KubernetesInstall-15.sh。
[root@gysl-master ~]# sh KubernetesInstall-15.sh
certificatesigningrequest.certificates.k8s.io/node-csr-FpTP2sCI0SiYDCxaIHa1SRukS_5u9BQN10BsTd6RU1Y approved
certificatesigningrequest.certificates.k8s.io/node-csr-YYfnPwAws2LxJzV-OgYjJ22zy_z9XQM8PT0MnqZN910 approved
脚本内容如下:
#!/bin/bash
CSRS=$(kubectl get csr | awk '{if(NR>1) print $1}')
for csr in $CSRS;
do
kubectl certificate approve $csr;
done
[root@gysl-master ~]# kubectl get cs
NAME STATUS MESSAGE ERROR
controller-manager Healthy ok
scheduler Healthy ok
etcd-2 Healthy {"health":"true"}
etcd-1 Healthy {"health":"true"}
etcd-0 Healthy {"health":"true"}
[root@gysl-master ~]# kubectl get node
NAME STATUS ROLES AGE VERSION
172.31.2.12 Ready 11m v1.13.2
172.31.2.13 Ready 11m v1.13.2
[root@gysl-master ~]# kubectl run nginx --image=nginx --replicas=3
kubectl run --generator=deployment/apps.v1 is DEPRECATED and will be removed in a future version. Use kubectl run --generator=run-pod/v1 or kubectl create instead.
deployment.apps/nginx created
[root@gysl-master ~]# kubectl expose deployment nginx --port=88 --target-port=80 --type=NodePort
service/nginx exposed
[root@gysl-master ~]# kubectl get pods
NAME READY STATUS RESTARTS AGE
nginx-7cdbd8cdc9-7h946 0/1 ContainerCreating 0 33s
nginx-7cdbd8cdc9-vtkqf 0/1 ContainerCreating 0 33s
nginx-7cdbd8cdc9-wdjtj 0/1 ContainerCreating 0 33s
[root@gysl-master ~]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.0.0.1 443/TCP 8h
nginx NodePort 10.0.0.2 88:46705/TCP 28s
[root@gysl-master ~]# kubectl get pods
NAME READY STATUS RESTARTS AGE
nginx-7cdbd8cdc9-7h946 1/1 Running 0 2m4s
nginx-7cdbd8cdc9-vtkqf 1/1 Running 0 2m4s
nginx-7cdbd8cdc9-wdjtj 1/1 Running 0 2m4s
[root@gysl-node1 ~]# curl http://10.0.0.2:88
Welcome to nginx!
Welcome to nginx!
If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.
For online documentation and support please refer to
nginx.org.
Commercial support is available at
nginx.com.
Thank you for using nginx.
如果此时在浏览器输入:
资源比较充裕的情况下,Master节点仅仅做为服务接口、调度、控制节点,必须部署的组件有:kube-apiserver、kube-controller-manager、kube-scheduler、kubectl、etcd。除此之外,一般还需要做HA等相关部署。如果Master节点资源比较充裕,有些实验也要求至少有三个节点在运行,那么也可以将Master节点部署设置为一般Node节点来使用。为此,直接执行脚本KubernetesInstall-17.sh。
[root@gysl-master ~]# KubernetesInstall-17.sh
脚本内容如下:
#!/bin/bash
KUBE_CONF=/etc/kubernetes
KUBE_SSL=$KUBE_CONF/ssl
IP=172.31.2.11
mkdir $KUBE_SSL
cp ~/kubernetes/server/bin/{kube-proxy,kubelet} /usr/local/bin/
cp $KUBE_CONF/ssl/{bootstrap.kubeconfig,kube-proxy.kubeconfig} $KUBE_CONF
cat>$KUBE_CONF/kube-proxy.conf</usr/lib/systemd/system/kube-proxy.service<$KUBE_CONF/kubelet.yaml<$KUBE_CONF/kubelet.conf</usr/lib/systemd/system/kubelet.service<1) print $1}')
kubectl get csr
kubectl label node 172.31.2.11 node-role.kubernetes.io/master='master'
kubectl label node 172.31.2.12 node-role.kubernetes.io/node='node'
kubectl label node 172.31.2.13 node-role.kubernetes.io/node='node'
kubectl get nodes
部署成功之后,将出现以下内容:
NAME STATUS ROLES AGE VERSION
172.31.2.11 Ready master 22m v1.13.2
172.31.2.12 Ready node 11h v1.13.2
172.31.2.13 Ready node 11h v1.13.2
4.1 Kubernetes的二进制安装部署是一个比较复杂的过程,其中涉及到的步骤比较多,需要理解清楚各节点及组件之间的关系,逐步进行,每一个步骤成功了再进行下一步,切不可急躁。
4.2 在安装部署的过程中,日志及帮助信息是十分重要的,journalctl命令较为常用,--help也会起到柳暗花明又一村的效果。
4.3 把执行步骤脚本化,显得清晰有效,在后续的工作、学习过程中要继续保持。
4.4 由于时间仓促,安装部署中的很多个性化配置并未配置,在后续过程中会根据实际使用情况进行完善。比如:每一个服务或组件并未将日志单独保存。
4.5 其他不尽如人意的地方正在完善。
4.6 文中的两张图片来源于互联网,如有侵权,请联系删除。
5.1 认证相关
5.2 证书相关
5.3 cfssl官方资料
5.4 Systemd相关资料
5.5 Kubernetes基本概念
5.6 本文涉及到的脚本及配置文件