在谷歌云上自建 K8s 集群并使用 VPC Native 方式实现容器网络互通( 二 )

 
2 安装 Docker
在所有的 VM 上安装 Docker:
sudoyum install -y yum-utilssudo yum-config-manager--add-repohttps://download.docker.com/linux/centos/docker-ce.reposudo yum install docker-ce docker-ce-cli containerd.iosudo systemctl start dockersudo docker run hello-worldsudo mkdir /etc/dockercat <<EOF | sudo tee /etc/docker/daemon.json{"exec-opts": ["native.cgroupdriver=systemd"],"log-driver": "json-file","log-opts": {"max-size": "100m"},"storage-driver": "overlay2"}EOFsudo systemctl enabledockersudo systemctl daemon-reloadsudo systemctl restart docker 
3 安装 kubeadm 及其工具
系统设置 , 以及工具安装:
sudosystemctl stop firewalld.servicesudo systemctl disablefirewalld.servicesudo modprobe br_netfiltercat <<EOF | sudo tee /etc/modules-load.d/k8s.confbr_netfilterEOFcat <<EOF | sudo tee /etc/sysctl.d/k8s.confnet.bridge.bridge-nf-call-ip6tables = 1net.bridge.bridge-nf-call-iptables = 1EOFsudo sysctl --systemcat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo[kubernetes]name=Kubernetesbaseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-$basearchenabled=1gpgcheck=1repo_gpgcheck=1gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpghttps://packages.cloud.google.com/yum/doc/rpm-package-key.gpgexclude=kubelet kubeadm kubectlEOF# Set SELinuxin permissive mode (effectively disabling it)sudo setenforce 0sudo sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/configsudo yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetessudo systemctl enable--now kubelet 
配置 kubeadm 初始化参数
cat <<EOF | sudo tee sudo/proc/sys/net/ipv4/ip_forward1EOF4 关闭 IP Alias 的 local route table
在 GCE 中 , GCE agent 的 network Daemon 会监控 IP Alias 地址 , 并添加相应的路由 。但在这种情况下 , 会造成 container 通讯的故障 。通过下面的命令在 Master 和各个 Node 系统内修改 GCE agent 配置 , 并重启 google guest agent 服务 。目的是 Pod 通过 Alias IP 地址互访时 , 不通过虚机实例的 Nat , 而访问其他地址时 , 采用 NAT 的方式 。
sudo sed -i 's/ip_aliases =true/ip_aliases = false/'/etc/default/instance_configs.cfg sudo systemctl restartgoogle-guest-agentiptables -P FORWARD ACCEPT ip route show table local#查看本地路由表, 确保alias IP段没有在eth0上【在谷歌云上自建 K8s 集群并使用 VPC Native 方式实现容器网络互通】
三 通过 kubeadm 安装 kubernetes
1 安装配置 Master 节点
A 配置 Master 节点
export name="$(hostname)"export pod_cidr="192.168.16.0/21"exportservice_cidr="172.16.16.0/21"cat <<EOF > /tmp/kubeadm-config.yamlapiVersion: kubeadm.k8s.io/v1beta3kind: InitConfigurationbootstrapTokens:- groups:-system:bootstrAppers:kubeadm:default-node-tokentoken: youzhi.0123456789abcdefnodeRegistration:name: $namekubeletExtraArgs:cloud-provider: gcenetwork-plugin: kubenetnon-masquerade-cidr: 0.0.0.0/0---apiVersion: kubeadm.k8s.io/v1beta3kind: ClusterConfigurationnetworking:podSubnet: ${pod_cidr}serviceSubnet: ${service_cidr}apiServer:extraArgs:enable-admission-plugins:DefaultStorageClass,NodeRestrictioncloud-provider: gcecontrollerManager:extraArgs:cloud-provider: gceconfigure-cloud-routes: "false"address: 0.0.0.0EOF
B 通过配置文件安装
sudo kubeadm init--config=/tmp/kubeadm-config.yaml如果执行成功 , 会有以下输出 。
Your Kubernetes control-planehas initialized successfully!To start using your cluster, you need to run the following as a regular user:mkdir -p $HOME/.kubesudo cp -i /etc/kubernetes/admin.conf$HOME/.kube/configsudo chown $(id -u):$(id -g) $HOME/.kube/configAlternatively, if you arethe root user, you can run:exportKUBECONFIG=/etc/kubernetes/admin.confYou should now deploy a pod network to the cluster.Run "kubectlapply -f [podnetwork].yaml" with one of the options listed at:https://kubernetes.io/docs/concepts/cluster-administration/addons/Then you can join any number of worker nodes by running the following on eachas root:sudo kubeadm join 10.122.16.10:6443 --token youzhi.0123456789abcdef--discovery-token-ca-cert-hashsha256:e5c96b2d0499287b6884c27b8ec7293e6aab1ab09540a0386b91ecbadfb38d4f
C 配置 kubectl 配置文件
mkdir -p $HOME/.kubesudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/configsudo chown $(id -u):$(id -g) $HOME/.kube/configD 安装 ip-masq-agent
默认配置下 ,  Pod-Pod 访问时会进行 SNAT 。使用 Alias IP 方式直接路由方式时 , 可以通过部署 ip-masq-agent 的方式关闭 SNAT 。


推荐阅读