装饰装修网站大全,苏州网站开发公司兴田德润优惠吗,企业网站后台管理系统模板,如何用ps做网站首页图片kubernetes持久化存储卷 kubernetes持久化存储卷一、存储卷介绍二、存储卷的分类三、存储卷的选择四、本地存储卷之emptyDir五、本地存储卷之 hostPath六、网络存储卷之nfs七、PV(持久存储卷)与PVC(持久存储卷声明)7.1 认识pv与pvc7.2 pv与pvc之间的关系7.3 实现nfs类型pv与pvc… kubernetes持久化存储卷 kubernetes持久化存储卷一、存储卷介绍二、存储卷的分类三、存储卷的选择四、本地存储卷之emptyDir五、本地存储卷之 hostPath六、网络存储卷之nfs七、PV(持久存储卷)与PVC(持久存储卷声明)7.1 认识pv与pvc7.2 pv与pvc之间的关系7.3 实现nfs类型pv与pvc7.4 subpath使用 八、存储的动态供给8.1 什么是动态供给8.2 使用NFS文件系统创建存储动态供给 kubernetes持久化存储卷
一、存储卷介绍
pod有生命周期生命周期结束后 pod 里的数据会消失(如配置文件,业务数据等)。
解决: 我们需要将数据与pod分离将数据放在专门的存储卷上
pod在k8s集群的节点中是可以调度的, 如果pod挂了被调度到另一个节点那么数据和pod的联系会中断。
解决: 所以我们需要与集群节点分离的存储系统才能实现数据持久化
简单来说: volume提供了在容器上挂载外部存储的能力
二、存储卷的分类
kubernetes 支持的存储卷类型非常丰富使用下面的命令查看:
# kubectl explain pod.spec.volumes或者参考: https://kubernetes.io/docs/concepts/storage/
kubernetes支持的存储卷列表如下:
awsElasticBlockStoreazureDiskazureFilecephfscinderconfigMapcsidownwardAPIemptyDirfc (fibre channel)flexVolumeflockergcePersistentDiskgitRepo (deprecated)glusterfshostPathiscsilocalnfspersistentVolumeClaimprojectedportworxVolumequobyterbdscaleIOsecretstorageosvsphereVolume
我们将上面的存储卷列表进行简单的分类:
本地存储卷 emptyDir pod删除数据也会被清除用于数据的临时存储hostPath 宿主机目录映射(本地存储卷) 网络存储卷 NAS类 nfs等SAN类 iscsi,FC等分布式存储 glusterfs,cephfs,rbd,cinder等云存储 aws,azurefile等
三、存储卷的选择
市面上的存储产品种类繁多但按应用角度主要分为三类:
文件存储 如:nfsglusterfscephfs等 优点: 数据共享(多pod挂载可以同读同写)缺点: 性能较差 块存储 如: iscsirbd等 优点: 性能相对于文件存储好缺点: 不能实现数据共享部分 对象存储 如: ceph 对象存储 优点: 性能好数据共享缺点: 使用方式特殊支持较少
面对kubernetes支持的形形色色的存储卷如何选择成了难题。在选择存储时我们要抓住核心需求
数据是否需要持久性数据可靠性 如存储集群节点是否有单点故障数据是否有副本等性能扩展性 如是否能方便扩容应对数据增长的需求运维难度 存储的运维难度是比较高的尽量选择稳定的开源方案或商业产品成本
总之存储的选择是需要考虑很多因素的熟悉各类存储产品了解它们的优缺点结合自身需求才能选择合适自己的。
四、本地存储卷之emptyDir 应用场景 实现pod内容器之间数据共享 特点 随着pod被删除该卷也会被删除
创建yaml文件
[rootk8s-master1 ~]# vim volume-emptydir.yml
apiVersion: v1
kind: Pod
metadata:name: volume-emptydir
spec:containers:- name: writeimage: centosimagePullPolicy: IfNotPresentcommand: [bash,-c,echo haha /data/1.txt ; sleep 6000]volumeMounts:- name: datamountPath: /data- name: readimage: centosimagePullPolicy: IfNotPresentcommand: [bash,-c,cat /data/1.txt; sleep 6000]volumeMounts:- name: datamountPath: /data volumes:- name: dataemptyDir: {}基于yaml文件创建pod
[rootk8s-master1 ~]# kubectl apply -f volume-emptydir.yml
pod/volume-emptydir created查看pod启动情况
[rootk8s-master1 ~]# kubectl get pods |grep volume-emptydir
NAME READY STATUS RESTARTS AGE
volume-emptydir 2/2 Running 0 15s查看pod描述信息
[rootk8s-master1 ~]# kubectl describe pod volume-emptydir | tail -10
Events:Type Reason Age From Message---- ------ ---- ---- -------Normal Scheduled 50s default-scheduler Successfully assigned default/volume-emptydir to k8s-worker1Normal Pulling 50s kubelet Pulling image centos:centos7Normal Pulled 28s kubelet Successfully pulled image centos:centos7 in 21.544912361sNormal Created 28s kubelet Created container writeNormal Started 28s kubelet Started container writeNormal Pulled 28s kubelet Container image centos:centos7 already present on machineNormal Created 28s kubelet Created container readNormal Started 28s kubelet Started container read验证
[rootk8s-master1 ~]# kubectl logs volume-emptydir -c write
[rootk8s-master1 ~]# kubectl logs volume-emptydir -c read
haha五、本地存储卷之 hostPath 应用场景 pod内与集群节点目录映射pod中容器想访问节点上数据例如监控只有监控访问到节点主机文件才能知道集群节点主机状态 缺点 如果集群节点挂掉控制器在另一个集群节点拉起容器数据就会变成另一台集群节点主机的了无法实现数据共享)
创建yaml文件
[rootk8s-master1 ~]# vim volume-hostpath.yml
apiVersion: v1
kind: Pod
metadata:name: volume-hostpath
spec:containers:- name: busyboximage: busyboximagePullPolicy: IfNotPresentcommand: [/bin/sh,-c,echo haha /data/1.txt ; sleep 600]volumeMounts:- name: datamountPath: /datavolumes:- name: datahostPath:path: /opttype: Directory基于yaml文件创建pod
[rootk8s-master1 ~]# kubectl apply -f volume-hostpath.yml
pod/volume-hostpath created[rootk8s-master1 ~]# kubectl get pods -o wide |grep volume-hostpath
volume-hostpath 1/1 Running 0 29s 10.224.194.120 k8s-worker1 none none
可以看到pod是在k8s-worker1节点上验证pod所在机器上的挂载文件
[rootk8s-worker1 ~]# cat /opt/1.txt
haha六、网络存储卷之nfs
搭建nfs服务器
[rootnfsserver ~]# mkdir -p /data/nfs
[rootnfsserver ~]# vim /etc/exports
/data/nfs *(rw,no_root_squash,sync)
[rootnfsserver ~]# systemctl restart nfs-server
[rootnfsserver ~]# systemctl enable nfs-server所有node节点安装nfs客户端相关软件包
[rootk8s-worker1 ~]# yum install nfs-utils -y
[rootk8s-worker2 ~]# yum install nfs-utils -y验证nfs可用性
[rootnode1 ~]# showmount -e 192.168.10.129
Export list for 192.168.10.129:
/data/nfs *
[rootnode2 ~]# showmount -e 192.168.10.129
Export list for 192.168.10.129:
/data/nfs *master节点上创建yaml文件
[rootk8s-master1 ~]# vim volume-nfs.yml
apiVersion: apps/v1
kind: Deployment
metadata:name: volume-nfs
spec:replicas: 2selector:matchLabels:app: nginxtemplate:metadata:labels:app: nginxspec:containers:- name: nginximage: nginx:1.15-alpineimagePullPolicy: IfNotPresentvolumeMounts:- name: documentrootmountPath: /usr/share/nginx/htmlports:- containerPort: 80volumes:- name: documentrootnfs:server: 192.168.10.129path: /data/nfs应用yaml创建
[rootk8s-master1 ~]# kubectl apply -f volume-nfs.yml
deployment.apps/nginx-deployment created在nfs服务器共享目录中创建验证文件
[rootnfsserver ~]# echo volume-nfs /data/nfs/index.html验证pod
[rootk8s-master1 ~]# kubectl get pod |grep volume-nfs
volume-nfs-649d848b57-qg4bz 1/1 Running 0 10s
volume-nfs-649d848b57-wrnpn 1/1 Running 0 10s
[rootk8s-master1 ~]# kubectl exec -it volume-nfs-649d848b57-qg4bz -- /bin/sh
/ # ls /usr/share/nginx/html/
index.html
/ # cat /usr/share/nginx/html/index.html
volume-nfs # 文件内容与nfs服务器上创建的一致
/ # exit
[rootk8s-master1 ~]# kubectl exec -it volume-nfs-649d848b57-wrnpn -- /bin/sh
/ # ls /usr/share/nginx/html/
index.html
/ # cat /usr/share/nginx/html/index.html
volume-nfs # 文件内容与nfs服务器上创建的一致
/ # exit七、PV(持久存储卷)与PVC(持久存储卷声明)
7.1 认识pv与pvc
kubernetes存储卷的分类太丰富了每种类型都要写相应的接口与参数才行这就让维护与管理难度加大。
persistenvolume(PV) 是配置好的一段存储(可以是任意类型的存储卷)
也就是说将网络存储共享出来配置定义成PV。
PersistentVolumeClaim(PVC)是用户pod使用PV的申请请求。
用户不需要关心具体的volume实现细节只需要关心使用需求。
7.2 pv与pvc之间的关系 pv提供存储资源(生产者) pvc使用存储资源(消费者) 使用pvc绑定pv 7.3 实现nfs类型pv与pvc
编写创建pv的YAML文件
[rootk8s-master1 ~]# vim pv-nfs.yml
apiVersion: v1
kind: PersistentVolume # 类型为PersistentVolume(pv)
metadata: name: pv-nfs # 名称
spec:capacity:storage: 1Gi # 大小accessModes:- ReadWriteMany # 访问模式nfs:path: /data/nfs # nfs共享目录server: 192.168.10.129 # nfs服务器IP访问模式有3种 参考 链接
ReadWriteOnce 单节点读写挂载ReadOnlyMany 多节点只读挂载ReadWriteMany 多节点读写挂载
cephfs 存储卷3种类型都支持我们要实现多个nginx跨节点之间的数据共享所以选择ReadWriteMany模式。
创建pv并验证
[rootk8s-master1 ~]# kubectl apply -f pv-nfs.yml
persistentvolume/pv-nfs created
[rootk8s-master1 ~]# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pv-nfs 1Gi RWX Retain Available 81s说明:
RWX为ReadWriteMany的简写Retain是回收策略 Retain表示需要不使用了需要手动回收参考 回收策略
编写创建pvc的YAML文件
[rootk8s-master1 ~]# vim pvc-nfs.yml
apiVersion: v1
kind: PersistentVolumeClaim # 类型为PersistentVolumeClaim(pvc)
metadata:name: pvc-nfs # pvc的名称
spec:accessModes:- ReadWriteMany # 访问模式resources:requests:storage: 1Gi # 大小要与pv的大小保持一致创建pvc并验证
[rootk8s-master1 ~]# kubectl apply -f pvc-nfs.yml
persistentvolumeclaim/pvc-nfs created
[rootk8s-master1 ~]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
pvc-nfs Bound pv-nfs 1Gi RWX 38s注意: STATUS 必须为Bound状态 (Bound状态表示pvc与pv绑定OK)
编写deployment的YMAL
[rootk8s-master1 ~]# vim deploy-nginx-nfs.yml
apiVersion: apps/v1
kind: Deployment
metadata:name: deploy-nginx-nfs
spec:replicas: 2selector:matchLabels:app: nginxtemplate:metadata:labels:app: nginxspec:containers:- name: nginximage: nginx:1.15-alpineimagePullPolicy: IfNotPresentports:- containerPort: 80volumeMounts:- name: wwwmountPath: /usr/share/nginx/htmlvolumes:- name: wwwpersistentVolumeClaim:claimName: pvc-nfs应用YAML创建deploment
[rootk8s-master1 ~]# kubectl apply -f deploy-nginx-nfs.yml
deployment.apps/deploy-nginx-nfs created验证pod
[rootk8s-master1 ~]# kubectl get pod |grep deploy-nginx-nfs
deploy-nginx-nfs-6f9bc4546c-gbzcl 1/1 Running 0 1m46s
deploy-nginx-nfs-6f9bc4546c-hp4cv 1/1 Running 0 1m46s验证pod内卷的数据
[rootk8s-master1 ~]# kubectl exec -it deploy-nginx-nfs-6f9bc4546c-gbzcl -- /bin/sh
/ # ls /usr/share/nginx/html/
index.html
/ # cat /usr/share/nginx/html/index.html
volume-nfs
/ # exit[rootk8s-master1 ~]# kubectl exec -it deploy-nginx-nfs-6f9bc4546c-hp4cv -- /bin/sh
/ # ls /usr/share/nginx/html/
index.html
/ # cat /usr/share/nginx/html/index.html
volume-nfs
/ # exit7.4 subpath使用
subpath 是指可以把相同目录下不同子目录挂载到容器中不同的目录中使用的方法。以下通过案例演示
编辑文件
# vim 01_create_pod.yaml编辑后查看保持内容一致即可
# cat 01_create_pod.yaml
apiVersion: v1
kind: Pod
metadata:name: pod1
spec:containers:- name: c1image: busyboxcommand: [/bin/sleep,100000]volumeMounts:- name: datamountPath: /opt/data1subPath: data1- name: datamountPath: /opt/data2subPath: data2volumes:- name: datapersistentVolumeClaim:claimName: pvc-nfs执行文件创建pod
# kubectl apply -f 01_create_pod.yaml
pod/pod1 created编辑文件
# vim 02_create_pvc.yaml查看编辑后文件保持内容一致即可
# cat 02_create_pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim # 类型为PersistentVolumeClaim(pvc)
metadata:name: pvc-nfs # pvc的名称
spec:accessModes:- ReadWriteMany # 访问模式resources:requests:storage: 1Gi # 大小要与pv的大小保持一致执行文件创建pvc
# kubectl apply -f 02_create_pvc.yaml
persistentvolumeclaim/pvc-nfs created编辑文件
# vim 03_create_pv_nfs.yaml查看编辑后文件保持内容一致注意修改nfs服务器及其共享的目录
# cat 03_create_pv_nfs.yaml
apiVersion: v1
kind: PersistentVolume # 类型为PersistentVolume(pv)
metadata:name: pv-nfs # 名称
spec:capacity:storage: 1Gi # 大小accessModes:- ReadWriteMany # 访问模式nfs:path: /sdb # nfs共享目录server: 192.168.10.214执行文件创建pv
# kubectl apply -f 03_create_pv_nfs.yaml
persistentvolume/pv-nfs created在nfs服务器查看pod中目录是否自动添加到nfs服务器/sdb目录中
[rootnfsserver ~]# ls /sdb
data1 data2八、存储的动态供给
8.1 什么是动态供给
每次使用存储要先创建pv 再创建pvc真累所以我们可以实现使用存储的动态供给特性。
静态存储需要用户申请PVC时保证容量和读写类型与预置PV的容量及读写类型完全匹配而动态存储则无需如此。管理员无需预先创建大量的PV作为存储资源
Kubernetes从1.4 版起引入了一个新的资源对象 StorageClass可用于将存储资源定义为具有显著特性的类Class而不是具体的PV。用户通过PVC直接向意向的类别发出申请匹配由管理员事先创建的PV或者由其按需为用户动态创建PV这样就免去了需要先创建PV的过程。
8.2 使用NFS文件系统创建存储动态供给
PV对存储系统的支持可通过其插件来实现目前Kubernetes支持如下类型的插件。
官方地址https://kubernetes.io/docs/concepts/storage/storage-classes/
官方插件是不支持NFS动态供给的但是我们可以用第三方的插件来实现
第三方插件地址: https://github.com/kubernetes-retired/external-storage
下载并创建storageclass
[rootk8s-master1 ~]# wget https://raw.githubusercontent.com/kubernetes-sigs/nfs-subdir-external-provisioner/master/deploy/class.yaml
[rootk8s-master1 ~]# mv class.yaml storageclass-nfs.yml[rootk8s-master1 ~]# cat storageclass-nfs.yml
apiVersion: storage.k8s.io/v1
kind: StorageClass # 类型
metadata:name: nfs-client # 名称,要使用就需要调用此名称
provisioner: k8s-sigs.io/nfs-subdir-external-provisioner # 动态供给插件
parameters:archiveOnDelete: false # 删除数据时是否存档false表示不存档true表示存档[rootk8s-master1 ~]# kubectl apply -f storageclass-nfs.yml
storageclass.storage.k8s.io/managed-nfs-storage created[rootk8s-master1 ~]# kubectl get storageclass
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
nfs-client k8s-sigs.io/nfs-subdir-external-provisioner Delete Immediate false 10s# RECLAIMPOLICY pv回收策略pod或pvc被删除后pv是否删除还是保留。
# VOLUMEBINDINGMODE Immediate 模式下PVC与PV立即绑定主要是不等待相关Pod调度完成不关心其运行节点直接完成绑定。相反的 WaitForFirstConsumer模式下需要等待Pod调度完成后进行PV绑定。
# ALLOWVOLUMEEXPANSION pvc扩容下载并创建rbac
因为storage自动创建pv需要经过kube-apiserver所以需要授权。
[rootk8s-master1 ~]# wget https://raw.githubusercontent.com/kubernetes-sigs/nfs-subdir-external-provisioner/master/deploy/rbac.yaml
[rootk8s-master1 ~]# mv rbac.yaml storageclass-nfs-rbac.yaml[rootk8s-master1 ~]# cat storageclass-nfs-rbac.yaml
apiVersion: v1
kind: ServiceAccount
metadata:name: nfs-client-provisioner# replace with namespace where provisioner is deployednamespace: default
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:name: nfs-client-provisioner-runner
rules:- apiGroups: []resources: [persistentvolumes]verbs: [get, list, watch, create, delete]- apiGroups: []resources: [persistentvolumeclaims]verbs: [get, list, watch, update]- apiGroups: [storage.k8s.io]resources: [storageclasses]verbs: [get, list, watch]- apiGroups: []resources: [events]verbs: [create, update, patch]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:name: run-nfs-client-provisioner
subjects:- kind: ServiceAccountname: nfs-client-provisioner# replace with namespace where provisioner is deployednamespace: default
roleRef:kind: ClusterRolename: nfs-client-provisioner-runnerapiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:name: leader-locking-nfs-client-provisioner# replace with namespace where provisioner is deployednamespace: default
rules:- apiGroups: []resources: [endpoints]verbs: [get, list, watch, create, update, patch]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:name: leader-locking-nfs-client-provisioner# replace with namespace where provisioner is deployednamespace: default
subjects:- kind: ServiceAccountname: nfs-client-provisioner# replace with namespace where provisioner is deployednamespace: default
roleRef:kind: Rolename: leader-locking-nfs-client-provisionerapiGroup: rbac.authorization.k8s.io[rootk8s-master1 ~]# kubectl apply -f rbac.yaml
serviceaccount/nfs-client-provisioner created
clusterrole.rbac.authorization.k8s.io/nfs-client-provisioner-runner created
clusterrolebinding.rbac.authorization.k8s.io/run-nfs-client-provisioner created
role.rbac.authorization.k8s.io/leader-locking-nfs-client-provisioner created
rolebinding.rbac.authorization.k8s.io/leader-locking-nfs-client-provisioner created创建动态供给的deployment 需要一个deployment来专门实现pv与pvc的自动创建
[rootk8s-master1 ~]# vim deploy-nfs-client-provisioner.yml
apiVersion: apps/v1
kind: Deployment
metadata:name: nfs-client-provisioner
spec:replicas: 1strategy:type: Recreateselector:matchLabels:app: nfs-client-provisionertemplate:metadata:labels:app: nfs-client-provisionerspec:serviceAccount: nfs-client-provisionercontainers:- name: nfs-client-provisionerimage: registry.cn-beijing.aliyuncs.com/pylixm/nfs-subdir-external-provisioner:v4.0.0volumeMounts:- name: nfs-client-rootmountPath: /persistentvolumesenv:- name: PROVISIONER_NAMEvalue: k8s-sigs.io/nfs-subdir-external-provisioner- name: NFS_SERVERvalue: 192.168.10.129- name: NFS_PATHvalue: /data/nfsvolumes:- name: nfs-client-rootnfs:server: 192.168.10.129path: /data/nfs[rootk8s-master1 ~]# kubectl apply -f deploy-nfs-client-provisioner.yml
deployment.apps/nfs-client-provisioner created[rootk8s-master1 ~]# kubectl get pods |grep nfs-client-provisioner
nfs-client-provisioner-5b5ddcd6c8-b6zbq 1/1 Running 0 34s测试存储动态供给是否可用
# vim nginx-sc.yaml
---
apiVersion: v1
kind: Service
metadata:name: nginxlabels:app: nginx
spec:ports:- port: 80name: webclusterIP: Noneselector:app: nginx
---
apiVersion: apps/v1
kind: StatefulSet
metadata:name: web
spec:selector:matchLabels:app: nginxserviceName: nginxreplicas: 2template:metadata:labels:app: nginxspec:imagePullSecrets:- name: huoban-harborterminationGracePeriodSeconds: 10containers:- name: nginximage: nginx:latestports:- containerPort: 80name: webvolumeMounts:- name: wwwmountPath: /usr/share/nginx/htmlvolumeClaimTemplates:- metadata:name: wwwspec:accessModes: [ ReadWriteOnce ]storageClassName: nfs-clientresources:requests:storage: 1Gi[rootk8s-master1 nfs]# kubectl get pods
NAME READY STATUS RESTARTS AGE
nfs-client-provisioner-9c988bc46-pr55n 1/1 Running 0 95s
web-0 1/1 Running 0 95s
web-1 1/1 Running 0 61s[rootnfsserver ~]# ls /data/nfs/
default-www-web-0-pvc-c4f7aeb0-6ee9-447f-a893-821774b8d11f default-www-web-1-pvc-8b8a4d3d-f75f-43af-8387-b7073d07ec01 扩展
批量下载文件
# for file in class.yaml deployment.yaml rbac.yaml ;
do wget https://raw.githubusercontent.com/kubernetes-incubator/external-storage/master/nfs-client/deploy/$file ;
done
