公司百度网站建设,wordpress改字体颜色,网站快照不更新了,域名申请 网站建设目录 写在前面材料准备一. OpenStack部1. 创建虚拟网络和虚拟机2. 配置离线环境3. 环境准备3.1 配置网络3.2 配置主机名并配置解析3.3 时间调整3.4 安装openstack客户端3.5 安装部署MariaDB3.6 安装部署RabbitMQ控制节点操作3.7 安装部署Memcache控制节点操作 4. 部署配置keyst… 目录 写在前面材料准备一. OpenStack部1. 创建虚拟网络和虚拟机2. 配置离线环境3. 环境准备3.1 配置网络3.2 配置主机名并配置解析3.3 时间调整3.4 安装openstack客户端3.5 安装部署MariaDB3.6 安装部署RabbitMQ控制节点操作3.7 安装部署Memcache控制节点操作 4. 部署配置keystone控制节点操作 5. 部署配置glance镜像控制节点操作 6. 部署配置placement元数据7. 部署配置nova计算服务7.1 控制节点配置7.2 计算节点配置compute01节点compute02节点7.3 配置主机发现控制节点节点 8. 配置基于OVS的Neutron网络服务8.1 控制节点配置8.2 计算节点配置compute01节点compute02节点8.3 校验neutron 9. 配置dashboard仪表盘服务10. 部署配置cinder卷存储10.1 控制节点配置10.2 计算节点配置compute01节点compute02节点10.3 校验cinder 11. 运维实战控制节点操作11.1 加载openstack环境变量11.2 创建路由器11.3 创建Vxlan网络11.4 将内部网络添加到路由器11.5 创建Flat网络11.6 设置路由器网关接口11.7 开放安全组11.8 上传镜像11.9 创建云主机11.10 创建卷类型11.11 卷类型添加元数据11.12 查看卷类型11.13 创建卷11.14 卷绑定云主机 二. Ceph集群部署1. 环境准备1.1 配置地址1.2 更改主机名 2. 配置hosts解析所有节点3. 制作离线源所有节点4. 配置时间同步5. 安装docker(所有节点)6. 安装cephadm(node1)7. 导入ceph镜像(所有节点)7.1 搭建制作本地仓库(node1)7.2 配置私有仓库 8. 引导集群(node1)9. 安装ceph-common工具(node1)10. 添加主机到集群(node1)11. 部署OSD12. 访问仪表盘查看状态 三. OpenStack对接Ceph平台1. 创建后端需要的存储池node1节点操作1.1 cinder卷的存储池1.2 glance存储池1.3 备份存储池1.4 创建实例存储池 2. 创建后端用户2.1 创建密钥node1节点操作2.1.1 创建用户client.cinder2.1.2 创建用户client.glance2.1.3 创建用户client.cinder-backup 2.2 创建存放目录2.3 导出密钥node1节点2.4 拷贝密钥node1节点操作2.4.1 控制节点准备2.4.2 计算节点准备 3. 计算节点添加libvirt密钥3.1 compute01添加密钥3.2 compute02添加密钥 4. 安装ceph客户端主要作用是OpenStack可调用Ceph资源 5. 配置glance后端存储controller节点 6. 配置cinder后端存储7. 配置卷备份compute01、compute02节点 8. 配置nova集成cephcompute01、compute02节点8.1 热迁移配置compute01、compute02节点 写在前面
学如何用openstack做一个用户或者考一个COA那你只需要centos7上用packstack部署一个环境即可。 但是如果你是想做云计算工程师和老板靠这个卖钱则一定要有一个虚拟化专用系统镜像虚拟化部署面板类似Fuel工具的方式。系统镜像里面预装了所有openstack组件这样通过pxe可以对云计算集群进行快速扩容。这个系统叫做统一虚拟化平台uniformed virtualization platform简称UVP。阿里云飞天架构和华为云的飞升架构都是如此。 想要做出UVP靠开源的什么packstack、什么kolla之流能行吗openstack一堆花里胡哨的部署工具能上生产的一个没有只是拿来玩一玩。上海有一家云计算公司因为用kolla作为生产力工具结果崩溃后无法恢复CTO被开除了。 想要做出UVP就需要你手动部署全套openstack并进行配置本篇立志于用Ubuntu server来模拟UVP手动部署openstack。
材料准备
openstack离线安装包ceph离线安装包 https://pan.baidu.com/s/1eSFwN80bdloM46TFHX5n_g?pwd6666 提取码6666主机规划 系统Ubuntu Server22.04LTS 虚拟化工具qemu-kvm和virt-manager 虚拟网络manage 10.0.0.0/24 nat模式 provider 203.0.113.0/24, nat模式
主机名IP磁盘CPUmemorycontroller网卡1:10.0.0.10 网卡2不配置sda100GB4C8GBcompute01网卡1:10.0.0.11 网卡2不配置sda100GBsdb50GB8C16GBconpute02网卡1:10.0.0.12 网卡2不配置sda100GBsdb50GB8C16GBDNS服务器
一. OpenStack部
1. 创建虚拟网络和虚拟机 安装一个Ubuntu 22.04作为模板机供后续克隆。模板机添加刚才创建的两个网络。模板机系统内两个网口默认配置dhcp
2. 配置离线环境
# 解压
tar zxvf openstackyoga.tar.gz -C /opt/# 备份文件
cp /etc/apt/sources.list{,.bak}
# 配置离线源
cat /etc/apt/sources.list EOF
deb [trustedyes] file:// /opt/openstackyoga/debs/
EOF# 清空缓存
apt clean all# 加载源
apt update3. 环境准备
3.1 配置网络
controller节点
cat /etc/netplan/00-installer-config.yaml EOF
# This is the network config written by subiquity
network:ethernets:ens33:dhcp4: falseaddresses: [10.0.0.10/24]routes:- to: defaultvia: 10.0.0.254nameservers:addresses: [114.114.114.114]ens38:dhcp4: falseversion: 2
EOF# 生效网络
netplan applycompute01节点
cat /etc/netplan/00-installer-config.yaml EOF
# This is the network config written by subiquity
network:ethernets:ens33:dhcp4: falseaddresses: [10.0.0.11/24]routes:- to: defaultvia: 10.0.0.254nameservers:addresses: [114.114.114.114]ens38:dhcp4: falseversion: 2
EOF# 生效网络
netplan applycompute02节点
cat /etc/netplan/00-installer-config.yaml EOF
# This is the network config written by subiquity
network:ethernets:ens33:dhcp4: falseaddresses: [10.0.0.12/24]routes:- to: defaultvia: 10.0.0.254nameservers:addresses: [114.114.114.114]ens38:dhcp4: falseversion: 2
EOF# 生效网络
netplan apply3.2 配置主机名并配置解析
controller节点更改主机名
hostnamectl set-hostname controller# 切换窗口
bashcompute01节点更改主机名
hostnamectl set-hostname compute01# 切换窗口
bashcompute02节点更改主机名
hostnamectl set-hostname compute02# 切换窗口
bash所有节点配置hosts解析
cat /etc/hosts EOF
10.0.0.10 controller
10.0.0.11 compute01
10.0.0.12 compute02
EOF3.3 时间调整
所有节点
# 开启可配置服务
timedatectl set-ntp true# 调整时区为上海
timedatectl set-timezone Asia/Shanghai# 将系统时间同步到硬件时间
hwclock --systohc控制节点
# 安装服务
apt install -y chrony# 配置文件
vim /etc/chrony/chrony.conf
20 server controller iburst maxsources 2
61 allow all
63 local stratum 10# 重启服务
systemctl restart chronyd计算节点
# 安装服务
apt install -y chrony# 配置文件
vim /etc/chrony/chrony.conf
20 pool controller iburst maxsources 4# 重启服务
systemctl restart chronyd3.4 安装openstack客户端
controller节点
apt install -y python3-openstackclient3.5 安装部署MariaDB
controller节点
apt install -y mariadb-server python3-pymysql配置mariadb配置文件
cat /etc/mysql/mariadb.conf.d/99-openstack.cnf EOF
[mysqld]
bind-address 0.0.0.0default-storage-engine innodb
innodb_file_per_table on
max_connections 4096
collation-server utf8_general_ci
character-set-server utf8
EOF重启根据配置文件启动
service mysql restart初始化配置数据库
mysql_secure_installation
输入数据库密码回车
可以在没有适当授权的情况下登录到MariaDB root用户当前已收到保护n
设置root用户密码n
删除匿名用户y
不允许远程root登录n
删除测试数据库y
重新加载数据库y3.6 安装部署RabbitMQ 控制节点操作 controller节点安装服务
apt install -y rabbitmq-server创建openstack用户 用户名为openstack密码000000
rabbitmqctl add_user openstack 000000允许openstack用户进行配置、写入和读取访问
rabbitmqctl set_permissions openstack .* .* .*3.7 安装部署Memcache 控制节点操作 controller节点安装服务
apt install -y memcached python3-memcache配置监听地址
vim /etc/memcached.conf
35 -l 0.0.0.0重启服务
service memcached restart4. 部署配置keystone 控制节点操作 创建数据库与用户给予keystone使用
# 创建数据库
CREATE DATABASE keystone;# 创建用户
GRANT ALL PRIVILEGES ON keystone.* TO keystone% IDENTIFIED BY keystoneang;controller节点安装服务
apt install -y keystone配置keystone文件
# 备份配置文件
cp /etc/keystone/keystone.conf{,.bak}# 过滤覆盖文件
grep -Ev ^$|# /etc/keystone/keystone.conf.bak /etc/keystone/keystone.confvim /etc/keystone/keystone.conf
[DEFAULT]
log_dir /var/log/keystone
[application_credential]
[assignment]
[auth]
[cache]
[catalog]
[cors]
[credential]
[database]
connection mysqlpymysql://keystone:keystoneangcontroller/keystone
[domain_config]
[endpoint_filter]
[endpoint_policy]
[eventlet_server]
[extra_headers]
Distribution Ubuntu
[federation]
[fernet_receipts]
[fernet_tokens]
[healthcheck]
[identity]
[identity_mapping]
[jwt_tokens]
[ldap]
[memcache]
[oauth1]
[oslo_messaging_amqp]
[oslo_messaging_kafka]
[oslo_messaging_notifications]
[oslo_messaging_rabbit]
[oslo_middleware]
[oslo_policy]
[policy]
[profiler]
[receipt]
[resource]
[revoke]
[role]
[saml]
[security_compliance]
[shadow_users]
[token]
provider fernet
[tokenless_auth]
[totp]
[trust]
[unified_limit]
[wsgi]填充数据库
su -s /bin/sh -c keystone-manage db_sync keystone调用用户和组的密钥库 这些选项是为了允许在另一个操作系统用户/组下运行密钥库
# 用户
keystone-manage fernet_setup --keystone-user keystone --keystone-group keystone# 组
keystone-manage credential_setup --keystone-user keystone --keystone-group keystone在Queens发布之前keystone需要在两个单独的端口上运行以容纳Identity v2 API后者通常在端口35357上运行单独的仅限管理员的服务。随着v2 API的删除keystones可以在所有接口的同一端口上运行5000
keystone-manage bootstrap --bootstrap-password 000000 --bootstrap-admin-url http://controller:5000/v3/ --bootstrap-internal-url http://controller:5000/v3/ --bootstrap-public-url http://controller:5000/v3/ --bootstrap-region-id RegionOne编辑/etc/apache2/apache2.conf文件并配置ServerName选项以引用控制器节点
echo ServerName controller /etc/apache2/apache2.conf 重新启动Apache服务生效配置
service apache2 restart配置OpenStack认证环境变量
cat /etc/keystone/admin-openrc.sh EOF
export OS_PROJECT_DOMAIN_NAMEDefault
export OS_USER_DOMAIN_NAMEDefault
export OS_PROJECT_NAMEadmin
export OS_USERNAMEadmin
export OS_PASSWORD000000
export OS_AUTH_URLhttp://controller:5000/v3
export OS_IDENTITY_API_VERSION3
export OS_IMAGE_API_VERSION2
EOF加载环境变量
source /etc/keystone/admin-openrc.sh创建服务项目后期组件将使用这个项目
openstack project create --domain default --description Service Project service验证
openstack token issue5. 部署配置glance镜像 控制节点操作 创建数据库与用户给予glance使用
# 创建数据库
CREATE DATABASE glance;# 创建用户
GRANT ALL PRIVILEGES ON glance.* TO glance% IDENTIFIED BY glanceang;创建glance浏览用户
openstack user create --domain default --password glance glance将管理员角色添加到浏览用户和服务项目
openstack role add --project service --user glance admin创建浏览服务实体
openstack service create --name glance --description OpenStack Image image创建镜像服务API端点
openstack endpoint create --region RegionOne image public http://controller:9292openstack endpoint create --region RegionOne image internal http://controller:9292openstack endpoint create --region RegionOne image admin http://controller:9292安装glance镜像服务
apt install -y glance配置glance配置文件
# 备份配置文件
cp /etc/glance/glance-api.conf{,.bak}# 过滤覆盖配置文件
grep -Ev ^$|# /etc/glance/glance-api.conf.bak /etc/glance/glance-api.conf# 配置项信息
vim /etc/glance/glance-api.conf
[DEFAULT]
[barbican]
[barbican_service_user]
[cinder]
[cors]
[database]
connection mysqlpymysql://glance:glanceangcontroller/glance
[file]
[glance.store.http.store]
[glance.store.rbd.store]
[glance.store.s3.store]
[glance.store.swift.store]
[glance.store.vmware_datastore.store]
[glance_store]
stores file,http
default_store file
filesystem_store_datadir /var/lib/glance/images/
[healthcheck]
[image_format]
disk_formats ami,ari,aki,vhd,vhdx,vmdk,raw,qcow2,vdi,iso,ploop.root-tar
[key_manager]
[keystone_authtoken]
www_authenticate_uri http://controller:5000
auth_url http://controller:5000
memcached_servers controller:11211
auth_type password
project_domain_name Default
user_domain_name Default
project_name service
username glance
password glance
[oslo_concurrency]
[oslo_messaging_amqp]
[oslo_messaging_kafka]
[oslo_messaging_notifications]
[oslo_messaging_rabbit]
[oslo_middleware]
[oslo_policy]
[oslo_reports]
[paste_deploy]
flavor keystone
[profiler]
[store_type_location_strategy]
[task]
[taskflow_executor]
[vault]
[wsgi]填充数据库
su -s /bin/sh -c glance-manage db_sync glance重启glance服务生效配置
service glance-api restart上传镜像验证
# 下载镜像
wget http://download.cirros-cloud.net/0.4.0/cirros-0.4.0-x86_64-disk.img# 上传镜像命令
glance image-create --name cirros --file cirros-0.4.0-x86_64-disk.img --disk-format qcow2 --container-format bare --visibilitypublic# 查看镜像运行状态
rootcontroller:~# openstack image list
------------------------------------------------------
| ID | Name | Status |
------------------------------------------------------
| 12a404ea-5751-41c6-a319-8f63de543cd8 | cirros | active |
------------------------------------------------------6. 部署配置placement元数据 作用placement服务跟踪每个供应商的库存和使用情况。例如在一个计算节点创建一个实例的可消费资源如计算节点的资源提供者的CPU和内存磁盘从外部共享存储池资源提供商和IP地址从外部IP资源提供者。 创建数据库与用户给予placement使用
# 创建数据库
CREATE DATABASE placement;# 创建用户
GRANT ALL PRIVILEGES ON placement.* TO placement% IDENTIFIED BY placementang;创建服务用户
openstack user create --domain default --password placement placement将Placement用户添加到具有管理员角色的服务项目中
openstack role add --project service --user placement admin在服务目录中创建Placement API条目
openstack service create --name placement --description Placement API placement创建Placement API服务端点
openstack endpoint create --region RegionOne placement public http://controller:8778openstack endpoint create --region RegionOne placement internal http://controller:8778openstack endpoint create --region RegionOne placement admin http://controller:8778安装placement服务
apt install -y placement-api配置placement文件
# 备份配置文件
cp /etc/placement/placement.conf{,.bak}# 过滤覆盖文件
grep -Ev ^$|# /etc/placement/placement.conf.bak /etc/placement/placement.conf# 配置文件
vim /etc/placement/placement.conf
[DEFAULT]
[api]
auth_strategy keystone
[cors]
[keystone_authtoken]
auth_url http://controller:5000/v3
memcached_servers controller:11211
auth_type password
project_domain_name Default
user_domain_name Default
project_name service
username placement
password placement
[oslo_middleware]
[oslo_policy]
[placement]
[placement_database]
connection mysqlpymysql://placement:placementangcontroller/placement
[profiler]填充数据库
su -s /bin/sh -c placement-manage db sync placement重启apache加载placement配置
service apache2 restart验证
rootcontroller:~# placement-status upgrade check
-------------------------------------------
| Upgrade Check Results |
-------------------------------------------
| Check: Missing Root Provider IDs |
| Result: Success |
| Details: None |
-------------------------------------------
| Check: Incomplete Consumers |
| Result: Success |
| Details: None |
-------------------------------------------
| Check: Policy File JSON to YAML Migration |
| Result: Success |
| Details: None |
-------------------------------------------7. 部署配置nova计算服务
7.1 控制节点配置
创建数据库与用户给予nova使用
# 存放nova交互等数据
CREATE DATABASE nova_api;# 存放nova资源等数据
CREATE DATABASE nova;# 存放nova等元数据
CREATE DATABASE nova_cell0;# 创建管理nova_api库的用户
GRANT ALL PRIVILEGES ON nova_api.* TO nova% IDENTIFIED BY novaang;# 创建管理nova库的用户
GRANT ALL PRIVILEGES ON nova.* TO nova% IDENTIFIED BY novaang;# 创建管理nova_cell0库的用户
GRANT ALL PRIVILEGES ON nova_cell0.* TO nova% IDENTIFIED BY novaang;创建nova用户
openstack user create --domain default --password nova nova将管理员角色添加到nova用户
openstack role add --project service --user nova admin创建nova服务实体
openstack service create --name nova --description OpenStack Compute compute创建计算API服务端点
openstack endpoint create --region RegionOne compute public http://controller:8774/v2.1openstack endpoint create --region RegionOne compute internal http://controller:8774/v2.1openstack endpoint create --region RegionOne compute admin http://controller:8774/v2.1安装服务
apt install -y nova-api nova-conductor nova-novncproxy nova-scheduler配置nova文件
# 备份配置文件
cp /etc/nova/nova.conf{,.bak}# 过滤提取文件
grep -Ev ^$|# /etc/nova/nova.conf.bak /etc/nova/nova.conf# 配置结果
vim /etc/nova/nova.conf
[DEFAULT]
log_dir /var/log/nova
lock_path /var/lock/nova
state_path /var/lib/nova
transport_url rabbit://openstack:000000controller:5672/
my_ip 10.0.0.10
[api]
auth_strategy keystone
[api_database]
connection mysqlpymysql://nova:novaangcontroller/nova_api
[barbican]
[barbican_service_user]
[cache]
[cinder]
[compute]
[conductor]
[console]
[consoleauth]
[cors]
[cyborg]
[database]
connection mysqlpymysql://nova:novaangcontroller/nova
[devices]
[ephemeral_storage_encryption]
[filter_scheduler]
[glance]
api_servers http://controller:9292
[guestfs]
[healthcheck]
[hyperv]
[image_cache]
[ironic]
[key_manager]
[keystone]
[keystone_authtoken]
www_authenticate_uri http://controller:5000/
auth_url http://controller:5000/
memcached_servers controller:11211
auth_type password
project_domain_name Default
user_domain_name Default
project_name service
username nova
password nova
[libvirt]
[metrics]
[mks]
[neutron]
[notifications]
[oslo_concurrency]
lock_path /var/lib/nova/tmp
[oslo_messaging_amqp]
[oslo_messaging_kafka]
[oslo_messaging_notifications]
[oslo_messaging_rabbit]
[oslo_middleware]
[oslo_policy]
[oslo_reports]
[pci]
[placement]
region_name RegionOne
project_domain_name Default
project_name service
auth_type password
user_domain_name Default
auth_url http://controller:5000/v3
username placement
password placement
[powervm]
[privsep]
[profiler]
[quota]
[rdp]
[remote_debug]
[scheduler]
[serial_console]
[service_user]
[spice]
[upgrade_levels]
[vault]
[vendordata_dynamic_auth]
[vmware]
[vnc]
enabled true
server_listen $my_ip
server_proxyclient_address $my_ip
[workarounds]
[wsgi]
[zvm]
[cells]
enable False
[os_region_name]
openstack 填充nova_api数据库
su -s /bin/sh -c nova-manage api_db sync nova注册cell0数据库
su -s /bin/sh -c nova-manage cell_v2 map_cell0 nova创建cell1单元格
su -s /bin/sh -c nova-manage cell_v2 create_cell --namecell1 --verbose nova填充nova数据库
su -s /bin/sh -c nova-manage db sync nova验证nova、cell0和cell1是否正确注册
su -s /bin/sh -c nova-manage cell_v2 list_cells nova重启相关nova服务加载配置文件
# 处理api服务
service nova-api restart
# 处理资源调度服务
service nova-scheduler restart
# 处理数据库服务
service nova-conductor restart
# 处理vnc远程窗口服务
service nova-novncproxy restart7.2 计算节点配置 compute01节点 安装nova-compute服务
apt install -y nova-compute配置nova文件
# 备份配置文件
cp /etc/nova/nova.conf{,.bak}# 过滤覆盖配置文件
grep -Ev ^$|# /etc/nova/nova.conf.bak /etc/nova/nova.conf# 完整配置
vim /etc/nova/nova.conf
[DEFAULT]
log_dir /var/log/nova
lock_path /var/lock/nova
state_path /var/lib/nova
transport_url rabbit://openstack:000000controller
my_ip 10.0.0.11
[api]
auth_strategy keystone
[api_database]
[barbican]
[barbican_service_user]
[cache]
[cinder]
[compute]
[conductor]
[console]
[consoleauth]
[cors]
[cyborg]
[database]
[devices]
[ephemeral_storage_encryption]
[filter_scheduler]
[glance]
api_servers http://controller:9292
[guestfs]
[healthcheck]
[hyperv]
[image_cache]
[ironic]
[key_manager]
[keystone]
[keystone_authtoken]
www_authenticate_uri http://controller:5000/
auth_url http://controller:5000/
memcached_servers controller:11211
auth_type password
project_domain_name Default
user_domain_name Default
project_name service
username nova
password nova
[libvirt]
[metrics]
[mks]
[neutron]
[notifications]
[oslo_concurrency]
lock_path /var/lib/nova/tmp
[oslo_messaging_amqp]
[oslo_messaging_kafka]
[oslo_messaging_notifications]
[oslo_messaging_rabbit]
[oslo_middleware]
[oslo_policy]
[oslo_reports]
[pci]
[placement]
region_name RegionOne
project_domain_name Default
project_name service
auth_type password
user_domain_name Default
auth_url http://controller:5000/v3
username placement
password placement
[powervm]
[privsep]
[profiler]
[quota]
[rdp]
[remote_debug]
[scheduler]
[serial_console]
[service_user]
[spice]
[upgrade_levels]
[vault]
[vendordata_dynamic_auth]
[vmware]
[vnc]
enabled true
server_listen 0.0.0.0
server_proxyclient_address $my_ip
novncproxy_base_url http://10.0.0.10:6080/vnc_auto.html
[workarounds]
[wsgi]
[zvm]
[cells]
enable False
[os_region_name]
openstack 检测是否支持硬件加速 如果结果返回0需要配置如下
# 确定计算节点是否支持虚拟机的硬件加速
egrep -c (vmx|svm) /proc/cpuinfo# 如果结果返回 “0” 那么需要配置如下
vim /etc/nova/nova-compute.conf
[libvirt]
virt_type qemu重启服务生效nova配置
service nova-compute restartcompute02节点 安装nova-compute服务
apt install -y nova-compute配置nova文件
# 备份配置文件
cp /etc/nova/nova.conf{,.bak}# 过滤覆盖配置文件
grep -Ev ^$|# /etc/nova/nova.conf.bak /etc/nova/nova.conf# 完整配置
vim /etc/nova/nova.conf
[DEFAULT]
log_dir /var/log/nova
lock_path /var/lock/nova
state_path /var/lib/nova
transport_url rabbit://openstack:000000controller
my_ip 10.0.0.12
[api]
auth_strategy keystone
[api_database]
[barbican]
[barbican_service_user]
[cache]
[cinder]
[compute]
[conductor]
[console]
[consoleauth]
[cors]
[cyborg]
[database]
[devices]
[ephemeral_storage_encryption]
[filter_scheduler]
[glance]
api_servers http://controller:9292
[guestfs]
[healthcheck]
[hyperv]
[image_cache]
[ironic]
[key_manager]
[keystone]
[keystone_authtoken]
www_authenticate_uri http://controller:5000/
auth_url http://controller:5000/
memcached_servers controller:11211
auth_type password
project_domain_name Default
user_domain_name Default
project_name service
username nova
password nova
[libvirt]
[metrics]
[mks]
[neutron]
[notifications]
[oslo_concurrency]
lock_path /var/lib/nova/tmp
[oslo_messaging_amqp]
[oslo_messaging_kafka]
[oslo_messaging_notifications]
[oslo_messaging_rabbit]
[oslo_middleware]
[oslo_policy]
[oslo_reports]
[pci]
[placement]
region_name RegionOne
project_domain_name Default
project_name service
auth_type password
user_domain_name Default
auth_url http://controller:5000/v3
username placement
password placement
[powervm]
[privsep]
[profiler]
[quota]
[rdp]
[remote_debug]
[scheduler]
[serial_console]
[service_user]
[spice]
[upgrade_levels]
[vault]
[vendordata_dynamic_auth]
[vmware]
[vnc]
enabled true
server_listen 0.0.0.0
server_proxyclient_address $my_ip
novncproxy_base_url http://10.0.0.10:6080/vnc_auto.html
[workarounds]
[wsgi]
[zvm]
[cells]
enable False
[os_region_name]
openstack 检测是否支持硬件加速 如果结果返回0需要配置如下
# 确定计算节点是否支持虚拟机的硬件加速
egrep -c (vmx|svm) /proc/cpuinfo# 如果结果返回 “0” 那么需要配置如下
vim /etc/nova/nova-compute.conf
[libvirt]
virt_type qemu重启服务生效nova配置
service nova-compute restart7.3 配置主机发现 控制节点节点 查看有那些可用的计算节点
openstack compute service list --service nova-compute发现计算主机
su -s /bin/sh -c nova-manage cell_v2 discover_hosts --verbose nova配置每5分钟主机发现一次
vim /etc/nova/nova.conf[scheduler]
discover_hosts_in_cells_interval 300重启生效配置
service nova-api restart校验nova服务
rootcontroller:~# openstack compute service list
------------------------------------------------------------------------------------------------------------------------
| ID | Binary | Host | Zone | Status | State | Updated At |
------------------------------------------------------------------------------------------------------------------------
| 68178099-13c5-4464-9a55-71ea0dd30bf5 | nova-scheduler | controller | internal | enabled | up | 2022-09-29T05:45:49.000000 |
| bd2a33be-1457-41c1-8ce8-3d4a8cb43551 | nova-conductor | controller | internal | enabled | up | 2022-09-29T05:45:49.000000 |
| 98b4584d-f9bf-4c10-9fd8-331899ebf70b | nova-compute | compute01 | nova | enabled | up | 2022-09-29T05:45:53.000000 |
| f809da57-8999-4ba4-8a32-5b60991f8838 | nova-compute | compute02 | nova | enabled | up | 2022-09-29T05:45:56.000000 |
------------------------------------------------------------------------------------------------------------------------8. 配置基于OVS的Neutron网络服务
8.1 控制节点配置
创建数据库与用给予neutron使用
# 创建数据库
CREATE DATABASE neutron;# 创建用户
GRANT ALL PRIVILEGES ON neutron.* TO neutron% IDENTIFIED BY neutronang;创建neutron用户
openstack user create --domain default --password neutron neutron向neutron用户添加管理员角色
openstack role add --project service --user neutron admin创建neutron实体
openstack service create --name neutron --description OpenStack Networking network创建neutron的api端点
openstack endpoint create --region RegionOne network public http://controller:9696openstack endpoint create --region RegionOne network internal http://controller:9696openstack endpoint create --region RegionOne network admin http://controller:9696配置内核转发
cat /etc/sysctl.conf EOF
# 用于控制系统是否开启对数据包源地址的校验关闭
net.ipv4.conf.all.rp_filter0
net.ipv4.conf.default.rp_filter0
# 开启二层转发设备
net.bridge.bridge-nf-call-iptables1
net.bridge.bridge-nf-call-ip6tables1
EOF加载模块 作用桥接流量转发到iptables链
modprobe br_netfilter生效内核配置
sysctl -p安装ovs服务
apt install -y neutron-server neutron-plugin-ml2 neutron-l3-agent neutron-dhcp-agent neutron-metadata-agent neutron-openvswitch-agent配置neutron.conf文件 用于提供neutron主体服务
# 备份配置文件
cp /etc/neutron/neutron.conf{,.bak}# 过滤提取配置文件
grep -Ev ^$|# /etc/neutron/neutron.conf.bak /etc/neutron/neutron.conf# 完整配置
vim /etc/neutron/neutron.conf
[DEFAULT]
core_plugin ml2
service_plugins router
allow_overlapping_ips true
auth_strategy keystone
state_path /var/lib/neutron
dhcp_agent_notification true
allow_overlapping_ips true
notify_nova_on_port_status_changes true
notify_nova_on_port_data_changes true
transport_url rabbit://openstack:000000controller
[agent]
root_helper sudo /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf
[cache]
[cors]
[database]
connection mysqlpymysql://neutron:neutronangcontroller/neutron
[healthcheck]
[ironic]
[keystone_authtoken]
www_authenticate_uri http://controller:5000
auth_url http://controller:5000
memcached_servers controller:11211
auth_type password
project_domain_name default
user_domain_name default
project_name service
username neutron
password neutron
[nova]
auth_url http://controller:5000
auth_type password
project_domain_name default
user_domain_name default
region_name RegionOne
project_name service
username nova
password nova
[oslo_concurrency]
lock_path /var/lib/neutron/tmp
[oslo_messaging_amqp]
[oslo_messaging_kafka]
[oslo_messaging_notifications]
[oslo_messaging_rabbit]
[oslo_middleware]
[oslo_policy]
[oslo_reports]
[placement]
[privsep]
[quotas]
[ssl]配置ml2_conf.ini文件 用户提供二层网络插件服务
# 备份配置文件
cp /etc/neutron/plugins/ml2/ml2_conf.ini{,.bak}# 过滤覆盖文件
grep -Ev ^$|# /etc/neutron/plugins/ml2/ml2_conf.ini.bak /etc/neutron/plugins/ml2/ml2_conf.ini# 完整配置
vim /etc/neutron/plugins/ml2/ml2_conf.ini
[DEFAULT]
[ml2]
type_drivers flat,vlan,vxlan,gre
tenant_network_types vxlan
mechanism_drivers openvswitch,l2population
extension_drivers port_security
[ml2_type_flat]
flat_networks physnet1
[ml2_type_geneve]
[ml2_type_gre]
[ml2_type_vlan]
[ml2_type_vxlan]
vni_ranges 1:1000
[ovs_driver]
[securitygroup]
enable_ipset true
enable_security_group true
firewall_driver neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver
[sriov_driver]配置openvswitch_agent.ini文件 提供ovs代理服务
# 备份文件
cp /etc/neutron/plugins/ml2/openvswitch_agent.ini{,.bak}# 过滤覆盖文件
grep -Ev ^$|# /etc/neutron/plugins/ml2/openvswitch_agent.ini.bak /etc/neutron/plugins/ml2/openvswitch_agent.ini# 完整配置
vim /etc/neutron/plugins/ml2/openvswitch_agent.ini
[DEFAULT]
[agent]
l2_population True
tunnel_types vxlan
prevent_arp_spoofing True
[dhcp]
[network_log]
[ovs]
local_ip 10.0.0.10
bridge_mappings physnet1:br-ens38
[securitygroup]配置l3_agent.ini文件 提供三层网络服务
# 备份文件
cp /etc/neutron/l3_agent.ini{,.bak}# 过滤覆盖文件
grep -Ev ^$|# /etc/neutron/l3_agent.ini.bak /etc/neutron/l3_agent.ini# 完整配置
vim /etc/neutron/l3_agent.ini
[DEFAULT]
interface_driver neutron.agent.linux.interface.OVSInterfaceDriver
external_network_bridge
[agent]
[network_log]
[ovs]配置dhcp_agent文件 提供dhcp动态网络服务
# 备份文件
cp /etc/neutron/dhcp_agent.ini{,.bak}# 过滤覆盖文件
grep -Ev ^$|# /etc/neutron/dhcp_agent.ini.bak /etc/neutron/dhcp_agent.ini# 完整配置
vim /etc/neutron/dhcp_agent.ini
[DEFAULT]
interface_driver neutron.agent.linux.interface.OVSInterfaceDriver
dhcp_driver neutron.agent.linux.dhcp.Dnsmasq
enable_isolated_metadata True
[agent]
[ovs]配置metadata_agent.ini文件 提供元数据服务元数据什么 用来支持如指示存储位置、历史数据、资源查找、文件记录等功能。元数据算是一种电子式目录为了达到编制目录的目的必须在描述并收藏数据的内容或特色进而达成协助数据检索的目的。
# 备份文件
cp /etc/neutron/metadata_agent.ini{,.bak}# 过滤覆盖文件
grep -Ev ^$|# /etc/neutron/metadata_agent.ini.bak /etc/neutron/metadata_agent.ini# 完整配置
vim /etc/neutron/metadata_agent.ini
[DEFAULT]
nova_metadata_host controller
metadata_proxy_shared_secret ws
[agent]
[cache]配置nova文件 主要识别neutron配置从而能调用网络
vim /etc/nova/nova.conf[default]
linuxnet_interface_driver nova.network.linux_net.LinuxOVSlnterfaceDriver[neutron]
auth_url http://controller:5000
auth_type password
project_domain_name default
user_domain_name default
region_name RegionOne
project_name service
username neutron
password neutron
service_metadata_proxy true
metadata_proxy_shared_secret ws填充数据库
su -s /bin/sh -c neutron-db-manage --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini upgrade head neutron重启nova-api服务生效neutron配置
service nova-api restart新建一个外部网络桥接
ovs-vsctl add-br br-ens38将外部网络桥接映射到网卡 这里绑定第二张网卡属于业务网卡
ovs-vsctl add-port br-ens38 ens38重启neutron相关服务生效配置
# 提供neutron服务
service neutron-server restart
# 提供ovs服务
service neutron-openvswitch-agent restart
# 提供地址动态服务
service neutron-dhcp-agent restart
# 提供元数据服务
service neutron-metadata-agent restart
# 提供三层网络服务
service neutron-l3-agent restart8.2 计算节点配置 compute01节点 配置内核转发
cat /etc/sysctl.conf EOF
# 用于控制系统是否开启对数据包源地址的校验关闭
net.ipv4.conf.all.rp_filter0
net.ipv4.conf.default.rp_filter0
# 开启二层转发设备
net.bridge.bridge-nf-call-iptables1
net.bridge.bridge-nf-call-ip6tables1
EOF加载模块 作用桥接流量转发到iptables链
modprobe br_netfilter生效内核配置
sysctl -p安装neutron-ovs服务
apt install -y neutron-openvswitch-agent配置neutron文件 提供neutron主体服务
# 备份文件
cp /etc/neutron/neutron.conf{,.bak}# 过滤提取文件
grep -Ev ^$|# /etc/neutron/neutron.conf.bak /etc/neutron/neutron.conf# 完整配置
vim /etc/neutron/neutron.conf
[DEFAULT]
core_plugin ml2
service_plugins router
auth_strategy keystone
state_path /var/lib/neutron
allow_overlapping_ips true
transport_url rabbit://openstack:000000controller
[agent]
root_helper sudo /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf
[cache]
[cors]
[database]
[healthcheck]
[ironic]
[keystone_authtoken]
www_authenticate_uri http://controller:5000
auth_url http://controller:5000
memcached_servers controller:11211
auth_type password
project_domain_name default
user_domain_name default
project_name service
username neutron
password neutron
[nova]
[oslo_concurrency]
lock_path /var/lib/neutron/tmp
[oslo_messaging_amqp]
[oslo_messaging_kafka]
[oslo_messaging_notifications]
[oslo_messaging_rabbit]
[oslo_middleware]
[oslo_policy]
[oslo_reports]
[placement]
[privsep]
[quotas]
[ssl]配置openvswitch_agent.ini文件 提供ovs网络服务
# 备份文件
cp /etc/neutron/plugins/ml2/openvswitch_agent.ini{,.bak}# 过滤提取文件
grep -Ev ^$|# /etc/neutron/plugins/ml2/openvswitch_agent.ini.bak /etc/neutron/plugins/ml2/openvswitch_agent.ini# 完整配置
vim /etc/neutron/plugins/ml2/openvswitch_agent.ini
[DEFAULT]
[agent]
l2_population True
tunnel_types vxlan
prevent_arp_spoofing True
[dhcp]
[network_log]
[ovs]
local_ip 10.0.0.11
bridge_mappings physnet1:br-ens38
[securitygroup]
enable_security_group True
firewall_driver neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver配置nova文件识别neutron配置
vim /etc/nova/nova.conf[DEFAULT]
linuxnet_interface_driver nova.network.linux_net.LinuxOVSlnterfaceDriver
vif_plugging_is_fatal true
vif_pligging_timeout 300[neutron]
auth_url http://controller:5000
auth_type password
project_domain_name default
user_domain_name default
region_name RegionOne
project_name service
username neutron
password neutron重启nova服务识别网络配置
service nova-compute restart新建一个外部网络桥接
ovs-vsctl add-br br-ens38将外部网络桥接映射到网卡 这里绑定第二张网卡属于业务网卡
ovs-vsctl add-port br-ens38 ens38重启服务加载ovs配置
service neutron-openvswitch-agent restartcompute02节点 配置内核转发
cat /etc/sysctl.conf EOF
# 用于控制系统是否开启对数据包源地址的校验关闭
net.ipv4.conf.all.rp_filter0
net.ipv4.conf.default.rp_filter0
# 开启二层转发设备
net.bridge.bridge-nf-call-iptables1
net.bridge.bridge-nf-call-ip6tables1
EOF加载模块 作用桥接流量转发到iptables链
modprobe br_netfilter生效内核配置
sysctl -p安装neutron-ovs服务
apt install -y neutron-openvswitch-agent配置neutron文件 提供neutron主体服务
# 备份文件
cp /etc/neutron/neutron.conf{,.bak}# 过滤提取文件
grep -Ev ^$|# /etc/neutron/neutron.conf.bak /etc/neutron/neutron.conf# 完整配置
vim /etc/neutron/neutron.conf
[DEFAULT]
core_plugin ml2
service_plugins router
auth_strategy keystone
state_path /var/lib/neutron
allow_overlapping_ips true
transport_url rabbit://openstack:000000controller
[agent]
root_helper sudo /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf
[cache]
[cors]
[database]
[healthcheck]
[ironic]
[keystone_authtoken]
www_authenticate_uri http://controller:5000
auth_url http://controller:5000
memcached_servers controller:11211
auth_type password
project_domain_name default
user_domain_name default
project_name service
username neutron
password neutron
[nova]
[oslo_concurrency]
lock_path /var/lib/neutron/tmp
[oslo_messaging_amqp]
[oslo_messaging_kafka]
[oslo_messaging_notifications]
[oslo_messaging_rabbit]
[oslo_middleware]
[oslo_policy]
[oslo_reports]
[placement]
[privsep]
[quotas]
[ssl]配置openvswitch_agent.ini文件 提供ovs网络服务
# 备份文件
cp /etc/neutron/plugins/ml2/openvswitch_agent.ini{,.bak}# 过滤提取文件
grep -Ev ^$|# /etc/neutron/plugins/ml2/openvswitch_agent.ini.bak /etc/neutron/plugins/ml2/openvswitch_agent.ini# 完整配置
vim /etc/neutron/plugins/ml2/openvswitch_agent.ini
[DEFAULT]
[agent]
l2_population True
tunnel_types vxlan
prevent_arp_spoofing True
[dhcp]
[network_log]
[ovs]
local_ip 10.0.0.12
bridge_mappings physnet1:br-ens38
[securitygroup]
enable_security_group True
firewall_driver neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver配置nova文件识别neutron配置
vim /etc/nova/nova.conf[DEFAULT]
linuxnet_interface_driver nova.network.linux_net.LinuxOVSlnterfaceDriver
vif_plugging_is_fatal true
vif_pligging_timeout 300[neutron]
auth_url http://controller:5000
auth_type password
project_domain_name default
user_domain_name default
region_name RegionOne
project_name service
username neutron
password neutron重启nova服务识别网络配置
service nova-compute restart新建一个外部网络桥接
ovs-vsctl add-br br-ens38将外部网络桥接映射到网卡 这里绑定第二张网卡属于业务网卡
ovs-vsctl add-port br-ens38 ens38重启服务加载ovs配置
service neutron-openvswitch-agent restart8.3 校验neutron
校验命令
rootcontroller:~# openstack network agent list
----------------------------------------------------------------------------------------------------------------------------------
| ID | Agent Type | Host | Availability Zone | Alive | State | Binary |
----------------------------------------------------------------------------------------------------------------------------------
| 5695085f-b03f-4ff2-b13f-a8e59036ca15 | Open vSwitch agent | controller | None | :-) | UP | neutron-openvswitch-agent |
| 77f6b5e6-a761-49c6-8694-de4d3d52509f | Metadata agent | controller | None | :-) | UP | neutron-metadata-agent |
| 87139cbc-27ee-4885-807e-96800816adca | Open vSwitch agent | compute01 | None | :-) | UP | neutron-openvswitch-agent |
| 891696fa-01af-4fd9-87f0-ad3d432f05d0 | L3 agent | controller | nova | :-) | UP | neutron-l3-agent |
| 91959f9b-db89-4021-b55e-888f71edb0b3 | DHCP agent | controller | nova | :-) | UP | neutron-dhcp-agent |
| e5598aa0-e71c-4a74-a11f-b415a2e4fdbb | Open vSwitch agent | compute02 | None | :-) | UP | neutron-openvswitch-agent |
----------------------------------------------------------------------------------------------------------------------------------9. 配置dashboard仪表盘服务
安装服务
apt install -y openstack-dashboard配置local_settings.py文件
vim /etc/openstack-dashboard/local_settings.py# 配置仪表板以在控制器节点上使用OpenStack服务
OPENSTACK_HOST controller# 在Dashboard configuration部分中允许主机访问Dashboard
ALLOWED_HOSTS [*]# 配置memcached会话存储服务
SESSION_ENGINE django.contrib.sessions.backends.cacheCACHES {default: {BACKEND: django.core.cache.backends.memcached.MemcachedCache,LOCATION: controller:11211,}
}# 启用Identity API版本3
OPENSTACK_KEYSTONE_URL http://%s:5000/v3 % OPENSTACK_HOST# 启用对域的支持
OPENSTACK_KEYSTONE_MULTIDOMAIN_SUPPORT True# 配置API版本
OPENSTACK_API_VERSIONS {identity: 3,image: 2,volume: 3,
}# 将Default配置为通过仪表板创建的用户的默认域
OPENSTACK_KEYSTONE_DEFAULT_DOMAIN Default# 将用户配置为通过仪表板创建的用户的默认角色
OPENSTACK_KEYSTONE_DEFAULT_ROLE user# 启用卷备份
OPENSTACK_CINDER_FEATURES {enable_backup: True,
}# 配置时区
TIME_ZONE Asia/Shanghai重新加载web服务器配置
systemctl reload apache2浏览器访问http://conntroller/horizon 10. 部署配置cinder卷存储
10.1 控制节点配置
创建数据库与用户给予cinder组件使用
# 创建cinder数据库
CREATE DATABASE cinder;# 创建cinder用户
GRANT ALL PRIVILEGES ON cinder.* TO cinder% IDENTIFIED BY cinderang;创建cinder用户
openstack user create --domain default --password cinder cinder添加cinder用户到admin角色
openstack role add --project service --user cinder admin创建cinder服务实体
openstack service create --name cinderv3 --description OpenStack Block Storage volumev3创建cinder服务API端点
openstack endpoint create --region RegionOne volumev3 public http://controller:8776/v3/%\(project_id\)sopenstack endpoint create --region RegionOne volumev3 internal http://controller:8776/v3/%\(project_id\)sopenstack endpoint create --region RegionOne volumev3 admin http://controller:8776/v3/%\(project_id\)s安装cinder相关服务
apt install -y cinder-api cinder-scheduler配置cinder.conf文件
# 备份文件
cp /etc/cinder/cinder.conf{,.bak}# 过滤覆盖文件
grep -Ev ^$|# /etc/cinder/cinder.conf.bak /etc/cinder/cinder.conf# 完整配置
vim /etc/cinder/cinder.conf
[DEFAULT]
transport_url rabbit://openstack:000000controller
auth_strategy keystone
my_ip 10.0.0.10
[database]
connection mysqlpymysql://cinder:cinderangcontroller/cinder
[keystone_authtoken]
www_authenticate_uri http://controller:5000
auth_url http://controller:5000
memcached_servers controller:11211
auth_type password
project_domain_name default
user_domain_name default
project_name service
username cinder
password cinder
[oslo_concurrency]
lock_path /var/lib/cinder/tmp填充数据库
su -s /bin/sh -c cinder-manage db sync cinder配置nova服务可调用cinder服务
vim /etc/nova/nova.conf[cinder]
os_region_name RegionOne重启nova服务生效cinder服务
service nova-api restart重新启动块存储服务
service cinder-scheduler restart平滑重启apache服务识别cinder页面
service apache2 reload10.2 计算节点配置 compute01节点 安装支持的实用程序包
apt install -y lvm2 thin-provisioning-tools创建LVM物理卷 磁盘根据自己名称指定
pvcreate /dev/sdb创建LVM卷组 cinder-volumes
vgcreate cinder-volumes /dev/sdb修改lvm.conf文件 作用添加接受/dev/sdb设备并拒绝所有其他设备的筛选器
vim /etc/lvm/lvm.conf
devices {
...
filter [ a/sdb/, r/.*/]安装cinder软件包
apt install -y cinder-volume tgt配置cinder.conf配置文件
# 备份配置文件
cp /etc/cinder/cinder.conf{,.bak}# 过滤覆盖文件
grep -Ev ^$|# /etc/cinder/cinder.conf.bak /etc/cinder/cinder.conf# 完整配置文件
vim /etc/cinder/cinder.conf
[DEFAULT]
transport_url rabbit://openstack:000000controller
auth_strategy keystone
my_ip 10.0.0.11
enabled_backends lvm
glance_api_servers http://controller:9292
[database]
connection mysqlpymysql://cinder:cinderangcontroller/cinder
[keystone_authtoken]
www_authenticate_uri http://controller:5000
auth_url http://controller:5000
memcached_servers controller:11211
auth_type password
project_domain_name default
user_domain_name default
project_name service
username cinder
password cinder
[lvm]
volume_driver cinder.volume.drivers.lvm.LVMVolumeDriver
volume_group cinder-volumes
target_protocol iscsi
target_helper tgtadm
volume_backend_name lvm
[oslo_concurrency]
lock_path /var/lib/cinder/tmp指定卷路径
vim /etc/tgt/conf.d/tgt.conf
include /var/lib/cinder/volumes/*重新启动块存储卷服务包括其依赖项
service tgt restartservice cinder-volume restartcompute02节点 安装支持的实用程序包
apt install -y lvm2 thin-provisioning-tools创建LVM物理卷 磁盘根据自己名称指定
pvcreate /dev/sdb创建LVM卷组 cinder-volumes
vgcreate cinder-volumes /dev/sdb修改lvm.conf文件 作用添加接受/dev/sdb设备并拒绝所有其他设备的筛选器
vim /etc/lvm/lvm.conf
devices {
...
filter [ a/sdb/, r/.*/]安装cinder软件包
apt install -y cinder-volume tgt配置cinder.conf配置文件
# 备份配置文件
cp /etc/cinder/cinder.conf{,.bak}# 过滤覆盖文件
grep -Ev ^$|# /etc/cinder/cinder.conf.bak /etc/cinder/cinder.conf# 完整配置文件
vim /etc/cinder/cinder.conf
[DEFAULT]
transport_url rabbit://openstack:000000controller
auth_strategy keystone
my_ip 10.0.0.12
enabled_backends lvm
glance_api_servers http://controller:9292
[database]
connection mysqlpymysql://cinder:cinderangcontroller/cinder
[keystone_authtoken]
www_authenticate_uri http://controller:5000
auth_url http://controller:5000
memcached_servers controller:11211
auth_type password
project_domain_name default
user_domain_name default
project_name service
username cinder
password cinder
[lvm]
volume_driver cinder.volume.drivers.lvm.LVMVolumeDriver
volume_group cinder-volumes
target_protocol iscsi
target_helper tgtadm
volume_backend_name lvm
[oslo_concurrency]
lock_path /var/lib/cinder/tmp指定卷路径
vim /etc/tgt/conf.d/tgt.conf
include /var/lib/cinder/volumes/*重新启动块存储卷服务包括其依赖项
service tgt restartservice cinder-volume restart10.3 校验cinder
校验命令
rootcontroller:~# openstack volume service list
-----------------------------------------------------------------------------------
| Binary | Host | Zone | Status | State | Updated At |
-----------------------------------------------------------------------------------
| cinder-scheduler | controller | nova | enabled | up | 2022-09-29T07:58:33.000000 |
| cinder-volume | compute01lvm | nova | enabled | up | 2022-09-29T07:58:29.000000 |
| cinder-volume | compute02lvm | nova | enabled | up | 2022-09-29T07:58:34.000000 |11. 运维实战 控制节点操作
11.1 加载openstack环境变量
source /etc/keystone/admin-openrc.sh11.2 创建路由器
openstack router create Ext-Router11.3 创建Vxlan网络
创建vxlan网络
openstack network create --provider-network-type vxlan Intnal创建vxlan子网
openstack subnet create Intsubnal --network Intnal --subnet-range 166.66.66.0/24 --gateway 166.66.66.1 --dns-nameserver 114.114.114.11411.4 将内部网络添加到路由器
添加命令
openstack router add subnet Ext-Router Intsubnal11.5 创建Flat网络
创建flat网络
openstack network create --provider-physical-network physnet1 --provider-network-type flat --external Extnal创建flat子网
openstack subnet create Extsubnal --network Extnal --subnet-range 10.0.0.0/24 --allocation-pool start10.0.0.30,end10.0.0.200 --gateway 10.0.0.254 --dns-nameserver 114.114.114.114 --no-dhcp11.6 设置路由器网关接口
openstack router set Ext-Router --external-gateway Extnal11.7 开放安全组
# 开放icmp协议
openstack security group rule create --proto icmp default# 开放22端口
openstack security group rule create --proto tcp --dst-port 22:22 default# 查看安全组规则
openstack security group rule list11.8 上传镜像
openstack image create cirros04 --disk-format qcow2 --file cirros-0.4.0-x86_64-disk.img11.9 创建云主机
创建ssh-key密钥
ssh-keygen -N 创建密钥
openstack keypair create --public-key ~/.ssh/id_rsa.pub mykey创建云主机类型
openstack flavor create --vcpus 1 --ram 512 --disk 1 C1-512MB-1G创建云主机
openstack server create --flavor C1-512MB-1G --image cirros04 --security-group default --nic net-id$(vxlan网络id) --key-name mykey vm01分配浮动地址
openstack floating ip create Extnal将分配的浮动IP绑定云主机
openstack server add floating ip vm01 $(分配出的地址)VNC查看实例
openstack console url show vm0111.10 创建卷类型
openstack volume type create lvm11.11 卷类型添加元数据
cinder --os-username admin --os-tenant-name admin type-key lvm set volume_backend_namelvm11.12 查看卷类型
openstack volume type list11.13 创建卷
指定lvm卷类型创建卷
openstack volume create lvm01 --type lvm --size 111.14 卷绑定云主机
将卷绑定云主机
nova volume-attach vm01 卷ID二. Ceph集群部署
1. 环境准备
主机名IP磁盘CPUmemorynode110.0.0.18sda100Gsdb50Gsdc50G2C4Gnode210.0.0.19sda100G sdb50Gsdc50G2C4Gnode310.0.0.20sda100G sdb50Gsdc50G2C4G
操作系统虚拟化工具Ubuntu22.04VMware15
1.1 配置地址
node1节点
cat /etc/netplan/00-installer-config.yaml EOF
# This is the network config written by subiquity
network:ethernets:ens33:dhcp4: falseaddresses: [10.0.0.18/24]gateway4: 10.0.0.254version: 2
EOF# 生效网络
netplan applynode2节点
cat /etc/netplan/00-installer-config.yaml EOF
# This is the network config written by subiquity
network:ethernets:ens33:dhcp4: falseaddresses: [10.0.0.19/24]gateway4: 10.0.0.254version: 2
EOF# 生效网络
netplan applynode3节点
cat /etc/netplan/00-installer-config.yaml EOF
# This is the network config written by subiquity
network:ethernets:ens33:dhcp4: falseaddresses: [10.0.0.20/24]gateway4: 10.0.0.254version: 2
EOF# 生效网络
netplan apply1.2 更改主机名
node1节点
hostnamectl set-hostname node1# 切换窗口
bashnode2节点
hostnamectl set-hostname node2# 切换窗口
bashnode3节点
hostnamectl set-hostname node3# 切换窗口
bash2. 配置hosts解析所有节点 cat /etc/hosts EOF
10.0.0.18 node1
10.0.0.19 node2
10.0.0.20 node3
EOF3. 制作离线源所有节点 解压离线包并配置本地仓库 tar zxvf ceph_quincy.tar.gz -C /opt/cp /etc/apt/sources.list{,.bak}cat /etc/apt/sources.list EOF
deb [trustedyes] file:// /opt/ceph_quincy/debs/
EOFapt-get clean all
apt-get update4. 配置时间同步 所有节点更改时区 # 可配置开启
timedatectl set-ntp true# 配置上海时区
timedatectl set-timezone Asia/Shanghai# 系统时钟与硬件时钟同步
hwclock --systohcnode1节点 # 安装服务
apt install -y chrony# 配置文件
vim /etc/chrony/chrony.conf
20 server controller iburst maxsources 2
61 allow all
63 local stratum 10# 重启服务
systemctl restart chronydnode2、node3节点 # 安装服务
apt install -y chrony# 配置文件
vim /etc/chrony/chrony.conf
20 pool controller iburst maxsources 4# 重启服务
systemctl restart chronyd5. 安装docker(所有节点) apt -y install docker-ce6. 安装cephadm(node1) apt install -y cephadm7. 导入ceph镜像(所有节点) 将准备好的离线镜像全部导入 docker load -i cephadm_images_v17.tar7.1 搭建制作本地仓库(node1) 启动仓库镜像 # 导入镜像
docker load -i registry.tar# 启动
docker run -d --name registry -p 5000:5000 --restart always 3a0f7b0a13ef配置仓库地址 cat /etc/docker/daemon.json EOF
{
insecure-registries:[10.0.0.18:5000]
}
EOFsystemctl daemon-reload
systemctl restart docker打地址标签 docker tag 0912465dcea5 10.0.0.18:5000/ceph:v17推入仓库 docker push 10.0.0.18:5000/ceph:v177.2 配置私有仓库 node2、node3节点配置私有仓库 cat /etc/docker/daemon.json EOF
{
insecure-registries:[10.0.0.18:5000]
}
EOFsystemctl daemon-reload
systemctl restart docker8. 引导集群(node1) 初始化mon节点 mkdir -p /etc/cephcephadm --image 10.0.0.18:5000/ceph:v17 bootstrap --mon-ip 10.0.0.18 --initial-dashboard-user admin --initial-dashboard-password 000000 --skip-pullps:
# 要部署其他监视器
ceph orch apply mon test01,test02,test03# 删除集群
cephadm rm-cluster --fsid d92b85c0-3ecd-11ed-a617-3f7cf3e2d6d8 --force9. 安装ceph-common工具(node1) 安装服务 apt install -y ceph-common10. 添加主机到集群(node1) 传输ceph密钥 ssh-copy-id -f -i /etc/ceph/ceph.pub node2ssh-copy-id -f -i /etc/ceph/ceph.pub node3集群机器发现 ceph orch host add node2ceph orch host add node311. 部署OSD
存储数据 node1机器 # 查看可用的磁盘设备
ceph orch device ls# 添加到ceph集群中,在未使用的设备上自动创建osd
ceph orch apply osd --all-available-devicesPS:
# 从特定主机上的特定设备创建OSD
ceph orch daemon add osd node1:/dev/sdb
ceph orch daemon add osd node2:/dev/sdb
ceph orch daemon add osd node3:/dev/sdb# 查看osd磁盘
ceph -sceph df12. 访问仪表盘查看状态
访问https://10.0.0.18:8443/ 访问https://10.0.0.18:3000/ 三. OpenStack对接Ceph平台
1. 创建后端需要的存储池 node1节点操作
1.1 cinder卷的存储池
ceph osd pool create volumes 321.2 glance存储池
ceph osd pool create images 321.3 备份存储池
ceph osd pool create backups 321.4 创建实例存储池
ceph osd pool create vms 322. 创建后端用户
2.1 创建密钥 node1节点操作 切换到ceph目录
cd /etc/ceph/在ceph上创建cinder、glance、cinder-backup、nova用户创建密钥允许访问使用Ceph存储池
2.1.1 创建用户client.cinder
对volumes存储池有rwx权限对vms存储池有rwx权限对images池有rx权限
ceph auth get-or-create client.cinder mon allow r osd allow class-read object_prefix rbd_children,allow rwx poolvolumes,allow rwx poolvms,allow rx poolimages# class-readx的子集授予用户调用类读取方法的能力# object_prefix 通过对象名称前缀。下例将访问限制为任何池中名称仅以 rbd_children 为开头的对象。2.1.2 创建用户client.glance
对images存储池有rwx权限
ceph auth get-or-create client.glance mon allow r osd allow class-read object_prefix rbd_children,allow rwx poolimages2.1.3 创建用户client.cinder-backup
对backups存储池有rwx权限
ceph auth get-or-create client.cinder-backup mon profile rbd osd profile rbd poolbackups# 使用 rbd profile 为新的 cinder-backup 用户帐户定义访问权限。然后客户端应用使用这一帐户基于块来访问利用了 RADOS 块设备的 Ceph 存储。2.2 创建存放目录
controller节点
mkdir /etc/ceph/compute01节点
mkdir /etc/ceph/compute02节点
mkdir /etc/ceph/2.3 导出密钥 node1节点 导出glance密钥
ceph auth get client.glance -o ceph.client.glance.keyring导出cinder密钥
ceph auth get client.cinder -o ceph.client.cinder.keyring导出cinder-backup密钥
ceph auth get client.cinder-backup -o ceph.client.cinder-backup.keyring2.4 拷贝密钥 node1节点操作
2.4.1 控制节点准备
拷贝glance密钥
scp ceph.client.glance.keyring rootcontroller:/etc/ceph/拷贝cinder密钥
scp ceph.client.cinder.keyring rootcontroller:/etc/ceph/拷贝ceph集群认证配置文件
scp ceph.conf rootcontroller:/etc/ceph/2.4.2 计算节点准备
拷贝cinder密钥
scp ceph.client.cinder.keyring rootcompute01:/etc/ceph/scp ceph.client.cinder.keyring rootcompute02:/etc/ceph/拷贝cinder-backup密钥backup服务节点
scp ceph.client.cinder-backup.keyring rootcompute01:/etc/ceph/scp ceph.client.cinder-backup.keyring rootcompute02:/etc/ceph/拷贝ceph集群认证配置文件
scp ceph.conf rootcompute01:/etc/ceph/scp ceph.conf rootcompute02:/etc/ceph/3. 计算节点添加libvirt密钥
3.1 compute01添加密钥
生成密钥PS注意如果有多个计算节点它们的UUID必须一致
cd /etc/ceph/UUID$(uuidgen)cat secret.xml EOF
secret ephemeralno privatenouuid$UUID/uuidusage typecephnameclient.cinder secret/name/usage
/secret
EOF执行命令写入secret
[rootcompute01 ~]# virsh secret-define --file secret.xml
Secret bf168fa8-8d5b-4991-ba4c-12ae622a98b1 created加入key
# 将key值复制出来
[rootcompute01 ~]# cat ceph.client.cinder.keyring
AQALyS1jHz4dDRAAEmtc8JlXWyzxmCx5vobZg[rootcompute01 ~]# virsh secret-set-value --secret ${UUID} --base64 $(cat ceph.client.cinder.keyring | grep key | awk -F {print $3})查看添加后端密钥
virsh secret-list3.2 compute02添加密钥
生成密钥PS注意如果有多个计算节点它们的UUID必须一致
cd /etc/ceph/UUIDbf168fa8-8d5b-4991-ba4c-12ae622a98b1cat secret.xml EOF
secret ephemeralno privatenouuid$UUID/uuidusage typecephnameclient.cinder secret/name/usage
/secret
EOF执行命令写入secret
[rootcompute02 ~]# virsh secret-define --file secret.xml
Secret bf168fa8-8d5b-4991-ba4c-12ae622a98b1 created加入key
# 将key值复制出来
[rootcompute02 ~]# cat ceph.client.cinder.keyring
AQALyS1jHz4dDRAAEmtc8JlXWyzxmCx5vobZg[rootcompute02 ~]# virsh secret-set-value --secret ${UUID} --base64 $(cat ceph.client.cinder.keyring | grep key | awk -F {print $3})# 忽略报错信息查看添加后端密钥
virsh secret-list4. 安装ceph客户端 主要作用是OpenStack可调用Ceph资源 controller节点
apt install -y ceph-commoncompute01节点
apt install -y ceph-commoncompute02节点
apt install -y ceph-common5. 配置glance后端存储 controller节点 更改glance密钥属性
chown glance.glance /etc/ceph/ceph.client.glance.keyring修改配置文件
vim /etc/glance/glance-api.conf
[glance_store]
#stores file,http
#default_store file
#filesystem_store_datadir /var/lib/glance/images/
stores rbd,file,http
default_store rbd
rbd_store_pool images
rbd_store_user glance
rbd_store_ceph_conf /etc/ceph/ceph.conf
rbd_store_chunk_size 8安装缺失aws的模块
apt install -y python3-boto3重启生效ceph配置
service glance-api restart上传镜像
openstack image create cirros04_v1 --disk-format qcow2 --file cirros-0.4.0-x86_64-disk.img到node1节点验证镜像
rbd images ls6. 配置cinder后端存储
更改cinder密钥属性controller、compute01、compute02节点
chown cinder.cinder /etc/ceph/ceph.client.cinder.keyring修改配置文件controller节点
vim /etc/cinder/cinder.conf
[DEFAULT]
# 指定存储类型否则在创建卷时类型为 __DEFAULT__
default_volume_type ceph# 重启服务生效配置
service cinder-scheduler restart修改配置文件compute01、compute02存储节点
vim /etc/cinder/cinder.conf
[DEFAULT]
enabled_backends ceph,lvm[ceph]
volume_driver cinder.volume.drivers.rbd.RBDDriver
rbd_pool volumes
rbd_ceph_conf /etc/ceph/ceph.conf
rbd_flatten_volume_from_snapshot false
rbd_max_clone_depth 5
rbd_store_chunk_size 4
rados_connect_timeout -1
glance_api_version 2
rbd_user cinder
rbd_secret_uuid bf168fa8-8d5b-4991-ba4c-12ae622a98b1
volume_backend_name ceph# 重启服务生效配置
service cinder-volume restart创建卷类型controller节点
openstack volume type create ceph设置卷类型元数据controller节点
cinder --os-username admin --os-tenant-name admin type-key ceph set volume_backend_nameceph查看存储类型controller节点
openstack volume type list创建卷测试controller节点
openstack volume create ceph01 --type ceph --size 1查看volumes存储池是否存在卷
rbd ls volumes7. 配置卷备份 compute01、compute02节点 安装服务
apt install cinder-backup -y更改密钥属性
chown cinder.cinder /etc/ceph/ceph.client.cinder-backup.keyring修改配置文件
vim /etc/cinder/cinder.conf
[DEFAULT]
backup_driver cinder.backup.drivers.ceph.CephBackupDriver
backup_ceph_conf/etc/ceph/ceph.conf
backup_ceph_user cinder-backup
backup_ceph_chunk_size 4194304
backup_ceph_pool backups
backup_ceph_stripe_unit 0
backup_ceph_stripe_count 0
restore_discard_excess_bytes true重启生效配置
service cinder-backup restart创建卷备份controller节点
openstack volume backup create --name ceph_backup ceph01 验证卷备份node1节点
rbd ls backups8. 配置nova集成ceph compute01、compute02节点 修改配置文件
vim /etc/nova/nova.conf
[DEFAULT]
live_migration_flag VIR_MIGRATE_UNDEFINE_SOURCE,VIR_MIGRATE_PEER2PEER,VIR_MIGRATE_LIVE[libvirt]
images_type rbd
images_rbd_pool vms
images_rbd_ceph_conf /etc/ceph/ceph.conf
rbd_user cinder
rbd_secret_uuid bf168fa8-8d5b-4991-ba4c-12ae622a98b1安装qemu支持rbd
apt install -y qemu-block-extra重启nova服务生效配置
service nova-compute restart创建实例测试controller节点
openstack server create --flavor C1-512MB-1G --image cirros04_v1 --security-group default --nic net-id$(vxlan网络id) --key-name mykey vm02# 安全组对应admin项目ID验证是否到ceph中的vms存储池
rbd ls vms8.1 热迁移配置 compute01、compute02节点 配置监听地址
vim /etc/libvirt/libvirtd.conf
listen_tls 0
listen_tcp 1
tcp_port 16509
listen_addr 10.0.0.12 # 注意自己的主机地址
auth_tcp none开启监听地址
vim /etc/default/libvirtd
LIBVIRTD_ARGS--listen屏蔽libvirtd服务
systemctl mask libvirtd.socket libvirtd-ro.socket libvirtd-admin.socket libvirtd-tls.socket libvirtd-tcp.socket重启libvirtd生效配置
service libvirtd restart重启计算节点nova服务
service nova-compute restart测试是否能互相通信连接 互通测试再进行热迁移 compute01连接compute02
virsh -c qemutcp://compute02/systemcompute02连接compute01
virsh -c qemutcp://compute01/system查看云主机
openstack server list查看需要迁移的云主机详细信息
openstack server show fdb31a02-9c44-481b-9d22-224c776e2304热迁移到另一个计算节点
nova live-migration fdb31a02-9c44-481b-9d22-224c776e2304 compute01
