本文分享openstack的Cinder存储服务组件，cinder服务可以提供云磁盘（卷），类似阿里云云盘

----------------------- 完美的分隔线  -----------------------------

# openstack-Mitaka-cinder块存储服务中文文档
# https://docs.openstack.org/mitaka/zh_CN/install-guide-rdo/cinder.html

# openstack-rocky版本Cinder官方安装文档
# https://docs.openstack.org/cinder/rocky/install/

9.0.Cinder概述

OpenStack块存储服务(cinder)为虚拟机添加持久的存储，块存储提供一个基础设施为了管理卷，以及和OpenStack计算服务交互，为实例提供卷。此服务也会激活管理卷的快照和卷类型的功能。

块存储服务通常包含下列组件：

1）cinder-api
接受API请求，并将其路由到``cinder-volume``执行。
2）cinder-volume
与块存储服务和例如``cinder-scheduler``的进程进行直接交互。它也可以与这些进程通过一个消息队列进行交互。``cinder-volume``服务响应送到块存储服务的读写请求来维持状态。它也可以和多种存储提供者在驱动架构下进行交互。
3）cinder-scheduler守护进程
选择最优存储提供节点来创建卷。其与``nova-scheduler``组件类似。
4）cinder-backup守护进程
``cinder-backup``服务提供任何种类备份卷到一个备份存储提供者。就像``cinder-volume``服务，它与多种存储提供者在驱动架构下进行交互。
5）消息队列
在块存储的进程之间路由信息。

9.1.在控制节点安装cinder存储服务

# Install and configure controller node
https://docs.openstack.org/cinder/rocky/install/cinder-controller-install-rdo.html

1）创建cinder数据库

# 创建相关数据库，授权访问用户

mysql -u root -p123456
----------------------------------------
CREATE DATABASE cinder;
GRANT ALL PRIVILEGES ON cinder.* TO 'cinder'@'localhost' IDENTIFIED BY 'cinder';
GRANT ALL PRIVILEGES ON cinder.* TO 'cinder'@'%' IDENTIFIED BY 'cinder';
flush privileges;
show databases;
select user,host from mysql.user;
exit
----------------------------------------

2）在keystone上面注册cinder服务（创建服务证书）

# 在keystone上创建cinder用户

cd /server/tools
source keystone-admin-pass.sh
openstack user create --domain default --password=cinder cinder
openstack user list

# 在keystone上将cinder用户配置为admin角色并添加进service项目，以下命令无输出

openstack role add --project service --user cinder admin

# 创建cinder服务的实体

openstack service create --name cinderv2 --description "OpenStack Block Storage" volumev2
openstack service create --name cinderv3 --description "OpenStack Block Storage" volumev3
openstack service list

# 创建cinder服务的API端点（endpoint）

openstack endpoint create --region RegionOne volumev2 public http://controller:8776/v2/%\(project_id\)s
openstack endpoint create --region RegionOne volumev2 internal http://controller:8776/v2/%\(project_id\)s
openstack endpoint create --region RegionOne volumev2 admin http://controller:8776/v2/%\(project_id\)s

openstack endpoint create --region RegionOne volumev3 public http://controller:8776/v3/%\(project_id\)s
openstack endpoint create --region RegionOne volumev3 internal http://controller:8776/v3/%\(project_id\)s
openstack endpoint create --region RegionOne volumev3 admin http://controller:8776/v3/%\(project_id\)s
openstack endpoint list

# 实例演示：

[root@openstack01 tools]# openstack user create --domain default --password=cinder cinder
+---------------------+----------------------------------+
| Field               | Value                            |
+---------------------+----------------------------------+
| domain_id           | default                          |
| enabled             | True                             |
| id                  | a1a276d12c4e442ebc9250e4d4148166 |
| name                | cinder                           |
| options             | {}                               |
| password_expires_at | None                             |
+---------------------+----------------------------------+
[root@openstack01 tools]# openstack user list
+----------------------------------+-----------+
| ID                               | Name      |
+----------------------------------+-----------+
| 26f88ba142d04735936d09caa7c76284 | placement |
| 82a27e65ca644a5eadcd54ff44e5e05b | glance    |
| a1a276d12c4e442ebc9250e4d4148166 | cinder    |
| cbb2b3830a8f44bc837230bca27ae563 | myuser    |
| cc55913a3da44a38939cdc7a2ec764cc | nova      |
| dd35b7396aa94342a01c807aaa707d21 | neutron   |
| e5dbfc8b394c41679fd5ce229cdd6ed3 | admin     |
+----------------------------------+-----------+
[root@openstack01 tools]# openstack role add --project service --user cinder admin
[root@openstack01 tools]# openstack service create --name cinderv2 --description "OpenStack Block Storage" volumev2
+-------------+----------------------------------+
| Field       | Value                            |
+-------------+----------------------------------+
| description | OpenStack Block Storage          |
| enabled     | True                             |
| id          | 5342850f7fd04f999ab6c6f787baa610 |
| name        | cinderv2                         |
| type        | volumev2                         |
+-------------+----------------------------------+
[root@openstack01 tools]# openstack service create --name cinderv3 --description "OpenStack Block Storage" volumev3
+-------------+----------------------------------+
| Field       | Value                            |
+-------------+----------------------------------+
| description | OpenStack Block Storage          |
| enabled     | True                             |
| id          | cba2b834789f49f5a9fdac76c09c5fae |
| name        | cinderv3                         |
| type        | volumev3                         |
+-------------+----------------------------------+
[root@openstack01 tools]# openstack service list
+----------------------------------+-----------+-----------+
| ID                               | Name      | Type      |
+----------------------------------+-----------+-----------+
| 5342850f7fd04f999ab6c6f787baa610 | cinderv2  | volumev2  |
| 63c882889b204d81a9867f9b7c0ba7aa | keystone  | identity  |
| 6c31f22e259b460fa0168ac206265c30 | glance    | image     |
| 854ca66666c64e2fbeff1e9c5cc1c4df | nova      | compute   |
| 90b5d791df5e4634848c00ba35390865 | neutron   | network   |
| a79d818312b34c4c8879d7dbbd41a78c | placement | placement |
| cba2b834789f49f5a9fdac76c09c5fae | cinderv3  | volumev3  |
+----------------------------------+-----------+-----------+
[root@openstack01 tools]# openstack endpoint create --region RegionOne volumev2 public http://controller:8776/v2/%\(project_id\)s
+--------------+------------------------------------------+
| Field        | Value                                    |
+--------------+------------------------------------------+
| enabled      | True                                     |
| id           | 1412aab234bf4793bbb55bf938dfabe9         |
| interface    | public                                   |
| region       | RegionOne                                |
| region_id    | RegionOne                                |
| service_id   | 5342850f7fd04f999ab6c6f787baa610         |
| service_name | cinderv2                                 |
| service_type | volumev2                                 |
| url          | http://controller:8776/v2/%(project_id)s |
+--------------+------------------------------------------+
[root@openstack01 tools]# openstack endpoint create --region RegionOne volumev2 internal http://controller:8776/v2/%\(project_id\)s
+--------------+------------------------------------------+
| Field        | Value                                    |
+--------------+------------------------------------------+
| enabled      | True                                     |
| id           | 5421883053d84778b222ed24b424ad71         |
| interface    | internal                                 |
| region       | RegionOne                                |
| region_id    | RegionOne                                |
| service_id   | 5342850f7fd04f999ab6c6f787baa610         |
| service_name | cinderv2                                 |
| service_type | volumev2                                 |
| url          | http://controller:8776/v2/%(project_id)s |
+--------------+------------------------------------------+
[root@openstack01 tools]# openstack endpoint create --region RegionOne volumev2 admin http://controller:8776/v2/%\(project_id\)s
+--------------+------------------------------------------+
| Field        | Value                                    |
+--------------+------------------------------------------+
| enabled      | True                                     |
| id           | 4947b9f1a61f4e5c858e1a2d6dd426eb         |
| interface    | admin                                    |
| region       | RegionOne                                |
| region_id    | RegionOne                                |
| service_id   | 5342850f7fd04f999ab6c6f787baa610         |
| service_name | cinderv2                                 |
| service_type | volumev2                                 |
| url          | http://controller:8776/v2/%(project_id)s |
+--------------+------------------------------------------+
[root@openstack01 tools]# openstack endpoint create --region RegionOne volumev3 public http://controller:8776/v3/%\(project_id\)s
+--------------+------------------------------------------+
| Field        | Value                                    |
+--------------+------------------------------------------+
| enabled      | True                                     |
| id           | 594078e79fd44a8383a9dba42931ff06         |
| interface    | public                                   |
| region       | RegionOne                                |
| region_id    | RegionOne                                |
| service_id   | cba2b834789f49f5a9fdac76c09c5fae         |
| service_name | cinderv3                                 |
| service_type | volumev3                                 |
| url          | http://controller:8776/v3/%(project_id)s |
+--------------+------------------------------------------+
[root@openstack01 tools]# openstack endpoint create --region RegionOne volumev3 internal http://controller:8776/v3/%\(project_id\)s
+--------------+------------------------------------------+
| Field        | Value                                    |
+--------------+------------------------------------------+
| enabled      | True                                     |
| id           | 67735ee3f61d48aea3dc3338d67a1ca8         |
| interface    | internal                                 |
| region       | RegionOne                                |
| region_id    | RegionOne                                |
| service_id   | cba2b834789f49f5a9fdac76c09c5fae         |
| service_name | cinderv3                                 |
| service_type | volumev3                                 |
| url          | http://controller:8776/v3/%(project_id)s |
+--------------+------------------------------------------+
[root@openstack01 tools]# openstack endpoint create --region RegionOne volumev3 admin http://controller:8776/v3/%\(project_id\)s
+--------------+------------------------------------------+
| Field        | Value                                    |
+--------------+------------------------------------------+
| enabled      | True                                     |
| id           | fed47d708ea2407bb2a986a4796719b2         |
| interface    | admin                                    |
| region       | RegionOne                                |
| region_id    | RegionOne                                |
| service_id   | cba2b834789f49f5a9fdac76c09c5fae         |
| service_name | cinderv3                                 |
| service_type | volumev3                                 |
| url          | http://controller:8776/v3/%(project_id)s |
+--------------+------------------------------------------+
[root@openstack01 tools]# openstack endpoint list
+----------------------------------+-----------+--------------+--------------+---------+-----------+------------------------------------------+
| ID                               | Region    | Service Name | Service Type | Enabled | Interface | URL                                      |
+----------------------------------+-----------+--------------+--------------+---------+-----------+------------------------------------------+
| 022711a6476648bda1446ecb7668f315 | RegionOne | placement    | placement    | True    | public    | http://controller:8778                   |
| 1291aa2f71104ce69f9b05905fbc2c8a | RegionOne | placement    | placement    | True    | admin     | http://controller:8778                   |
| 1412aab234bf4793bbb55bf938dfabe9 | RegionOne | cinderv2     | volumev2     | True    | public    | http://controller:8776/v2/%(project_id)s |
| 1cba9e89dc91422390a5b987dbeffdb6 | RegionOne | neutron      | network      | True    | internal  | http://controller:9696                   |
| 2bcda9f77cdb4c06be6f35a3c3312e3d | RegionOne | neutron      | network      | True    | admin     | http://controller:9696                   |
| 3f293d128470468683d5f82a66301232 | RegionOne | placement    | placement    | True    | internal  | http://controller:8778                   |
| 43960ef2a79a45d49bfd22a2dbf4c2ce | RegionOne | nova         | compute      | True    | internal  | http://controller:8774/v2.1              |
| 4947b9f1a61f4e5c858e1a2d6dd426eb | RegionOne | cinderv2     | volumev2     | True    | admin     | http://controller:8776/v2/%(project_id)s |
| 5421883053d84778b222ed24b424ad71 | RegionOne | cinderv2     | volumev2     | True    | internal  | http://controller:8776/v2/%(project_id)s |
| 594078e79fd44a8383a9dba42931ff06 | RegionOne | cinderv3     | volumev3     | True    | public    | http://controller:8776/v3/%(project_id)s |
| 67735ee3f61d48aea3dc3338d67a1ca8 | RegionOne | cinderv3     | volumev3     | True    | internal  | http://controller:8776/v3/%(project_id)s |
| 7129fffdb2614227aca641b10635efdf | RegionOne | nova         | compute      | True    | admin     | http://controller:8774/v2.1              |
| 7226f8f9c7164214b815821b77ae3ce6 | RegionOne | glance       | image        | True    | admin     | http://controller:9292                   |
| 756084d018c948039d2ae55b13fc7d4a | RegionOne | glance       | image        | True    | internal  | http://controller:9292                   |
| 7f0461c745b340ef83372059782d22ee | RegionOne | nova         | compute      | True    | public    | http://controller:8774/v2.1              |
| b8dabe6c548e435eb2b1f7efe3b23236 | RegionOne | keystone     | identity     | True    | admin     | http://controller:5000/v3/               |
| eb72eb6ea51842feb67ba5849beea48c | RegionOne | keystone     | identity     | True    | internal  | http://controller:5000/v3/               |
| ed17939d7623456bb203bb7197fc16c4 | RegionOne | neutron      | network      | True    | public    | http://controller:9696                   |
| f13c44af4e8d45d5b0229ea870f2c24f | RegionOne | glance       | image        | True    | public    | http://controller:9292                   |
| f172f6159ad34fbd8e10e0d42828d8cd | RegionOne | keystone     | identity     | True    | public    | http://controller:5000/v3/               |
| fed47d708ea2407bb2a986a4796719b2 | RegionOne | cinderv3     | volumev3     | True    | admin     | http://controller:8776/v3/%(project_id)s |
+----------------------------------+-----------+--------------+--------------+---------+-----------+------------------------------------------+

# ok

3）安装cinder相关软件包

yum install openstack-cinder -y

4）快速修改cinder配置

openstack-config --set  /etc/cinder/cinder.conf database connection  mysql+pymysql://cinder:cinder@controller/cinder
openstack-config --set  /etc/cinder/cinder.conf DEFAULT transport_url  rabbit://openstack:openstack@controller
openstack-config --set  /etc/cinder/cinder.conf DEFAULT auth_strategy  keystone 
openstack-config --set  /etc/cinder/cinder.conf keystone_authtoken auth_uri  http://controller:5000
openstack-config --set  /etc/cinder/cinder.conf keystone_authtoken auth_url  http://controller:5000
openstack-config --set  /etc/cinder/cinder.conf keystone_authtoken memcached_servers  controller:11211
openstack-config --set  /etc/cinder/cinder.conf keystone_authtoken auth_type  password
openstack-config --set  /etc/cinder/cinder.conf keystone_authtoken project_domain_name  default 
openstack-config --set  /etc/cinder/cinder.conf keystone_authtoken user_domain_name  default
openstack-config --set  /etc/cinder/cinder.conf keystone_authtoken project_name  service 
openstack-config --set  /etc/cinder/cinder.conf keystone_authtoken username  cinder
openstack-config --set  /etc/cinder/cinder.conf keystone_authtoken password  cinder
openstack-config --set  /etc/cinder/cinder.conf DEFAULT my_ip 192.168.1.81
openstack-config --set  /etc/cinder/cinder.conf oslo_concurrency lock_path  /var/lib/nova/tmp

# 检查生效的cinder配置

egrep -v "^#|^$" /etc/cinder/cinder.conf
grep '^[a-z]' /etc/cinder/cinder.conf

# 实例演示：

---------------------------------------------
[root@openstack01 tools]# egrep -v "^#|^$" /etc/cinder/cinder.conf
[DEFAULT]
transport_url = rabbit://openstack:openstack@controller
auth_strategy = keystone
my_ip = 192.168.1.81
[backend]
[backend_defaults]
[barbican]
[brcd_fabric_example]
[cisco_fabric_example]
[coordination]
[cors]
[database]
connection = mysql+pymysql://cinder:cinder@controller/cinder
[fc-zone-manager]
[healthcheck]
[key_manager]
[keystone_authtoken]
auth_uri = http://controller:5000
auth_url = http://controller:5000
memcached_servers = controller:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = cinder
password = cinder
[matchmaker_redis]
[nova]
[oslo_concurrency]
lock_path = /var/lib/nova/tmp
[oslo_messaging_amqp]
[oslo_messaging_kafka]
[oslo_messaging_notifications]
[oslo_messaging_rabbit]
[oslo_messaging_zmq]
[oslo_middleware]
[oslo_policy]
[oslo_reports]
[oslo_versionedobjects]
[profiler]
[sample_remote_file_source]
[service_user]
[ssl]
[vault]
---------------------------------------------
[root@openstack01 tools]# grep '^[a-z]' /etc/cinder/cinder.conf
transport_url = rabbit://openstack:openstack@controller
auth_strategy = keystone
my_ip = 192.168.1.81
connection = mysql+pymysql://cinder:cinder@controller/cinder
auth_uri = http://controller:5000
auth_url = http://controller:5000
memcached_servers = controller:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = cinder
password = cinder
lock_path = /var/lib/nova/tmp
---------------------------------------------

# ok

5）同步cinder数据库

# 有35张表

su -s /bin/sh -c "cinder-manage db sync" cinder

# 验证数据库

mysql -h192.168.1.81 -ucinder -pcinder -e "use cinder;show tables;"

# 实例演示：

[root@openstack01 tools]# mysql -h192.168.1.81 -ucinder -pcinder -e "use cinder;show tables;"
+----------------------------+
| Tables_in_cinder           |
+----------------------------+
| attachment_specs           |
| backup_metadata            |
| backups                    |
| cgsnapshots                |
| clusters                   |
| consistencygroups          |
| driver_initiator_data      |
| encryption                 |
| group_snapshots            |
| group_type_projects        |
| group_type_specs           |
| group_types                |
| group_volume_type_mapping  |
| groups                     |
| image_volume_cache_entries |
| messages                   |
| migrate_version            |
| quality_of_service_specs   |
| quota_classes              |
| quota_usages               |
| quotas                     |
| reservations               |
| services                   |
| snapshot_metadata          |
| snapshots                  |
| transfers                  |
| volume_admin_metadata      |
| volume_attachment          |
| volume_glance_metadata     |
| volume_metadata            |
| volume_type_extra_specs    |
| volume_type_projects       |
| volume_types               |
| volumes                    |
| workers                    |
+----------------------------+

# ok

6）修改nova配置文件

# 配置nova调用cinder服务

openstack-config --set  /etc/nova/nova.conf cinder os_region_name  RegionOne

# 检查生效的nova配置

grep '^[a-z]' /etc/nova/nova.conf |grep os_region_name

7）重启nova-api服务

systemctl restart openstack-nova-api.service
systemctl status openstack-nova-api.service

8）启动cinder存储服务

# 需要启动2个服务

systemctl start openstack-cinder-api.service openstack-cinder-scheduler.service
systemctl status openstack-cinder-api.service openstack-cinder-scheduler.service

systemctl enable openstack-cinder-api.service openstack-cinder-scheduler.service
systemctl list-unit-files |grep openstack-cinder |grep enabled

# 实例演示：

[root@openstack01 tools]# systemctl start openstack-cinder-api.service openstack-cinder-scheduler.service
[root@openstack01 tools]# systemctl status openstack-cinder-api.service openstack-cinder-scheduler.service
● openstack-cinder-api.service - OpenStack Cinder API Server
   Loaded: loaded (/usr/lib/systemd/system/openstack-cinder-api.service; disabled; vendor preset: disabled)
   Active: active (running) since 二 2018-10-30 16:01:27 CST; 600ms ago
 Main PID: 19104 (cinder-api)
   CGroup: /system.slice/openstack-cinder-api.service
           └─19104 /usr/bin/python2 /usr/bin/cinder-api --config-file /usr/share/cinder/cinder-dist.conf --config-file /etc/cinder/cinder.conf --logfile /var/log/cinde...

10月 30 16:01:27 openstack01.zuiyoujie.com systemd[1]: Started OpenStack Cinder API Server.
10月 30 16:01:27 openstack01.zuiyoujie.com systemd[1]: Starting OpenStack Cinder API Server...

● openstack-cinder-scheduler.service - OpenStack Cinder Scheduler Server
   Loaded: loaded (/usr/lib/systemd/system/openstack-cinder-scheduler.service; disabled; vendor preset: disabled)
   Active: active (running) since 二 2018-10-30 16:01:27 CST; 700ms ago
 Main PID: 19105 (cinder-schedule)
   CGroup: /system.slice/openstack-cinder-scheduler.service
           └─19105 /usr/bin/python2 /usr/bin/cinder-scheduler --config-file /usr/share/cinder/cinder-dist.conf --config-file /etc/cinder/cinder.conf --logfile /var/log...

10月 30 16:01:27 openstack01.zuiyoujie.com systemd[1]: Started OpenStack Cinder Scheduler Server.
10月 30 16:01:27 openstack01.zuiyoujie.com systemd[1]: Starting OpenStack Cinder Scheduler Server...
[root@openstack01 tools]# systemctl enable openstack-cinder-api.service openstack-cinder-scheduler.service
Created symlink from /etc/systemd/system/multi-user.target.wants/openstack-cinder-api.service to /usr/lib/systemd/system/openstack-cinder-api.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/openstack-cinder-scheduler.service to /usr/lib/systemd/system/openstack-cinder-scheduler.service.
[root@openstack01 tools]# systemctl list-unit-files |grep openstack-cinder |grep enabled     
openstack-cinder-api.service                  enabled 
openstack-cinder-scheduler.service            enabled

# 至此，控制端的cinder服务安装完毕，在dashboard上面可以看到项目目录中多了一个卷服务

# 接下来安装块存储节点服务器storage node

9.2.在存储节点服务器安装cinder存储服务

# 存储节点建议单独部署服务器（最好是物理机），测试时也可以部署在控制节点或者计算节点
# 在本文，存储节点使用LVM逻辑卷提供服务，需要提供一块空的磁盘用以创建LVM逻辑卷
# 我这里在VMware虚拟机增加一块100GB的磁盘

1）安装LVM相关软件包

yum install lvm2 device-mapper-persistent-data -y

2）启动LVM的metadata服务并配置开机自启动

systemctl start lvm2-lvmetad.service
systemctl status lvm2-lvmetad.service

systemctl enable lvm2-lvmetad.service
systemctl list-unit-files |grep lvm2-lvmetad |grep enabled

3）创建LVM逻辑卷

# 检查磁盘状态

fdisk -l

# 创建LVM 物理卷 /dev/sdb

pvcreate /dev/sdb

# 创建 LVM 卷组 cinder-volumes，块存储服务会在这个卷组中创建逻辑卷

vgcreate cinder-volumes /dev/sdb

# 实例演示：

[root@openstack02 ~]# fdisk -l

磁盘 /dev/sda：536.9 GB, 536870912000 字节，1048576000 个扇区
Units = 扇区 of 1 * 512 = 512 bytes
扇区大小(逻辑/物理)：512 字节 / 512 字节
I/O 大小(最小/最佳)：512 字节 / 512 字节
磁盘标签类型：dos
磁盘标识符：0x0003970d

   设备 Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048     1026047      512000   83  Linux
/dev/sda2         1026048  1044381695   521677824   83  Linux
/dev/sda3      1044381696  1048575999     2097152   82  Linux swap / Solaris

磁盘 /dev/sdb：107.4 GB, 107374182400 字节，209715200 个扇区
Units = 扇区 of 1 * 512 = 512 bytes
扇区大小(逻辑/物理)：512 字节 / 512 字节
I/O 大小(最小/最佳)：512 字节 / 512 字节

[root@openstack02 ~]# pvcreate /dev/sdb
  Physical volume "/dev/sdb" successfully created.
[root@openstack02 ~]# vgcreate cinder-volumes /dev/sdb
  Volume group "cinder-volumes" successfully created

# ok

4）配置过滤器，防止系统出错

# 默认只会有openstack实例访问块存储卷组，不过，底层的操作系统也会管理这些设备并尝试将逻辑卷与系统关联。
# 默认情况下LVM卷扫描工具会扫描整个/dev目录，查找所有包含lvm卷的块存储设备。如果其他项目在某个磁盘设备sda，sdc等上使用了lvm卷，那么扫描工具检测到这些卷时会尝试缓存这些lvm卷，可能导致底层操作系统或者其他服务无法正常调用他们的lvm卷组，从而产生各种问题，需要手动配置LVM，让LVM卷扫描工具只扫描包含"cinder-volume"卷组的设备/dev/sdb，我这边磁盘分区都是格式化的手工分区，目前不存在这个问题，以下是配置演示

vim /etc/lvm/lvm.conf
-----------------------------
devices {
filter = [ "a/sdb/", "r/.*/"]
}
-----------------------------

# 配置规则：

# 每个过滤器组中的元素都以a开头accept接受，或以 r 开头reject拒绝，后面连接设备名称的正则表达式规则。
# 过滤器组必须以"r/.*/"结束，过滤所有保留设备。
# 可以使用命令:vgs -vvvv来测试过滤器。

# 注意：

# 如果存储节点的操作系统磁盘/dev/sda使用的是LVM卷组，也需要将该设备添加到过滤器中，例如：
filter = [ "a/sda/", "a/sdb/", "r/.*/"]
# 如果计算节点的操作系统磁盘/dev/sda使用的是LVM卷组，也需要修改这些节点的/etc/lvm/lvm.conf，在过滤器中增加该类型的磁盘设备，例如：
filter = [ "a/sda/", "r/.*/"]

5）在存储节点安装配置cinder组件

yum install openstack-cinder targetcli python-keystone -y

6）在存储节点快速修改cinder配置

openstack-config --set  /etc/cinder/cinder.conf database connection  mysql+pymysql://cinder:cinder@controller/cinder
openstack-config --set  /etc/cinder/cinder.conf DEFAULT transport_url  rabbit://openstack:openstack@controller
openstack-config --set  /etc/cinder/cinder.conf DEFAULT auth_strategy  keystone 
openstack-config --set  /etc/cinder/cinder.conf keystone_authtoken www_authenticate_uri  http://controller:5000
openstack-config --set  /etc/cinder/cinder.conf keystone_authtoken auth_url  http://controller:5000
openstack-config --set  /etc/cinder/cinder.conf keystone_authtoken memcached_servers  controller:11211
openstack-config --set  /etc/cinder/cinder.conf keystone_authtoken auth_type  password
openstack-config --set  /etc/cinder/cinder.conf keystone_authtoken project_domain_name  default 
openstack-config --set  /etc/cinder/cinder.conf keystone_authtoken user_domain_name  default
openstack-config --set  /etc/cinder/cinder.conf keystone_authtoken project_name  service 
openstack-config --set  /etc/cinder/cinder.conf keystone_authtoken username  cinder
openstack-config --set  /etc/cinder/cinder.conf keystone_authtoken password  cinder
openstack-config --set  /etc/cinder/cinder.conf DEFAULT my_ip 192.168.1.82
openstack-config --set  /etc/cinder/cinder.conf lvm volume_driver cinder.volume.drivers.lvm.LVMVolumeDriver
openstack-config --set  /etc/cinder/cinder.conf lvm volume_group cinder-volumes
openstack-config --set  /etc/cinder/cinder.conf lvm iscsi_protocol  iscsi
openstack-config --set  /etc/cinder/cinder.conf lvm iscsi_helper  lioadm
openstack-config --set  /etc/cinder/cinder.conf DEFAULT enabled_backends  lvm
openstack-config --set  /etc/cinder/cinder.conf DEFAULT glance_api_servers  http://controller:9292
openstack-config --set  /etc/cinder/cinder.conf oslo_concurrency lock_path  /var/lib/cinder/tmp

# 如果存储节点是双网卡，选项my_ip需要配置存储节点的管理IP，否则配置本机IP
# 检查生效的cinder配置

egrep -v "^#|^$" /etc/cinder/cinder.conf
grep '^[a-z]' /etc/cinder/cinder.conf

# 实例演示：

[root@openstack02 ~]# egrep -v "^#|^$" /etc/cinder/cinder.conf
[DEFAULT]
transport_url = rabbit://openstack:openstack@controller
auth_strategy = keystone
my_ip = 192.168.1.82
enabled_backends = lvm
glance_api_servers = http://controller:9292
[backend]
[backend_defaults]
[barbican]
[brcd_fabric_example]
[cisco_fabric_example]
[coordination]
[cors]
[database]
connection = mysql+pymysql://cinder:cinder@controller/cinder
[fc-zone-manager]
[healthcheck]
[key_manager]
[keystone_authtoken]
www_authenticate_uri = http://controller:5000
auth_url = http://controller:5000
memcached_servers = controller:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = cinder
password = cinder
[matchmaker_redis]
[nova]
[oslo_concurrency]
lock_path = /var/lib/cinder/tmp
[oslo_messaging_amqp]
[oslo_messaging_kafka]
[oslo_messaging_notifications]
[oslo_messaging_rabbit]
[oslo_messaging_zmq]
[oslo_middleware]
[oslo_policy]
[oslo_reports]
[oslo_versionedobjects]
[profiler]
[sample_remote_file_source]
[service_user]
[ssl]
[vault]
[lvm]
volume_driver = cinder.volume.drivers.lvm.LVMVolumeDriver
volume_group = cinder-volumes
iscsi_protocol = iscsi
iscsi_helper = lioadm
---------------------------------------------------------
[root@openstack02 ~]# grep '^[a-z]' /etc/cinder/cinder.conf
transport_url = rabbit://openstack:openstack@controller
auth_strategy = keystone
my_ip = 192.168.1.82
enabled_backends = lvm
glance_api_servers = http://controller:9292
connection = mysql+pymysql://cinder:cinder@controller/cinder
www_authenticate_uri = http://controller:5000
auth_url = http://controller:5000
memcached_servers = controller:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = cinder
password = cinder
lock_path = /var/lib/cinder/tmp
volume_driver = cinder.volume.drivers.lvm.LVMVolumeDriver
volume_group = cinder-volumes
iscsi_protocol = iscsi
iscsi_helper = lioadm

7）在存储节点启动cinder服务并配置开机自启动

# 需要启动2个服务

systemctl start openstack-cinder-volume.service target.service
systemctl status openstack-cinder-volume.service target.service

systemctl enable openstack-cinder-volume.service target.service
systemctl list-unit-files |grep openstack-cinder |grep enabled
systemctl list-unit-files |grep target.service |grep enabled

# 实例演示：

[root@openstack02 ~]# systemctl start openstack-cinder-volume.service target.service
[root@openstack02 ~]# systemctl status openstack-cinder-volume.service target.service
● openstack-cinder-volume.service - OpenStack Cinder Volume Server
   Loaded: loaded (/usr/lib/systemd/system/openstack-cinder-volume.service; disabled; vendor preset: disabled)
   Active: active (running) since 二 2018-10-30 18:23:10 CST; 668ms ago
 Main PID: 2075 (cinder-volume)
    Tasks: 1
   CGroup: /system.slice/openstack-cinder-volume.service
           └─2075 /usr/bin/python2 /usr/bin/cinder-volume --config-file /usr/share/cinder/cinder-dist.conf --config-file /etc/cinder/cinder.conf --logfile /var/log/cinder/volume.log

10月 30 18:23:10 openstack02.zuiyoujie.com systemd[1]: Started OpenStack Cinder Volume Server.
10月 30 18:23:10 openstack02.zuiyoujie.com systemd[1]: Starting OpenStack Cinder Volume Server...

● target.service - Restore LIO kernel target configuration
   Loaded: loaded (/usr/lib/systemd/system/target.service; disabled; vendor preset: disabled)
   Active: active (exited) since 二 2018-10-30 18:23:11 CST; 49ms ago
  Process: 2076 ExecStart=/usr/bin/targetctl restore (code=exited, status=0/SUCCESS)
 Main PID: 2076 (code=exited, status=0/SUCCESS)

10月 30 18:23:10 openstack02.zuiyoujie.com systemd[1]: Starting Restore LIO kernel target configuration...
10月 30 18:23:11 openstack02.zuiyoujie.com target[2076]: No saved config file at /etc/target/saveconfig.json, ok, exiting
10月 30 18:23:11 openstack02.zuiyoujie.com systemd[1]: Started Restore LIO kernel target configuration.
[root@openstack02 ~]# systemctl enable openstack-cinder-volume.service target.service
Created symlink from /etc/systemd/system/multi-user.target.wants/openstack-cinder-volume.service to /usr/lib/systemd/system/openstack-cinder-volume.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/target.service to /usr/lib/systemd/system/target.service.
[root@openstack02 ~]# systemctl list-unit-files |grep openstack-cinder |grep enabled
openstack-cinder-volume.service               enabled 
[root@openstack02 ~]# systemctl list-unit-files |grep target.service |grep enabled
target.service                                enabled

# 至此，在存储节点安装cinder服务就完成了

9.3.在控制节点进行验证

1）获取管理员变量

cd /server/tools/
source keystone-admin-pass.sh

2）查看存储卷列表

openstack volume service list

# 实例演示：

[root@openstack01 tools]# openstack volume service list
+------------------+-------------------------------+------+---------+-------+----------------------------+
| Binary           | Host                          | Zone | Status  | State | Updated At                 |
+------------------+-------------------------------+------+---------+-------+----------------------------+
| cinder-scheduler | openstack01.zuiyoujie.com     | nova | enabled | up    | 2018-10-31T10:55:19.000000 |
| cinder-volume    | openstack02.zuiyoujie.com@lvm | nova | enabled | up    | 2018-10-31T10:55:21.000000 |
+------------------+-------------------------------+------+---------+-------+----------------------------+

# 返回以上信息，表示cinder相关节点安装完成

9.4.cinder云磁盘使用建议

1）云磁盘可以进行磁盘迁移，扩容，缩容等操作，但不建议在生产环境进行尝试，测试环境可以尝试，但也要注意备份数据

2）涉及重要数据的话尽量还是不使用云磁盘，而采用本地磁盘存储数据，如果出现问题至少各个磁盘数据是分开的，磁盘文件还在

3）总的来说，使用openstack部署企业私有云，生产环境用本地磁盘就好，测试环境可以以尝试使用云磁盘

======== 完毕，呵呵呵呵 ========

作者：天生帅才