1,先安装pip:
pip: http://pypi.python.org/packages/source/p/pip/pip-1.1.tar.gz#md5=62a9f08dd5dc69d76734568a6c040508
 
tar zxpf pip-1.1.tar.gz
cd pip-1.1
python setup.py install
 
2,下载swift源码
yum install git
git clone https://github.com/openstack/swift.git
 
https://github.com/openstack/swift
 
3,下载依赖库,用pip把依赖库下载到pipdowns文件夹中
 
mkdir pipdowns
pip install -r swift-master/requirements.txt -d pipdowns/ --no-install
 
 
4,升级setuptool
pip install --upgrade setuptools
 
5,安装依赖库
 
pip install pipdowns/*
 
需要安装gcc
 
yum install gcc gcc-c++ kernel-devel
 
需要安装
 
yum install python-devel.x86_64
yum libffi-devel.x86_64
 
另外如果是存储节点需要安装
 
yum install python-setuptools xinetd xfsprogs
 
6,安装swift
cd swift
python setup.py install
 
7,代理节点安装配置
 
yum install memcached
pip install netifaces
 
启动memcached
chkconfig memcached on && service memcached start
 
创建用户、组合配置文件目录
 
- useradd –Mr swift
- mkdir /etc/swift
- chown –R swift:swift /etc/swift
 
创建swift.conf
 
该文件/etc/swift/swift.conf在所有节点必须完全一样,内容如下
 
- [swift-hash]
- # random unique string that can never change (DO NOT LOSE)
- swift_hash_path_suffix = fLIbertYgibbitZ
 
 
 
创建用于SSL的证书文件
 
Swift默认使用https协议,因此需要创建证书文件,这也是使用Cyberduck作为客户端所必须的,当然在这里先不打算使用https因此可以省略
 
1. cd /etc/swift
2. openssl req -new -x509 -nodes -out cert.crt -keyout cert.key
 
创建代理服务的配置文件
 
代理服务的配置文件为/etc/swift/proxy-server.conf,内容如下:
 
1. [DEFAULT]
2. bind_port = 8888
3. user = swift
4. #cert_file = /etc/swift/cert.crt
5. #key_file = /etc/swift/cert.key
6. [pipeline:main]
7. pipeline = catch_errors healthcheck cache swift3 authtoken keystone proxy-server
8. [app:proxy-server]
9. use = egg:swift#proxy
10. account_autocreate = true
11. [filter:swift3]
12. use = egg:swift#swift3
13. [filter:keystone]
14. paste.filter_factory =keystone.middleware.swift_auth:filter_factory
15. operator_roles = admin, swiftoperator
16. [filter:authtoken]
17. paste.filter_factory =keystone.middleware.auth_token:filter_factory
18. # Delaying the auth decision is required tosupport token-less
19. # usage for anonymous referrers ('.r:*').
20. delay_auth_decision = true
21. auth_protocol = http
22. service_port = 5000
23. service_host = 127.0.0.1
24. auth_port = 35357
25. auth_host = 127.0.0.1
26. auth_token = 012345SECRET99TOKEN012345
27. admin_token = 012345SECRET99TOKEN012345
28. [filter:cache]
29. use = egg:swift#memcache
30. set log_name = cache
31. [filter:catch_errors]
32. use = egg:swift#catch_errors
33. [filter:healthcheck]
34. use = egg:swift#healthcheck
这里有两个token需要设置成keydtonek admin token一样。如果打算使用https的话需要修改端口为443,创建前面据说的证书文件,去掉证书文件配置前的注释,去掉[filter:authtoken]下的auth_protocol = http。如果打算使用非本地的或多个memcached服务器的话,需要在[filter:cache]下添加一项,多个地址用逗号分开,如下:
 memcache_servers = 10.1.2.3:11211,10.1.2.4:11211 
 
先尝试配置存储节点,最后再来配置ring
创建相应的rings
 
1. cd /etc/swift
2. swift-ring-builder account.builder create 18 3 1
3. swift-ring-builder container.builder create 18 3 1
4. swift-ring-builder object.builder create 18 3 1
 
这里得注意中间这个3它表示存储对象在swift中的拷贝数,这个值得少于等于你的zone数,否则在后面rebalance的时候会报list index out of range错,在产品环境中推荐的值是3。创建完ring后会生成对应的builder文件,这些文件很重要,应当实时备份。
将每个存储节点的每个存储设备添加到各个ring
语法格式如下
 
1. swift-ring-builder account.builder add z<ZONE>-<STORAGE_LOCAL_NET_IP>:6002/<DEVICE> 100
2. swift-ring-builder container.builder add z<ZONE>-<STORAGE_LOCAL_NET_IP_1>:6001/<DEVICE> 100
3. swift-ring-builder object.builder add z<ZONE>-<STORAGE_LOCAL_NET_IP_1>:6000/<DEVICE> 100
 
比如说我这里计划将10.61.2.13作为zone 1,计划在上面建一个分区/dev/sdb1挂载到/srv/node/sdb1,那我的命令如下:
 
1. swift-ring-builder account.builder add z1-10.61.2.13:6002/sdb1100
2. swift-ring-builder container.builder add z1-10.61.2.13:6001/sdb1 100
3. swift-ring-builder object.builder add z1-10.61.2.13:6000/sdb1 100
 
如果还有其它分区要添加的话依样就是了,最后的100是一个表示权重的浮点数文档建议的值是100XTB,就是你分区有1TB大小就设100,2TB大小就设200。Zone的数是从1开始的依次递增。
验证ring
 
通过以上命令添加完实体后可通过以下命令来列出所ring的信息进行验证
 
1. swift-ring-builder account.builder
2. swift-ring-builder container.builder
3. swift-ring-builder object.builder
Rebalance rings
 
添加完全后要rebalance 这些rings,命令如下
 
1. swift-ring-builder account.builder rebalance
2. swift-ring-builder container.builder rebalance
3. swift-ring-builder object.builder rebalance
完全后会产生account.ring.gz, container.ring.gz和 object.ring.gz三个文件,这些文件以后要复制到每个代理节点(如果添加了多个的话)及存储节点的/etc/swift目录下
 
最后确保配置文件的用户属性并运行代理服务
 
1. chown -R swift:swift /etc/swift
2. swift-init proxy start
 
修正:修正proxy启动时的ValueError:invalid literal for int() with base 10: 'true'错误,方法见这:
https://github.com/openstack/keystone/commit/bc803a4cede7ed2f39f4dc5c74977eedf46eb205 vim /usr/lib/python2.6/site-packages/keystone-2012.1-py2.6.egg/keystone/middleware/auth_token.py按下图修改 
1,先安装pip:
pip: http://pypi.python.org/packages/source/p/pip/pip-1.1.tar.gz#md5=62a9f08dd5dc69d76734568a6c040508
 
tar zxpf pip-1.1.tar.gz
cd pip-1.1
python setup.py install
 
2,下载swift源码
yum install git
git clone https://github.com/openstack/swift.git
 
https://github.com/openstack/swift
 
3,下载依赖库,用pip把依赖库下载到pipdowns文件夹中
 
mkdir pipdowns
pip install -r swift-master/requirements.txt -d pipdowns/ --no-install
 
 
4,升级setuptool
pip install --upgrade setuptools
 
5,安装依赖库
 
pip install pipdowns/*
 
需要安装gcc
 
yum install gcc gcc-c++ kernel-devel
 
需要安装
 
yum install python-devel.x86_64
yum libffi-devel.x86_64
 
另外如果是存储节点需要安装
 
yum install python-setuptools xinetd xfsprogs
 
6,安装swift
cd swift
python setup.py install
 
7,代理节点安装配置
 
yum install memcached
pip install netifaces
 
启动memcached
chkconfig memcached on && service memcached start
 
创建用户、组合配置文件目录
 
- useradd –Mr swift
- mkdir /etc/swift
- chown –R swift:swift /etc/swift
 
创建swift.conf
 
该文件/etc/swift/swift.conf在所有节点必须完全一样,内容如下
 
- [swift-hash]
- # random unique string that can never change (DO NOT LOSE)
- swift_hash_path_suffix = fLIbertYgibbitZ
 
 
 
创建用于SSL的证书文件
 
Swift默认使用https协议,因此需要创建证书文件,这也是使用Cyberduck作为客户端所必须的,当然在这里先不打算使用https因此可以省略
 
1. cd /etc/swift
2. openssl req -new -x509 -nodes -out cert.crt -keyout cert.key
 
创建代理服务的配置文件
 
代理服务的配置文件为/etc/swift/proxy-server.conf,内容如下:
 
1. [DEFAULT]
2. bind_port = 8888
3. user = swift
4. #cert_file = /etc/swift/cert.crt
5. #key_file = /etc/swift/cert.key
6. [pipeline:main]
7. pipeline = catch_errors healthcheck cache swift3 authtoken keystone proxy-server
8. [app:proxy-server]
9. use = egg:swift#proxy
10. account_autocreate = true
11. [filter:swift3]
12. use = egg:swift#swift3
13. [filter:keystone]
14. paste.filter_factory =keystone.middleware.swift_auth:filter_factory
15. operator_roles = admin, swiftoperator
16. [filter:authtoken]
17. paste.filter_factory =keystone.middleware.auth_token:filter_factory
18. # Delaying the auth decision is required tosupport token-less
19. # usage for anonymous referrers ('.r:*').
20. delay_auth_decision = true
21. auth_protocol = http
22. service_port = 5000
23. service_host = 127.0.0.1
24. auth_port = 35357
25. auth_host = 127.0.0.1
26. auth_token = 012345SECRET99TOKEN012345
27. admin_token = 012345SECRET99TOKEN012345
28. [filter:cache]
29. use = egg:swift#memcache
30. set log_name = cache
31. [filter:catch_errors]
32. use = egg:swift#catch_errors
33. [filter:healthcheck]
34. use = egg:swift#healthcheck
这里有两个token需要设置成keydtonek admin token一样。如果打算使用https的话需要修改端口为443,创建前面据说的证书文件,去掉证书文件配置前的注释,去掉[filter:authtoken]下的auth_protocol = http。如果打算使用非本地的或多个memcached服务器的话,需要在[filter:cache]下添加一项,多个地址用逗号分开,如下:
 memcache_servers = 10.1.2.3:11211,10.1.2.4:11211 
 
先尝试配置存储节点,最后再来配置ring
创建相应的rings
 
1. cd /etc/swift
2. swift-ring-builder account.builder create 18 3 1
3. swift-ring-builder container.builder create 18 3 1
4. swift-ring-builder object.builder create 18 3 1
 
这里得注意中间这个3它表示存储对象在swift中的拷贝数,这个值得少于等于你的zone数,否则在后面rebalance的时候会报list index out of range错,在产品环境中推荐的值是3。创建完ring后会生成对应的builder文件,这些文件很重要,应当实时备份。
将每个存储节点的每个存储设备添加到各个ring
语法格式如下
 
1. swift-ring-builder account.builder add z<ZONE>-<STORAGE_LOCAL_NET_IP>:6002/<DEVICE> 100
2. swift-ring-builder container.builder add z<ZONE>-<STORAGE_LOCAL_NET_IP_1>:6001/<DEVICE> 100
3. swift-ring-builder object.builder add z<ZONE>-<STORAGE_LOCAL_NET_IP_1>:6000/<DEVICE> 100
 
比如说我这里计划将10.61.2.13作为zone 1,计划在上面建一个分区/dev/sdb1挂载到/srv/node/sdb1,那我的命令如下:
 
1. swift-ring-builder account.builder add z1-10.61.2.13:6002/sdb1100
2. swift-ring-builder container.builder add z1-10.61.2.13:6001/sdb1 100
3. swift-ring-builder object.builder add z1-10.61.2.13:6000/sdb1 100
 
如果还有其它分区要添加的话依样就是了,最后的100是一个表示权重的浮点数文档建议的值是100XTB,就是你分区有1TB大小就设100,2TB大小就设200。Zone的数是从1开始的依次递增。
验证ring
 
通过以上命令添加完实体后可通过以下命令来列出所ring的信息进行验证
 
1. swift-ring-builder account.builder
2. swift-ring-builder container.builder
3. swift-ring-builder object.builder
Rebalance rings
 
添加完全后要rebalance 这些rings,命令如下
 
1. swift-ring-builder account.builder rebalance
2. swift-ring-builder container.builder rebalance
3. swift-ring-builder object.builder rebalance
完全后会产生account.ring.gz, container.ring.gz和 object.ring.gz三个文件,这些文件以后要复制到每个代理节点(如果添加了多个的话)及存储节点的/etc/swift目录下
 
最后确保配置文件的用户属性并运行代理服务
 
1. chown -R swift:swift /etc/swift
2. swift-init proxy start
 
修正:修正proxy启动时的ValueError:invalid literal for int() with base 10: 'true'错误,方法见这:
https://github.com/openstack/keystone/commit/bc803a4cede7ed2f39f4dc5c74977eedf46eb205 vim /usr/lib/python2.6/site-packages/keystone-2012.1-py2.6.egg/keystone/middleware/auth_token.py按下图修改 
 
 
 
 
 
存储节点安装、配置和运行
 
创建存储节点使用磁盘:
1. fdisk /dev/sdb (依次输入n<Enter>p<Enter>1<Enter><Enter><Enter>w<Enter>)
2. mkfs.xfs -i size=1024 /dev/sdb1
3. echo "/dev/sdb1 /srv/node/sdb1 xfs noatime,nodiratime,nobarrier,logbufs=8 0 0" >> /etc/fstab
4. mkdir -p /srv/node/sdb1
5. mount /srv/node/sdb1
6. useradd –Mr swift
7. chown -R swift:swift /srv/node
 
我这里是使用虚拟机所以使用下面的步骤
 
为swift创建存储区域
truncate -s 10GB swift-disk
1.44和1.45在 /home下
9.101和9.105在 /mnt/sdb下
 
mkfs.xfs swift-disk
 
mkdir /srv/node/sdb1 -p
echo "/home/swift-disk /srv/node/sdb1 xfs loop,noatime,nodiratime,nobarrier,logbufs=8 0 0" >> /etc/fstab
 
mount /srv/node/sdb1/
useradd -Mr swift
chown -R swift:swift /srv/node
 
 
 
ntp配置
 
配置控制节点
[root@cc ~]# vim /etc/ntp.conf
 在:
 # Hosts on local network are less restricted.
 #restrict 192.168.1.0 mask 255.255.255.0 nomodify notrap
 下添加你要提供时间同步服务的网段,如我的为:
 restrict 192.168.1.0 mask 255.255.255.0 nomodify notraprestrict 192.168.9.0 mask 255.255.255.0 nomodify notrap
去掉以下两行前的注释:
 server 127.127.1.0 # local clock
 fudge 127.127.1.0 stratum 10 
[root@cc ~]# vim /etc/sysconfig/ntpd
 在后面添加:
 SYNC_HWCLOCK=yes
 使时间更新到bios
 启动ntp:
 [root@cc ~]# chkconfig ntpd on && service ntpd start 
 
2.1.2配置启动计算节点的ntp
 [root@node1 ~]# vim /etc/ntp.conf
 以下三行:
 server 0.centos.pool.ntp.org
 server 1.centos.pool.ntp.org
 server 2.centos.pool.ntp.org
 替换为:
 server 192.168.1.44
 这里的ip还是你的控制节点的ip[root@node1 ~]# vim /etc/sysconfig/ntpd
 在后面添加:
 SYNC_HWCLOCK=yes
 使时间更新到bios
 启动ntp:
 [root@node1 ~]# chkconfig ntpd on && service ntpd start 
 
2.3关闭selinux
 selinux会引起一些权限的问题,因此我选择关闭
 [root@node1 ~]# setenforce 0
 [root@node1 ~]# vim /etc/selinux/config
 修改SELINUX=disabled 
 
设置rsync
创建文件/etc/rsyncd.conf
 
1. uid = swift
2. gid = swift
3. log file = /var/log/rsyncd.log
4. pid file = /var/run/rsyncd.pid
5. #address = <STORAGE_LOCAL_NET_IP>
6. [account]
7. max connections = 2
8. path = /srv/node/
9. read only = false
10. lock file = /var/lock/account.lock
11. [container]
12. max connections = 2
13. path = /srv/node/
14. read only = false
15. lock file = /var/lock/container.lock
16. [object]
17. max connections = 2
18. path = /srv/node/
19. read only = false
20. lock file = /var/lock/object.lock
创建文件 /etc/swift/account-server.conf:
 
1. [DEFAULT]
2. #bind_ip = <STORAGE_LOCAL_NET_IP>
3. workers = 2
4. [pipeline:main]
5. pipeline = account-server
6. [app:account-server]
7. use = egg:swift#account
8. [account-replicator]
9. [account-auditor]
10. [account-reaper]
 
创建文件 /etc/swift/container-server.conf:
1. [DEFAULT]
2. #bind_ip = <STORAGE_LOCAL_NET_IP>
3. workers = 2
4. [pipeline:main]
5. pipeline = container-server
6. [app:container-server]
7. use = egg:swift#container
8. [container-replicator]
9. [container-updater]
10. [container-auditor]
 
创建文件 /etc/swift/object-server.conf:
1. [DEFAULT]
2. #bind_ip = <STORAGE_LOCAL_NET_IP>
3. workers = 2
4. [pipeline:main]
5. pipeline = object-server
6. [app:object-server]
7. use = egg:swift#object
8. [object-replicator]
9. [object-updater]
10. [object-auditor]
11. [object-expirer]
 
以上配置文件中有一项bind_ip设置,目的是只监听内网ip的相应端口,但是这样把配置文件拷到每个存储节点时都要修改,我觉得可以把这项去掉,这样所有存储节点的配置就是一样了,直接用就是,真需要考虑安全的话可以在iptables上只开放内网的ip及端口即可。
启动存储节点
1. swift-init object-server start
2. swift-init object-replicator start
3. swift-init object-updater start
4. swift-init object-auditor start
5. swift-init container-server start
6. swift-init container-replicator start
7. swift-init container-updater start
8. swift-init container-auditor start
9. swift-init account-server start
10. swift-init account-replicator start
11. swift-init account-auditor start
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
安装问题记录:
 
error in setup command: Error parsing /root/swift/setup.cfg: Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository.
 
出现这个问题时,重新用git下载源码
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
存储节点安装、配置和运行
 
创建存储节点使用磁盘:
1. fdisk /dev/sdb (依次输入n<Enter>p<Enter>1<Enter><Enter><Enter>w<Enter>)
2. mkfs.xfs -i size=1024 /dev/sdb1
3. echo "/dev/sdb1 /srv/node/sdb1 xfs noatime,nodiratime,nobarrier,logbufs=8 0 0" >> /etc/fstab
4. mkdir -p /srv/node/sdb1
5. mount /srv/node/sdb1
6. useradd –Mr swift
7. chown -R swift:swift /srv/node
 
我这里是使用虚拟机所以使用下面的步骤
 
为swift创建存储区域
truncate -s 10GB swift-disk
1.44和1.45在 /home下
9.101和9.105在 /mnt/sdb下
 
mkfs.xfs swift-disk
 
mkdir /srv/node/sdb1 -p
echo "/home/swift-disk /srv/node/sdb1 xfs loop,noatime,nodiratime,nobarrier,logbufs=8 0 0" >> /etc/fstab
 
mount /srv/node/sdb1/
useradd -Mr swift
chown -R swift:swift /srv/node
 
 
 
ntp配置
 
配置控制节点
[root@cc ~]# vim /etc/ntp.conf
 在:
 # Hosts on local network are less restricted.
 #restrict 192.168.1.0 mask 255.255.255.0 nomodify notrap
 下添加你要提供时间同步服务的网段,如我的为:
 restrict 192.168.1.0 mask 255.255.255.0 nomodify notraprestrict 192.168.9.0 mask 255.255.255.0 nomodify notrap
去掉以下两行前的注释:
 server 127.127.1.0 # local clock
 fudge 127.127.1.0 stratum 10 
[root@cc ~]# vim /etc/sysconfig/ntpd
 在后面添加:
 SYNC_HWCLOCK=yes
 使时间更新到bios
 启动ntp:
 [root@cc ~]# chkconfig ntpd on && service ntpd start 
 
2.1.2配置启动计算节点的ntp
 [root@node1 ~]# vim /etc/ntp.conf
 以下三行:
 server 0.centos.pool.ntp.org
 server 1.centos.pool.ntp.org
 server 2.centos.pool.ntp.org
 替换为:
 server 192.168.1.44
 这里的ip还是你的控制节点的ip[root@node1 ~]# vim /etc/sysconfig/ntpd
 在后面添加:
 SYNC_HWCLOCK=yes
 使时间更新到bios
 启动ntp:
 [root@node1 ~]# chkconfig ntpd on && service ntpd start 
 
2.3关闭selinux
 selinux会引起一些权限的问题,因此我选择关闭
 [root@node1 ~]# setenforce 0
 [root@node1 ~]# vim /etc/selinux/config
 修改SELINUX=disabled 
 
设置rsync
创建文件/etc/rsyncd.conf
 
1. uid = swift
2. gid = swift
3. log file = /var/log/rsyncd.log
4. pid file = /var/run/rsyncd.pid
5. #address = <STORAGE_LOCAL_NET_IP>
6. [account]
7. max connections = 2
8. path = /srv/node/
9. read only = false
10. lock file = /var/lock/account.lock
11. [container]
12. max connections = 2
13. path = /srv/node/
14. read only = false
15. lock file = /var/lock/container.lock
16. [object]
17. max connections = 2
18. path = /srv/node/
19. read only = false
20. lock file = /var/lock/object.lock
创建文件 /etc/swift/account-server.conf:
 
1. [DEFAULT]
2. #bind_ip = <STORAGE_LOCAL_NET_IP>
3. workers = 2
4. [pipeline:main]
5. pipeline = account-server
6. [app:account-server]
7. use = egg:swift#account
8. [account-replicator]
9. [account-auditor]
10. [account-reaper]
 
创建文件 /etc/swift/container-server.conf:
1. [DEFAULT]
2. #bind_ip = <STORAGE_LOCAL_NET_IP>
3. workers = 2
4. [pipeline:main]
5. pipeline = container-server
6. [app:container-server]
7. use = egg:swift#container
8. [container-replicator]
9. [container-updater]
10. [container-auditor]
 
创建文件 /etc/swift/object-server.conf:
1. [DEFAULT]
2. #bind_ip = <STORAGE_LOCAL_NET_IP>
3. workers = 2
4. [pipeline:main]
5. pipeline = object-server
6. [app:object-server]
7. use = egg:swift#object
8. [object-replicator]
9. [object-updater]
10. [object-auditor]
11. [object-expirer]
 
以上配置文件中有一项bind_ip设置,目的是只监听内网ip的相应端口,但是这样把配置文件拷到每个存储节点时都要修改,我觉得可以把这项去掉,这样所有存储节点的配置就是一样了,直接用就是,真需要考虑安全的话可以在iptables上只开放内网的ip及端口即可。
启动存储节点
1. swift-init object-server start
2. swift-init object-replicator start
3. swift-init object-updater start
4. swift-init object-auditor start
5. swift-init container-server start
6. swift-init container-replicator start
7. swift-init container-updater start
8. swift-init container-auditor start
9. swift-init account-server start
10. swift-init account-replicator start
11. swift-init account-auditor start
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
安装问题记录:
 
error in setup command: Error parsing /root/swift/setup.cfg: Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository.
 
出现这个问题时,重新用git下载源码