一.硬件及软件环境
操作系统:SLES10 SP2 内核2.6.16.60-0.21-bigsmp.
硬件环境:x3650M2 2台 双HBA卡
DS3400磁盘柜1台 300G硬盘×8 Raid 10阵列
网络环境:节点1:192.168.31.31 私有地址:10.10.10.31 虚拟地址:192.168.31.51
节点2:192.168.31.32 私有地址:10.10.10.32 虚拟地址:192.168.31.52
RAC操作系统环境概况
主机名
|
操作系统
|
处理器
|
内存
|
ria15
|
SUSE 10 SP2
|
Xeon dual E5430 2.66GHz*2
|
4GB
|
ria16
|
SUSE 10 SP2
|
Xeon dual E5430 2.66GHz*2
|
4GB
|
STORAGE分布情况
客户操作系统上的磁盘
|
挂载点
|
大小(MB)
|
描述
|
/dev/sdb1
/dev/sdc1
|
/dev/raw/raw1
/dev/raw/raw2
|
512m
512m
|
做ocr磁盘,两个互为镜象
|
/dev/sdd1
/dev/sde1
/dev/sdf1
|
/dev/raw/raw3
/dev/raw/raw4
/dev/raw/raw5
|
512m
512m
512m
|
做voting disk磁盘,三个磁盘互为镜象
|
/dev/sdg1
/dev/sdh1
/dev/sdi1
|
|
256G
256G
256G
|
ASM磁盘组
|
/dev/sdj1
|
|
256G
|
ASM闪回恢复区
|
主机名
|
RAC实例名
|
数据库名
|
数据库文件存储
|
OCR
|
表决磁盘(Voting Disk)
|
ria15
|
orcl1
|
orcl
|
ASM
|
raw1-2
|
raw3-5
|
ria16
|
orcl2
|
orcl
|
ASM
|
raw1-2
|
raw3-5
|
1.安装suse10 sp2操作系统,相关文档请看sesu10安装手册.
2.装多路径软件(让主机支持开机就可以识别存储设备,此处使用的是rdac-LINUX-09.03.0C05.0030-source.tar)
ria16:~ # tar zxvf rdac-LINUX-09.03.0C05.0030-source.tar.gz
ria16:~ # cd linuxrdac-09.03.0C05.0030/
ria16:~ # make clean
ria16:~ # make
ria16:~ # make install
…….
You must now edit your boot loader configuration file, /boot/grub/menu.lst, to
add a new boot menu, which uses mpp-2.6.16.60-0.21-bigsmp.img as the initrd p_w_picpath.
Now Reboot the system for MPP to take effect.
The new boot menu entry should look something like this (note that it may
vary with different system configuration):
...
title SUSE Linux (2.6.16.60-0.21-bigsmp) with MPP support
kernel (hd1,3)/boot/vmlinuz root=/dev/hdb4 vga=0x31aselinux=0 splash=silent console=tty0 resume=/dev/hda2 elevator=cfq showopts
initrd (hd0,8)/boot/mpp-2.6.16.60-0.21-bigsmp.img (修改menu.lst时会使用到)
修改/boot/grub/ menu.lst让系统调用多路径镜象文件[添加最后四行] 两节点
# Modified by YaST2. Last modification on Mon Oct 12 11:19:46 UTC 2009
default 3 [此处根据实际情况更改]
timeout 8
gfxmenu (hd0,0)/message
##YaST - activate
###Don't change this comment - YaST2 identifier: Original name: linux###
title SUSE Linux Enterprise Server 10 SP2
root (hd0,0)
kernel /vmlinuz-2.6.16.60-0.21-bigsmp root=/dev/disk/by-id/scsi-3600605b00107b58012204cd819b466b1-part6 vga=0x31a resume=/dev/sda5 splash=silent showopts
initrd /initrd-2.6.16.60-0.21-bigsmp
###Don't change this comment - YaST2 identifier: Original name: failsafe###
title Failsafe -- SUSE Linux Enterprise Server 10 SP2
root (hd0,0)
kernel /vmlinuz-2.6.16.60-0.21-bigsmp root=/dev/disk/by-id/scsi-3600605b00107b58012204cd819b466b1-part6 vga=normal showopts ide=nodma apm=off acpi=off noresume nosmp noapic maxcpus=0 edd=off 3
initrd /initrd-2.6.16.60-0.21-bigsmp
title SUSE Linux Enterprise Server 10 SP2 MPP
root (hd0,0)
kernel /vmlinuz-2.6.16.60-0.21-bigsmp root=/dev/disk/by-id/scsi-3600605b00107b58012204cd819b466b1-part6 vga=0x31a resume=/dev/sda5 splash=silent showopts
initrd /mpp-2.6.16.60-0.21-bigsmp.img
3.配置NTP服务 一个节点
编辑/et/ntp.conf
fudge 192.168.31.32stratum 10 # LCL is unsynchronized 将ip地址修改为与自己实际ip地址对应
在另一节点上用yast配置客户端并停止client上的ntp服务
Service ntp stop
Chkconfig ntp off
4.磁盘分区 一个节点
重起服务器,重起服务器后,系统自动识别存储,可以通过fdisk –l查看,如果能够看到分区,说明存储成功挂载.如果未检测到磁盘,请重新安装多路径软件及检查/boot/grub/menu.lst文件配置是否正确.
此处将所有磁盘分为一个区,(使用fdisk /dev/sdx x为对应磁盘)分区完后,通过fdisk –l来进行查看.[注:分区只在一台服务器上面执行,另一台服务器执行partprob即可]
5.安装包 两节点
rpm -ivh binutils-2.17.50.0.3-1.i386.rpm
rpm -ivh libaio-0.3.104-14.2.i586.rpm
rpm -ivh compat-db-4.1.25-9.i386.rpm
rpm -ivh libstdc++-4.1.2_20070115-0.21.i586.rpm
rpm -ivh libstdc++-devel-4.1.2_20070115-0.21.i586.rpm
rpm -ivh openmotif-2.2.4-21.12.i586.rpm
rpm -ivh openmotif-demo-2.2.4-21.12.i586.rpm
rpm -ivh freetype2-devel-2.1.10-18.14.i586.rpm
rpm -ivh fontconfig-devel-2.3.94-18.16.i586.rpm
rpm -ivh xorg-x11-devel-6.9.0-50.58.i586.rpm
rpm -ivh openmotif-devel-2.2.4-21.12.i586.rpm
rpm -ivh openmotif-libs-2.2.4-21.12.i586.rpm
rpm -ivh openmotif21-libs-2.1.30MLI4-143.2.i586.rpm
rpm -ivh setarch-1.3-1.i386.rpm
6.安装asm包 两节点
oracleasm-2.6.16.60-0.21-bigsmp-2.0.4-1.SLE10.i586.rpm
oracleasm-support-2.1.3-1.SLE10.i386.rpm
oracleasmlib-2.0.4-1.SLE10.i386.rpm
三.环境变量配置及内核修改
1.修改/etc/passwd让oracle用户可以登陆 两节点
Oracle:……. /bin/bash
2.修改/etc/group将oracle用户添加到disk组 两节点
disk:x:6:oracle
3.为oracle用户设置密码 两节点
ria16:~ # passwd oracle (oracle)
Changing password for oracle.
New Password:
Bad password: too simple
Reenter New Password:
Password changed.
4.修改/etc/hosts 两节点
192.168.31.52 ria16
192.168.31.51 ria15
10.10.10.32 ria16-priv
10.10.10.31 ria15-priv
192.168.31.32 ria16-vip
192.168.31.31 ria15-vip
5.修改内核参数/etc/sysctl.conf (如果是suse系统则不用添加) 两节点
net.core.rmem_max = 1048536
net.core.rmem_default = 262144
net.core.wmem_max = 1048536
net.core.wmem_default = 262144
net.ipv4.ip_local_port_range = 1024 65000
kernel.shmall = 2097152
kernel.shmmax = 2147483648
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
fs.file-max = 65536
RHEL则添加
[root@host186 ~]# vi /etc/sysctl.conf
net.ipv4.ip_local_port_range = 1024 65000
kernel.sem = 250 32000 100 128
net.core.rmem_default = 4194304
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 262144
6.修改/etc/sericutiy/limits.conf 两节点
oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
oracle soft memlock 4000000
oracle hard memlock 4000000
7.修改/etc/init.d/boot.local建立心跳 两节点
/sbin/insmod /lib/modules/2.6.16.60-0.21-bigsmp/kernel/drivers/char/hangcheck-timer.ko hangcheck-timer hangcheck_tick=30 hangcheck_margin=180
8.修改/etc/profile.d/oracle.sh建立用户环境变量 两节点
ORACLE_BASE=/opt/oracle
export ORA_CRS_BASE=/oracle_crs
ORACLE_HOME=$ORACLE_BASE/product/10.2/db_1
# ORACLE_HOME=$ORACLE_BASE/product/10gR2/db
ORACLE_SID=orcl1/2
export ORACLE_BASE ORACLE_HOME ORACLE_SID
# For RAC
export ORA_CRS_HOME=$ORA_CRS_BASE/product/10.2/crs
export ORA_ASM_HOME=$ORACLE_BASE/product/10.2/asm
9.建立crs家目录 两节点
ria16:~ # mkdir -p /oracle_crs/product/10.2/crs
ria16:~ # chown -R root:oinstall /oracle_crs/
ria16:~ # chmod -R 775 /oracle_crs/
10.建立raw设备映射文件/etc/raw (两个ocr和3个voting disk) 两节点
raw1:sdb1
raw2:sdc1
raw3:sdd1
raw4:sde1
raw5:sdf1
11.修改/et vi /etc/udev/rules.d/50-udev-default.rules 将默认裸设备注释掉 两节点
#KERNEL=="raw[0-9]*", SUBSYSTEM=="raw", NAME="raw/%k", GROUP="disk"
新建文件/etc/udev/rules.d/99-udev-raw.rules
KERNEL=="raw[1-2]", SUBSYSTEM=="raw", NAME="raw/%k",OWNER="root", GROUP="oinstal
l",MODE="660"
KERNEL=="raw[3-5]", SUBSYSTEM=="raw", NAME="raw/%k",OWNER="oracle", GROUP="oinstall",MODE="660"
12格式化asm磁盘
Oracle ASM与Oracle数据库紧密集成在一起,并与Oracle的数据管理工具套件配合工作。它可以简化数据库存储管理,并提供原始磁盘I/O的性能。
配置ASMLib。以root用户身份在两个节点上配置ASMLib。 两个节
ria15:~ # /etc/init.d/oracleasm configure
Configuring the Oracle ASM library driver.
This will configure the on-boot properties of the Oracle ASM library
driver. The following questions will determine whether the driver is
loaded on boot and what permissions it will have. The current values
will be shown in brackets ('[]'). Hitting <ENTER> without typing an
answer will keep that current value. Ctrl-C will abort.
Default user to own the driver interface []: oracle
Default group to own the driver interface []: dba
Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: done
Initializing the Oracle ASMLib driver: done
Scanning the system for Oracle ASMLib disks: done
ria15:~ # /etc/init.d/oracleasm enable
Writing Oracle ASM library driver configuration: done
Initializing the Oracle ASMLib driver: done
Scanning the system for Oracle ASMLib disks: done
创建ASM磁盘。以root用户身份在任何一个节点上创建ASM磁盘。
ria15:~ # /etc/init.d/oracleasm createdisk VOL1 /dev/sdg1
Marking disk "VOL1" as an ASM disk: done
ria15:~ # /etc/init.d/oracleasm createdisk VOL2 /dev/sdh1
Marking disk "VOL2" as an ASM disk: done
ria15:~ # /etc/init.d/oracleasm createdisk VOL3 /dev/sdi1
Marking disk "VOL3" as an ASM disk: done
ria15:~ # /etc/init.d/oracleasm createdisk VOL4 /dev/sdj
sdj sdj1
ria15:~ # /etc/init.d/oracleasm createdisk VOL4 /dev/sdj1
Marking disk "VOL4" as an ASM disk: done
ria15:~ #
ria15:~ # /etc/init.d/oracleasm scandisks
Scanning the system for Oracle ASMLib disks: done
ria15:~ # /etc/init.d/oracleasm 列出了oracleasm所有命令
Usage: /etc/init.d/oracleasm {start|stop|restart|enable|disable|configure|createdisk|deletedisk|querydisk|listdisks|scandisks|status}
ria15:~ # /etc/init.d/oracleasm listdisks
VOL1
VOL2
VOL3
VOL4
在ria16上执行
ria16:~ # /etc/init.d/oracleasm enable
Writing Oracle ASM library driver configuration: done
Initializing the Oracle ASMLib driver: done
Scanning the system for Oracle ASMLib disks: done
ria16:~ # /etc/init.d/oracleasm listdisks
VOL1
VOL2
VOL3
VOL4
13.建立等效ssh-keygen 两节点
oracle@ria16:~> ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/opt/oracle/.ssh/id_rsa):
Created directory '/opt/oracle/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /opt/oracle/.ssh/id_rsa.
Your public key has been saved in /opt/oracle/.ssh/id_rsa.pub.
The key fingerprint is:
9a:51:5a:25:29:60:2c:f8:5c:72:86:d6:d1:e8:b7:93 oracle@ria16
oracle@ria16:~> ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/opt/oracle/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /opt/oracle/.ssh/id_dsa.
Your public key has been saved in /opt/oracle/.ssh/id_dsa.pub.
The key fingerprint is:
7e:34:3a:ad:58:57:1c:04:a3:8f:ff:32:6a:1a:bc:87 oracle@ria16
将双方公钥文件拷贝到对方
oracle@ria16:~>cat /opt/oracle/.ssh/ id_dsa.pub >>/opt/oracle/.ssh/authorized_keys.ria16
oracle@ria16:~>cat /opt/oracle/.ssh/ id_rsa.pub >>/opt/oracle/.ssh/authorized_keys.ria16
oracle@ria15:~>cat /opt/oracle/.ssh/ id_dsa.pub >>/opt/oracle/.ssh/authorized_keys.ria15
oracle@ria15:~>cat /opt/oracle/.ssh/ id_rsa.pub >>/opt/oracle/.ssh/authorized_keys.ria15
oracle@ria15:~>cd /opt/oracle/.ssh
oracle@ria15:~>scp authorized_keys.ria15 ria16:/opt/oracle/.ssh/
oracle@ria15:~>scp ria16: /opt/oracle/.ssh/authorized_keys.ria16 .
oracle@ria15:~>cat .ssh/authorized_keys.ria15 authorized_keys.ria16
oracle@ria15:~>mv authorized_keys.ria16 authorized_keys
oracle@ria15:~>mv authorized_keys.ria15
oracle@ria16:~>cd /opt/oracle/.ssh
oracle@ria16:~>cat .ssh/authorized_keys.ria16 authorized_keys.ria15
oracle@ria16:~>mv authorized_keys.ria15 authorized_keys
oracle@ria16:~>mv authorized_keys.ria16
oracle@ria15:~> date ;ssh ria16 date
Tue Oct 13 14:57:50 CST 2009
Tue Oct 13 14:57:50 CST 2009
oracle@ria15:~> date ;ssh ria16-priv date
Tue Oct 13 14:57:53 CST 2009
Tue Oct 13 14:57:53 CST 2009
oracle@ria15:~> date ;ssh ria15-priv date
Tue Oct 13 14:57:57 CST 2009
Tue Oct 13 14:57:57 CST 2009
oracle@ria15:~> date ;ssh ria15 date
Tue Oct 13 14:58:03 CST 2009
Tue Oct 13 14:58:03 CST 2009
oracle@ria15:~>
四.安装Oracle集群件
下载软件包,在ria15上以oracle用户身份执行(此处未做截图,做文字说明)
$./runInstaller -ignoresysprereqs
1. 欢迎页面:单击Next。
2. 指定Inventory目录和证书:
o 输入inventory目录的完整路径:/opt/oracle/app/oracle/oraInventory。
o 指定操作系统组名:oinstall。
3. 指定Home详细内容:
o /oracle_crs/product/10.2/crs
4. 特定于产品的必要条件检查:
o 忽略有关物理内存要求的警告。
5. 指定集群配置:单击Add。
o 公共节点名称:ria16
o 专用节点名称:ria16-priv
o 虚拟主机名称:ria16-vip
6. 指定网络接口用法:
o 接口名称:eth0
o 子网:192.168.31.0
o 接口类型:Public
o 接口名称:eth1
o 子网:10.10.10.0
o 接口类型:Private
7. 指定Oracle集群注册表(OCR)位置。不镜像表ocr
o 指定OCR位置:/dev/raw/raw1,/dev/raw/raw2
8. 指定表决磁盘位置.不镜像表决磁盘。
o 表决磁盘位置:/dev/raw/raw3,/dev/raw/raw4./dev/raw/raw5
9. 摘要:单击Install。
10. 执行配置脚本:以root用户身份按顺序执行以下脚本(一次执行一个)。在当前脚本完成后,再继续执行下一个脚本。
o 在ria15上执行/u01/app/oracle/oraInventory/orainstRoot.sh。
o 在ria16上执行/u01/app/oracle/oraInventory/orainstRoot.sh。
o 在ria15上执行/u01/app/oracle/product/10.2.0/crs_1/root.sh。
o 执行完上面脚本后修改$ORA_CRS_VASE/bin/vipca和srvctl将export LD_ASSME_KENEL注释掉
o 在ria16上执行/u01/app/oracle/product/10.2.0/crs_1/root.sh。
o 执行完脚本后升级集群软件到10..0.2.04,然后在手动执行vipca
ria16上的root.sh脚本会自动调用VIPCA,但会因为“The given interface(s), "eth0" is not public.Public interfaces should be used to configure virtual IPs.”错误而失败。如果您的公共接口使用不可路由的IP地址(192.168.x.x),则Oracle集群验证实用程序(CVU)将无法找到合适的公共接口。一种解决方法是手动运行VIPCA。在执行vipca前,先升级集群软件
在ria16上执行/u01/app/oracle/product/10.2.0/crs_1/root.sh后,解压集群升级软件
执行./runinstaller –ignoresystemprereqs点击next继续
选择ocr家目录 next
安装模式,集群模式,选择两个节点ria15,ria16,next
检查环境,所有通过,nxet
查看整个设置,如果与设置对不上,back回去修改,next
安装升级软件
升级软件安装成功(下面需要执行脚本,查看please remember,首先以root用户停止crs服务,然后执行脚本
以root用户停止crs服务
第2步要求以root用户执行脚本
在ria15上面执行./root102.sh
执行完脚本后,退出
执行vipca脚本
选择支持的网络接口,都选择上
修改ip alias name修改后如下图所示,next
建立vip等,完成后退出
11. 返回到ria15单击OK。
到此集群软件安装成功
五.安装Oracle数据库10g第2版
下载后,在ria15上以oracle用户身份执行
./runInstaller ignoresysprereqs
选择next
选择enterprise edition (1.24GB),next
选择oracle数据库的家目录,next
选择安装模式,选择集群安装模式,并选择ria15,ria16节点 next
先安装数据库软件,选择install database software only next
查看设置,如果所有设置正确,next
开始安装数据库软件
同步到ria16
安装完成,执行脚本root.sh以root用户在两节点执行,执行完后点ok,完成安装
ria15
ria16执行
升级数据库软件,进入执行./runInataller -ignoresysprereqs
next
选择数据库家目录next
选择升级方式,此处为集群方式,选择两节点 next
检查环境设置 next
next
查看配置情况,无误直接next
开始升级
完成,需要以root用户执行在两节点上面执行root.sh脚本
单击ok完成安装
到此,数据库软件已经成功安装在ria15,ria16上面
六.创建库
以oracle用户执行dbca来创建库
选择集群数据库next
配置asm磁盘选择configure automatic storage managerment next
选择管理asm磁盘节点,select all next
设置密码并选择数据库初始化文件类型如图 next
创建asm实列
创建asm磁盘组create new
创建数据组DATA ,选择Extemal选择前三个磁盘
开始创建asm磁盘组
完成后
继续create new创建磁盘组
磁盘组名FLASHRECOVER
点击finish完成磁盘创建
创建database
选择create a database next
选择节点select all ,next
选择创建数据库类型 next
输入全局数据库名orcl ,next
next
创建密码 next
选择数据存储管理方式 next
next
选择use oracle-managed +DATA next
选择flashrecover next
配置recovery选项如下图265144M=256*1024 next
next
next
配置进程数
设置SGA,PGA信息
配置语言环境
创建数据库next
next
配置信息ok
开始创建数据库
创建完数据库
启动数据库实列
到此处,RAC成功安装