在拷900G大数据时出现如下错误;


客户端产生的错误日志:文件写不进去;查看rsync问题,没有解决


day_140121/20140121_0c9cb0e43a3ee1cd57bd492YbauZrxQ8.jpg
      32768  25%   15.62MB/s    0:00:00
rsync: writefd_unbuffered failed to write 4 bytes to socket [sender]: Broken pipe (32)
rsync: close failed on "/day_140121/.20140121_09389f3cdfa66f9a3a73rMgZU2MLNTo3.png.V65wiu": Input/output error (5)
rsync error: error in file IO (code 11) at receiver.c(730) [receiver=3.0.6]
rsync: connection unexpectedly closed (1434 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(600) [sender=3.0.6]
tail: /tmp/a:文件已截断
day_131019/20131019_e6e35aa3326cdbb0bd19YDtGdzS8ay36_120x120.jpg
       7843 100%    0.00kB/s    0:00:00 (xfer#1, to-check=1031/135274)
rsync: close failed on "/day_131019/.20131019_e6e35aa3326cdbb0bd19YDtGdzS8ay36_120x120.jpg.AKrGPn": Input/output error (5)
rsync error: error in file IO (code 11) at receiver.c(730) [receiver=3.0.6]
rsync: connection unexpectedly closed (1673 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(600) [sender=3.0.6]
tail: /tmp/a:文件已截断    



mfsmaster端错误日志,提示分配了元数据,但是块服务器没找到块的地址;


Jan 24 13:42:13 to8tobbs_master1 mfsmaster[7458]: (192.168.2.111:9422) chunk: 0000000000345B30 creation status: IO error
Jan 24 13:42:13 to8tobbs_master1 mfsmaster[7458]: (192.168.2.185:9422) chunk: 0000000000345B30 creation status: IO error
Jan 24 13:43:39 to8tobbs_master1 mfsmaster[7458]: chunk 0000000000345B30 has only invalid copies (2) - please repair it manually
Jan 24 13:43:39 to8tobbs_master1 mfsmaster[7458]: chunk 0000000000345B30_00000001 - invalid copy on (192.168.2.185 - ver:00000000)
Jan 24 13:43:39 to8tobbs_master1 mfsmaster[7458]: chunk 0000000000345B30_00000001 - invalid copy on (192.168.2.111 - ver:00000000)
Jan 24 13:43:42 to8tobbs_master1 mfsmaster[7458]: chunk 0000000000345B2F has only invalid copies (2) - please repair it manually
Jan 24 13:43:42 to8tobbs_master1 mfsmaster[7458]: chunk 0000000000345B2F_00000001 - invalid copy on (192.168.2.185 - ver:00000000)


在块服务器的日志里提示如下错误:


ERROR:No space left on device

到google.com里查了下,说是磁盘inode是文件的索引节点不够用了;


查看块服务器的磁盘inode

df -i

全部用完了;

只好重新格式化磁盘,设置足够用的inode节点;重拷数据;


这里大家在部署mfs分布式文件系统时,块服务器的磁盘inode一定要设置清楚;

[root@web1 140123]# df -i
文件系统              Inode  已用(I)  可用(I) 已用(I)%% 挂载点
/dev/sda3            6766592  699208 6067384   11% /
tmpfs                6172064       1 6172063    1% /dev/shm
/dev/sda1              51200      44   51156    1% /boot
/dev/sdb1            122028032 15234160 106793872   13% /img
192.168.3.62:9421    1014750381 13744354 1001006027    2% /img_pic


这里我原来/img_pic的磁盘inode只有3456125一下子用完了,大于2T的磁盘分区大多数用于视频数据,所以默认的inode节点很小的,大家要注意下自己的数据是小文件多的情况,就要关注下inode;


[root@web1 140123]# df -h
文件系统              容量  已用  可用 已用%% 挂载点
/dev/sda3             102G   54G   43G  56% /
tmpfs                  24G     0   24G   0% /dev/shm
/dev/sda1             194M   45M  140M  25% /boot
/dev/sdb1             1.8T  900G  841G  52% /img
192.168.3.62:9421     6.2T  1.7T  4.6T  27% /img_pic