理解inode

文件储存在硬盘上,硬盘的最小存储单位叫做"扇区" Sector,每个扇区储存512字节(相当于0.5KB
操作系统读取硬盘的时候,不会一个个扇区地读取,这样效率太低,而是一次性连续读取多个扇区,即一次性读取一个"块" block,这种由多个扇区组成的"块",是文件存取的最小单位。"块"的大小,最常见的是4KB,即连续八个sector组成一个block 文件数据都储存在"块"中,必须有一个地方储存文件的元信息,比如文件的创建者、文件的创建日期、文件的大小等等。这种储存文件元信息的区域就叫做inode,中文译名为"索引节点"。

环境说明

[root@C71 ~]# cat /etc/redhat-release 
CentOS Linux release 7.6.1810 (Core) 
[root@C71 ~]# uname -a
Linux C71.wcy 3.10.0-957.21.3.el7.x86_64 #1 SMP Tue Jun 18 16:35:19 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

但文件系统还有剩余空间,却无法创建文件得时候,可能为inode满了,可以df -i检查一下

[root@C71 ~]# df -hT
文件系统                类型      容量  已用  可用 已用% 挂载点
/dev/mapper/centos-root xfs       126G  2.2G  124G    2% /
devtmpfs                devtmpfs  1.9G     0  1.9G    0% /dev
tmpfs                   tmpfs     1.9G     0  1.9G    0% /dev/shm
tmpfs                   tmpfs     1.9G   12M  1.9G    1% /run
tmpfs                   tmpfs     1.9G     0  1.9G    0% /sys/fs/cgroup
/dev/sda1               xfs       497M  179M  318M   37% /boot
tmpfs                   tmpfs     378M     0  378M    0% /run/user/0
/dev/mapper/dfi-dfi--lv ext4      4.8G   28M  4.6G    1% /dfi
[root@C71 ~]# 
[root@C71 ~]# df -i
文件系统                   Inode 已用(I)  可用(I) 已用(I)% 挂载点
/dev/mapper/centos-root 65800192   60438 65739754       1% /
devtmpfs                  479688     364   479324       1% /dev
tmpfs                     482689       1   482688       1% /dev/shm
tmpfs                     482689    1232   481457       1% /run
tmpfs                     482689      16   482673       1% /sys/fs/cgroup
/dev/sda1                 256000     335   255665       1% /boot
tmpfs                     482689       1   482688       1% /run/user/0
/dev/mapper/dfi-dfi--lv   327680  327680        0     100% /dfi

表现为

[root@C71 ~]# touch /dfi/test.txt 
touch: 无法创建"/dfi/test.txt": 设备上没有空间

更改ext4文件系统为xfs

  • 因为ext4无法动态调整inode空间占比,所以选择将ext4改为xfs(支持动态调整inode空间占比)
    备份文件,将ext4文件系统转为xfs
[root@C71 ~]# cp -r /dfi /backup
[root@C71 ~]# umount /dev/dfi/dfi-lv  
[root@C71 ~]# df -hT
文件系统                类型      容量  已用  可用 已用% 挂载点
/dev/mapper/centos-root xfs       126G  2.4G  124G    2% /
devtmpfs                devtmpfs  1.9G     0  1.9G    0% /dev
tmpfs                   tmpfs     1.9G     0  1.9G    0% /dev/shm
tmpfs                   tmpfs     1.9G   12M  1.9G    1% /run
tmpfs                   tmpfs     1.9G     0  1.9G    0% /sys/fs/cgroup
/dev/sda1               xfs       497M  179M  318M   37% /boot
tmpfs                   tmpfs     378M     0  378M    0% /run/user/0

如果遇到卸载不掉得情况,可以fuser强制删除用户关联

[root@C71 ~]# umount /dev/dfi/dfi-lv 
umount: /dfi:目标忙。
        (有些情况下通过 lsof(8) 或 fuser(1) 可以
         找到有关使用该设备的进程的有用信息)
[root@C71 ~]# fuser -m -k /dev/dfi/dfi-lv 
/dev/dm-1:            1834c
[root@C71 ~]# umount /dev/dfi/dfi-lv

删除lvs逻辑卷

[root@demo1 ~]# vgs
  VG     #PV #LV #SN Attr   VSize   VFree
  centos   1   1   0 wz--n- <25.51g    0 
  dfivg    1   1   0 wz--n-  <5.00g    0 
[root@demo1 ~]# lvs
  LV     VG     Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  root   centos -wi-ao---- <25.51g                                                    
  dfi-lv dfivg  -wi-a-----  <5.00g                                                    
[root@demo1 ~]# pvs
  PV         VG     Fmt  Attr PSize   PFree
  /dev/sda3  centos lvm2 a--  <25.51g    0 
  /dev/sdb   dfivg  lvm2 a--   <5.00g    0 
[root@demo1 ~]# vgremove dfivg
Do you really want to remove volume group "dfivg" containing 1 logical volumes? [y/n]: y
Do you really want to remove active logical volume dfivg/dfi-lv? [y/n]: y
  Logical volume "dfi-lv" successfully removed
  Volume group "dfivg" successfully removed
[root@demo1 ~]# pvremove /dev/sdb
  Labels on physical volume "/dev/sdb" successfully wiped.
[root@demo1 ~]# vgs
  VG     #PV #LV #SN Attr   VSize   VFree
  centos   1   1   0 wz--n- <25.51g    0 
[root@demo1 ~]# lvs
  LV   VG     Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  root centos -wi-ao---- <25.51g                                                    
[root@demo1 ~]# pvs
  PV         VG     Fmt  Attr PSize   PFree
  /dev/sda3  centos lvm2 a--  <25.51g    0

重新分区,根据提示,依次输入"mktable","gpt","y","mkpart","minio","ext4","0","100%","ignore","p","q"

[root@C71 ~]# parted /dev/sdb
GNU Parted 3.1
使用 /dev/sdb
Welcome to GNU Parted! Type 'help' to view a list of commands.                              
(parted) mktable                                                      
新的磁盘标签类型? gpt                                                    
警告: The existing disk label on /dev/sdb will be destroyed and all data on this disk will be lost. Do you want to continue?
是/Yes/否/No? y                                                           
(parted)                                                                  
(parted) mkpart                                                           
分区名称?  []? dfi                                                       
文件系统类型?  [ext2]? ext4                                              
起始点? 0                                                                
结束点? 100%                                                             
警告: The resulting partition is not properly aligned for best performance.
忽略/Ignore/放弃/Cancel? Ignore                                           
(parted) p                                                                
Model: VMware, VMware Virtual S (scsi)
Disk /dev/sdb: 5369MB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags: 

Number  Start   End     Size    File system  Name  标志
 1      17.4kB  5369MB  5369MB               dfi

(parted) q                                                                
信息: You may need to update /etc/fstab.

格式化磁盘使用make.xfs,可加-f参数强制格式化

[root@demo1 ~]# mkfs.xfs /dev/sdb1 
meta-data=/dev/sdb1              isize=512    agcount=4, agsize=327678 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=0, sparse=0
data     =                       bsize=4096   blocks=1310711, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
log      =internal log           bsize=4096   blocks=2560, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
[root@demo1 ~]# mount /dev/sdb1 /dfi/

恢复挂载后查看inode使用情况

[root@demo1 ~]# df -i
文件系统                   Inode 已用(I)  可用(I) 已用(I)% 挂载点
/dev/mapper/centos-root 13373440  384910 12988530       3% /
devtmpfs                  479687     355   479332       1% /dev
tmpfs                     482688       1   482687       1% /dev/shm
tmpfs                     482688    1213   481475       1% /run
tmpfs                     482688      16   482672       1% /sys/fs/cgroup
/dev/sda1                 256000     335   255665       1% /boot
tmpfs                     482688       1   482687       1% /run/user/0
/dev/sdb1                2621376       3  2621373       1% /dfi

查看/dfi目录inode空间占比,可以看到占比imaxpct=25

[root@demo1 ~]# xfs_info /dfi/
meta-data=/dev/sdb1              isize=512    agcount=4, agsize=327678 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=0 spinodes=0
data     =                       bsize=4096   blocks=1310711, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
log      =internal               bsize=4096   blocks=2560, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

动态扩容inode

xfs文件系统动态扩容inode空间占比为30%(后续可以根据存储情况调整)

[root@demo1 ~]# xfs_growfs -m 30 /dfi/
meta-data=/dev/sdb1              isize=512    agcount=4, agsize=327678 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=0 spinodes=0
data     =                       bsize=4096   blocks=1310711, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
log      =internal               bsize=4096   blocks=2560, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
inode max percent changed from 25 to 30

查看inode,对比之前可用2621373,增加524328个

[root@demo1 ~]# df -i
文件系统                   Inode 已用(I)  可用(I) 已用(I)% 挂载点
/dev/mapper/centos-root 13373440  384910 12988530       3% /
devtmpfs                  479687     355   479332       1% /dev
tmpfs                     482688       1   482687       1% /dev/shm
tmpfs                     482688    1222   481466       1% /run
tmpfs                     482688      16   482672       1% /sys/fs/cgroup
/dev/sda1                 256000     335   255665       1% /boot
tmpfs                     482688       1   482687       1% /run/user/0
/dev/sdb1                3145704       3  3145701       1% /dfi

恢复备份文件

[root@demo1 ~]# cp -r /backup/ /dfi/
[root@demo1 ~]# df -i
文件系统                   Inode 已用(I)  可用(I) 已用(I)% 挂载点
/dev/mapper/centos-root 13373440  385039 12988401       3% /
devtmpfs                  479687     355   479332       1% /dev
tmpfs                     482688       1   482687       1% /dev/shm
tmpfs                     482688    1222   481466       1% /run
tmpfs                     482688      16   482672       1% /sys/fs/cgroup
/dev/sda1                 256000     335   255665       1% /boot
tmpfs                     482688       1   482687       1% /run/user/0
/dev/sdb1                3145704  327674  2818030      11% /dfi
[root@demo1 ~]# df -i -kh /dfi/
文件系统       Inode 已用(I) 可用(I) 已用(I)% 挂载点
/dev/sdb1       3.0M    320K    2.7M      11% /dfi

恢复文件系统为LVS逻辑卷

[root@demo1 ~]# umount /dfi/
[root@demo1 ~]# df -h
文件系统                 容量  已用  可用 已用% 挂载点
/dev/mapper/centos-root   26G  2.1G   24G    8% /
devtmpfs                 1.9G     0  1.9G    0% /dev
tmpfs                    1.9G     0  1.9G    0% /dev/shm
tmpfs                    1.9G   12M  1.9G    1% /run
tmpfs                    1.9G     0  1.9G    0% /sys/fs/cgroup
/dev/sda1                497M  179M  318M   37% /boot
tmpfs                    378M     0  378M    0% /run/user/0
[root@demo1 ~]# parted /dev/sdb
GNU Parted 3.1
使用 /dev/sdb
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) mktable                                                          
新的磁盘标签类型? gpt                                                    
警告: The existing disk label on /dev/sdb will be destroyed and all data on this disk will be lost. Do you want to continue?
是/Yes/否/No? y                                                           
(parted) p                                                                
Model: VMware, VMware Virtual S (scsi)
Disk /dev/sdb: 5369MB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags: 

Number  Start  End  Size  File system  Name  标志

(parted) q                                                                
信息: You may need to update /etc/fstab.
[root@demo1 ~]# lsblk 
NAME            MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda               8:0    0   30G  0 disk 
├─sda1            8:1    0  500M  0 part /boot
├─sda2            8:2    0    4G  0 part [SWAP]
└─sda3            8:3    0 25.5G  0 part 
  └─centos-root 253:0    0 25.5G  0 lvm  /
sdb               8:16   0    5G  0 disk 
[root@demo1 ~]# pvcreate /dev/sdb
  Device /dev/sdb excluded by a filter.

导致问题的原因是添加的磁盘是在虚拟机中已经有了分区表,现在的虚拟机并不能识别磁盘的分区表,运行parted命令重做分区表

[root@demo1 ~]# parted /dev/sdb
GNU Parted 3.1
使用 /dev/sdb
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) mklabel msdos
警告: The existing disk label on /dev/sdb will be destroyed and all data on this disk will be lost. Do you want to
continue?
是/Yes/否/No? y                                                           
(parted) q                                                                
信息: You may need to update /etc/fstab.

[root@demo1 ~]# pvcreate /dev/sdb                                         
WARNING: dos signature detected on /dev/sdb at offset 510. Wipe it? [y/n]: y
  Wiping dos signature on /dev/sdb.
  Physical volume "/dev/sdb" successfully created.
[root@demo1 ~]# vgcreate dfivg /dev/sdb
  Volume group "dfivg" successfully created
[root@demo1 ~]# lvcreate -l +100%FREE -n dfi-lv dfivg
  Logical volume "dfi-lv" created.
[root@demo1 ~]# vgs
  VG     #PV #LV #SN Attr   VSize   VFree
  centos   1   1   0 wz--n- <25.51g    0 
  dfivg    1   1   0 wz--n-  <5.00g    0 
[root@demo1 ~]# pvs
  PV         VG     Fmt  Attr PSize   PFree
  /dev/sda3  centos lvm2 a--  <25.51g    0 
  /dev/sdb   dfivg  lvm2 a--   <5.00g    0 
[root@demo1 ~]# lvs
  LV     VG     Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  root   centos -wi-ao---- <25.51g                                                    
  dfi-lv dfivg  -wi-a-----  <5.00g                                                    
[root@demo1 ~]# mkfs.xfs /dev/dfivg/dfi-lv 
meta-data=/dev/dfivg/dfi-lv      isize=512    agcount=4, agsize=327424 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=0, sparse=0
data     =                       bsize=4096   blocks=1309696, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
log      =internal log           bsize=4096   blocks=2560, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
[root@demo1 ~]# mount /dfi/
mount: 在 /etc/fstab 中找不到 /dfi/
[root@demo1 ~]# mount /dev/dfivg/dfi-lv /dfi/
[root@demo1 ~]# df -hT
文件系统                  类型      容量  已用  可用 已用% 挂载点
/dev/mapper/centos-root   xfs        26G  2.1G   24G    8% /
devtmpfs                  devtmpfs  1.9G     0  1.9G    0% /dev
tmpfs                     tmpfs     1.9G     0  1.9G    0% /dev/shm
tmpfs                     tmpfs     1.9G   12M  1.9G    1% /run
tmpfs                     tmpfs     1.9G     0  1.9G    0% /sys/fs/cgroup
/dev/sda1                 xfs       497M  179M  318M   37% /boot
tmpfs                     tmpfs     378M     0  378M    0% /run/user/0
/dev/mapper/dfivg-dfi--lv xfs       5.0G   33M  5.0G    1% /dfi
[root@demo1 ~]# df -i
文件系统                     Inode 已用(I)  可用(I) 已用(I)% 挂载点
/dev/mapper/centos-root   13373440  385042 12988398       3% /
devtmpfs                    479687     356   479331       1% /dev
tmpfs                       482688       1   482687       1% /dev/shm
tmpfs                       482688    1226   481462       1% /run
tmpfs                       482688      16   482672       1% /sys/fs/cgroup
/dev/sda1                   256000     335   255665       1% /boot
tmpfs                       482688       1   482687       1% /run/user/0
/dev/mapper/dfivg-dfi--lv  2619392       3  2619389       1% /dfi
[root@demo1 ~]# xfs_info /dfi/
meta-data=/dev/mapper/dfivg-dfi--lv isize=512    agcount=4, agsize=327424 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=0 spinodes=0
data     =                       bsize=4096   blocks=1309696, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
log      =internal               bsize=4096   blocks=2560, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
[root@demo1 ~]# xfs_growfs -m 30 /dfi/
meta-data=/dev/mapper/dfivg-dfi--lv isize=512    agcount=4, agsize=327424 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=0 spinodes=0
data     =                       bsize=4096   blocks=1309696, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
log      =internal               bsize=4096   blocks=2560, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
inode max percent changed from 25 to 30
[root@demo1 ~]# xfs_info /dfi/
meta-data=/dev/mapper/dfivg-dfi--lv isize=512    agcount=4, agsize=327424 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=0 spinodes=0
data     =                       bsize=4096   blocks=1309696, imaxpct=30
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
log      =internal               bsize=4096   blocks=2560, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
[root@demo1 ~]# df -i
文件系统                     Inode 已用(I)  可用(I) 已用(I)% 挂载点
/dev/mapper/centos-root   13373440  385042 12988398       3% /
devtmpfs                    479687     356   479331       1% /dev
tmpfs                       482688       1   482687       1% /dev/shm
tmpfs                       482688    1226   481462       1% /run
tmpfs                       482688      16   482672       1% /sys/fs/cgroup
/dev/sda1                   256000     335   255665       1% /boot
tmpfs                       482688       1   482687       1% /run/user/0
/dev/mapper/dfivg-dfi--lv  3143264       3  3143261       1% /dfi