生产环境LVM卷ext4文件系统故障修复处理

admin管理员组

文章数量:1535110

一、问题描述

某项目因存储视频流泪数据，数据量较大，生产环境当时已达158TB，采用LVM+Ext4存储，在某次LVM在线扩容过程中，扩容失败，报错：inode_counter 溢出，从字面看就i节点数量超过最大限制了，被lvresize命令忽略，报出警告：Invalid argument While checking for on-line resizing support

LVM利用Linux内核的device-mapper功能来实现存储系统的虚拟化（系统分区独立于底层硬件）。 通过LVM，你可以实现存储空间的抽象化并在上面建立虚拟分区（virtual partitions），可以更简便地扩大和缩小分区，可以增删分区时无需担心某个硬盘上没有足够的连续空间，避免为正在使用的磁盘重新分区的麻烦、为调整分区而不得不移动其他分区的不便。

二、分析处理过程

1、lvreduce后LVM逻辑卷分区已损坏

#lvreduce过程，下面为一个网络示例
lvreduce -L 20g /dev/VG01/var   //输出如下，注意这里是缩容到20G不是缩容20G；-L：指定逻辑卷的大小，单位为“kKmMgGtT”字节；-l：指定逻辑卷的大小（LE数）。

  WARNING: Reducing active and open logical volume to 20.00 GiB.
  THIS MAY DESTROY YOUR DATA (filesystem etc.)
Do you really want to reduce VG01/var? [y/n]: y  //需要确认
  Size of logical volume VG01/var changed from 31.50 GiB (8065 extents) to 20.00 GiB (5120 extents).
  Logical volume VG01/var successfully resized.
ll /etc/lvm/archive #查看最近一次修改vgs的备份
vgcfgrestore VG名称 --test -f /etc/lvm/archive/恢复的时间的备份.vg  //--test测试恢复，没有报错属于正常,如果可恢复的话，可恢复到修改前的vgs大小
vgcfgrestore sihua2 --test -f /etc/lvm/archive/sihua2_00008-211645777.vg

  TEST MODE: Metadata will NOT be updated and volumes will not be (de)activated.
  Restored volume group sihua2
vgcfgrestore sihua2 -f /etc/lvm/archive/sihua2_00008-211645777.vg

  Restored volume group sihua2  //现场验证恢复后，文件系统还是错误的，执行下列命令未成功修复，怀疑会对文件系统造成二次危害，仅用于测试环境验证

mount /dev/sihua2/video2 /opt/ceshi2  //报错如下
mount: /opt/ceshi2: wrong fs type, bad option, bad superblock on /dev/mapper/sihua2-video2, missing codepage or helper program, or other error.

e2fsck -fy /dev/sihua2/video2		//报错如下//报错如下
e2fsck 1.44.1 (24-Mar-2018)
The filesystem size (according to the superblock) is 456340480 blocks
The physical size of the device is 2883584 blocks
Either the superblock or the partition table is likely to be corrupt!
Abort? yes

//后经测试：可检查lvm下的执行lvreuce前的备份，可直接less
# Generated by LVM2 version 2.02.187(2)-RHEL7 (2020-03-24): Mon Dec 12 22:18:01 2022

contents = "Text Format Volume Group"
version = 1
#下面这行可以
description = "Created *before* executing 'lvreduce -L -5G /dev/mapper/sihua-video'"

creation_host = "zq-sihua-63-cms"       # Linux zq-sihua-63-cms 3.10.0-1160.25.1.el7.x86_64 #1 SMP Wed Apr 28 21:49:45 UTC 2021 x86_64
creation_time = 1670854681      # Mon Dec 12 22:18:01 2022

sihua {
   
        id = "pdUxaK-jZ3G-nhJX-Fevl-uw4n-Vbc6-i5WO2O"
        seqno = 11
        format = "lvm2"                 # informational
        status = ["RESIZEABLE", "READ", "WRITE"]
        flags = []
        extent_size = 8192              # 4 Megabytes
        max_lv = 0
        max_pv = 0
        metadata_copies = 0

        physical_volumes {
   

                pv0 {
   
                        id = "EyYE8P-c2xr-20KA-XR0d-m8Ix-rcNg-Ha9wIX"
                        device = "/dev/sdc1"    # Hint only

                        status = ["ALLOCATABLE"]
                        flags = []
                        dev_size = 2147481600   # 1024 Gigabytes
                        pe_start = 2048
                        pe_count = 262143       # 1024 Gigabytes
                }
……
pv9 {
   
                        id = "JxXiat-id6Z-l96J-beUT-dVVo-d5VN-r32j6M"
                        device = "/dev/sdl1"    # Hint only

                        status = ["ALLOCATABLE"]
                        flags = []
                        dev_size = 68719476669  # 32 Terabytes
                        pe_start = 2048
                        pe_count = 8388607      # 32 Terabytes
                }
        }

        logical_volumes {
   

                video {
   
                        id = "Yo2vL7-yysC-YkHe-fkvW-P1hR-YcHE-Cmv7tI"
                        status = ["READ", "WRITE", "VISIBLE"]
                        flags = []
                        creation_time = 1657971739      # 2022-07-16 19:42:19 +0800
                        creation_host = "zq-sihua-63-cms"
                        segment_count = 10

                        segment1 {
   
…………
						segment10 {
   
                                start_extent = 59244535
                                extent_count = 8388607  # 32 Terabytes

                                type = "striped"
                                stripe_count 

本文标签： 系统故障环境文件lvm