Friday 27 June 2008

Change disks used by an aggregate

Recently we needed to transfer an aggregate that was hosted on an external shelf to disks housed internally in our test FAS2020 and the same trick can be used to move an aggregate from one shelf to another.

The trick uses the disk-replace command. It works serially, and so it is a little time consuming, BUT it retains exactly the same layout on disk, since it's a bit-for-bit copy.

In short the command to use is 'disk replace start < new diskname> e.g. 
 disk replace start 0b.19 0c.00.0

So here's what my disk layout looks like - I am using traditional volumes
Volume vol1 (online, raid_dp) (block checksums)
  Plex /vol1/plex0 (online, normal, active)
    RAID group /vol1/plex0/rg0 (normal)

      RAID Disk Device          HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
      --------- ------          ------------- ---- ---- ---- ----- --------------    --------------
      dparity   0c.00.6         0c    0   6   SA:1   -  SATA  7200 211377/432901760  211977/434130816
      parity    0c.00.1         0c    0   1   SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0b.19           0b    1   3   FC:B   -  ATA   7200 211377/432901760  211921/434014304
      data      0c.00.3         0c    0   3   SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0c.00.11        0c    0   11  SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0c.00.10        0c    0   10  SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0c.00.9         0c    0   9   SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0c.00.8         0c    0   8   SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0c.00.7         0c    0   7   SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0c.00.4         0c    0   4   SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0b.29           0b    1   13  FC:B   -  ATA   7200 211377/432901760  211921/434014304
      data      0c.00.5         0c    0   5   SA:1   -  SATA  7200 211377/432901760  211977/434130816


Spare disks

RAID Disk       Device          HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
---------       ------          ------------- ---- ---- ---- ----- --------------    --------------
Spare disks for block or zoned checksum traditional volumes or aggregates
spare           0a.16           0a    1   0   FC:A   -  ATA   7200 211377/432901760  211921/434014304 (not zeroed)
spare           0a.20           0a    1   4   FC:A   -  ATA   7200 211377/432901760  211921/434014304
spare           0a.22           0a    1   6   FC:A   -  ATA   7200 211377/432901760  211921/434014304
spare           0a.24           0a    1   8   FC:A   -  ATA   7200 211377/432901760  211921/434014304
spare           0a.26           0a    1   10  FC:A   -  ATA   7200 211377/432901760  211921/434014304
spare           0b.17           0b    1   1   FC:B   -  ATA   7200 211377/432901760  211921/434014304 (not zeroed)
spare           0b.25           0b    1   9   FC:B   -  ATA   7200 211377/432901760  211921/434014304
spare           0b.27           0b    1   11  FC:B   -  ATA   7200 211377/432901760  211921/434014304
spare           0c.00.0         0c    0   0   SA:1   -  SATA  7200 211377/432901760  211977/434130816 (not zeroed)

All the disks marked with a 'CHAN' type of FC:A or FC:B are actually SATA drives on the other side of a FC->SATA bridge, and are basically SATA disks in an external shelf. The disks marked SA:1 are SATA drive internal to the FAS2020. In the output above I am part-way through migrating the disks. Some of the disks in my volume/aggregate are internal and some are external. On of the disks that is still on the external shelf is disk 0b.19. Using the command below I will transfer the data on that individual disk to an internal SATA disk (0c.00.0). The command I use is.
 disk replace start 0b.19 0c.00.0

Then I receive a warning messge
*** You are about to copy and replace the following file system disk ***
  Disk /vol1/plex0/rg0/0b.19

      RAID Disk Device          HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
      --------- ------          ------------- ---- ---- ---- ----- --------------    --------------
      data      0b.19           0b    1   3   FC:B   -  ATA   7200 211377/432901760  211921/434014304
***
Really replace disk 0b.19 with 0c.00.0? y
disk replace: Disk 0b.19 was marked for replacing.

The output of sysconfig -r shows something like this
Volume vol1 (online, raid_dp) (block checksums)
  Plex /vol1/plex0 (online, normal, active)
    RAID group /vol1/plex0/rg0 (normal)

      RAID Disk Device          HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
      --------- ------          ------------- ---- ---- ---- ----- --------------    --------------
      dparity   0c.00.6         0c    0   6   SA:1   -  SATA  7200 211377/432901760  211977/434130816
      parity    0c.00.1         0c    0   1   SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0b.19           0b    1   3   FC:B   -  ATA   7200 211377/432901760  211921/434014304 (replacing)
      data      0c.00.3         0c    0   3   SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0c.00.11        0c    0   11  SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0c.00.10        0c    0   10  SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0c.00.9         0c    0   9   SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0c.00.8         0c    0   8   SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0c.00.7         0c    0   7   SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0c.00.4         0c    0   4   SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0b.29           0b    1   13  FC:B   -  ATA   7200 211377/432901760  211921/434014304
      data      0c.00.5         0c    0   5   SA:1   -  SATA  7200 211377/432901760  211977/434130816

And then once the replacement gets under way.. I see my copy under way.
Volume vol1 (online, raid_dp) (block checksums)
  Plex /vol1/plex0 (online, normal, active)
    RAID group /vol1/plex0/rg0 (normal)

      RAID Disk Device          HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
      --------- ------          ------------- ---- ---- ---- ----- --------------    --------------
      dparity   0c.00.6         0c    0   6   SA:1   -  SATA  7200 211377/432901760  211977/434130816
      parity    0c.00.1         0c    0   1   SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0b.19           0b    1   3   FC:B   -  ATA   7200 211377/432901760  211921/434014304 (replacing, copy in progress)
      -> copy   0c.00.0         0c    0   0   SA:1   -  SATA  7200 211377/432901760  211977/434130816 (copy 1% completed)
      data      0c.00.3         0c    0   3   SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0c.00.11        0c    0   11  SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0c.00.10        0c    0   10  SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0c.00.9         0c    0   9   SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0c.00.8         0c    0   8   SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0c.00.7         0c    0   7   SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0c.00.4         0c    0   4   SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0b.29           0b    1   13  FC:B   -  ATA   7200 211377/432901760  211921/434014304
      data      0c.00.5         0c    0   5   SA:1   -  SATA  7200 211377/432901760  211977/434130816

And finally when the copy is done, 0c.00.0 is part of the aggregate/volume and 0b.19 is spare.
Volume vol1 (online, raid_dp) (block checksums)
  Plex /vol1/plex0 (online, normal, active)
    RAID group /vol1/plex0/rg0 (normal)

      RAID Disk Device          HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
      --------- ------          ------------- ---- ---- ---- ----- --------------    --------------
      dparity   0c.00.6         0c    0   6   SA:1   -  SATA  7200 211377/432901760  211977/434130816
      parity    0c.00.1         0c    0   1   SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0c.00.0         0c    0   0   SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0c.00.3         0c    0   3   SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0c.00.11        0c    0   11  SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0c.00.10        0c    0   10  SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0c.00.9         0c    0   9   SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0c.00.8         0c    0   8   SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0c.00.7         0c    0   7   SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0c.00.4         0c    0   4   SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0c.00.2         0c    0   2   SA:1   -  SATA  7200 211377/432901760  211977/434130816
      data      0c.00.5         0c    0   5   SA:1   -  SATA  7200 211377/432901760  211977/434130816


Spare disks

RAID Disk       Device          HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
---------       ------          ------------- ---- ---- ---- ----- --------------    --------------
Spare disks for block or zoned checksum traditional volumes or aggregates
spare           0a.16           0a    1   0   FC:A   -  ATA   7200 211377/432901760  211921/434014304 (not zeroed)
spare           0a.20           0a    1   4   FC:A   -  ATA   7200 211377/432901760  211921/434014304
spare           0a.22           0a    1   6   FC:A   -  ATA   7200 211377/432901760  211921/434014304
spare           0a.24           0a    1   8   FC:A   -  ATA   7200 211377/432901760  211921/434014304
spare           0a.26           0a    1   10  FC:A   -  ATA   7200 211377/432901760  211921/434014304
spare           0b.17           0b    1   1   FC:B   -  ATA   7200 211377/432901760  211921/434014304 (not zeroed)
spare           0b.19           0b    1   3   FC:B   -  ATA   7200 211377/432901760  211921/434014304 (not zeroed)
spare           0b.25           0b    1   9   FC:B   -  ATA   7200 211377/432901760  211921/434014304
spare           0b.27           0b    1   11  FC:B   -  ATA   7200 211377/432901760  211921/434014304
spare           0b.29           0b    1   13  FC:B   -  ATA   7200 211377/432901760  211921/434014304 (not zeroed)
Posted by at
Labels: , , , ,

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)