LVM on Linux Tutorial: Adding Hard Drives For More Space

LVM (Logical Volume Management) on Linux is quite handy for dynamically adding space to a Linux system, often times while the system is still up and running.

This is a tutorial (with accompanying video) of me adding three hard drives to a system, combining them into a volume group, then creating virtual disks a.k.a. logical volumes, and then making them into filesystems.

LVM on Linux diagram

Typographical Conventions

BOLD: command line prompt
BOLD BLUE: Things to type
BOLD RED: Things outputted to look for

The basics of LVM:

Physical Hard Disk: A physical hard disk is added to the system

PV (Physical Volume): You “prep” it for LVM use by presenting it as a PV (physical volume)

VG (Volume Group): A collection of one or more PVs

LV (Logical Volume): An amount of disk space carved out from a VG

A Video “LVM on Linux” Tutorial

For those that like video better

A Written “LVM On Linux” Tutorial

The operating system is CentOS 6.3 (a.k.a. Redhat 6.3)

Platform is Virtual machine for easier screen recording

I have one disk for OS (/dev/sda) and have just inserted three more disks of size 7gig, 8gig, 9gig.

(Scary huh? ( because 7 8 9) haha :) )

So the first neat little trick is to see if Linux will see the disks without rebooting. This has worked for me about 85-90% of the time for both VM and physical platforms (and for both drives added to chassis or added via external storage)

You need to tell linux to rescan its scsi channels

Currently “fdisk -l” only sees one disk (/dev/sda) (which by default of OS install is created into LVM)


Sponsored Links



[root@centos-testnode ~]# fdisk -l

Disk /dev/sda: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000bda82

Device Boot Start End Blocks Id System
/dev/sda1 * 1 64 512000 83 Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2 64 2611 20458496 8e Linux LVM

Disk /dev/mapper/vg_centostestnode-lv_root: 16.8 GB, 16819159040 bytes
255 heads, 63 sectors/track, 2044 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/mapper/vg_centostestnode-lv_swap: 4127 MB, 4127195136 bytes
255 heads, 63 sectors/track, 501 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

[root@centos-testnode ~]#

Now we use the directory /sys/class/scsi_host to rescan our scsi bus. You will see multiple “host[N]” directories. One of these is your scsi/fiber card that talks to the disks. You can spend time figuring it out or just tell all three to rescan.

Use thie command “echo “- – -” > host[N]/scan”

[root@centos-testnode ~]# cd /sys/class/scsi_host
[root@centos-testnode scsi_host]# ls
host0 host1 host2
[root@centos-testnode scsi_host]# echo "- - -" > host0/scan
[root@centos-testnode scsi_host]# echo "- - -" > host1/scan
[root@centos-testnode scsi_host]# echo "- - -" > host2/scan
[root@centos-testnode scsi_host]#

Using “fdisk -l” should see the new disks, if not then a reboot is required

[root@centos-testnode scsi_host]# fdisk -l

Disk /dev/sda: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000bda82

Device Boot Start End Blocks Id System
/dev/sda1 * 1 64 512000 83 Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2 64 2611 20458496 8e Linux LVM

Disk /dev/mapper/vg_centostestnode-lv_root: 16.8 GB, 16819159040 bytes
255 heads, 63 sectors/track, 2044 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/mapper/vg_centostestnode-lv_swap: 4127 MB, 4127195136 bytes
255 heads, 63 sectors/track, 501 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/sdb: 7516 MB, 7516192768 bytes
255 heads, 63 sectors/track, 913 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/sdc: 8589 MB, 8589934592 bytes
255 heads, 63 sectors/track, 1044 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/sdd: 9663 MB, 9663676416 bytes
255 heads, 63 sectors/track, 1174 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

[root@centos-testnode scsi_host]#

We can do a “pvdisplay” first and we will see only the one disk.


[root@centos-testnode scsi_host]# pvdisplay
--- Physical volume ---
PV Name /dev/sda2
VG Name vg_centostestnode
PV Size 19.51 GiB / not usable 3.00 MiB
Allocatable yes (but full)
PE Size 4.00 MiB
Total PE 4994
Free PE 0
Allocated PE 4994
PV UUID WEN4mp-vBj0-gMv1-3XF9-guJy-26DF-7SZbLn

[root@centos-testnode scsi_host]#

So to prep physical disks for LVM use need to use “pvcreate” command. Then do “pvdisplay” to check the results.


[root@centos-testnode scsi_host]# pvcreate /dev/sdb /dev/sdc /dev/sdd
Writing physical volume data to disk "/dev/sdb"
Physical volume "/dev/sdb" successfully created
Writing physical volume data to disk "/dev/sdc"
Physical volume "/dev/sdc" successfully created
Writing physical volume data to disk "/dev/sdd"
Physical volume "/dev/sdd" successfully created
[root@centos-testnode scsi_host]# pvdisplay
--- Physical volume ---
PV Name /dev/sda2
VG Name vg_centostestnode
PV Size 19.51 GiB / not usable 3.00 MiB
Allocatable yes (but full)
PE Size 4.00 MiB
Total PE 4994
Free PE 0
Allocated PE 4994
PV UUID WEN4mp-vBj0-gMv1-3XF9-guJy-26DF-7SZbLn

"/dev/sdb" is a new physical volume of "7.00 GiB"
--- NEW Physical volume ---
PV Name /dev/sdb
VG Name
PV Size 7.00 GiB
Allocatable NO
PE Size 0
Total PE 0
Free PE 0
Allocated PE 0
PV UUID kvLPdc-HP94-KSIh-rV2Y-v8n3-8JWd-4zOg7x

"/dev/sdc" is a new physical volume of "8.00 GiB"
--- NEW Physical volume ---
PV Name /dev/sdc
VG Name
PV Size 8.00 GiB
Allocatable NO
PE Size 0
Total PE 0
Free PE 0
Allocated PE 0
PV UUID GfBx3K-q1MK-mdSs-ZjgS-6Ppj-nHqL-WSNN1w

"/dev/sdd" is a new physical volume of "9.00 GiB"
--- NEW Physical volume ---
PV Name /dev/sdd
VG Name
PV Size 9.00 GiB
Allocatable NO
PE Size 0
Total PE 0
Free PE 0
Allocated PE 0
PV UUID BXcukd-4T0G-onOs-QrOc-dQok-eqau-XS1w1q

[root@centos-testnode scsi_host]#

Now let’s create a VG out of all three disks using “vgcreate” command. The name of VG is arbitrary. The command is “vgcreate [name of vg] [pv devices]” and you can use “vgdisplay” to see the results.

In the video I used all three PVs in my “vgcreate”. I thought it would be useful to also show how to use “vgextend” in case you wanted to add a disk to an existing vg. So I will use the “vgcreate” with the first two PVs, and a “vgextend” with the third PV.



[root@centos-testnode ~]# vgcreate geekswingvg /dev/sdb /dev/sdc

Volume group "geekswingvg" successfully created
[root@centos-testnode ~]# vgextend geekswingvg /dev/sdd

Volume group "geekswingvg" successfully extended
[root@centos-testnode ~]# vgdisplay geekswingvg
--- Volume group ---
VG Name geekswingvg
System ID
Format lvm2
Metadata Areas 3
Metadata Sequence No 1
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 0
Open LV 0
Max PV 0
Cur PV 3
Act PV 3
VG Size 23.99 GiB
PE Size 4.00 MiB
Total PE 6141
Alloc PE / Size 0 / 0
Free PE / Size 6141 / 23.99 GiB
VG UUID h0aHSW-j0GP-iBI0-heMS-haMn-7vK7-kv3Dux

[root@centos-testnode ~]#

A few things to notice:


Sponsored Links



  • You’ll see the VG size is about 24Gigabytes (which is 7 + 8 + 9).
  • The PE size (physical extent) is about 4mb. Think of a PE like a block of a disk.
  • The Total PE = 6141 (6141 * 4mb = about 24gig)If you do not specify a PE size, the OS will pick one it thinks is best.
  • The allocated PE = 0 (none have been used yet)
  • The Free PE = 6141

Two common ways to create LVs is to specify a size (usually in gigabytes)
or specify the number of PE. I’ll go ahead and do both. I’ll create three LVs, the first two 5gig and 10gig by specifying the LV size. The third I will create using the rest of the disk available (specifying the number of PEs)

The commands to use:

“lvcreate –name [name of lv] –size [N]G [vgname]”

or

“lvcreate –name [name of lv] –extents [number of PE] [vgname]”

[root@centos-testnode ~]# lvcreate --name geekswinglv1 --size 5G geekswingvg
Logical volume "geekswinglv1" created
[root@centos-testnode ~]# lvcreate --name geekswinglv2 --size 10G geekswingvg
Logical volume "geekswinglv2" created
[root@centos-testnode ~]#

Using vgdisplay we can see how many allocated and how many free PE there are.

[root@centos-testnode ~]# vgdisplay geekswingvg
--- Volume group ---
VG Name geekswingvg
System ID
Format lvm2
Metadata Areas 3
Metadata Sequence No 3
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 2
Open LV 0
Max PV 0
Cur PV 3
Act PV 3
VG Size 23.99 GiB
PE Size 4.00 MiB
Total PE 6141
Alloc PE / Size 3840 / 15.00 GiB
Free PE / Size 2301 / 8.99 GiB
VG UUID h0aHSW-j0GP-iBI0-heMS-haMn-7vK7-kv3Dux

[root@centos-testnode ~]#

In are case there are 2301 PEs left. I’ll create the third LV with 2301 PEs.

[root@centos-testnode ~]# lvcreate --name geekswinglv3 --extents 2301 geekswingvg
Logical volume "geekswinglv3" created
[root@centos-testnode ~]#

Use the “lvdisplay” command to see the LVs you created on your VG

[root@centos-testnode ~]# lvdisplay geekswingvg
--- Logical volume ---
LV Path /dev/geekswingvg/geekswinglv1
LV Name geekswinglv1
VG Name geekswingvg
LV UUID dbNKv8-6PeV-59W7-80Mq-Ij0e-EBAy-pTfMYk
LV Write Access read/write
LV Creation host, time centos-testnode, 2013-02-08 20:52:51 -0500
LV Status available
# open 0
LV Size 5.00 GiB
Current LE 1280
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:2

--- Logical volume ---
LV Path /dev/geekswingvg/geekswinglv2
LV Name geekswinglv2
VG Name geekswingvg
LV UUID emVaGX-7Xoj-gJCs-Gr23-lkTX-kHWT-u0koBb
LV Write Access read/write
LV Creation host, time centos-testnode, 2013-02-08 20:52:59 -0500
LV Status available
# open 0
LV Size 10.00 GiB
Current LE 2560
Segments 2
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:3

--- Logical volume ---
LV Path /dev/geekswingvg/geekswinglv3
LV Name geekswinglv3
VG Name geekswingvg
LV UUID hyYPIt-9JtW-WWE3-2smW-qMi0-lMm9-2DOp07
LV Write Access read/write
LV Creation host, time centos-testnode, 2013-02-08 20:54:19 -0500
LV Status available
# open 0
LV Size 8.99 GiB
Current LE 2301
Segments 2
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:4

[root@centos-testnode ~]#

You will see these devices in /dev/mapper. Now you can use /dev/mapper/[lvname] just like you would a physical disk
device like /dev/sdb, /dev/sdc


[root@centos-testnode ~]# cd /dev/mapper
[root@centos-testnode mapper]# ls -1
control
geekswingvg-geekswinglv1
geekswingvg-geekswinglv2
geekswingvg-geekswinglv3
vg_centostestnode-lv_root
vg_centostestnode-lv_swap
[root@centos-testnode mapper]#

I’ll create a filesystem and mount it now

[root@centos-testnode mapper]# mkfs -t ext3 /dev/mapper/geekswingvg-geekswinglv1
mke2fs 1.41.12 (17-May-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
327680 inodes, 1310720 blocks
65536 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=1342177280
40 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736

Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 32 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
[root@centos-testnode mapper]# mkdir /testmountpoint
[root@centos-testnode mapper]# mount /dev/mapper/geekswingvg-geekswinglv1 /testmountpoint
[root@centos-testnode mapper]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/vg_centostestnode-lv_root
16166824 2294224 13051352 15% /
tmpfs 936816 0 936816 0% /dev/shm
/dev/sda1 495844 37311 432933 8% /boot
/dev/mapper/geekswingvg-geekswinglv1
5160576 141436 4756996 3% /testmountpoint
[root@centos-testnode mapper]#

WOO HOO DONE!!!!

Thanks for reading! Thanks for watching!!

, , , , Linux

2 thoughts on “LVM on Linux Tutorial: Adding Hard Drives For More Space

Leave a Comment

Your email address will not be published. Required fields are marked *

Spam protection by WP Captcha-Free