Working with Logical Volume Manager (LVM)

LVM

Logical Volume Manager – LVM

The Logical Volume Manager also know as LVM, provides a great amount of flexibility for applications that require expanding disk space. LVM work by creating a virtual layer of storage on top of physical devices. The OS operates against the LVM disk as if it was a regular disk. However the advantages allow us to expand the disk without having to reformat the file system or create new partitions.

 

Scenario

Consider that your application is using database that runs on an EC2 instance. The data is stored in EBS volumes. You application has become very popular and grows widely in turn increasing the size of your database on daily basis. You are in a situation where you need to frequently increase the size of your disk space where the data gets stored. This should be done seamlessly without affecting your application at run time.

 

Solution

This can be achieved easily by  leveraging on  Logical Volume Managers (LVMs), which provide the option to easily increase the size of their volume by adding more EBS volumes. With multiple EBS volumes, network performance is increased between EC2 instances and EBS volumes.

LVMs create an abstraction layer on top of underlying disk layouts and allows sysadmins to create virtual partitions over physical volumes. You can have multiple virtual partitions on a physical volume or a single virtual partition over multiple physical volumes. LVMs provide the ability to dynamically expand and shrink virtual partitions and make additions to physical volumes on the fly for existing LVMs.

Lets get our hands dirty and practically see how we can achieve this.

In this article we will see how to expand the disk space of the mount /var/lib/mysql where our DB gets stored.  We will  start by creating physical volume, volume groups and then the Logical Volume. Upon completion, we will then learn how to expand or “extend” the logical volume by attaching a new device to the volume group.

 

How to do it ?

We will be working on EC2 instance and EBS volume for this example. Launch an EC2 instance and attach 3 additional EBS volume (disks) to that instance. In this example, the additional EBS volumes apart from ‘/’ are given the following device names.

  • /dev/xvdf
  • /dev/xvdg
  • /dev/xvdj

You can view your device names by listing /dev directory

 

Prepare the physical disks

As we know, a physical disk is just an attached piece of storage. This is normally what we would use for
storage on a file system if we were not using LVM. Lets us first create partition on the first two devices ( /dev/xvdf and /dev/xvdg).
We will be using gdisk  command for partitioning the device.
[root@devopsideas-LVM dev]# gdisk /dev/xvdf
GPT fdisk (gdisk) version 0.8.6

Partition table scan:
MBR: not present
BSD: not present
APM: not present
GPT: not present

Creating new GPT entries.

Command (? for help): n
Partition number (1-128, default 1): <Enter>
First sector (34-41943006, default = 2048) or {+-}size{KMGTP}: <Enter>
Last sector (2048-41943006, default = 41943006) or {+-}size{KMGTP}: <Enter>
Current type is 'Linux filesystem'
Hex code or GUID (L to show codes, Enter = 8300): L
0700 Microsoft basic data  0c01 Microsoft reserved    2700 Windows RE
4200 Windows LDM data      4201 Windows LDM metadata  7501 IBM GPFS
7f00 ChromeOS kernel       7f01 ChromeOS root         7f02 ChromeOS reserved
8200 Linux swap            8300 Linux filesystem      8301 Linux reserved
8e00 Linux LVM             a500 FreeBSD disklabel     a501 FreeBSD boot
a502 FreeBSD swap          a503 FreeBSD UFS           a504 FreeBSD ZFS
a505 FreeBSD Vinum/RAID    a580 Midnight BSD data     a581 Midnight BSD boot
a582 Midnight BSD swap     a583 Midnight BSD UFS      a584 Midnight BSD ZFS
a585 Midnight BSD Vinum    a800 Apple UFS             a901 NetBSD swap
a902 NetBSD FFS            a903 NetBSD LFS            a904 NetBSD concatenated
a905 NetBSD encrypted      a906 NetBSD RAID           ab00 Apple boot
af00 Apple HFS/HFS+        af01 Apple RAID            af02 Apple RAID offline
af03 Apple label           af04 AppleTV recovery      af05 Apple Core Storage
be00 Solaris boot          bf00 Solaris root          bf01 Solaris /usr & Mac Z
bf02 Solaris swap          bf03 Solaris backup        bf04 Solaris /var
bf05 Solaris /home         bf06 Solaris alternate se  bf07 Solaris Reserved 1
bf08 Solaris Reserved 2    bf09 Solaris Reserved 3    bf0a Solaris Reserved 4
bf0b Solaris Reserved 5    c001 HP-UX data            c002 HP-UX service
ed00 Sony system partitio  ef00 EFI System            ef01 MBR partition scheme
ef02 BIOS boot partition   fb00 VMWare VMFS           fb01 VMWare reserved
fc00 VMWare kcore crash p  fd00 Linux RAID
Hex code or GUID (L to show codes, Enter = 8300): 8e00
Changed type of partition to 'Linux LVM'

Command (? for help): w

Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
PARTITIONS!!

Do you want to proceed? (Y/N): Y
OK; writing new GUID partition table (GPT) to /dev/xvdf.
The operation has completed successfully.

As you can see, I have created just a single partition for the entire device for this example. You can create multiple partitions based upon your requirement.

Follow the same steps for partitioning the second device /dev/xvdg

 

Create Physical Volume

Each LVM logical volume has an underlying physical storage unit. This is the physical volume
component of LVM. A physical volume can either be a partition of a device or the entire disk. In
order to use a physical volume, the physical volume must be initialized as a physical volume (PV).

Now that our disks have been created correctly, we need to prepare them for use by LVM. We will use pvcreate and pvdisplay command to achieve this.

 

Run the below command to create the physical volume.

[root@devopsideas-LVM dev]# pvcreate /dev/xvdg1 /dev/xvdf1
Physical volume "/dev/xvdg1" successfully created
Physical volume "/dev/xvdf1" successfully created

 

Run the below command to display the available Physical Volumes

[root@devopsideas-LVM dev]# pvdisplay
"/dev/xvdg1" is a new physical volume of "20.00 GiB"
--- NEW Physical volume ---
PV Name               /dev/xvdg1
VG Name
PV Size               20.00 GiB
Allocatable           NO
PE Size               0
Total PE              0
Free PE               0
Allocated PE          0
PV UUID               HUratO-JjQT-QB4T-63HX-TSNx-Aofg-iw06N3

"/dev/xvdf1" is a new physical volume of "20.00 GiB"
--- NEW Physical volume ---
PV Name               /dev/xvdf1
VG Name
PV Size               20.00 GiB
Allocatable           NO
PE Size               0
Total PE              0
Free PE               0
Allocated PE          0
PV UUID               jf0clR-fFq6-ZHzq-x3HX-921V-2xlg-a1doHQ

 

As you can see, the two physical volumes that we created are listed in the pvdisplay command. Each device has a total physical volume size of 20 GiB.

 

Create Volume Group

A volume group is a group of physical volumes that make up our allocatable disk space for a logical
volume. A physical volume is made up of extents; an extent is also mapped from the volume group to
the physical volume. This makes it easy to move data from one volume to another or transfer data from
an old physical storage to a new physical storage device without interrupting our file system.
Run the below command to create the volume group
[root@devopsideas-LVM dev]# vgcreate DB /dev/xvdg1 /dev/xvdf1
/proc/devices: No entry for device-mapper found
/proc/devices: No entry for device-mapper found
Volume group "DB" successfully created

 

We have named our volume group as DB. We have added the two physical volumes that we created in the earlier step as part of this volume group.

Run the below commands to view the details on volume group in that particular system.

[root@devopsideas-LVM dev]# vgdisplay
--- Volume group ---
VG Name               DB
System ID
Format                lvm2
Metadata Areas        2
Metadata Sequence No  1
VG Access             read/write
VG Status             resizable
MAX LV                0
Cur LV                0
Open LV               0
Max PV                0
Cur PV                2
Act PV                2
VG Size               39.99 GiB
PE Size               4.00 MiB
Total PE              10238
Alloc PE / Size       0 / 0
Free  PE / Size       10238 / 39.99 GiB
VG UUID               pT3zj7-Zmdi-NH2p-Pt9I-4v15-ZZvi-hm62MD


[root@devopsideas-LVM dev]# vgdisplay -s
  "DB" 39.99 GiB [20.00 GiB used / 19.99 GiB free]


[root@devopsideas-LVM dev]# vgscan
  Reading all physical volumes.  This may take a while...
  Found volume group "DB" using metadata type lvm2

 

As you can see from the output that we get a total Volume Group size of ~40 GiB and PE (Physical Extent) Size of 4.00 MiB. An Extent is the smallest unit of space that can be assigned to a volume group. Volume group extents are referred to as physical extents and a logical volume is allocated into sets of logical extents that are the same size as the physical extents. Thus the logical volume extents map to the physical volume extents and that is how a logical volume communicates with the physical volume data.

 

Create the Logical Volume

Now that we have prepared our physical volumes and associated them with a volume group, we can
now create a logical volume. The logical volume will use disk space available from the volume group,
which again maps to the physical volume.
We will use lvcreate command to create the Logical Volume. lvcreate command has several flags associated with it.
  • -n (volume name)
  • ­-L size of the volume in bytes i.e 1M 1G 20G 1M etc.
  • ­-l size in physical extents. (If a physical extent is 4MiB and you want to create an 8MiB LVM volume then -l would be 2.)

Lets us create LVM named mysql from our DB Volume Group and assign it 20GB of space

[root@devopsideas-LVM dev]# lvcreate -n mysql -L 20G DB
Logical volume "mysql" created

 

We can view the details of LVM by typing the lvdisplay, lvdisplay -C and lvscan commands

[root@devopsideas-LVM dev]# lvdisplay
--- Logical volume ---
LV Path                /dev/DB/mysql
LV Name                mysql
VG Name                DB
LV UUID                k6M1SF-0PYU-yI0r-s8Sj-bCy9-CoSV-iJiyvV
LV Write Access        read/write
LV Creation host, time devopsideas-LVM, 2016-10-02 13:47:40 +0000
LV Status              available
# open                 0
LV Size                20.00 GiB
Current LE             5120
Segments               2
Allocation             inherit
Read ahead sectors     auto
- currently set to     8192
Block device           253:0


[root@devopsideas-LVM dev]# lvdisplay -C
  LV    VG   Attr       LSize  Pool Origin Data%  Move Log Cpy%Sync Convert
  mysql DB   -wi-a----- 20.00g  


[root@devopsideas-LVM dev]# lvscan
  ACTIVE            '/dev/DB/mysql' [20.00 GiB] inherit
    

 

Create a File System and Mount the Logical Volume

Run the below command to create file system on the LVM that we created and mount it to /var/lib/mysql directory which is where our database gets stored in our example.

 

[root@devopsideas-LVM DB]# mkfs.ext4 /dev/DB/mysql
mke2fs 1.42.9 (28-Dec-2013)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
1310720 inodes, 5242880 blocks
262144 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=2153775104
160 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000

Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done


[root@devopsideas-LVM mysql]# mount /dev/DB/mysql /var/lib/mysql

[root@devopsideas-LVM mysql]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/xvda1            8.0G  1.4G  6.7G  18% /
devtmpfs              482M     0  482M   0% /dev
tmpfs                 497M     0  497M   0% /dev/shm
tmpfs                 497M   19M  478M   4% /run
tmpfs                 497M     0  497M   0% /sys/fs/cgroup
/dev/mapper/DB-mysql   20G   45M   19G   1% /var/lib/mysql

df -h command indicates that our filesystem got mounted at /var/lib/mysql

 

Extending Logical Volumes and Volume Groups

Since our application became so popular, the current space allocated for DB is not enough. We got a request to increase the the disk space by another 30GB.

We can easily do this by,

  1. Creating a new EBS volume of required size and attaching it to our instance
  2. Create a PV with the newly attached device
  3. Add the PV as part of already created Volume Group (DB)
  4. Extend the Logical Volume
  5. Resize the filesystem for the new size to take effect.

We already have a device /dev/xvdj, which we haven’t used yet. We will use this device to extend the size of our Logical Volume and in turn the filesystem size.

Repeat the same procedure for preparing the device, creating a Physical Volume  as seen above. Once creating the PV for /dev/xvdj1, we need to add it to Volume Group (DB). We will use vgextend command to achieve this.

[root@devopsideas-LVM dev]# vgextend DB /dev/xvdj1
Volume group "DB" successfully extended

 

We can run the vgdisplay command to verify the Volume Group details

[root@devopsideas-LVM dev]# vgdisplay -C
VG   #PV #LV #SN Attr   VSize  VFree
DB     3   1   0 wz--n- 59.99g 39.99g

 

We can see that the DB Volume Group size is now increased to ~60GB.

Present size of our Logical Volume “mysql” is 20 GB. We got a request to increase another 30GB to our mysql file system. We have enough space in our Volume Group now to allocate 30GB to mysql LVM. We can do this by running lvextend command.

[root@devopsideas-LVM dev]# lvextend -L +30G /dev/DB/mysql
Size of logical volume DB/mysql changed from 20.00 GiB (5120 extents) to 50.00 GiB (12800 extents).
Logical volume mysql successfully resized.

 

We can verify the above operation by running lvdisplay command

[root@devopsideas-LVM dev]# lvdisplay
--- Logical volume ---
LV Path                /dev/DB/mysql
LV Name                mysql
VG Name                DB
LV UUID                k6M1SF-0PYU-yI0r-s8Sj-bCy9-CoSV-iJiyvV
LV Write Access        read/write
LV Creation host, time devopsideas-LVM, 2016-10-02 13:47:40 +0000
LV Status              available
# open                 1
LV Size                50.00 GiB
Current LE             12800
Segments               3
Allocation             inherit
Read ahead sectors     auto
- currently set to     8192
Block device           253:0

 

Now we have everything in place and we should resize our filesystem for the changes to take effect.

[root@devopsideas-LVM linuxacademy]# resize2fs /dev/mapper/DB-mysql
resize2fs 1.42.9 (28-Dec-2013)
Filesystem at /dev/mapper/DB-mysql is mounted on /var/lib/mysql; on-line resizing required
old_desc_blocks = 3, new_desc_blocks = 7
The filesystem on /dev/mapper/DB-mysql is now 13107200 blocks long.

 

Lets verify whether the File sytem size got increased by running the df command.

 [root@devopsideas-LVM linuxacademy]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/xvda1            8.0G  1.4G  6.7G  18% /
devtmpfs              482M     0  482M   0% /dev
tmpfs                 497M     0  497M   0% /dev/shm
tmpfs                 497M   19M  478M   4% /run
tmpfs                 497M     0  497M   0% /sys/fs/cgroup
/dev/mapper/DB-mysql   50G   52M   47G   1% /var/lib/mysql
tmpfs                 100M     0  100M   0% /run/user/1001

 

We can see that the size of our mysql File System is 50GB. We were able to increase the size from 20 to 50GB on the fly without any downtime.

We can further add more storage to the volume group and use it for other file systems. For example, we still have 9.99GiB left in our DB Volume Group

[root@devopsideas-LVM linuxacademy]# vgdisplay -C
VG   #PV #LV #SN Attr   VSize  VFree
DB     3   1   0 wz--n- 59.99g 9.99g

 

Lets say, we are running Orable DB in the same server and we would like to manage it as part of LVM, we can follow the same steps that we saw above.

 

[root@devopsideas-LVM linuxacademy]# lvcreate -n oracle -L 9G DB
Logical volume "oracle" created.
[root@devopsideas-LVM linuxacademy]# mkfs.ext4 /dev/DB/oracle
mke2fs 1.42.9 (28-Dec-2013)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
589824 inodes, 2359296 blocks
117964 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=2151677952
72 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

[root@devopsideas-LVM linuxacademy]# mount /dev/DB/oracle /var/lib/oracle
[root@devopsideas-LVM linuxacademy]# df -h
Filesystem             Size  Used Avail Use% Mounted on
/dev/xvda1             8.0G  1.4G  6.7G  18% /
devtmpfs               482M     0  482M   0% /dev
tmpfs                  497M     0  497M   0% /dev/shm
tmpfs                  497M   19M  478M   4% /run
tmpfs                  497M     0  497M   0% /sys/fs/cgroup
/dev/mapper/DB-mysql    50G   52M   47G   1% /var/lib/mysql
tmpfs                  100M     0  100M   0% /run/user/1001
/dev/mapper/DB-oracle  8.8G   37M  8.3G   1% /var/lib/oracle

 

To summarize, we have seen how to create Physical Volume, Volume Group and Logical Volume Manager. We also saw how to increase the disk space without any downtime using LVM.

That’s it for this article. Hope this will be of help to someone!!