🔰🔰🔰Integrating LVM with Hadoop and providing Elasticity to DataNode Storage🔰🔰🔰

Hello, connection…..

first of all, we have to add a hard disk to the virtual box on the data node.

Step to add hard disk:-

1. Shutdown the virtual box and go to the settings.
2. In setting go to storage click “Controller SATA”.
3. The last symbol in controller SATA is to ‘Add harddisk’ click on it.
4. create a virtual hard disk and choose it.

Start the machine and type ‘fdisk -I'm to check the hard disk added.

Then we have to create a new extended partition because by default we have only 3 primary and 1 extended partition. In the extended partition, we have to create a subpartition up to 15 partitions.

Steps to create Logical Volume Manager (LVM):-

1. Physical Volume (pv):-

Physical volumes are the base "block" that you need to manipulate a disk using Logical Volume Manager. LVM allows you to create physical volumes using multiple partitions, it is recommended that you use a single partition for a PV.

2. Volume Group (vg):-

A volume group is the central unit of the Logical Volume Manager (LVM) architecture. It is what we create when we combine multiple physical volumes to create a single storage structure, equal to the storage capacity of the combined physical devices. Physical volumes are devices that are initialized using LVM, i.e., hard disk drives, solid-state drives, partitions, etc.

3. Logical Volume (lv):-

LVM is a tool for logical volume management which includes allocating disks, striping, mirroring, and resizing logical volumes. With LVM, a hard drive or set of hard drives is allocated to one or more physical volumes. LVM physical volumes can be placed on other block devices which might span two or more disks.

To create Physical group command “pvcreate /dev/<partition_name>”

To show physical volume command is “pvdisplay”

Then we have to create Volume Group command “vgcreate <volume_name> /dev<partition_name>”

To show volume group command “vgdisplay”

Then create logical volume command “lvcreate -L/ — size +15G -n <name_of_lvm> <name_volume_group>

To show the created logical group command is “lvdisplay”

we have to formated the allocated space command “mkfs.ext4 <path_of_LV>

Then, mount the folder which is shared to namenode of hadoop, in my case shared folder is ‘/dn1’ and the command is to mount the folder “mount <path_of_LV> <shared_folder_location>”

Check my folder is mounted or not… by the command ‘df -h’

yeah… my folder is mounted and starts the name node and the data node. Let see is it connected are not…

here, my data node is started…..

yes, it is connected.

Here, I am extended the storage of data node command “lvextend -L +5G /dev/vg1/lvm7”

To show storage will extended command is “lvdisplay”

To update the storage we will provide to it by the command is “resize2fs /dev/vg1/lvm7”

you can see “/dn1” logical volume is increased by +5GB by “df -h” command:

Now you can check on the name node a data node added to it with a size of 20GB by using the command on Name node : “hadoop dfsadmin -report”.

Thank you ….