VDO is a Linux device-mapper driver that reduces disk space usage on block devices, and minimizes the replication of data, saving disk space and even increasing data throughput.
VDO consists of two kernel modules and two commands:
Kernel Modules:
- kvdo – This module transparently controls data compression.
- uds– This module is for deduplication.
Commands:
- vdo– This command is used to create, remove, start, and stop VDO volumes, as well as perform other configuration changes.
- vdostats -This command is used to report on various aspects of VDO volumes, including effective reduction and physical volume utilization. Think of this as ‘df’ for VDO capacity.
The VDO layer is placed on top of an existing block storage device, such as a RAID device or a local disk. Those block devices can also be encrypted devices. The storage layers, such as LVM logical volumes and file systems are placed on top of a VDO device.
VDO config is written in /etc/vdoconf.yml
1:10 ratio for VDO logical size Virtual machines, containers
1:3 for object storage
The following diagram shows the placement of VDO in an infrastructure consisting of KVM virtual machines that are using optimized storage devices.
VDO applies three phases to data in the following order to reduce the footprint on storage devices:
1. Zero-Block Elimination filters out data blocks that contain only zeroes (0) and records the information of those blocks only in the metadata. The nonzero data blocks are then passed
to the next phase of processing. This phase enables the thin provisioning feature in the VDO devices.
2. Deduplication eliminates redundant data blocks. When you create multiple copies of the same data, VDO detects the duplicate data blocks and updates the metadata to use those
duplicate blocks as references to the original data block without creating redundant data blocks. The universal deduplication service (UDS) kernel module checks the redundancy of the
data through the metadata it maintains. This kernel module ships as part of the VDO.
3. Compression is the last phase. The kvdo kernel module compresses the data blocks using LZ4 compression and groups them on 4 KB blocks.
1. Enabling VDO
Install the vdo and kmod–kvdopackages to enable VDO in the system.
If VDO services not started, then you need to start.
2. Creating a VDO Volume
To create a VDO volume, run the vdo create command.
Make sure that you have a spare disk – or at least a partition – available for use by VDO.
[root@tower ~]# lsblk
Verify the availability of the vdo1 volume using the vdo list command.
Verify that the vdo1 volume has both the compression and deduplication features enabled.
Use grep to search for the lines containing the string Deduplication in the output of the vdo status –name=vdo1 command.
Use grep to search for the lines containing the string Compression in the output of the vdo status –name=vdo1 command.
Format the vdo1 volume with the XFS file-system type and mount it on /mnt/vdo1.
Format the vdo1 volume with the XFS file system using the mkfs command.
The -K option in the preceding mkfs.xfs command prevents the unused blocks in the file system from being discarded immediately which lets the command return faster.
Use the udevadm command to register the new device node.
Create the /mnt/vdo1 directory using the mkdir command.
Mount the vdo1 volume on /mnt/vdo1 using the mount command.
Permanent mount for VDO
View the initial statistics and status of the volume using the vdostats command.
To start & stop particular VDO volumes
To activate/De-activate VDO volumes
Check the status of vdo
Enable / Disable De-duplication
Compression
To increase the logical size of the VDO volume
To increase the Physical size of the VD0 volume
Remove VDO volumes
umount -f /dev/mapper/vdo1 udevadm settle dmsetup remove vdo1 vdo remove --force --name=vdo1
run discard unused blocks manually
Conclusion
We have seen how to install and perform a basic configuration of VDO on RHEL8 / CentOS8. VDO provides native deduplication for Linux, reducing required storage capacity by eliminating redundant data from a sub-file level.
