lv-convert2cache v0.0.5 [08 Apr 2022] by Dominic
Description
TL;DR: lv-convert2cache.sh supercharges a Logical Volume.
lv-convert2cache.sh is a wrapper for LVM functions to convert an existing LVM logical volume (LV) into a read/write 'cache LV'. A typical scenario is that an existing LV (in an existing volume group (VG)) is currently stored on one (or more) physical volume(s) (PV(s)) which are on one (or more) slower devices (e.g. spinning disks), and we want to make this LV faster for read and write by (optionally creating and) using a new PV on a smaller but faster device (e.g. NVMe SSD) as a 'cache pool'.
After some sanity tests, lv-convert2cache.sh asks you (once) to confirm before any action is taken.
When lv-convert2cache.sh has completed, the new cache LV will have the same name as, and be usable as, the original LV. There will also be two hidden LVs - the data LV on the original slower drive (named as the original LV with '_corig' suffixed) and a cache pool LV on the faster device (named as the original LV with '_CacheVol_cvol' suffixed). Internally the cache pool LV will hold both the cached data and the metadata for managing the cache. All LVs can be seen with: lvs --all --options +devices
Usage
lv-convert2cache.sh [options] existing_VG/existing_LV PV_for_cache ["further_options_for_lvconvert_(at_cache_creation)"]
Example
./lv-convert2cache.sh timedicer/home /dev/sda1 "--chunksize 512"
Options
-h see this help
-l see changelog
-v show existing cache LVs, if any
Notes
The device (typically a drive partition) to be used for the new PV (PV_for_cache i.e. the cache pool) must exist, but need not already be a PV. Any contents on such device must be expendable - they will be lost.
I have not seen any guidance on the recommended size ratio between the cache pool device (PV_for_cache) and the underlying data LV (existing_LV). Most examples use small cache pools (e.g. 10GB), maybe because they were written when larger SSDs were expensive. I have used a 200GB cache pool (for a 2TB source LV) without problems, except having to specify "--chunksize 512" when creating it.
- Of 2 possible cache methods, lv-convert2cache.sh uses 'dm-cache' (alternative 'dm-writecache' only speeds up writing not reading)
- Of 2 possible cache options, lv-convert2cache.sh uses 'cachevol' (the cached copies of data blocks and the metadata for managing the cache are both stored in the same (hidden) LV on the same (fast) device; the more complex alternative option 'cachepool' allows having cache data & metadata on different (LVs and) devices [which could make the cached LV operate marginally faster])
- Unsurprisingly, to make changes lv-convert2cache.sh must be run by superuser
- I made this utility to help me and you, but you use it at your own risk
You are unable to resize logical volumes of cache type. To workaround this limitation: remove the cache pool, extend the (now-uncached) logical volume, and then recreate the cache pool, something like this:
# cache pool 'home_CacheVol_cvol' on '/dev/sda1'
# - find this data with 'lv-convert2cache.sh -v'
lvremove timedicer/home_CacheVol_cvol
lvextend -L+100G -r /dev/timedicer/home # extend including underlying fs
lv-convert2cache.sh timedicer/home /dev/sda1 "--chunksize 512"
Kudos
The following pages were helpful in devising this utility and writing the help text -
https://blog.delouw.ch/2020/01/29/using-lvm-cache-for-storage-tiering/
https://manpages.ubuntu.com/manpages/focal/man7/lvmcache.7.html
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/configuring_and_managing_logical_volumes/enabling-caching-to-improve-logical-volume-performance_configuring-and-managing-logical-volumes
https://blog.pcfe.net/hugo/posts/2019-04-28-grow-cached-lv/
Changelog
0.0.5 [08 Apr 2022] - Help text clarifications
0.0.4 [06 Apr 2022] - Add help about how to resize/extend a cache type LV, add -v option
0.0.3 [01 Apr 2022] - Help text clarifications
0.0.2 [11 Feb 2022] - Help text clarifications
0.0.1 [02 Feb 2022] - Help text changes, bug fix, check if existing LV is already cached
0.0.0 [11 Jan 2022] - Initial version
Download lv-convert2cache.sh
Donation
I have provided this software free gratis and for nothing. If you would like to thank me with a contribution, please let me know and I will send you a link. Thank you!
My Other Sites
- TimeDicer - Onsite/offsite data backup for Windows (uses rdiff-backup)
- Finding a 4D Backup Solution
- Web Scraping How To - extracting data from web sites
My Programs
Here is a selection of some (other) programs I have written, most of which run from the command line (CLI), are freely available and can be obtained by clicking on the links. Dependencies are shown and while in most cases written for a conventional Linux server, they should run on a Raspberry Pi, and many can run under Windows using Windows Subsystem for Linux (WSL) or Cygwin. Email me if you have problems or questions, or if you think I could help with a programming requirement.
Backup Utilities
- TimeDicer - Onsite/offsite data backup for Windows (uses rdiff-backup) [ GNU/Linux & MS Windows©: 2008-20 ]
- rdiff-backup-regress - GNU/Linux script to regress an rdiff-backup archive. [ GNU/Linux: 2012-16 ]
Debian/Ubuntu kernel and LVM Utilities
- kernel-remove - GNU/Linux script to list the installed GNU/Linux kernels in a Debian-based distro (e.g. Ubuntu), and can be used to remove an unwanted kernel and related packages, updating grub appropriately. [ GNU/Linux-Debian/Ubuntu: 2010-22 ]
- lvm-usage - GNU/Linux script to show available disk space and how it is used; run as cron job to warn if usage is above a set percentage. Provides additional information if LVM is in use. [ GNU/Linux: 2012-21 ]
- lvm-delete-snapshot - GNU/Linux script to remove LVM snapshot that has been left over by another process. [ GNU/Linux: 2012-21 ]
- netnames - GNU/Linux script shows current name, biosdevname and 'predictable name' of network device - helps with network device name scheme migration. [ GNU/Linux-Debian/Ubuntu: 2020-20 ]
Dellmont - VoIP and Mobile Phone Account Utilities
- dellmont-credit-checker - GNU/Linux script to check credit balance on many Dellmont / Finarea / Betamax portals such as voicetrading.com and voipdiscount.com. [ GNU/Linux: 2008-19 ]
- sms-sender - GNU/Linux script to send text messages using Dellmont’s voicetrading.com. [ GNU/Linux: 2012-18 ]
- get-vt-cdrs - GNU/Linux script to download CDRs (call detail records) from Dellmont’s voicetrading.com or voippro.com. [ GNU/Linux: 2010-19 ]
- saynoto0870 - For UK, a GNU/Linux script which performs automated lookup of the www.saynoto0870.com database, finding cheap or free geographic number replacements for expensive non-geographic (087* or 084*) numbers. [ GNU/Linux: 2012-12 ]
Miscellaneous Programs
- sleepwalker - Windows© program which can be run from a remote machine to 'wake up' a Windows© machine behind a router, wait for it to start and then initiate Remote Desktop session. [MS Windows©: 2008-16]
- relay-enforcer - GNU/Linux program enabling a postfix-based mail server relaying to Gmail to act on reports from Gmail about blocked emails. [ GNU/Linux: 2016-22 ]
- bind9-resolved-switch - GNU/Linux program for switching permanently between using bind9 or systemd-resolved as the system DNS resolver. [ GNU/Linux: 2016-22 ]
- unlock - GNU/Linux remote program for easy entering of decrypt passphrase on a remote machine which has root dm-crypt+LUKS. [ GNU/Linux: 2017-18 ]
- wifi-updown - GNU/Linux program to take down wifi interface if there is a working wired interface (or restore wifi if not). [ GNU/Linux: 2018-22 ]
- routefix - GNU/Linux program to restore a default ip traffic route if there is none such (e.g. after running wifi-updown). [ GNU/Linux: 2018-19 ]
- pdf-compress - GNU/Linux program to create smaller b/w pdf file from an original large pdf file, especially when original resulted from scanning. [ GNU/Linux: 2016-22 ]
- form-extractor - GNU/Linux program to extract form tags from a web page or downloaded file. [ GNU/Linux: 2012-20 ]
- mythic-dns-sync - GNU/Linux program to update DNS record at mythic-beasts.com to match local external ip. [ GNU/Linux: 2016-22 ]
- tiny-device-monitor - GNU/Linux program to test webpages (including password-protected) or machines to check they are live; use as a cron job for your own websites, for hardware presenting a webpage, or for any machines with a presence on your local LAN or on the internet. [ GNU/Linux: 2009-22 ]
- dutree - GNU/Linux program to show a tree-style list of files and directories at the specified location and greater than the specified size (default 1GB). [ GNU/Linux: 2012-20 ]
- Accounts - Multi-business multi-currency accounting software, uses Access [MS Windows©: 1996-2022]
- Rents Program - Residential lettings/landlord front office program, with many special features for UK market [MS Windows©: 1991-2022]
If you have a comment or question, please email me, thank you.