rsync
The rsync utility is an extremely flexible copying tool that works locally as well as remotely through SSH. To use rsync it is best to setup passwordless SSH authenticates with public/private keys between the servers that will require it. As we have done before, let’s see how it works and what options is offers by example:
# synchronise the 2nd folder /backup/home/john with the 1st one /home/john
marc:~> rsync /home/john /backup/home/john
.
# using the specified marc’s SSH keyfile, sync the folder Pictures from the remote host
# with the contents of Pictures from localhost without creating Pictures itself
marc:~> rsync -e “ssh -i $HOME/.ssh/id_rsa” Pictures/ marc@sl7ora112:~/Pictures/
.
# same as above but using compression (-z) and archival mode (-rlptgoD but no -H,-A,-X)
# -r means recursive, -l symlinks, -p preserve permissions, -t preserve times, -g preserve
# groups, -o preserve owner, -D copy device & special files
marc:~> rsync -az Pictures/ marc@sl7ora112:~/Pictures/
.
# same as above but with hard links (-H), ACLs (-A) and extended attributes (-X)
marc:~> rsync -aAHXz Pictures/ marc@sl7ora112:~/Pictures/
.
# same as above but checksums everything (c) rather than checking first ctime and size
marc:~> rsync -aAHXzc Pictures/ marc@sl7ora112:~/Pictures/
.
# skip files with newer mtimes (-u) on the destination
marc:~> rsync -azu Pictures/ marc@sl7ora112:~/Pictures/
.
# unlike with “-l” the “-L” option copies the files pointed to rather than the symlink
marc:~> rsync -azL Pictures/ marc@sl7ora112:~/Pictures/
.
# same as above but deletes extraneous files in the destination that do not exist in source
marc:~> rsync -azL delete Pictures/ marc@sl7ora112:~/Pictures/
.
The rsync command has a multitude of options to cater for very specific circumstances. If your needs to beyond the examples above it’s better to check man rsync for further examples and explanations.
blockdev
The blockdev command is used to execute ioctl calls against devices from the command line. The most common calls we can issue are:
root:/tmp> blockdev -v --flushbufs /dev/sdb -> flush device buffers
flush buffers succeeded.
.
root:/tmp> blockdev -v --getalignoff /dev/sdb -> get align offset
get alignment offset in bytes: 0
.
root:/tmp> blockdev -v --getbsz /dev/sdb -> get block size
get blocksize: 4096
.
root:/tmp> blockdev -v --getdiscardzeroes /dev/sdb -> get discard zero support (0 no, 1 yes)
get discard zeroes support status: 0
.
root:/tmp> blockdev -v --getfra /dev/sdb -> get read-ahead in sectors
get filesystem readahead: 8192
.
root:/tmp> blockdev -v --getiomin /dev/sdb -> get min I/O size in bytes
get minimum I/O size: 1048576
.
root:/tmp> blockdev -v --getioopt /dev/sdb -> get optimal I/O size in bytes
get optimal I/O size: 2097152
.
root:/tmp> blockdev -v --getmaxsect /dev/sdb -> get max sectors per request
get max sectors per request: 65535
.
root:/tmp> blockdev -v --getpbsz /dev/sdb -> get physical block size in bytes
get physical block (sector) size: 4096
.
root:/tmp> blockdev -v --getra /dev/sdb -> get read-ahead in bytes
get readahead: 8192
.
root:/tmp> blockdev -v --getro /dev/sdb -> get read-only status (0 no, 1 yes)
get readonly: 0
.
root:/tmp> blockdev -v --getsize64 /dev/sdb -> get device size in bytes
get size in bytes: 10737418240
.
root:/tmp> blockdev -v --getss /dev/sdb -> get logical sector size in bytes
get logical block (sector) size: 512
.
root:/tmp> blockdev -v --getsz /dev/sdb -> get device size in sectors
20971520
.
root:/tmp> blockdev -v --rereadpt /dev/sdb -> get re-read partition table
reread partition table succeeded.
.
root:/tmp> blockdev -v --setro /dev/sdb -> set device/partition to read-only
set read-only succeeded.
.
root:/tmp> blockdev -v --setrw /dev/sdb -> set device/partition to read-write
set read-write succeeded.
SSM – System Storage Manager
The last command we will cover in this chapter is the system-storage-manager or ssm. It might need to be installed first…
# dnf -y install system-storage-manager
We can list volumes, devices, pools, filesystems and snapshots with it:
# ssm list volumes
------------------------------------------------------------------------------------------------------
Volume Pool Volume size FS FS size Free Type Mount point
------------------------------------------------------------------------------------------------------
/dev/vg/root vg 7.59 GB xfs 7.58 GB 1.45 GB linear /
/dev/vgdata/documentation vgdata 20.00 GB xfs 19.99 GB 7.59 GB striped /home/marc/documentation
/dev/vgdata/sl7ora112 vgdata 24.00 GB ext4 24.00 GB 4.85 GB striped /sl7ora112
/dev/vg/var vg 3.78 GB xfs 3.77 GB 2.05 GB linear /var
/dev/vg/home vg 29.76 GB xfs 29.74 GB 24.95 GB linear /home
/dev/vgdata/music vgdata 10.00 GB xfs 9.99 GB 4.53 GB striped /home/marc/music
/dev/vgdata/vigilante vgdata 1.00 GB xfs 1017.23 MB 969.88 MB striped /home/marc/vigilante
/dev/vgdata/photos vgdata 150.00 GB xfs 149.93 GB 27.86 GB striped /home/marc/photos
/dev/vgdata/companies vgdata 1.00 GB xfs 1017.23 MB 878.99 MB striped /home/marc/companies
/dev/vgdata/software vgdata 80.00 GB xfs 79.96 GB 31.62 GB striped /home/marc/software
/dev/vgdata/personal vgdata 5.00 GB xfs 4.98 GB 4.93 GB striped /home/marc/personal
/dev/vgdata/win7 vgdata 16.00 GB xfs 15.99 GB 981.26 MB striped /win7
/dev/vgdata/oracle vgdata 6.00 GB xfs 5.99 GB 4.77 GB striped /oracle
/dev/md127 md 41.29 GB raid1
/dev/sda1 476.00 MB vfat part /boot/efi
/dev/sdb1 4.28 GB ext4 4.28 GB 3.86 GB /tmp
.
# ssm list devices
--------------------------------------------------------------------------------------------
Device Free Used Total Pool Mount point
--------------------------------------------------------------------------------------------
/dev/md127 164.00 MB 41.12 GB 41.29 GB vg
/dev/sda 931.51 GB PARTITIONED
/dev/sda1 476.00 MB /boot/efi
/dev/sda2 3.81 GB SWAP
/dev/sda3 0.00 KB 41.29 GB 41.32 GB md
/dev/sda4 729.41 GB 156.50 GB 885.91 GB vgdata
/dev/sdb 931.51 GB
/dev/sdb1 4.28 GB /tmp
/dev/sdb2 0.00 KB 41.29 GB 41.32 GB md
/dev/sdb3 729.41 GB 156.50 GB 885.91 GB vgdata
.
# ssm list pools
--------------------------------------------------------------------------------------------
Pool Type Devices Free Used Total
--------------------------------------------------------------------------------------------
vg lvm 1 164.00 MB 41.12 GB 41.29 GB
vgdata lvm 2 1.42 TB 313.00 GB 1.73 TB
.
# ssm list filesystems
------------------------------------------------------------------------------------------------------
Volume Pool Volume size FS FS size Free Type Mount point
------------------------------------------------------------------------------------------------------
/dev/vg/root vg 7.59 GB xfs 7.58 GB 1.45 GB linear /
/dev/vgdata/documentation vgdata 20.00 GB xfs 19.99 GB 7.59 GB striped /home/marc/documentation
/dev/vgdata/sl7ora112 vgdata 24.00 GB ext4 24.00 GB 4.85 GB striped /sl7ora112
/dev/vg/var vg 3.78 GB xfs 3.77 GB 2.05 GB linear /var
/dev/vg/home vg 29.76 GB xfs 29.74 GB 24.95 GB linear /home
/dev/vgdata/music vgdata 10.00 GB xfs 9.99 GB 4.53 GB striped /home/marc/music
/dev/vgdata/vigilante vgdata 1.00 GB xfs 1017.23 MB 969.88 MB striped /home/marc/vigilante
We can list the snapshots taken from filesystems with:
# ssm list snapshots
With ssm we can also create LVM logical volumes, creating the underlying filesystem and mounting it in one go:
# ssm create –fs ext4 -s 1G -n ssmvol -p vgdata –raid 1 /mnt/snapvol
It is faster to create logical volumes with ssm but the range of options is far more limited (i.e. only raid
0/1/10 are supported, no thin volumes can be created, etc).
We can also remove logical volumes …
# ssm remove vgdata/ssmvol
… and resize them ….
# ssm resize -s -2G vgdata/ssmvol -> shrinks volume to 2G
# ssm resize -s +3G vgdata/ssmvol -> expands volume to 3G
# ssm resize -s 18G vgdata/ssmvol /dev/sdf3 -> resizes volume to 18G by adding another PV
… and check for consistency …
# ssm check /dev/sdb3
… and add devices …
# ssm add -p vgdata /dev/sdd3 /dev/sdd4