As is known, "computer users are divided into those who make backups and those who will do them". In this article, we will look at various ways backup (backup) of the entire system and, accordingly, recovery from backup.

It should be noted right away that all operations should not be carried out “live”, i.e. not on a running system, but from a liveCD or system installed on an adjacent partition / flash drive / usb-hdd. In cases where a downtime of a few minutes is critical for the system, it is possible to copy the system from under itself, but some additional conditions must be taken into account, which are not yet considered in this article.

From now on, for actions performed as root, the sudo command will be used, which is the standard for Ubuntu. On other systems it is possible to gain superuser privileges via su , some liveCD systems run in superuser mode by default

tar

One of the most popular ways to create a simple backup is to archive data using tar . The advantages of this method are the possibility of incremental backup (adding files to an existing archive, deleting or changing them), the ability to extract from an archive individual files, as well as the presence of tar on almost any Linux system.

Creating an archive

First, create mount points for the root partition and for the partition on which you are going to create a backup, like this

Mount both partitions. For greater reliability, you can mount the root partition in read-only mode (read-only) to eliminate the possibility of accidental data changes

sudo mount /dev/sdXY /mnt/root -o ro sudo mount /dev/sdXY /mnt/backup

(Instead of "sdXY" use your values ​​for the desired partitions. You can determine them with sudo fdisk -l or sudo blkid)

In case you use separate partitions for /boot, /usr, /home, etc. and want to include their contents in the backup, mount them in the appropriate folders

sudo mount /dev/sdXY /mnt/root/usr -o ro sudo mount /dev/sdXY /mnt/root/home -o ro

If necessary, create a folder on the backup partition where you want to place the archive, for example

sudo mkdir -p /mnt/backup/ubuntu/root

Now you can start creating the archive. To create a gzip compressed archive, run

Sudo tar -cvzpf -C /mnt/root /mnt/backup/ubuntu-sda1.tar.gz .

(The -p switch enables saving owners and permissions for files)

For bzip2 compression use

Sudo tar -cvjpf /mnt/backup/ubuntu-sda1.tar.bz2 /mnt/root

For lzma compression

sudo tar --lzma -cvpf /mnt/backup/ubuntu-sda1.tar.lzma /mnt/root

Similarly for lzo compression - switch --lzop instead of --lzma

Different compression algorithms give different archive sizes and also differ in performance.

At the end of the process, unmount all mounted partitions

Sudo umount /mnt/root(/boot,/var,/home,) /mnt/backup

Restoring from an archive

Create mount points for the root partition and the partition where your archive is stored

Sudo mkdir /mnt/(root,backup)

Mount the backup partition

sudo mount /dev/sdXY /mnt/backup -o ro

Format the root partition to the same (or different) FS. If you are using separate partitions for /usr, /boot, etc. and have archived them, format them too

(if you are restoring the system to a new HDD, partition it with fdisk/gparted and format the partitions)

Some file systems support setting a UUID when formatting. This makes it possible to create a file system with the same UUID as the old one, which will avoid the need to edit fstab .

For ext2/3/4, the UUID is set using the -U switch, and you can simplify the task even more with a command like

sudo mkfs.ext4 -L "label" -U "$(sudo blkid -o value -s UUID /dev/sda1)" /dev/sda1

If you used archiving when creating the image file, first unpack it using the same archiver, for example

Bzip2 -dv /media/backup/sda5.dd.bz

Now you can mount the image

sudo mount /media/backup/sda5.dd -o loop /mnt

(With the loop option, the mount program will automatically "hook up" the image file to a free loop device, and then mount the file system)

Now you can work with the contents of the image as with a regular file system, all your changes will be written to the image. When finished, unmount the image as a regular FS

sudo umount /mnt

dd - copy the entire hard drive

In this case, we will use dd again, only this time we will save all the contents hard drive- with the partition table, the partitions themselves and all the data. The advantage of this method is that you can save all the systems installed on this hard drive in one step without having to back up each partition separately. In addition, with such a backup, all data related to the bootloader will be saved - thus, after restoring from a backup, you will not need additional manipulations, you can immediately boot from this hard drive.

Creating an image

In general, the procedure is similar to that described above for backing up individual partitions. In this case, the advice about clearing free space with “zeros” is also applicable - if you have free time, do this with all sections.

Before starting the operation, make sure that none of the partitions of this hard disk is mounted. You can do this by running the mount command with no options.

Select the partition where you are going to place the obarz file. Of course, this must be a partition of another hard drive. Also make sure that there is enough free space on this partition (for example, using the df utility) - the amount of free space should correspond to the size of the copied hard disk (when compressed, the image will be smaller, but this depends on the type of data stored).

Mount the backup partition

sudo mount /dev/sdXY /mnt

Now you can start

sudo dd if=/dev/sdX bs=1M conv=noerror,sync | lzma -cv > /mnt/hdd.dd.lzma

(here "sdX" is a disk, not a partition! for copying without compression, the command is similar to the one above for a partition backup)

Depending on the size of the hard drive and the performance of your computer, the procedure may take a long time (up to several hours). When done, unmount the backup partition

sudo umount /mnt

Restoring from an image

Attention! This method involves a complete rollback to the state at the time of creating the archive with the replacement of all data!

Before starting work, make sure the power supply is reliable. Connect network adapter if you have a laptop, and if possible, use a UPS or stabilizer. High write intensity increases the risk of disk damage in the event of a power failure

Make sure that no partition of the disk being restored is in use. Mount the backup partition

sudo mount /dev/sdXY /mnt

You can start the process

Bzip2 -dc /mnt/hdd.dd.bz | sudo dd of=/dev/sdX bs=1M conv=sync,noerror

Or for an uncompressed image

sudo dd if=/mnt/hdd.dd.bz of=/dev/sdX bs=1M conv=sync,noerror

When done, unmount the backup partition

sudo umount /mnt

If you want to extract the image to another hard drive, it must be at least as large as the original. In case the new disk is larger, you can expand the partitions or create a new partition on free space using parted/fdisk/gparted/etc

Don't use both hard drives("duplicate" and "original") at the same time! If both drives are connected, the system will have two partitions for each UUID, which will lead to problems in operation or inability to boot

Mounting the image

By analogy with a partition image, you can work with a hard disk image just like a normal hard disk. In this case, the procedure is somewhat more complicated, since the image contains several sections.

If the image is compressed, unpack it. Now "hook" the image to the loop device

sudo losetup -fv /media/backup/sda.dd

(With the -f switch, the program will automatically find a free loop device, otherwise you must explicitly specify it)

losetup will display the name of the used device - unless you are working with other image files (iso, encrypted containers, etc.), it will most likely be /dev/loop0

Now we have a device that is a hard disk for the system, but we do not have access to its partitions. The kpartx program will help you get to the partitions (you may need to install the package of the same name)

sudo kpartx -av /dev/loop0

(Switch -a - add partitions for the specified device; -v - informative output)

The program will display the names of the created devices for the disk partitions: loop0p1 for the first partition, loop0p2 for the second, by analogy with partitions regular disk. Device files will be in the /dev/mapper folder

Now you can work with partitions and FS on them. For example, mount the former sda5 and write files to it

sudo mount /dev/mapper/loop0p5 /mnt

When done, unmount the partition

sudo umount /mnt

Remove partition devices with kpartx

sudo kpartx -dv /dev/loop0

and release the loop device

sudo losetup -v -d /dev/loop0

Everything! The changes have been written, and your image has become a regular file again.

cp

Here we will consider a backup using the cp utility, i.e. using simple copy. Actually, this is not the best way, and it is more suitable for copying the system to another hard disk / partition / computer, rather than creating a backup.

On the other hand, this method has several advantages:

Versatility - cp can be found on any Linux system

Low resource requirements (due to the lack of compression and the simplicity of the mechanism)

Ease of further work with the backup (adding / changing / deleting files, extracting the necessary data, etc.)

Making a copy

Create mount points for root and backup partitions

Sudo mkdir /mnt/(root,backup)

Mount both partitions

sudo mount /dev/sdXY -o ro /mnt/root sudo mount /dev/sdXY /mnt/backup

Mount partitions for /usr, /boot, etc., if any

sudo mount /dev/sdXY -o ro /mnt/root/home

Create a folder on the backup partition for your backup

Sudo mkdir /mnt/backup/ubuntu

You can start

sudo cp -av /mnt/root/* /mnt/backup/ubuntu

(the -a switch enables copying links "as is", saving all possible file attributes and recursive mode. the -v switch - output information about what is happening)

At the end of the process, unmount all partitions

In the future, you can archive your data in any convenient way.

Restoring from a copy

Attention! This method involves a complete rollback to the state at the time of creating the archive with the replacement of all data!

Create mount points for partitions

Sudo mkdir /mnt/(root,backup)

Mount the backup partition

sudo mount /dev/sdXY -o ro /mnt/backup

Format the root partition and partitions /usr, /boot, etc., if any. (For formatting partitions with saving UUID, see the section about)

sudo mkfs.reiserfs -l "root" /dev/sdXY sudo mkfs.ext2 -L "boot" /dev/sdXY sudo mkfs.ext4 -L "home" /dev/sdXY

Mount the newly created FS

The process of copying is similar, only in the opposite direction.

sudo cp /mnt/backup/ubuntu/* -av /mnt/root

Once the copy is complete, edit fstab to correct the UUID of the partitions

Unmount partitions

sudo umount /mnt/backup /mnt/root/(usr,home,)

squashfs

sudo mkfs.reiserfs -l "root" /dev/sdXY sudo mkfs.ext2 -L "boot" /dev/sdXY sudo mkfs.ext4 -L "home" /dev/sdXY

Mount the newly created FS

sudo mount /dev/sdXY /mnt/root sudo mount /dev/sdXY /mnt/root/usr sudo mount /dev/sdXY /mnt/root/var

You can start! The unsquashfs utility is used to unpack the image.

sudo unsquashfs -d /mnt/root -f /mnt/backup/ubuntu-root.sqfs

(The -d key specifies the path for unpacking; with the -f key, the program will use existing folders instead of trying to create new ones)

As with image creation, you'll see a progress bar and lots of other useful information.

When finished, edit fstab to replace the UUIDs of the partitions with the new ones (if you formatted partitions with the same UUIDs, skip this step)

Sudo nano /mnt/root/etc/fstab

Save the file and unmount all partitions

sudo umount /mnt/backup /mnt/root(/usr,/var,)

Mounting the image

squashfs is mounted like any other image - via a loop device. Kernel support for squashfs is included in many distributions, including Ubuntu, so it will be enough just to use the mount command with the loop option

sudo mount /media/backup/ubuntu-root.sqfs -o ro,loop /mnt

(The ro option is not required, because writing there still won't work)

Now you can copy any desired files from the image. Adding something in this way will not work, for this you will need to use mksquashfs again

When finished, unmount the image as a normal filesystem

sudo umount /mnt

rsync

Like cp , rsync operates on files, not block devices. The peculiarity of rsync is that it does not copy files that are already on the destination. By default, it checks the size and modification time of files, but you can also check the hash (usually this is done when increased security is needed).

Easy use

The syntax for rsync is similar to cp:

Rsync -a /mnt/root /mnt/backup

The -a option is very often enough, it provides the most necessary: ​​recursive copying of directories, saving owner and group information, and so on. To display detailed information about copying, the -v switch is used, be careful with it, you can skip the error message in the data stream. The -x switch ensures that rsync will not go beyond the specified filesystem.

The documentation for rsync describes a lot of options. For example, there are some that allow you to copy over SSH, or remove a file from the destination if it was deleted in the source directory.

"Smart" copying reduces system downtime. We run rsync directly on a running system, the data in which is constantly changing, rsync copies the data, say, for several hours. Then we put the system in read-only, run rsync again, now it copies only those files that have changed in these few hours. In a few minutes we have a complete copy of the original FS. At the same time, downtime was reduced by an order of magnitude compared to offline copying. And in some cases, one online copy will be enough without transferring the system to read-only.

Keeping Previous Copies

Strictly speaking, rsync is not a backup tool - it is a synchronization tool. This is important when making regular backups, because if any important file is deleted in the working directory of the source, rsync will delete it in the backup as well. To increase the safety of data, it is advisable to keep old backups. However, simply saving multiple copies will require a lot of hard disk space. If copies have many of the same files, then this leads to unnecessary redundancy. This problem can be solved by using hard links.

The bottom line is that in modern file systems (including Ext4) file addressing is performed in two stages: the file name points to a unique file number (index descriptor or i-node), and the data itself is associated with this number. Any filename is, in fact, a hard link to that number. Consequently, a file (data set) can have several names and be in different directories, and this allows you to eliminate redundancy if you need to duplicate files (after all, a hard link takes up little memory). The data itself is not deleted until the removal of the last hard link is requested.

A significant limitation is that hard links are only possible within the same file system.

Synchronizing the contents of the directory for the current backup with the source directory:

rsync \ --archive \ --delete --delete-excluded \ # remove from the backup non-existent in the source and excluded files--progress \ # print information about transfer progress"/home/user/Files/" \ # source directory"/backup/latest/" \ # directory for current backup--exclude="/Public/" # exclude unnecessary directories

In the /backup/latest/ directory, a copy of all the necessary files and directories from the source will be created and everything unnecessary will be removed.

Create another current backup without redundancy:

cp\ --archive\ # save all additional information about files--link \ # use hard links for files - eliminating redundancy"/backup/latest/" \ # source is the current backup obtained above "/backup/$(date +%Y-%m-%d_%H-%M-%S) /" # destination is a directory with the date in the name for convenience (see man date)

The next time a backup is created, rsync will delete files in the /backup/latest/ directory that have been deleted/excluded/modified in the source directory (changed files are first deleted and then written to a new version). However, only the names of the files (the same hard links) will be deleted, the files themselves (data) are saved, since hard links were created on them in the neighboring directory with the “cp” command.

Other tools

There are many applications for creating backups in Linux. You can search for "backup" in the Ubuntu Software Center to find backup software available in Ubuntu.

For a corporate environment and just for fairly large-scale and critical backup tasks, we can recommend understanding one of the most popular and powerful backup systems for Linux called Bacula

By the way, you can also find Russian-language manuals on the net.

Parted Magic

Parted Magic is another great one, but paid a distribution kit containing a whole collection of tools for backing up and restoring information, working with disks and partitions, as well as recovering lost data. It supports many file systems, LVM2 and RAID (both hardware and software) and contains such tools as fsarchiver , GParted , Clonezilla mentioned above, and everything that is required for the methods described in this article. In addition, the distribution kit includes a web browser and some other additional software. The distribution kit is translated into several languages, including Russian, and has a full-fledged graphical interface.

LParted

LParted is a full-featured LiveCD designed primarily for working with partitions. hard drives(HDD), permanently deleting or restoring data and testing equipment. LiveCD based on Lubuntu Linux. LParted is a functional analogue of Parted Magic.

Here I would add about SystemRescueCD and others

More about saving data

For important data, you can make a mirror partition on two disks. To do this, it is not necessary to have a RAID controller and disks of the same size - you can, for example, assemble a mirror from an 80 GB old screw and an 80 GB partition on a new one. Mirroring can be implemented using LVM or software RAID. However, this method is useless in case, for example, ~220V voltage hits the +5V bus or a meteorite falls on system unit computer.

IT geeks who have their own server at home can extend the idea of ​​mirroring and use DRBD. The same RAID-1, but hard drives are situated in different computers, which improves reliability.

A modern convenient solution is to back up data to the "clouds", for example, using Ubuntu One, Dropbox, http://www.adrive.com/ and others.

Neither mirroring nor replication on Ubuntu One will save you from accidentally pressing Delete, so make “classic” backups anyway. And one fine terrible day, all your labors and efforts will be rewarded.

This is a configurable setting that determines the type and use of the bootable USB drive that is created.

There are four download methods available in the Rufus utility, one of which must be selected from the drop-down list. In the list they are designated as MS-DOS, FreeDOS, ISO-image, DD-image. The first two are available only for computers and devices with a BIOS - they must be selected from the drop-down list in the "Partition scheme and system interface type" line.

This upload method in Rufus creates bootable flash drive with MS-DOS operating system. This is a disk operating system that may be required in the following cases:

• to update and flash the BIOS, it is safer to perform this procedure from under DOS;
• for diagnosing and testing a computer or laptop, since most utilities for in-depth diagnostics of RAM or a hard drive work only under DOS;
• if you need to access files on your hard drive with a missing or corrupted operating system;
• when using old console programs (for example, for programming peripheral devices), which may not work correctly under Windows;
• if you need access to file system hard drives of a computer or laptop at a lower level.

FreeDOS is an operating system similar to MS-DOS that is free under a free license and available from Rufus. It also allows you to update the BIOS, repair the hard drive using special programs, test RAM(RAM) and more. Actions are performed by special commands from the command line.

The ISO image boot method should be selected to transfer the finished boot disk ISO image to a USB flash drive. An ISO image must be created in advance using the programs designed for this. This method can be used in operating systems ah Windows, Linux, and others. Received boot disk can then be used for disaster recovery operating system and hard drive diagnostic tools.

Rufus DD image upload method

Used to create boot disks from images created in the operating system Linux program D.D. The fundamental difference of this loading method is that the DD program does not actually create an image, but makes a bit-by-bit copy of the given directory.

Rufus is not something unique in the field of creating bootable media, its main advantage is ease of use.

So, suppose you have downloaded an ISO file with a distribution image and prepared a flash drive.

The terminal command for writing the iso image of the distribution is as follows:

sudo dd if=path to .iso of=/dev/sdb

Where instead path to .iso you need to specify the path to the iso file from your folder Downloads.
sdb at the end of the command, this is the flash drive on which the image will be written. To identify your flash drive, run in a terminal:

And if your flash drive is defined differently, change it in the command.

To correctly enter the path to the file in the command, open the folder with the file, open the nearby terminal and drag the file into it:

The terminal will display the path to the file and the .iso file itself. Copy it without quotes and paste it into the command, and in the terminal press the keyboard shortcut ctrl+c to reset and display a new prompt string.

The easiest way to create a command is in a text editor.
I got the following command:

Now insert the USB flash drive into the port of the computer, copy the received command from text editor, type in the terminal and execute (press Enter):

The terminal cursor will blink and it seems that nothing is happening, but in fact there is a recording. Wait for the image recording to finish and at the end you should see something like the following in the terminal:

That's all. Your flash drive has become bootable and you can now install the distribution on any computer.

To create a hard disk image, it is not necessary to use utilities like Acronis True Image or Norton Ghost, a fairly simple dd utility that comes with most Unix-like operating systems (Linux, FreeBSD, Solaris, etc.) This article shows you a simple way to create a hard disk image backup using dd. The first step is to prepare for the backup. In this article, we introduce the following notation:

• /dev/sda - disk to be imaged;
• /dev/sdb - the disk where the image will be written to.

Substitute your own values ​​if necessary.

Preparing to create a hard drive image

The first step is to boot from any available Live-CD that has the dd utility and enter command line as superuser. Create a mount point for backup.

mkdir /mnt/backup

Mount the hard drive on which you want to save the image.

Create a hard drive image

dd if=/dev/sda of=/mnt/backup/sda.img bs=8M conv=sync,noerror

• if=/dev/sda - copy the entire hard drive sda;
• of=/mnt/backup/sda.img - copy to /mnt/backup/sda.img;
• bs=8M - set the size of the hard disk cache to speed up the copying procedure (otherwise the data will be dumped in small portions of 512 bytes);
• conv=sync,noerror - tells dd to copy bit-for-bit, ignoring read errors.

To reduce the size of a hard disk image, you can compress it with any archiver.

dd if=/dev/sda bs=8M conv=sync,noerror | gzip -c > /mnt/backup/sda.img

Restoring a hard drive image

To restore a hard disk image, you need to follow the procedure reverse to the procedure for creating this image.

dd if=/mnt/backup/sda.img of=/dev/sda bs=8M conv=sync,noerror

When using compression in parallel, you must unzip the image.

gunzip -c /mnt/backup/sda.img | dd of=/dev/sda conv=sync,noerror bs=8M

Migrating the system to another hard drive

To migrate the entire system to another hard drive, you must set the destination of the new drive as the destination.

dd if=/dev/sda of=/dev/sdb bs=8M conv=sync,noerror

Then, if necessary, install the boot from this hard drive. Provided that the new hard drive is larger than the old one, it will remain unallocated area. It should be marked and formatted according to existing requirements.

Copy statistics in dd

The main disadvantage in dd is the lack of a visual representation of the statistics of the execution of the copy procedure. However, this disadvantage can be easily circumvented. It is enough to connect to another terminal.

Determine the process number under which dd is running.

Send periodically this process kill -USR1 dd_process_number.

watch -n 5 kill -USR1 dd_process_number

• watch -n 5 - run command every 5 seconds;
• kill -USR1 dd_process_number - show copy statistics.

Let's use the dd utility to create an image of a flash drive with archiving free space. Backup Image useful if the original suddenly stops working with important information. For example, a flash drive with private keys of electronic signatures of the organization's management. So, we have a 4 GB / dev / sdd flash drive, the information on which takes about 90 MB.
du-sh/run/media/aleksey/Transcend

89M /run/media/aleksey/Transcend

All commands are executed on behalf of the user root. Or, in the respective distributions, before the commands, add sudo.
fdisk -l /dev/sdd

Disk /dev/sdd: 3.7 GiB, 3904897024 bytes, 7626752 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0xc653eaa4 Device Boot Start End Sectors Size Id Type /dev/sdd1 2048 7628543 7626496 3.7G b W95 FAT32

Disk /dev/sdd: 3.7 GiB, 3904897024 bytes, 7626752 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0xc653eaa4 Device Boot Start End Sectors Size Id Type /dev/sdd1 2048 7628543 7626496 3.7G b W95 FAT32

By creating a simple image with the command
dd if=/dev/sdd of=sdd. iso bs=4M conv=noerror,
we doom ourselves to storing a 4GB file. What if the flash drive was 64GB? And not alone? This problem will help us to solve the usual archiver, take the standard gzip.
dd if=/dev/sdd bs=4M conv=noerror | gzip - c > sdd . iso. zip
where is the key -c allows you to work with standard output.
After the work is completed, let's look at the resulting file. ls-al sdd*

The resulting file is approximately 25MB in size. A real saving of disk space even compared to a 4GB file!
To restore a flash drive from an image, we use the reverse order of the commands.
gunzip - c sdd . iso. zip | dd of=/dev/sdd conv=noerror bs=4M

You can also archive hard disk images, where the volumes are an order of magnitude larger.

By the way! For clarity of the process, due to the lack of a progress bar in dd, I suggest using a small utility progress- Coreutils Progress Viewer. Installing it on Fedora is easy.
dnf install progress
For other distributions, the required repository can be found at https://pkgs.org/download/progress .
After installing and running the utility with the command watch progress in the second terminal (in the first one we have an archiver and dd running) on ​​behalf of the same user, we will see something like this.

manprogress will show you various useful keys of this utility.