Archive and compress data using Linux

Behind the phrase “data archiving” is the basic idea of backing up files or entire di­rec­to­ries and storing them in a secure location, often in a com­pressed form. For reasons of data security, archiving was an important factor in server en­vi­ron­ments at an early stage: Orig­i­nal­ly server data was stored on tape drives – a backup method which is still used for large data volumes. To make this archiving method as efficient as possible, the packing program tar (short for tape archiver) was developed for Unix systems in 1979. With the help of tar, files and di­rec­to­ries can to this day still be packed into a single data file and then recovered with the user rights still remaining intact – as long as the source and target both support the Unix or Linux data file di­rec­to­ries.

For the archiving process to free up ad­di­tion­al storage space, .tar-data files are often com­pressed with the help of different tools, like gzip, bzip2, or lzop. But what are the different com­pres­sion programs and formats? And why are they still so important today for systems like Linux tar?

There are a number of free com­pres­sion tools for Linux dis­tri­b­u­tions that all have one thing in common: they can be operated via command line or terminal. Short commands can quickly compress data files, such as HTML documents, to save storage space and bandwidth when sending via networks or the internet. In addition, there are standard graphic in­ter­faces for these tools, as well as archive managers, which combine several com­pres­sion programs – that must be installed as well – into a single visual user interface. Control of the graphic interface obviously requires ad­di­tion­al system resources, which is why use of a terminal generally remains the best choice for com­pres­sion.

The main dif­fer­ence between the in­di­vid­ual programs is the com­pres­sion rate, which is ac­com­pa­nied by different com­pres­sion durations. In most cases, however, different modes can also be selected in the tool itself to offer either the best possible storage reduction or the quickest possible com­pres­sion time. Another feature that dif­fer­en­ti­ates com­pres­sion software is the output format. Due to the different al­go­rithms used by the various programs, com­pressed files have different pack format and require specific programs to be unpacked.

In terms of speed, the free pack program is still among the top options, which is why common web server software such as Apache, IIS, or NGINX usually implement it in the form of their own modules to answer user queries with com­pressed data packets in the shortest possible time. Ad­di­tion­al in­for­ma­tion about the func­tion­al­i­ty and use of the GPL-licensed com­pres­sion tool can be found in our article on the program.

For the standard com­pres­sion process it’s not necessary to specify options. This is only required if you want to change com­pres­sion settings, access the overview menu, or unpack a .bz2 file. For example, to pack the text document test.txt, you just need to complete the command

to delete the original file and replace it with the com­pressed file test.txt.bz2. By placing the documents together, you can also package multiple files with a single command:

If you want to de­com­press a packed document, it’s necessary – as mentioned earlier – to set the cor­re­spond­ing option pa­ra­me­ters (-d):

Here’s an overview of some other bzip2 command options:

The archiving program tar has been in operation for over 30 years and has hardly lost any of its value. Partially, this is because the tool allows data to be archived while retaining file de­f­i­n­i­tions. Mainly, though, it’s because it allows for the packing of complete file di­rec­to­ries. This makes tar the perfect partner of com­pres­sion­al tools like gzip and bzip2, which only allow for single file data com­pres­sion.

In the first step, the packing program compiles all data files in a selected directory into a single archive file without unlinking any of the contained files. In the second step, the files are com­pressed using one of the specific com­pres­sion programs. As a result of this com­pres­sion, which is either described as pro­gres­sive, compact, or solid, the archive files are given extended formats, such as .tar.gz (.tgz for short) or .tar.bz2 (.tbz2 for short). The packing program also allows for the sub­se­quent unpacking of such files (e.g. file type .tar.gz).

The com­bi­na­tion of tar and a com­pres­sion tool isn’t required, so you can also combine files in an archive that you haven’t pre­vi­ous­ly packaged or don’t want to compress. For example, if you want to bundle the un­com­pressed test documents test.txt and test2.txt in the same archive named archive.tar, the following command will suffice:

To unzip this archive on Linux, replace the –c (create new archive) parameter with –x (extract files from archive). If not only a certain archive component is to be unzipped, then the file(s) can be omitted:

Al­ter­na­tive­ly, if you aim to pack a com­pressed archive – for example, on the basis of the gzip com­pres­sion, including the extended for­mat­ting .tar.gz – then tar also offers cor­re­spond­ing options. Since the program has im­ple­ment­ed options for com­pres­sion and de­com­pres­sion with the bzip2, xz, compress, and gzip pack programs, this is also possible with a single command:

 The command to unpack .tar.gz differs from the equiv­a­lent for un­com­pressed di­rec­to­ries only through the spec­i­fi­ca­tion of the pack program parameter:

In addition to the pre­vi­ous­ly listed command options for easy file archiving, there are several ad­di­tion­al pa­ra­me­ters to specify the pack or unpack process. These include the com­pres­sion methods already mentioned, options for setting up di­rec­to­ries, as well as options for checking and pre­view­ing the tar archives:

Update contents of an archive (doesn’t include sub­di­rec­to­ries!)

Expand contents of an archive

Compare contents of an archive with the file system (run in the archive directory!)

File Roller is a graphic user interface for various com­pres­sion tools and packing programs, which is standard for the operation of command lines. The archive manager is available for the GNOME and Unity desktop en­vi­ron­ments, and has been dis­trib­uted under the GNU General Public License since 2001. It allows contents of various archive files to be viewed, as well as for files to be extracted, deleted, or added to them. It is also possible to create new com­pressed or unchanged files and archives, as well as to convert them to another format. For this purpose, the main window of the software offers various buttons and menus alongside a drag-and-drop function. In addition to tar archive formats like tar.gz, File Roller supports the following formats:

• .7z
• .tar
• .gzip
• .bzip2
  
• .ar
• .jar
• .cpio

File Roller is pre­in­stalled on some Linux dis­tri­b­u­tions, such as Ubuntu, by default, but can also be installed manually using the re­spec­tive package manager or from the official homepage. An al­ter­na­tive for the desktop en­vi­ron­ment KDE is Ark.