How To Compress Files Using The Linux Command Line

Introduction

There are a number of different ways to compress files using the Linux command line. In this guide you will find practical examples showing how to use the zip command to compact and organize files within your file system.

Common uses for zip files include the need to save space and for copying a large number of files from one place to another.

If you have 10 files which are all 100 megabytes in size and you need to transfer them to an ftp site, then depending on your upload speed this could take a considerable amount of time.

If you compress all 10 files into a single zip archive and the compression reduces the file size to 50 megabytes per file, then you only have to transfer half a gigabyte of data instead of the full gigabyte required to send an uncompressed file.

How To Create An Archive Of All the Files In A Folder

To start with here is a simple command showing how to create an archive of all the files in the current folder.

Imagine you had a folder of songs with the following MP3 files in it:

AC/DC Highway To Hell
Night Prowler.mp3
Love hungry man.mp3
Get It Hot.mp3
Walk all over you.mp3
Highway to hell.mp3
If you want blood you got it.mp3
Show down in flames.mp3
Touch too much.mp3
Beating around the bush.mp3
Girls Got Rhythm.mp3

To create an archive called ACDC_Highway_To_Hell.zip enter the following command:

zip ACDC_Highway_To_Hell *

Text will scroll up the screen showing the files as they are being added.

Type ls -lt *.zip and you will see that the archive has been created.

How To Include Hidden Files In An Archive

The previous command is fine for archiving all of the files in a folder but it only includes files that aren't hidden.

But it's not always so simple. Imagine you wanted to zip up your home folder so that you can back it up to a USB drive or external hard drive.

Running the command ls -lt will show a number of folders including Pictures, Templates, Music, Public etc.

Now run the command ls -a. You will now see folders such as .gnome and .config which will also need to be included in a zip file. Your .bashrc file is also a hidden file.

To compress all the files in a folder including the hidden files run the following command:

zip home * .*

This will create a file called home.zip with all the files within the home folder. (You must be in the home folder for this to work). The problem with this command is that it only includes the files in the home folder and not the folders, which brings us to the next example.

How To Archive All Files And Sub Folders In A Zip File

To include all of the files and subfolders within an archive run the following command:

zip -r home .

How To Add New Files To An Existing Zipped Archive

If you want to add new files to an existing archive or update the files in an archive, simply use the same name for the archive file when running the zip command.

For instance, imagine you had a music folder with four albums in it and you created an archive called music.zip to keep as a backup. Now imagine one week later you had downloaded two new albums.

To add the new albums to the zip file, simply run the same zip command as you did the previous week.

For example to create the original music archive run the following code:

zip -r music /home/yourname/music/

To add new files to the archive run the same command again.

If the zip file had a list of files in it and one of the files on the disk had changed then the amended file will be updated within the zip file.

You can also supply the -u command to update a zip file which does pretty much the same thing as the standard zip command but if the zip file doesn't already exist a warning message is displayed.

This helps if you know you are supposed to be updating a file and accidentally enter the wrong zip file name.

How To Update The Existing Files In An Zipped Archive

If you have a zip file that is supposed to contain the same file names every time and you want to update that file with any changes that have been made to those files then the -f switch helps you do this.

For example imagine you had a zip file with the following files:

/home/yourname/documents/file1

/home/yourname/documents/file2

/home/yourname/documents/file3

/home/yourname/documents/file4

/home/yourname/documents/file5

/home/yourname/documents/file6

Now imagine that during the week you added 2 new files and amended 2 files so that the folder /home/yourname/documents now looks like this:

/home/yourname/documents/file1

/home/yourname/documents/file2

/home/yourname/documents/file3

/home/yourname/documents/file4 (updated)

/home/yourname/documents/file5 (updated)

/home/yourname/documents/file6

/home/yourname/documents/file7

/home/yourname/documents/file8

When you run the following command the zip file will contain the updated files (file4 and file5) but file7 and file8 will not be added.

zip zipfilename -f -r /home/yourname/documents

How To Delete Files From An Zipped Archive

So you created a massive zip file with hundreds of files and now realize that there are about four or five files within the zip file that you don't need to be there. Without having to zip up all those files again you can just run the zip command with the -d switch as follows:

zip zipfilename -d "name of file in archive"

For example, if you had a file in the archive with the name home/documents/test.txt you would delete it with this command:

zip zipfilename -d home/documents/test.txt

How To Copy Files From One Zip File To Another

If you have files in one zip file and you want to copy them to another zip file without extracting them first and re-zipping them, use the -u switch.

If you realize that you have a zip file called "variousmusic.zip" with music from various artists one of which is AC/DC. you can copy the AC/DC songs out of the variousmusic.zip file into your ACDC.zip file using the following command:

zip variousmusic.zip -U --out ACDC.zip "Back_In_Black.mp3"

The above command copies the file "back in black" from variousmusic.zip to ACDC.zip. If the zip file you are copying to doesn't exist it will be created.

How To Use Pattern Matching And Piping To Create An Archive

The next switch is a really useful one as it lets you use the output of other commands to insert files into your zip file. For instance, imagine you want to create a file called lovesongs.zip which contains every song which has the word love in the title.

To find the files with love in the title you can use the following command:

find /home/yourname/Music -name *love*

The above command isn't 100% perfect because it picks up words like "clover" as well, but you get the idea. To add all the returned results from the above command to a zip file called lovesongs.zip simply run this command:

find /home/yourname/Music -name *love* | zip lovesongs.zip -@

How To Create A Split Archive

If you are backing up your computer but the only media you have available for backing up to is a set of blank DVDs, then you have a choice. You can keep zipping up files until the zip file is 4.8 gigabytes (well just under) and burn the DVD, or you can create something called a split archive which keeps creating a new archive in a set after it has reached the limit that you specify.

For example:

zip mymusic.zip -r /home/myfolder/Music -s 670m

How To Customise Progress Report Of Zipping Process

There are various ways to customize the output that appears whilst zipping is in progress.

The switches available are as follows:

-db = displays how many bytes have been zipped and how many are left to go
-dc = displays a count of files zipped and how many are left to go
-dd = displays dots for every 10 megabytes of file that has zipped
-ds = sets how often dots appear (ie 10m, 1g, 100k)
-du = displays the uncompressed size of each file

For example:

zip myzipfilename.zip -dc -r /home/music

How To Fix A Zip File

If you have a zip archive which is broken, you can try and fix it using the -F command and if that fails the -FF command.

This is useful if you have created a split archive using the -s switch and you have lost one of the archive files.

For example, try this one first:

zip -F myfilename.zip --out myfixedfilename.zip

and then

zip -FF myfilename.zip --out myfixedfilename.zip

How To Encrypt An Archive

If you have sensitive information that you want to store in a zip file use the -e command to encrypt it. You will be asked to enter a password and to repeat the password.

For example:

zip myfilename.zip -r /home/wikileaks -e

How To Show What Will Be Zipped

If you know you are going to be creating a large archive, make sure that the correct files are going to be added to the zip file you can see the expected results of a zip command by specifying the -sf switch.

For example:

zip myfilename.zip -r /home/music/ -sf

How To Test An Archive

After backing up files to a zip file, it is tempting to save disk space by deleting the original files. Before you do that, it's a good idea to test the zip file works properly.

You can use the -T switch to test that the zip file is valid.

For example:

zip myfilename.zip -T

Output from this command when an archive is invalid may be something like:

zip warning: missing end signature--probably not a zip file

Remember you can try the -F command to fix broken zip files.

It is worth noting that the -T can produce false positives in that it says a zip file is corrupt even though when you open it you can extract all the files.

How To Exclude Files

Sometimes you will want to exclude certain files from a zip file. For example if you have copied the files from your phone or digital camera you will have a mixture of videos and images. You might wish to zip up the photos to photos.zip and videos to videos.zip.

Here is one way to exclude the videos when creating photos.zip

zip photos.zip -r /home/photos/ -x *.mp4

How To Specify Compression Level

When you compress files into a zip file it works out whether you compress the file or just store it. mp3 files, for example, are already compressed so there is little point in compressing them further and thus they are usually just stored as is within a zip file.

You can, however, specify a compression level between 0 and 9 which compresses a file further. This takes longer to do but can make significant space savings.