User login

File Compression for Bandwidth and Storage Efficiency

(May 2005) posted on Mon Jun 06, 2005

Discover what data-compression options are available and how to select the best ones for your files.


By Mark A. Coudray

Working with digital image files leads to the inevitable situation of having to store or move them. In the case of storage, we may need to archive the temporary working files, intermediate work, and the final image for future reference and retrieval. Moving files could involve transfers over a network from a remote file server, from workstation to workstation, from creator to client, or over the Internet via attachment to e-mail, from Website to client, or by file transfer protocol (FTP) from a dedicated server. Regardless of the means, we are concerned with speed of transmission, assurance that the file(s) will arrive intact and uncorrupted, and minimizing our consumption of bandwidth.

Increases in digital-camera pixel densities and computer speed have led to decreases in awareness about file size and its impact on transmission bandwidth. Old LANs passed data at 10 Megabits/sec, while transfer speeds in the Gigabit range are much more common today. Slow dial-up modem connections limited file-size transmission. Today, broadband DSL, cable, T-1, and fiber-optic lines are very common. All of these advances make designers lazy. They create graphics at very high resolutions with dozens of high-res layers. Files that were 5-10 MB in the past now arrive in the hundreds of Megabytes—even into the Gigabytes. Almost all of the information that makes these image files so large is redundant and can be flattened and compressed.

You can store files in their final, archival form on CD as long as they're not larger than about 700 MB. But the huge image files that we receive these days may have to be archived on DVDs, which are higher-density media that can store data in the 4-5 GB range. Also keep in mind that the drive you use to write to CDs may not be able to write to DVDs, so saving huge image files may require the purchase of additional hardware.

Finally, we must consider the time and data-integrity issues involved in the transmission of digital files over networks, intranets, and the Internet. Time spent sending and receiving files varies according to transmission method, speed, and bandwidth of the network. Many ISPs still charge for bandwidth, in addition to the basic monthly access charges. This is a particular problem if you are co-locating your own image servers. A low-cost ISP plan will typically allow monthly access to 5 GB of traffic before you may begin to incur bandwidth fees. Many screen printers are really in the digital file business, so it is not uncommon for these shops to move 5 GB of data per day. To minimize this, we want our files to be as small as possible in order to keep the transmission time as short as possible and our bandwidth fees at a manageable level. File compression lets us do just that.

Lossy vs. lossless compression

File compression falls into two different classes: lossy and lossless. With lossy compression, a mathematical algorithm is used to discard what appears to the naked eye to be redundant, unused, or imperceptible information. We are actually throwing away data under the pretext that our eyes won't be able to detect the lost information. Screen printing is a relatively low-resolution printing process, so it is generally capable of tolerating fairly large amounts of compression.

Lossy compression can result in very large reductions in file size. It is not uncommon to have files one fifth to one twentieth their original size after compression. While this is highly attractive, it comes with a cost—image degradation. The negative impact of lossy compression on an image can affect it once or progressively, depending on what type of compression method is used.

Lossless compression is much more reliable, but the corresponding reductions in file size are not as impressive. They are usually limited to a range of half to one tenth of the original file size. However, there are exceptions to this that I'll discuss in a moment. With most types of lossless compression, you will need some type of software to encode or compress the image, and the recipient will need to decode the file to restore it to its original size.

Types of lossy compression

The most common type of lossy compression is JPEG or JPG. JPEG was developed by the Joint Photographic Experts Group as a method of compressing large amounts of continuous-tone information. This is the default method of compression for digital cameras. It is a progressive compression method, which means you can decide how much compression your image can tolerate.

The higher the pixel density of your original image, the less noticeable any compression damage will be. Most digital images produced at 5-megapixel resolution will not show any noticeable loss. However, when these images are enlarged, the artifacts become readily apparent. The more you compress the file, the greater the damage to the image and the greater the generation of visible artifacts in the image. Programs like Adobe Photoshop allow you to save your files in JPEG format and to choose how much compression you will use. JPEG does a very good job on a single compression cycle for most images that will be screen printed.

Image compression in the range of 10:1 to 20:1 usually does not create noticeable degradation. However, it is very important to note that if you choose JPEG and continually re-save the same image in JPEG format, you will progressively degrade the image. There will be very noticeable artifacts in the image after only two or three cycles that can rarely be corrected. For this reason I avoid JPEG formatting for any kind of archiving.

One of the biggests problems with JPEG formatting is the effect it can have on color separating. If the original image is in RGB format, compressed with JPEG, converted to CMYK, and re-saved for separation, there is a high likelihood that artifacts will show up in the final separated image. I recommend you use JPEGs for Web graphics and for transmission to clients as low-resolution FPO (For Position Only) placeholders, not as files intended for final separation—especially if they're enlarged from the original capture resolution.

Options for lossless compression

We have an array of possibilities available to us when it comes to lossless compression. Programs like StuffIt and WinZip are commonly used to reduce file size for archiving or transfer over the Internet. You may have one of these already on your computer, as these types of applications often come loaded with the operating system. With this approach, you must remember that if you send a compressed or archived file, the receiver must also have the software to expand the compressed file. Years ago, exchanges of compressed files were best kept to a specific platform—PC to PC or Mac to Mac. But now it's usually no problem at all to encode and decode compressed files between PCs and Macs. Sticking with more popular compression formats will help prevent any cross-platform conflicts.

My favorite application is StuffIt from Allume Systems, Inc. (previously Aladdin Systems, Inc.). The commercial version of the program will compress files for Mac or PC and can handle .sit, .zip, and other compression formats. The StuffIt Expander utility is free, but it's limited to decompressing only. However, it will open all known forms of compression on either platform. You can download StuffIt Expander from www.stuffit.com.

If you use TIFF (Tagged Image File Format), one of the most useful lossless compression formats is LZW. It works on the basis of areas of color. The more flat the color and the larger the areas of that flat color, the greater the compression possible. Depending on the image, LZW can compress files to between one half and one tenth their original sizes. It can handle multiple channels and layers as long as they are in TIFF format. Photoshop gives you the option of saving all TIFF images with LZW enabled. This works very well if you plan to keep the image in Photoshop. But as soon as you move the image to another application, that program must have the ability to decode the LZW compression; otherwise you will have what looks like colored static when you open the image.

The next option is fractal-based compression. This technology has been around for a number of years and is now accessible very inexpensively for both the Mac and PC. It allows for huge file reductions with little if any degradation. It is excellent for file transmission and for the compression and transport of large-format digital images, and it works with multi-channel and multi-layer graphics. The only real limitation is that once the original image is encoded, the receiver must have the decoding software to open the file. For this reason it is essential that you confirm that the receiver can open the file before you send it or transport it.

The last type of compression I want to discuss is Adobe's Portable Document Format (PDF). More people are using PDF each day, because it is platform independent—any computer can receive, open, and view a PDF file as long as Adobe Acrobat Reader is installed. This is freeware available from www.adobe.com. PDF is best used for transmitting and viewing assembled documents. All necessary fonts and graphics can be embedded so that the document will display as expected on any computer. PDF files are generally one tenth to one fifteenth of the original file size.

PDF compression is completely transparent to the user. There are many different ways to achieve PDF compression. The best way to make a PDF file is to use Adobe Acrobat Distiller. Distiller gives you many different options to choose from. For instance, you can choose compression for high-resolution printing or for use on the Web. You can also include different types of security options to prevent unauthorized duplication or copying of the file.

PDF Writer software was popular in the 1990s. The computer recognized PDF Writer as a printer and would enable users to generate PDFs by simply "printing" the document. Nowadays you can often export a PDF directly from the desktop-publishing or graphic-design software you use. Current versions of Quark XPress, Adobe InDesign, Adobe Illustrator, and others support this function.

The need for compression

File compression is another area of the business where you must be proficient. Simply receiving files that have been arbitrarily compressed is not good enough. You must know when it is appropriate to use each type of compression and how to educate your clients about sending and receiving compressed graphics files. More and more companies find the need to send and receive graphics via networks and the Internet each day, which means you will receive a variety of compressed file types as a regular part of accepting incoming jobs. Compression technology continues to develop, so knowing what your options are and how they affect the quality of your final reproduction are critical to your success.


Terms:

Did you enjoy this article? Click here to subscribe to the magazine.