Important Note: If you are not a techie, do not be frightened away by the words “digital age” in the title of this post; this piece is written for you – for everyone. If your interests lie more in the arts and their appreciation than in the sciences and technology, this post nonetheless has an important message for you!
Remember Those Ugly, Over-Reproduced Cartoons?
Most of us have seen those really funny cartoons that made the rounds through countless office complexes and faculty rooms. Someone, somewhere, cut out a printed “original” from the funny pages of the newspaper or perhaps the New Yorker magazine (great cartoons back then). They brought it to the office and copied it (on a Xerox machine in those days) to share laughs with their friends and colleagues. As the copies made the rounds, copies of the copies were made and those copies themselves multiplied in all directions as they traveled from employee to employee. After a while, most recipients were looking at copies of copies of copies……well, you get the idea – and the images were terrible! Why should that be?
Thou Cannot Reproduce or Store “Things” without Degradation
Is This Some Divinely-Ordained Law of Nature?
Nature has long had a way of exacting a price on anything that man attempts to reproduce, transmit, or store for long periods of time – as exemplified by our cartoon example, browned pages, faded photographs, and LP sound recordings on vinyl which tend to get “noisy” over time. That inevitable degradation seemed like a veritable “law of nature,” operating without exception as does Newton’s law of universal gravitation or the laws of thermodynamics. In the early days of radio, noisy “static” fell in the same category as the imperfections which attach themselves to seemingly all reproduction/transmission/storage processes. Noise on radio transmissions due to static was very irritating, and radio engineers expressed the hopelessness of trying to eliminate static by predicting that, “Static, like the poor, will always be with us.” The development of FM (frequency modulation) broadcasting in 1935 by Edwin Howard Armstrong greatly reduced static. Frequency modulation involves a very different process for radio transmission compared to the old familiar AM, or amplitude modulation; while drastically reducing the annoying effects of static, FM did not completely eliminate it. Armstrong’s announcement and demonstration of FM in 1935 ranks high in the annals of technical innovation even though it did not completely solve the static interference problem.
The remainder of this post describes, in layperson’s terms, a technological miracle for lovers of the arts, particularly the archivists dedicated to the survival and future transmission of man’s finest accomplishments.
Digital Information Processing and Storage to the Rescue!
The digital age has not only provided us with obvious advantages in distributing and storing various forms of information, it has actually solved the problem of reproduction, transmission and storage degradation.
Here is how it works – in layperson’s terms! As referenced in my earlier post explaining the basics of digital technology (April 7, 2013; “Digital” Technology: What Does That Mean?), the term “digital” refers to storing and processing all information as numbers in a computer. What kind of information? Anything can be digitized. You do that when you scan/copy on your printer or you record sounds using your mobile phone. Everything can be digitized – video, still images, documents, and sound. The numbers stored in a computer which represent your information are expressed in data “bits” using the binary, or two-digit number system. The two digits utilized are the ubiquitous computer data 1’s and data 0’s we all hear about. A data bit is assigned the value of either “1” or “0”. On a magnetic disk drive (still the main data storage vehicle for larger computers), 1’s and 0’s are determined by the magnetic orientation of tiny “spots” on the disk’s magnetic surface film. The simplest illustration: Magnetization of the surface film in one spatial direction implies a data “1” for a given bit; magnetization in the other direction implies a data “0”. These data bits are organized in specific data bit- arrays which are determined by format standards such as that used with .jpg photo images or that programmed into the specific host device used to store data (such as a disk drive). As we cram more and more data bits onto a given surface area of a magnetic disk (economics), the magnetized “spots” become smaller in size and consequently more difficult to correctly detect and “read” as a data “1” or a data “0”.
So, you might ask, “How, then, is digital storage any different from the noise susceptibility of the “analog” system as represented by the old Xerox technology and the scratchy vinyl grooves in LP music albums?” Digital data is, in fact, also susceptible to “noise” degradation meaning that data 0’s occasionally become mistakenly detected as data 1’s – and vice-versa. That certainly constitutes a noise-susceptible process. So, back to the original question, “How is digital storage/processing different?”
Digital Error Correction Codes (ECC) Provide the Answer
Stay with me, here – what follows is not difficult. By appending a small number of additional binary bits (known as ECC bits) to the end of the bit stream which comprises the information data sequence, virtually ANY statistically plausible errors made in retrieving the data sequence (accurately detecting data 1’s and data 0’s) can be automatically corrected by the computer or digital device using the appended ECC bits. The end-user never sees the process! Typically 10 – 15% additional ECC bits are appended to the information data bit stream in order to implement the robust self-correcting capability of ECC. Hopefully, the overview of the process as just presented was relatively simple to digest.
What is NOT simple, indeed very complex, is the mathematical basis used to determine those appended error-correction data bits so they can provide the promised level of data protection. Do not worry; we are NOT going there – here! The ECC bit-values (“1” or “0”) are determined from the data bit stream itself as both are being recorded together. Complex math algorithms do this “on the fly” as we say. During my career as an electrical design engineer responsible for recording/decoding data to/from magnetic storage devices, I attended several highly specialized technical course-seminars dealing with error-correction codes. Engineering/Math PhD theses are written on the subject.
Brief Personal/Occupational Sidebar:
They did not teach this material in college back in my day….pause for laughter! In fact, vacuum tubes were still giving way to transistorized electronics in the late 1950’s! My not-so-subtle message to aspiring engineers: Go for it, but be prepared to absorb new technologies and to re-invent yourselves several times over during the course of your career – just to keep up! End of sidebar.
Profound Implications for Society and Lovers of the Arts
Now we can endlessly reproduce data with no degradation whatsoever, and that is wonderful news for archivists and society in general. For the arts, it is a god-send. Photographs, painted images, recordings, documents, old newsreels, and motion pictures need never be lost or hopelessly degraded over the decades to come.
There is one caveat, one price to pay, however: Even the laser-recorded CD and the DVD are not impervious to chemical/material changes over the years which could ultimately degrade the binary data bits stored on them and render them unreadable/uncorrectable even with digital ECC (Error Correction Codes). These codes cannot correct data which is severely compromised; they are designed to correct all statistically plausible random errors due to noise and slight imperfections in the media upon which they are recorded. If a CD/DVD substrate reaches a notable level of deterioration which results in massive data detection errors, ECC will be ineffective and the data compromised.
Periodic Data “Refreshing” is Mandatory
Want to keep an old movie forever? Digitize it carefully along with ECC bits and periodically “read” the data digitally with the aid of ECC. Using that ECC-corrected result which is a faithful copy of the original data, re-record it on fresh, new media – along with the proper ECC bits. How often? Check with the chemists and the material folks who understand media and degradation effects. Perhaps once every twenty years will be sufficient, keeping in mind that each time the movie is transferred using ECC, it will be a perfect copy UNLESS the media has degraded beyond the formidable capabilities of ECC to correct errors. At that point, the information will be lost or compromised.
The real challenge here? Refreshing the data content periodically implies retaining/maintaining a compatible “player/format” for years or decades capable of reading the stored information data 1’s and 0’s and temporarily transferring them to a digital device (computer). From the computer, the data 1’s and 0’s are input to the latest-technology storage device which, using perhaps a new data array format, records it on the latest-technology media – still in binary 1’s and 0’s format. The binary format with its data 1’s and 0’s will continue to be used; only the organizational format used for storing those data bits in digital devices will change with time. Future firmware engineers will know how to take data 1’s and 0’s from today’s organizational format and put them into the newer format standards which will evolve. Accordingly, the data itself will never change.
Summarizing the “Whole Mish-Mash”
The upshot for you, me, and society? Everything having to do with sight and sound can be represented digitally and stored in computers as numbers using no more than ordered-arrays of 1’s and 0’s, those two (binary) states which determine the value of one data bit.
Let us step away from details of the engineering world for one moment and ponder a profound concept: That movies, pictures, and sound can be represented by nothing more than extensive ordered-collections of data 1’s and 0’s. It is both an incredible reality and a tribute to our technology!
Your CDs and DVDs represent nothing more than a huge, ordered collection of data bits which, when reconstructed by computer circuitry on playback, resurrect all the pleasures of Mozart, Tommy Dorsey, The Beatles, Michelangelo, Van Gogh, and so on.
And here is the real coup for consumers, archivists, and those of us who love to ponder such things: In the DIGITAL world, everything digitized can be copied over and over with absolutely no degradation in quality. Note: I do not advocate nor do I support outright piracy; I merely make the point that even if the substrate material in our CDs and DVDs might crumble in 20 years, the digital data stored on it can theoretically be re-recorded on new media well before that time in order to precisely preserve it. Tenth -generation copies of copies using digital data with ECC will be as good as the original – a boon for archivists and for society.
It seems rather like cheating mother-nature by skirting one of her seemingly fundamental and inviolable laws. It is truly a blessing that great works of art and music as well as our favorite photographs and movies can now be perfectly preserved forever. For Hollywood, in particular, the DIGITAL age came just in time as the old celluloid film copies disintegrate. How wonderful it is to see Fred Astaire and Ginger Rogers dancing on a clear, shimmering silver screen at the nearby Stanford Theatre, increasingly the product of DIGITAL preservation….. and restoration (yet another story).
For a great example of film preservation/restoration, see the Criterion Collection DVD of the 1955 movie Summertime with Kathryn Hepburn. This was filmed on location in Venice, one of Hollywood’s very first overseas ventures. Check the vibrant colors and the splendid photography of Venice and compare the restored film to the faded, drab, unrestored color film trailer which is included on the disk!
Do you believe in miracles? You should!