Back Grazing in Familiar Pastures; Shannon’s Milestone Book on Communication Theory


Do you use the internet and personal communication devices such as cell phones? Since you are here, you must! Who doesn’t these days? One look at people in public places with eyes riveted on phone screens or tablets speaks to the popularity of personal communication. DSL (Direct Subscriber Line) services like AT&T’s U-Verse reliably bring broadband television and the Internet into our homes over lowly, antiquated, but ubiquitous twisted-pair phone wire connections. That miracle is only possible thanks to the power of modern digital communication theory.

The gospel of the engineering/mathematics that enable that capability is this 1949 book edition by Claude Shannon of AT&T’s famous Bell Telephone Laboratories. Its title: The Mathematical Theory of Communication. “Bell Labs” made immense contributions to our body of technical knowledge over many decades through its familiar, blue-wrappered Technical Journal. The authors of its featured papers include many of the most important scientists, engineers, and mathematicians of the past century.

Claude Shannon was one of them; the contents of his 1949 book, published by the University of Illinois Press, first appeared in the Bell System’s Journal in 1948. The paper’s unique and important approach to reliably sending electrical/optical signals from one point (the source) to another (the destination) through a “channel” was instrumental in realizing today’s communication miracles. Shannon’s methods are not limited to this or that specific channel technology; rather, his work applies to virtually all forms of communication channels – from digital audio/video disks, to AM/FM broadcasting, to the technology of the Internet, itself. The wide applicability of Shannon’s insights to communication systems as diverse as Samuel Morse’s original telegraph system and modern satellite communications is quite remarkable and underlines the importance of his findings.

Claude_Elwood_Shannon_(1916-2001)[1]Interestingly, some of the foundation for Shannon’s ideas emanated from the early design of Morse’s first telegraph system which began service in 1844 between Washington and Baltimore. The first message sent over that line was Morse’s famous utterance in Morse code to his assistant, Alfred Vail: “What hath God wrought?” While Claude Shannon is fairly identified as the “father of communication theory” thanks to his famous 1948/49 publications, there were also many grandfathers! Most of them made valuable contributions to the speed and reliability of early communication vis-à-vis the telegraph and early telephony, as pioneered by Alexander Graham Bell. One of the early, key contributors to communication technology was R.V.L. Hartley who, in the July, 1928 issue of the Bell System Technical Journal, published a very original treatise titled Transmission of Information. This paper of Hartley’s and one in the 1924 Journal by Harry Nyquist were acknowledged by Shannon as prime foundational sources for his later ideas.

Hartley Bell Journal_2 1928 Journal w/ Hartley’s Paper: Transmission of Information

What Were Claude Shannon’s Contributions?

A brief but inclusive answer comes from the well-regarded book of J.R. Pierce, Symbols, Signals and Noise. I quote, here:

“The problem Shannon set for himself is somewhat different. Suppose we have a message source which produces messages of a given type, such as English text. Suppose we have a noisy communication channel of specified characteristics. How can we represent or encode messages from the message source by means of electrical signals so as to attain the fastest possible transmission over the noisy channel? Indeed, how fast can we transmit a given type of message over a given channel without error? In a rough and general way, this is the problem that Shannon set himself and solved.”

Although Shannon impressively refined our concepts regarding the statistical nature of communication, Samuel Morse and his assistant, Alfred Vail, had, long ago, recognized statistical ramifications, and that fact was reflected in their telegraph code. Notably, they made certain that the most commonly used letters of the alphabet had the simplest dot/dash implementations in the Morse code – to minimize the overall transmission time of messages. For example, the most commonly used letter “e” was assigned a short, single “dot” as its telegraphic representation. Reportedly, this “code optimization” task was handled by Vail who merely visited a local printing shop and examined the “type bins,” equating the frequency of use in print for a specific letter to the size of its type bin! The printing industry had a good handle on text statistics of the English language long before electrical technology arrived on the scene. The specific dot/dash coding of each letter for Morse’s code proceeded accordingly. From that practical and humble beginning, statistical communication theory reached full mathematical bloom in Shannon’s capable hands. As in Morse’s time, coding theory remains an important subset of modern digital communication theory.

Revisiting Communication Theory:
Grazing Once Again in Technical Pastures of the Past

The most satisfying portion of my engineering career came later – particularly the last ten years – when I became immersed in the fundamentals of communication theory while working in the computer disk drive industry, here in Silicon Valley. My job as electrical engineer was to reliably record and retrieve digital data using the thin, magnetic film deposited on spinning computer disks. As the data demands of personal computers rapidly increased in the decade of the 1990’s, the challenge of reliably “communicating” with the magnetic film and its increasingly dense magnetically recorded bits of data was akin to the DSL task of cramming today’s broadband data streams down the old, low-tech telephone twisted-pair wires which have been resident in phone cables for many decades. Twisted-pair wires make a very poor high speed communication cable compared to coaxial cable or the latest fiber-optic high-speed cable, but they had one huge advantage/motivation for DSL’s innovators: They already fed most every home and business in the country!

I retired from engineering in 2001 after a thirty-seven year career and now find myself wandering back to “technical pastures of the past.” During the last ten and most exciting years of my career, I came to know and work with two brilliant electrical engineering PhDs from Stanford University. They had been doctoral students there under Professor John Cioffi who is considered the “father of DSL.” The two were employed by our company to implement the latest communication technologies into disk storage by working closely with our product design teams. Accordingly, the fundamental communication theories that Shannon developed which enabled the DSL revolution were applied to our disk drive channels to increase data capacity/reliability. Under the technical leadership of the two Stanford PhDs, our design team produced the industry’s first, successful production disk drive utilizing the radically new technology. IBM had preceded our efforts somewhat with their “concept” disk drive, but it never achieved full-scale production. After the successful introduction of our product, the disk drive industry never looked back, and, soon, everyone else was on-board with the new design approach known as a “Partial Response/Maximum Likelihood” channel.

I always appreciated the strong similarities between the technology we implemented and that which made DSL possible, but I recently decided to learn more. I purchased a book, a tech-bible on DSL, co-authored in 1999 by Professor Cioffi. Thumbing through it, I recognize much of the engineering it contains. I have long felt privileged that I and our design team had the opportunity to work with the two young PhD engineers who studied with Cioffi and who knew communication theory inside-out. Along with their academic, theoretical brilliance, the two also possessed a rare, practical mindset toward hardware implementation which immensely helped us transfer theory into practice – in the form of a commercially successful, mass-produced computer product. Everyone on our company staff liked and deeply respected these two fellows.

When the junior of the two left our company as our drive design was nearing full production, he circulated a company-wide memo thanking the organization for his opportunity to work with us. He cited several of us engineers by name for special thanks, an act which really meant a lot to me…and, surely, to my colleagues – an uncommon courtesy, these days, and a class act in every sense of the word!

Even in this valley of pervasive engineering excellence, that particular experience was one of a select few during my career which allowed me a privileged glimpse into the rarified world of “top-minds” in engineering and mathematics – the best of the best. A still-higher category up the ladder of excellence and achievement is that of “monster minds” (like Einstein, Bohr, and Pauli) which the Nobel physicist, Richard Feynman, so humorously wrote about in his book, Surely You’re Joking Mr. Feynman. A very select club!

The recent event which tuned me in, once again, to this technology and my past recollections was the subject of my May 2, 2015 blog post, Two Books from 1948 : Foundations of the Internet and Today’s Computer Technology (click on the link). In it, I describe the incredible good fortune of stumbling upon one of the two scarce, foundational books on communication theory and computer control: Cybernetics by Norbert Wiener. More recently, I acquired a nice copy of Claude Shannon’s 1949 first edition, The Mathematical Theory of Communication (the other book). That one came at no give-away price like my copy of Cybernetics, but, given its importance, it still represents a bargain in my mind.

IMG_2479 PSLike many engineers who are familiar with Shannon and his influence, I had never read his book, although I had taken a course on statistical communication theory in my master’s degree program over 45 years ago. Unlike many engineers, today, whose gaze is fixed only upon the present and the future, I have a deep interest in the history of the profession and a healthy respect for its early practitioners and their foundational work. Accordingly, I have been brushing off some technical rust and am now immersed in Shannon’s book for the first time and in the subject material, once again.

Old, familiar pastures – a bit like coming home, again, to peacefully graze. While the overall “view” improves with age and experience, the “eyesight” is not so keen, anymore. But my curiosity is up, yet again, and I will soldier-on through the technical difficulties and see where that takes me, all the while relishing the journey and the landscape.

A New Workplace and a Changing Society; The “Two Faces” of Technology

Two Faces   002

Those of you who have been following my blog for some time may be aware of my background as a Silicon Valley (California) electrical engineer (retired). You may also realize from previous posts that I view technology as a dual-edged sword which can cut both ways – for good and for bad. On the “good” side of the ledger, are the obvious benefits, both real and potential, that technology brings to our lives….and to our spirits when judiciously used. The “bad” side will be determined by us individually and collectively as we merge technology ever more into our lives while struggling to reconcile it with our basic humanity. Our decisions on both the personal and the societal levels regarding technology’s role will affect us immensely. In fact, we have now reached a pivotal point in that process.

The “Middle-Class” As We Have Known It
 Is An Endangered Species!

Reading a book review in Thursday’s Wall Street Journal inspired this post, but only because the subject has long been on my mind. The book is titled Average Is Over and purportedly states that American society is headed toward a dual-class reality with an elite 10-15% at the top and the rest in an “underfunded future of lower economic expectations, shantytowns and an endless diet of beans” according to the reviewer. Those at the top will be the high-achievers, the self-motivated ones who largely embrace technology and can adapt to its demands. Those at the bottom will be everyone else! In essence, the middle-class as we have known it disappears.

It appears to me that we are headed in that direction – due largely to the influence of technology. Long-gone is the widespread need for unskilled labor as exemplified by a union worker in Detroit whose only responsibility was bolting-on automobile rear-view mirrors for $25 an hour and excellent benefits. Replacing legions of such workers, today, is a two-tiered, much smaller team of workers: robotics engineers who design robots to do such jobs and tech-skilled workers who can monitor and maintain the complex machinery of the robots themselves. The ultimate cost-savings in high-volume production lines such as those in the automotive industry is staggering – and quality improves as well.

Many former hourly factory workers find themselves unemployed today; their prospects for good wages and benefits in today’s society look bleak – unless they have the motivation, capability, and opportunity to go back to school and acquire the higher-tech skills that today’s workplace needs, indeed demands.

Henry Ford pioneered mass-production with his Model T Ford assembly lines beginning in 1908. The new methods he introduced were based initially more on common-sense logistics (time-motion studies) than on advanced manufacturing technologies, yet the die was cast and manufacturing along with the workplace has been changing ever since. Tom Friedman, the noted columnist, commentator, and author has written extensively on the evolving workplace and its effects on society. He would be the first to endorse the importance of technology in the workplace.

Despite technology’s rapid pace, it has taken a while for us to reach this point in time, the point at which technology’s promise of cheaper mass-produced goods of superior quality and functionality critically impacts the employment fabric of our society. Even though we have just now arrived at that critical juncture, technology has had an evolving influence on product manufacturing and the work force for some time. The semiconductor industry has relied on very sophisticated production automation almost from its inception some 50 years ago.

When I was designing computer disk-drives in the late 1990’s, here in Silicon Valley, California, I spent time at our factory in Japan. Even then, the automated assembly lines could turn out 10,000 of those technically-sophisticated memory storage devices a day! When you scanned the vast assembly-line area, you saw only an occasional human being visible over the humming conveyer lines. Aside from the economies of manufacturing such volumes, the argument is easily made that robotics was the only alternative to insure the uniform accuracy and quality required.



One humorous aspect of our factory in Japan was the way disk-drive assemblies were transported to other stations within the plant. As we walked through the factory isles, we had to be constantly aware of the little “robot-pallets” passing us by, busily moving themselves (except for computer control) from station to station within the plant by following a flat metallic strip on the floor. It was intriguing, almost comedic to watch a robot-pallet turn into a shipping area with a load of completed drives and begin to unload them onto the shipping racks for packaging – totally without direct human assistance! All of this took place over fifteen years ago; manufacturing technology has certainly not stood still since then!

Charlie Chaplin made a prophetic, satirical feature film in 1935 called Modern Times in which he acts and satirically spoofs the onset of the mechanical age of manufacturing and its effect on workers. The film has many funny and outlandish situations which bring to mind the robot pallets in Japan which I described. Who’s in control here!

The Reality Today

This is the reality of our industrial society today. I am led to believe that the middle-class in this country is an endangered species and that the impact of technology will continue to grow. The increasing flow of capital into the most affluent 15% of our population will increase unabated. I see this as very troubling. I cannot envision a happy and economically successful society in which 15% live an affluent lifestyle while the other 85% experience something well-below the heretofore accepted middle-class existence in this country. In addition to the great job opportunities open to the technically proficient in the society, there will be opportunities for talented craftsmen, accomplished musicians and artists, and others with significant niche skills to do well.

The vast numbers of hourly workers being put out of work by automation technology today along with future job-seekers who are shut out by the demands of a technical education will have a difficult road ahead. So will the rest who are not sufficiently motivated to retrain – to embrace technology, or to excel in the niche skills mentioned above. Gone forever are the formerly ubiquitous 8-5 time-clock, union factory jobs which paid well in wages and benefits.

A Caution: Retirement for Future Employees

Generous government, union, and private-sector pensions are now declining or non-existent. That is a problem middle-class hourly workers often did not need to face. Even talented, future tech-industry professionals who can look forward to jobs with good salaries face a three-tiered problem: Overseas outsourcing, technical obsolescence, and retirement. It is very difficult to keep pace with technology, and future engineering candidates would be well-advised to prepare to refresh their knowledge-base several times over during their careers. To remain gainfully employed with a good salary after age sixty in today’s engineering environment is, indeed, quite a feat – and there never were many company pensions for retired engineers.

Life will be harder for most, and I indeed worry about the society as a whole. Are we ready and capable to deal with the situation? Don’t count on Congress to be of much help; the outcome is going to be determined by choices made at the grass-roots level, person by person.

Author/historian David McCullough has noted that people have been claiming for decades that the country was going to hell, and the fact that it seems so now to some people is nothing new. However, I do think the country and its society will be quite different in the years to come due to the prevailing winds. Fasten your seat-belts.