From Silicon To Memes: Part 2

by Seth Noble |  Blog |  Aug 08, 2019

This is part two of a four-part series published on Forbes.com which explores how automated arithmetic machines have come to dominate human communication and the consequences for our culture and economy.

The Mechanisms Of Digital Communication

In the first part of this series, we looked at the history of computing and how, over the past eighty years, these devices have evolved from automated calculators into the backbone of human communication. In the midst of all this change, one component has remained surprisingly static. The transmission control protocol (TCP) is the software that performs the computation behind moving data for nearly all of your familiar applications and services. Yet its fundamental design has not changed in decades. In this part, we look at how electronic communications have evolved and the role TCP plays in our lives.

Analog electronic communications, such as telegraphs and telephones, consisted of a circuit of physical wires connecting devices. The circuit may pass through many junctions and repeaters, but an electrical signal at one end is propagated all the way to the other. The public switched telephone network (PSTN) allowed such connections to be created on demand, but the communication was still basically a wired circuit between two points.

Digital communications provide an abstraction away from the vagaries of electrical signal propagation, but early digital devices such as Teletype terminals and modems came to rely on the PSTN as a convenient way to establish a circuit between distant systems. This concept of circuit-switched networks carried over into technologies like asynchronous transfer mode (ATM), which was once touted as the future of digital communications. Computers were expected to call one another, exchange information and then release the circuit.

Packet-switched networks brought a very different approach. Messages between systems are broken into small packets of bytes. Much like a letter through the postal system, each packet carries its destination address and return address. This allows many packets to be sent to, or received from, many different destinations without having to establish a dedicated circuit each time.

Packet switching added tremendous flexibility to digital communications but also introduced several new problems. First, nearly all computer software still communicated point-to-point in streams of bytes, not small packets. Second, packet-switched networks such as ARPANET (and its descendant, the internet) are "best effort," meaning there is no guarantee any particular packet will make it all the way to its destination. Third, a device might know the speed of its immediate connection, but the capabilities of the network beyond are completely unknown and could change at any time. Send data too fast, and packets will simply disappear.

In 1974, Vinton Cerf and Robert Kahn came up with an elegant solution to meet the needs of the day. Their Transmission Control Protocol (TCP) (originally called the Transmission Control Program) solved all three problems by creating a virtual circuit on top of the packet-switched network. TCP automatically breaks up a stream of bytes into packets and reassembles those packets into the original order at the other side. It detects and repeats lost packets and uses that loss as a signal to temporarily slow down so as to avoid more loss. This software system of reliable, congestion-controlled byte pipes allowed distant software to communicate as if it had a virtual circuit, without having to worry about the details of packets.

Today, practically all digital communication networks are based on the Internet Protocol (IP) and nearly all applications use TCP. While some implementation details have changed, modern TCP still uses the exact same full-duplex, virtual-circuit model as it did forty-five years ago. All of those popular applications, all of the ideas they carry, and all the innovations they create still pass through this one technology.

While that is a great testament to TCP's design, a lot has changed in the last four decades. Almost every aspect of modern networking has not only grown by a factor of millions but has also become millions of times more varied. Speeds range from tens of kilobits to tens of gigabits per second, latency (the time for a packet to cross the network) ranges from microseconds to tens of seconds, and millions of simultaneous data flows compete for bandwidth over backbone paths.

In the face of such change, TCP has shown its age. Enterprise users, whose livelihood depends on moving large amounts of data quickly, see that data sent by TCP often fails to reach anywhere near their network hardware capacity. Consumers see slow loading web pages, hung applications and delayed messages. Buffering and dropped connections are a part of internet life. Ever faster hardware speeds have only made the problem more obvious: TCP was never intended to scale this far.

With nearly every aspect of our modern culture and economy dependent on digital communication, even seemingly small inefficiencies can have profound effects.

In the next part, we will examine how gaps in the functionality of decades-old software are costing businesses and consumers billions of dollars and begin to look at ways to bridge those gaps.