Columbia Professor Duncan J. Watts builds on the work of mathematicians, physicists, biologists, sociologist, economists and others to advance the new science of networks. "Six Degrees: The Science of a Connected Age " brings a sociologists' perspective to a field relevant to those dealing with complex systems and their robustness and fragility under stress. The science of networks has significance for those wrestling with current issues of law and public policy in a wide spectrum of applications including electric power grids, insurance markets and anti-terror measures.
Chapters one and two introduce the basics of the study of how individual behavior aggregates to collective behavior. In chapters three through five, Watts addresses the elements of "small world" networks, the role of scale-free networks and hubs, and the mechanics of search on the network. In chapters six through nine, he deals with epidemics, "the madness of crowds" and the dynamics of adaptation. He closes with views on learnings from recent crises, including the terror attacks of September 11, and with five "lessons for a connected age."
In addition to my notes below, see Watts' February piece for The Chronicle of Higher Education "Unraveling the Mysteries of the Connected Age. "
Watts, D.J. Six Degrees: The Science of a Connected Age (W.W. Norton & Co., 2002) (More ... )
Chapters One and Two
"The Connected Age."
Watts starts with a description of the 1996 power blackout in the western United States as an example of "cascading failure." In that case, the devices that protected individual elements of the system made the whole more likely to collapse. He discusses "emergence" of new phenomena when individual behavior aggregates at a level of groups, systems and populations.
The emergence of a new science.
He reviews the history of the science of random graphs developed by Erdos and Renyi, who found phase transitions at a critical point of increasing connectivity, resulting in the formation of giant components. But random graph theory lacked a way to deal with dynamics on the networks, particularly with how globally coherent activity emerges without central control.
When physicists entered the study of networks, they brought with them earlier studies of phase transitions that have the same behavior as those observed in evolving social networks. This revealed elements of universality in the emerging science. Universality is the quality that the behavior of physical or biological systems (e.g. chemical reactions or metabolic processes) can exhibit fundamental similarities with social systems (e.g. complex group behavior) and other types of apparently unrelated systems.
Watts says that "this tells us that at least some of the properties of extremely complicated systems can be understood without knowing anything about their detailed structure or governing rules. * * * This is a tremendously hopeful message for anyone interested in understanding the emergent behavior of complex social and economic systems like friendship networks, firms, financial markets, and even societies." Watts, Six Degrees, p. 65.
Chapters Three Through Five
Clusters and random short cuts make for small worlds.
"Small world" effects describe the ability of individuals to find connecting paths to strangers in far away locations in relatively few jumps. Studies found that the keys to finding short paths within networks are the existence of local clusters of related nodes or individuals (cliques) plus "short cuts" within the network. In mathematical models, only a few random short cuts caused the average path length to drop like a stone. Because the short cuts could be random to have this effect, it didn't matter how they were formed. This discovery enabled the transfer of discoveries and formulae from other sciences to the study of social networks.
Scale-free networks and hubs.
In 1999, Barabasi and Albert published a ground breaking paper "Emergence of Scaling in Random Networks." Science, 286, 509-512. (1999). This paper showed that certain connections in real world networks don't have a normal ("bell curve") distribution but rather follow a power law distribution. This means that there is an increased likelihood of extreme events in such "scale free" networks. As a result, in scale-free networks, as networks evolve, a few nodes will be "hubs" with an extraordinary number of connections. Barabasi and Albert also found that the evolution of these hubs depended on the combination of network growth and "preferential attachment" - the tendency for new nodes to connect to those already well connected (the "rich get richer" effect).
Watts applied the physicists' formulae (with some modifications) to social networks. He demonstrated that in social networks, hubs were not necessary in order for small world effects to appear. He found those effects in "networks of overlapping cliques, locked together via the co-membership of individuals in multiple groups. Because this feature is a property of the representation of the network, and not of any particular matching procedure, it is true regardless of how individuals and groups are matched. Even * * * random affiliation networks will always be small-world networks." Watts, p. 128.
Searching the network.
In times of crisis and rapid change, the ability to find short paths to the right information becomes particularly important. Network search can follow two modes, broadcast or directed. A broadcast search is impractical on large networks (unless you are a virus). Directed searches have their own issues. How to direct a search in a large network of which one knows only the local portion?
Mathematicians had demonstrated that to solve the problem of directed search, one need only forward the request to the individual node that seems closest to the desired information or destination, then let that node do the same until the search is successful. In a social network, people measure "closeness" to others on multiple dimensions (e.g. common geography, occupation, college attendance).
Watts's research found that if individuals were allowed to use 2 or 3 dimensions of social "closeness," they could easily find randomly chosen targets, even in networks characterized by close-knit clusters or cliques. All without using a hub. "Searchability is, therefore, a generic property of social networks,". Watts, p. 156.
Chapters Six Through Nine
Epidemics and Failures.
Epidemiology focuses on reducing the rate of growth of a disease. A disease can be thought of as doing a network search for an uninfected ("susceptible") host, and so epidemics can be analyzed using network science. If the disease can find a short cut, such as an infectious person flying from the isolated source to a highly connected hub city, or having intimate contact with many suspectibles, or both, an otherwise slow-to-spread disease can explode, as did AIDS.
Physicists contribute "percolation theory" to epidemiology. Various physical processes develop toward a change, then suddenly percolate from one state to another. The formation of crystals in a super-saturated solution is an example familiar to high school chemistry students. Percolation depends upon the development of a "percolating cluster" -- a single cluster of susceptibles that is connected with the entire population. When it is triggered, the entire system changes.
In 2000, physicists Barabasi and Albert looked at the "robustness" of scale-free networks during epidemics and similar challenges. They found that such networks are more resistant to random failures, because the small minority of hubs were unlikely to be hit. However, for the same reason, they were more vulnerable to attacks targeted at the highly connected hubs. "Attack and Error Tolerance of Complex Networks." Nature, 406, 378-382 (2000). Using percolation mathematics, Watts' team found similar effects in networks that were not scale free.
Decisions, Delusions and the Madness of Crowds.
In 1841, Mackay wrote his classic about panics and group manias, "Extraordinary Popular Delusions and the Madness of Crowds." Documented examples date to Roman times, and include the "Dot.com" bubble at the end of the 20th century. The real mystery of financial markets is that they are both rational and irrational at times. The evolution of cooperative behavior is similarly paradoxical. The "Diner's Dilemma" and the "Tragedy of the Commons" illustrate how outcomes unfortunate for the whole group can be the result of each participant acting rationally.
An "information cascade" is an event during which individuals spontaneously stop behaving like individuals and start to cooperate, acting like a coherent mass. Information cascades, such as financial panics, riots or revolutions, can be triggered by a small initial shock, then propagate throughout an entire networked system.
Watts looks at studies of "information externalities" (information issues outside of a transaction that affect the transaction decision, like other consumers' opinions), "market externalities" (value components derived from the presence of something else, like a fax machine), and "complementaries" (separate products that increase each others' value). The combination of market externalities and complementaries can generate the positive feedback effect of increasing returns. Watts also suggests the presence of "coordination externalities" (the likelihood that others will follow your action) affects the decision to go along with a budding information cascade.
Thresholds, Cascades and Predictability.
Transposing epidemiology to network science, Watts calls information cascades "social contagion." He contends that unlike biological contagion, social contagion is a process highly contingent upon coordination externalities. The greater the percentage of one's social "neighbors" making a choice, the greater the probability one will make the same choice. The probability jumps once the percentage hits a critical "threshold." "Highly connected" nodes with many neighbors tend to be "stable," because the tipping of any one or two neighbors will not reach their threshold percentage.
Individuals have different thresholds. "Early adopters" may tip to an innovation based on the influence of just one neighbor. As they tip, they influence their more stable neighbors who require the example of a larger percentage of neighbors. Early adopters with few neighbors are more likely to constitute the threshold percentage and trigger a cascade within their small clique than will those with many neighbors in a large clique. Under Watts' theory, highly-connected clusters (large cliques) tend to be stable rather than vulnerable, thus not the likely origin of a global cascade.
After a survey of the mathematics used to study global cascades, Watts proposes that global cascades can be started either by 1) lowering the average threshold of the population (e.g., by increasing the appeal of the innovation) or by 2) reducing the average density of the network (size of connected cliques). When both thresholds and density are high, the system tends to be stable, except in the small fraction of all nodes where a small clique includes an early adopter that could tip to the innovation.
Cascades, therefore, will tend to be rare, unless an innovation is directly targeted at one of those vulnerable clusters. If converted to the innovation, that vulnerable cluster exposes its neighboring stable nodes to the phenomenon of multiple neighbors converting, leading to a landslide of conversion to the innovation. In the context of marketing new technology, Geoffery Moore of Intel calls this phenomenon "crossing the chasm".
Watts takes from this the insight that the nature of the trigger matters less than the connectivity of the target. To prevent (or precipitate) a global cascade, the trick is "to focus not on the stimulus itself but on the structure of the network the stimulus hits." Watts, p. 249.
Innovation, Adaptation and Recovery.
Watts reviews the story of the Toyota-Aisin crisis and the "self-healing" response. On February 1, 1997, the only factory for the sole supplier of a brake component essential to Toyota's entire production system burned down with all of the specialized equipment needed for its production. Within 24 hours, all of Toyota's "just in time" manufacturing operation halted. The halt ended Toyota's demand for the other components supplied by some 200 companies whose businesses were dependent upon Toyota. Without central control or supervision, the 200 cooperated so as to find replacement equipment and re-establish production of the component within 3 days, averting a disaster for the entire Japanese industrial system.
Watts uses the Aisin crisis as a case study to sketch the history of industrial organizational theory from Adam Smith's The Wealth of Nations through Ronald Coase's The Nature of the Firm and The Second Industrial Divide by Michael Piore and Charles F. Sabel. The latter work identifies "flexible specialization" as the key to adaptation in an ambiguous environment of rapid change. Flexible specialization is a characteristic of the craft model of organization that was almost wiped out by the "economies of scale" methods introduced during the Industrial Revolution. Working together, Sabel and Watts proposed that when solving complex problems in ambiguous situations, individuals compensate for their individual limitations by using the network search methods Watts had been studying, to find those individuals with the knowledge or assets the searchers lack.
The hierarchical nature of the typical post-Industrial Revolution firm performs poorly at the task of flexible redistribution of information needed in a crisis. Yet the encouragement of local "multiscale" teams at multiple levels of the hierarchy dramatically improved the capacity for such search and redistribution. Because such "multiscale networks" not only minimized the likelihood of failures but also optimized recovery from failures, Watts calls them "ultra robust." He sees such crisis recovery skills developing naturally by network participants dealing with everyday ambiguity and change.
Chapter Ten : "The End of the Beginning."
The terror attacks of September 11, 2001 tested the robustness of the complex network that is Manhattan. Besides the physical destruction, public and private organizations suffered an unprecedented organizational crisis. The sudden loss of communications, transportation and information infrastructures was compounded by the loss of so many firefighters, police officers, business managers and staff. Entire firms were in peril, and some did not recover. Within 24 hours, a response self-organized and Manhattan's complex network rewired itself and marched on, though the global effects continue today.
As with the Toyota Aisin crisis, "the capability to recover from the catastrophe could not have been consciously designed. * * * So whatever it was about the system that enabled it to recover so rapidly had to have been there beforehand and had to have evolved principally for other purposes." Watts, p. 295.
Before providing a difficulty-rated guide for further reading, Watts closes with the admission that his book leaves unanswered questions, and the expectation that scientists will keep exploring the new science of networks until they get to the bottom of it ... and then keep going. Just like Manhattan.
DougSimpson.com/blog
Posted by dougsimpson at September 4, 2003 10:25 AM | TrackBack