Innovation: Together or Alone

It is very, very rare to find cases where somebody on their own, working alone, in a moment of sudden clarity has a great breakthrough that changes the world.
Steven Johnson

It seems that the determinants of success and creative achievement have become a matter of political debate. We have one presidential candidate who attributes it to individual initiative and one who says, “we succeed because we do things together," one who champions “Big Capital” and one who supports government and corporate collaboration.

In last Sunday’s Times (9/23/12) Steven Johnson, the author of Where Good Ideas Come From, writes about this issue by taking up the question of who created the Internet. In discussing this case, Johnson returns to his claim that, like so many other technologies,

“the Internet was created by—and continues to shaped by—centralized groups of scientists and programmers and hobbyists. (and more than a few entrepreneurs) freely sharing the fruits of their intellectual labor…”

I wrote about Johnson’s views here in discussing the culture of the coffee house and here in discussing Where Good Ideas Come From. I imagine he returns to these themes in his latest book, Future Perfect: The Case for Progress in a Networked Age.

In his Times article he claims that we live in a society that credits the development of the most important new products to corporations or government agencies. Johnson takes issue with this view:

“…the institutions responsible for the technology itself were neither governments nor private start-ups. They were much closer to the loose, collaborative organizations of academic research. They were networks of peers.”

They are not motivated by economic incentives, copyright protections, or centralized control of their work. Rather, they are primarily interested in sharing their ideas, and in collaborating with others in developing them further.

Johnson is concerned with “products” as he calls them, to a large extent, technological products. But surely there are other forms of creative achievement. What role do peer networks play in the arts, in writing a great novel or composing a memorable piece of music?

I doubt Tolstoy or Proust sat around with their buddies working on their novels or that Beethoven went down to the coffee house to try out his new compositions with his friends. Their “products” were creations of talented individuals, working alone, long hours, collaborating only with their thoughts and imagination.

Works of art are every bit an innovation as the technologies Johnson discusses. They constitute, at least, one important exception to his view. No doubt there are others that call into question the necessity of collaborative peer networks for creative achievements.

Where Do Good Ideas Come From?

How are innovations developed? Do they spring forth in the mind of a single individual? Or are they crafted over time in a dialogue with others? Is there some recurring pattern that can account for their formation? These are the types of questions Steven Johnson considers in his densely packed, heavily researched book, Where Good Ideas Come From: The Natural History Innovation.

In this book Johnson undertakes an analysis of 300 of the most influential innovations in areas ranging from the discovery of air conditioning, evolution, vacuum tubes, vacuum cleaners, the Internet and the World Wide Web.

It is an intellectual tour-de-force that is continually fascinating. Johnson argues that certain environments breed innovations effortlessly. Listing the major themes of this factually rich volume is, I think, the clearest way to illustrate his findings.

• Good ideas rarely spring de novo in the mind of a single person. It’s very, very rare to find cases where somebody on their own, working alone, in a moment of sudden clarity has a great breakthrough that changes the world.

• Innovations are built out of a collection of existing ideas and they are limited to those ideas that happen to be around at any given time. Ideas are works of bricolage; they are built out of that detritus [of old ideas]…in reality they’ve been cobbled together with spare parts that happened to be sitting in the garage.

• Innovations are often generated in open networks of ideas, individuals or settings that meld together seamlessly. …[a creative] space…sees information spillover as a feature, not a flaw. It is designed to leak. In this sense it shares some core values with the liquid networks of dense cities.

• Good ideas often come from hunches that develop over a longer period of time. …the idea (Darwin’s theory of natural selection) didn’t arrive in a flash; it drifted into his consciousness over time, in waves… The Web came into being as an archetypal slow hunch: from a child’s exploration of a hundred year old encyclopedia, to a freelancer’s idle side project designed to help him keep track of his colleagues, to a deliberate attempt to build a new information platform that could connect computers across the planet.

• Good ideas often occur by chance, say during a dream or while on a walk. The history of innovation is replete with stories of good ideas that occurred to people while they were out on a stroll. (A similar phenomenon occurs with long showers or soaks in a tub…)

• Innovations are often crafted from a tool or idea originally developed for a specific application but is then transformed or “gets hijacked” for a completely new one. Gutenberg’s printing press was a classic combinatorial innovation…Each of the key elements that made it such a transformative machine—the moveable type, the ink, the paper, and the press itself—had been developed separately well before Gutenberg printed his first bible.

• Good ideas often develop after a long series of false starts and errors. Being wrong forces you to explore…When we’re wrong, we have to challenge our assumptions, adopt new strategies.

• Finally good ideas are often formed in environments that encourage and promote exchanges between fields and methods of analysis.….[hotbeds of innovation] where different kinds of thoughts could productively collide and recombine.

In reviewing the 300 innovations (Each one is briefly described in the Appendix) that he studied, Johnson concludes that by far the majority developed in open environments, where there were no barriers (copyright protections), no economic incentives (market forces), and where ideas flowed freely in unregulated channels of communication.

For those eager to create the next big thing, Johnson concludes with this bit of advice: Go for a walk; cultivate hunches; write everything down, but keep your folders messy; embrace serendipity; make generative mistakes; take on multiple hobbies; frequent coffeehouses and other liquid networks; follow the links; let others build on your ideas; borrow, recycle, reinvent.

(He also penned some not-so-subtle advice that is clearly aimed my way: New ideas do not thrive on archipelagoes.)

Coffee House Culture

The other day Starbucks announced it will now offer its customers a digital network providing free e-books, movies, and access to some sites you normally pay for such as The Wall Street Journal, as well as other gratis sites including The New York Times, USA Today, Apple's iTunes, etc. Of course, they are only accessible as long as you remain at Starbucks--presumably paying for drinks, pastries, sandwiches, etc. Starbucks also gets a share of anything sold on its in-store network.

So at Starbucks coffee houses you’ll be peering at your iPhone, listening to your iPod or reading on your computer. This is not Paris after all where you can buy a cup of coffee and stay at your favorite table all day working away on your masterpiece or coming up with the next big thing with your friends.

Whatever happened to those days of the coffee house culture? Do you remember them? Sitting around for hours dreaming up new ideas, talking about the latest new book, or new discovery in the lab? According to Steven Johnson in Where Good Ideas Come From, a great many came during those coffee house conversations.

Johnson believes good ideas don't come from a lone genius working alone in a laboratory or at his desk. Instead, they frequently come from interactions between your colleagues and intellectual friends. He points to the coffee houses of Europe during the Age of Enlightenment. "People would hang out in this intellectual hub and have these free-floating conversations about all these different interests and passions.”

He claims Benjamin Franklin’s Club of Honest Whigs that used to meet at the London Coffeehouse when he was in England would hash over ideas that were instrumental in Franklin’s thinking. Johnson writes, "There should be a plaque to commemorate that coffeehouse. It was really a tremendously generative space.”

In Where Good Ideas Come From he describe these coffee houses as 'liquid networks” where people from entirely different perspectives gather together to thrash out whatever was on their minds. He cites several examples:

• Eighteenth century English coffeehouse where everything from the science of electricity, to the insurance industry, to democracy itself was discussed

• Freud’s salon “where physicians, philosophers, and scientists gathered to help shape the emerging field of psychoanalysis”

• Those legendary Paris cafes where writers, poets, artists and architects laid the foundations of contemporary culture

• The Homebrew Computer Club in the 1970s where Johnson claims “a ragtag assemblage of amateur hobbyists, teenagers, digital entrepreneurs, and academic scientists managed to spark the personal computer revolution…”

The coffee house model of innovation has a certain romantic appeal to it. And yet I can’t help but be skeptical of the claims Johnson attaches to it. On the surface, of course, one imagines these were and continue to be settings for lively intellectual exchange. However, we have no independent record of what in fact was said during these exchanges. Nor does Johnson cite any evidence made by participants of the role they played in formulating their ideas.

I am not suggesting they aren’t sources of innovation. But what came first—the idea or the conversation? And if the idea came first, how was it further developed at the coffee house? Or if the conversation came first, what is the evidence for this claim?

A network can support a person’s ideas, can be a setting where they can be expressed, and where perhaps they are refined in the process. And they can motivate someone to continue their line of thinking or their research program. But do we have a record of any of this?

Perhaps I’m being pedantic. But those are the kinds of questions I always ask, especially given the importance Johnson attributes to the coffee house model of creativity. While it has enormous appeal and while the experience of being with your friends in those settings is often stimulating, I wonder if any of the innovations Johnson discusses in his book, were, in fact, “born” there. To be sure, they may have been discussed and perhaps clarified, but that is a different matter than origination.

"Lyrical Connections"

I am reading Steven Johnson’s Where Good Ideas Come From: The Natural History of Innovation. In his chapter on The Slow Hunch he mentions the important role the commonplace book tradition played in the gradual “evolution” of Darwin’s ideas. This surprised and pleased me and while it didn’t exactly render me speechless, it reminded me once again of the unrecognized potential of a person’s commonplace book.

For example, Johnson speaks of the important memory-enhancing powers of maintaining a commonplace book. That is surely one of the principal ways I use it. In 1988 I started keeping an electronic record of notable quotes from the books and articles I had read. Any time I want to recall one of them, all I need to do is “open” my commonplace book document and run a Find search on the book or author in question. This yields a set of passages that I can draw upon in the same way one would use an encyclopedia, article or book review.

Johnson writes, “There is a distinct self-help quality to the early descriptions of commonplacing’s virtues: maintaining the books enabled one to ‘lay up a fund of knowledge, from which we may at all times select what is useful in several pursuits of life.’” This is the way commonplace books were used from Antiquity until the era of the Enlightenment when they reached their peak of popularity as a source of knowledge.

Johnson clearly believes a person’s commonplace book can be an important source of innovations. It stimulates “surprising new links of association” and is a central repository of a “vast miscellany of hunches.” For me the problem has always been how to capture these relationships, what method can be employed. You have to do more than simply review your entries from time to time. A tool for organizing, categorizing, or indexing them is required to locate those new associations.

Johnson reviews John Locke’s scheme. “The beauty of Locke’s scheme was that it provided just enough order to find snippets when you were looking for them, but at the same time it allowed the main body of the commonplace book to have its own unruly, unplanned meanderings.”

In practice, Locke’s method entails an ever more complex set of categories (“heads”) to use in assigning each saved passage. With more and more entries the scheme becomes entirely too complex and cumbersome to make much use of. You need a much more sophisticated, speedier system for generating relationships in a commonplace book that becomes increasingly larger as the reader’s collection grows.

Enter the electronic revolution. Johnson describes the way he uses a software program called DEVONthink where he stores all his work—essays, blogs, notes, book chapters, etc.

“DEVONthink features a clever algorithm that detects subtle semantic connections between distinct passages of text. These tools are smart enough to get around the classic search-engine failing of excessive specificity…Modern indexing software like DEVONthink’s learns associations between individual words by tracking the frequency with which words appear near each other. This can create almost lyrical connections between ideas.”

I’ve been searching for this type of software package for years. When I analyzed the 300 pages of Volume 1 of my Commonplace Book, it took me months of going through one page after another to identify its major themes. I lacked the skills to do much more than that.

Volume 2 is currently an equivalent number of pages and waiting to be analyzed. My hope is to somehow capture those “lyrical connections” that Johnson claims DEVONthink can generate. More importantly, can it generate new combinations of passages that are some sense novel or that might be considered original and as Johnson suggests may turn out to be genuinely innovative?

This is the way he does it. He plugs a passage or paragraph into the software and asks it to find other passages in his collection that are similar. Instantly, a set of quotes appears that, in turn, give rise to new ideas whereupon he plugs those into the software for a further search. “Before long a new idea takes shape in my head, built upon the trail of associations the machine has assembled for me.”

Can DEVONthink accomplish something like this with my Commonplace Book? Dream on, Richard.

Is Google a Commonplace Book?

Look closely at the photo. It’s a page (9th) of a Google search for “Search Engine.” Note at the top it shows 81-90 of 87,600,000 listings that were obtained by Google in lightning speed--0.07 seconds.

Is that page or any of those that precede or follow it a commonplace book? Steven Johnson in his lecture The Glass Box and The Commonplace Book says it is:

What I want to suggest to you is that, in some improbable way, this page is as much of an heir to the structure of a commonplace book as the most avant-garde textual collage.

To understand Johnson’s claim consider the various meanings of commonplace book. There seem to me to be three. The first refers to a set of quotations from a text that follow one another sequentially without any structure or organization. An example is the blog simply known as Commonplace Book that posts from time to time a referenced quotation without a comment or note.

The second meaning is the one I follow in my commonplace book where I list the passages I have made note of in each of the books or articles I read. After I have finished reading the work, I type in a Word document a heading with the name of the author followed by the title of the text. I follow this with each of the passages I’ve marked. My commonplace book consists now of well over 600 typed written, single spaced pages of such passages.

Thus, it isn’t organized in any systematic fashion, say by subject matter, theme, or category. That has to be done subsequently during a time-consuming analysis. While I’ve done that for the first 300 pages, the second remains a task for the future. I am hoping to find a method that will be simpler and more efficient than the one I employed initially.

I might have used the approach John Locke used as long ago as 1652 in A New Method of Making Commonplace Books that represents the third and most complete meaning of the term. Locke developed an elaborate system for indexing and categorizing the contents of his commonplace book by creating at the outset an index keyed to each letter of the alphabet that was, in turn, divided into five separate sections corresponding to one of the five vowels. He explained his procedure this way:

When I meet with any thing that I think fit to put into my Common-Place-Book, I first find a proper head. Suppose for example, that the Head be EPISTOLA, I look into the Index for the first Letter and the following Vowel which in this instance are E.I. If in the space marked E. there is any number, That directs me to the Page designed for words that begin with an E and whose first Vowel, after the initial Letter, is I. I must then write under the word EPISTOLA in that Page what I have to remark.

Locke did not begin with a pre-determined set of topics or “heads” as he referred to them; instead, they were created during the course of his readings. They included a broad array of themes followed by the passage he selected to fit the theme, and a comment of his own. This sounds like a commonplace book at his best. He not only copied passages but he classified, indexed, and annotated them at the same time. I know of no one who keeps a commonplace book in this manner today. It really isn’t that intricate, although it clearly takes a good deal more time than the two other methods. It also encourages the commonplacer to identify the reason the passage was selected in the first place.

Does a Google search employ this method? In citing Locke’s example, Johnson claims that is precisely what its algorithm does and it does so almost instantly. However, every commonplace book I know about has been created by a real person. Although the Google algorithm performs some of the same functions, insofar as I can tell, it is not such a creature.

On review, I should have included the “real person” component to the commonplace book concept. So while a Google search does provide a set of references (without form or structure), still it is not generated from the reading experience of an individual reader. And while it does provide a list of citations with a brief description (snippet) for the specific search phrase, it doesn’t index them or organize the sub-set in any particular way. Rather they are enumerated one after the other without regard to informative value, author or quality. If I really stick with a Google search for page after page, I am often startled to find the very document I want or the one that is most useful a good many pages beyond the initial one.

Johnson’s really fine lecture has stimulated me to think further about the commonplace book concept, as well a new way of viewing the results of a Google search. It seems to me such a results page or cluster of pages might be viewed as a second or third order variation of a commonplace book, something in between the first and second forms I have described.

It is more a remixing, recombination of ideas and references from a vast range of sources, as in a collage or cento, that are almost miraculously put together in not much more than a nanosecond. But a cento is not a commonplace book in the strict sense of the term, although it may be derived from one.

Johnson ends this portion of his lecture optimistically: “But all of this magic was predicated on one thing: that the words could be copied, re-arranged, put to surprising new uses in surprising new contexts. By stitching together passages written by multiple authors, without their explicit permission or consultation, some new awareness could take shape.”