People move on because they seem to assume that this is incompetence at work. Search is sooo 1.0, right? It’s been solved, and we’re just fumbling the execution, right? You usually get some sort of ironic joke along the lines of “wow, it is so easy to find stuff out there on the public Web, and here with all our resources, we can’t even do search right.”

And then the conversation tends to move on to more obviously “2.0″ things like blogs, wikis, how to increase participation, and my personal pet peeve: annoying moaning about “culture change.”

Hold on. Rewind. Let’s go back to search and think for a moment. I have a theory here, and I’d like to see if all you smart E 2.0 guys agree. I have reached a radical conclusion: broken search is the problem, but fixing search is not the solution. Search breaks behind the firewall for social, not technical reasons.

How Search Breaks Behind the Firewall

Let’s start with the blindingly obvious, and then draw some weird conclusions. Here are the most common reasons that come up:

1. Icebergs: Too much stuff that “ought to be shared” is on people’s desktops, or sitting in emails in attachment form.
2. Digital Landfills: There’s still tons of those random fileservers all over the place, where people simply dump files. They “ought” to put it into content management systems.
3. It ain’t a net: Web-based content is just easier to crawl. You have all those nice, friendly sociable things like robots.txt files that tell crawlers where to go and not go. Intranets are much ruder. It’s hard to take a proper inventory of everything that’s not on the searchable “grid” even it is nominally “online” in some form. Corporate IT “ought” to clamp down on undisciplined asset sprawl.
4. Those pesky formats: PDFs, PPTs, Excel spreadsheets, random proprietary databases with query interfaces.
5. Permissions: Lots of need-to-know systems with non single-sign-on protections. No way you can give your friendly Intranet spider ALL the skeleton keys, right? Corporate IT “ought” to enforce corporation-wide single-sign on.

Sounds like a purely technical problem doesn’t it? Get it all SOAn up (apologies for the terrible pun), create the holy grail “email attachment killing” technology (or alternately, figure out a way to make email “socially searchable”).

Why Search Breaks Behind the Firewall

Okay, I’ve been leading you to what I believe is the wrong solution. Everything above is merely a symptom. You could put the most talented engineers onto the problem, spend billions, and still end up with a mess.

You see the problem is not technical. It is social.

The key here is all those “ought” phrases.  Who’s recommending the “ought”? Who is being recommended as the “ought to do something” constituency? There’s no such thing as a person or organization without an agenda within a corporation. If the IT czar is a psycho with crazy ambitions, do you really want him/her to have all the skeleton keys? Is the person saying “ought” things about email attachments a librarian trying to make his/her job easier? Or somebody who feels cut out of certain conversations? Or somebody who is genuinely pointing out a non-political problem with sloppy sharing?

We often forget that the public Web is a very democratic place. Authority is democratically earned. PageRank is democratic. We link to stuff we like, authors we like. Google’s algorithms mine a vast democratic vote. Stuff on the public Web is online primarily because people WANT it online. They WANT it to bubble up. There are no big devious intentions. On the public Web, popularity equals power in very direct and simple ways.

Now step behind the firewall. What do you see?

1. Every single email ends up being an act of political judgment. To, cc and Bcc are three words about which I could write an entire book.
2. Need to know and organization charts/cascade patterns beat democratic content popularity and “Word of Mouth” information travel hands down. This surprises people: we often remark about how much work gets done at the watercooler. But this does not mean more gets done that way than through formal channels. The formal channels carry 80% of the communication. The reason we focus on the watercooler is that most of the politically sensitive stuff travels that way.
3. On the Web, people read people they like. Inside the firewall, everybody is constantly trying to figure out who’s important, who has the money, whose stock is up, whose stock is down. Whose coat-tails to ride. Which fool to suffer gladly for the moment, which battle to pick.  In other words there is very low correlation between information flows and friendship networks. You may pay most attention to people you hate or fear. Your best friend may be your lunch buddy from another department, and while you may be on each other’s blogrolls outside work, you have no connection inside and don’t vote up each other’s information contributions. Oneworkgroup may be trying to stay under the radar in skunkworks mode, while another may have a reason for wanting a dog-and-pony influence roadshow this year.

Forget blogs, forget wikis. Quite often intranet bloggers and wiki champions are the zero-influence noise signals (and I say this despite being one myself!). Most of the powerful people won’t blog. Let’s say a bright young intern writes a killer strategy brief/analysis of the company’s positioning as an internal blog, which gets a lot of comments and chatter.  Sure, an alert VP might spot it and hire the kid and groom him/her (okay I’ll admit it, that happened to me!). But look at what that does to search.  That page might well top the rankings by normal “democratic” search logic for the word “strategy,” but truth be told, the most important strategy documents are probably some low-distribution emails/meeting minutes, or some very carefully judged piece of officialese that comes out as a fait accompli, when it is useless to everybody else. Most people take one look at it, and yawn, but it is designed to send the right coded, and tactically timed signals about who has what piece of turf, to the few people who can interpret it. This document may get buried.

There are even rules. Like actual government, go-to-jail rules, that prevent the most important stuff from being shared at all beyond corporate officers, let alone discussed.

Some of the polyannish types might claim that this implies the organization is broken and that there is a need for more transparent, open, democratic governance.

I say, HELL no! This stuff may sound horrible, but large companies are not democratic for a reason. They exist to make money, not as laboratories for empowerment or social engineering.  The authority structures, communication flow patterns, emergent models of what’s hidden and how and why, versus what’s open, are all designed to work. Sure, you can find a lot of alternative models that are starting to work, but let’s not throw out the old without understanding how and why it works.

Which brings me to my main point. Search is political.

Search is Political

We often forget that Web 2.0 is built on top of Web 1.0. It assumes functioning search as part of the canvas.  Google and other search majors attempt strongly to maintain a “search neutrality” that matches “Net neutrality.” You may not like to hear this, but “search neutrality” is a really bad idea inside the firewall.

Inside the organization, the main form of authority and control is based on patterns of information sharing. We assume that function is accomplished by permission controls on content management systems and the like. Not true. Most information flow control is not based on restricting access, but limiting the very knowledge that a certain piece of information even exists.

And this is a good thing. If a lunatic in some toxic silo has evil designs on another department, then even the knowledge that a certain document exists may need to be hidden from him/her. There may be very good tactical reasons for burying some information in a complicated email thread, rather than being shared with clarity. Harsh but true: there are many good things that get done because someone was smart enough to cover their actions with nominal disclosure (“but it was in that email I sent 3 weeks ago, don’t act surprised.”)

Yes, such toxic politics can be really bad for an organization, but the solution is not to deny that it exists, but be smart about managing it.  Ben Horowitz has a brilliant recent post about what to do about minimizing politics from the CEO point of view, but it is really everybody’s job to be sophisticated about this stuff. And shoving a democratic search engine into your information ecosystem is a pretty naive thing to do.

A large organization is a contentious place. Alignment isn’t some idealized state of harmony to aspire to. It is a decision-by-decision evolving reality as power and influence shift through the organization chart. And again, this is a good thing. If things are going badly, you want different organizations to debate and express dissent, negotiate and if necessary, yes, hide information. You might even want them to work against each other for a bit of fairly-structured internal competition if you are not sure who has it right.

Public Web models of search blunder into this intricate and complex piece of machinery like clueless bulls in a china shop. This is the reason why they very rarely get much traction. It is in almost nobody’s interest to have powerful enterprise-wide search in place. This may sound perverse, but the ability to penetrate the opacity of organizations is to a certain extent the right test of whether you deserve access at all. If you can get to it, you are smart enough to use it wisely. It’s not a great system, but it’s better than naive search.

Instead of Search

I’ve been thinking hard about this problem, and I don’t think enterprise search can or should be “fixed.” The fundamental social and information flow assumptions of “search” need to be deconstructed and reconstructed for the enterprise.  Local/silo search within single sites/assets is fine. Enterprise-wide search in its naive form is a terrible idea.  And it is actually a good thing it is being done badly. Doing enterprise search well will provide a LOT more visibility to the most useless parts of the information ecosystem within an enterprise, and provide more incentives to hide the important information. Search infrastructure can provide a false sense of security, openness, transparency and “being informed.”

What can we do instead of search? I have some ideas, but they are still at a very early stage. But I do know that a smart discovery function is essential before the rest of the E 2.0 suite can transform organizations. The Web 1.0 agenda is still unfinished within the enterprise, so let’s be cautious about moving on to the E 2.0 agenda. Plumbing before furniture.

I am curious about how many people agree with my diagnosis or have alternative ideas/mental models here. Apologies if I’ve offended any huge advocates of enterprise search.

(Anyone know of any interesting forums/email lists about this issue? If not, I might be interested in starting one. Email me if you are interested, and I’ll start some sort of Google group or something if there’s enough interest).

Venkatesh Rao is an Entrepreneur-in-Residence at Xerox, and the manager of the Trailmeme project. He blogs at Ribbonfarm.com. These views are his own, and do not reflect those of his employer.

I thought the E 2.0 gang would appreciate a pointer to a new book by Rob Salkowitz, Young World Rising, where he examines the bottom-up revolution being created by young entrepreneurs in parts of the world with a young and growing working-age population. I have previously talked about Rob and his work on the interplay of demographics, generational effects and 2.0 technologies (in my SM vs. KM post, and in my review of Rob’s previous book, co-authored with Dan Rasmus). This is easily his best work so far.

I posted an interview with Rob on the Trailmeme blog, with links to the book. Rob shared some fascinating views on technology and demographics, and I have included a brief introduction to his work for those who are new to this important subject. Check it out.

Soldiers, Privateers, Mercenaries and Web Technology

One of the fun aspects of Web product development is that you get to think up usage scenarios, and interesting personas to go with them. A few months back, an old classmate, who now works for a big Wall Street firm, emailed asking if we had an enterprise version of trailmeme available. We don’t, and at that point we didn’t even have fully-thought-through ideas; we only had some poorly-defined conceptual vaporware for potential enterprise markets. But the email got me thinking, and a startling thought hit me: the old idea that you develop a Web 2.0 product for consumer/SMB users and harden/evolve it for enterprise users is changing very rapidly. This is because the enterprise itself is changing very rapidly, due to the emergence of three very different IT-user personas, who demand very different future enterprise IT strategies. The three user personas I’ve defined are soldier, privateer and mercenary. Depending on which one dominates the future of work, enterprise IT could evolve in three very different directions. Which means the enterprise design for products like trailmeme could evolve in radically different ways.

One of the more radical conclusions in the article is that perhaps there is no real need for a separate category called “Enterprise 2.0″ software/IT, given the way the workforce (and as a result, the enterprise) are changing, with blurring boundaries and increasing numbers of people employed in non-traditional models.

Read the full post at here

The Problem of Enterprise Document Integration

A typical large-scale Intranet is a cheerful mess of Websites running on different platforms, often from entirely different generations and technology paradigms. Within Xerox for instance, we have an ancient 1.0 level default Intranet (sorry any Xerox IT guys reading this, but you know it is true!), several repositories running on our homegrown Docushare CMS, some Sharepoint installs, and of course, a whole bunch of special-purpose local workgroup sites running on systems ranging from Confluence to Mediawiki. This is as it should be — a few slowly-evolving sensible enterprise defaults, complemented by some faster-moving local solutions for local problems. Besides this general content management infrastructure, like every large corporation, we naturally have all those specialized platforms for functions like elearning and HR.

This information environment though, doesn’t lend itself to easy organization and re-organization around changing and often transitory purposes. How, for example, would you pull together all the online resources needed to help employees from a newly acquired company orient themselves? Or give your employees an easy way to explore all the information related to a particular market?

The traditional 1.0 answer (which we know does not work) is “portal.” Portals are simply too hard to set up, too slow to adapt, and too centralized (I often rant about “Portalitis” — multiple silos each wanting to centralize information around themselves, leading to a proliferation of portals with a lot of redundancies). In reality, valuable information exists all over in a distributed way, and is constantly shifting.  Most of us are reduced to emailing bunches of cut-and-paste links and attachments to each other for every situation.

This is the problem of enterprise document integration, EDI: The problem of rapidly assembling and sharing an organized set of online resources to suit a given situation.

I don’t claim to have the complete, definitive answer, but I do think the idea of a “trail” is one potential solution. So here’s my quick soapbox pitch, before I get back to discussing the broader EDI problem. Keep in mind that what I am describing below is the simpler consumer version that we already have out there. The stuff isn’t yet ready to take on full-fledged EDI.

What is Xerox Trails?

Trails allow you to collect, organize and present (in short, “blaze”) collections of Web content using the user experience metaphor of a “trail” (as in hiking). You navigate a trail using a “Trail Map” view and a “Following” view. Show is easier than tell, so here is an example trail I created on our destination site, trailmeme.com, about the “Social Media vs. Knowledge Management” debate I triggered a year and a half ago, organizing the original article and some of the most interesting responses. Here is a screenshot of the “Trail Map” view of the trail (Click here to go to the live, clickable map: you’ll need to have Flash enabled in your browser).

And here is the “Following” view (think of it as a “ground-level/street-level” view). The idea is to support navigation with the sorts of signs you might see on a highway or a hiking trail at the ground level, by framing the original source with a useful navigation overlay. Click here to start “following” the trail.

This is the basic user experience. We also have a plugin that implements this experience on WordPress blogs (our project blog has this enabled), and we have a similar plugin for MediaWiki in the works. These allow you to create the “trails” user experience on your own site. Both are open source (GPL) and if you are interested in supporting trails on other platforms, talk to me.

And of course, since we are Xerox, there is a print and publishing angle to all of this. Trails (when appropriate) can be sequenced and converted to instant PDFs with one click (currently only supported on the WordPress and MediaWiki plugin version).

So that’s the technology itself.  Now let’s get back to the EDI question.

The Varieties of EDI Problems

Here is a quick half-dozen variants of the EDI problem that occur to me, that frequently appear in the enterprise. One of the big things I am interested in hearing from you guys is: are these the right EDI use cases? Are there better/more important ones?

1. A bid team for a major and complex enterprise sale collaborating to rapidly assemble existing resources into a raw material package to support the proposal writing
2. An HR/eLearning department creating an ad-hoc, just-in-time “course pack” out of stuff already scattered across the intranet, to support a course on the eLearning system
3. A workgroup involved in competitive analysis gradually putting together a “market view” picture by putting together internal material, analyst reports, press releases, competitor news and so forth, into a sort of constantly evolving online intelligence dossier
4. Creating an ad-hoc org chart out of wherever people choose to post their Intranet (or even public) profiles (even for companies that have a single social network/expert locator system, the ordinary model of organizing with tags and a couple of matrix dimensions reflecting the static organizational structure, rarely reflects the real org chart of ad-hoc teams, workgroups, dotted-line reporting and so forth)
5. Creating a weekly or monthly newsletter out of press coverage of your company
6. Creating a collection of information for a new “community of practice” type group, without creating yet another silo with redundant copies of information

Some of these problems, of course, can be solved with other technologies, including internal social bookmarking or collections of RSS feeds in a truly SOA-enabled Intranet. Where trails add value is where there is a need to add some meaningful sequencing (“read this before that”), so that the collection becomes a guided narrative of sorts. Some questions that my research team and I are interested in include:

1. What are the most critical EDI problems and pain points?
2. When the existing solutions to these problems work poorly, how do they fail?
3. Lost opportunities: what problems are being finessed or not faced at all, due to the difficulty of the problem?
4. How transient are information organization needs? How much organization lasts for days, versus months, versus quarters, versus years?
5. How dynamic are information collections, how often do they change during active and quiescent phases of the respective workflows?
6. What sort of security and permissions architecture constraints impact EDI?
7. What regulatory and “need to know” constraints impact EDI?

Invite Code and an Invitation to Dream

One of the smart things Xerox does is organize “dreaming sessions” between researchers and customers (who otherwise rarely meet).  That’s kinda what I am hoping to create here, virtually. If there is enough interest and I am able to scrounge together a budget, we might even do a face-to-face dreaming session at some point.

So here you go, here is how you can experiment with the technology in its current consumer incarnation:

• Go to the login/registration page of trailmeme
• Request an invitation to register with the coupon code: e2blog

You should see the invitation in your email in 10-15 minutes. If you don’t, check your spam folder. Once you register, you can create your own trails. The code will work for the first 100 users who sign up. We’ll release more batches of invites soon.

Keep in mind that this is NOT a commercially launched Xerox product. It is a labs project that is in research beta, so yes, some things (okay, several things) are kinda klutzy at the moment. But I hope you will be won over by the sheer coolness of it all.

There is a FAQ, and quick-and-dirty screencasts to help you follow/blaze trails. You can keep up with our updates  on the project blog at blog.trailmeme.com

So play around, create some trails, and feel free to post any comments/feature suggestions/high-level conceptual thoughts on EDI, anything you like. You can also email me directly at venkatesh.rao@xerox.com if you want to share private comments or just want to connect and talk about this.

If you are itching to use this inside the firewall, and you happen to run WordPress inside your org, you can of course try the plugin right away.

And I’d appreciate any reblogs. Feel free to share the code too.

Venkatesh Rao is a researcher in the Xerox Innovation Group. His personal blog is at ribbonfarm.com. While his posts on this blog usually come with the disclaimer that the opinions are not that of his employer, in this case, that is obviously not true.

McAfee and the Left Brain

McAfee’s book is dry, to the point and somewhat formulaic, but it gets the job done. That job is to serve as ammunition for E 2.0 evangelists, change agents and executive sponsors everywhere.  The book is a well-balanced mix of anecdotes and concepts, with a couple of new acronyms thrown in (ESSP: emergent social software platform). Astutely, McAfee starts with a personal story that gently encourages skeptics to identify with his own trajectory. He recounts how he was personally skeptical of the claims of the Web 2.0 movement until he decided to look up “Skinhead” on Wikipedia (the idea being that as a contentious flame-bait topic, it would stress Wikipedia’s claims of being able to produce level-headed articles on contentious subjects). Wikipedia passed the test, won over McAfee, and presumably, at this point, the book will sort of draw the skeptical reader into adopting a more open mindset.

From there, the book proceeds systematically to a chapter-length answer to the basic “what problems does this solve?” question asked by passive aggressive blockers and nay-sayers every evangelist must face, titled “vexations and missed opportunities in group work.”

We then get a brief history (too brief, as we’ll see) and a treatment of how E 2.0 does what earlier generations of Enterprise groupware and Knowledge Management (KM) failed to do (yay! for my early KM vs. SM thesis). The usual laundry list of design principle differences are trotted out, starting with the basic distinction between folksonomies versus top-down imposed taxonomies.

After what are essentially “neutralize the nay-sayers” ammunition chapters, we get a more positively-framed chapter listing the benefits. That concludes Part I. If you are using this as your tactical evangelism manual, by this time you should have convinced your quarry of the basic value proposition, and have him or her prepped for the deployment discussion.

Part II proceeds with a higher-level neutralization task: the roadblocks to deployment and operationalization. A particularly valuable section is “red herrings” — the huge list of idiotic objections that slow adoption (arising rarely from genuine ignorance, and usually from basic risk aversion, inertia, posturing by self-styled “critics” of flavors of the month, and intentional passive aggression). This bit should help newbie evangelists avoid particularly vicious and tortuous bunny trails and arguments that go nowhere, with people who don’t intend on moving, and who don’t need to be moved. Then we get a sketch of the the legitimate concerns, and a wise caution to prepare for the long haul. I was pleased to see a strong bit of advice to avoid the ROI question and focus on “are we moving?” as the core operational strategy question (the book does not address business model strategy at any length).

The book concludes with two chapters titled “Going Mainstream” and “Looking Ahead” which are roadmap fodder for the unfortunate middle managers everywhere tasked with creating those powerpoint slides for senior exec briefings. “Looking Ahead” has one nice bit about a “Model 1″ (goal-directed/waterfall) vs. “Model 2″ (open-ended/agile) approaches, and should help evangelists set the right expectations.

Overall, the book is a workman-like tactical manual aimed at driving change. That does make it somewhat tedious and boring to those of us who’ve already been down this road and learned the lessons the hard way, but it is critical to understand that this sort of book is essential for the army of second-generation evangelists out there (think Marines vs. Army), to make sure they are solving the right problems, and solving them right. A big part of this is knowing which battles to pick, and knowing who must be persuaded and who needs to be sidestepped.

If I have one criticism, it is that the book is almost too cynically level-headed and focused on being the ammunition needed by the frontline warriors. Even frontline folks have souls and need some inspiration and a sense of wonder at what we still don’t understand, to keep them going.  Rather than focusing on converting the resistance through sparking genuine “Aha!” moments, the book focuses on helping believers win, outmaneuver the resistance, and push through their agenda. Winning hearts, minds and souls isn’t something the book will help with.

The  book will also not help those who already have frontline experience in their search for a broader philosophical synthesis and intellectual stimulation. That book is the Fraser/Dutta one.

Fraser/Dutta and the Right Brain

Throwing Sheep in the Boardroom is a very different book. It eschews all talk of mundane practical issues except where such talk helps to make broader social-psychological or historical points. The book also does not suffer fools gladly, and is written at a high, nearly academic level. I suspect (and I am being pessimistic here) that the book will simply be beyond the comprehension of the majority of bright-eyed and clueless evangelists who just want to whine about (ugh) “culture change.” While their efforts may be propped up and rendered safe in the short term by good practical manuals like McAfee’s, to operate at higher levels of sophistication, understanding at this level is needed. Most will not achieve it. They will be doomed to energetic and misguided evangelism that will do more harm than good.

With that caveat, a look at the book. I have less to say here since it is a dense, closely-argued book that is anything but formulaic.

The overall narrative attempts to make a single broad point: Enterprise 2.0 tools up-end power structures in radical ways, and the higher-level social psychology questions that matter are all about the various tensions that play out in individual power struggles and authority battles.

To this end, the book situates the E 2.0 debate in a much broader context, drawing its inspiration from the power struggles between the “hierarchical” monarchies and “horizontal” Knights Templar in medieval European Christendom. There is a nuanced discussion of power, status, authority (ascriptive as well as real), and how these evolve when there is a struggle between hierarchical and network modes of organization. There are deep discussions of social identity, privacy, loyalty and friendship. Where McAfee and others take ideas like ratings at face value, Fraser and Dutta dive deep into the motivations driving rating behavior.

The book is very loosely structured at an overt level, but is very tight at the sentence level. There are three parts titled Identity, Status and Power, each with 5 chapters. There are no crutches for those with weak stomachs. No anecdote boxes, no graphs, no bullet lists of key concepts and talking points. Instead, each chapter is a single densely argued essay that dives into a few tricky themes. The only concession to “popular” accessibility is a single well-chosen cartoon at the start of each part. Don’t expect to easily translate this book to Powerpoint.

It is a keep-up-or-shut-up book that takes no prisoners. While not scholarly in the strict sense, there are plenty of references to the sort of original literature that most readers would never think of reading. Probably the best thing about the book is that it pointedly avoids annoying Kumbaya talk about how social software is all about world peace and Burning Man sensibilities. It is not, it is about the same old ornery self-interested, calculating individualists that power is always about. Again, I feel vindicated (see The Unsociable, Radically Individualist Soul of Social Media)

The overall effect of the book is that it delivers a brilliant and much-needed boost to the level of the discourse. This is the level at which we should be talking at today. This is the level at which fresh insights are still available. This is the level at which the conversation needs to operate if what McAfee calls the “long haul” challenge is to be met. Unfortunately, I think maybe 10% of the people who need to be talking at this level will actually be able to do so. I don’t mean to be elitist here. This has simply been my personal experience. Enthusiasm for a cause, and competence in supporting it are not the same thing.

Which Should You Read?

Ideally, you should read both. Use McAfee’s book to help you fight your battles. Even if you already know all that stuff and have gone beyond, the book is the ammunition you need. Put a little bit of your own soul into the battle though.

The Fraser/Dutta material is far more dangerous and subversive. If you really get what is being talked about, you should be cautious in selecting who you discuss the ideas with. The ideas openly frame the conflicts the right way, and only those who have been won over heart and soul will be prepared to engage you at this level.

If you can read and follow McAfee, but find that Fraser/Dutta leaves you confused and fumbling, you might want to gently step back from what you are doing before you do too much damage.

Venkatesh G. Rao writes a blog on business, innovation, and other stuff at www.ribbonfarm.com, and is a Web technology researcher at Xerox. The views expressed in this post are his personal ones and do not represent the views of his employer.

The New Adoption Threshold

My actions (which I suspect were pretty common for yesterday) show that the biggest barriers to adoption these days arise from two questions we ask:

a) Whether the marginal value of a service is greater than the startup-costs of registering and integrating it into your routine and…

b) Whether you think the thing will survive long enough to be worth learning.

Keep in mind that for a), supporting open registration models (OpenID, Facebook, Google, Twitter) is not enough: the startup costs of using a service go beyond merely creating and remembering new login info. You must additionally do something nobody can automate for you: integrate the new beast into your routine.

The Friendfeed acquisition and my response to it suggests that a lot of people will react like me: only believing in a site’s survivability (even if it is VERY good), if it integrates into a larger, more stable ecosystem. In a similar piece of news yesterday, the URL shortener tr.im announced that it was going out of business. Twitter, which accounts for most shortened URL use, picked bit.ly as their official shortener a while back, and after that it was only a matter of time before the mass extinction event began. Another victim of the era of standardization.

What both these incidents show is that “social networking in the wild” (SNITW) is finally starting to take off. You could interpret it equally as “domestication of the Web 2.0 wilderness.”

Acronym of the Day: SNITW

What is SNITW? It means your identity following you around wherever you go, as opposed to just inside a silo site like Facebook. Think of it as single-logon on steroids, and spanning your entire life as opposed to just your work life.

For example, if you go to popular techie blog ReadWriteWeb and try to post a comment, you will see options to post via Twitter and Facebook, 2 of the bigger “in the wild” identity suppliers. Your in-the-wild activity can also feed right back into your in-the-sandbox activity via services like Friendfeed which pick up and re-domesticate all your wild wanderings.

Other contenders: your Google ID (which is part of Google’s open social model), Gravatar by Automattic (the folks behind WordPress.com), OpenID (a distributed model where you can be your own identity provider, centralizing your life around your blog for instance) and Disqus (which centralizes your commenting life). It’s been a long journey. Remember Microsoft Passport?

So here’s what your social-virtual life will look like in the coming years:

• Source: You will establish 2-3 “base camp” identities via services like Facebook and Google (on average, we consumers are not very trusting, and the B2B world naturally resists monopolists, so it isn’t going to be just one)
• Process: Using these 2-3 base-camp identities, you will use services all over the Web (mostly social, non-financial). Many sites will support all major base-camp identity suppliers, but a few will have gaps, which is a technical reason you’ll need more than one base-camp. Sort of like those places that don’t accept American Express, so you will always need a Visa.
• Sink: Using re-integration services like FriendFeed, you will put your Humpty-Dumpty life together again. If you want to. If the source base-camps represent the “theory” of your life, the sink systems represent the logs of experimental data.

In this scheme, services which do NOT offer open registration models will need 2x-3x the value add to reach same penetration levels.

The interesting thing here is data-sharing. Apparently a random new service xyz.com will lose their most valuable resource: customer data. Not quite. The major base-camp identity providers will need to meet their own set of customer expectations (both business and 3rd party provider), namely involving open API access. Apple is learning this the hard way right now. So the balance of power among providers is shifting quite a bit, but not as radically as it might appear. Advertisers still need the high eyeball-surface-area provided by a large number of services, which gives these services enough collective bargaining power to rein in excess by the base-camp or sink providers.

Closing the Loop

Another interesting point is this: the Facebook acquisition signals a “close the loop” event, since a major “source” base-camp now owns a major “sink” re-integration service. But in general, I don’t think the 2 will converge neatly. Both consumers and 3rd party services will hesitate about putting all their eggs in one basket, and one of the most powerful ways to resist data monopolization is to keep source and sink ecosystems somewhat separate. Kinda like after Enron the SEC created mechanisms to ensure corporations, banks and auditors weren’t all in bed together. Another reason is technical: some kinds of stuff just don’t feed back nicely into the structure of something like Facebook. The last reason is social-psychological. Facebook is my “social resume” and LinkedIn is my “work resume.” There is no particularly good reason either should be the home of my “comment posting” persona.

So the re-integration ecosystem will organize itself around patterns of social activity, where the source base-camp providers will specialize in terms of social identity.

The New Order

What are the pluses/minuses of this?

• Pluses: life gets simpler for you and me, worst-case security thresholds start rising as the base-camp identity providers start imposing minimum security standards, anonymity starts to fade somewhat (if you’ve seen these SNITW sites, you’ll see things like people’s real names and pictures showing up automatically).
• Minuses: The whole system has far fewer points of failure. Successful security breaches can cause much more damage as a result, and will be harder to contain. Like the electric grid, the co-dependencies between major services can cause cascading failures (we saw a preview of that with the Facebook 2nd order consequences of the Twitter denial of service attack), anonymity starts to fade somewhat.

Some consequences that aren’t pluses or minuses for you and me directly, but still interesting. With all the focus on real-time search challenging Google, we seem to forget that the identity wars are creating an even bigger blindspot for Google than real-time: Facebook for instance, cannot be indexed well by Google (by design). So search is an example of an infrastructure service that might get balkanized as the source/sink identity providers leverage their capabilities into search. Tellingly, Facebook just revealed a big search play today, and Friendfeed is also known for having a great search component.

What about Financial Identity?

The big thing that’s missing in this picture is integration between unified social identities and unified financial identities. On the latter front, services like PayPal and Google Checkout and Amazon Payments are slowly creating a parallel payments ecosystem. The cutting edge is services like Mint.com which attempt to do to your financial life what Friendfeed does to your social life.

But these 2 universes don’t talk to each other. And perhaps they shouldn’t.

And Enterprise/Work Identity?

This is where there may be unpleasant news in store for Enterprise 2.0 vendors. As job-hopping and free-agent lifestyle trends increase, individuals will be increasingly reluctant to maintain a whole separate inside-the-firewall identity. I (and I imagine you) would rather fill out a special “private/corporate” section of my LinkedIn profile, were I to join a new company, than go sign-up for and populate a whole new internal system. In fact, the ability to centralize my work life around me, rather than my employer, could well become a major factor in decisions made by prospective employees between competing job offers.

The math is unforgiving. If a new employee expects to stay in a job for only 2 years, and the company, in the worst case, has a god-awful paper forms/triplicate signoff based system to get yourself into major databases for various purposes, and you estimate it might take 3 months of annoying paperwork at the beginning and end, that’s 6 months, or 25% of your work-time, spent with annoying bureaucratic process headaches bogging you down. I might choose a job with maybe a 20% pay cut rather than accept that I’ll spend that much time in hell.

The Holy Grail: Healthcare Identity

You may have noticed, but we’ve worked our way down the Maslow pyramid.  Google Health is the thin end of the wedge. If the current attempt at healthcare reform actually creates portable health insurance, the trifecta will be complete.

Once you’ve got all 3 into one broader architecture that is individual-centric, great powers and terrors could be unleashed. By controlling finely-graduated access to my 3-ring identity circus (outermost: social, second: financial, innermost: health), I could potentially eliminate 90% of the huge burden of paperwork “living” today entails. I expect that translates to about 15-20% in core, intrinsic productivity gains (not “economic” productivity as in GDP divided by population).

The flip side: I am my own single point of failure.

And no, let’s stop here. No global physical+digital passports and Facebook-managed systems of visas for international travel. That I think is not going to happen within our lifetimes. The irresistable force that is Web 2.0 will finally meet its immovable object when it runs up against national borders and the biggest “old” organization of all: the international system of nation-states.

Social Cash?

A friend made a point once in a conversation about the nature of money: the anonymous paper-currency economy, besides being a favorite of mobsters, serves as a check and balance against the tyranny of any information-based governance system. The more a tax regime is viewed as unfair, the more money moves out of the trackable economy and into the cash economy. Mess with money supply through over-free printing of money or lowering of interest rates below inflation, and wealth migrates further from cash to gold.

With all this identity unification fun, a similar check and balance is needed. We need social cash. True, anonymous, paper-currency-like cash. Cory Doctorow’s Whuffie will not do, since it is personally identifiable. There is a real need for somebody to invent a system whereby I can take some of my online social capital (earned through my blog’s rankings for instance) and converting it into an anonymous trust currency. Imagine being able to post a one word comment, “bullshit” as an anonymous comment on a blog and actually be listened to. Today such a comment would be ignored as noise (since it has neither the credibility of known identity, nor the credibility of intrinsic content). But if I could associate with such a comment 1 million reputation dollars say, that I’ve earned elsewhere, the comment would get taken seriously, the way anonymous donors to foundations are taken seriously.

Isolation, Interdependence, or Unification?

When you consider other major technologies that have gone through a Wild-West youth, followed by an era of standardization and integration (such as railroads, container shipping, credit cards, ACH-driven banking), you realize that there are two major tradeoffs that drive the dynamics. The first is the consumer’s trade-off between making life more simple and trusting any one infrastructure system too much. The second is the exact same dynamic at the B2B level. In various historical cases, we’ve seen outcomes ranging from pretty much complete unification (such as railroad gauges) to oligopolies (the top 2-3 credit card companies, metric and FPS systems) to continued anarchy (English isn’t going to be the universal language/human-brain-OS anytime soon).

For Web identities, I think the future is going to be “interdependence.” A small set of interlocked and co-dependent identity services, which are coupled enough to function well together (and take each other down in crises, but able to recover individually).

(p.s. I’ve had to withdraw from being a regular contributor here, thanks to relentlessly increasing demands at work, but a “Hi there!” to everybody I’ve had the pleasure to get to know through this site).

Venkatesh G. Rao writes a blog on business and innovation at www.ribbonfarm.com, and is a Web technology researcher at Xerox. The views expressed in this blog are his personal ones and do not represent the views of his employer.

Business at its Best and Boringest

Think of the most exciting parts of business. The charged atmosphere of a sales office “bullpen” in the morning, as salespeople make their morning calls before hitting the road. The boiler room excitement of a trading room. The bated-breath 3-2-1-ignition! pressure of a product launch. The blow-by-blow of a complex battle to steal a major customer from a competitor. Compared to how gut-real business can get at its best, the atmosphere of the virtual communities inhabiting Enterprise 2.0 infrastructure can feel pretty anemic (with the exception of IM and email, which I don’t count as “social media” anyway). From what I’ve heard from friends, this seems to be broadly true across all but the most aggressive and young small companies.

I am thinking here primarily of broad, enterprise-wide elements like a popular intranet blog or wiki, or an intra-company laconi.ca or yammer microblog. But even workgroup-level stuff, though closer to “mission critical,” rarely rises to the level of “exciting.” Staff organizations seem to do more with social media than line organizations. Innocuous fun stuff and watercooler banter goes ’2.0′ pretty quickly. Caucus groups post their event roundups.  Extra-curricular groups jump in to share picnic or softball game pictures. Armchair strategists, safely distant from the real strategy tables, post interesting news articles and uncontroversial comments about industry trends.

Official but low-impact/mostly harmless committees love the 2.0 medium too. You know what I mean: the ones chartered to do various non-critical-path things like “look into” a new long-term trend, champion green practices, organize a seminar series, or whatever.  The medium is a natural home for their activities. Don’t pretend you haven’t seen something like this:

“We are starting this blog to share thoughts and findings from our committee on blah blah blah, and provide a forum where community members can have their voices heard, share their thoughts and provide feedback. Readers can find a collection of resources at this repository. Let the fun begin!”

The signs are unmistakable: a slightly self-important, editorial “committee” voice, and a hopelessly retarded attempt to humanize and be “engaging.” Quite often, there are overtones of social justice, empowerment and other such themes.  Cheap shots aside, I’ve run or participated in a couple of such committee efforts myself. While they have their place and value, they are not where the action is. The evidence is pretty compelling: the part of the enterprise that is going “2.0″ fastest is the periphery, the part that views itself as disenfranchised, not the center. For every 2 blogs or wiki entries that display a wicked sense of humor or dare to raise truly consequential issues, there are 8 mostly-harmless dullsville pieces.

You could argue that this behavior is justified risk aversion. People don’t want to go on the record saying potentially unsafe, job-threatening stuff, or stuff that could be read as criticizing peers. People certainly don’t want to widely distribute sensitive or need-to-know information. Unfortunately that explanation doesn’t hold water. There is typically plenty of room between the reasonable boundary and the boundary at which people actually seem to stop. You don’t have to put together the communication around a million-dollar deal in a blog-fishbowl to make things interesting.

Why the Excitement is Missing

Here’s why I believe the exciting stuff is missing. The exciting people, by and large, are missing. One part of the reason is hard to fix. The exciting people — say the guy leading the consequential re-org, or managing the “bet the company” product launch, is probably far busier than everybody else. But I suspect there is another reason: to put it in terms of an American high school analogy, it is the same reason the “cool kids” avoid the “loser kids.”  Enterprise 2.0 is mostly populated by the equivalent of band geeks. The equivalent of football players and cheerleaders are possibly avoiding it. Just possibly, they might be thinking “nobody who is anybody goes there; nothing that matters happens there.” And it becomes a self-fulfilling prophecy. Enterprise 2.0 stuff  fails to deliver value, despite popularity and heavy activity, because the valuable stuff isn’t happening on it to begin with.

The world of business inherits its exciting elements from the market economy, the only truly Darwinian ecosystem (modulo bailouts) that modern information workers participate in. Those who participate in the risks, participate in the excitement. Those who want safety and security do their best to stay out of the line of fire.

It is critically important that Enterprise 2.0 tools get adopted by the risk takers and in-the-line-of-fire people actually driving the business. If we speculate that 20% of the employees are responsible for 80% of the results, we need that proportion reflected in online activity. The people who don’t pull their punches. The ones who dare to call a spade a spade. The ones who know how to tell the truth without unnecessary collateral damage. Without them, the revolution that Enterprise 2.0 thinking is capable of triggering will not happen.

Venkatesh G. Rao writes a blog on business and innovation at www.ribbonfarm.com, and is a Web technology researcher at Xerox. The views expressed in this blog are his personal ones and do not represent the views of his employer.

The Overview

Here’s the list of 14, with a self-rating of the level of provocation in each piece:

1. Breaking the Second Moore’s Law: Clouds and the Two Enterprise Cultures (Apr 10, 2009, 0 comments, provocation: low)
2. There is No Such Thing as Culture Change (Apr 1, 2009, 28 comments, provocation, high)
3. Diffusecasting: The New Model for Mass Influence (Mar 24, 2009, 5 comments, provocation: medium):
4. The Unsociable, Radically-Individualist Soul of Social Media (Feb 26, 2009, 13 comments, provocation: high):
5. Grown Up Digital, by Don Tapscott (Feb 6, 2009, 4 comments, provocation: low):
6. A Social Media Capability Maturity Model: Part II (Jan 28, 2009, 1 comment, provocation: very low)
7. A Social Media Capability Maturity Model: Part I (Jan 27, 2009, 0 comments, provocation: very low)
8. The Future According to Microsoft (Jan 21, 2009, 1 comment, provocation: very low)
9. The Last Page of Web 2.0 (Jan 14, 2009, 9 comments, provocation: low)
10. Will Your Enterprise 2.0 even make sense in 2009? (Dec 15, 2008, 3 comments, provocation: medium)
11. The ‘Hit by a Bus’ Social Media ROI Method (Dec 9, 2009, 9 comments, provocation: medium)
12. Has Seth Godin Peaked? (Nov 20, 2008, 7 comments, provocation: high)
13. Social Media vs. Knowledge Management: The Reactions (Oct 16, 7 comments, provocation: medium)
14. Social Media vs. Knowledge Management: A Generational War (Sept 28, 46 comments, provocation: very high)

You don’t need me to plot the correlation between level of provocation and level of reaction (comments, as well as what I haven’t computed: blog responses and tweets). If you provoke intelligently and skillfully, this pattern will emerge. And contrary to what you might think you may or may not always want a high-intensity response.

Manufacturing Dissent

When I first began writing online, back in 1999 (I wrote furiously for a couple of years, took a 5-year break, and returned to the online scene around 2006), I had pretty decent raw ability to create reactions, but very little control over the process. Back then, I was very active on Sulekha.com, both as a columnist (this was before the word ‘blog’ gained traction) and on the bulletin boards. My posts routinely provoked big reactions, and I gained some notoriety and even a couple of dedicated stalkers who liked to react pretty poisonously to anything I said. I knew that half of what I said would create debate and dissent. The problem was, I couldn’t predict which half.  I also couldn’t predict or control the quality of the debate.

Now, 10 years later, I have fairly good control over the rhetoric of online conversations (and much less interest in creating reactions for the sake of reactions). I can modulate my writing to achieve a desired quality/quantity of responses (within limits set by how well known I am, of course), with fairly high probability. On my own blog, ribbonfarm.com, where I try to post very enduring content on fundamental themes, I aim for low provocation. I don’t have the time to manage a very active discussion scene there. My intent there is to provide my audience food for thought; stuff they really have to think about. Here, I like to stir the pot and challenge the biggest pathology in the world of work: groupthink and assumed consenus. A pathology that leads to the formation and perpetuation of mutual-admiration cliques that do not talk to each other, thereby silencing public discourse around the most important divides. I think this is particularly useful for the Enterprise 2.0 space, which has the potential to change the entire philosophy of organizational management, not just its IT infrastructure. So what have I learned about manufacturing dissent?

1. Calibrate your range: depending on your fame, there is a baseline level of reaction you will get on social media no matter what you say or how stupid it is. Oprah will get 100x more retweets for her most inane thoughts than I will for my most useful tweet. That is your lower limit. Your upper limit depends on your skill, but will be limited at some point by your reach.
2. Separate your mood from your rhetorical intent (but not all the way): 10 years ago, the reason I couldn’t control the level of provocativeness in my writing or the probable intensity of the reaction was that I hadn’t yet developed the maturity to separate rhetorical intent from my own feelings.  It is one thing to spout vituperative nonsense because you are mad about something, and quite another to (for example) dramatically posit a “War” between social media and knowledge management because you think that will start an interesting debate. On the other hand, you can get too detached from the content of what you are saying, and get addicted to the sociopathic delights of baiting and button-pushing. Don’t. Stay detached enough to control, close enough to care.
3. Really understand your rhetorical intent: Self-awareness is key. Don’t delude yourself that you are trying to foster useful dialog when all you are doing is creating linkbait to drive up your page rank, or rationalizing a desire to vent or spew venom about someone/something. The absolute best intention behind provocation is personal learning. If you are the kind who is into provocation as part of an SEO (search-engine optimization) strategy, smart people will soon tune you out, and you will find yourself surrounded by idiots and other SEOers.
4. Dichotomize, dichotimize, dichotomize: Your two favorite colors should be black and white, and in any given provocation post, you should talk about both and pick one. Don’t insult your audience’s intelligence by offering philosophy 101 lessons on the futility and falsehood of all dichotomies. Not any dichotomy will do: picking the right dichotomy to poke at is like finding a social earthquake fault-line. The latent energy has to be out there, not in your head. Usually in the form of two non-talking cliques that are silently wooing fence-sitters on some issue. A newbie mistake is to assume that any old silly post with a title like “X vs. Y” or “A or B?” will work.
5. Be careful about adopting polarization as a goal: Dichotomies are necessary to create fertile framings for public discourses. But if your goal is to create a deeper chasm between two sides (polarization) and more cohesion on both sides, you are basically preparing for escalated conflict. There are some who believe this is never a legitimate goal for a public discourse, but I tend to be somewhat agnostic. I am willing to believe that there are sometimes irreconcilable positions, and that you have to help move the public discourse towards a winner-take-all conflict.
6. Qualifications are your cadmium rods: A friend of mine has a great rule of thumb for crisp writing: round off to the nearest simple assertion. The more you qualify bald statements, the more you will dampen the response. That’s not a bad thing. That’s how you exercise calibrated control, like the cadmium rods that prevent nuclear reactors from turning into nuclear bombs. Baseline high-clarity assertions, where 10 words articulate 80 % of your nuanced “internal truth” position, should be your starting point. Every additional 10 words worth of qualification will get you 80% of what remains. So 10 words will create a perception that is 80% true to what you believe, 20 words will get you to 96%, and so forth. You need to put in a minimal amount of qualification based on your estimation of the sophistication of your audience, and then add more to the extent that you want to dampen debate.
7. Target a level of provocation that matches your own level of uncertainty: Here is a paradoxical rule: the less sure you are of your position, the more provocative you should be. If you are unsure what you believe, a highly-qualified 96% true-to-you articulation will result in a turgid, gray, useless article that merely expresses your own confusions. An 80% true-to-you articulation will create room for others to challenge and support you. But here’s where newbie agent provocateurs make a mistake. If your level of confidence in your own position falls below a certain threshold (that you need to intuitively judge), you basically don’t know enough to play the catalytic agent provocateur role at all, and you should shut up. Pick topics where you have more to offer than the average listener, but aren’t 100% certain of your position either.
8. Decide what you want out of the conversation: It goes without saying that despite your best efforts at managing the ensuing conversations in your own comments sections, others’ blogs, and places like twitter, the overall public conversation may not reach closure at all, may converge to multiple different conclusions in different places (or even the same place), or solidly converge to a “you are WRONG!” outcome that you still disagree with. You need to be very very modest in setting your own goals. My goal is rarely persuasion, polarization, conversion or bringing a choir closer to me. My goal is nearly always to draw my own private conclusions from the conversation, move to a more refined position myself, and bank the learning credits. Only rarely do I share my revised position in a follow-up post because that rarely serves any purpose (though I usually publicly thank/acknowledge those I learned from). The public conversation you have caused will likely have too much momentum and locked-in equilibrium positions for you to strongly influence further in the same medium. Further influence will need a higher impact medium. To the extent that I have a public-spirited goal for others, I hope others take away some private learning like I do.

These are just the most important “advanced” principles. But a comment on the 101 stuff is in order. There are many basics that I haven’t talked about, like ignoring the jerks and knee-jerks, selectively engaging only those with the sophistication to understand that your stance is rhetorical rather than absolute, knowing when someone is on the other side of an unbridgeable divide and so forth. If you are still at that level of the game, you have only two choices. You could just give up on the challenge of developing your dissent skills and decide to become a purely harmony-driven blogger (and accepting a much lower ceiling in your ability to create reactions), OR you could fumble around and learn like I did. Hopefully it will take you less time than it did me.

Venkatesh G. Rao writes a blog on business and innovation at www.ribbonfarm.com, and is a Web technology researcher at Xerox. The views expressed in this blog are his personal ones and do not represent the views of his employer.

There are two basic business model architectures, complex systems and volume operations, and the two cannot and should not mix, or share best practices. Businesses with one architecture should not covet the benefits of the other.

The distinction is a nuanced one, but has almost biblical clarity (Moore actually uses the word “covet”). Think “high touch mega-deal business” vs. “mass production of widgets” for starters.  Like every good dogma, it catalyzes a lot of creative thought when you attempt to think of ways to break it. My question to the Enterprise 2.0 crowd is this: does cloud technology provide a way to break the Second Moore Law? I think it does, but it will take some extraordinary business creativity to do so.

The Two Enterprise Cultures

The Second Moore’s Law gets at a far more fundamental distinction in business than small/large or closed-and-hierarchical/open-and-flat. It gets at the very definition of a business, the purpose of which, if you recall your Papa Drucker, is “to create and keep a customer.”  Complex systems customers, like buyers of enterprise healthcare plans, or individuals buying homes, require a high-touch, high-customization approach. Volume operations customers, like individuals buying coffee, require a low-touch, a few-sizes-fit-all approach, where customization, if available at all, is in DIY form. Here are two pictures illustrating the difference, taken from Moore’s book website.

The complex systems model involves selling a solution which involves an integration platform over legacy infrastructure, followed by a technology architecture layer, a mix of your modules and third-party modules, a “solution architecture” that maps the bottom of the cake to the customer’s needs, and topped by an icing of consulting and integration services. You may instantly think “IBM” for the whole thing, and “Dell” or “Microsoft” for the modules, but think broadly, beyond IT, to businesses like Boeing aircraft, space launch services, or commercial real-estate construction.

The volume operations model on the other hand, involves starting with a base technology that spawns a portfolio of offerings which spread out through a ring of channel operations to a huge base of consumers, who generate cashflow as a river of small-ticket transactions. This is most familiar in retail, but be careful not to conflate volume/complex with consumer/enterprise or small/large (I’ll leave you to work out the nuances).

The book has a rich discussion of the two, but I’ll highlight just a few elements. Let’s return to Drucker for guidance in how to interpret the assertion that complex systems and volume operations business architectures do not mix. Two of Drucker’s ideas are pertinent. First, that there are only two really essential functions in business, marketing and innovation. That’s what you need to create and keep a customer. Second, maximizing profitability is not the goal of business (that, as we said, is to “create and keep a customer”). Profitability is a constraint. You have to make a profit primarily to manage risk and uncertainty in order to ensure survivability and meet the “keep customer” part of the definition of a business, which you cannot do if you aren’t around. So based on Drucker’s guidelines, here are the parts of the Second Moore’s law that are worth pointing out:

1. Marketing: The marketing function (which in the broad sense encompasses everything from positioning and competitive strategy to market research, not just selling) could not be more different. The high-touch/low-touch distinction is an obvious one, as is the difference in the length of the sales cycle and the revenue balance between initial buy and post-sale. But there are more subtle ones. Complex systems architecture companies must orchestrate a large coalition of companies to deliver a solution, so reputation and collective positioning matter more. Volume operations work primarily by pull rather than push, so branding and stand-alone segment positioning is key (rather than coalition positioning). Marketers who are good at one are rarely good at the other.
2. Innovation: Moore’s book is based on the technology adoption lifecycle and the category maturity cycle. He has a thorough discussion of 15 different types of innovation, from initial disruption to end-of-life innovations and harvesting. The big point, which he elaborates painstakingly, is that all  types play out very differently under the two architectures. For example, in the first kind, disruptive innovation, which comes at the birth of a category, complex businesses disrupt via technlogy innovations (example, Oracle in abstracting databases from mainframes), while volume businesses typically disrupt through business model innotavtions (for example, Kindle and free Wi-Fi downloads).
3. Profitability Constraint: Each business architecture also has a very different gross margin model (set by Wall Street expectations for that architecture). Roughly, “messy cost structure, large margins, low velocity” versus “lean cost structure, tight margins, high velocity.”

These three “business basics” distinctions provide powerful reasons why the two don’t mix. It goes right down to the personalities and strengths in your talent base. To make it explicit, Moore offers a “worked” example with a simple 7-stage linear value chain (the “skeleton” of a business), involving [Market] Research, Design, Sourcing, Making, Marketing, Selling and Servicing. Each value chain element, he shows, ends up getting structured differently. For example, the first step in the value chain, market research is primarily driven by qualitative scenarios in the complex systems architecture, because the customer base is small and diverse (example the airlines Boeing sells to), with no useful statistical structure as a “market.” Metaphors and story telling guide market research.  In volume operations by contrast, quantitative market research and analytics dominat (Netflix is the classic here).

So there you have it, that’s the two cultures for you. And Moore is pretty much resolute in his recommendation: the two cannot mix. The meet in the marketplace, and play a countercyclical dance, as complex offerings get increasingly tightly integrated, modularized and commoditized, thus moving from “complex” to “volume” regimes (example: mainframes to PCs), leaving the complex systems incumbent to look for new value at the next layer of the offering. But despite this partnership and meeting, they cannot play each other’s games.

Or maybe they can. Can the cloud break this law?

Clouds and the Two Cultures

Though Moore’s book has a brief discussion of Salesforce.com, and has an obscure discussion of how products can become platforms, it doesn’t really get to the idea of the cloud comprehensively (the book was published in 2005).

So is the idea of the cloud (whether it is Amazon bare-metal, Google’s App Engine level, or Microsoft Azure) a game changer at the level of business models? Certainly not in the general sense. Building Boeing jetliners will never be the same game as building Hyundai cars (automobiles are the most complex of the volume operations businesses, according to Moore) or Bics. I am an aerospace engineer, and I am pretty sure there is no currently feasible way to offer air-travel platforms that scale by the seat. You’d need something like a highly aerodynamic and fuel-efficient formation-flying set of auto-piloted 1-person aircraft to get to the aerospace equivalent of Amazon EC2. Since I studied formation flight in grad school, I am fairly sure that isn’t going to happen in the next 100 years.

But within the narrower world of IT systems, where the laws of physics are less constraining, can it happen, at least upon maturity in a few years, if not today?

I believe it can, and it can happen through the complex model becoming mostly unnecessary, and the bulk of revenue moving into the volume model. The complexity doesn’t go away, it just gets absorbed into market-based on-demand coordination models that are owned by nobody, not even the “integration” vendor, who can then become the “platform” vendor.

1. The Free-Premium Stack: Every module vendor  has grand visions of a stack where a free (possibly open source supported) base props up an SMB volume operations “premium” version, and a “super-premium” one for large enterprises. Unlike the integration challenges of 1.0 technology (like setting up special customizations to copies of Windows sold to a Fortune 100 company), there is a secret hope at work that SaaS, WebTops and XML-based SOA integration standards like RSS, integrating into a complex system will require no extra customer-specific work. Even if you are selling a 10,000 seat license. Holy grail, yes, but conceivable in a way perpetual motion machines are not.
2. The “Open Platform”: If module vendors are dreaming low-touch integration dreams, the platform vendors have similar hopes. With the evolution of open APIs and SOA as legitimate technology strategies, there is the hope that integration work will not be demanding on the platform end either. Perhaps it can be as simple as the CIO’s staff checking off a bunch of boxes for supported modules, and IT ecosystems simply culling out module species that don’t talk the main integration languages.
3. The “Sip” Sales Cycle vs. the Mega-Deal: Another big reason to believe the Second Moore Law can be broken is that it is now possible to engineer sales models where even very complex offerings can be sold with an entry-purchase in the volume regimes. Cloud-metered by-the-sip sales in fact. Dell sells to large enterprises by the truck-load. Amazon EC2 can sell by the box-hour. The goal is still millions of dollars from a single customer, but you don’t get that with a starter sale of hundreds of thousands. You can get started with one unit. Stowe Boyd just pointed out a great Niall Kennedy piece on the power of flexible pricing, and the implications for business model architectures are seriously significant. Google actually charges by the type of database access command.
4. Open Source as an Integration-Operations Game Changer: Moore has a quick treatment of open source, but I believe he fails to realize its true significance, which is this: much of the operational complexity of complex systems business architecture comes about because it is attacked top-down within a single umbrella sale or engagement. When your offerings are based on open source, much of this complexity is simply shunted out to the open marketplace, where it is solved by market/crowdsourced mechanisms rather than complex operations. If you think about it, the idea that complex systems integrations offer Quality of Service (QoS) guarantees within the SLA (Service Level Agreement) is a specious one. Well-organized open source communities beat formal QoS and SLA models hollow on specific fronts like bug fixing, security threat resolution and so forth. Slowly, open-source coordination models are taking on more and more of the integration challenges through their market-like models, and winning.
5. The Moving Target Effect: The customer isn’t standing still in all this. If “bazaar” models are taking over the open economy of free agents via open source, the insides of companies are also going from cathedral to bazaar architectures. This makes the complex-systems architecture slightly less necessary. Thomas Malone’s “Future of Work” (2004) chronicles many of these internal changes, and things are evolving so quickly, his book is is already outdated.
6. The Invisible Hand and the Single Wringable Neck: Perhaps the biggest reason complex systems architectures have been required in the past is that CEOs of Fortune 100 companies have demanded a “single wringable neck.” IT systems are now so large and complex, and so critical to operations, that if something bad happens on a large scale, the CEO needs to be able to call up a single powerful VP level account manager (or even the vendor CEO) and yell. What happens when there is no single wringable neck? Everybody expecting the mix of automated plumbing systems and market mechanisms inside and outside the firewall to resolve everything? This is perhaps the biggest unknown. In a sense, the single wringable neck is replaced by the invisible hand. As with Obama and the financial crisis, there are people to blame, but no single person who can be called and yelled at to get things fixed when a complex system breaks. My opinion is simple: it is evolution. We are now at a point of complexity in IT systems, where limiting things to levels where “single wringable necks” can exist is to limit business itself. Individual large businesses must be willing to operate as mini-economies rather than controlled systems.  You get the threat of meltdowns and messy recoveries, but you also get the promise of a brave new age of next-generation business model species. The dinosaurs are dying. The mammals are coming.

Venkatesh G. Rao writes a blog on business and innovation at www.ribbonfarm.com, and is a Web technology researcher at Xerox. The views expressed in this blog are his personal ones and do not represent the views of his employer.