Darwin Discovery Engine Blog

Predicting the Strength of Ties Through Social Media

Eric Gilbert and Karrie Karahalios at the University of Illinois at Urbana-Champaign recently published an interesting paper on Predicting Tie Strength With Social Media. Building on the theoretical work on strong and weak ties started by the classic Mark Granovetter 1973 paper “The Strength of Weak Ties,” they looked at whether social media behavior could be used as a predictor of tie strength. In this paper the researchers present a predictive model that maps social media data to tie strength. The model builds on a dataset of over 2,000 Facebook relationships and distinguishes between strong and weak ties with over 85% accuracy.  

They asked 35 people to rate a number of their Facebook friends that were randomly selected.  Then they looked at the Facebook behavior of these people to see if could accurate predict the strength of their ties as reported by the participants. Dimensions such as the use of words associated with intimacy and intensity, duration of communication, social distance, and other factors were looked at for their predictive power. The complete set of variables was combined to form the predictive model that worked in 85% of the instances. 

I am sure that the researchers would be the first to agree that more work needs to be done. As a former researcher I can see many potential confounding variable but, at the same time, the work looks promising.  They offer some practical applications. For example, a “system that prioritizes via tie strength, or allows users to tune parameters that incorporate tie strength, might provide more useful, timely and enjoyable activity streams.”

This is especially true as many people are building friends or followers that number in the hundreds, if not thousands. This would also apply within the enterprise as the use of social media within the firewall continues to grow.

 

Enterprise Search Summit 2010 Comes to NYC and KM World 2010 Moves to Washington DC

This year’s Enterprise Search Summit 2010 will be at the New York Hilton on May 11-12. The KM World Conference & Exhibition 2010 has moved to Washington, D.C. It will be November 16 - 18, 2010 at Marriott Renaissance,  It is nice to see these two conferences on the East Coast. I went last year in the November to the combined conference in San Jose.

Thierry Hubert and I will be presenting at the Enterprise Search Summit 2010. We are presenting on Deriving Order From Chaos Through Discovery and Awareness. Our session is on Wednesday May 12 at 1PM. The Enterprise Search Summit says that the “emphasis for Enterprise Search Summit is on how enterprise search enables Information Access. Search can no longer be viewed as a stand-alone application. It is increasingly part of everything we do and has become the de facto gateway to information in the enterprise.” I would certainly agree and we hope to see you there.

The KM World announcement said that this year's theme is KnowHow: the Knowledge-Driven Enterprise. “The knowledge of how to do something smoothly and efficiently with technical skill and expertise -- also known as "knowhow" -- drives every enterprise, agency, and organization. Creating knowledge bases, sharing them effectively, using them for decision-making and innovation.”

This sounds familiar but it is great that it is still very relevant. The session shave a 2.0 flavor like last year. I hope to be able to attend. Deadline for speaker submissions is March 1. Here are my notes from the combined event last year.

KM World and Enterprise Search Summit: Is Semantic Technology Real?

KM World and Enterprise Search Summit: Evolve From a Tactical E-Discovery Approach to Search and E-Discovery

KM World and Enterprise Search Summit: The Role of Social Techniques in Search & How It Impacts Your Organization

KM World and Enterprise Search Summit Sessions: From Birth to Billions: The Life Story of Google Enterprise Search

KM World and Enterprise Search Summit Sessions: Fundamentals of Enterprise Search

KM World and Enterprise Search Summit Sessions: Enterprise Search Technologies

Information Awareness and Discovery for Life Sciences and Pharmaceutical Companies

Success for lIfe sciences and pharmaceutical companies is often associated with their ability to manage knowledge transition and information sharing. This is especially true as as they deal with the challenges of many recent mergers in the industry.  The segregated systems where the departments’ collaborative efforts are stored represent a significant barrier to business continuation and leveraging knowledge.  As system integration analysis and process streamlining efforts struggle to unify these systems, the companies are losing precious time to better manage their R&D, clinical trials, and manufacturing, all the while  ensuring efficient CRO and CMO monitoring relationships.

Considering that these companies are knowledge intensive, it is critical for them to improve R&D productivity.  However, the increasing complexity of information systems and the increasing amount of semi-structured collaborative technologies increases the level of information overload, growing work flow complexity, and knowledge isolation through diverse communication and storage tools.

These companies’ organizational models are in the perpetual struggle of managing rigorous processes and needing to be flexible and organic in their structure. 

Mastering knowledge in a timely manner is key to their evolution and discovery processes.  This is why Darwin Ecosystem takes a different approach to accelerating information consumption by breaking the silos of knowledge to allow for meaningful discovery.  Our technology aggregates the information made accessible through simple syndication channels (RSS) and correlates it to deliver an awareness experience according to the user’s context of interest.  This  presentation, known as a Scan Cloud, immediately  displays what is happening across the organization’s departments by outlining correlated clusters and dimension classes that make it easy to grasp what is happening and direct attention  to the information of interest.

Darwin’s Awareness Engine is designed to evolve with the company and its users, and it can process new sources on demand, thus adding to your corpus of correlated information.  There  is no need for complex dashboards, strenuous process analysis and complex system integration.  All that is required is to make the content available to the service as an RSS feed.   Darwin’s Awareness Engine will correlate the content as it comes.  The experience can also be enhanced by customizing the taxonomy according to the company’s cultures and context. Darwin provides an awareness of what is d going both inside and outside the organization so that emerging themes do not get lost in silos.

Illustrations:

The following illustrates a correlation from mainstream media and blogs the morning of December 14, 2009.  The Scan Cloud™ show the connections between the emerging events associated with the Monsanto company.  A selection of on the Monsanto key word further filtered the sources and allowed for social media content to emerge as seen in the second screen capture.  The selected blog entry reveals a criticism of Monsanto. 

 

Webcom 09: La Peur Mal Placée des Médias Traditionnels

Ceci est une translation en français d’un blog Webcom 09: The Misplaced Fear of the Mainstream Media publié récemment afin d’atteindre les communautés de langue française et anglaise qui s'intéressent aux moteurs de découverte et de veille sur internet.

Un sujet central au récent événement Webcom 2009 fut celui de la crainte de voir les médias traditionnels perdre leur position de leader au profit des media sociaux.

La seule réalité que la consommation de l’information se déplace vers un dialogue sur l’internet oblige les médias traditionnels à entrer en concurrence avec ce qu’ils qualifient de contenus non vérifiés et par conséquent pas digne de confiance. En d’autres termes, on assiste à une nouvelle démonstration de leur difficulté à distinguer entre les contenus non vérifiés et la sagesse collective. J’ai souvent entendu de la part de ces médias traditionnels les commentaires suivants: “ Comment les gens peuvent-ils lire des commentaires aussi pauvres et invérifiables?” “ Notre mission est de fournir des contenus de qualité qui sont de loin plus solides.”

Les fournisseurs de contenus formels restent convaincus que sans leur production contrôlée, les gens auraient une vision déformée et s’appuieraient plus sur des opinions que sur des faits avérés. On peut comprendre leur point de vue. Si les gens acceptent l’information issue du media social comme une vérité validée, ils auraient raison….

Et pourtant, n’est-ce pas ce que nous faisions déjà? Le simple fait que les opinions et les discussions soient transposées de l’oral vers l’écrit ne change rien par rapport au risque de désinformation. Face à l’information, indépendamment de sa source, les gens modifient, fabriquent et adoptent leur propre point de vue en fonction de leur réseau social, qu’il soit physique ou via l’internet. La bataille n’est donc pas du formel contre l’informel, mais plutôt du comment les fournisseurs de contenu formel écoutent et participent au dialogue.

J’ai aussi été amusé en entendant les médias traditionnels et les grandes sociétés du media remettre en question la valeur du media social et de son contenu. Ces sociétés dépensent des fortunes en études marketing pour étudier les tendances, avec des analyses de type voo-doo, et qui regardent dans le rétroviseur l’impact des campagnes précédentes afin d’influencer la suivante. La chose amusante est qu’ils ont religieusement appliqué cette méthode pour mesurer les opinions et les appréciations des consommateurs pour mieux les attirer, et pourtant ils s’inquiètent du volume de contenu créé par ces mêmes personnes qu’ils essaient de comprendre et d’anticiper par croisement d’analyses. Quelqu’un peut il me pincer?

La perception qui doit changer n’est pas celle des gens, mais celle des sociétés des médias qui ont un comportement bien plus féodal car elles craignent de perdre leur pouvoir. Je pense que la réflexion de Thomas W. Malone en 2004 The Future of Work sur la transformation du modèle d’affaires est encore plus vraie aujourd’hui à la lumière du challenge auquel font face les médias traditionnels. Mais les médias traditionnels sont-ils capables de conduire de façon distribuée les délibérations et la création de contenu avec l’émergence d’une sagesse écrite des foules.

La possibilité d’avoir accès à des communications écrites en temps réel a changé les concepts de lecteurs et d’audience. C’est devenu un dialogue! Et de ce point de vue, l’industrie des médias est transformée à jamais. Les médias traditionnels doivent repenser leur relation avec leur clients (oui, clients puisqu’ils ne sont plus passifs lecteurs). Ils ont besoin de se recentrer sur leur coeur de métier et valeurs d’une information vérifiée ou validée, et d’atteindre un nouveau niveau qui préservera leur intégrité alors que leurs hypothèses sur les cycles de vie d’une information sont en train de changer. Certains comprendront que leur processus de gestion du business est leur ennemi et qu’ils doivent en créer de nouveaux.

Les sociétés des médias se perdent dans leur tentative de devenir elles aussi une destination du media social, plutôt que de se concentrer sur leurs forces et atouts. Elles prennent le risque de devenir des suiveurs de la révolution des médias au lieu d’essayer d’en être les leaders ou les champions des sources validées et digne de confiance. Il me semble qu’ils font une grave erreur en essayant de rattraper une tendance plutôt que de songer à innover et devenir les propres acteurs de leur transformation.

Je vous invite à lire le post de mon collègue Bill Ives qui a été publié récemment à propos de  What killed the Rocky Mountain News comme un exemple de média traditionnel qui n’a pas été capable de résister à la montée du média social. Dans cette session, l'ancien journaliste John Temple a dit que les journaux ont besoin de passer plus de temps à innover, et pas seulement essayer d’accroitre leur Chiffre d’Affaires avec des versions en ligne de ce qu’ils ont déjà fait ou publié.

Complete Listing of Our KM World 2009 Session Notes

We have been blogging some of the excellent KM World sessions. Here are links to the seven sessions we posted on last week. There is an emphasis on search and discovery as this most relates to Darwin Ecosystem. This was a great conference from our perspective. 

We hope these notes are useful and welcome your thoughts on the sessions.

KM World Sessions: Is Semantic Technology Real?

KM World Sessions: Evolve From a Tactical E-Discovery Approach to Search and E-Discovery

KM World Sessions: The Role of Social Techniques in Search & How It Impacts Your Organization

KM World Sessions: From Birth to Billions: The Life Story of Google Enterprise Search

KM World Sessions: Fundamentals of Enterprise Search

KM World Sessions: Enterprise Search Technologies

KM World Sessions: Resetting the Enterprise With 2.0 Collaborative Tools

KM World Sessions: Is Semantic Technology Real?

This is another in a series of notes from the 2009 KM World. It is titled: Is Semantic Technology Real? It is moderated by Rob Gonzalez, Platform Product Manager, Endeca Technologies and include: Michael J. Cataldo, CEO, Cambridge Semantics, Daniela Barbosa, Business Development Manager, Dow Jones Client Solutions, Dow Jones & Company, Lorenzo Thione, Founder / Principal Program Manager, Powerset / Bing Microsoft, Inc.  Here is the session description.

“Semantic technology is all the rage, sometimes even dubbed “Web 3.0.” However, many people—especially those making technology decisions for enterprises—wonder whether semantic technology has meaningful applications in the enterprise. Based on hands-on experience working with semantic tools, this panel of experts will establish the boundaries between reality and hype and help you understand what enterprises can gain from semantic technology in the here and now.”

Cambridge Semantics provides semantic development tools.  Dow Jones Client Solutions helps clients with taxonomy solutions, as well as other information needs. Powerset was acquired by Microsoft. It is Web semantic search tool.  Rob said, in answer to the panel title question, that semantic search has been applied successfully inside the enterprise and on the Web.

Rob asked the panel if there any examples of financially successful Web applications of semantic technologies. Lorenzo said yes. This happens more when you use semantic technologyto improve an existing site or product on such applications as consumer search. It seems harder for new sites such as Wolfram Alpha.

Daniela said in publishing there have been successful examples such as the Huffington Post and some Web advertising that use semantic web but do not promote it. The sites that promote that they are semantic Web are less successful.  Best Buy is using semantic web with their whole catalog but it is not promoted as a semantic web site. Google bought a semantic search company and it formed the foundation for Adsense and Adwords.

Semantic web appls let you combine different data sets like comparing the odds that you will be killed by sharks vs. vending machines.  It turns out that Vending machines are more deadly.  Semantic search helps people better define what they want in search.  The definition of semantic technology has been fluid.  Rob said it is everything from natural language document processing, and sentiment analysis to geotagging so it is metadata on top of data.  It is finding new meaning in data. Michael said semantic technologies are based on emerging standards on ontologies, distributed query, and other related things.  Semantics helps you find concepts even when there are not exact word matches. For example, you can find out how many sub-prime mortgages are in your loan portfolio even if they not are labeled that way. You define the characteristics and then find them. Here is a paper on semantic technologies and ontologies. 

The panel moved to semantics in the enterprise. There have been a number of successful examples where you can use a computer to connect data in two different places. How do you know that different things are similar? This is the answer that semantic technologies can address. It is about understanding concepts. FAST was given as an example. Consulting companies are starting practices around implementing semantic technologies.  At Dow Jones they are seeing companies interested in making better use of existing data through semantic technologies..

Daniela Is hoping that semantic technologies will get implemented like wikis, start small and spread.  This will be better than starting big.  Michael said that ontologies can grow over time. 

Some one asked what is new about semantic technologies? Daniela said the open vs. closed concept.  You can find better connections now. Companies do not know they are already part way there so some stuff is not new.  Now you can get your siloed data connecting and get better access to data outside the enterprise. Semantic technologies are trying to find structure in data where it does not explicitly exist in the same way a person might.

Merck is using semantic technologies to automate the integration of research results from three different relational databases and several spread sheets so they can be uploaded into a common database. Prior to this people had to cut and paste content by hand into a template for the upload.

Semantic technologies support discovery in contrast to simple search when you know what you are looking for. There remains a lot to do to improve the interfaces of semantic technology tools. 

KM World Sessions: Evolve From a Tactical E-Discovery Approach to Search and E-Discovery

This is another in a series of notes from the 2009 KM World. This is part of the Enterprise Search Summit.   It is titled: Evolve From a Tactical E-Discovery Approach to Search and E-Discovery by Brian W. Hill, Senior Analyst, Forrester Research, Here is the session description.

During the next 18 months, initiatives to cut costs and rising regulatory demands will drive enterprise e-discovery investments. Although search is a critical component of e-discovery, enterprises Forrester surveyed use a grab bag of search tools and techniques for litigation or regulatory inquiries. Lacking an end-to-end approach to gather and filter information, few enterprises report having a holistic approach to e-discovery. A full two-thirds of Forrester respondents consider their e-discovery strategy reactive rather than proactive. This indicates overall enterprise readiness for e-discovery is dismal. In this session, Hill will demonstrate the need for information and knowledge management (I&KM) professionals to collaborate with IT, legal, and business teams to understand how search technology can help cut e-discovery costs and mitigate risk. Formalized, cross-functional e-discovery programs that facilitate internal communication, encourage standardized methodologies, and provide a “big picture” view will help drive results. These programs can help support initiatives to synchronize e-discovery,
records management, and archiving.

Brian started by saying that eDiscovery solutions are not cheap but not using them can be much more expensive. Companies are grappling with regulatory issues and getting ready for possible litigation. Organizations have responded to this by rolling out a series of archives. The result is content and technology chaos that is difficult to search across and does not manage to mitigate legal risk. 

There is massive fragmentation within many enterprises. Security and metadata can be very different and not connected.

eDiscovery is a large market but it is also immature. May organizations are having challenges here. There are different stakeholders including legal and regulatory and large set of technology options. eDiscovery now invites fear uncertainty and doubt. At the same time organizations are trying to take out cost.  (see my posts: Equivio Releases Relevance™ to Enhance eDiscovery and Darwin Ecosystem Brings Awareness Engine™ to the Web and within the Enterprise)

There are different content types: email, collaboration systems, desktop files, etc. Email is the fastest growing content type.  However, the law says that all types of electronic content is subject to eDiscovery, including things like voice mail.  Only 20% of organizations surveyed felt their records were accessible and searchable.

There is a change point at the 5,000 employee level. There is more pain above 5,000 employees. Companies at this level feel more vulnerable and have much more complexity to deal with.  There is more to it than search. There needs to be proper legal hold capability within records management.   Brian looked at the records management vendors. Many people are using Sharepoint but it is not designed for this task. (see Forrester ECM wave report)

Less than half of stakeholders are satisfied with their records management.   Technology is not the answer. You need a well defined process that has proper stakeholder alignment. The market is fragmented and there are about 500 venders that address this need in some way.  But again the right process is the main need and it is perceived as the top challenge.  Defining defensible, repeatable process is key.

Records managers report all over the place in organizations. This is another example of fragmentation. There is gradual trend toward reporting form IT and legal.  IT and legal are key players here and Brian said he often finds that IT and legal have not met before.

There is a wide variety of maturity of this function within large organizations for those just starting to those very well developed. The company’s legal risk profile is a main driver of maturity, as well as how many times they have been sued.

One search approach is key words but you only find what you are looking for. You might not know the right key words.  I will add that this is a place where Darwin Ecosystem can help as it allows for discovery of relationships that you were not necessarily looking for. There are tools that use fuzzy logic, clustering, and other means to uncover key words that you did not think about.

The most commonly used technology to support search is operating system and desktop search. There are a number of shortcomings with this approach.  A very low percentage are using enterprise search.

In summary, it is much more than technology.  You need to define objectives, build queries, measure success, automate, review and analyze. You need a team of experts: legal, search, statistics, etc.   Make sure you have the right exec support. Get the right stakeholders. Develop a records retention strategy and policies. Create a consistent process.  Need to have integration between records management and search.  Should stay on top evolving case law.

Brian's slides are available at  http://www.forrester.com/esswest09 

KM World Sessions: The Role of Social Techniques in Search & How It Impacts Your Organization

This is another in a series of notes from the 2009 KM World. These notes are done real time so please excuse typos.  It is titled: The Role of Social Techniques in Search & How It Impacts Your Organization by Charlene Li,  Partner, Altimeter Group.  Here is the session description.

“Social technologies are transforming the way that people use the web and, with it, the way that companies engage with their customers and employees. Search is certainly being affected by the increasingly social nature of online activities. Impacting the socialization of search are the factoring in of the social graph and social activities into search results. Also, online people increasingly turn to their social networks when seeking information, recognizing that these people are likely to lead them to results. Li, a former Forrester analyst, provides insights into how social technologies are transforming the way people search for and discover information and how you can prepare your organization for—and create business advantage with—this shift.”

Charlene is now writing a book on how social technologies are impacting organizations and making them more open.  She started her talk with search.  Googlesearch results now provide a lot of personal context.  Then she showed Twitter results on Bing. It shows the most popular links about a topic. She said search will change to provide more social content both inside the enterprise and on the Web.

There are three ways search will improve. First, semantic search will better understand the meaning behind your search query.  For example, people will use the words Saints differently. Some people will think of a football team. I would. It will also be more personalized and more social.

Your social content will be integrated everywhere. You will not have to go to a site like Facebook but it will be found in many contexts.  Now there is a greater culture of sharing.  The engagement pyramid starts with watchers, then sharers, then commentors, then producers, and finally at the top are curators.  The curators mange the content produced by the pyramid.  She said most organizations support the top of the pyramid but the focus should be at the bottom so the watchers are well served.  These people are the foundation.

Facebook Connect puts Facebook content on other sites. For example, you can comment on Huffington Post and it will appear on my Facebook and my friends can see it.  So you can bring your friends into Huffington Post and bring Huffington Post into Facebook. This openness is the future.

Then she showed Tech Crunch with Google Sidewiki. You can see the side conversations around this site. Tech Crunch cannot control this conversation but now you can see it easily.

Next, she showed Amazon with GetGlue that allows you to see comments on a book from multiple sites.  You can see what people are saying and where they are saying it.  The comments are being separated from the site and pivot around the topic or the person.

With these tools you can target marketing much more. You can use social graphics: who is like you who might have the same interests. Media6 identifies who is closest to you. Then they can target those people with ads based on what you do and like.  You can also map relationships within organizations to target business communication.

Be careful not to get too wrapped up in the technologies. Start with what types of relationships you want with your employees and your customers.  Loyal and constant relationships are more valuable.

Obama redefined political campaigns.  The relationship with the candidate was more personal and got more engagement with supporters. This type of connection will be very powerful for businesses.

You also need to create learning organizations. There are many tools for seeing what is happening on the Web. Good to get these tools in the hands of everyone so they can see what is happening inside and outside the organization. This is a role that Darwin Ecosystem serves as you can see the relationships between topics on a near real time basis.

There also needs to be a culture of sharing. Internal micro-blogging tools such as Yammer can promote this sharing.  You need to create new information workflows to support this sharing. Everyone is getting the information that used to be department focused and siloed like marketing.  Comcast is using Twitter to respond to customer issues very quickly and the whole organization has become more customer focused. I have conversed with Comcast through Twitter and they are very responsive.

Kohl’s has a very interactive Facebook page and responds quickly to comments put on the wall. Saleforce.com is integrating a Twitter-like comment feature so you can talk about sales issues.

Openness requires accountability. The Red Cross gave guidelines for its employees on how to use social media and avoid risks. Finally you need to be open to fail and but you need to learn from these failures – Google says fail fast and fail smart.  Walmart failed many times at first with soclal media through fake blogs and other stuff. Finally they started a blog for their buyers that worked. 

In summary, Charlene said you cannot ignore social networks as they will be everywhere and integrated into many things. You need to be prepared to give up control to succeed.

 

KM World Sessions: From Birth to Billions: The Life Story of Google Enterprise Search

This is another in a series of notes from the 2009 KM World. This session is titled, “From Birth to Billions: The Life Story of Google Enterprise Search” by Cyrus Mistry, Enterprise Product Manager – Google. Here is the session description.

“This talk examines the state of Google’s Enterprise Search business, highlighting the origins of the iconic yellow box and the way the product and business search market has evolved over the past decade. Mistry highlights the challenges facing enterprises and providers of business search technology, as well as possible solutions and case studies of companies and organizations leading the way in search innovation.”

Cyrus began by describing the common problem of having too much information at work. The average worker creates over a gigabyte of information at work and much of this is in email. So Google were asked if they could bring Google into the enterprise.  At first they resisted but there was too much demand so they started Google enterprise search.

They found people wanted three things. First, people want simplicity.  The average number of words put into a Google search field is shrinking but people have the same expectations for good results.  They also want speed of response.  They did studies and found that the faster the results the more relevant people felt the results were. The third thing people want is relevance.

But things are different in the enterprise, Page rank is not relevant as there is not the same linking as on the Web.  In 2001 they launched Google for the enterprise in a small yellow pizza size box at a small price point that opened it up to more businesses. Cyrus said it also help lower the price of many other search tools. In 2004 they added more security. 

People felt Google was not connected so they did Open Source and established the connector framework.  Next after the basics were done they looked at the search experience.  They wanted to bring all types of results (text, images, etc.)  into a single screen.  This provided intelligent simplicity.  Next they looked at scalability so you can reach all documents. With Google you can add more boxes to get scale even up to a billion documents.

Now they are looking at social signals. Cyrus said this is where the innovation will be found.  The feature of user added results has been the most popular capability. People get really excited about this.  Another popular social feature is query suggestions to see related things that others are searching for.  They found a big jump is usage after this feature was adopted. They also look at where people click to see if a result should be elevated.  They use a complex algorithm to overcome any bias of current rank of this elevation.

 

 

KM World Sessions: Resetting the Enterprise With 2.0 Collaborative Tools

This is another in a series of notes from the 2009 KM World. These are near real time notes so please excuse typos and spacing issues. This is the Opening Keynote: Resetting the Enterprise With 2.0 Collaborative Tools by Andrew McAfee. Here is part of the session description.

“Andrew McAfee focuses on how emergent social software platforms are benefiting enterprises, and how smart organizations and their leaders are making effective use of them to share knowledge, inspire innovation, and enable decision making. He shares strategies, stories, and real-world examples of successful enterprise collaboration using 2.0 tools.”

Andy began with his definition of enterprise 2.0, It is “the use of emergent social software platforms by organizations in pursuit of their goals.”  Here is what I wrote recently. (see What is Your Definition of Enterprise 2.0? Here is Mine.) I think they are very similar. Andy said that now technology is not being used to tell people what to do but instead it is being implemented to let people decide what to do about it. I really like this. He then asked those who find their intranet easier to find stuff than the Web. Five people raised their hands. This is the first time that I have seen anyone raise their hand as I have been asking audiences the same question with attribution after I first heard Andy ask it several years ago.

Andy told a story about the recent time when he was in a rental car that did not work. So he tweeted and asked for help and got 16 responses in a few minutes, most from people he did not know. People do want to help each other. I found this out last weekend when I had a Twitter spam attack and got lots of help to fix it. So he said let’s stop worrying about the risk of social media within the enterprise. People are usually helpful. He finds very few horror stories inside the enterprise.

One difference from the Web is that your identity is traceable within the enterprise so bad behavior will be found out. I would add that you are also working with colleagues who hopefully share a common goal. In fact, I think that enterprise 2.0 can help people be more aligned around common goals. Andy said that enterprise 2.0 has improved his view of humanity.

He then said to avoid the idea that there is one way to do things. Innovation is more the issue now than strategy. There are now chief innovation officers. Crowdsourcing is becoming more common. Companies are now sending problems out on the Web for others outside the enterprise to help them solve the issue. Merck is one example.  The diversity of people looking at an issue increases the rate of solving a problem.

The key is building communities than people want to join and participate.  Verizon has opened up a portion of their web site and let others help them solve customer issues.  Some people are spending a lot of their time for free helping Verizon with customer service.  These people do get status and recognition.  

Andy said he used to be against the wisdom of crowds. He thought crowds would get dumber when they got together. Now he has completely changed his mind.  He gave an example where a group prediction market beat an expert and a synthesis of the polls on the electoral college vote spread in the 2008 US presidential election.  The lesson: crowds can be very wise. You should enable peer review and experiment with collective intelligence.

The great benefit of using Enterprise 2.0 is not sharing documents for collaborating. This is helpful but it is not the biggest benefit. Instead it is connecting the dots on issues. He gave the example of the wiki set up by the US intelligence community: Intellipedia. Because of this wiki people now know better who is doing relevant work on topics of interest.  They are discovering useful people on their topics of interest.  There is less parallel play and re-inventing the wheel.

He gave some major benefits from a recent McKinsey study that found increased customer satisfaction, increased innovation, access to knowledge, access to internal experts, and employee satisfaction.  While these results are likely from true believers, the magnitude is impressive.  Adopting enterprise 2.0 seems to be a necessary move.  Now the internal processes of the organization can be supported through technology.

He concluded with some pointers for success.  He twisted this by talking about how to fail.  Declare war on traditional management and technology, Bad idea for two reasons. It is bad marketing and it is wrong.  Allow walled gardens to flourish. Do not have disconnected content and teams. Accentuate the negative.  Do not worry too much about the risks.

Another thing to avoid: Try to replace email.  It will be hard and email does have its uses, especially as a single source.  We tend to overweight the advantages of the technology we are using.  A replacement technology needs to be ten times better than what you are using.  Email is the current technology and is not that bad.  Over night success occurs when there is no existing technology on the issue, examples are Facebook and Twitter.

Another thing to avoid: put into many features. Keep it simple like an iPod, Google, etc.  Technologists love features but users respond to simplicity even it then think they want more features.

Another thing to avoid: Overuse the word “social.” This is not top of mind for senior execs. While I would agree with this I would also not go with enterprise 2.0 with execs. It seems that technology labels are difficult in general. Just promote what it does.

 

 

 

 

 

KM World Sessions: Enterprise Search Technologies

This is another in a series of notes from the 2009 KM World. Enterprise Search Technologies was a preconference workshop.These notes are done near real time so please excuse any typos or spacing issues.  It was led by independent consultant Miles Kehoe from New Idea Engineering, Inc. Here is the session description.

“This workshop, by a vendor neutral consultant who has hands-on experience with a broad range of “out of the box,” open source, commercial, and home grown solutions, provides an overview of the enterprise search technology landscape. It reviews technologies currently on the market; discusses pros and cons, strengths and weaknesses, and specific requirements Kehoe shares case studies that illuminate how search technologies are leveraged in different types of organizations; and provides a good introduction to and understanding of the enterprise search world.”

Miles said the characteristics of great search include conversational capabilities, open ended, flexible, and smart.  Conversational search allows you to interact to focus your quest. This is especially important for enterprise search, as search is much harder inside the enterprise.  Never provide just - no hits. Ask more if you cannot find anything.  Every search engine is built around a set of indices. This even applies to Google who creates an index through its spiders. Different search engines just add different stuff around the indices.  Every search engine goes through a process. Some expose parts of it which gives you added flexibility to pull specific information out.

It used to be you got plain search results pages in enterprise search like basic Google Web search. Now you get what he called enterprise search 2.0 with visualization, navigation, people, facets, etc. strung around the basic results. 

There are two basic parts of search: indexing and the actual search. It is better to take time at indexing (when people are not waiting) than search when people are waiting. However, I asked if the real time capabilities of Twitter were changing those expectations.  People want to see stuff as soon as it exists.

Before he reviewed vendors, Miles said it is not the technology but the methodology. It is how you implement the search engine. I can agree with this.

As he started to review vendors Miles mentioned Lucene/Soir, a free open source search engine that is behind a number of search engines, including some commercial ones. It is Java based with an Apache license, prolific documentation, many implementations, and you have total control over search and relevance. However, there is some implementation work required and it is hard to find answers. There are limited enterprise support options. SearchBlox is packaged Lucene. Lucid Imagination is packaged Soir.

Miles’ tier one vendors are: Autonomy, Endeca, Exalead, Fast Search (the original independent version), Google, Vivisimo. I have reviewed Exalead (see Exalead’s CloudView Offers Integrated Search Capabilities and Exalead Provides Ability to Integrate with EMC Documentum).  His criteria are: broad enterprise presence, multi-platform search, market penetration, and clear product vision.  People like the Google brand so they have a perception that Google enterprise search works well. Not being in Tier One is not necessarily bad, just not meeting all the criteria. Other vendors I have reviewed that Miles also mentioned include Attivio (see Attivio Aligns with Traction and Releases New Features) that is newer and Recommind (see Recommind Provides Axcelerate eDiscovery 3.0 with New Features) which is more vertical focused.

Dates are important but web servers provide bad data so it is hard to trust what you get. Miles gave the example of a 1996 document appearing as new because it had just been re-indexed. 

The wifi started working so Miles showed us Web sites with good search capabilities. Globrix is a UK real estate site that uses FAST and you could see a lot of facets in home listings such as number of bedroom, bathrooms, price range, etc. Then we looked at Newssift that displays sentiment on topics. We looked at Kosmix that provides an example of exploratory search.  It shows things that are related and loosely related. 

Next we covered supporting technologies including document filters, connectors, social search, and federation.  Document filters are part of the indexing process that converts binary source documents (PDF, Office, etc.) into a stream of text for indexing.  Connectors are utility tools to provide a clearly defined interface between a search engine and external content. Some relate to indexing and others to display. Connectbeam is an example (see my reviews: Connectbeam Offers New Social Networking Application Integration Possibilities).

Social search is a popular term that applies to the capability to search corporate personal profiles to find people in an organization with certain skills or experience. It typically requires user to explicitly self-profile in order for searches to return accurate results. Some products now track user behavior to implicitly associate interest to users.

Federation refers to a program that can dispatch user queries to one or more external data sources (search engines, RDBMS systems, etc.) and present the combined results to the user. Federation from unsecure resources is fairly easy. Because relevance from each source is calculated differently, it is sometimes difficult to integrate results in a meaningful way.

Entity extraction recognizes people, places, or things during indexing. In unsupervised extraction entities are recognized through algorithms. In supervised extraction, the process is seeded by human operators prior to processing.

Sentiment analysis recognizes positive or negative sentiment algorithmically during indexing. It is easier to tell positive sentiment than negative.

Results clustering groups sets of documents into categories base don content. It looks like facets and entity extraction however clustering can be done independent of the query. Clustering is often used in search results to assist the user to discover additional related terms and content.

Facted search is the result of assigning documents in a search result list into a pre-defined taxonomy-like order. Unlike clustering, which can appear similar, facets are base don the query and populate pre-defined classes of content (authors location, etc.). Facets are often used to encourage interaction with user.

A key to having good search is to monitor it over time after the initial implementation. Look at what is happening and make corrections. Look at what people are searching for and accommodation them. You need to pull together a diverse collection of skills to have a great search function (e.g., business domain experts and corporate librarians, beyond just technical skills).

Miles mentioned two blogs on the topic that he writes: EnterpriseSearchBlog,com and SearchComponentsOnline.com. 

   

KM World Sessions: Fundamentals of Enterprise Search

This is the first in a series of notes from the 2009 KM World. Please forgive typos as this isi real time or near real time. Fundamentals of Enterprise Search was a preconference workshop. It was led by Avi Rappoport, Principal - Search Tools Consulting Editor, SearchTools.com. She is an independent consultant not connected with a vendor. Here is the session description.

“Search engines, big and small, have certain standard elements and processes. The more you understand them, the easier to tune them to solve your real information needs. This practical overview provides a big picture view of how search fits within enterprise and websites, and a focused introduction to search technology and user experience. Elements of search covered include robot spiders, database connectors and other tools for locating content, indexing issues, query parsing, retrieval, relevance ranking, and designing usable search interfaces. The workshop addresses common search problems and solutions, security issues, languages, new interface elements, important (and unimportant) features as well as providing tools for choosing a search engine or evaluating an existing one.”

Avi began with similarities and differences of enterprise search and Web search. The differences include:  limited scope, fewer meaningful hyperlinks for link analysis. Security and access control issues, content in databases, more control (for specifying value ranking, etc.), and no search spam.

Next she covered text search vs. data base search. Text search indexes multiple content sources and uses simple search commands instead of SQL. There is flexible indexing and retrieval and relevance ranking (major issue). There are new features such as spell check, auto completes, and facets. It works in the real world (e.g., eBay, Google).

Then she covered how information architecture works with search.  Information architecture is the art and science of organizing information for access and use. It creates order and systems and provides standard vocabulary. Search can supplement information architecture through user vocabularies and dynamic changes with new content.   

KM and search are opposite ways of approaching content. KM organizes stuff, and search finds it.   There are two main types of search: known item with short queries and “good enough” answers and exploratory search for research purposes. This is what Darwin Ecosystems excels at.  It can help you discover relationships between content that you did know exits and were not explicitly looking for. Avi said that search is an iceberg and people often see it as magic.

It is useful to index everything as it is hard to know in advance what people want.  Twitter has changed expectations for real time indexing, even for intranets. Three minutes is a good expectation. Here is another impact of consumer Web on enterprise computing.

Index security is an issue. (see my post: Attivio Aligns with Traction and Releases New Features) Without the right security you can see stuff you should not see. Need to work with security people on search issues to avoid this and have capabilities in the search tool.  The first step requires knowing what needs to be controlled and then you can determine how to do this.  Be aware of privacy laws.

After you determine what to secure, then deal with access control. Best to keep access control info as part of document store (what Attivio does - see post mentioned above). There are four levels of access control. One – access to search engine, two – collection-level access control (to portions of the search engine), three – locked results for a teaser for subscription, four – hit-level access control – link to a access control database at the point of display - hardest to do, useful when constantly changing rules.

Robot spiders start with a base URL for all hosts. For each page they repeat this process: read text info into internet format, save document in cache, save words into index, extract all links and check for rules, if they are new URLs add them to the list. It can repeat process over and over.

There are common problems with robots that SEO tries to avoid. Spiders can be disallowed by robots.txt or robots meta. Also cannot handle URLs with ? and & (but all spiders should handle these now), Javascript, forms, and interactive dynamic links, session IDs that change, multiple views of same data (wikis and Lotus Notes). 

External sources that have APIs like Twitter can be brought into enterprise search. You might want to partition this so it does not clutter standard search.  This is one way you can use Darwin Ecosystem inside the enterprise to look outside it. Relevance is relative. (sounds obvious but need to remember this when creating relevance listing).

Indexing multimedia needs to be dealt with now. There can be internal and external metadata to support this.  Best to use human judgment rather than automated systems. Automated systems can be a starting point but they need to be fine tuned by people. Speech to text and other automated capabilities are still buggy.

Stop words are common terms or ubiquitous terms. Traditionally you excluded them but there are consequences – such as copyright mentions. Best to index everything, especially since storage is much cheaper now. Avi gave a good example of excluding stop words by searching for phrase “whatever well be,” a song title.  On the other hand you can lots of irrelevant stuff.  Another example, the rock band, The Who, Here is where relevance can help so you get a lot if you include stop words but only need to see best ten examples. I tired this on Google and it did work as the top results related to the band even though there were 457,000,000. Avi said that Google may have set up an exception for this term. She also said you might get different results on wordpress.com.

Dealing with duplicate documents can be complex. First you need to decide what is a duplicate and then what is the primary if there are some slight differences (e.g. typo corrections).  Exact match is easy but similarity is more useful, harder but worth it.  Best to remove duplicates from index and hide results unless requested. This is what Google does.  Can create rules for handling duplicates. This is a good idea. However, you need human supervision but it is worth it.

Avi went through the search process: search form – query parser – query engine (goes to inverted index and back)  – relevance ranker (goes to document store, get stuff and brings back – formatter – search results.  This all happens very fast now. Queries come from many sources, not simply search fields. There are alerts, saved searches, automated searches, geographic information systems, and others. You need to balance relevance and completeness. You cannot have both.

Relevance ranking algorithms work differently. The most common is TF-DF or term frequencies : inverse document frequency. How often is the query word in document and how often is word in the index? There are others but this is most efficient. Look at title, metadata, and top of document. Remember relevance is task specific. There is no such thing as objective relevance. You can never please everyone. More like berry picking than hunting, Try different stuff instead of locking on single goal. Here is where Darwin Ecosystem can help with correlations of different topics with target key words.

Be sure to limit the user interface complexity. Google is a great example. Use familiar use interface elements. Put search into navigation so it appears everywhere. With auto-complete, use a drop down menu of matching words. Base this on search logs and use 7 - 10 most popular in alphabetic order

In summary, with enterprise search you have much more control on the capabilities and decisions with your search capability than on the Web. Make good use of these decisions. 

Webcom 09 - The Misplaced Fear of the Mainstream Media

Permeating the recent Webcom 2009 event was the discomfort and fear of mainstream media losing ground to the rise of social media.

The simple reality that information consumption is migrating into a Web dialogue leaves the mainstream media having to compete with what they label as unverified content, and thus accordingly, untrustworthy information. Again, a major perception shift from an industry that has a difficult time distinguishing between unverified written expressions and the wisdom of crowd. I often heard from many traditional media people attending such comments as: “How can people consume such rubbish and unverifiable comments?” “We are dedicated to verifying and delivering quality content that is far more reliable!”.

Formal content providers truly believe that without their controlled content people will have a distorted view of what matters and believe opinions over facts. From their perspective they are absolutely right! If people accept social media information as validated truth, we would have... well, isn’t this what we already have? The simple fact that opinions and discussions are transposed from the oral to the written form does not mean that it changes much regarding the risk of people being misinformed. People distort, fabricate and adopt their own points of view according to their social networks, whether it be physical or the Web. The battle is not one of formal versus informal, but one where formal content providers need to listen and engage in the dialogue.

I was also amused when I heard that mainstream media and large companies were questioning the value of social media and its content. These companies and traditional media spend fortunes on market trend analysis with voodoo-like rearview mirror reporting that only serves to give them an impact appreciation for what they did in order to improve the next campaign. The amusing part is that they have traditionally done this to measure opinions and perceptions to best attract customers, yet today they are worried about the volume of content created by the same people they have been trying to mind-probe through inference investigation. Can someone pinch me here?

The perception that has to change is not the one of the people (it is only a transposition for them), it is the one of the media companies who are far more feudal than the people, as they fear a loss of power. I believe that Thomas W. Malone’s 2004 The Future of Work a great tome on the transformation of business, is more meaningful today in the light of the traditional media companies’ challenge. But are traditional media companies capable of distributed deliberation and content creation with the emerging wisdom of crowds?

The ability to have a real-time written communication has changed the concept of readers and audiences. It is now a dialogue! And from that basis the media industry is transformed forever. Traditional media needs to rethink its relationship with its customers (yes customers as they are no longer passive). They need to focus on their core value of reliable information and take it to a new level that will preserve their integrity as their assumptions regarding the information life-cycles and distribution are changing. Some will understand that their business operating processes are the enemy and that they must create new ones. 

Many media companies are becoming consumed by finding ways to become social destinations themselves than thinking about their core business value proposition and processes. These companies are taking the risk of becoming followers in the media revolution instead of becoming the leaders and champions of valued and verified information. In my opinion they are making a mistake as they are driven by following to catch-up instead of innovating and managing their own transformation.  

See the post by my colleague Bill Ives that appeared here this Monday on What killed the Rocky Mountain News for an example of a mainstream news company that did not successfully handle the rise of social media. In this session the former Editor, John Temple, said that newspapers now need to spend more time on innovation and not simply work on trying to get more revenue for online versions of what they have always done. 

Post script: The Seattle Times recently took the innovative step of inviting editors from some neighborhood blogs to question mayoral candidates for a town hall-style debate organized by the newspaper and its news partner, KING-TV. Here is a good example of creative use of soical media with traditional media. 

Prendre le Pouls 2.0 du succes

Ceci est une translation en français d’un blog Taking the Pulse 2.0 of Success publié récemment afin d’atteindre les communautés de langue française et anglaise qui s'intéressent aux moteurs de découverte et de veille sur internet.

Les entreprises de toutes tailles sont convaincues que leur vision du futur doit guider leurs actions à mener. Afin de s'assurer que leur actions sont alignées avec cette vision, le défi que doit relever le top management est de pouvoir gagner la confiance, susciter l'inspiration et faire agir leurs employés dans le sens de cette vision. Un Top Management  expérimenté s'assurera également d'inclure le point de vue de tous les acteurs intéressés à l'entreprise pour s'assurer leur engagement et faciliter l'exécution des tactiques au service de la vision.

Entretenir et ré-inventer sa vision est un processus très itératif dont dépendent les résultats de l'entreprise. Comme tout système asservi, ce processus sera le plus efficace si l'on peut observer des résultats mesurables le plus tôt possible dans le cycle de vie de la vision. Les résultats sont souvent utilisés pour mesurer si la vision était correcte ou nécessite des ajustements. Aujourd'hui, ce genre de mesures n'offre qu'un pilotage par rétroviseur. Par exemple, quand une entreprise regarde sa courbe des ventes, elle mesure le résultat d'un ensemble d'efforts et d'actions menés en collaboration plusieurs mois auparavant, bien avant de pouvoir devenir un indicateur de succès.

Aujourd'hui, le Top Management a besoin d'une visibilité et d'une lecture en temps-réel de l'alignement des actions et des motivations avec la vision stratégique de l'entreprise. Avec l'émergence des outils de collaboration Web 2.0, beaucoup d'équipes ou d'experts utilisent les espaces de dialogue instantané via un écran, les blogs, les wikis et autres moyens pour coordonner leurs activités et mettre en oeuvre les retours d'expérience pour aboutir à leur objectif plus facilement/rapidement. Malheureusement, cette information n'est que rarement utilisée comme une mesure du pouls de ce qui se passe au jour-le-jour. Ces outils restent isolés, ambigus et produisent du contenu éphémère ou non comptabilisé, simplement parce qu'il est trop hétérogène et que l'on ne sait pas le mesurer. Et pourtant, les employés les utilisent pour délibérer, obtenir des décisions, déléguer, voire à des fins stratégiques.

Avec Darwin, nous offrons une plate-forme qui sait mesurer le pouls des différents initiatives à travers l'univers Web 2.0 de collaboration de l'Entreprise. Pour en savoir plus, contactez nous sur www.Darwineco.com.

Votre entreprise est-elle 2.0? conférence du 28 août à Montréal

Ceci est une traduction en français d’un blo "Is your enterprise 2.0? (Montréal 8/24/09 Conference)” publié récemment afin d’atteindre les communautés de langue française et anglaise qui s'intéressent aux moteurs de découverte et de veille sur internet. Nous avons donné une présentation fin Août à l'événement d’Allard-Hervieu; “Votre entreprise est-elle 2.0?” . L'événement s’est tenu en français et nous étions heureux de noter la présence de représentants des sociétés parmi les plus importantes basées à Montréal (publiques et privées)

Plus de 75 dirigeants étaient présents, apportant leur expérience significative en solutions de collaboration et leur compréhension de l’impact grandissant des technologies Web 2.0. Contrairement à l’habitude pour ce genre de conférences qui deviennent souvent des actions de prospection, Allard-Hervieu a su choisir le bon message et les bons interlocuteurs pour rendre l'expérience mutuellement enrichissante. Les présentations furent animées par 3 orateurs se succédant suivant une histoire logique. Elle démarra avec l'émergence et le positionnement du Web 2.0 dans l’univers de l’entreprise. Elle se poursuivit avec une introduction aux solutions et technologies qui animent et capture les contenus, et les bienfaits de plans de communication pour leur déploiement, le tout illustré par des exemples concrets. Enfin, l’histoire se termine sur comment peut-on finalement mieux consommer toute cette information générée et chaotique afin de mesurer le pouls de ce qui se passe dans et autour de l’entreprise grâce à des classifications par émergence et corrélation organique, en opposition avec les portails complexes, les classements par popularité et outils de KM.

Ce fut l’occasion pour nous de faire pour la première fois une présentation ouverte de notre produit Darwin.... Nous avons utilisé des exemples vivant et temps-réel directement liés aux entreprises de notre audience.... Hydro-Québec, le sujet H1N1 et les tendances des nouvelles canadiennes. Le cas Hydro-Québec fut particulièrement parlant puisqu’il a souligné la controverse du moment sur les donations faites aux écoles privées. Notre technologie a permis de montrer le contraste (mais corrélé) entre le point de vue des médias (formels) et l’opinion du public (informel). De plus, nous avons aussi pu découvrir que le débat était surtout francophone, en montrant un rapport de 10 à 1 en ajustant la préférence linguistique sur l’anglais.

Cette présentation d’une corrélation des thèmes à travers des sources multiples et aussi différentes que le formel vs l’informel a fait réaliser à notre audience le potentiel d’une telle solution pour des applications à l'intérieur de l’intranet de l’entreprise, ou bien en liaison avec le Web.

Improving communication during work-awareness.

Recent surveys and research show that executives finally value social media favorably to improve relationships with customers and build their brand (“Social Media: Embracing the Opportunities, Averting the Risks” paper from Russell Herder). Paradoxically, these same executives fail to embrace these technologies within their organizations to improve productivity through better collaboration, idea sharing, faster initiative adoption and, of course, knowledge capitalization. The reason for this can be attributed to the many past failures of Knowledge Management (KM) and collaborative technologies that were assaulted by armies of consultants and system integrators over-thinking communication empowerment. Today we know that most KM initiatives have failed due to exaggerated efforts to harness tacit knowledge into explicit processes. Having spent most of my carrier in the space of KM, collaboration and social technologies since 1988, I can testify to how many times the Web 2.0 in the enterprise was adopted under different names; Groupware, Shareware, Communication Intranets and others...the bottom line is that they all aimed at improving communication during work-awareness.

Let me expend on work-awareness. Work-awareness in state of mind where the individual is focused on his/her activity and driven by accomplishing the task at hand. If a company is able to empower the individual with communication and information discovery tools, then the information being captured is likely to be rich in relevance and valued as a sharable asset for others. Many power-blogger and micro-bloggers (Twitter) practice this concept to make their knowledge valuable for personal gain and knowledge acquisition. This is an observable phenomenon that companies can use to improve capturing knowledge from their workforce as well as empowering them to share this knowledge to solve problems and improve productivity.

Now back to the bad rep of KM and earlier versions of the Web 2.0. KM, like the early efforts of artificial intelligence, is a noble cause rooted in predictability and measurements that can rival the best structure data analysis. The reality is that the data source is messy and very human. As such, emerging movements and personal interpretations are more meaningful than a well defined empirical dashboard. Basically it requires a human being to extract value from this corpus of unstructured data.

Most of my readers know I have been developing a Web 2.0 awareness technology (Darwin Ecosystem) for the last two years that can can extract relevance out of the chaos of unstructured communication. I was a keynote speaker at the Enterprise 2.O conference in Montreal, Canada last week. Since the conference, I have been exposed to many Canadian companies who are struggling with making sense of their communication platform as they are exploring the implementation of Web 2.0 solutions. After every demonstration of Darwin’s Awareness Engine these companies were amazed at the technology’s ability to aggregate and correlate meaningful information on one screen; one called it “The dashboard of our voices” and an other said “I can finally make sense of what is going on without endless meetings.” Needless to say, we are now discussing the best way to implement Darwin’s Awareness Engine in environments that have only emails, Lotus Notes/Domino, Sharepoint and others with proven Web 2.0 solutions. I look forward to harvesting what they have in their legacy communication tools to show them that the value of collaboration is not about the platform you choose (the standard now), but instead it is about the knowledge hidden and captured during work (the future).

Economist Intelligent Unit Finds Strong Belief in Business Value of Social Software

The Economist Intelligence Unit reported that Web 2.0 has moved from buzzword to reality in many of the world’s largest corporations. They conducted a survey of 406 senior executives worldwide and found that 79% of respondents see the collaborative web as a way to boost revenues and cut costs. They said that, “perhaps the most interesting finding is that a full 85% of C-suite executives see the sharing and collaboration aspects of Web 2.0 as an opportunity to increase revenue and/or margins, versus 75% of middle management.” Nothing like having top down sponsorship delivered to a largely already convinced group of middle managers. This top down effect also extend to the view that Web 2.0 is transformative, affecting all parts of the business (35% versus 28%) and is making a significant impact on the company’s business model (41% versus 22%). They asked good questions.

The top perceived potential for Web 2.0 was to transform the way that companies interact with their customers. Many of these study participants were part of large companies already using Web 2.0 tools and methods. The most cited use was the creation of online communities to help with product marketing or product development. Next was the use of blogs or wikis to initiate conversations and share knowledge inside or outside the company.

It seems that so far executives are more convinced of external uses of Web 2.0 than internal enterprise 2.0 efforts. The study found that 68% of executives expect Web 2.0 methods and tools to be the single biggest factor changing the ways their company interacts with customers. While 49% said these tools will change how employees interact with each other and the company. This is not surprising given the current usage patterns within the sample reported above and the broader awareness of Web 2.0 marketing efforts than enterprise 2.0 collaboration efforts.

Building on this nearly 60% of those surveyed said that their companies are asking customers to contribute their product development and/or enhancement, or that they plan to do so within the coming two years. In addition, “about half of companies say they are, or are planning to, treat customers as co-developers of products that are in a constant state of improvement.” The wisdom of crowds has caught on.

Is your enterprise 2.0? (Montréal 8/24/09 Conference)

I gave a presentation last week in Montreal at the Allar-Hervieu event; “Votre entreprise est-elle 2.0?”.  Of course, it was in French and I was pleased that it  attracted some of the more important Montreal-based public and private companies. 

Over 75 executives, with extensive experience in the importance of collaborative computing and the emerging impact of  Web 2.0 technologies, were present at the event.  This was a breeze of fresh air as many such conferences are occupied by Web industry enthusiasts looking for networking opportunities and prospects.  Not this time!  This time the communication firm Allar-Hervieu of Montreal targeted the right audience with the right message.

The presentation was made by 3 speakers delivering a logical story-line that addressed the positioning and reality of the emergence of the Web 2.0 in the enterprise, the use of content-capturing technologies and the need for thoughtful communication plans through concrete examples. and lastly how this chaos of information can be best consume to help the enterprise measure the pulse of what is happening through organic correlation and emergence ranking in opposition to complex portals, popularity ranking and KM tools.

This event was our first open product demonstration.  I used live examples relating to the audience's   organizations... Hydro-Québec, H1N1 trends and Canadian news events.  The Hydro-Québec live demonstration was particularly relevant since it outlined a current controversy about donations to private schools. Our technology was able to show the formal media perspective in contrast (and correlated) with social media citizen contributions.  Additionally, we also noticed that the Hydro-Québec debate was clearly francophone-driven as a contrast of 1 to 10 was visible when tuning the linguistic preference. 

This demonstration, of correlation sources originating from the formal and informal Web, demonstrated to the audience how they can benefit from this technology within their intranet as well as reaching to Web content.

Taking the Pulse 2.0 of Success

Organizations of every size believe their vision for the future must lead their actions to be realized. To ensure that actions are aimed towards that end, the challenge of the executive team is about gaining the trust, leading the inspiration and ensuring that the vision will be adopted and acted upon. An experienced executive team will also ensure that the organization’s stakeholders express their vision and share its ownership. This provides for a better feasibility assessment and execution commitment.

Maintaining and reinventing a vision is a highly iterative process that shapes future outcomes. This process can be most effective when observing measurable results as early as possible in the vision’s life cycle. The results are often used to measure if the vision was correct or requires improvement. Nowadays this form of measurement provides leadership with a rear-view mirror decision-making perspective. In fact, when an organization measures sales, it is the outcome of coordinated collaborative efforts that have often occurred months before becoming a key indicator of success. 

Today management needs to have real-time visibility on the company’s vision adoption and actions that lead to the measured result. With the emergence of Web 2.0 collaboration tools, many teams and knowledge workers use chat rooms, blogs, wikis and other means to coordinate their activities and implement best practices aimed at reaching their common goal. Unfortunately, this information and knowledge is rarely observed as a measurement of the pulse of what is going on day-to-day. These tools remain segregated, ambiguous and lead to perishable or ignored content simply because they are too heterogeneous to provide measurements. Yet employees use them for deliberation, decision buy-in, delegation and strategic purposes.

We have a platform that provides an overview of pulse of different initiatives across the enterprise’s Web 2.0 collaboration.

Web 2.0: A Collective Conscientiousness Platform?

I have been exploring the role that the WEB 2.0 can have as platform for the representation of Collective Conscientiousness. With blogs, wikipedia, wikinews, social networking, MySpace, LinkedIn, YouTube and other highly interactive and social network-enabled features, we are starting to see popular themes and sub-culture groups emerge to the mainstream. With the interaction between people’s interests and the need to express themselves (their lives) with these systems, we are providing our society with near-real-time awareness of what matters to people. 

I believe that we will be able to harness the commonality of the expressions found across these systems and be capable of viewing the expressions of the Collective Conscientiousness. I think that this awareness can influence actions towards the evolving Collective Conscientiousness that can be of use to market analysis, political campaigns and governance.

Please comment and let me know if this is a topic of interest and if you would like to join a focus group to further discuss and explore the possibilities of a visible collective conscientiousness.

Social Media Leads Future of Technology – HBS Panel

Here is an interesting brief report from Harvard Business School on a panel moderated by HBS professor David Yoffie. The session was on "The Technology Revolution and its Implications for the Future," with panelists James Breyer (HBS MBA '87), partner of the venture capital firm Accel Partners; Susan L. Decker (HBS MBA '86), president of Yahoo! Inc.; and Eric Kim (HBS MBA '81), senior vice president and general manager of Intel Corporation's Digital Home Group

Some useful stats include: “Of 6.5 billion people in the world, about 1.5 billion have Internet access, more than 300 million have broadband access to the home, and 3 billion have cell phones, a growing number of which offer Internet access.”

I was especially interested in the quote from James Breyer, "Each year there is more information created on the Web than in all the previous years combined. Investment initiatives are around participating in the information flow. We [at Accel] are interested in companies that help us understand how to structure information, communicate, categorize some of that self-generated information, and then act on it."

Susan Decker added, "But fundamentally, removing the complexity and adding simplicity so you can easily access in an open way everything you want, and leverage a lot of social connections rather than going to multiple ones, is how the user experience will evolve."

These are issues that we are working on at Darwin. The alpha version is accessible by invitation only. Contact us through the web site if you are passionate about Knowledge Management and Semantic Web organic-based architectures and computations.