Darwin Discovery Engine Blog

Enterprise Search Summit 2010 Comes to NYC and KM World 2010 Moves to Washington DC

This year’s Enterprise Search Summit 2010 will be at the New York Hilton on May 11-12. The KM World Conference & Exhibition 2010 has moved to Washington, D.C. It will be November 16 - 18, 2010 at Marriott Renaissance,  It is nice to see these two conferences on the East Coast. I went last year in the November to the combined conference in San Jose.

Thierry Hubert and I will be presenting at the Enterprise Search Summit 2010. We are presenting on Deriving Order From Chaos Through Discovery and Awareness. Our session is on Wednesday May 12 at 1PM. The Enterprise Search Summit says that the “emphasis for Enterprise Search Summit is on how enterprise search enables Information Access. Search can no longer be viewed as a stand-alone application. It is increasingly part of everything we do and has become the de facto gateway to information in the enterprise.” I would certainly agree and we hope to see you there.

The KM World announcement said that this year's theme is KnowHow: the Knowledge-Driven Enterprise. “The knowledge of how to do something smoothly and efficiently with technical skill and expertise -- also known as "knowhow" -- drives every enterprise, agency, and organization. Creating knowledge bases, sharing them effectively, using them for decision-making and innovation.”

This sounds familiar but it is great that it is still very relevant. The session shave a 2.0 flavor like last year. I hope to be able to attend. Deadline for speaker submissions is March 1. Here are my notes from the combined event last year.

KM World and Enterprise Search Summit: Is Semantic Technology Real?

KM World and Enterprise Search Summit: Evolve From a Tactical E-Discovery Approach to Search and E-Discovery

KM World and Enterprise Search Summit: The Role of Social Techniques in Search & How It Impacts Your Organization

KM World and Enterprise Search Summit Sessions: From Birth to Billions: The Life Story of Google Enterprise Search

KM World and Enterprise Search Summit Sessions: Fundamentals of Enterprise Search

KM World and Enterprise Search Summit Sessions: Enterprise Search Technologies

Information Awareness and Discovery for Life Sciences and Pharmaceutical Companies

Success for lIfe sciences and pharmaceutical companies is often associated with their ability to manage knowledge transition and information sharing. This is especially true as as they deal with the challenges of many recent mergers in the industry.  The segregated systems where the departments’ collaborative efforts are stored represent a significant barrier to business continuation and leveraging knowledge.  As system integration analysis and process streamlining efforts struggle to unify these systems, the companies are losing precious time to better manage their R&D, clinical trials, and manufacturing, all the while  ensuring efficient CRO and CMO monitoring relationships.

Considering that these companies are knowledge intensive, it is critical for them to improve R&D productivity.  However, the increasing complexity of information systems and the increasing amount of semi-structured collaborative technologies increases the level of information overload, growing work flow complexity, and knowledge isolation through diverse communication and storage tools.

These companies’ organizational models are in the perpetual struggle of managing rigorous processes and needing to be flexible and organic in their structure. 

Mastering knowledge in a timely manner is key to their evolution and discovery processes.  This is why Darwin Ecosystem takes a different approach to accelerating information consumption by breaking the silos of knowledge to allow for meaningful discovery.  Our technology aggregates the information made accessible through simple syndication channels (RSS) and correlates it to deliver an awareness experience according to the user’s context of interest.  This  presentation, known as a Scan Cloud, immediately  displays what is happening across the organization’s departments by outlining correlated clusters and dimension classes that make it easy to grasp what is happening and direct attention  to the information of interest.

Darwin’s Awareness Engine is designed to evolve with the company and its users, and it can process new sources on demand, thus adding to your corpus of correlated information.  There  is no need for complex dashboards, strenuous process analysis and complex system integration.  All that is required is to make the content available to the service as an RSS feed.   Darwin’s Awareness Engine will correlate the content as it comes.  The experience can also be enhanced by customizing the taxonomy according to the company’s cultures and context. Darwin provides an awareness of what is d going both inside and outside the organization so that emerging themes do not get lost in silos.

Illustrations:

The following illustrates a correlation from mainstream media and blogs the morning of December 14, 2009.  The Scan Cloud™ show the connections between the emerging events associated with the Monsanto company.  A selection of on the Monsanto key word further filtered the sources and allowed for social media content to emerge as seen in the second screen capture.  The selected blog entry reveals a criticism of Monsanto. 

 

Global Most Admired Knowledge Enterprises (MAKE) Winners Announced for 2009

This certainly brings back memories. The Most Admired Knowledge Enterprises (MAKE) award has been around for some time. Back when I was with a large consulting company and involved in our knowledge management practice, I spoke at the awards meetings in London several times.  These were the days of better travel funding and the relatively early days of KM. I am pleased that this award is still around and still relevant. Knowledge management seems to be one of the longer running management initiatives. Now enterprise 2.0 has stepped up the potential for KM and better content curation is emerging as new capability. Darwin Ecosystem is designed to play a part in this trend. 

For the first time, Apple was named the overall Global MAKE Winner. North America took top honors with 7 Winners, followed by Asia (5 Winners) and Europe (3 Winners). Teleos in association with The KNOW Network conducts this award. There were also 5 'Global' Winners whose organizational structures reflect the 21st global knowledge economy.

A panel of experts made the picks based on these criteria:

- creating an enterprise knowledge-driven culture

- developing knowledge workers through senior management leadership

- developing and delivering knowledge-based products/services/solutions

- maximizing enterprise intellectual capital

- creating an environment for collaborative knowledge sharing

- creating a learning organization

- delivering value based on stakeholder knowledge

- transforming enterprise knowledge into shareholder/stakeholder value

This year’s winners include some long time winners and some major companies that you would not associate wit KM in the 90s. 

Complete Listing of Our KM World 2009 Session Notes

We have been blogging some of the excellent KM World sessions. Here are links to the seven sessions we posted on last week. There is an emphasis on search and discovery as this most relates to Darwin Ecosystem. This was a great conference from our perspective. 

We hope these notes are useful and welcome your thoughts on the sessions.

KM World Sessions: Is Semantic Technology Real?

KM World Sessions: Evolve From a Tactical E-Discovery Approach to Search and E-Discovery

KM World Sessions: The Role of Social Techniques in Search & How It Impacts Your Organization

KM World Sessions: From Birth to Billions: The Life Story of Google Enterprise Search

KM World Sessions: Fundamentals of Enterprise Search

KM World Sessions: Enterprise Search Technologies

KM World Sessions: Resetting the Enterprise With 2.0 Collaborative Tools

KM World Sessions: From Birth to Billions: The Life Story of Google Enterprise Search

This is another in a series of notes from the 2009 KM World. This session is titled, “From Birth to Billions: The Life Story of Google Enterprise Search” by Cyrus Mistry, Enterprise Product Manager – Google. Here is the session description.

“This talk examines the state of Google’s Enterprise Search business, highlighting the origins of the iconic yellow box and the way the product and business search market has evolved over the past decade. Mistry highlights the challenges facing enterprises and providers of business search technology, as well as possible solutions and case studies of companies and organizations leading the way in search innovation.”

Cyrus began by describing the common problem of having too much information at work. The average worker creates over a gigabyte of information at work and much of this is in email. So Google were asked if they could bring Google into the enterprise.  At first they resisted but there was too much demand so they started Google enterprise search.

They found people wanted three things. First, people want simplicity.  The average number of words put into a Google search field is shrinking but people have the same expectations for good results.  They also want speed of response.  They did studies and found that the faster the results the more relevant people felt the results were. The third thing people want is relevance.

But things are different in the enterprise, Page rank is not relevant as there is not the same linking as on the Web.  In 2001 they launched Google for the enterprise in a small yellow pizza size box at a small price point that opened it up to more businesses. Cyrus said it also help lower the price of many other search tools. In 2004 they added more security. 

People felt Google was not connected so they did Open Source and established the connector framework.  Next after the basics were done they looked at the search experience.  They wanted to bring all types of results (text, images, etc.)  into a single screen.  This provided intelligent simplicity.  Next they looked at scalability so you can reach all documents. With Google you can add more boxes to get scale even up to a billion documents.

Now they are looking at social signals. Cyrus said this is where the innovation will be found.  The feature of user added results has been the most popular capability. People get really excited about this.  Another popular social feature is query suggestions to see related things that others are searching for.  They found a big jump is usage after this feature was adopted. They also look at where people click to see if a result should be elevated.  They use a complex algorithm to overcome any bias of current rank of this elevation.

 

 

KM World Sessions: Fundamentals of Enterprise Search

This is the first in a series of notes from the 2009 KM World. Please forgive typos as this isi real time or near real time. Fundamentals of Enterprise Search was a preconference workshop. It was led by Avi Rappoport, Principal - Search Tools Consulting Editor, SearchTools.com. She is an independent consultant not connected with a vendor. Here is the session description.

“Search engines, big and small, have certain standard elements and processes. The more you understand them, the easier to tune them to solve your real information needs. This practical overview provides a big picture view of how search fits within enterprise and websites, and a focused introduction to search technology and user experience. Elements of search covered include robot spiders, database connectors and other tools for locating content, indexing issues, query parsing, retrieval, relevance ranking, and designing usable search interfaces. The workshop addresses common search problems and solutions, security issues, languages, new interface elements, important (and unimportant) features as well as providing tools for choosing a search engine or evaluating an existing one.”

Avi began with similarities and differences of enterprise search and Web search. The differences include:  limited scope, fewer meaningful hyperlinks for link analysis. Security and access control issues, content in databases, more control (for specifying value ranking, etc.), and no search spam.

Next she covered text search vs. data base search. Text search indexes multiple content sources and uses simple search commands instead of SQL. There is flexible indexing and retrieval and relevance ranking (major issue). There are new features such as spell check, auto completes, and facets. It works in the real world (e.g., eBay, Google).

Then she covered how information architecture works with search.  Information architecture is the art and science of organizing information for access and use. It creates order and systems and provides standard vocabulary. Search can supplement information architecture through user vocabularies and dynamic changes with new content.   

KM and search are opposite ways of approaching content. KM organizes stuff, and search finds it.   There are two main types of search: known item with short queries and “good enough” answers and exploratory search for research purposes. This is what Darwin Ecosystems excels at.  It can help you discover relationships between content that you did know exits and were not explicitly looking for. Avi said that search is an iceberg and people often see it as magic.

It is useful to index everything as it is hard to know in advance what people want.  Twitter has changed expectations for real time indexing, even for intranets. Three minutes is a good expectation. Here is another impact of consumer Web on enterprise computing.

Index security is an issue. (see my post: Attivio Aligns with Traction and Releases New Features) Without the right security you can see stuff you should not see. Need to work with security people on search issues to avoid this and have capabilities in the search tool.  The first step requires knowing what needs to be controlled and then you can determine how to do this.  Be aware of privacy laws.

After you determine what to secure, then deal with access control. Best to keep access control info as part of document store (what Attivio does - see post mentioned above). There are four levels of access control. One – access to search engine, two – collection-level access control (to portions of the search engine), three – locked results for a teaser for subscription, four – hit-level access control – link to a access control database at the point of display - hardest to do, useful when constantly changing rules.

Robot spiders start with a base URL for all hosts. For each page they repeat this process: read text info into internet format, save document in cache, save words into index, extract all links and check for rules, if they are new URLs add them to the list. It can repeat process over and over.

There are common problems with robots that SEO tries to avoid. Spiders can be disallowed by robots.txt or robots meta. Also cannot handle URLs with ? and & (but all spiders should handle these now), Javascript, forms, and interactive dynamic links, session IDs that change, multiple views of same data (wikis and Lotus Notes). 

External sources that have APIs like Twitter can be brought into enterprise search. You might want to partition this so it does not clutter standard search.  This is one way you can use Darwin Ecosystem inside the enterprise to look outside it. Relevance is relative. (sounds obvious but need to remember this when creating relevance listing).

Indexing multimedia needs to be dealt with now. There can be internal and external metadata to support this.  Best to use human judgment rather than automated systems. Automated systems can be a starting point but they need to be fine tuned by people. Speech to text and other automated capabilities are still buggy.

Stop words are common terms or ubiquitous terms. Traditionally you excluded them but there are consequences – such as copyright mentions. Best to index everything, especially since storage is much cheaper now. Avi gave a good example of excluding stop words by searching for phrase “whatever well be,” a song title.  On the other hand you can lots of irrelevant stuff.  Another example, the rock band, The Who, Here is where relevance can help so you get a lot if you include stop words but only need to see best ten examples. I tired this on Google and it did work as the top results related to the band even though there were 457,000,000. Avi said that Google may have set up an exception for this term. She also said you might get different results on wordpress.com.

Dealing with duplicate documents can be complex. First you need to decide what is a duplicate and then what is the primary if there are some slight differences (e.g. typo corrections).  Exact match is easy but similarity is more useful, harder but worth it.  Best to remove duplicates from index and hide results unless requested. This is what Google does.  Can create rules for handling duplicates. This is a good idea. However, you need human supervision but it is worth it.

Avi went through the search process: search form – query parser – query engine (goes to inverted index and back)  – relevance ranker (goes to document store, get stuff and brings back – formatter – search results.  This all happens very fast now. Queries come from many sources, not simply search fields. There are alerts, saved searches, automated searches, geographic information systems, and others. You need to balance relevance and completeness. You cannot have both.

Relevance ranking algorithms work differently. The most common is TF-DF or term frequencies : inverse document frequency. How often is the query word in document and how often is word in the index? There are others but this is most efficient. Look at title, metadata, and top of document. Remember relevance is task specific. There is no such thing as objective relevance. You can never please everyone. More like berry picking than hunting, Try different stuff instead of locking on single goal. Here is where Darwin Ecosystem can help with correlations of different topics with target key words.

Be sure to limit the user interface complexity. Google is a great example. Use familiar use interface elements. Put search into navigation so it appears everywhere. With auto-complete, use a drop down menu of matching words. Base this on search logs and use 7 - 10 most popular in alphabetic order

In summary, with enterprise search you have much more control on the capabilities and decisions with your search capability than on the Web. Make good use of these decisions. 

Webcom 09 - The Misplaced Fear of the Mainstream Media

Permeating the recent Webcom 2009 event was the discomfort and fear of mainstream media losing ground to the rise of social media.

The simple reality that information consumption is migrating into a Web dialogue leaves the mainstream media having to compete with what they label as unverified content, and thus accordingly, untrustworthy information. Again, a major perception shift from an industry that has a difficult time distinguishing between unverified written expressions and the wisdom of crowd. I often heard from many traditional media people attending such comments as: “How can people consume such rubbish and unverifiable comments?” “We are dedicated to verifying and delivering quality content that is far more reliable!”.

Formal content providers truly believe that without their controlled content people will have a distorted view of what matters and believe opinions over facts. From their perspective they are absolutely right! If people accept social media information as validated truth, we would have... well, isn’t this what we already have? The simple fact that opinions and discussions are transposed from the oral to the written form does not mean that it changes much regarding the risk of people being misinformed. People distort, fabricate and adopt their own points of view according to their social networks, whether it be physical or the Web. The battle is not one of formal versus informal, but one where formal content providers need to listen and engage in the dialogue.

I was also amused when I heard that mainstream media and large companies were questioning the value of social media and its content. These companies and traditional media spend fortunes on market trend analysis with voodoo-like rearview mirror reporting that only serves to give them an impact appreciation for what they did in order to improve the next campaign. The amusing part is that they have traditionally done this to measure opinions and perceptions to best attract customers, yet today they are worried about the volume of content created by the same people they have been trying to mind-probe through inference investigation. Can someone pinch me here?

The perception that has to change is not the one of the people (it is only a transposition for them), it is the one of the media companies who are far more feudal than the people, as they fear a loss of power. I believe that Thomas W. Malone’s 2004 The Future of Work a great tome on the transformation of business, is more meaningful today in the light of the traditional media companies’ challenge. But are traditional media companies capable of distributed deliberation and content creation with the emerging wisdom of crowds?

The ability to have a real-time written communication has changed the concept of readers and audiences. It is now a dialogue! And from that basis the media industry is transformed forever. Traditional media needs to rethink its relationship with its customers (yes customers as they are no longer passive). They need to focus on their core value of reliable information and take it to a new level that will preserve their integrity as their assumptions regarding the information life-cycles and distribution are changing. Some will understand that their business operating processes are the enemy and that they must create new ones. 

Many media companies are becoming consumed by finding ways to become social destinations themselves than thinking about their core business value proposition and processes. These companies are taking the risk of becoming followers in the media revolution instead of becoming the leaders and champions of valued and verified information. In my opinion they are making a mistake as they are driven by following to catch-up instead of innovating and managing their own transformation.  

See the post by my colleague Bill Ives that appeared here this Monday on What killed the Rocky Mountain News for an example of a mainstream news company that did not successfully handle the rise of social media. In this session the former Editor, John Temple, said that newspapers now need to spend more time on innovation and not simply work on trying to get more revenue for online versions of what they have always done. 

Post script: The Seattle Times recently took the innovative step of inviting editors from some neighborhood blogs to question mayoral candidates for a town hall-style debate organized by the newspaper and its news partner, KING-TV. Here is a good example of creative use of soical media with traditional media. 

A New Age of Communication-Centric Collaboration

Twenty years ago Lotus Development Corporation released Lotus Notes.  As one of the fortunate alpha tester and early adopters, Notes changed the way I relate to collective information and inspired my career in collaboration and social computing.  Since then  the platforms have changed but collaboration computing remains rooted in Ray Ozzie’s Lotus Notes vision.  As an extension to Ray’s view of collaboration, I created in 1996 the first social network (InterCommunity) on the Lotus Domino platform (Web server) and delivered the first Web-based community social network upon which partners could build community applications (similar to FaceBook).  The solutions we see on the Web today were understood and often existed in large enterprises as well as being available to a handful of early business Web communities throughout the ‘90s.   

For the last 20 years I have also observed companies going through the painful transition of archaic email and file sharing to Wikis and collaboration solutions.  The simple fact that collaboration is stuck in application silos is deeply rooted in the segregation of process and communication tools.  When you think about it,communication and deliberation precede the formality of process, yet most information systems have it backwards and do not connect the two effectively (even if they have all the latest Web 2.0 tools).  Frankly, other than Web protocols, the core usage challenges have not changed since 1989 despite the Web’s richer applications and social computing innovations; connectivity and open source is what made the difference thus far, not the applications themselves.  As we are naturally motivated in transposing what we know over understanding the opportunities that a new environment offers, we often continue to make the same mistakes and build technologies to over-think what is natural; our innate ability to communicate and  evolve.

But now Google shows us Google Wave.  Not since my introduction to Lotus Notes twenty years ago have I been wowed with innovation in that space.  Google Wave truly rethinks communication streams and collaboration as it should be with today’s connected world.  Google did not reinvent the wheel, they just proved that it a sphere.  It is perhaps too server-centric for  my taste, but it  is just a matter of time before the model can be truly internet-based instead of mainframe-like.  Nonetheless, Google Wave’s API does open new ways to incorporate content, applications and structure in the stream of human communication in real-time.  Lotus Notes first version had some very basic features that made it possible to build solutions that promoted collaboration and content distribution, as well as  access control, replication, forms, views, scripting language, mail and later Web standards.  With these integrated basics developers created applications that exceeded Lotus‘ expectations.  Google Wave’s core architecture has the building blocks, openness and a framework based on today’s Web computing reality that will allow the user and developer community to marvel us with their clever use of Google Wave.  The communication-centric framework of Google Wave is simply brilliant and users will gradually embrace its many features as well as enrich their use through third-party plug-ins. 

I have read the more skeptical and critical press on Google Wave.  In my opinion they have short memory and grossly under estimate what Google is doing to the Web with this truly new paradigm.  Facebook and other messaging technologies need to think about ways to connect and leverage Google Wave or they will be left in the dust by its user community.  As to the critics who claim that the UI is too complex for every day users... I would like to add that if you survived and supported FaceBook’s user experience, Google Wave will not be a problem.  This is not your father’s Web so get with the program and compute with passion and intelligence.  We are about to discover new ways to have a return of investment for all the time we wasted promoting ourselves on the Web 2.0.  Isn’t it time for something new? 

I see Google Wave as new platform capable of creating a much needed movement to revive business opportunities and bring new players to the Web.  I also feel fortunate that the Darwin Awareness Engine brings a solution to the information overload that Google Wave generates.  If history repeats itself, Google Wave’s composition is destined to succeed. 

Improving communication during work-awareness.

Recent surveys and research show that executives finally value social media favorably to improve relationships with customers and build their brand (“Social Media: Embracing the Opportunities, Averting the Risks” paper from Russell Herder). Paradoxically, these same executives fail to embrace these technologies within their organizations to improve productivity through better collaboration, idea sharing, faster initiative adoption and, of course, knowledge capitalization. The reason for this can be attributed to the many past failures of Knowledge Management (KM) and collaborative technologies that were assaulted by armies of consultants and system integrators over-thinking communication empowerment. Today we know that most KM initiatives have failed due to exaggerated efforts to harness tacit knowledge into explicit processes. Having spent most of my carrier in the space of KM, collaboration and social technologies since 1988, I can testify to how many times the Web 2.0 in the enterprise was adopted under different names; Groupware, Shareware, Communication Intranets and others...the bottom line is that they all aimed at improving communication during work-awareness.

Let me expend on work-awareness. Work-awareness in state of mind where the individual is focused on his/her activity and driven by accomplishing the task at hand. If a company is able to empower the individual with communication and information discovery tools, then the information being captured is likely to be rich in relevance and valued as a sharable asset for others. Many power-blogger and micro-bloggers (Twitter) practice this concept to make their knowledge valuable for personal gain and knowledge acquisition. This is an observable phenomenon that companies can use to improve capturing knowledge from their workforce as well as empowering them to share this knowledge to solve problems and improve productivity.

Now back to the bad rep of KM and earlier versions of the Web 2.0. KM, like the early efforts of artificial intelligence, is a noble cause rooted in predictability and measurements that can rival the best structure data analysis. The reality is that the data source is messy and very human. As such, emerging movements and personal interpretations are more meaningful than a well defined empirical dashboard. Basically it requires a human being to extract value from this corpus of unstructured data.

Most of my readers know I have been developing a Web 2.0 awareness technology (Darwin Ecosystem) for the last two years that can can extract relevance out of the chaos of unstructured communication. I was a keynote speaker at the Enterprise 2.O conference in Montreal, Canada last week. Since the conference, I have been exposed to many Canadian companies who are struggling with making sense of their communication platform as they are exploring the implementation of Web 2.0 solutions. After every demonstration of Darwin’s Awareness Engine these companies were amazed at the technology’s ability to aggregate and correlate meaningful information on one screen; one called it “The dashboard of our voices” and an other said “I can finally make sense of what is going on without endless meetings.” Needless to say, we are now discussing the best way to implement Darwin’s Awareness Engine in environments that have only emails, Lotus Notes/Domino, Sharepoint and others with proven Web 2.0 solutions. I look forward to harvesting what they have in their legacy communication tools to show them that the value of collaboration is not about the platform you choose (the standard now), but instead it is about the knowledge hidden and captured during work (the future).

Taking the Pulse 2.0 of Success

Organizations of every size believe their vision for the future must lead their actions to be realized. To ensure that actions are aimed towards that end, the challenge of the executive team is about gaining the trust, leading the inspiration and ensuring that the vision will be adopted and acted upon. An experienced executive team will also ensure that the organization’s stakeholders express their vision and share its ownership. This provides for a better feasibility assessment and execution commitment.

Maintaining and reinventing a vision is a highly iterative process that shapes future outcomes. This process can be most effective when observing measurable results as early as possible in the vision’s life cycle. The results are often used to measure if the vision was correct or requires improvement. Nowadays this form of measurement provides leadership with a rear-view mirror decision-making perspective. In fact, when an organization measures sales, it is the outcome of coordinated collaborative efforts that have often occurred months before becoming a key indicator of success. 

Today management needs to have real-time visibility on the company’s vision adoption and actions that lead to the measured result. With the emergence of Web 2.0 collaboration tools, many teams and knowledge workers use chat rooms, blogs, wikis and other means to coordinate their activities and implement best practices aimed at reaching their common goal. Unfortunately, this information and knowledge is rarely observed as a measurement of the pulse of what is going on day-to-day. These tools remain segregated, ambiguous and lead to perishable or ignored content simply because they are too heterogeneous to provide measurements. Yet employees use them for deliberation, decision buy-in, delegation and strategic purposes.

We have a platform that provides an overview of pulse of different initiatives across the enterprise’s Web 2.0 collaboration.

Web 2.0: A Collective Conscientiousness Platform?

I have been exploring the role that the WEB 2.0 can have as platform for the representation of Collective Conscientiousness. With blogs, wikipedia, wikinews, social networking, MySpace, LinkedIn, YouTube and other highly interactive and social network-enabled features, we are starting to see popular themes and sub-culture groups emerge to the mainstream. With the interaction between people’s interests and the need to express themselves (their lives) with these systems, we are providing our society with near-real-time awareness of what matters to people. 

I believe that we will be able to harness the commonality of the expressions found across these systems and be capable of viewing the expressions of the Collective Conscientiousness. I think that this awareness can influence actions towards the evolving Collective Conscientiousness that can be of use to market analysis, political campaigns and governance.

Please comment and let me know if this is a topic of interest and if you would like to join a focus group to further discuss and explore the possibilities of a visible collective conscientiousness.