The evolution of knowledge work: from information overload to insight generation

In today's digital landscape, we're overwhelmed with information but always wanting more insights. Knowledge workers have access to more data than ever, yet many organizations struggle to translate this information abundance into meaningful and actionable understanding. This blog explores why most knowledge management tools focus mainly on organization and retrieval rather than knowledge creation and why human involvement remains essential.

Information vs. knowledge vs. insight: understanding the differences

Before diving deeper, let's clarify three key concepts:

• Information: Raw facts, data, and content from any type of source. Think of spreadsheets, documents, and databases but also facts from conversations, task statuses, emails, calendar events and more.
• Knowledge: Processed information that a person understands and can apply. This includes both explicit knowledge (written or stated facts) and tacit knowledge (experiential know how).
• Insight: Novel connections and conclusions drawn from knowledge that create new understanding or value. For example, insights often emerge when we connect previously unrelated pieces of knowledge or notice something that’s missing in the current knowledge landscape.

The progression from information to insight is the essence of high value knowledge work, yet most tools only support the first stage of this journey.

‍

Historical context: from filing cabinets to digital databases

The modern concept of "knowledge work" emerged in the post-WWII era, with Peter Drucker coining the term in 1959 as white collar information workers began outnumbering manual laborers.¹ Technological milestones, from mainframe computers in the 1960s to personal computing in the 1970s and the World Wide Web in 1991, rapidly transformed how we handle information and created entirely new professions.² By 2023, over 60% of U.S. workers were classified as "white collar," up from a much smaller fraction mid-century, confirming Drucker's prediction that knowledge workers would be the key asset of the 21st century.³

Most knowledge management initiatives throughout this evolution focused on capturing, organizing, and retrieving explicit knowledge. The assumption was simple: make workers more productive by storing information for easy access. By the late 1990s, however, thought leaders began recognizing that knowledge isn't just stored in databases, it lives in people.⁴ Organizations started promoting communities of practice and storytelling to share tacit know how that couldn't be captured in formal documents. For example, Xerox's Tech Reps dramatically improved printer repair rates by sharing "war stories" in informal gatherings rather than following technical manuals. Similarly, the World Bank established thematic communities of practice where experts could exchange experiential knowledge across geographical boundaries, significantly improving project outcomes in developing countries.

This shift revealed a vital pattern: we've evolved from treating knowledge as an object to be cataloged toward understanding knowledge as a process requiring human interaction and interpretation.

‍

Current landscape: the organization<>creation imbalance

Today's knowledge management ecosystem encompasses a diverse array of tools. Enterprise platforms like Microsoft SharePoint (used by over 190 million people) and Confluence (used by over 300,000 people) serve as central repositories. Personal knowledge management apps like Evernote (250 million registered users by 2024) and Notion (100 million users by 2024) help individuals organize their digital notes. 

These tools excel at what they were designed for – organization, storage, and retrieval of information. They aim to create centralized "single sources of truth" where documents and data can be stored and searched, preventing knowledge loss and redundant work.

Most tools treat knowledge as static information rather than dynamic insight. They manage explicit knowledge but overlook tacit knowledge. This "stock" concept of knowledge, treating it like a static stockpile of information, still underpins many tools. It leads these tools to become individual digital filing cabinets rather than insight generators.

The real world impact of this limitation is substantial. Consider a product manager who needs to synthesize customer feedback from multiple sources to identify a new market opportunity. She might store survey data in one tool, interview transcripts in another, and competitor analysis in another system. Despite knowing where all these pieces of information live, connecting the dots across these tool silos to reach a meaningful conclusion becomes a manual, time consuming process that relies entirely on her mental bandwidth. Knowledge workers lose up to 30% of their time just searching for information across these disconnected systems, adding to the time it takes for synthesizing actionable insights.

Another limitation is information overload and organization fatigue. Users can spend excessive time tagging, linking, and organizing notes, which yields a beautifully organized archive but not necessarily new knowledge. The investment in organization to make later search for information faster paradoxically overshadows the synthesis of information. The sheer tedium and effort of 'gardening' new pieces of information often becomes the primary task, diminishing the actual generation of insights that should be the ultimate goal.

All this work around organizing knowledge can lead to confusion about what knowledge work really entails. Christian Tietze, in his critique of modern note taking systems, notes the "Collector's Fallacy," the tendency to collect huge amounts of information under the assumption that having information equals knowing it.  Author Umberto Eco similarly observed how students often confuse photocopying with learning; they copy piles of articles and "relax as if they had read it." ⁶

‍

The knowledge creation gap: beyond information storage

A crucial cognitive distinction exists between storing information, processing knowledge, and generating insights. This three stage progression represents the value chain of intellectual work; to restate:

• Information consists of raw facts quickly perceived, recorded and transferred—the basic building blocks of understanding. 
• Knowledge represents the meaningful processing of this information within existing mental frameworks. When we develop knowledge, we organize information into coherent structures that we can understand and apply.
• Insights, at the highest level, emerge when we connect previously unrelated pieces of knowledge in novel ways that create new understanding or value. An insight is the "aha!" moment that transforms how we see a situation. 

Our brains don't work like databases. Information is stored, but knowledge and insights arise through association and pattern recognition. Research shows that insight generation happens when the brain shifts from focused attention to a more relaxed, diffuse mode that allows distant concepts to connect.⁷ This explains why the most valuable ideas often emerge not while actively searching for information, but during periods of reflection or even unrelated activities.

This has practical implications for daily work. When a strategy consultant is developing recommendations for a client, they progress through all three stages: collecting information (industry reports, competitor analyses), developing knowledge (understanding industry dynamics and client position), and finally generating insights (seeing novel connections that lead to competitive advantage). Current tools excel at helping with information collection and sometimes knowledge organization but offer little support for the crucial final stage of insight generation.

‍

The human advantage: our competitive edge with knowledge

In spite of this era of rapid AI advancements, humans possess unique cognitive capabilities that give us an edge in knowledge and insight creation. Intuition, the result of experience and subconscious pattern recognition, allows experts to make rapid decisions based on tacit knowledge they've internalized over the years. As Nobel laureate Herbert Simon described, intuition occurs when an expert makes a quick decision but is explicitly "unable to describe in detail the reasoning or process that produced the answer." ⁸ And we’ve heard this at Liminary as well when speaking to dozens of professional researchers: insights come from the mysterious black box workings of the brain once it’s truly, deeply processed relevant information.

Humans excel at pattern recognition and generalization across contexts. We can see a few examples which then let us  infer a general principle or take a principle and apply it to a novel situation. Our emotional and social intelligence helps us ask the right questions (what problems are worth solving?) and know which solutions will matter in practice.

Human judgment outperforms purely algorithmic approaches in various domains. During the Flash Crash of 2010, traders with intuition intervened where algorithms failed. AI can mimic existing styles in creative fields, but human artists originate new expressions drawing on personal experiences and intentional creativity. In medical diagnostics, experienced clinicians still outperform AI in cases requiring contextual understanding, recognizing when symptoms don't fit standard patterns and require novel approaches to treatment.

The phrase "humanity is our competitive advantage" suggests that our uniquely human qualities drive true innovation in an automation age. We handle ambiguity and open ended problems better than algorithms designed for narrow goals. We can ask new questions, not just answer existing ones. And we possess ethical judgment shaped by living in the real world.

‍

Thinking about the future: balancing technology and human insight

The recent advances in LLMs have caused a lot of questioning about what the role of humans will be if they are replaced by or supplemented with AI. Our belief is that more and more, future systems must be designed explicitly to bridge the gap between information management and knowledge creation. Our tools will shift from a "storage and search" primary use case to "augmentation". Tools should enhance human intelligence, not replace it.

It’s not just about tooling but also about organizational culture that facilitates knowledge creation. Companies known for innovation have cultural practices that reward questioning, allow failure, and encourage cross pollination of ideas. Pixar's "Braintrust" meetings, 3 M's exploratory project time, and NASA's combination of formal knowledge bases with storytelling sessions all exemplify this balance.⁹

There’s also how an organization is managed; specifically, how does an organization know that innovation is happening and reward it? Measuring knowledge creation requires thoughtful metrics beyond counting documents in a repository. Innovation outcomes (new products or services), employee surveys about the climate for innovation, participation in knowledge activities, and social network analysis can all provide insight into an organization's knowledge creation health.¹⁰

‍

The path forward: stop managing information, start creating insights

But, back to tooling. While current tools excel at organizing information, they miss the critical step of helping knowledge workers transform that information into valuable insights. Liminary solves this problem.

Liminary isn't just another note-taking app or file storage repository; it's a synthesis companion that transforms how knowledge workers create insights from information. Unlike traditional tools that focus on organization and retrieval, Liminary actively supports the knowledge creation process by:

• Connecting all your knowledge in one place: Eliminating information silos and fragmentation that slow your work. When a project manager imports meeting notes, research findings, and strategy documents, Liminary automatically identifies relationships between these previously disconnected pieces.
• Discovering hidden relationships: Surfacing unexpected connections between concepts and ideas that might otherwise remain overlooked. For example, when a financial analyst reviews market trends, Liminary might reveal a connection to customer behavior patterns they hadn't considered, leading to a more comprehensive investment strategy.
• Freeing mental bandwidth for what matters: Offloading the mechanical aspects of knowledge management so you can focus on critical thinking. Rather than spending hours organizing notes and searching across tools, a consultant can spend that time developing deeper insights for clients.
• Providing contextual intelligence: Suggesting relevant pieces of information precisely when you need them in your workflow. When writing a recommendation, Liminary might surface supporting evidence from the research you conducted months ago that you'd forgotten was relevant.

What makes Liminary different is that we're not trying to replace human thinking—we're amplifying it. Knowledge workers face a sea of unstructured information that grows in volume every day. Liminary helps with the heavy lifting of capturing and connecting information, accelerating the process of synthesizing insights. It's a companion that frees up your mental bandwidth and energy to draw connections only a human can.

‍

Try it out for yourself

Experience a new way of working with information that bridges the gap between information management and insight creation. 

Join the Liminary beta waitlist today and be among the first to discover how our synthesis companion can revolutionize your approach to knowledge work.