Are You Using Your Dark Data Effectively?

Physicists believe that dark matter makes up 85% of the matter in the universe. That percentage is less than some experts estimate is the dark data in most companies. IBM believes that dark data represents 90% of data collected, while others think it is as low as 50%. But what is dark data?

Dark Data = Dark Matter

Gartner defines dark data as the information assets organizations collect, process, and store during regular business activities but generally fail to use for other purposes. Like dark matter, dark data often comprises most organizations’ universe of information assets. Thus, organizations often retain dark data for compliance purposes only. Storing and securing data typically incurs more expense – think of running a large data center, and more significant risk – think of the impact of a data breach of consumer information. Not leveraging the data through analytics incurs expense and risk, and can easily exceed the cost of data storage.

Dark data is distinct from typical transactional data, such as revenue tallies or total calls in a call center. Dark data, for our purposes, is data that artificial intelligence can unlock to provide information. Dark data falls into four areas:

• First up is Vision, which includes video and images of or from customers.
• Next is Audio from call centers, machinery acoustics, and setting recordings.
• Third is Industrial IoT, or the Internet of Things, which includes asset tracking, temperature tracking, and geo-location.
• Last is Natural Language Processing or NLP of Text that provides data for analysis from surveys, chats, email, and social media.

Transactional Data Offers Insights

Point of sale, inventory systems, CRM tools, and ERP are all dark data sources. Combining dark data with typical data sources expands the value of analytics by adding additional complexity and understanding. For example, traditional analytics may report the total sales by employee for a retail location. Combining that data with AI-driven computer vision, organizations can analyze the interpersonal behavior between the employee and customer and identify which behaviors are related to increased sales and higher customer satisfaction.

Over the next few years, forward-thinking organizations will implement these AI tools to unlock the value of dark data. This approach will help them advance along their path on the analytics maturity model, bringing more business value with each increase in sophistication in analytics use.