There’s no denying the pivotal role data plays in businesses today. Companies are racing to provide customers with the best services tailor-made for each user’s needs – and it’s all made possible through data-driven insights.

However, there is a hidden layer to the information that companies collect. Dark data is defined as the data that organizations have but are unaware of. Dark data is generally the by-product of day-to-day business activities and can be generated by various sources — transactional records of customer data, financial statements, applications, servers, network configurations, call records, sensor data, log data, audio/video snippets, email and chat messages, and more.

Such a treasure trove of data can either immensely benefit organizations or turn out to be a colossal liability, depending on whose hands the data falls into.

Impact of Dark Data

Most businesses have a hard time accessing relevant information when they need it. Given the amount of data being generated by organizations, it can be challenging to monitor what data is being created, where it is being stored, and how it can be leveraged. Reports have found that 7.5 sextillion gigabytes of data is created every day, of which 6.75 septillion megabytes of data becomes dark data. If ignored, the proliferation of dark data can quickly cost companies thousands of dollars in storage costs.

On top of storage costs, more data presents more risks and increases the attack surface. Unaccounted and uncategorized data is a prime target for attackers looking for ways to breach an organization. A hacker could unearth sensitive information like payment records, old documents, or previous employee information that might be exploited for financial gain. The average cost of a data breach in 2019 was $3.92 million, a 1.5 percent increase from 2018. Given the amount of money organizations already spend on storing tremendous amounts of dark data, the additional brunt of a data breach could be catastrophic.

Besides storage costs and security challenges, organizations might be unwittingly violating privacy and compliance regulations due to dark data archives. Most compliance regulations require businesses to maintain in-depth records of all kinds of data they possess and regularly audit the effectiveness of their data storage processes. However, if an organization is unaware of these data caches in the first place, achieving compliance becomes a challenge. In addition to potentially incurring millions of dollars in fines, organizations risk losing the trust of their customers.

The challenges dark data present are too huge to ignore. The key is to mitigate the liabilities dark data reveals while maximizing its potential to provide actionable intelligence that can ensure greater security for organizations.

Turning on the Light

As a first step, organizations should carefully sift through all the data they have collected. A good place to start is monitoring log files from app servers, firewall devices, file servers, network devices, and other IT resources. Log analysis can provide detailed information on an organization’s activities and can help with keeping a close eye on any suspicious events occurring within the network.

Another often overlooked source of dark data is the numerous devices employees bring into the organization. The rise of bring your own device (BYOD) environments has led to an exponential increase in new, unstructured data types. People use personal devices containing all kinds of applications at work for a number of activities. The information generated by and stored on mobile devices, social networking sites, and file sharing services can quickly lead to information governance challenges. This growth in data volumes and formats makes it difficult to discover, retrieve, and utilize generated information. This is why personal devices need to be managed and monitored closely for potential threats.

Creating an Action Plan

What organizations need to do is create a clear action plan on how to tackle this data. Review all data collection processes; collate the data collected over the years; securely store it in organized repositories; encrypt all sensitive information files, customer call records, geolocation data, and archived content contained in this dark data; and provide accountable accessibility to the data, depending on how it will be utilized.

Educating and training employees and top leadership about the importance of data governance and monitoring is also essential. Organizational silos and unresponsive leadership are some of the top causes for poor data management practices. A data-literate workforce is not only empowered with intelligent insights, but will have a leg up on the competition that neglects leveraging these powerful insights.

Conclusion

Data can quite literally make or break a business. While dark data certainly has its share of challenges, if organizations invest in the right tools and processes to extract value from this data, they can unlock immense business potential. From understanding customer spending behavior to analyzing website traffic patterns, dark data has considerable insight to offer.

But effective Data Management isn’t a one-time project – rather, it’s an ongoing practice that should be routinely applied across operations. Ultimately, prioritizing good Data Governance supported by senior leadership will provide long-term benefits to modern, data-driven organizations.

Disclosure: This article was originally published in Dataversity.