Data Lake

Lake of DATA? What??

What makes a Data Lake turn into Data Swamp?

1. Lack of Metadata

Metadata is often understood as “data about data”. In a more specific manner, it can be described as data that provides information about other data. Think of it as hashtags we use in our twitter posts or LinkedIn posts. In a similar manner, when metadata are used in a data lake, it acts as a tagging system and thus making the data search easy. Data swamp lacks metadata which makes the data search very difficult and thus creating a problematic scenario.

2. Irrivalent Data

Often information gathering from various sources results in the collection of data with no goal. Thus, a data lake can easily turn into a data swamp if data that is being collected is done without setting any parameters about the kinds of data they want to gather and why. Thus turning a well-organized data lake into a data swamp flooded with data that may never be needed.

3. Lack of Data Governance

Data Governance is very important from an organization's point of view. It defines how to treat data, who should handle it, how long the company should retain it and where it should go, etc. Excellent data governance and high data quality go hand in hand. Data lake lacks in data governance which results in dumping all the information at one place thus putting a question mark on data quality. Also, a lack of governance can put an organization at risk when it comes to audits.

4. Lack of Data Cleaning Strategy

There is an immense lack of data cleaning strategy when it comes to data swamp. A data swamp might contain duplicate data as well as some erroneous data. As such, it's just like dumping everything in one place and never bothering about what is present there and what needs to be cleaned. Hence, it is necessary and very important for an organization to make and stick to plans for regularly cleaning their data.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store