Data like Lakes, Swamps or Graveyards – which will it be?
Very seldom do our data lakes get filled in a well-planned, organized and consistent manner. Gathering, storing and analyzing all sorts of data (for example: raw customer data, customer behavior data, location data, website clickstreams, images, social media trails, video and audio files plus regular structured data); requires intensive planning and must be implemented based on de facto standards. Otherwise Lakes of Data quickly become Data Swamps and eventually Data Graveyards.
Data Graveyards are evidence of wasted time, money and efforts.
How to avoid this data-death trap: Instead of haphazardly collecting data, it is more sensible to deliver information for decision making purposes in an orchestrated stream of bite-size chunks while demonstrating progress, spreading cost and gaining business users trust.
Recognize risk-increasing factors, such as appointing inexperience staff and/or forsaking proper training, not engaging with business people to collect requirement and designing technology driven solutions, to name a few.
Curb risks by demonstrating early success albeit in small steps. Increase the success rate through addressing each aspect of the process of building a data lake (aka data warehouse). Pre-determine all factors involved from collecting, consolidating, cleaning and structuring data right through to data analytics, guided by a blueprint of how it should be done. Be driven by the motto of “Pragmatism and Flexibility”.
To avoid Data Lakes becoming Swamps and Graveyards attend the ever popular course presented by Alicornio Africa Data Warehouse Concepts and Dimensional Modeling Techniques that teaches sound DW design principles no matter what technology is used.