Lawrence Corr takes a close look at Data Warehousing trends in 2005 and beyond

Susan Andre 2005

Data warehouse design expert Lawrence Corr returns to South Africa in November for the sixth year in succession to present a master class on the subject. Below he comments on recent trends in data warehousing. A question central to trends forecasting is: is data warehousing getting easier? Lawrence Corr provides his input on the matter with the answer being that it should be!

According to Corr, “The tumbling cost of storage and processing power has lowered the barriers that have prevented many companies from taking full advantage of data warehousing. However, over the last few years these hardware costs have dropped so quickly that the thinking of many warehousing experts has yet to catch up with this. It is worth noting that disk storage costs are several thousand times less today than when many of the ideas on data modelling and database design were cast in stone. I think it’s fair to say that many data warehouse practitioners haven’t yet fully embraced this sea of change. Many continue the ‘best practices’ which were once economically essential for a data warehouse to take flight but now only yield savings of a few hundreds of dollars on storage. Instead we should all be using new techniques that by yesterdays standards ‘waste’ disk space in favour of reduced development time and increased query and reporting capabilities. Having said that, it is a mistake to think we can buy our way past every problem by throwing hardware at it. We can’t raise user expectations and satisfy analytical requirements solely with brute force database hardware; we must still concentrate on designing simple, understandable models that appeal to the business users needs.

Specific data warehousing trends I have noticed in the last twelve months include web data slowly but surely making its way into the data warehouse. Clickstream is still taking a long time to become main stream but more and more companies, beyond the obvious dotcoms, are starting to find value in analysing web activity in the same way as their traditional business processes.

Within many organisations that have successfully adopted data warehousing, I see an increasing amount of data mart and data warehouse design and development being done upfront while new source systems are being developed. The positive aspect of this is that data warehousing teams can have valuable input into the data capture requirements. The drawback is that warehouse development is doubly challenging while the source is a moving target and little representative data is available yet. It requires good ETL prototyping skills.

14 – 18 November 2005 at the Unisys Auditorium in Sunninghill, Johannesburg, Lawrence Corr will present a Data Warehouse Design and Development Master Class with Joe Caserta, co-author of ‘The Data Warehouse ETL Toolkit’. For further information and registrations, contact Alicornio Africa on (011) 258 8739, email info@alicornio.co.za or visit www.alicornio.co.za.

Lawrence Corr is a leading data warehouse designer and highly experienced educator. As a former associate of Dr Ralph Kimball he has taught data warehousing classes for Kimball University in Europe and South Africa and is recognised as a leader in the field of dimensional modelling. His company DecisionOne Consulting specialises in data warehouse design review.

About Alicornio Africa

For the past six years, Alicornio Africa has serviced an impressive array of blue chip and other companies. In addition, they have continuously added to the knowledge pool of their industry by hosting leading international experts in the field in training and discussion forums. They have established themselves as leaders in the education of the expert and layman alike in the intricacies of the highly complex arena of business intelligence, information integration management, data warehousing, architecture, design and implementation. Alicornio Africa’s consulting services are both product and non-product specific. The company distributes and implements DOC1 for Customer Communication Management, and they are distributors and implementation specialists of the Sagent Solution for Information Integration.