Data warehousing market poised for record growth

Susan Andre1999

The data warehousing market is set for record growth over the next 12 months. This is the prediction from Ralph Kimball, widely considered to be the "father" of data warehousing. "I have never seen greater demand," says Kimball. "Conference companies are enjoying their highest forward booking in years for data warehousing conferences; and data warehousing vendors are seeing very strong forward purchase intent from customers."

Kimball himself had more than 5 000 people from 4 000 companies around the world book for his recent paid-for Webcast.

This growth after a year of worldwide slowdown in business intelligence deployment will be driven by parallel events, says Kimball:

The Year 2000 issue is coming to an end, freeing up huge amounts of resources and discretionary funds in budgets which have been tied upresolving the Y2K crisis. Consequent pent-up demand for business intelligence will need to be addressed and satisfied.

Increasing awareness by organisations that high-value, new-style applications such as electronic commerce and customer relationship management (CRM) will simply not yield their full value unless the data issue has been addressed. A need to analyse the vast amounts of data being generated by Internet-based business.

"We are seeing that companies are finding their need to apply e-commerce or CRM is hampered by the lack of data integration and consolidation. Multiple and disparate data sources inhibit the continuity of customer experience that is essential for CRM to work; and e-commerce generates such a flood of consumer data that we simply have to tap into it. Not to do so is to lose a major opportunity to profile customers based on their recorded actions on the Internet."

Kimball refers to the example of Microsoft, which is taking a billion hits a day on its sites globally. "This volume of hits exceeds the number of phone calls made on every telco in the world," notes Kimball. "Cumulatively they represent consumer behaviour and they need to be tracked. But it is beyond the capability of current business intelligence tools to track this behaviour, which means suppliers of such software are having to develop a new wave of products to address this requirement."

The impact of the Internet has brought about a new phenomenon that Kimball terms the "data webhouse". This is, on the one hand, the evolution of the data warehouse to address the new demands of the Web; and on the other, the deployment of the data warehouse's capabilities over the Web.

The lack of data is inhibiting corporate decision-making, adds Kimball. "Without one set of consistent, homogeneous data we cannot create one unified customer experience. Yet the majority of medium-sized to large organisations are wrestling with multiple customer views from legacy and ERP systems. They need to bring this under control, and the data warehouse represents a solution."

Kimball was the founder of Red Brick Systems and is currently involved with full lifecycle data mart company, Sagent. He was in South Africa to present a data warehousing masterclass in conjunction with data warehousing vendor Sagent SA. Eighty-five people from South African corporates attended the masterclass.