Enterprise Information Integration

Susan Andre Survival Strategy Column

The Dawn of a New Era

Markets moving in a new direction have caught many software suppliers and software users off guard. Essentially, the demand for businesses to supply users with data in a variety of formats, is rapidly increasing. The primary challenge however, is finding a solution to access the data from multiple sources, integrate it on-the-fly, present and distribute it – all through the utilisation of easy to use, flexible and open solutions.

Recognised Needs – Key Drivers for Change

There are a number of key drivers fuelling renewed interest in information integration for more effective use and delivery, these include:

1. To obtain an overall view of business in order to facilitate more informed and faster decision-making.
2. To enable efficient data analysis, thereby increasing staff productivity and avoiding data collection from various places.
3. To speed up react time to respond to events, in highly competitive environments.
4. To empower the knowledge worker.
5. To ensure transparency for Corporate Governance.

To date, data integration has been largely problematic therefore making it difficult for management to obtain an overall view of their business. The attempt of various BI vendors to slap a glitzy front-end on has failed, and in doing so, highlighted the fact that unless the integration challenges are solved, the above issues will still not be addressed.

Emerging Trends

As was predicted by Gartner, CurrentAnalysis and other analysts, vendors in this market are currently driving towards meeting market demands, through consolidating business efforts. By taking progressive steps to meet the demand and grow the industry, vendors are drawing on either their spending power to make necessary acquisitions or alternatively strategic alliances are being forged.

The front-end delivery market, specifically, reporting tools and ROLAP/OLAP solutions can no longer survive without taking the step to marry with back-end (ETL) solutions. However given that it will take considerable time for most vendors to truly integrate these very different technologies, it is likely that customers will not gain any immediate benefit. Instead, it may even create a scenario whereby the customer is paying more for something that could potentially increase labour output.

In addition, for many vendors, aging software solutions are simply falling short in supporting the market demands, due to the fact that their software solutions were never originally designed and developed as a truly integrated solution. Furthermore, the software was probably developed using outdated techniques and technologies. Clearly, this makes it near impossible for vendors to take advantage of future advances, without re-developing their product suite. The down side for the customer is that their existing investment is not well protected.

On the Upside

Be that as it may, there is a positive outcome for those organisations that made the right choice from the start. Businesses that originally selected a technology that is truly integrated have since benefited a great deal from these solutions, and the good news is, that the advantages are still rolling in. Ultimately, a further increase in value can be expected, and the possibility of extending the lifetime of the ETL and BI solution is fast becoming a reality. For those utilising less than wholly integrated systems, this may just be the time to clear the deck and start afresh.

Extend the Use of Traditional BI and ETL Tools

On the front of extending the use of the traditional ETL solutions there are a number of significant advances in technologies, which prompts us to examine where else it can be applied. Take for example personalised CRM communications. It has been the Holy Grail of marketers for years. But few, if any, communications systems have been able to create and distribute personalised communications across all channels. One hindering factor is that it requires a great deal of effort (and cost) to collect the correct information. Once this has been achieved, a further, rather complex, process ensues to ensure efficient delivery to a complete customer communications system that works across a range of delivery channels. These channels include both paper and digital more specifically, mailed, printed materials, browser and email.

In essence, if effectively applied and provided, and the ETL / BI technology can cope with these demands, BI should enable more personal customer communications across multiple channels to any customer, anywhere, at any time.

When looking beyond ordinary information delivery mechanisms, imagine how the ETL and BI solution can enhance document composition tools. With the utilisation of strong segmentation techniques to incorporate CRM and one-to-one marketing components, interactive desktop management capabilities, and archiving and retrieval functions the result speaks for itself: a service offering a complete customer communications system that works across all delivery channels.

To fulfil these needs one requires integrated information. Again the existing ETL and BI environment must be able to do advanced Enterprise Information Integration (EII). However, the required data is typically scattered across applications, stored in different types of databases and in a variety of different formats. The growing backlog of requests for specialised data sets and ad-hoc data requests calls for efficient and effective information integration.

The Solution

EII uses data integration techniques to source from heterogeneous applications and data structures, and accordingly performs in-line transformation and analysis. EII combines and conforms data to the needs of an end-user or business process and delivers it to an analytic tool or application, for display and manipulation.

Business Analysts that implement and utilise an EII tool to discover what is needed to satisfy their businesses rapidly changing information needs, can gain considerable advantages, and is a further example of how this technology can be effectively put to use.

Effective Tools for Business Analysts

In terms of data warehousing projects, the use of high level EII technology can be instrumental in avoiding costly mistakes and keeping the project on track. For the Business Analyst, this is an ideal tool, and if the technology is implemented in the early stages of the project, an attempt can be made to understand how data will be used, prior to designing and loading it into a data warehouse.

The consequence of implementing a physically well-designed dimensional data warehouse is that it ultimately fails to connect with the organisations' strategic objectives and user requirements. Should more emphasis be placed on the definition of high-level business measures, the unfortunate and frequent scenario where priority is given to the physical design of data storage, during the analysis and design phases, can be avoided. In the event that the physical design supercedes end-user requirements, attempts to use technologies such as reporting tools or analytical applications are invariably problematic and cumbersome.

The value of early implementation can be demonstrated, and the above mentioned issues aptly addressed by adopting the EII technology at concept origination and project inception stages. The considerable advantages of this include:

1. Increase in productivity through time and output management.
2. Use of the technology as a communication tool.
3. Enabling the Business Analyst and business user to jointly define the requirements.
4. Identifying the data required to connect with strategic objectives.

Approaching Information Integration and the delivering of information from a unique angle, gives one scope to enhance the existing BI infrastructure, in addition to integrating it with other information delivery applications. Utilising EII can also improve the quality of data warehouse implementations. When EII is used to solve day-to-day information problems, it reveals opportunities for strategic and permanent additions to an organisation's analytic data model.

Rather than adding unnecessary data, Business Analysts, BI Project Managers and business stakeholders can systematically assess the value of new information created through EII. If the selected information proves to have lasting value, it can then be added to the warehouse, ideally using the same software that is used for EII.

Extending Solutions into New Market Spaces

Only if the original software solution is based on robust product architecture, will it be possible for the vendor to adapt and adjust to industry demands and to take advantage of future technologies without compromising customers existing investments. Another big plus is, if the solution is able to facilitate changing techniques, businesses will ultimately be free to make use of their BI technologies beyond the limiting boundaries. Successful integration of these techniques will eventually separate products, thereby clearly differentiating high-level and valuable tools from the clutter.

Out with the Old, In with the New

The days of the traditional Extract Transform and Load (ETL) and Business Intelligence (BI) solutions are numbered, since businesses are rightfully expecting to gain a lot more from these technologies. Software is often very powerful but frequently grossly under utilised. By embracing change and development, a new era in EII hails the time to take full advantage of these immensely valuable software solutions, to ensure the greatest return on investment. Not only will this new wave in technology facilitate the implementation of proper organisational support, but EII will achieve an extension in the software solution capacity. Whilst supplying flexible and diverse data is the immediate objective, greater productivity and smart business over an extended period of time, through the effective use of tools and technologies, is the ultimate goal