Businesses can gain greater value from their BI investment by improving the way in which data flows to the BI system are managed. Many problems result when the ETL process is handled by a patchwork of scripting, custom coding, and various built-in schedulers that are part of existing ETL solutions, because these systems do not provide end-to-end execution, monitoring, and control of the ETL process.
TMWhitepaper | Tidal Software
Automating Data Flows to Your
Business Intelligence SystemBy Wayne Greene
Wayne Greene Business Intelligence (BI) systems increasingly provide the data that drives both strategic and tactical VP Product Management decisions for an enterprise. Many businesses have already invested heavily to aggregate data from diverse Tidal Software systems and applications in order to create a whole-enterprise view to fully reflect the daily state of the Wayne Greene leads the product business, as well as support more effective, informed decisions. As businesses use BI systems more heavily, the management team at Tidal and challenge for IT is to keep the underlying databases accurate so decisions are made on trustworthy data.drives the product strategy across the Businesses need many types of complex and dynamic data integration to remain competitive and improve company. Prior to working at Tidal, profitability. One example is the compilation of whole customer profiles that are kept updated in near real-he was the Director of Competitive time. This consolidated, finger-tip information about the customers and their activities is created by pulling and Market Intelligence at the Wily data from a number of systems to create a view that is essential for targeting sales in real-time; for example, it Technology Division of Computer can enable up-sell campaigns during customer service calls. Another use of the customer profile is as an input Associates. Prior to CA, Wayne was at to fraud detection systems. A second example of a data integration need is the consolidation of data across Hewlett Packard for 18 years in a variety multiple portfolio accounting systems to support risk mitigation and portfolio analysis. Further examples of different roles, spending the last include financial modeling for cost allocation of services; trades and transfers reconciliation checking; and 6 years at HP in the Corporate IT and compliance and audit reporting. As the operational warehouse matures, new applications for the data typically Outsourcing Services division where emerge. These are based on the business's experience with the information that is available.he was responsible for the enterprise A BI approach is just one of the solutions to enterprise-wide data integration. To many, its benefits are ready management toolset. He led a team accessibility and scalability. However, it does have its challenges. The information retrieved from a BI system of approximately 120 engineers, was is only as trustworthy as the data put into it. Generally bad information is not the result of bad source data. responsible for system, network, and More often, bad information is the result of pulling out-of-sync data into the system due to problems in the application monitoring, help desk data processing flow. This process flow, referred to as Extract-Transform-Load (ETL), can not only result in ticketing and case exchange solutions, bad or inaccurate information, but can also suffer from other problems in the areas of information availability, enterprise application integration, process auditability, and agility in responding to changing business needs. While bad information can lead to job scheduling, datawarehousing bad decisions, which can negatively affect the course of the business, the full scope of problems leads to high and reporting, as well as end user costs for IT, potential issues in governance and compliance, and inability to provide a whole-enterprise view support portals. Wayne has a PhD from that is kept current with the business needs. University of California at Berkeley, and a Bachelor of Science from M.I.T. Bad data, or data that is not current, has another, broader impact in leading to distrust of the system. When the BI system cannot be fully relied on, its uses rapidly become limited. This requires complex and time-consuming adjustments for errors and re-running of the downstream analysis. Ultimately this leads to the business losing faith in the data. Perhaps the biggest hit is that strategic business planning is no longer strategic or too late to capitalize on changing market opportunities. A well run BI system reliably presents current data from all of the available, different data types and becomes an integral part of the business applications. The longer term result is for the business to ... [download for more]
