Introduction Introduction To To Enterprise Enterprise Data Data Systems Systems Introduction to Data Warehousing Introduction Introduction To To Enterprise Enterprise Data Data Systems Systems Dr Hamid Nemati ISOM 602 Ten years ago I could have told you how Doritos were selling west of the Mississippi Today not only can I tell you how well Doritos sell west of the Mississippi I can tell you how well they are selling in California in Orange County in the town of Irvine in the local main street s supermarket in the special promotion at the end of aisle four on Thursdays D Wayne Calloway CEO of Pepsico The Age of the Data Glut Introduction Introduction To To Enterprise Enterprise Data Data Systems Systems Today the amount of data available to most organizations is staggering and is continuously growing The volume of information that is available to corporations is rapidly increasing and frequently overwhelming A Bell Labs report has indicated that the amount of data doubles every 5 years and that only about 5 of it is used Reasons for Explosion in Data Introduction Introduction To To Enterprise Enterprise Data Data Systems Systems Increases in information processing capabilities wide spread and routine use of point of sale data gathering methods such as bar codes and the use of credit cards possible the computerization of many business transactions Problems With Large Volume of Data Reasons for Explosion in Data Unfortunately because of large volume of data Introduction Introduction To To Enterprise Enterprise Data Data Systems Systems Network Bandwidth is increasing at the same time the price of high bandwidth is decreasing Technological advances in data storage devices have made it possible to store large amounts of data efficiently and inexpensively New and efficient and more powerful database management and processing systems have allowed for efficient access to large amount of data Introduction Introduction To To Enterprise Enterprise Data Data Systems Systems traditional methods of data process analysis and data inquiry may no longer be adequate Understanding the patterns in data overwhelms human capabilities We seldom know which questions should be asked or which questions can be asked The need for data analysis tools that intelligently and automatically support the decision maker in accessing processing and sifting through mountains of data is apparent and immediate 1 Forces that are Driving the DW Sam Never Imagine This The New Business Environment Introduction Introduction To To Enterprise Enterprise Data Data Systems Systems Global competition Shorter and shorter product windows Higher and higher quality Smarter and smarter products Smarter Teams and Smarter Workers Fundamental changes in organizational structures Introduction Introduction To To Enterprise Enterprise Data Data Systems Systems The New Technological Environment New classes of computers New classes of communications New classes of technology image sound video multimedia New classes of software Much more complex technical environment Cooperative Processing Client Server Distributed Data Bases LANs WANs etc How Big Is It Really Mid 97 Introduction Introduction To To Enterprise Enterprise Data Data Systems Systems Wal Mart s initial data warehouse was 700 gigabyte It has grown to 24 terabytes as new types of data such as inventory forecast demographic modular markdowns returns and market basket added Now the world s largest commercial data warehouse The largest table is 300 gigabytes with 5 billion rows In all it had 19 7 billion rows in Teradata warehouse The data warehouse is stocked with 65 weeks of data kept by item by store by day etc from 80 000 items in 3 000 store locations The system has handled as many as 48 000 SQL commands in one day and can handle 4 500 users It handles 25 000 queries a week with 75 000 lookup queries What is all the fuss Introduction Introduction To To Enterprise Enterprise Data Data Systems Systems So What is all the excitement about Introduction Introduction To To Enterprise Enterprise Data Data Systems Systems The system allows managers buyers and vendors query information to analyze sales trends by item and by store to make informed decisions on replenishment look at customer buying trends analyze seasonal buying trends make mark down decisions react to merchandise volume and movement at any time An hour and half after each store closes Wal Mart knows where it stands operationally and financially Wal Mart has improved its in stock percentage by 10 points to 96 percent the highest rates in the industry The system is the basis for all inventory purchasing marketing and staffing decisions Sam Walton opened his first Wal Mart store in Rogers Arkansas in 1962 Today this retailer has annual revenues of nearly 110 billion 3 000 stores and 600 000 employees In a typical business day Wal Mart department store generates over 200 million transaction updates At the end of the business day a high bandwidth telecommunication network is used to transfer all the sales data from each store a centralized data repository Data Warehouse Runs on a 32 node NCR WorldMark 5100M massively parallel server and a 365 AMP NCR 3600 server running the NCR Teradata database using a network of 5000 Unix servers Wal Mart s buyers merchandisers logistics and forecasting associates as well as 3 500 of Wal Mart s vendor partners have direct access to the data warehouse With more than 30 applications running on the system users can ask virtually any question The data warehouse has enabled Wal Mart to better focus on replenishment at the store level and provide precise information to its buyers and suppliers Enterprise Information Needs Introduction Introduction To To Enterprise Enterprise Data Data Systems Systems There are two fundamentally different types of information systems needs exists in all organizations They are Operational systems Informational systems Therefore why not create specific DBs whose role is to make large scale end user access easy to the operational DBs i e a Data Warehouse 2 Separation of Operational and Informational Concerns Introduction Introduction To To Enterprise Enterprise Data Data Systems Systems Operational Systems Response Time Reliability Security Recoverability The Data Warehouse Introduction Introduction To To Enterprise Enterprise Data Data Systems Systems Informational Systems Flexibility Performance Ease of Navigation Large numbers of different views Manage Huge Amounts