Data Warehousing !!!!
This no more is the thing with only IT companies, the Mechanical companies, The electrical companies, The Manufacturing Industry, The Hospitals, The Banks, The Telecom Companies all are using the Data Warehousing, since all have understood the power of storing the data in sophisticated manner and using it for Reporting and Analytical purposes.
The DATAwarehousing, as the name suggests is a game played with data. The data for today, yesterday, day before yesterday, a week old data, an year old data stretching may be up to the ages of your grand grand father.
This no more is the thing with only IT companies, the Mechanical companies, The electrical companies, The Manufacturing Industry, The Hospitals, The Banks, The Telecom Companies all are using the Data Warehousing, since all have understood the power of storing the data in sophisticated manner and using it for Reporting and Analytical purposes.
The DATAwarehousing, as the name suggests is a game played with data. The data for today, yesterday, day before yesterday, a week old data, an year old data stretching may be up to the ages of your grand grand father.
The data warehouse is mainly the term
used in the companies to manage the data. The larger the company, the bigger
the data, the bigger requirement to manage the data.
The concept of a data warehouse is to
have a centralized database that is used to capture information from different
parts of the business process. The definition of a data warehouse can be
determined by the collection of data and how it is used by the company and
individuals that it supports. Data warehousing is the method used by businesses
where they can create and maintain information for a company wide view of the
data.
The History and Evolution of Data Warehousing
Many people feel that data warehousing is only copying data from one place to another, They feel “That doesn’t make any sense! Why waste time copying and moving data, and storing it in a different database? Why not just get it directly from its
original location when someone needs it?” This is totally incorrect. Data Warehouses are very much evolved as compared to Databases.
Data Warehouses do far more then just storing the data.
Going to 1970-80's years back, the computing world was dominated by
the mainframe in those days. Real data-processing applications, the ones run on
the corporate mainframe, almost always had a complicated set of files or
early-generation databases (not the table-oriented relational databases most applications use today) in which they stored data.
Although the applications did a fairly good job of performing routine data processing functions, data created as a result of these functions (such as
information about customers, the products they ordered, and how much
money they spent) was locked away in the depths of the files and databases. The way data was stored made it more complex to solve the purpose.
It was almost impossible, for example, to see how retail stores in one
region were doing against stores in the other region, against their competitors, or even against their own performance in some earlier period.
Between 1976 and 1979, the concept
grew out of research, driven from discussions with Citibank’s advanced
technology group, called TARADATA.The name Teradata was chosen
to symbolize the ability to manage terabytes (trillions of bytes) of data.
technology group, called TARADATA.The name Teradata was chosen
to symbolize the ability to manage terabytes (trillions of bytes) of data.
Early days of Data Warehousing - 1980's
As the era began, the computers were
the only name making there presence everywhere. The organizantion started to
have computers every where in each department. How could an organization hope
to compete if its data was scattered all over the place on different computer
systems that weren’t even all under the control of the centralized
data processing department? (Never mind that even when the data was all stored
on mainframes, it was still isolated in different files and databases, so it
was just as inaccessible.)
A special software then came into existence which made the life simple for the user and everyone, using the data
or analyzing the data. This new type of software, called a distributed database management system (distributed DBMS, or DDBMS), would magically pull the requested data from databases across the
organization, bring all the data back to the same place, and then consolidate
it, sort it, and do whatever else was necessary to answer the user’s question.
organization, bring all the data back to the same place, and then consolidate
it, sort it, and do whatever else was necessary to answer the user’s question.
Although this was thing all were
expecting, but the chocolate is always better if its sweeter, the sweetest the
perfect. The chocolate being RDBMS (relational database management system) for decision support — the world’s
first.
Why Data Warehouse over Normal Databases !!!
The need has been established for a
company wide view of data in operational systems. Date warehouses are designed
to help management and businesses analyze data and this helps to fill the need
for subject- oriented concepts. Integration is closely related to subject
orientation. Data warehouses must put data from disparate sources into a
consistent format. They must resolve such problems as naming conflicts and
inconsistencies among units of measure. When this concept is achieved, the data
warehouse is considered to be integrated.
The form of the stored data has
nothing to do with whether something is a data warehouse. A data warehouse can be normalized
or de-normalized. It can be a relational database,
multidimensional database, flat file, hierarchical database, object database,
etc. Data warehouse
data often gets changed. Also, data warehouses will most often be directed to a
specific action or entity.
Data warehouse success cannot be
guaranteed for each project. The techniques involved can become quite
complicated and erroneous data can also cause errors and failure. When
management support is strong, resources committed for business values, and an
enterprise vision is established, the end results may turn out to be more
helpful for the organization or business. The main factors that create needs
for data warehousing for most businesses today are requirements for the
companywide view of quality information and departments separating
informational from operational systems for improved performance for managing
data.
With this basic history of Data Warehouse, lets try to gather more information about the Giant - The Data Warehouse.
There are numerous pages, blogs, websites these days which talk about Datawarehousing. No doubt few of them have really good information.
I have gone through many of them – Its great to see the enormous knowledge people gather in there life time and the more better work they do is share that knowledge with others.
When I shuffle through the websites, blogs I sometimes feel annoyed that none location provide everything at its best. Definition of Datawarehousing is written in great manner in one blog, OLAP and OLTP is good at other ........!!!
Other bad things – Who the hell is going to tell me how do I relate all these to world around me so that I can understand its importance.
Now try reading few other pages on internet – and you will understand what I am trying to say in my last sentence above.
After working of quite a few Data Warehousing projects and gathering lots of information from the pioneers article - here are few of my real life examples for all the topics which we say “DATAWAREHOUSING CONCEPTS” !!!
I will try to provide some real life scenario for each concept which will be useful in helping you gather real life knowledge of Datawarehousing.
Now try reading few other pages on internet – and you will understand what I am trying to say in my last sentence above.
After working of quite a few Data Warehousing projects and gathering lots of information from the pioneers article - here are few of my real life examples for all the topics which we say “DATAWAREHOUSING CONCEPTS” !!!
I will try to provide some real life scenario for each concept which will be useful in helping you gather real life knowledge of Datawarehousing.
