Authored by:Chintan
What is Master Data? Most of the IT organizations have their data dispersed and stored at various locations. But the data is shared and used by several of the applications that make up a data warehouse. For example, an ERP system has data from Customer Master, Item Master and Account Master. The master data is one of the key assets of any company.
Need of Master Data Management: Since master data is used by multiple applications, an error in master data can cause errors in all the applications that use it. For example, all important documents, bills, checks are sent to the wrong person because of an incorrect address in Customer Master. Similarly, an incorrect price in Item Master can be a marketing disaster for an organization. An incorrect account number in an Account Master can lead to huge fines. To overcome such hazards, maintaining high quality and consistent set of master data is necessary for all organizations.
What is Master Data Management (MDM)? The technology, tools and processes required to create and maintain consistent and accurate lists of master data is known as Master Data Management. MDM is a continuous, iterative process. There are many factors considered for MDM which involves requirements, priorities, resource availability, time frame and the size of the problem.
MDM Life Cycle: MDM project involves many stages as follows:
1- Identify sources of master data.
2- Identify the producers and consumers of the master data.
3- Collect and analyze metadata about master data.
4- Appoint data stewards.
5- Develop the master-data model.
6- Choose a toolset.
7- Finalize and receive approval for the process.
8- Design and implement the process.
9- Test the master data.
10- Modify the producing and consuming systems.
11- Implement the maintenance processes.
Conclusion: In recent times, creating and maintaining accurate and complete master data has become a business imperative. Both large and small businesses must develop data maintenance and governance processes and procedures, to obtain and maintain accurate master data.
Tuesday, June 16, 2009
Wednesday, June 10, 2009
Data Quality
In 2003 the Data Warehousing Institute calculated that bad data quality leads to a whopping loss of, approx $600 billion annually.
Data Quality Improvement is the processes and technologies involved in ensuring the conformance of data values to business requirements and acceptance criteria.
The reasons that adversely affect the data quality are:
Legacy Systems and data: Legacy Systems may/may not have validations in built into them. Legacy Systems tend to have redundant data, composite keys and referential integrity issues.
Application Evolution : Applications evolve over time and the data entry operations, client and server side validations are often overlooked resulting in bad data quality.
System Work-Around: More often than not, immediate results and often temporary measures are deployed to meet time deadlines or technology limitations.
Time Decay: The best of the systems cannot stand the test of the time. What better example than Y2K bug. Data quality deteriorates with time.
Lack of common data standards: Companies do not always invest time and resources into creating best practices, standards and checklists. Simple tasks such as having universal naming conventions can improve quality of the data.
Data Entry issues: Data entry issues are top1 reason for adversely affecting the quality of the data. If data entry is performed by customers or web based users, it is most likely that junk and misplaced information will be gathered. Even internal data entry operations are compromised because of the ‘remarks’ or ‘comments’ sections.
So how can this data be cleared up?
DIY: Do It Yourself by looking into databases, forms, applications etc. Of course, this is not the best choice. But is a beginner’s step that can lead you to a roadmap for data cleansing.
Invest in Data Cleansing Tools: Outsource to who can do it best. Yes, now we are talking business. Invest in identifying the suitable tools in the market. Here are some:
· Informatica Power Center
· Trillium Software
· Business Objects Data Integrator
· Data Flux
So, going forward how do you prevent rather than cure? Here are some ideas gathered by us.
Data Profiling – analyze the date for correctness, completeness, uniqueness, consistency, and reasonability. This must be done in the order of column profiling, dependency profiling, and then redundancy profiling.
Data Cleansing – Detect and correct corrupt or inaccurate records from a record set, table, or database. The common methods are parsing, data transformation, duplicate elimination, and many statistical methods.
Data Defect Prevention –Set up a data governance group that will take control and responsibility of the various databases and enforce/introduce data quality rules. They will conduct regular audits and data cleansing programs. They also have to take charge of the training of data entry and other personnel.
Data Quality: Authored by Vivek and Devi. Cleansed by Vai :-)
Data Quality Improvement is the processes and technologies involved in ensuring the conformance of data values to business requirements and acceptance criteria.
The reasons that adversely affect the data quality are:
Legacy Systems and data: Legacy Systems may/may not have validations in built into them. Legacy Systems tend to have redundant data, composite keys and referential integrity issues.
Application Evolution : Applications evolve over time and the data entry operations, client and server side validations are often overlooked resulting in bad data quality.
System Work-Around: More often than not, immediate results and often temporary measures are deployed to meet time deadlines or technology limitations.
Time Decay: The best of the systems cannot stand the test of the time. What better example than Y2K bug. Data quality deteriorates with time.
Lack of common data standards: Companies do not always invest time and resources into creating best practices, standards and checklists. Simple tasks such as having universal naming conventions can improve quality of the data.
Data Entry issues: Data entry issues are top1 reason for adversely affecting the quality of the data. If data entry is performed by customers or web based users, it is most likely that junk and misplaced information will be gathered. Even internal data entry operations are compromised because of the ‘remarks’ or ‘comments’ sections.
So how can this data be cleared up?
DIY: Do It Yourself by looking into databases, forms, applications etc. Of course, this is not the best choice. But is a beginner’s step that can lead you to a roadmap for data cleansing.
Invest in Data Cleansing Tools: Outsource to who can do it best. Yes, now we are talking business. Invest in identifying the suitable tools in the market. Here are some:
· Informatica Power Center
· Trillium Software
· Business Objects Data Integrator
· Data Flux
So, going forward how do you prevent rather than cure? Here are some ideas gathered by us.
Data Profiling – analyze the date for correctness, completeness, uniqueness, consistency, and reasonability. This must be done in the order of column profiling, dependency profiling, and then redundancy profiling.
Data Cleansing – Detect and correct corrupt or inaccurate records from a record set, table, or database. The common methods are parsing, data transformation, duplicate elimination, and many statistical methods.
Data Defect Prevention –Set up a data governance group that will take control and responsibility of the various databases and enforce/introduce data quality rules. They will conduct regular audits and data cleansing programs. They also have to take charge of the training of data entry and other personnel.
Data Quality: Authored by Vivek and Devi. Cleansed by Vai :-)
Tuesday, June 2, 2009
What's new in Cognos8.4 GO! Family
Cognos on the GO!
Go! Mobile
The newer Go! Mobile version gained location intelligence and query capabilities as well as the ability to deliver prompted, scheduled and bursted reports. The product takes advantage of GPS information with Blackberry, Symbian and similar mobile devices.
Go! Dashboards
This is an Adobe Flash-based dashboarding tool that lets users visualize information in drag-and-drop fashion. This new feature gives dashboards a slick appearance and it supports dynamic interaction. So, visualizations change as you move sliders and drill down on data.
Go! Search
The earlier version of this product was limited to searching preexisting Cognos reports. Version 8.4 delivers original query results, as well as cubes and unstructured (Word and PDF) documents and reports.
Go! Mobile
The newer Go! Mobile version gained location intelligence and query capabilities as well as the ability to deliver prompted, scheduled and bursted reports. The product takes advantage of GPS information with Blackberry, Symbian and similar mobile devices.
Go! Dashboards
This is an Adobe Flash-based dashboarding tool that lets users visualize information in drag-and-drop fashion. This new feature gives dashboards a slick appearance and it supports dynamic interaction. So, visualizations change as you move sliders and drill down on data.
Go! Search
The earlier version of this product was limited to searching preexisting Cognos reports. Version 8.4 delivers original query results, as well as cubes and unstructured (Word and PDF) documents and reports.
Monday, June 1, 2009
What's new in Cognos 8.4 Query Studio?
In QS 8.3, users could only filter on the fields in the report body.
-QS 8.4 allows filtering on any field in the package(works only with relational packages).
- QS 8.4 allows filtering using wildcards
That's for today. More tomorrow.
-QS 8.4 allows filtering on any field in the package(works only with relational packages).
- QS 8.4 allows filtering using wildcards
That's for today. More tomorrow.
Subscribe to:
Posts (Atom)
