Poor data quality spans corporate silos, servers and applications, business units and people … all the way down to the manufacturing floor where an error of a single keystroke has the power to halt production. Nearly every large organization wrestles with data-quality problems, particularly companies that have multiple divisions and rich histories of growth and acquisition, and many of these companies are just beginning to understand the problem.
“People call us in because their business is failing in thousands of tiny different ways,” says Tom Kennedy, CEO of BackOffice Associates, a company that specializes in providing data migration and governance solutions. “[Potential clients] say, ‘this is happening and this is happening and this is happening,’ and they don’t know why. Most data-quality problems are hidden until a business process fails” (see “Deep Dark Secrets” below).
Deep Dark Secrets
If most companies don’t like to talk publicly about IT security, they positively loathe talking about poor data quality. “If you talk about the data, you’re talking about the integrity of the business,” says Philip Say, an SAP director of solution marketing for ERP and finance applications. “Companies are grappling with understanding what they have really created — what’s really running their business.”
Corporate governance and compliance regulations work primarily to shed light on the accuracy of financial performance or whether companies adhere to proper health and safety regulations governing controlled substances. In this age of compliance, companies have extreme disincentives to openly admit data-quality problems that could cast doubt on their ability to manufacture or distribute products or services, much less turn a profit. “Suddenly, [Sarbanes-Oxley] is like the new ISO standard,” says Mark Feldman, Virsa’s senior VP of strategy and business development. “Some institutional investors have been more concerned about companies passing a Sarbanes-Oxley audit than an ISO audit.” If you admit to any material weaknesses in your operational data, your share price goes down.
Kennedy says every company knows it has a data problem. “If we walk in and ask if they have a data problem, they’ll shut the door and say, ‘Yes, we have a really bad data problem, but we don’t know how big the problem is. We just know it’s causing problems all over the place.’ ”
But there is good news. Awareness of bad data is on the rise, and the technology and tools needed to address data integrity issues across the enterprise are now available. These lead to strategies and best practices for cleaning your data and ensuring that it’s correct moving forward.
In the past, the issue of poor data quality rested solely on the shoulders of the IT department. “IT people were getting beat up for something they couldn’t control, and they’d go to great lengths to try to fix the problem,” explains Bill Grasham, a principal in Deloitte Consulting’s Technology Integration practice. “Businesses would put in place what I call ‘human middleware’ — they’d have a large number of people extracting data into spreadsheets, parsing it, and trying to clean it up just so they could get meaningful analytics.” There are various ways that companies deal with data governance issues (see “The 4 Models of Data Governance” below).
The 4 Models of Data Governance
1. No data governance: No special definitions here; it’s simply the most widely used “model.” The primary problem is a consistent inability to understand why business processes fail.
2. Center of excellence: When an organization realizes that the rigors of inputting data into SAP are too complex for many people to handle, it specially trains a group to centrally handle all master-data requests. The primary problem is that the center of excellence becomes a bottleneck that is difficult to bypass.
3. Passive data governance: Employees enter their own data into SAP, but the organization uses tools and reporting processes to identify and catch data-related errors. The primary problem is that you can still enter bad data into SAP or create incomplete data objects.
4. Active data governance: All the data required for a business process is collected in a workflow environment — driven, for example, by email notifications and Web-based front ends — and when all the data is completely and accurately collected so that it’s business-ready for SAP, it’s automatically entered into SAP ready to run. The primary problems are that few organizations understand the scope of their own data problems well enough to implement active data-governance solutions across the enterprise.
(Source: BackOffice Associates)
As compliance regulations, such as the Sarbanes-Oxley Act, have entered the IT landscape, the ultimate responsibility for accurate data has been shifting from IT to business leaders. The reality, however, is that an organization’s IT group is often the most powerful catalyst for change, and savvy IT pros know it. The hard part is using technology in concert with business action.
Let’s explore four common data issues and, more importantly, some key strategies you can use to tackle data problems head-on.
1] Integrated applications: Organizations that use SAP solutions, particularly those that use SAP NetWeaver to connect various SAP components covering ERP, mySAP Product Lifecycle Management (mySAP PLM), mySAP Supply Chain Management (mySAP SCM), and mySAP Supplier Relationship Management (mySAP SRM), as well as custom-built or best-of-breed applications, are more likely to see the effects of bad data management.
“Unlike older systems, which were very loosely connected, SAP’s system is very tightly connected; it needs more data to work because it has more functionality than older systems,” says Bill Swanton, VP of research for AMR Research. “It’s really important that the data be consistent.” In a loosely connected system where applications and data are isolated in departmental silos or regions, a data error may not be used directly with other applications or business units and it may go unnoticed. Tightly connected systems, however, tend to use a single instance of data as a catalyst that affects many other applications, causing a ripple effect based on the single point of the problem.
A single purchase of material, Swanton notes, may require hundreds or even thousands of pieces of information about that material to make it business-ready. “There may be 10 to 20 people who have to supply different parts of it, and the purchase has to be done consistently across all the different applications or you just get garbage results,” Swanton explains. “The major problem is, how do you feed all the information into SAP so that it does what you expect it to?”
A lot of data problems are inadvertently hidden from your IT department simply because of human nature. At a certain point, just to get the job done, your business processes change to reflect manual workarounds whenever a business application doesn’t perform as you expect. For instance, if some unit of material in your warehouse has a waste factor of 0.01 (meaning that one out of every 100 units is defective), what happens if a worker accidentally makes the waste factor 0.10? In this case, your inventory-control application would operate as if 10 — rather than one — units out of every 100 were defective. This could cause you to automatically order more shipments of material than you can use. A relatively simple overstock problem in your warehouse means you pay a supplier for more units than you need, which affects your budget and financial reports, not to mention your business relationship with your supplier.
“Approximately 30 percent of the trouble tickets a help desk gets from an ERP system can be traced back to problems with master data,” Swanton notes. When you get into supply chain planning, that number rises to more than 80 percent.
2] Master data management: Whenever organizations agree to exchange a lot of data — perhaps in some sort of buyer/supplier relationship — they need to agree on the standards they will use to describe the information they pass and receive. For instance, the Global Data Synchronization Network (GDSN) is a network of interoperable data pools used with the GS1 Global Registry so that organizations can share standardized, synchronized master data. Different industries may have their own data and process standards, but the need to transform thousands of inputs into accurate data output is common across businesses.
Most organizations reach compliance with their trading-partner organizations through specific business-application silos — often third-party applications and services that have been used over the years. These non-ERP solutions have solved problems for specific business units, but they tend not to address data governance as a whole across the enterprise.
SAP NetWeaver Master Data Management (SAP NetWeaver MDM) gives you the ability to consolidate all your master data into a single version of accurate data. You can then use this single version for cross-system synchronization and distribution within the company and among your trading partners. Using SAP NetWeaver MDM helps to ensure that the data you use to run your business is accurate. When you have to attest to the quality of the data contained in your financial reports, something that organizations falling under the Sarbanes-Oxley Act must do, knowing that your data is accurate is important.
3] Harmonization: One of SAP NetWeaver MDM’s capabilities is to harmonize master data. Harmonization lets you take disparate data references to any item across geographies, business units, and IT systems — even if the part numbers and names have a dozen variations — and combine them into a single unit without needing to change or fix the data at each source.
This ability alone is powerful, but when it’s combined with SAP NetWeaver BI, it can help businesses understand how they are performing. Instead of having several different instances of “the truth,” businesses can finally begin to use business intelligence accurately to make decisions based on reality. What SAP NetWeaver MDM doesn’t have, however, is a comprehensive method for ensuring that data errors don’t arise in the first place.
4] Responsibility and management: When organizations look at master data management and data-governance issues, they can break them down into two major problem areas: responsibility and management. Most people don’t fully understand what data they are responsible for, and when they do, they often misunderstand their true role. “Different groups within a company say they own this data and take care of it in their spreadsheet, and you just can’t do that in an ERP setting,” Swanton explains. “We use the term, ‘data stewardship,’ instead of ‘data ownership.’ You are responsible for maintaining the data for the good of the whole company.”
The shift in responsibility for data quality is also a shift in organizational thinking, and it has to be driven from the top-down. Your executives and business managers have to understand the scope of the data-quality problem and get behind the development of a governance process. Once your power players are committed, your entire organization needs to know. Your workers need to understand that SAP isn’t broken and doesn’t act irrationally — it’s the bad or missing data elements that create havoc.
Simply alerting the company to be on the lookout for bad data isn’t enough, however. You have to create a data-architecture plan that can systematically find, remove, and, ideally, prevent data errors from occurring. Some key elements that you need to include in your plan are shown in “Strategies for Data Governance” below.
Strategies for Data Governance
|Create a charter
||The idea is to raise awareness in your organization about the importance of enterprise data management and how it’s critical to the business. In addition, don’t forget to state a clear goal.
|Establish a steering committee
||You need to build awareness and momentum by involving business managers and workers responsible for key data. You not only want their buy-in, but their participation in the solution, as well.
|Build your data management organization
||“A data management organization doesn’t have to be physical,” Grasham notes. “It can be virtual, and it can have shared responsibilities.” You need to identify people who will manage certain areas of data so that you can set standards and processes and a governance structure that is consistent across all boundaries.
|Identify trouble spots
||Your organization might struggle with product issues, financials, customer interactions, or any particular business unit or silo. Pick the key areas your data management organization needs to address first.
|Create and publish standards
||Basically, you need to describe the data in your business in consistent ways, and workers need to be able to understand and interpret your standards. One example Kennedy has seen again and again in global organizations is a classic problem with units of measure: If you always use 1,000 items at a time, your unit of measure for one unit would be “1.” So, if you need 8,000 units, you enter “8.” No problem … until one day a new worker who has a screaming headache needs 8,000 items and enters 8,000. Now, your business applications think you want 8,000,000 items. “People don’t have standards on these sorts of things, so they make poor choices,” Kennedy says, noting that you need to consider how people will try to interpret the data your applications need.
|Clean up your bad data
||Before you can truly move forward with a corporate governance plan, you have to get your existing data cleaned up. A data harmonization model, by the way, helps you to consistently identify multiple usages of the same data, consolidating it into a single usable instance, but it doesn’t correct all the problematic data that can undermine your business. For instance, a harmonization plan would not particularly help you if users can easily enter incorrect data. So, what you’re after here is a way to ensure that old problems don’t undermine your plan for data standardization.
|Create and use metrics
If you can’t measure the nature and scope of your problem, you have no way of knowing if you’re making progress. For example, one large client of BackOffice Associates thought it had a business-process problem in which SAP-based solutions kept messing up, but the company couldn’t pinpoint why. In four days BackOffice Associates found 50,000 materials that were not business-ready. “They were in shock. They hired us to put a process in place,” Kennedy says. “And every week they published metrics — these are materials that are manufactured that don’t have a bill of materials (BOM) and a routing; these are items that don’t have a sales view; these are items that have poor units of measure; these are items that have bad descriptions.” It took the company five months to clean up its data, but now that it’s done, if anyone finds an error in data, they know it’s one of only a very few errors — not one of 50,000.
|Identify the root cause
||Once you know how to find and measure the errors, at one level you can take steps to retrain the people who are causing them. On another level, you can address the business process in ways that reduce errors and have correct approval methods. Alternately, you’ll need to build or buy diagnostic tools that can cross-check data for accuracy and consistency.
|Establish patterns for correction
||Once you know how to find the root causes, you need to empower people with the ability to correct them. It’s an obvious step, but one that’s often overlooked and usually needs IT help. If you need to be SOX-compliant, for instance, you’ll need to have a documented process for data correction. If your financial compliance requirements are small, you still need to find the most efficient way for authorized users to correct data errors and omissions.
|Use business processes to prevent errors
Preventing errors from entering the ERP environment in the first place is the gold standard that few organizations have been able to meet. There are two basic areas you have to cover to make it possible: custom workflows and data-entry rules.
“What we typically see is people building a workflow process — routing applications and checking the data along the way,” Swanton explains. “The second thing is to create a lot of automation to look for problems, changes in business rules, and automatic checking for data quality. You need to be constantly cleaning this stuff up because if you don’t, it tends to degrade over time.”
Most organizations have some rudimentary governance applications — Web-based front ends that collect data from different workers for specific actions, for example, that are then integrated into SAP automatically or manually. Keep the good ones, make data-governance enhancement to others, and build new ones for your most troublesome business processes.
|Plan organizational rollouts
||AMR Research reports that large companies can easily spend $250,000 to $500,000 on service-intensive engagements to find, fix, and prevent data-governance problems; consequently, they recommend that you roll out your governance processes over time. If you’ve got a dozen people putting out fires, the cost is relatively minor, compared to employee compensation — the benefits of uninterrupted business, better agility, and smoother business-partner relationships usually make up the difference in initial data-governance costs. There’s a good chance you’ve rolled out specific solutions with your financial compliance-related controls for transactional data; use the best of those processes and principles for your master data.
Catalyst for Change
Bad data has been around for a long time, but most companies are just beginning to understand the domino effect that thousands of small problems bring to large organizations. Those who are tackling data governance head-on, however, are quickly learning how to prevent bad data from recurring. They use each new error they find and correct to establish better business processes and applications.
“I think we’re in an early stage of this whole [data governance] issue, but SAP NetWeaver provides us with a solid platform for addressing these problems as they come up,” notes SAP’s Say. “Those organizations that may not be looking at it right now — or those that don’t have a technique — will have an advantage if they can standardize on one platform such as SAP NetWeaver.”