GRC
HR
SCM
CRM
BI


Article

 

Archiving in the Age of Big Data

Move from a Siloed Approach to a Holistic Strategy with the SAP Information Lifecycle Management Database Storage Option

by Helmut Stefani | SAPinsider, Volume 15, Issue 4

October 1, 2014

Maintaining and updating data in your database can be a huge resource drain, especially considering the rapid growth of data over the last few years. Read this article to see how SAP is helping customers simplify their landscapes while addressing their archiving requirements through SAP Information Lifecycle Management, providing organizations with a less complicated, more efficient alternative for archiving SAP ERP data.

 

With enterprise applications capturing wide swaths of information about customers and business activities, organizations are accumulating massive piles of big data that need to be managed. An important part of this growing volume of information is historical data. Archived data from business-complete processes — such as manufacturing, HR, and financials data from SAP ERP — has intrinsic value that can benefit current or future business decisions by helping to reveal trends, patterns, and correlations. Historical data also often needs to be retained for many years to comply with legal requirements, such as laws governing tax audits or product liability, or FDA regulations in the pharmaceutical industry. 
 
Keeping historical data available in the main application database is a resource drain, however, so organizations generally move this data to a secondary storage location. While solutions such as SAP Information Lifecycle Management provide comprehensive functionality for data management tasks such as archiving, retention management, and system decommissioning, adding various storage locations to support these tasks can lead to increased — and costly — landscape complexity.
 
To help customers simplify their landscapes while addressing their archiving requirements, SAP offers a new database storage option within SAP Information Lifecycle Management that provides organizations with a less complicated, more efficient alternative for archiving SAP ERP data.1

Traditional Approaches to Storing Archived Data

In SAP environments, archived operational data, such as transactional data or master data from an SAP ERP system, is traditionally stored in a third-party storage system. This storage system needs to be certified for use with the SAP ArchiveLink interface to enable integration with the standard SAP archiving system, or, to work with SAP Information Lifecycle Management for data retention and system decommissioning, it needs to be a write once, read many (WORM)-like store that supports the storage of structured information using the WebDAV standard.

Organizations that are also using SAP Business Warehouse (SAP BW) have the option to move less frequently accessed (“cold” or “aged”) analytical data from their SAP BW systems to the near-line storage area of SAP IQ — a highly optimized data analysis server that enables SQL-based reporting and improved performance during data load and access (see the sidebar “What Is SAP IQ?”) — via a near-line storage interface. As part of SAP BW, SAP delivers a native ABAP-based implementation of the SAP BW near-line storage interface, but you can also leverage certified partner implementations, which are developed using a near-line software development kit (SDK) included in SAP BW. The near-line storage option is available for SAP BW systems regardless of whether they are based on SAP HANA or a traditional RDBMS database.

What Is SAP IQ?

SAP IQ (formerly known as SAP Sybase IQ) is a high-performance analytical platform designed for quickly loading and accessing large amounts of data. Its columnar table design enables high data compression rates (up to 90% compared to a conventional RDBMS database), improved search capabilities, and increased performance for archiving and reporting.
 
SAP IQ enables the secure storage of data in its read-only table space, for which no special analytics schema is required. SAP IQ supports standard connectivity, such as ODBS and JDBS interfaces, and standard SQL.
 
SAP IQ (currently in release 16) runs on standard hardware and supports a large number of operating systems. Learn more at www.sap.com/pc/tech/database/software/sybase-iq-big-data-management/index.html.

If implemented discretely, these traditional approaches to storing archived SAP ERP and SAP BW data create two silos of data (see Figure 1), which ends up adding to the complexity of your system landscape and increases costs.

 

The traditional, siloed approach to storing archived SAP ERP and SAP BW data

Figure 1 — The traditional, siloed approach to storing archived SAP ERP and SAP BW data

A New Way to Store Archived Data with SAP IQ

Available with SAP NetWeaver 7.31 support package stack (SPS) 07, the SAP Information Lifecycle Management database storage option allows customers to consolidate their storage infrastructure on a single platform and substantially reduce storage-related costs. Instead of storing SAP ERP data in a traditional third-party data store via SAP ArchiveLink or in a WORM-like data store, you simply move the data to an SAP IQ database that can also store your SAP BW data (see Figure 2). This option not only allows you to take advantage of the archiving and reporting performance advantages of SAP IQ, it also enables you to leverage the full set of retention management capabilities of SAP Information Lifecycle Management, such as propagation of expiration dates, legal hold support, and automated rule-based data destruction, while storing the data in SAP IQ.

The SAP Information Lifecycle Management database storage option

Figure 2 — The SAP Information Lifecycle Management database storage option supports the storage of SAP ERP data on an SAP IQ database that can also store SAP BW data, enabling a holistic approach to data storage

 
Detailed information on installing and configuring the SAP Information Lifecycle Management database storage option is available at SAP Service Marketplace.2 Before embarking on an implementation, however, it is important to understand the types of content that can be archived using this option, and the benefits provided by the storage capabilities of SAP IQ, so that you can get the most out of your investment. Let’s take a closer look.

What Types of Content Are Supported?

The SAP Information Lifecycle Management database storage option enables you to store two types of structured database content from SAP ERP in SAP IQ: archive files, which contain archived data, and archive indices, which provide access to the archive files. Many companies, such as retailers, generate large volumes of data in their SAP applications and need to archive items — iDocs, for example — on a daily basis. To be able to respond to inquiries — from shop personnel regarding certain materials or promotions, for instance — organizations maintain large archive indices of these archived items. 

 

Note: The SAP Information Lifecycle Management database storage option is included with SAP NetWeaver 7.31 SPS 07 (archive index support only) and SAP NetWeaver 7.31 SPS 10 (archive file support). To use this option, you must have the SAP IQ server installed with a minimum required version of 15.4 ESD3 (15.4.0.3027). If you are running SAP IQ 16.0, you must be at least on version 16.0 SP1 PL1 (64-bit). Additionally, the code page of the server must be CESU-8. For the full details, refer to SAP Note 1737415.

 
Archive indices are archive information structures created with the SAP standard Archive Information System (transaction SARI) and stored in transparent database tables with the technical name ZARIX* (e.g., ZARIX1, ZARIX2). While archived data moves from the application database to the archive, the indices traditionally remain in the application database, and large indices in particular can still cause volume problems. Storing indices in columnar tables in SAP IQ allows the database to benefit from the high compression rates of SAP IQ, relieving it from unnecessary strain. To enable this functionality, you simply establish a connection to a secondary database (SAP IQ) in the Archive Information System via transaction DBCO (see Figure 3).3 The storage of archive indices in SAP IQ is supported as of SAP NetWeaver 7.31 SPS 07.

Establishing a connection to the SAP IQ database for storage of archive indices

Figure 3 — Establishing a connection to the SAP IQ database for storage of archive indices

 
Archive files are Archive Development Kit files created by the data archiving process in your SAP ERP system. The archiving system writes the data to be archived to archive files according to the structure defined by the corresponding archiving objects, which must be enabled for SAP Information Lifecycle Management for use with its functionality. To store archive files from SAP ERP in SAP IQ, in transaction IRMPOL (SAP Information Lifecycle Management policy management), you simply specify that the SAP Information Lifecycle Management store point to SAP IQ. The files are stored as BLOBs in the read-only table space of the SAP IQ database. Archive files created in the SAP Information Lifecycle Management retention warehouse for system decommissioning can also be stored in SAP IQ. The storage of archive files in SAP IQ is supported as of SAP NetWeaver 7.40 SPS 05 and SAP NetWeaver 7.31 SPS 10.
 
In an upcoming innovation cycle, SAP intends to add support to the SAP Information Lifecycle Management database storage option for storing SAP ArchiveLink documents (which contain unstructured content, such as scanned invoices, that is attached to business objects in the SAP system) in SAP IQ. With this support, customers will no longer need to maintain an SAP ArchiveLink store for storing SAP ArchiveLink documents, enabling further optimization of the storage infrastructure.

Choosing the Right Information Management Approach

How does the SAP Information Lifecycle Management database storage option relate to traditional enterprise content management (ECM) solutions, and when does it make sense to use which approach?
 
If your primary focus is on data archiving and retention management, and your main objective is to reduce the complexity and total cost of ownership of your IT landscape, then the SAP Information Lifecycle Management database storage option is a compelling alternative storage solution. As an added benefit, using this option enables SAP IQ to serve as an integrated archiving platform and infrastructure for both SAP ERP and SAP BW.
 
If your primary focus is on managing and archiving large volumes of unstructured information (such as text, images, emails, and social media content), then using an ECM solution — such as the SAP Extended ECM application by OpenText, the SAP Archiving application by OpenText, or the SAP Document Access application by OpenText — would be a better choice. This is also the case if your business processes are very document-centric, where employees are jointly creating documents in a collaborative fashion and attaching them to business objects in the SAP system (in plant maintenance applications or in SAP Enterprise Asset Management, for instance). As an added value, SAP Document Access provides easy access to your SAP documents and a connection to write once, read many (WORM)-like storage devices. Learn more about these solutions at www.sap.com/pc/tech/enterprise-information-management/software/content-management/index.html.

A Fast and Efficient Deployment

To help customers quickly and efficiently adopt a cost-effective data aging and retention management strategy, SAP delivers the SAP IQ Near-Line Storage and Retention Management rapid-deployment solution. 
 
This rapid-deployment solution, available since August 2014, helps organizations get up and running quickly to address typical business challenges caused by large data volumes, including:

  • High maintenance effort for managing large data volumes in SAP BW and SAP ERP systems
  • Poor performance of SAP BW and SAP ERP applications due to large data volumes
  • IT complexity caused by heterogeneous system landscapes, including non-SAP near-line storage solutions
  • Risks due to costly implementation efforts and unforeseeable implementation problems

The SAP IQ Near-Line Storage and Retention Management rapid-deployment solution includes best practices and services for a complete data aging strategy for SAP BW and retention management for SAP ERP applications. Figure 4 provides an overview of its delivery scope.4
 
Once you have fully set up your storage infrastructure using the rapid-deployment solution, you can move data from your online SAP BW and SAP ERP applications to secondary, secure, and cost-effective near-line storage based on SAP IQ software. This helps reduce the maintenance effort, increase the performance of your SAP BW and SAP ERP applications, and fulfill your retention obligations.

Delivery scope of the rapid deployment solution

Figure 4 — Delivery scope of the rapid-deployment solution

Paving the Way to a Holistic Big Data Strategy

By storing historical data from your SAP ERP or SAP BW applications on SAP IQ, you can:

  • Minimize costs: You get all components, including storage, from one, trusted vendor. You also reduce costs by storing archive indices more efficiently, thanks to the high compression rate of SAP IQ compared to traditional databases. If you use the SAP BW near-line storage solution, you can further reduce the data footprint in SAP BW and optimize your investment in SAP IQ.
  • Increase performance: With its columnar-style database, SAP IQ enables a fast archive index read without additional secondary database indices, resulting in high performance when accessing archived data. It also offers enhanced search capabilities using broader indices (since database space is no longer an issue), which saves time and enables faster I/O because the system accesses fewer layers (software, network, storage, and hardware, for example) compared to a standard solution using an external archive store.
  • Reduce system complexity: By eliminating the need for third-party data stores and software, you can reduce the complexity of your IT landscape. This is particularly true if you also leverage SAP IQ for storing analytical data using the SAP BW near-line storage solution.

The SAP Information Lifecycle Management database storage option helps organizations simplify and streamline their archiving storage infrastructure, in turn paving the way to a holistic and efficient big data strategy. You can learn more about SAP’s end-to-end approach to information management at www.sap.com/pc/tech/enterprise-information-management/software/information-lifecycle/index.html.

 

1 While the database storage options support archiving data from all SAP Business Suite applications, SAP ERP currently has the most use cases, and is the main focus of this article. [back]

2 See the “Installation and Configuration Guide” for the SAP Information Lifecycle Management database storage option at http://service.sap.com/~011000358700001208472013E. [back]

3 For more information, refer to the SAP Help Portal under SAP ERP > SAP ERP Cross-Application Functions > Cross-Application Components > SAP Information Lifecycle Management > Additional Functions > Defining a Secondary Database (SAP IQ). [back]

4 More details on the SAP IQ Near-Line Storage and Retention Management rapid-deployment solution, including availability information, is available at SAP Service Marketplace (http://service.sap.com/rds-nls). [back]

An email has been sent to:





 

Helmut Stefani
Helmut Stefani

Helmut Stefani (helmut.stefani@sap.com) joined SAP in 1997, working on the documentation, product management, solution management, and rollout of data archiving, information lifecycle management (ILM), and enterprise content management (ECM) topics. Helmut has authored several publications on data management, data archiving, and ILM, including the book Archiving Your SAP Data (2nd Edition) (SAP PRESS; 2007). He is also the author of a chapter on ILM in the book Enterprise Information Management with SAP (SAP PRESS; 2012). Currently, Helmut is a member of the SAP HANA Platform Global Center of Excellence.



More from SAPinsider



COMMENTS

Please log in to post a comment.

SAPinsider
FAQ