E2E100 - End to End Root Cause Analysis

E2E100 End to End Root Cause Analysis

E2E100

End to End Root Cause Analysis

THE BEST-RUN BUSINESSES RUN SAP © SAP AG 2006

© SAP AG 2007

Solution Manager 4.0 SP10 2007/Q2 Material number 50084566

Copyright

Copyright 2007 SAP AG. All rights reserved. No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice.

© SAP AG 2006

Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors. Microsoft, Windows, Excel, Outlook, and PowerPoint are registered trademarks of Microsoft Corporation. IBM, DB2, DB2 Universal Database, OS/2, Parallel Sysplex, MVS/ESA, AIX, S/390, AS/400, OS/390, OS/400, iSeries, pSeries, xSeries, zSeries, System i, System i5, System p, System p5, System x, System z, System z9, z/OS, AFP, Intelligent Miner, WebSphere, Netfinity, Tivoli, Informix, i5/OS, POWER, POWER5, POWER5+, OpenPower and PowerPC are trademarks or registered trademarks of IBM Corporation. Adobe, the Adobe logo, Acrobat, PostScript, and Reader are either trademarks or registered trademarks of Adobe Systems Incorporated in the United States and/or other countries. Oracle is a registered trademark of Oracle Corporation. UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group. Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, and MultiWin are trademarks or registered trademarks of Citrix Systems, Inc. HTML, XML, XHTML and W3C are trademarks or registered trademarks of W3C®, World Wide Web Consortium, Massachusetts Institute of Technology. Java is a registered trademark of Sun Microsystems, Inc. JavaScript is a registered trademark of Sun Microsystems, Inc., used under license for technology invented and implemented by Netscape. MaxDB is a trademark of MySQL AB, Sweden.

SAP, R/3, mySAP, mySAP.com, xApps, xApp, SAP NetWeaver, and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other product and service names mentioned are the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specifications may vary. The information in this document is proprietary to SAP. No part of this document may be reproduced, copied, or transmitted in any form or for any purpose without the express prior written permission of SAP AG. This document is a preliminary version and not subject to your license agreement or any other agreement with SAP. This document contains only intended strategies, developments, and functionalities of the SAP® product and is not intended to be binding upon SAP to any particular course of business, product strategy, and/or development. Please note that this document is subject to change and may be changed by SAP at any time without notice. SAP assumes no responsibility for errors or omissions in this document. SAP does not warrant the accuracy or completeness of the information, text, graphics, links, or other items contained within this material. This document is provided without a warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability, fitness for a particular purpose, or noninfringement. SAP shall have no liability for damages of any kind including without limitation direct, special, indirect, or consequential damages that may result from the use of these materials. This limitation shall not apply in cases of intent or gross negligence. The statutory liability for personal injury and defective products is not affected. SAP has no control over the information that you may access through the use of hot links contained in these materials and does not endorse your use of third-party Web pages nor provide any warranty whatsoever relating to thirdparty Web pages.

Course Prerequisites

Required Knowledge: Web Application Server ABAP and Java (SAPTECH) Basics of SAP Solution Manager Recommended Knowledge: End to End Solution Support (E2E50)

© SAP AG 2006, Title of Presentation / Speaker Name / 1

Target Audience

This course is intended for the following audiences: Application Management Team SAP Technology Operations Team Business Process Experts Technical Quality Managers SAP Service and Support Engineers

Duration: 5 days


User notes These training materials are not a teach-yourself program. They complement the explanations provided by your course instructor. Space is provided on each page for you to note down additional information. There may not be sufficient time during the course to complete all the exercises. The exercises provide additional examples that are covered during the course. You can also work through these examples in your own time to increase your understanding of the topics.

Course Goals

This course will enable you to: Perform E2E Root Cause Analysis to isolate the component causing the problem in a heterogeneous IT landscape Prove solution stabilization (success of continuous root cause analysis in a war room approach)


Course Content

Preface Unit 1

Introduction

Unit 5

End to End Trace

Unit 2

Incident Management

Unit 6

Unit 3

End to End Change Diagnostics

End to End Exception Analysis

Unit 7

End to End Workload Analysis

Application and Data Inconsistency Analysis

Unit 8

SAP EarlyWatch Alert

Unit 4

Appendix: SAP Solution Operations Standards


End-to-End Root Cause Analysis Certification Will

Title: Application Management Expert – Root Cause Analysis Duration: 1 hour, at Friday 1pm Number of Questions per Certification Topic: 2 (5%) 4 (10%) 12 (30%) 12 (30%) 6 (15%) 2 (5%) 2 (5%) --------40


Introduction and Incident Management End-to-End Change Diagnostics End-to-End Workload Analysis End-to-End Trace Analysis End-to-End Exception Analysis Application and Data Inconsistency Analysis SAP EarlyWatch Alert TOTAL

Root Cause Analysis Roadmaps Guide Through the Class

Root Cause Analysis (cross-component)

?

Is it a performance problem? End-to-End Workload Analysis Is it a functional problem?

? ?

System or application termination End-to-End Exception Analysis

?

Inconsistent data End-to-End Data Inconsistency Analysis

?

Incorrect results or error messages in application End-to-end Application Root Cause Analysis


If the customer is facing a performance problem you can start with end-to-end workload analysis.

Root Cause Analysis Roadmaps Overview E2E RCA (cross-component)

E2E Workload Analysis JAVA component analysis

E2E Trace Analysis

E2E Exception Analysis

Client Trace Analysis

JAVA Exception Analysis

Network Trace Analysis

ABAP Exception Analysis

Analyze CPU consumption Analyze Disk utilization

JAVA Trace Analysis

Java Memory analysis

ABAP Traces Analysis

Java Thread Dump analysis Java workload analysis

ABAP component analysis ABAP workload analysis Hardware bottlenecks analysis Database workload analysis ABAP Memory analysis

E2E Change Analysis System Landscape Analysis Overall change Analysis Technical Configuration Analysis

E2E Data Inconsistency Analysis User Level Analysis Application Level Analysis Technology Level Analysis

Business Configuration, Content and Coding Analysis

Interfaces and GUI analysis

Database workload analysis Performance Problem © SAP AG 2006, Title of Presentation / Speaker Name / 1

Functional Problem

Example: End-to-End Trace Analysis Roadmap

End-to-end trace ?

Client Network Server

Problem is on client side Client analysis with BMC AppSight

?

Problem is network related Network analysis (not in scope of this course)

?

Problem with Java-based component Analysis with Wily Introscope Transaction Tracer

?

Problem with ABAP-based component ABAP tracing with SQL Trace and Application Tracing Instrumented Browser

WAN Network

Presentation Logic

Enterprise Portal


Application Logic

ECC

Database

Example for Fact Sheet: E2E Workload Analysis Use Case

Level 2 (must know)

Get workload information of your complete solution Analyze overall performance bottlenecks Provides Key performance indicators for different components Best Practice for Investigation / Rules of Thumb Check the workload overview for bad response times Choose „Portfolio“ view and search for „bubbles“ with high average response time and high values of accumulated times. They indicate performance bottlenecks (high workload in conjunction with high av. Response time) Check the component specific KPIs for deviation from average values

© SAP AG 2006

Limitations: aggregation level is 1 hour (no ad hoc analysis) Prerequisites: Solution Manager 4.0 SP10

Introduction to End-to-End Diagnostics

Introduction to End-to-End Diagnostics Incident Management End-to-End Change Diagnostics End-to-End Workload Analysis End-to-End Trace End-to-End Exception Analysis Application and Data Inconsistency Analysis © SAP AG 2006

© SAP AG


E2E100

1-1

Introduction to End-to-End Diagnostics

Contents: Scope of End-to-End Diagnostics SAP Standards for End-to-End Diagnostics Incident Management End-to-End Root Cause Analysis Remote Support Solution Monitoring

End-to-End Root Cause Analysis Tool Overview Open Diagnostics Platform

Savings Through End-to-End Diagnostics

© SAP AG 2006, E2E Root Cause Analysis 1

© SAP AG

E2E100

1-2

Introduction to End-to-End Diagnostics: Unit Objectives

After completing this unit, you will be able to: Explain the SAP standards related to End-to-End diagnostics, including incident management, End-to-End root cause analysis, remote support, and solution monitoring Explain the scope of End-to-End root cause analysis


© SAP AG

E2E100

1-3

Introduction to End-to-End Diagnostics: Unit Overview

Introduction to End-to-End Diagnostics Lesson 1: Scope of End-to-End Diagnostics Lesson 2: SAP Standards and Benefits of End-to-End Diagnostics


© SAP AG

E2E100

1-4

SAP Solution Manager Support User

RFID

Devices

Portal

SAP Portal

Java World

APO BI SRM SEM

Office

Duet

Analytics

.net World

Office Integration and Self-Services

Web Services and xApps

External Applications

Rendering

R/3

ERP

APO ESOA SEM SRM

SAP Solution Manager © SAP AG 2006, E2E Root Cause Analysis 1

To be able to support today’s SAP solutions, the following need to be supported: Client-side technologies such as mobile devices, portals, Adobe forms, Microsoft Office, and Internet browsers Server-side applications such CRM, SRM, SCM, and BI on multiple platforms (ABAP, Java, .net, and C++) SAP Solution Manager provides key capabilities to support distributed environments across different technology stacks. SAP Solution Manager provides open hubs to plug in external applications.

© SAP AG

E2E100

1-5

Tools for Problem Detection and Resolution PROBLEM DETECTION = SOLUTION MONITORING SAP Solution Manager

Health Check Monitor and Alert

More aggregation, less skill specialization

IT Reporting Analytics

Aggregate Evaluate

Cross-Component Diagnostics

• E2E Change Analysis • E2E Workload Analysis • E2E Exception Analysis • E2E Trace Analysis

Isolate Component

Select

Component Diagnostics ABAP

Nail it Down

Java

.net

C(++)

PROBLEM RESOLUTION = ROOT CAUSE ANALYSIS © SAP AG 2006, E2E Root Cause Analysis 1

Each system or software component relying on a certain technology stack (most often either ABAP, Java, .net, or C++) provides proprietary diagnostics to fulfill common diagnostics tasks on that component. SAP Solution Manager introduces a new layer of diagnostics that aims to isolate the component that is the root cause for an incident. On top of this root cause analysis (problem resolution), Solution Monitoring and IT Reporting aim to detect problems proactively and report on capacity trends, service level agreements, and solution quality. SAP Solution Manager 4.0 features four-cross component diagnostics capabilities: E2E Change Analysis: Makes changes to the productive landscape transparent (“Yesterday it worked, today it does not work; What has changed?“) E2E Workload Analysis: Isolates general performance bottlenecks in a solution landscape E2E Exception Analysis: Isolates exceptional situations, such as fatal entries in logs or dumps in a solution landscape E2E Trace Analysis: Records the activity of a single user or a single process in detail

© SAP AG

E2E100

1-6

End-to-End Root Cause Analysis: Overview Client

Server

Storage

Portal

CRM

Browser

SRM

ECC

SAP GUI

IO Subsystem

APO Mobile

BI

XI

MDM

Office

E2E Change Analysis Introscope

E2E Workload Analysis E2E Exception Analysis

BMC Appsight for SAP Client Diagnostics Introscope

E2E Trace Analysis


While End-to-End Change Analysis, End-to-End Workload Analysis, and End-to-End Exception Analysis cover the server side, End-to-End Trace Analysis spans the full cycle of a user request - from client to server to disk. SAP has licensed two third-party products to complete the key capabilities in root cause analysis: CA‘s Wily Introscope for server-side root cause analysis, and BMC AppSight for client-side root cause analysis. Both products are highly adaptable and are delivered by SAP fully preconfigured for SAP products (containing instrumentation and dashboards for SAP). CA Wily Introscope is fully integrated in the server-side root cause analysis infrastructure of SAP Solution Manager. Wily Introscope is included in SAP Solution Manager for SAP standard components. If customers want to support customer applications like SAP does, they can buy an add-on for SAP Solution Manager. This SAP Price List component is known as the Solution Support Enablement Pack. BMC AppSight is also included in the Solution Support Enablement Pack. In SAP Solution Manager, the customer can record with the black box and its SAP configuration; however, the customer can only analyze with the BMC AppSight Console when they have licensed the Solution Support Enablement Pack. SAP engineers have full access both to Wily Introscope and BMC AppSight.

© SAP AG

E2E100

1-7

End-to-End Workload Analysis: Example An Employee Self-Service (ESS) user experiences a performance problem in the browser. Browser

WAN Network

Presentation Logic

Enterprise Portal

Application Logic

Database

ECC

Analysis Path

Finding

Use End-to-End Workload Analysis to check server side performance

No bottleneck on server side, so issue must be on client, network or performance of single execution on server side

Use End-to-End Trace Analysis to record the activity of a single user – browser to disk

Time is spent on client

Use BMC Appsight for SAP Client Diagnostics to identify the root cause of bad client performance

Bad Java Script coding: over 20.000 loops over an operation on a date field


The Employee Self-Service scenario is distributed over physical hosts with installations of SAP Enterprise Portal and mySAP ERP Human Capital Management as a back-end system. End-to-end workload analysis should find out which technology component the problems are related to.

© SAP AG

E2E100

1-8

Third Party Tools Included in Diagnostics Introscope is a performance management solution that enables you to monitor complex Web applications in production environments 24x7. http://www.wilytech.com (part of CA)

Appsight is a system built on a unique problem resolution architecture that was designed from the ground up with patented Black Box technology to optimize the problem resolution process. http://www.identify.com (part of BMC)

SAP Standard for

Server Performance Diagnostics

Client Diagnostics

SAP Solution Manager 4.0 includes a license

... of Wily Introscope for SAP delivered instrumentation and dashboards

… for the recording agent of Appsight („blackbox“)

Solution Support Enablement Pack includes a license

to create instrumentation for custom code

to use Appsight Console to analyze client logs recorded by the blackbox

to create own dashboards (including defintion of customized monitoring tresholds)

to create of own instrumentation

Scope


Solution Support Enablement Pack (SEP) is a billable addon to SAP Solution Manager. SAP Support can use the full scope of both products to provide service and support to customers.

© SAP AG

E2E100

1-9

Introduction to End-to-End Diagnostics: Unit Overview

Introduction to End-to-End Diagnostics Lesson 1: Scope of End-to-End Diagnostics Lesson 2: SAP Standards and Benefits of End-to-End Diagnostics


© SAP AG

E2E100

1-10

SAP Standards for End to End Diagnostics 1

End to End Root Cause Analysis

2

Incident Management

3

Remote Support

4

Solution Monitoring


End-to-End Diagnostics requires four standards: End-to-End Root Cause Analysis Incident Management Remote Support Solution Monitoring For more information on all SAP Solution Operation standards, see http://service.sap.com/supportstandards

© SAP AG

E2E100

1-11

End-to-End Root Cause Analysis

End-to-end root cause analysis consists of roadmaps and tools to work out a corrective action and a final solution for any incident: Used by all SAP ecosystems: Customer, partner, ISV, and SAP Covers all SAP products and is open for fast integration of any ISV product One standard tool for each diagnostics task

Root Cause Analysis = Problem Resolution At least provides a workaround! Root Cause Analysis ? Notes Search and Forward to Development

© SAP AG 2006

SAP solution operations is based on 16 standards. Four of them are related to diagnostics: Root Cause Analysis Incident Management (in SAP Solution Manager: Service Desk) Remote Support Solution Monitoring (System Monitoring and Business Process/Interface Monitoring) Root cause analysis provides roadmaps and tools that enable a support engineer in the SAP ecosystem to work out a corrective action and a final solution for any incident. It is used by the entire SAP ecosystem, covers all SAP products, and is open for fast integration of any third party. For each diagnostic task, the standard features one diagnostic tool. This class and certification is focused on root cause analysis. The class presents the most typical component-independent support tasks along root cause analysis roadmaps.

© SAP AG

E2E100

1-12

Incident Management

Incident management is a standard process to: Describe, categorize, and prioritize an incident Collect context data of the incident Clarify the incident and search for an existing solution both in the customer solution database and the SAP Notes database Determine the root cause and, if necessary, forward the incident inside the customer support organization to ISV or to SAP Define and implement the actions required to eliminate the root cause Validate that the corrective action will prevent recurrence of the incident Make the status of the incident transparent to all parties


Incident management is required to administer the workflow of creating and processing any incident in a customer solution. Efficient incident management requires the key capabilities shown here. The SAP standard for incident management is the Service Desk in SAP Solution Manager.

© SAP AG

E2E100

1-13

Incident Management in SAP Solution Manager: Service Desk Creates

End User

Message

Solution Support

1 SAP system

Searches for Solution

SAP Service Marketplace

2

Provides

Finds

Solution

Solution SAP Solution Manager

or

SAP Notes database Service products Best practices DB

Forwards Message

4 Provides

Customer's solution database

Solution

3

SAP Support

Provides Solution

Customer

SAP


The SAP best practice for Incident Management is to use the Service Desk in SAP Solution Manager. The maintenance contract includes the Service Desk for managing incidents within SAP scope. Unlike any other ticket system, the Service Desk collects the context of an incident from Web Application Server, and is strongly integrated with the SAP Service Marketplace (part of the SAP service backbone).

© SAP AG

E2E100

1-14

Remote Support Central Gateway to customer solution landscape

Remote Today, 98% of installed SAP solutions have a line connection to SAP. Every SAP developer may log on from his or her desktop to any live customer installation to perform root cause analysis.

Safe Customer-enforced change control: The SAP standard authorization profile does not allow changes.

SAP Support Safe access through SAP router

SAP Customer Solution Manager Diagnostics Change Control

OS monitor Landscape DB monitor Config files SAP Log files agent Java dumps Java stats


SAP Solution Manager is the central support hub for SAP employees. It provides safe remote access to diagnostics data in the customer solution landscape. Changes to production are always enforced by the customer change management workflow: SAP only changes settings in the customer development system if asked by the customer, complying with the customer’s change management and validation workflow. The customer is responsible to transport and apply the changes in the quality assurance and production environment. SAP Solution Manager has a read-only authorization policy. In the SAP Solution Manager standard configuration, a support user, SAPSUPPORT, is created. This user does not allow changes to the customer solution landscape, ensuring that changes to the customer environment are enforced by the customer change management workflow.

© SAP AG

E2E100

1-15

Solution Monitoring

Business Process & Interface Monitoring and System Monitoring Detect performance problems or errors before they affect business continuity Trigger alerts based on a central alert definition (thresholds) Provide notification to IT responsible for a critical system situation Display and process alert

Provide transparency on IT operations excellence Service levels Capacity trends Solution quality


To detect incidents before they affect business continuity, Solution Monitoring detects incidents proactively. When Solution Monitoring detects performance problems or errors, it and triggers alerts that can be broadcasted to the responsible persons in IT.

© SAP AG

E2E100

1-16

End-to-End Root Cause Analysis is triggered by Solution Monitoring or Incident Management End-to-End Diagnostics Solution Monitoring Business Process Operations BP and Interface Monitoring

SAP Technology Operations System Monitoring

End User Incident Management

End-to-End Root Cause Analysis

Process alerts before they affect business

Restore service operation as quickly as possible


SAP Solution Monitoring comprises both technical monitoring (system monitoring) and monitoringrelating diagnostics data in a business process context, called Business Process and Interface Monitoring. End users trigger incidents via incident management. Both Solution Monitoring and Incident Management can lead to root cause analysis for problem resolution.

© SAP AG

E2E100

1-17

End-to-End Diagnostics: Organizational View Customer’s Business Unit Global Business Process Champion Regional Business Process Champion

End User Incident management

PMO (Program Office)

Customer’s IT Application Management End to End Root Cause Analysis, Remote Support

Custom Development

Business Process Operations BP and Interface Monitoring

SAP Technology Operations System Monitoring

IT Infrastructure SAP standards in operations reduce total cost of operations and enable mission-critical support.

SAP standards available Defined SAP interfaces


Standard end-to-end root cause analysis and remote support are owned by Application Management in the customer IT organization. Custom development, business process operations, and SAP technology operations also work with the end-to-end root cause analysis tools in SAP Solution Manager and the component diagnostics tools of the satellite systems.

© SAP AG

E2E100

1-18

SAP Solution Manager Scenarios Implementation of SAP solutions SAP methods and tools Global rollout Customizing synchronization E-learning management Test management

Upgrade of SAP solutions

Solution Monitoring

CORE BUSINESS PROCESSES

SAP methods and tools E-learning management Test management

Change Request Management Follows ITIL standards Maintenance processes

=

Required for the delivery of SAP standard support services

System monitoring Business process monitoring Central system administration Solution reporting Service level reporting SAP EarlyWatch Alert

Service Desk SAP Best Practices for incident management Integration of third-party help desks

Root Cause Analysis Delivery of SAP Services On-site/remote delivery Issue management

Safe remote access Performance measurement Logs and dumps Traces Technical configuration


Root Cause Analysis is one of the SAP Solution Manager scenarios that is required for delivery of SAP support.

© SAP AG

E2E100

1-19

Savings Through End-to-End Diagnostics Reduce license costs Tools for service desk, root cause analysis and solution monitoring are included in SAP Solution Manager

Reduce costs for support experts Root cause analysis by a targeted approach: one-step dispatching to the expert for component root cause analysis Monitoring by automated notification of alerts IT reporting by automated report generation and broadcasting (powered by SAP BI) Standardization and certification keep consultant rates at an acceptable level

Reduce time of disrupted production Mission-critical support: Incidents that are not solved in a timely manner can mean business loss


Significant savings can be achieved with end-to-end diagnostics by reducing license costs, costs for support experts, and disrupted production time.

© SAP AG

E2E100

1-20

End to End Diagnostics: Open Hub Architecture SAP Solution Manager: End-to-End Diagnostics Open Diagnostics Hub Infrastructure Workload

Exceptions

Java World

Traces

SAP Portal

Technical Config

APO BI SRM SEM

Duet

Analytics

.net World

Office Integration and Self-Services

Web Services and xApps

External Applications

Changes

R/3

ERP

APO

ESA

SEM SRM


Diagnostics has an open hub infrastructure that allows external applications or new SAP applications to hook in quickly (usually about one week of project to ensure basic support). Open hubs exist for: Workload: A hub for performance and resource metrics Exceptions: A hub for critical status of an engine or an application (such as log entries of severity fatal and error, as well as dumps) Traces: A hub for the component to obtain part of the end-to-end trace in SAP Solution Manager Configuration: A hub providing technical configuration, by default once a day Changes: A hub that enables SAP Solution Manager to provide unified reporting on configuration, software, and content SAP Solution Manager ensures that all diagnostics data is displayed consistently and is understandable over component borders. SAP will connect all significant supported products to end-to-end diagnostics. A component is not hooked into end-to-end diagnostics until the component diagnostics is complete, stable, and understood. If SAP does not hold a license and maintenance contract with its customers, products can be supported by SAP on customer request via a premium engagement (SAP MaxAttention). In this case, SAP would assess the component diagnostics of the third party and integrate the component into end-to-end diagnostics if appropriate.

© SAP AG

E2E100

1-21

Introduction to End-to-End Diagnostics: Unit Summary

You should now be able to: Explain the SAP standards surrounding end-to-end diagnostics, including incident management, end-to-end root cause analysis, remote support, and system administration and monitoring Explain the scope of end-to-end root cause analysis


© SAP AG

E2E100

1-22

Technical Architecture of the example ESS scenario

The ESS scenario consists of a SAP XSS application and a business package for ESS which are deployed on the Web Application Server Java stack. The backend functionality is provided from a SAP ECC on Web Application Server ABAP stack. A backend function module call from the Java stack is done via the Java Connector Service of the J2EE.

© SAP AG

E2E100

1-23

Incident Management


© SAP AG


E2E100

2-1

Incident Management

Contents: Incident Management Workflow


© SAP AG

E2E100

2-2

Incident Management: Unit Objectives

After completing this unit, you will be able to: Explain the incident management workflow Explain the role of SAP Solution Manager to manage incidents across all parties in the SAP support network


© SAP AG

E2E100

2-3

Incidents, Problems, and Requests for Change ITIL

Incident Service Desk/Incident Management

Problem Problem Management

Known Error Problem Management

Request for Change

“Any event which is not part of the standard operation of a system that causes, or may cause, an interruption to, or a reduction in, the quality of service.” Goal: Restore service operation as quickly as possible, with minimal disruption to end users “One or more incidents with unknown reason” Goal: Reduce the number and severity of incidents, and provide documentation for first- and second-level support Reason is known; workaround or solution is available. Change is necessary, so request for change will be prepared

Request for change

Change Management


An incident is any event that is not part of the standard operation of a system which causes, or may cause, an interruption to, or a reduction in, the quality of service. The goal of incident management is to restore service operations as quickly as possible, with minimal disruption to end users. A problem is one or more incidents of unknown reason. The goal of problem management is to reduce the number and severity of incidents and to provide documentation for first and second-level analysis so that known errors can be identified and a solution can be proposed when the incident next occurs. A request for change is created following the customer’s Change Request Management process

© SAP AG

E2E100

2-4

E2E Diagnostics Within the Support Process End User

First-Level Support Help Desk

Second-Level Support SAP GSC

Incident

Create Create message message Classify Classify Prioritize Prioritize

SAP

Customer


A user creates an incident, classifies it, and prioritizes it.

© SAP AG

E2E100

2-5

E2E Diagnostics Within the Support Process (2) End User



Incident Complete Complete description description

Create Create message message Classify Classify

Gather Gather context context details details Check Check priority priority

Prioritize Prioritize

SAP

Customer


The help desk completes the description, gathers context details, and checks priorities.

© SAP AG

E2E100

2-6

Problem Description: Gather Context Details

Incident

What What is is the the complete complete description description of of the the problem? problem? -- Specify Specify the the sequence sequence of of action. action. -- What What are are the the end-user end-user expectations expectations from from the the system system at at this this point? point? When When did did the the problem problem start? start? How How frequently frequently does does the the problem problem occur? occur? Is Is the the problem problem reproducible? reproducible? IfIf yes, yes, how how and and when? when? Does Does an an error error message message appear appear on on the the screen? screen? What What is is the the business business impact impact of of the the problem? problem?

Goal Capture Capture all all details details about about the incident the incident

Methodology Methodology Gather Gather details details from from the the original original source source of of the the incident incident (end (end user, user, reports) reports) Gather Gather supporting supporting evidence evidence of of the the incident incident (screenshots, (screenshots, reproducible reproducible steps) steps)

Purpose Each Each piece piece of of information information provides insight provides insight on on how how to later perform the to later perform the root root cause cause analysis analysis


Gathering context details: What is the problem? When did the problem start? What were the last changes? What components are involved? How frequently does the problem occur? What is the impact of the problem? How many users are on the system? Is the problem reproducible and if so, when? What steps did the customer take to fix the problem? What analyses have been done so far? What are the versions of the components involved?

© SAP AG

E2E100

2-7




Incident Complete Complete description description Gather Gather context context details details

Create Create message message

Check Check priority priority

Classify Classify Prioritize Prioritize

Search Search in in solution solution database database Search Search for for SAP SAP Notes Notes Provide Provide solution solution

Message is closed

SAP

Customer


Search for known errors in the customer solution database and SAP Notes. First-level support results are: Complete and detailed description of the problem, including versions of the primary components Summary of the analysis performed to date The priority of the problem is set correctly (SAP Note 67739) Solution/workaround found after SAP Note Search or internal solution database search

© SAP AG

E2E100

2-8




Incident Cross-component Cross-component root root cause cause analysis analysis

Check Check priority priority

Classify Classify Prioritize Prioritize

Identify Identify responsible responsible component component

Provide Provide solution solution

Forward

Gather Gather context context details details

Create Create message message

Forward

Complete Complete description description

Message is closed

Component Component root root cause cause analysis analysis

Search Search in in solution solution database database Search Search for for SAP SAP Notes Notes



Message is closed

Message is closed

SAP

Customer


Root cause analysis starts with cross-component root cause analysis, unless the responsible component is already known. Component root cause analysis drills down to the root cause of the problem in the component.

© SAP AG

E2E100

2-9

Incident Management: Unit Summary

You should now be able to: Explain the incident management workflow Explain the role of SAP Solution Manager to manage incidents across all parties in the SAP support network


© SAP AG

E2E100

2-10

End-to-End Change Analysis

Introduction to End-to-End Diagnostics Incident Management End-to-End Change Analysis End-to-End Workload Analysis End-to-End Trace End-to-End Exception Analysis Application and Data Inconsistency Analysis © SAP AG 2006

© SAP AG


E2E100

3-1

End-to-End Change Analysis

Contents: Detection of Version Information Configuration and File Reporting End-to-End Change Analysis Change and Transport System Enhanced Change and Transport System


© SAP AG

E2E100

3-2

End-to-End Change Analysis: Unit Objectives

After completing this unit, you will be able to: Check the current information on software components and their versions Look up recent changes in customer solutions, including: Technical configuration Business configuration Content Coding Compare technical configuration between landscapes


© SAP AG

E2E100

3-3



E2E Trace Analysis







JAVA Trace Analysis









Database workload analysis Performance Problem

Functional Problem

© SAP AG 2006

© SAP AG

E2E100

3-4

Why Change Diagnostics? Understand the big picture before starting deeper root cause analysis Check system landscape: the satellite systems, their main functions and how they connect with each other Check software components and their versions

Symptoms: The system behaves differently after certain date or some changes One landscape works differently from the other one. For example, QAS is working fine, but PRD has issues.


© SAP AG

E2E100

3-5

Typical Questions

How many transports were imported last week? When did we import support packages?

Did we change any technical configuration parameters?

Is there ONE place where all changes in the solution are listed?

Which systems of my solution were changed?


End-to-End Change Analysis is an important part of end-to-end root cause analysis. If an unknown problem comes up, End-to-End Change Analysis answers the question, “What was changed in the system?”

© SAP AG

E2E100

3-6

Change Diagnostics Roadmap

Change Diagnostics (Cross Component)

?

Which components are involved and which versions do they have? Display system landscape documentation in SAP Solution Manager

?

Which changes were applied recently?

?

Overall changes in a solution End-to-End Change Analysis

?

Changes to Technical Configuration Configuration and File Reporting

?

Changes to Business Configuration, Content and Coding Enhanced Change and Transport System


The picture shows the change diagnostics roadmap with the two main problem areas: Control of the current version information Control of recent changes

© SAP AG

E2E100

3-7

End-to-End Change Analysis: Overview

End-to-End Change Analysis Tools Configuration Tracking

Type of changes

Technical Configuration

Enhanced Change and Transport System

Patches, SP, Release

Business Configuration

“Content” EP, XI, BI, SLD

Coding

Changes to Production

Productive Environment


This diagram shows the different tools in End-to-End Change Analysis. Configuration tracking extracts configuration parameters from Java or ABAP-based systems and displays the parameters and their history in Diagnostics. The Change and Transport System records changes to business configuration (customizing requests) and coding (workbench requests). The new Enhanced Change and Transport System is also able to transport non-ABAP objects such as Java archives. Releases, Support Packages, and patches are documented in SAP Solution Manager. End-to-end change analysis provides BI-based reporting on all changes in a solution.

© SAP AG

E2E100

3-8



?


?


?


?


?




© SAP AG

E2E100

3-9

Transaction SMSY: Landscape Version Information

System ID, Product Version, Main Instance Installation Number Data Source Message Server, System Number, DB

Software components and version information


Transaction SMSY in SAP Solution Manager contains the version information for the entire system landscape. In the tree structure on the left side of transaction SMSY, choose Systems and select the system you are currently analyzing. In the header data, check the data source. It should be SLD with a current time stamp. If the data was entered manually or if the time stamp is old, the data might be outdated and unreliable. Go to the Software Components tab to see the releases and Support Package levels of the different software components of the solution. Check data source for version information reliability. If data was refreshed recently, you can rely on the version information provided in SMSY. For Java-based systems, the version information including patch information can be found in the component information: http://:/sap/monitoring/ComponentInfo

© SAP AG

E2E100

3-10

Types of Content Corrections in SAP Web AS

SAP Web AS ABAP

SAP Web AS JAVA

SMSY

SMSY

Release

Release

Support Package Stack

Support Package Stack

Support Packages

Support Packages

SAP Notes

Patches /ComponentInfo SAP Hotfix

SNOTE

Software Deployment Manager


© SAP AG

E2E100

3-11

Note Browser

The Note Browser gives you a list of all SAP Notes. Select all notes with implementation status Completely implemented. Use transaction SNOTE, then choose SAP Note Browser. © SAP AG 2006, Title of Presentation / Speaker Name / 1

The Note Browser gives you a list of all SAP Notes that are imported into an ABAP system. Select all notes with implementation status Completely implemented. Use transaction SNOTE and then choose SAP Note Browser.

© SAP AG

E2E100

3-12

Fact Sheet: SMSY in SAP Solution Manager Level 2 (must know)

Use Case

Check hardware and software inventory Check current software components and support package level Provides Software Components and Support Package Level Best Practice for Investigation / Rules of Thumb Search for the SID by using the “Find” button Check in which project and solution landscapes the system is used by using the “Where-Used List” Check the reliability of the data by checking the data source and the “Last Changed” date. Only data which was updated recently is reliable


© SAP AG

E2E100

3-13



?


?


?


?


?




© SAP AG

E2E100

3-14

E2E Change Analysis - Overview

BSO

B70


E2E Change Analysis is integrated into the Diagnostics. Navigate to Root Cause Analysis -> Configuration -> E2E Change Analysis. In the first screen you see the number of changes for all systems of the solution in the selected timeframe.

© SAP AG

E2E100

3-15

Navigation Block


In the next step select a system. When you open the Navigation Block you can design individual queries. For example, you can display the number of changes per system, change type and calendar day.

© SAP AG

E2E100

3-16

Detail view per change type and change day

For the details jump into the Config Stores, displaying the parameter for the selected system and calendar day © SAP AG 2006, Title of Presentation / Speaker Name / 1

The number of changes is context sensitive. Right-click on the figures in order to navigate into the context menu. Then select Goto -> Config Details in order to display the complete configuration set for the selected day.

© SAP AG

E2E100

3-17

E2E Change Analysis - Architecture

E2E change analysis

Number of changes

InfoCube

Data collection once a day

Data collection once a day

Diagnostics Diagnostics agents agents

Solution Solution tool tool plug-ins plug-ins Solution Manager

Java-based Java-based installations installations

Config store

ABAP-based ABAP-based installations installations

Parameter values

Configuration and file reporting © SAP AG 2006, Title of Presentation / Speaker Name / 1

The data is collected in Java-based systems via diagnostics agents, and in ABAP-based systems via Solution Tool Plug-ins. End-to-end change analysis uses the same tables as configuration and file reporting. It enhances the functionality of the former configuration and file reporting by adding data from ABAPbased systems and by providing BI-based reporting capabilities. Change types are the smallest unit that is counted and displayed in End-to-End Change Analysis. Change types are bundled into change groups. Planned change groups and change types are: XI parameter CRM parameter - CRM back end - Mobile Infrastructure SEM / BI parameter OS parameter DB parameter - ORA - MAXDB - DB6 - DB4 - DB2 - INF ABAP parameter J2EE parameter TREX parameter © SAP AG

E2E100

3-18

Software version changes - Notes - Patches - Support Packages - Releases Transport requests - Customizing - Workbench - Content (CTS+) Configuration templates can be displayed using transaction SMDIAG_TEMPLATE in the SAP Solution Manager.

© SAP AG

E2E100

3-19

Required Software Versions

Release

Support Package

Planned Availability

NW04s

SPS13

August 2007

Solution Manager

SP13

August 2007

BI_Cont

SP6

July 2007

ST-A/PI

01J_CRM500

May 2007


© SAP AG

E2E100

3-20

Fact Sheet: E2E Change Analysis Level 2 (must know)

Use Case

Check all changes in a solution (OS, DB, ABAP parameters, Java parameters, transport requests, Support Packages, etc.) Central entry point for Root Cause Analysis Provides Number of changes per system, change category and day Best Practice for Investigation / Rules of Thumb Check all changes in a solution since the day when a certain error occurred In the navigation block, drill down into the changes per day In the result area, right-click on a number and go to the configuration details. This will show a list of parameters which were changed on a certain day If the change was recorded in a transport request, you can find the details in the managed system


Benefits: Central entry point for analyzing changes in a solution BI methods to drill down from overviews to detailed lists of changes Provides hints for root cause analysis with data and trends Provides accurate information on configuration parameters and their history (for example, database parameters, ABAP parameters, or Java parameters) Enables the organization to standardize the solution configuration Improves security by controlling the versions of configurations in use

© SAP AG

E2E100

3-21



?


?


?


?


?




© SAP AG

E2E100

3-22

End-to-End Change Analysis: Example 1

End-to-end Change Analysis example with Changes to Technical Configuration By testing of a Java application in Development system it has been noted that the performance of this application is much worse as in comparison to the production system. To locate the root cause, the technical configuration of J2EE server node in the Development system is to be compared with technical configuration of J2EE server node in the Production system.


© SAP AG

E2E100

3-23

Compare Two Landscapes Agent

Web AS Instance

Agent

D

Web AS Instance

Agent

D

Web AS Instance

Production

D

F15 S

Agent

S

S

Web AS Instance D

S

Agent

S

S

Agent

S

Web AS Instance

S

S

Web AS Instance D

S

Agent

S

S

SAP Solution Manager

Web AS Instance D

QA

What is S S S S S S difference from configuration point of view ??? Web AS Agent

D S

S

Instance

Development

D S

S

Config store

S

S

NC2

© SAP AG 2006

© SAP AG

E2E100

3-24

View Configuration Data (ABAP components)

6.Save (if required)

2.Select 1.Click

3.Load

5.Display

4.Select

© SAP AG 2006

Before we start with comparing of two landscapes let us first to see how the configuration data could be displayed for non-ABAP and ABAP components Configuration Change Reporting collects the configuration settings for all ABAP and non-ABAP components in the customer solution and provides navigation via landscape trees to each configuration setting in diagnostics. A history of changes is provided per single configuration setting (based on daily snapshots). In addition, it is possible to display the configuration as it existed for a given point in time. You can also compare the configuration settings between two components within any landscape, each at any given point in time. Configuration and File Reporting is integrated into Diagnostics. To start the application, choose Root Cause Analysis Configuration Configuration and File Reporting.

© SAP AG

E2E100

3-25

View Configuration Data (JAVA components)

6.Save (if required)

2.Display

1.Click © SAP AG 2006, Title of Presentation / Speaker Name / 1

© SAP AG

E2E100

3-26

View Configuration Data: Attribute History 2.Display history

1.Click

3.Compare 4.Save


© SAP AG

E2E100

3-27

Compare Two Landscapes

NC2

S

F15

S

Result


Use case – Compare two landscapes or two snapshots of the same landscape: Choose the desired time stamp and click the Compare button. Select the appropriate landscape in the Landscape2 dropdown list. Choose the desired time stamp for Landscape2. Check the node to compare (for example, Landscape, NW, SAP J2EE Engineer, Enterprise Portal). Click the Compare button. If you select a node other than Landscape, you need to select the item to compare in Landscape2 in a dialog box after you click Compare. Expand the configuration hierarchy to compare the attribute values. The procedure for comparing two snapshots of the same landscape is similar to that described above.

© SAP AG

E2E100

3-28

Compare Two Landscapes: Result

NC2

S

F15

S

(…)


© SAP AG

E2E100

3-29

End-to-End Change Analysis: Example 2

End-to-end Change Analysis example with Changes to Technical Configuration Several employees of a company have complained about high system response times during the usage of ESS scenarios over the last few days. It has been noted that all these employees are logged on to the same J2EE server node and this single J2EE server node demonstrates a bad overall performance in comparison to the other J2EE server nodes. To locate the root cause, the technical configuration of problematic J2EE server node is to be compared with technical configuration of other J2EE server nodes showing better performance.


© SAP AG

E2E100

3-30

Compare instances from the same Landscape Web AS Instance

Agent

Agent

D

Agent

S

Web AS Instance

S

Agent

D

S

S

Web AS Instance

S

Agent

D S

S

S

S

Web AS Instance

Production D

S

Web AS Instance

Agent

D S

Agent

D

S

S

Web AS Instance

S

Agent

S

S

Web AS Instance

What is D difference from S S S S configuration point of view ???

S

QA

Config store

Web AS Instance D

S

SAP Solution Manager

S

Development S

© SAP AG 2006

Here you see a J2EE system landscape with development, QA, and production systems. Each system consists of one or several hosts. Each host consists of one or several instances. Each instance consists of several nodes (dispatcher, server, and so on). A diagnostics agent is located on each host. It transfers configuration data to Solution Manager once per day. The need to track configuration settings first came up in J2EE systems. J2EE systems have a large number of configuration parameters in different locations. Formerly, it was difficult to track changes or to keep the configuration parameters consistent within the solution landscape.

© SAP AG

E2E100

3-31

Compare instances from the same Landscape

Node ID= 17694650 S Node ID= 26686150 S


© SAP AG

E2E100

3-32

Compare instances from the same Landscape: Results

Node ID= 26686150

Node ID= 17694650

S

S

Unchanged value Different value Additional value Missing value © SAP AG 2006, Title of Presentation / Speaker Name / 1

© SAP AG

E2E100

3-33

Compare Two Landscapes: Result Node ID= 17694650

Node ID= 26686150 S

S

Unchanged value Different value Additional value Missing value


In the Result screen, you can drill down into the system parameters. You will see different icons for unchanged values, different values, additional values, and missing values.

© SAP AG

E2E100

3-34

Fact Sheet: Configuration and File Reporting Level 2 (must know)

Use Case

Check configuration parameters of Java systems Compare configuration parameters of multiple instances Provides Configuration parameters of Java systems and their history Best Practice for Investigation / Rules of Thumb Compare configuration parameters in the same system but for different timestamps. This is a typical use case in Root Cause Analysis Compare configuration parameters within the transport landscape Compare configuration parameters between multiple instances in order to achieve homogeneous parameter settings


Important parameters include the Java memory parameters (heapsize, -Xms, -XX:NewSize, etc.)

© SAP AG

E2E100

3-35



?


?


?


?


?



© SAP AG

E2E100

3-36

Detect changes to Business Configuration, Content and Coding Example: Usage of Enhanced Change and Transport System to identify a transport which has caused a program error Starting with a certain date a user complains about an error message by executing an ABAP-based (e.g. an ABAP report) or non-ABAP application (e.g. an Enterprise Portal iView). Before that time this application was executed without errors. Enhanced Change and Transport System is to be used to identify a transport which has caused a program error.


© SAP AG

E2E100

3-37

Import History

The import history shows all transports that were imported in a selected time frame and the import time stamp. If the Transport Management System is configured, the import history can be displayed for any system in the transport domain. From the import history, you can navigate into the object list and into the transport log files.


The import history can be reached from transaction STMS or by choosing Overview Imports Goto Import History. Here you can find the list of all transports that were imported into a system, along with the import time stamp. From the import history, you can navigate into the object list and into the transport log files. The import history is often used to recreate the latest developments after a system copy.

© SAP AG

E2E100

3-38

1. Track Changes in Import History ABAP + non-ABAP 1. Call transaction STMS Goto Imports

Overview Import History

2. Choose corresponding time interval

3. Check the corresponding transports requests © SAP AG 2006, Title of Presentation / Speaker Name / 1

Java applications can inherit all the features of the ABAP Change and Transport System. The import history can be reached by using transaction STMS or by choosing Overview Imports Goto Import History. Here you can find the list of all transports that were imported into a system, along with the import time stamp From the import history, you can navigate into the object list and into the transport log files. The import history can be called from any system in the transport domain.

© SAP AG

E2E100

3-39

2. Navigate to the Object List (non-ABAP) non-ABAP

4. Display the Object Lists for this request


From the import history, you can navigate into the object list. By double-clicking on the non-ABAP object GUID, you reach the non-ABAP object details. Here you can find the name of the archive. When an object is changed in the development system, it is added to a transport request. You can display the transport request’s object list to see which objects it contains.

© SAP AG

E2E100

3-40

2. Navigate to the Object List (ABAP) ABAP

The object list shows all objects in a transport request, and whether they were imported and generated successfully.


When an object is changed in the development system, it is added to a transport request. You can display the transport request’s object list to see which objects it contains.

© SAP AG

E2E100

3-41

3. Navigate to the Transport Log Files ABAP + non-ABAP

5. Check the transport log file


From the import history, you can also navigate into the transport log files. In the detailed log file for the Deployment step, you can see if the deployment was successful. You can also see other information, such as the SDM host and port.

© SAP AG

E2E100

3-42

3. Navigate to the Transport Log Files (ABAP) ABAP

Return code

The transport log files show the return code for each import step: 0: The import was successful 4: A warning is issued, but all objects were transported successfully 8: A warning is issued, and at least one object could not be transported successfully 12 or higher: A critical error has occurred (generally not caused by the objects in the request)


In the transport logs, return codes indicate the success or failure of the export: 0: The export was successful 4: A warning is issued, but all objects were transported successfully 8: A warning is issued, and at least one object could not be transported successfully 12 or higher: A critical error has occurred (generally not caused by the objects in the request) The system administrator must check the log files after an import. The return codes for all steps must be 0 or 4.

© SAP AG

E2E100

3-43

4. Display active and available Versions (ABAP) ABAP 1. Click

2.Display

3. Compare


© SAP AG

E2E100

3-44

Version Management

Versions of ABAP Repository objects can be:

Development DB

Compared Active and temporary versions

Restored

Version Overview

Version DB

Version REL Request Date Time Author Versions in the development database act. 4.6C DEVK900110 10.05.1998 10:10:00 Developer

All versions saved through releasing change requests

Versions in the version database 00002 4.6C DEVK900105 08.05.1998 18:10:00 Developer 00001 4.6C DEVK900096 07.05.1998 08:04:00 Developer

Report Source Versions: ZZABAP © SAP AG 2006, Title of Presentation / Speaker Name / 1

When a change request is released, a new version of each Repository object in the change request is written to the version database, which contains a complete change history of all Repository objects. In addition to the versions created automatically by the release of change requests, users can create temporary versions at any time. To do this, in the maintenance transaction for the Repository object, choose Create version. You can access version management from: The Repository Browser (transaction SE80) The Transport Organizer (transaction SE09 or SE10) Display and maintenance transactions for all Repository objects In the version overview, the active and temporary versions are displayed in the development database, and versions saved as a result of released change requests are displayed in the version database. The version database resides in the development system. Versions cannot be transported between SAP systems. If the development system is removed from the system landscape, all versions in the version database are lost.

© SAP AG

E2E100

3-45

Organizer Tools Organizer Tools (SE03)

Components Components that that can can be be analyzed: analyzed: Objects Objects Objects Objects in in requests requests Object Object directory directory Request Request // tasks tasks Administration Administration


In Organizer Tools (transaction SE03), you can perform various tasks related to transportable objects and their transport requests. Most important for root cause analysis is to find objects in requests and tasks. If a certain object is not correct, you can see in which transport request it was changed and who is the owner of that transport request.

© SAP AG

E2E100

3-46

Fact Sheet: CTS Import History Level 2 (must know)

Use Case

Check which transport requests were imported into a system at a certain point in time This tool can be used for both ABAP and Java objects Provides List of imported transport requests and objects Best Practice for Investigation / Rules of Thumb STMS Overview Imports Goto Import History. Select the time interval in which the error occured Drill down into the transport logfiles to see if the import was successful Drill down into the object list in order to see which ABAP or Java objects were affected


© SAP AG

E2E100

3-47

Manage Heterogeneous Development Environments Enhanced Change and Transport System (CTS+)

ABAP Workbench SE80 Exchange Infrastructure Integration Builder Developer Studio and NWDI

Development Landscape

Transport

Quality Landscape

in check

Transport

Production Landscape

Deploy

SCA

EPA

Enterprise Portal Content Administrator

Deploy

Quality Component 1

Production Component 1

. . .

. . .

Quality Component n

Production Component n

… (open Interface for non-ABAP objects) © SAP AG 2006, Title of Presentation / Speaker Name / 1

The Enhanced Change and Transport System (CTS+) improves the proven Change and Transport System. It allows changes from heterogeneous development environments to be transported with the existing Change and Transport System. Java archives can be checked into transport requests as transportable objects. On the target system, after-import methods are executed to deploy the Java archives via the Software Deployment Manager (SDM). Transport of ABAP transport objects Enterprise Portal objects (EPA, PAR) NWDI objects of (SCAs) Objects from XI There is also an interface for non-SAP applications Will be developed further (for example, integration with Maintenance Optimizer) Change and Transport System (CTS) and Netweaver Development Infrastructure provide powerful functions to control transports in ABAP and Java. What was missing? Synchronized import into double-stack systems A solution for the transport of Portal content A central administration interface for all types of transports and systems Tracking and management of non-ABAP objects with Change Request Management These open issues are addressed with the Enhanced Change and Transport System © SAP AG

E2E100

3-48

End-to-End Change Analysis: Unit Summary

You should now be able to: Check the current information on software components and their versions Look up recent changes in customer solutions, including: Technical configuration Business configuration Content Coding Compare technical configuration between landscapes


© SAP AG

E2E100

3-49

© SAP AG

E2E100

3-50

Exercises Unit:

E2E Change Diagnostics

Lesson:

Detection of Version Information

At the conclusion of this exercise, you will be able to: • Find version information for a customer solution in SAP Solution Manager TT4 is the Solution Manger system which contains all information about the system landscape. TT5 is a satellite system which is managed by TT4.

1-1

Check the support package level in the SAP Solution Manager System TT4 1-1-1

Name the transaction to find the system landscape description in SAP Solution Manager: Answer: ______________________________________________

1-1-2

Call transaction SMSY and find information for the SAP ECC System TT5. What is the current Support Package Level for the software component SAP_APPL? Answer: ______________________________________________ What is the data source of the version information? Answer: ______________________________________________ When was the data last refreshed? Answer: ______________________________________________

1-1-3

Find version information for the Java Stack of system TT5 (“System Components” “Java System” “TT5”). What is the current Support Package Level for the software component CORETOOLS?

© SAP AG

E2E100

3-51

© SAP AG

E2E100

3-52

Exercises Unit:


Lesson:

Configuration and File Reporting

At the conclusion of this exercise, you will be able to: • Use the configuration and file reporting in the Solution Manager Diagnostics This exercise has to be executed in the Solution Manager Diagnostics of system TT4.

1-2

Logon to the Solution Manager Diagnostics in TT4 and perform the following analyses. 1-2-1

Call the Configuration and File Reporting Application.

1-2-2

Select the solution provided by trainer. Check for which timestamps configuration data is available. Load the configuration data for the most recent timestamp.

1-2-3

In the “View Configuration” screen search for the string “version.txt”. Go to the first occurance. What is the ConfigStore path and ConfigStore name? Answer: ______________________________________________ When was the file last changed? Answer: ______________________________________________

1-2-4

Click on the line to display the configuration parameters. Which of the parameters were changed? Answer: ______________________________________________ Mark a parameter which was changed and display the attribute history. When was the parameter changed and what was the previous value? Answer: ______________________________________________

© SAP AG

E2E100

3-53

1-2-5

Compare the complete configuration for the solution provided by trainer with an older version.

1-2-6

Perform a Deep Compare for a single ConfigStore.

© SAP AG

E2E100

3-54

Exercises

Unit:


Lesson:

Change and Transport System

At the conclusion of this exercise, you will be able to: • Use the tracking features of the Change and Transport System to identify a transport which has caused a program error. This exercise must be executed in the mySAP ERP system TT5.

1-3

A user complains about an error in the report ZZ_CHANGE_CONTROL_TEST in the system TT5. An error message comes up when the button “No” in the first popup screen is pressed. The error appears since 06.03.2007. 1-3-1

Which transport request was imported into TT5 on 06.03.2007? Answer: ______________________________________________

1-3-2

Which user is the owner of this transport request? Answer: ______________________________________________

1-3-3

Which objects does this transport request contain? Answer: ______________________________________________

1-3-4

Compare the new version of the imported object with the previous version. Which line of coding was changed? Answer: ______________________________________________

1-3-5

How are the objects in the transport request related with the report ZZ_CHANGE_CONTROL_TEST? Answer: ______________________________________________

© SAP AG

E2E100

3-55

© SAP AG

E2E100

3-56

Solutions

1-1

Unit:


Lesson:

Detection of Version Information

Check the support package level in the SAP Solution Manager System TT4 1-1-1

Name the transaction to find the system landscape description in SAP Solution Manager: The transaction name is SMSY

1-1-2

Call transaction SMSY and find information for the SAP ECC System TT5. What is the current Support Package Level for the software component SAP_APPL? In the tree structure on the left hand side of transaction SMSY drill down into Landscape Components Systems SAP ECC TT5 SAP ECC Server. On the right hand side click on the tab “Software Components”. Search for the software component SAP_APPL and check the SP Level. What is the data source of the version information? Click on the tab “Header Data” and check the value for “Data Source” When was the data last refreshed? check the value for “Last Changed On/By”

1-1-3

Find version information for the Java Stack of system TT5 (Landscape Components System Components Java System TT5). What is the current Support Package Level for the software component CORETOOLS? In the tree structure on the left hand side of transaction SMSY drill down into Landscape Components System Components Java System TT5. On the right hand side click on the tab “Software Components”. Search for the software component CORE-TOOLS and check the SP Level.

© SAP AG

E2E100

3-57

© SAP AG

E2E100

3-58

Solutions

1-2

Unit:


Lesson:

Configuration and File Reporting

Logon to the Solution Manager Diagnostics in TT4 and perform the following analyses. 1-2-1

Call the Configuration and File Reporting Application. Call the Solution Manager Diagnostics http://:50000/smd In the menu choose “Root Cause Analysis” “Configuration” “Configuration and File Reporting”

1-2-2

Select the solution provided by trainer. Check for which timestamps configuration data is available. Load the configuration data for the most recent timestamp. Select the solution provided by trainer. Then select the server and click on the “Start” button. Check which timestamps are available in the listbox. Select the most recent timestamp and click on the “Load” button.

1-2-3

In the “View Configuration” screen search for the string “version.txt”. Go to the first occurance. What is the ConfigStore path and ConfigStore name? In the “View Configuration” screen enter the filename “version.txt” in the Search field and click on the “Search” button. In the following screen click on the “Go to next” button in order to find the first occurance. The ConfigStore path is “DVEBMGS50/j2ee/cluster/dispatcher/”. The ConfigStore name is “version.txt”. When was the file last changed? Check the “Date” entry in the end of the line for the ConfigStore name.

1-2-4

Click on the line to display the configuration parameters. Which of the parameters were changed? All parameters with the icon

were changed.

Mark a parameter which was changed and display the attribute history. When was the parameter changed and what was the previous value? Click on the button “Attribute History” to display the change date and the © SAP AG

E2E100

3-59

previous version. 1-2-5

Compare the complete configuration for the solution provided by trainer with an older version. Go back to the screen “Display landscape Production System of solution provided by trainer. Click on the “Compare” button. In Landscape 2 select an older timestamp and click on the “Compare” button. Now the screen “Select Subtrees for Compare” appears.

1-2-6

Perform a Deep Compare for a single ConfigStore. In the screen “Select Subtrees for Compare” select a single ConfigStore, e.g. “Path = DVEBMGS50/j2ee/”. Click on the button “Deep Compare”. In the next screen mark the same ConfigStore for landscape2, e.g. “ConfigStores Path = DVEBMGS50/j2ee/” and click on “Select”. If there are differences in the configuration versions a new screen “Compare Results” appears. It shows all ConfigStores with changed parameters.

© SAP AG

E2E100

3-60

Solutions

1-3

Unit:


Lesson:

Change and Transport System

A user complains about an error in the report ZZ_CHANGE_CONTROL_TEST in the system TT5. An error message comes up when the button “No” in the first popup screen is pressed. The error appears since 06.03.2007. 1-3-1

Which transport request was imported into TT5 on 06.03.2007? Go to the import history of system TT5 (STMS Overview Imports (F5) Doubleclick on the TT5 Import Queue Goto Import History (Ctrl+F7) Press the filter and enter the time interval: The name of the transport request is DEVK900019

1-3-2

Which user is the owner of this transport request? The owner of the transport request is WALZJ

1-3-3

Which objects does this transport request contain? Doubleclick on the transport request in order to navigate into the objectlist. The objectlist contains the following transport objects: LIMU REPS ZZ_CHANGE_CONTROL_TEST_Z_CHF01 (Source) LIMU REPT ZZ_CHANGE_CONTROL_TEST (Report Text)

1-3-4

Compare the new version of the imported report source with the previous version. Which line of coding was changed? Doubleclick on the name of the report source ZZ_CHANGE_CONTROL_TEST_Z_CHF01 in order to navigate into the ABAP editor. From the ABAP editor select Utilities Versions Version Management. Select the active version and the previous version and press the Compare button ( ). The following lines were changed: 64 CLEAR e_user_name. 64 * CLEAR e_user_name.

© SAP AG

E2E100

3-61

1-3-5

How are the objects in the transport request related with the report ZZ_CHANGE_CONTROL_TEST? Go back into the ABAP editor and put the cursor on the include ZZ_CHANGE_CONTROL_TEST_Z_CHF01. Press the “Where Used” button ( ). In the next screen select “Programs” and execute. You see that the report ZZ_CHANGE_CONTROL_TEST uses the include ZZ_CHANGE_CONTROL_TEST_Z_CHF01.

© SAP AG

E2E100

3-62

End-to-End Workload Analysis


© SAP AG


E2E100

4-1

End-to-End Workload Analysis

Contents: End-to-End Workload Analysis Using SAP Solution Manager ABAP Workload Analysis Java Workload Analysis Java Thread Dump Analysis Workload Analysis using CA Wily Introscope Java Memory Analysis Operating System Analysis Database and Storage Subsystem Analysis


© SAP AG

E2E100

4-2

End-to-End Workload Analysis: Unit Objectives

After completing this unit, you will be able to: Analyze a performance bottleneck using End-to-End Workload Analysis Analyze an issue with ABAP workload Analyze an issue with SAP J2EE workload Analyze operating system resources


© SAP AG

E2E100

4-3



E2E Trace Analysis







JAVA Trace Analysis










Functional Problem

© SAP AG 2006

© SAP AG

E2E100

4-4

Root Cause Analysis Roadmap (2)

Root cause analysis (Cross-Component)

?

Is it a performance problem? End-to-end workload analysis

?

Is it a functional problem?

?

System or application termination End-to-end exception analysis

?

Inconsistent data End-to-end data inconsistency analysis

?

Incorrect results or error messages in application End-to-end application root cause analysis


If the customer is facing a performance problem you can start with end-to-end workload analysis.

© SAP AG

E2E100

4-5

SAP Web AS Architecture (ABAP Stack)

1/3

SAP Web AS ABAP One infrastructure for ABAP applications Database with ABAP schema

Browser

ICM

ICM

Gateway Makes the RFC interface between the SAP instances available (within a SAP System and beyond system boundaries)

ABAP-Dispatcher Gateway

Handles web requests (http, https) or SMTP and calls the relevant local handler for the URL to serve Implemented as a separate process, which is started and monitored by the dispatcher Use transaction SMICM to monitor and administrate the ICM

WP

WP

Message Server Enqueue Server

ABAP Central Instance

Work process Each WP is single-threaded and corresponds to an OS process Shared memory is used to store common data Several types: Dialog / Update / Background / Enqueue / Spool Message Server Exchanges messages and balances the load in the SAP System

DB

Enqueue Server Maintains locks on objects within transactions + ensures data consistency by controlling the concurrent accesses to objects © SAP AG 2006, Title of Presentation / Speaker Name / 1

Work Process A process that as a component of an application server executes an ABAP application. To process SAP requests from several front ends, an SAP application server has a dispatcher, which collects the requests and forwards them to work processes for execution. There are the following types of work process: - Dialog: For executing dialog programs - Update, Upd2: For asynchronous database updates - Background (batch): For executing background jobs - Enqueue: For executing lock operations - Spool: For print formatting Work processes can be assigned to dedicated application servers. In the service overview (SM51), you can see which work process types are provided by the individual servers. Each work process is logged onto the database system as a user for the entire runtime of the SAP system. Each work process is assigned for the duration of a dialog step to an ABAP program.

© SAP AG

E2E100

4-6

SAP Web AS Architecture (JAVA Stack) SAP Web AS Java One infrastructure for Java All kinds of Java & J2EE applications Database with Java schema Message Server Handles communication between J2EE server nodes The only component that knows all the participating J2EE server nodes Server nodes communicate exclusively via the Message Service Enqueue Server Applications can request locks for read/write access to logical objects (i.e. Java objects) from the Enqueue Service Locking of logical objects in front of the database is usually more efficient than locking database records directly Software Deployment Manager Manage and deploy Software Packages - Software Deployment Archives (SDAs) and Software Component Archives (SCAs) that you receive from SAP Central Services Cluster wide communication Central locking objects

2/3 Browser HTTP

Java-Dispatcher Server Node

SDM

Server Node

Java Central Instance

Message Server Enqueue Server Central Services

DB


SAP Web AS Java Provides an installation option to have one infrastructure for Java for all kinds of Java + J2EE applications. Java dispatcher and server are running each on a separate Java Virtual Machine (JVM) instance The database has only a Java schema

© SAP AG

E2E100

4-7

SAP Web AS Architecture (Double Stack) Browser

ICM MPI

HTTP

Java-Dispatcher

ABAP-Dispatcher Gateway

SAP Web AS ABAP + Java Common installation, configuration and monitoring All kinds of ABAP, Java & J2EE applications Database with ABAP and Java schema JCO Enables communication between ABAP and Java Inbound messages: Java calls BAPIs and RFC-enabled function modules (RFM) on the ABAP side Outbound messages: ABAP calls functions on the Java side Communication is based on Remote Function Call (RFC) JCo supports synchronous, transactional, and queued RFC

3/3

WP

WP

Message Server

JCo

Server Node

SDM

Server Node

Java Central Instance

Enqueue Server ABAP Central Instance

Message Server Enqueue Server Central Services

DB


The picture above shows the architecture elements of an double stack installation.

© SAP AG

E2E100

4-8

End-to-End Workload Analysis: ABAP vs. Java ABAP Work Processes vs. Java Threads Java VM (jlaunch)

SAP ABAP Application Server

Customer A

Shared memory WP dw.exe

WP

WP

WP Shared write access Must be synchronized

... Thread-1

myThread1 Thread-17

new Thread(r)

CALL FUNCTION IN NEW TASK Fixed number of work processes

Threads created and destroyed on demand


Java and the SAP Web Application Server have completely different process models: SAP Web AS has a fixed number of work processes (configured by profile parameters). Each of the work processes is single-threaded and corresponds to a separate OS process. Shared memory is used to store common data. If concurrency is needed, a new task is run in a different work process (and therefore in a different OS process). Java provides concurrency within a single OS process by means of threads. Each thread is a flow of execution. Threads are created and destroyed on the fly on demand. Since multiple applications may run in the same OS process in Java, one badly behaved application may impact all other applications in the same Java Virtual Machine. Note: For SAP Web AS, a single work process can be debugged and restarted without impact on other processes or applications.

© SAP AG

E2E100

4-9

End-to-End Workload Analysis: ABAP vs. Java

Workload Analysis Area

ABAP

Solution-wide workload analysis

Java End-to-End workload analysis

System-wide ST03N, STAD performance statistics

Wily Introscope

Operating system analysis

ST06

Wily Introscope

Database analysis

ST04

ST04

Current application activity

SM50, SM66 (work process overview)

Thread dump analysis

Memory usage

ST02

Java memory analysis

© SAP AG 2006

© SAP AG

E2E100

4-10

End-to-End Workload Analysis: Example End-to-end Workload Analysis example with an Employee Self-Service (ESS) scenario Several employees of a company have complained about high system response times during the usage of ESS scenarios over the last few days. The company support organization is informed about the problems and is starting root cause analysis using SAP Solution Manager. To begin the analysis of the problem, the end-to-end workload analysis application will be used to locate the root cause. Browser

WAN Network

Presentation Logic

Enterprise Portal

Application Logic

Database

ECC


The Employee Self-Service scenario is distributed over physical hosts with installations of SAP Enterprise Portal and mySAP ERP Human Capital Management as a back-end system. End-to-end workload analysis should find out which technology component the problem is related to.

© SAP AG

E2E100

4-11

Fact Sheet: E2E Workload Analysis Level 2 (must know)

Use Case Get workload information of your complete solution Analyze overall performance bottlenecks Provides Key performance indicators for different components Best Practice for Investigation / Rules of Thumb

Check the workload overview for bad response times Choose „Portfolio“ view and search for „bubbles“ with high average response time and high values of accumulated times. They indicate performance bottlenecks (high workload in conjunction with high av. Response time) Check the component specific KPIs for deviation from average values

© SAP AG 2006

Limitations: aggregation level is 1 hour (no ad hoc analysis) Prerequisites: Solution Manager 4.0 SP10

© SAP AG

E2E100

4-12

End-to-End Workload Analysis: Start Select time frame

Component-specific workload metrics

Select diagram type

Workload metrics overview


To start an end-to-end workload analysis, log on to Solution Manager and execute transaction DSWP (solution_manager). Then select Goto and choose Start Solution Manager Diagnostics. In Solution Manager diagnostics, choose Workload E2E Workload Analysis/. Choose your solution and select all components. Click Start to begin with your investigation. The figure above shows the entry screen with feature description. Here you can get a good overview of the workload in your solution for the chosen time frame. The values of the diagram can be displayed in different formats: Scatter: x-axis shows the accumulated response time; y-axis shows the average response time ( to help identify application scaling problems) Portfolio: x-axis shows the average response time; y-axis shows the hour of day; size of the balloon represents the accumulated response time. Critical situations can be identified by high ‚flying‘ big balloons.

© SAP AG

E2E100

4-13

Portfolio: „Rules of Thumb“ Accumulated Response time = load on the system night hours with low load on the system

1 sec

Av. Response Time

time range

t

Working hours with higher load Hardware is correctly sized and able to handle high load on the system

5 sec

Performance problem: high average response time + high accumulated response time

4 sec

Expensive application is executed not often

2 sec

Working hours with high workload Hardware is not correctly sized and not able to handle high load. As a result Av. Response Time is going higher

1 sec

t © SAP AG 2006, Title of Presentation / Speaker Name / 1

© SAP AG

E2E100

4-14

End-to-End Workload Analysis: Example (SAP ECC) Checks to identify general performance problem on SAP ECC system

Depends on customer requirements (no general rule)

Average CPU time < 40 % of response time (wait time)

Average DB time < 40 % of response time (wait time)

Wait time < 10% of response time

Average roll wait time < 200 ms


If a problem is detected, the data in the workload monitor (transaction ST03N) can be used to identify the area of the system where the problem is located. First check for general performance problems affecting all transactions. Good general performance is normally indicated by: Wait time < 10% response time Average roll-in time < 20 ms Average roll-wait time < 200 ms Average load (and generation) time < 10 % of response time (< 50 ms) Average database request time < 40 % of (response time - wait time) Average CPU time < 40 % of (response time - wait time) Average CPU time not much less than processing time Average response time depends on customer requirements; there is no general rule

© SAP AG

E2E100

4-15

End-to-End Workload Analysis: Example (SAP ECC) (2)


DIALOG: Transaction step is normally started online by the user (such as editor dialogs or manual postings).

© SAP AG

E2E100

4-16



UPDATE: Transaction step of the SAP update process; it is automatically started by the dispatcher on a host with an active update process (update processes are usually installed on the database server).

© SAP AG

E2E100

4-17



© SAP AG

E2E100

4-18



© SAP AG

E2E100

4-19

End-to-End Workload Analysis: Example (EP)

Checks to identify general performance problems on Enterprise Portal system

Check for high average response times in correlation to high number of executions per second


The above figure shows the workload summary for an Enterprise Portal. By default you can see three metric types - iViews, Servlets, and Web Dynpro Applications with two key figures - Average Response Time and Executions.

© SAP AG

E2E100

4-20

End-to-End Workload Analysis: Example (EP) (2)

Set filter options for more selective data view

Click


To add more key figures, enter the Navigation section and choose additional key figures such as Executions and Measuring Interval. Click on Transfer to have the new key figures displayed in the Metric Data Display.

© SAP AG

E2E100

4-21

End-to-End Workload Analysis: Architecture Component-specific workload analysis

ABAP

Cross-component workload analysis

Reporting / EarlyWatch Alert

RFC Interface

e.g., R/3, BW, XI, APO, CRM

Java

Diagnostics Agent

SAP Business Intelligence

e.g., IPC

J2EE

Diagnostics Agent

e.g., Portal, JavaWD, XI, BillerDirect, CRM-ISA

.Net

Introscope Enterprise Manager

Diagnostics Agent

SAP Solution Manager (J2EE / ABAP stack)

e.g., Duet, .Net PDK, MobileClient

C / C++

Diagnostics Agent

e.g., TREX, ITS, Business One, MDM

Statistic performance data for ABAP Aggregated performance data for non-ABAP


To analyze the workload of an solution consisting of distributed technology components, it is necessary to install a Diagnostic Agent for each satellite. The Diagnostics Agent aggregates and transfers online diagnostics data to the Introscope Enterprise Manager (EM) every 15 seconds. On ABAP-based components, the workload related data is extracted from statistical files and is written in BI. The data from Enterprise Manager is extracted hourly to BI.

© SAP AG

E2E100

4-22



The time profile shows the most important workload metrics for a chosen time frame in graphical form.

© SAP AG

E2E100

4-23



Check for iViews with high average response time and a high number of executions per second

© SAP AG

E2E100

4-24



Check for expensive SQL statements.

© SAP AG

E2E100

4-25



A Java-based system interacts with an ABAP-based system via Java Connector (JCo). Long-running JCo calls point to a possible workload problem on the called system.

© SAP AG

E2E100

4-26



Check the CPU and memory utilization of the selected system.

© SAP AG

E2E100

4-27

End-to-End Workload Analysis Roadmap

End-to-end workload analysis

?

System components with unusually high response time or throughput Component workload analysis

?

Single transaction or client or network not performing End-to-end trace


The next steps of the root cause analysis process depend on the results of the end-to-end workload analysis.

© SAP AG

E2E100

4-28

End-to-End Workload Analysis Roadmap (2)

End-to-end workload analysis

?


?

Single transaction or client or network not performing End-to-end trace


© SAP AG

E2E100

4-29

General Component Analysis Roadmap

Component workload analysis

?

Is it a Java component? Java component workload analysis

?

Is it an ABAP component? ABAP component workload analysis

?

Is it a database problem? Database workload analysis


© SAP AG

E2E100

4-30

General Component Analysis Roadmap (2)


?


?


?



© SAP AG

E2E100

4-31



E2E Trace Analysis







JAVA Trace Analysis










Functional Problem

© SAP AG 2006

© SAP AG

E2E100

4-32

Java Workload Analysis Roadmap

Java workload analysis Average response time per user click not acceptable

? !

Operating system analysis ?

CPU time of the Java processes is unexpectedly high Analyze CPU consumption with Wily Introscope

?

High disk utilization Analyze disk utilization

!

SAP J2EE analysis ?

High memory utilization by Java processes (close to max heap after full GC) Java memory analysis

?

Threads are blocked or in deadlock situations Java thread dump analysis

?

High workload in specific Java application Workload analysis with Wily Introscope


© SAP AG

E2E100

4-33

Java Workload Analysis Roadmap (2)


? !


?


?


!

SAP J2EE analysis

?


?


?



© SAP AG

E2E100

4-34

Java Workload Analysis: Example

Operating system resource consumption check with Wily Introscope The support organization of the company finished the endto-end workload analysis and identified a Java-based system (for example, Enterprise Portal) as root cause for the performance problems. The first step is to exclude a hardware bottleneck as root cause for the poor performance. Support experts use the Wily Introscope tool to check the operating system resource consumption.


Wily Introscope is integrated into the diagnostics part of Solution Manager. Operating system related data can easily be checked with predefined dashboards.

© SAP AG

E2E100

4-35

Fact Sheet: Wily Introscope Level 2 (must know)

Use Case Ad hoc analysis of resource consumption and system behavior Analyze health status of the system Provides Live and historical data in graphical form Best Practice for Investigation / Rules of Thumb get a system status overview via the dashboards check the component specific dashboards check specific values in the investigator mode


Limitations: application specific data available for systems based on Java and .NET Prerequisites: Byte code has to be instrumented Information resources for Introscope - SAPNet quicklink /diagnostics - SAP Note 797147 - SAP Service Marketplace

Downloads Installations and Upgrades SAP Technology Components SAP SOLUTION MANAGER SAP SOLUTION MANAGER 4.0 SAP Solution Manager Diagnostics WILY INTROSCOPE AGENT 7 WILY INTROSCOPE ENTPR MGR 7 WILY INTROSCOPE WEBVIEW 7 - CSS component XX-PART-WILY for customer messages WebView requires the SVG browser plug-in to display performance data.

© SAP AG

E2E100

4-36

Introscope Architecture

Diagnostics Agent

Level 3 Expert

Introscope Workstation

Real-time, historical, and performance alerting

Introscope Enterprise Manager

Introscope WebView Level 2 (must know)

Persistent store


Introscope Enterprise Manager Central persistent repository for all Introscope performance data Installed by default on Solution Manager diagnostics machine in directory /usr/sap/ccms/wilyintroscope Should run as Windows service or UNIX background process Introscope WebView Web-based graphical UI to display Introscope performance data Deployed on the Solution Manager diagnostics J2EE Engine, a link to WebView is available in Solution Manager diagnostics navigation Introscope Workstation Swing-based UI with two major components: the Console and the Explorer

© SAP AG

E2E100

4-37

Introscope Web View

J2EE Server Analysis / Wily Introscope


Customers and partners have to create an own user as described in the Introscope Setup guide (http://service.sap.com/instguides).

© SAP AG

E2E100

4-38

J2EE Server Analysis: WebView Basics

After logging on to WebView, you will see the main WebView screen. Main functions include: Console Customized views on performance data

Investigator Tree view of all available data Transaction Viewer Perform and display traces for individual users


© SAP AG

E2E100

4-39

J2EE Server Analysis: WebView Basics (2)

Introscope WebView console

Click

Choose dashboard from list


Dashboards are predefined views on metric data presented in graphical form. They can also be combined with traffic lights to give a quick overview of the current status of the solution. Use the dropdown list to select a dashboard and then choose View. Organization of the dashboards: Dashboards are hyperlinked Overview dashboard in each module (traffic lights) Triage dashboard in each module (several small charts with key performance indicators) Topic-specific dashboards: E.g., JDBC, JCO Zoom dashboards: Enlarge individual diagrams Dashboard numbering indicates the level of analysis: Higher number = more detail 0 – Site map 1 – Overview (links and traffic lights only) 2 – Triage (Small charts and traffic lights, covering large area like EP) 3 – Topic overview / detail (few charts, typically, e.g. JCO, JDBC)

© SAP AG

E2E100

4-40

J2EE Server Analysis: WebView Usage Traffic lights are hyperlinks to a more detailed view

Click on traffic light


On each dashboard there are graphic objects, text objects, and data viewers. Data viewers are used to show data in metrics or metrics groupings, or as represented by elements such as alerts. Graphic and text objects are used to create a layout that helps users to scan the dashboard, and clarify the meaning of data shown by data viewers.

© SAP AG

E2E100

4-41

J2EE Server Analysis: WebView Usage (2)

Console lens: Select one or more monitored J2EE nodes for the Console (CTRL+Click to select more than one)


If the connected solution consists of many monitored systems the graphical output could become difficult to read as all agents are presenting their data in the same view. For this reason, you can select a single agent or a smaller group of agents via the Lens function.

© SAP AG

E2E100

4-42


Live / historical data Toggle between live data and historical data Traffic lights are not active for historical data By default, 30 days of history are available Old data is more “coarse-grained” and condensed, with fewer data points


With this function, you can display historical data: Refresh the historical data Change historical range options Zoom in on specific time frames of the data

© SAP AG

E2E100

4-43


Useful functions in the context menu Zoom control Download raw data as .csv file (can be opened in Microsoft Excel) Details view: Aggregated data for all curves in a diagram


Right-click to reach the context menu for a specific data viewer. You then have access to zoom options and a menu to download data.

© SAP AG

E2E100

4-44


Response times are always measured in milliseconds A response time of 1K equals 1 second

Clicking on the line leads you to the location in the explorer tree


SVG – Scalable Vector Graphics, Adobe Plug In for the browser Context Menu: Zoom in/ zoom out CTRL+click: Zoom in Shift+CTRL+click: Zoom out CTRL+drag mouse: Open a rectangle to zoom in ALT+mouse: Drag the zoomed view

© SAP AG

E2E100

4-45

J2EE Server Analysis: WebView Basics (4)

Site map


Dashboard organization: Dashboards are organized by applications (EP, XI, CRM, …) Dashboards are hyperlinked Overview dashboard in each module (traffic lights) Triage dashboard in each module (several small charts with key performance indicators) Topic-specific dashboards, for example, JDBC or JCo Zoom dashboards enlarge individual diagrams

© SAP AG

E2E100

4-46

J2EE Server Analysis

Start OS resource analysis with 1. SAP J2EE Overview dashboard

Click


The OS resource analysis can be started with dashboard number 1 (SAP J2EE Overview). In this dashboard view, you have several traffic lights that indicate the state of your system at OS level, such as CPU and memory consumption.

© SAP AG

E2E100

4-47

J2EE Server Analysis (2)

Get a more detailed view with “3. SAP J2EE System Detail” dashboard

Check for bottleneck


Please check the third dashboard (SAP J2EE System Detail) to get a more detailed overview of OS resource consumption. Clear signs of a bottleneck would be CPU consumption of 95% to 100% over a long time, or heap usage close to the maximum heap.

© SAP AG

E2E100

4-48

J2EE Server Analysis: WebView Investigator

Investigator mode


For deeper analysis it is recommended that you to check OS-relevant data in the Introscope Investigator view. Open the Investigator view from Introscope WebView by clicking on Investigator in the top-level menu.

© SAP AG

E2E100

4-49

J2EE Server Analysis: WebView Investigator (2)

Preview pane

Tree pane © SAP AG 2006, Title of Presentation / Speaker Name / 1

The figure above shows the general structure of the investigator. You can navigate in the tree pane on the left side and choose the data that you want to see in graphical diagram form in the preview pane.

© SAP AG

E2E100

4-50

J2EE Server Analysis: WebView Investigator (3)

Metric tree items

Host Process Agent Resource Metric Called resource


Here you can see the structure of the metric tree. You navigate in the tree by choosing the host, then the monitored process, followed by the agent name. Below the agent, all instrumented resources are visible. Open the Resources folder to see the different metric views.

© SAP AG

E2E100

4-51

J2EE Server Analysis: OS Resource Analysis Check in Investigator view for metric tree item SAP OsCol

Check for bottleneck in OS resources


You can find OS-related metrics such as CPU consumption in the SAP OsCol folder.

© SAP AG

E2E100

4-52

Analyze High Java process CPU Utilization

Analyze high J2EE CPU utilization

?

Java process consumes unexpectedly high CPU (>90%)

?

Are there other Java processes with low CPU utilization in the J2EE cluster? Check the load balancer behavior

?

Is there only one Java instance and a Java process consumes > 90% CPU or consumes unexpectedly high CPU? Follow the steps in SAP Note 742395 to analyze the high CPU and forward the message to SAP


You can check the CPU utilization for each OS process via the OS command console. Open the OS command console from Solution Manager diagnostics. Then choose the command group System and the command ProcessStat. The tool displays all processes on OS level with the same detail as from ‘Task Manager‘ after a click on Send Command. A sign of a possible problem with the load balancer would it be if you find a application server jlaunch process with a much higher CPU consumption then the other application server jlaunch processes in the cluster. If just one application server process or all application server processes have the same high CPU consumption, it is necessary to check SAP Note 742395 to continue the analysis.

© SAP AG

E2E100

4-53

Check the Load Balancer Behavior

Are the HTTP sessions well distributed?


The load distribution for a J2EE cluster can be checked by checking the amount of HTTP sessions per J2EE application server node. You can do this in the Wily Introscope Console with, for example, dashboard number 3 (SAP J2EE Servlets Detail). In a well-configured system the number of HTTP sessions should not differ significantly from one J2EE server node to the next. Keep in mind that there will be some differences between server nodes because of session stickiness (to keep the user context) most of the time. Here we are checking for a significant difference. Stickiness: When a user first logs on the load balancer should identify that the user does not have an existing session and load balance them to a WAS java dispatcher. The dispatcher will then load balance the new user session to one of it's server nodes. For future requests (within the same session), the load balancer should identify that you have an existing session and pass it to the same dispatcher (known as stickiness). The dispatcher will in turn pass it to the same server node. So once you have a session, you should be sent to the same server node. The stickiness can be ensured by different techniques like IP address based or via a session cookie.

© SAP AG

E2E100

4-54



? !


?


?


!

SAP J2EE analysis

?


?


?



© SAP AG

E2E100

4-55

Java Workload Analysis: Disk Utilization

Check disk utilization via Introscope


You can use Wily Introscope to check the disk utilization on your system. High disk utilization leads to long response times and should therefore be optimized.

© SAP AG

E2E100

4-56

Java Workload Analysis: Disk Utilization (2)

Possible root causes for high disk utilization: Severity level for logging / tracing too high – check with log configuration

Diagnostics

Archiving of log files – check with file system browser

Occurrence of many messages (e.g., exceptions) – check with log viewer


The figure above shows some possible reasons for high disk utilization.

© SAP AG

E2E100

4-57



? !


?


?


!

SAP J2EE analysis

?


?


?



© SAP AG

E2E100

4-58

Memory Analysis

Memory/capacity analysis ?

High memory utilization by Java process (close to the max heap after full GC)

?

Other java processes in the J2EE cluster consume significantly low amounts of memory Analyze the load balancer behavior

?

One Java instance available and the Java process utilizes memory close to the max heap after full GC Analyze memory leak conditions (potential OOM issues)


© SAP AG

E2E100

4-59

Memory Analysis (2)

Memory/capacity analysis ?

High memory utilization by Java process (close to the max heap after full GC)

?

Other java processes in the J2EE cluster consume significantly low amounts of memory Analyze the load balancer behavior

?

One Java instance available and the Java process utilizes memory close to the max heap after full GC Analyze memory leak conditions (potential OOM issues)


© SAP AG

E2E100

4-60

Java Workload Analysis: Example Java Memory Analysis with Solution Manager The support organization from the company completed the operating system analysis and have excluded a hardware bottleneck as root cause for the performance problems. They now proceed with checking the garbage collection behavior and memory consumption of the SAP J2EE, as these are performance-critical areas of a Java-based system.


Java memory analysis is integrated into the diagnostics part of Solution Manager.

© SAP AG

E2E100

4-61

Java Memory Analysis: Ad-Hoc GC Analysis (4)

Permanent increase of memory after full GC

Looks like a memory leak

Will lead to an OutOfMemory crash shortly!


This is an example of an memory leak ending with an out-of-memory condition in the system. There is a marked, constant decrease of available memory, even after several full GC cycles.

© SAP AG

E2E100

4-62

Java Memory Analysis: Basics Garbage Collection: Basics (JVM memory management) The Java Virtual Machine has two primary jobs: Execute Java coding Manage memory Memory management: JVM directly allocates memory from OS JVM distributes memory to the Java applications internally JVM manages memory allocation / deallocation Memory allocation occurs implicitly on object creation (is done by the JVM itself and hidden from the programmer) Memory deallocation is executed by a special Java agent – the Garbage Collector (GC) – which is a part of the Java VM.

© SAP AG 2006

Objects become garbage when they are no longer reachable from the root set. The root set consists of: Static reference fields in classes Local references The parameters shown above are just valid for SUN and HP. IBM has different names for the parameters.

© SAP AG

E2E100

4-63

Java Memory Analysis: Garbage Collector Analysis user address space of the Java process

Perm space

Java heap

-Xms / -Xmx

Spaces Survivor

Eden (young generation)

-XX:PermSize, …thread local (stack, etc.)

DLLs

Old (tenured) generation

-XX:NewSize, …

Note: The organization into Young and Old generations as shown here is used in the SUN, HP, and IBM J9 Java VM.


Perm space is reserved for Java VM, and the objects from the application should be always in the memory. The Java heap in Windows should not be more then 1.7 GB (it should normally be 1 to 1.2 GB). In the Eden area, objects are stored that are used and may be deleted. Eden is cleared during minor GC, and old generation are cleared during full GC. Note: IBM JVM 1.4 has a hybrid approach and you can choose between J9 and Sovereign. The IBM Sovereign VM has a different architecture concept and therefore J9 is recommended for SAP applications.

© SAP AG

E2E100

4-64

Java Memory Analysis: Garbage Collector Analysis (2)

The JVM (SUN, HP) has two types of GC cycle: When the new generation fills up, it triggers a minor collection, in which surviving objects are moved to the tenured generation. When the tenured generation fills up, it triggers a major collection, which involves the entire object heap.

Regular GC (minor collection)

Full GC (major collection)

Queries only newly created objects before clearing those without references. This is performed frequently and has a relatively short duration (~0.2 seconds).

Queries all the objects in the heap before clearing those without references. Full GC is less frequent, and can be very time consuming, between 3 and 5 seconds per cycle. On very large heaps (>1.5G) a full GC cycle can take more then 10 seconds.


The garbage collection mechanism is responsible for marking and deleting objects that are no longer in use. During a collection cycle, objects that are no longer referenced are marked for deletion and deleted, and the space that they occupied is returned to the heap. Garbage collection is a process that is constantly active during runtime of a Java program, and shares the same resources, such as CPU time and address space. As such, it has significant influence on the performance of the Java program.

© SAP AG

E2E100

4-65


user address space of the Java process

Perm space

Java heap

-Xms / -Xmx

Spaces Survivor

Eden (young generation)


DLLs

Old (tenured) generation

Scanned area


The red bar in the figure above shows the scanned memory area during a minor garbage collection. The red arrows represent the direction in which object are copied.

© SAP AG

E2E100

4-66


user address space of the Java process

Perm space

Java heap

-Xms / -Xmx

Spaces Survivor

Eden


DLLs

Old generation

Scanned area


During a full garbage collection cycle, the complete heap space will be scanned. This is one reason for the long time needed for a full GC cycle.

© SAP AG

E2E100

4-67

Java Memory Analysis: Ad-Hoc GC Analysis GC example: Normal operation

File upload capability Selection time interval

Higher activity during work hours Selection J2EE nodes System crash caused by OutOfMemeory

Low activity in night hours

© SAP AG 2006

In Solution Manager diagnostics, you can access Java memory analysis by choosing Workload JAVA. Java memory analysis presents the GC behavior of the chosen Java instance during a selected time frame. The figure above shows normal behavior. The memory is allocated until a GC cycle starts and deallocates memory. What do I get? Online monitoring of abnormal aggregated and non-aggregated GC behavior data per JVM / J2EE server node Reporting of derived KPIs to characterize the healthiness of a JVM / J2EE server node, and conclusion regarding healthiness of J2EE cluster How does it work? If appropriate JVM setting is implemented, the GC behavior information is written in standard output file (format depending on JVM vendor) Ad-hoc GC analysis: - JavaMemoryAnalysis application triggers the diagnostics agent application to transfer standard output file to central Solution Manager diagnostics system - Standard output file is scanned by JavaMemoryAnalysis and GC behavior information can be displayed in Solution Manager diagnostics Permanent GC analysis: - Standard output file is scanned on a regular basis by Diagnostics agent and GC behavior information is transferred to IS Enterprise Manager - GC behavior information can be transferred from IS EM to Solution Manager BI to conduct GC KPI reporting (EWA / IT reporting) © SAP AG

E2E100

4-68

Java Memory Analysis: OutOfMemoryError 2

181974.276: [Full GC 181974.276: [Tenured: 856089K->856089K(873472K), 1.1814274 secs] 916052K->916050K(1004800K), [Perm : 128742K->128741K(262144K)], 1.1815444 secs]

FATAL: Caught OutOfMemoryError! Node will exit with exit code 666 getThreadDump : Mon Feb 12 10:03:14 2007

std_server0.out

1 1 2

In the GC diagram you can see that the system is busy with Full GC In the std_server0.out file you can see that after Full GC the JVM is not able to free Java Heap or/and Perm Space and as a result OutOfMemoryError is caught

© SAP AG 2006

© SAP AG

E2E100

4-69

Java Memory Analysis: Ad-Hoc GC Analysis (3) GC analysis: Statistics view

GC time vs. elapsed time? % of full GC vs. minor GC? Recycled bytes may be useful for comparison Average GC / full GC times? Available memory going down? Adequate evaluation may point you to A memory leak Too-high memory consumption © SAP AG 2006, Title of Presentation / Speaker Name / 1

You can see statistics about the system GC behavior by clicking on Details View. A productive EP system should show the following characteristics: GC time vs. elapsed time should be <= 10% % of full GC vs. minor GC: full GC should be much less frequent What you get from GC analysis: Find out if Java memory is a problem Find out if GC is the reason for poor performance What you do not get: Find out who is consuming memory Find the reason for a memory leak

© SAP AG

E2E100

4-70

OOM Reasons: User Session Overload Out of memory!

Effective heap size

User sessions

caches

runtime time Startup phase

Warm-up phase

Normal operation phase


Abstract and idealized: How to determine when the system is “warm“ Startup already fills and uses some caches Currently, we cannot differentiate the different heap areas

© SAP AG

E2E100

4-71

OOM Reasons: Cache Oversizing Out of memory!

Effective heap size

User sessions

caches


Warm-up phase




© SAP AG

E2E100

4-72

OOM Reasons: Application Leak Out of memory!

Effective heap size

User sessions caches


Warm-up phase




© SAP AG

E2E100

4-73

Java Memory Analysis: Ad-Hoc GC Analysis (2)

Java memory analysis: Events Overview

On click


The Events Overview function displays important events in graphical form for a chosen time frame. If you click on an event, a table view will appear with a detailed event description below the graphic.

© SAP AG

E2E100

4-74

Java Memory Analysis: Permanent GC Analysis

Permanent GC analysis is integrated into the E2E Workload Analysis tool.

Should be < 5%

Should be >>prom. rate

Should be approx. 50%

Should be < 5%

Should be << alloc.rate


The goal of the data shown in the Java Memory Usage section of the end-to-end workload analysis tool is to quantify and analyze dynamic GC behavior. The overall result shows the average of all other values and is therefore meaningless and will be deleted in the next version All displayed values will be explained in detail on the following slides The percentage of GC time (Last 5 minutes) is a fluently calculated value that shows the percentage of GC time over a time frame of 5 minutes (detailed explanation on the next slide).

© SAP AG

E2E100

4-75

GCA KPIs: GC Time per Time Interval (Definition)

Time spent in garbage collections is one of the most important KPIs Directly measures GC performance impact Is calculated by dividing overall GC duration time by time of considered time interval Time interval spans from first to last GC Time spent in GC

Time free for processing ti

GCi - 10

GCi

ΔT Time interval

KPI should be small (e.g., < 5%); if parameter is increased, it is an indicator of critical situations © SAP AG 2006, Title of Presentation / Speaker Name / 1

One of the most important parameters used to assess GC behavior is the total time spent in garbage collections: This parameter directly measures the performance impact on the JVM by the garbage collections. Time spent in a GC is not available as free processing time. Increased GC time is usually indicative of critical situations in general. This parameter is calculated using the sliding window approach, described above. On the interval of a fixed number of garbage collections, the sum of durations of all GCs in that interval is calculated. The time is then divided by the total real time interval of the sliding window. This time interval lasts from the beginning of the first GC to the end of the last GC in the interval. We can deduce that this dimensionless parameter will be large if each GC takes a substantial amount of time and the remaining time free for processing is small. This indicates two phenomena: long GC duration and frequent GCs. The reason for both is worth further investigation.

© SAP AG

E2E100

4-76

GCA KPIs: Ratio FGC to All GCs (Definition)

Percentage of Full Garbage Collections - fairly robust KPI For a “well behaved” JVM, minor garbage collections (minor GC) should be much more frequent than full garbage collections (full GC). Consecutive full GCs are typical for a GC cycle that is out of balance. On each considered time interval, the number of full GCs is divided by total number of GCs (minor GCs + full GCs). ti GCi - 10

GCi

NFGC = 4 NGC = 11 FullGC

RFGC = 37 %


A fairly robust parameter is the percentage of full garbage collections on a given GC interval. This parameter is motivated by the fact that for a well behaved JVM, minor garbage collections must be much more frequent than full garbage collections. Anything else is contradictory to the concept of generational GC. On the other hand, it is obvious that any number of consecutive full GCs is critical, as it implies that the garbage collector is trying to remove objects from old space and failing – which triggers the next full GC if the criteria for a full GC are still fulfilled. Thus, a high percentage of full GCs are typical for a GC cycle that is out of balance. To calculate this parameter, use a sliding window approach to determine the fraction of full GCs on the interval of all GCs.

© SAP AG

E2E100

4-77

GCA KPIs: MAX OGR (Definition)

Old space growth rate (OGR): Catches phases of excessive old space growth Trend analysis of memory available after full GC KPI should not grow excessively

Non-critical

Memory after full GC

Memory after full GC

Implication is that objects cannot be removed from old space

Critical

t

t Linear Trend Analysis

If persistently larger than zero, you have a critical situation © SAP AG 2006, Title of Presentation / Speaker Name / 1

In general, resource consumption is difficult to quantify for dynamic systems such as the JVM GC cycle. However, it is clear that a healthy GC cycle is characterized by the fact that on average, objects stored in the old space have sufficient time to “die” and thus can always be removed. Apart from a static residue and minor growth trends, the old space used should not grow over time. The opposite is true for critical situations. Here we often observe objects being accumulated in the old space even after a full GC. The old space growth rate (OGR) parameter tries to catch such growth phases by performing a linear trend analysis on the memory available after a full GC. This parameter should not grow excessively. Depicted here are two examples, a non-critical one on the left and a critical one on the right. The memory after GC as a function of time is displayed. You can identify the memory after a full GC by the red bullets. For the further analysis, remove the original curve of all GCs in favor of the full GC only On this reduced data set, you can then perform a simple linear trend analysis. The slope of the resulting straight line is the parameter OGR. It is clear that it should be small: almost zero in the non-critical case, yet positive in the critical case. Studies have shown that a distinction can be made between critical and non-critical situations based on OGR.

© SAP AG

E2E100

4-78

GCA KPIs: PR / AR and GR / YR (Definition)

Calculation of memory rates (based on example by SUN) AR = Allocation Rate: Memory load created by applications PR = Promotion Rate: Memory shifted from young to old generation GR = Garbage Loss Rate: Memory actually removed as garbage YR = Young Generation Loss Rate: Memory removed from young generation

Requests

AR

Young Generation

YR

PR GR

RMI

Old Generation

Heap Minor Garbage Collection


© SAP AG

E2E100

4-79

Java Workload Analysis Roadmap


? !

Operating system analysis ?


?


!

SAP J2EE analysis ?


?


?



© SAP AG

E2E100

4-80

Java Memory Analysis: Roadmap Java workload analysis ?

Is there really a memory-related problem? Is the garbage collector behavior okay? Is there a general memory problem or an application-specific memory problem? Does the memory configuration have enough headroom for expected load increase and peaks? Is there an aging observable over time (increased memory usage and higher GC overhead)? Garbage Collector analysis based on IntroScope, Java standard Output, and JMX data

?

What amount of memory is occupied by what area (session sizes, cache sizes, and application sizes)? What are the fastest growing memory areas? What classes are consuming the most memory? What are the fastest growing classes? In case of memory leaks, what area or class is leaking? IntroScope LeakHunter, IntroScope Instance Counter, Class histogram analysis

?

How much memory is allocated by a single request (memory profiling)? What specific method causes the generation of instances in a leaking class or collection (stack trace analysis)? What is the typical content of a user session (session browsing)? In case of memory leaks, why is a certain class leaking? What is the programming logic behind it? Session browsing tools, memory profiling / tracing tools, and heap dump Analysis


Java memory analysis can be divided into the tree sections, which you can see in the above figure. The goal for the first section, Memory Problem Detection and Classification, is it to answer the following questions: Is there really a memory-related problem? Is the garbage collector behavior okay? Is there a general memory problem (for example, a less-configured heap size) or an applicationspecific memory problem (for example, leaking class)? Does the memory configuration have enough headroom for expected load increase and peaks? Is there an aging observable over time (increased memory usage and higher GC overhead)? The goal for the second section, Quantitative Memory Analysis, is it to answer the following questions: What amount of memory is occupied by what area (session sizes, cache sizes, and application sizes)? What are the fastest growing memory areas? What classes are consuming the most memory? What are the fastest growing classes (instance number or occupied bytes, for example)? In case of memory leaks, what area or class is leaking? The goal for the third section, Qualitative Memory Analysis, is it to answer the following questions: How much memory is allocated by a single request (memory profiling)? What specific method causes the generation of instances in a leaking class or collection (stack trace analysis)? What is the typical content of a user session (session browsing)? In case of memory leaks, why is a certain class leaking? What is the programming logic behind it?

© SAP AG

E2E100

4-81

Expert Analysis of Java Memory Problems

Deeper root cause analysis of Java memory problems has to be done with expert tools like:

Level 3 (Expert)

Wily Introscope LeakHunter IntroScope estimates the fastest growing collections based on heuristics

Class Histogram analysis Identification of critical objects is based on multiple memory snapshots

Heap dump analysis Identification of root cause of memory problem based on object reference graph


How does it work: Wily Introscope LeakHunter: - Via setting in IntroScope configuration and instrumentation, the appropriate data is recorded to IntroScope Enterprise Manager Class Histogram analysis: - Via JVM setting (for example, XX:+PrintClassHistogram) you can force to conduct class histogram snapshot in addition to standard thread dump - Class histogram provides a snapshot of all classes regarding number of instances and number of occupied bytes Heap Dump analysis: - Via JVM settings (for example, XX:+HeapDumpOnCtrlBreak) we can force heap dump snapshots instead of a standard thread dumps - Heap dumps provide the most precise and complete picture of complete heap with nearly no runtime overhead (only when writing the dump to file system)

© SAP AG

E2E100

4-82

Fact Sheet: JAVA Memory Analysis Level 2 (must know)

Use Case Analyze Java memory consumption Check if GC is the root cause for a performance problem Provides Memory consumption and GC duration data Best Practice for Investigation / Rules of Thumb

Check if the diagram shows an increasing memory allocation even after full GC cycles Check for long running full GC’s Check frequency of GC’s Criteria for memory leak: At least ~10 full GC: Memory usage after the full GC seems to increase Even better indicator: Memory consumption does not go down as expected during phases of low load (e.g., night hours) Problem: Caches are designed to keep a lot of data in memory; do they have an upper bound? Is the upper bound adequate? © SAP AG 2006, Title of Presentation / Speaker Name / 1

Limitations: does not show who consumes the memory Prerequisites: SAP Solution Manager 4.0

© SAP AG

E2E100

4-83

Java Memory Analysis

Exercise Java Memory Analysis


© SAP AG

E2E100

4-84



? !


?


?


!

SAP J2EE analysis

?


?


?



© SAP AG

E2E100

4-85

Java Component Analysis: Example Thread dump analysis with Solution Manager The company’s support organization needs to identify a single Java application that causes performance problems. The experts are going to check the state of all Java threads. They are using the thread dump analysis tool to narrow down the problem to a specific Java package or application.


Thread dump analysis is integrated into the diagnostics part of Solution Manager.

© SAP AG

E2E100

4-86

Fact Sheet: JAVA Thread Dump Analysis Level 2 (must know)

Use Case Trigger thread dumps and analyze the status of the JAVA thread activity Analyze “system hang situations” Provides Status and stack trace for every JAVA thread Best Practice for Investigation / Rules of Thumb

Always create sequences of thread dumps, not only one Look for threads with status “waiting on monitor entry” Check the existence of deadlocks Look for high activity in customer coding sections


Limitations: does not show CPU and Memory consumptions of the threads Prerequisites: SAP Solution Manager 4.0

© SAP AG

E2E100

4-87

Thread Activity Analysis

Features of thread activity analysis: Central triggering and scheduling of JVM-based thread dumps for selected J2EE node(s) Analysis of the thread dump data via appropriate user interface

Expert mode


You can find the Thread Activity Analysis feature by choosing Exceptions Application Selection Java. In ABAP terms, full thread dumps are the “SM50 for Java.“ You get a complete overview of what the worker threads (work processes) are doing. Usage scenarios: System hangs and does not respond to requests trigger thread dump System or some requests are very slow trigger a series of thread dumps in short intervals System seems to be aging and dies after some time trigger a series of dumps with longer intervals (for example, scheduled every 15 minutes over several days)

© SAP AG

E2E100

4-88

Full Thread Dump: Example Thread dumps and garbage collection traces are written by the Java VM in plain text format Thread name

Native thread ID

Execution state

Stack trace


List of all threads – for each thread: Thread name assigned by the Java application (for example, by the J2EE Engine) Java thread ID is usually not useful Native thread ID is the same thread ID that is displayed in OS tools; this way, you can map the info Execution state of the thread: runnable or waiting The C stack location in the process address space (not of interest) Stack trace of this thread

© SAP AG

E2E100

4-89

Thread Activity Analysis: Entry Screen

Triggers a full thread dump on the chosen component(s) Expert mode to a create series of thread dumps Starts analysis of thread dumps from the selected component(s)

Starts sequence of thread dumps Generates a task in the Solution Manager diagnostics scheduler © SAP AG 2006, Title of Presentation / Speaker Name / 1

The user has three options to trigger dumps: Trigger single dump now for ad hoc analysis (click on Trigger Dump) Trigger a series in short intervals (click on Start Now) - Active as long as you keep the browser session open - Example: Trigger 100 dumps, one dump every 30 seconds, during a load test Schedule triggering of dumps (click on Schedule Thread Dump Task) - Via the Expert mode scheduler function - Example: Server node dies from time to time, so trigger a dump every 10 minutes over several days - Usage in two steps: Schedule Thread Dump Task creates a task in the scheduler that contains all the parameters that you fill in in the form: selected nodes, number of dumps, and interval between dumps. This saves fiddling with the cryptic syntax of ThreadDumpTask parameters. Go to the scheduler, edit the task (when to start, and so on.) and enable it (you need Administrator rights).

© SAP AG

E2E100

4-90

Thread Activity Analysis (Scheduler): Enable the Task

Enable (and optionally modify) the task


If you want to trigger 10 dumps in sequence with short intervals for a node (-list), this is a single execution of the scheduler task.

© SAP AG

E2E100

4-91

Thread Activity Analysis: Usage

Trigger a new dump

Check for new dumps and GC on the analyzed nodes

Download dumps and GC data to your PC

Restart and uptime since previous restart of cluster node

Uptime at the time of analysis

Full thread dumps © SAP AG 2006, Title of Presentation / Speaker Name / 1

The figure above shows the Summary table, where you have one column per analyzed J2EE node. If dumps are triggered bypassing the startup framework (for example, kill -3 on UNIX), you will not see a time stamp for the dump. Features for offline analysis: Download the complete Java VM log (for example, std_server0.out) via thread dump analysis Download information about a deadlock Download individual full thread dumps - If you plan to attach the download to a message, always download the complete VM log; developers will request this so that they have a complete picture - Upload a Java VM log to the thread dump analyzer

© SAP AG

E2E100

4-92

Thread Activity Analysis: Usage (2)

On click

On click

Single dump Thread detail © SAP AG 2006, Title of Presentation / Speaker Name / 1

The summary analysis reads the console logs from the selected components and shows it in a tab. The summary table structure is independent of the chosen J2EE node ID. It is not possible to choose Restart. If a dump contains a deadlock, it will be shown in summary and marked with a red triangle.

© SAP AG

E2E100

4-93


First focus on application threads; they handle user requests Check how many threads are in state waiting for monitor entry. These threads are blocked

critical!

Check which threads are in state runnable. How many threads are in this state? What is their stack trace? Which package prefixes can you determine?

Example: 39 application threads are busy

Example: 1 thread is idle


You can do a simple thread dump evaluation by following the steps on shown in this figure. Your analysis should be focused on application threads, since these are the ones that are processing the application logic. Check every single dump for threads with condition “waiting for monitor entry.” These threads want to enter into a method that is currently blocked by another thread. Of course, this has a direct impact on the response time and is therefore a critical condition. The next step for root cause analysis is it to find out what is blocking the thread. If you cannot find threads that are blocked, it make sense to check the threads that are currently in condition “runnable”. If you have a sequence of thread dumps, check for long-running threads and try to find out what are they doing.

© SAP AG

E2E100

4-94


These threads are blocked!

Find out who is blocking them!


To identify the blocking thread, go to threads detail and select a thread that has the condition “waiting on monitor entry.” The stack trace of the marked thread will be shown in the lower part of the screen. Then click the find lock owner button. The lock owner will be shown in the Locks view.

© SAP AG

E2E100

4-95


The lock holder keeps the other threads from working. Click to get details on blocking thread.


You can see the blocking thread (lock owner) in the upper part of the screen. Below, all threads that are blocked from this one are listed. You can select the lock owner to see his stack trace in the Thread Details view.

© SAP AG

E2E100

4-96


What non-standard Java packages are involved? Quick check to find out…


For root cause analysis, it is helpful to have a quick overview of non-SAP components that are involved in the thread dump.

© SAP AG

E2E100

4-97


All Dumps Intermittent performance problems should be analyzed by a series of dumps. Analysis: “All Dumps” and “Thread Activity” “All Dumps” structure: One column per dump; X-axis is the timeline One row per combination (thread name prefix, state, stack line) Number in the cell indicates number of threads that match High numbers indicate hotspot or (idle) pools

Compare dumps: What changes and what remains unchanged?


The All Dumps view allows you to identify a tendency of thread conditions over a sequence of thread dumps. The goal is it to identify increasing or decreasing numbers of threads with specific conditions that could lead you to the root cause of a problem.

© SAP AG

E2E100

4-98

Thread Dump Analysis: Usage

Thread Activity “Thread Activity” structure: One column per dump; X-axis is the timeline One row per thread Mark in cell indicates if the thread (state or stack) has changed in comparison to previous dump


The Threads Activity view gives a quick overview of the thread status of a series of thread dumps. Threads with critical condition (blocked) are marked with an X. You can identify the lock holder by clicking on the X.

© SAP AG

E2E100

4-99

Java Thread Dump Analysis

Exercise Java Thread Dump Analysis 1 and 2


© SAP AG

E2E100

4-100



? !


?


?


!

SAP J2EE analysis

?


?


?



© SAP AG

E2E100

4-101

Workload Analysis with Wily Introscope (EP)

SAP EP Overview SAP EP Triage

SAP EP JDBC SAP EP iViews SAP EP Components SAP EP PCD SAP EP Content Management


Wily Introscope is delivered with a default instrumentation to cover the most common problem areas for SAP Java applications such as Enterprise Portal, Exchange Infrastructure, and so on. You can access this application-specific data view via Introscope WebView or Introscope Workstation. In WebView, you will find a predefined set of dashboards for your specific solution. The figure above shows as an example the dashboard structure for the SAP Enterprise Portal application.

© SAP AG

E2E100

4-102

Workload Analysis with Wily Introscope


This is an example of the SAP Enterprise Portal Overview dashboard. Here you can find the following data: Number of requests HTTP sessions iView response time EP request average response time Performance data of connected systems (TREX, LDAP, enqueue service, database) System resource consumption

© SAP AG

E2E100

4-103

Workload Analysis with Wily Introscope (2)

Application analysis in investigator mode

Navigate in the tree to your application

Check the application-specific workload metrics in the preview pane


You can check the application workload with Introscope Workstation. Here you will find a specific metric folder depending on your application. The example above shows the metric tree for SAP Enterprise Portal.

© SAP AG

E2E100

4-104



E2E Trace Analysis







JAVA Trace Analysis










Functional Problem

© SAP AG 2006

© SAP AG

E2E100

4-105

General Component Analysis Roadmap (3)


?


?


?



© SAP AG

E2E100

4-106

Example: End to End Workload Analysis Workload Analysis with ABAP specific transactions The support organization from the company finished the E2E Workload analysis and identified a ABAP based system (e.g. R/3) as root cause for the performance problems.

The support experts are using the ABAP transaction ST03N as entry point for a detailed workload analysis of the system.


© SAP AG

E2E100

4-107

Performance Monitors for the Technology issues

Transaction ST03N

Workload Monitor

Work Process Overview

Operating System Monitor

Database Monitor

Setups Buffers

Transaction SM50 / SM66

Transaction ST06

Transaction ST04

Transaction ST02

… Details in the following slides © SAP AG 2006, Title of Presentation / Speaker Name / 1

To begin a workload analysis on an ABAP stack, use the Workload Monitor (Transaction ST03N). If you find statistics that indicate a performance problem use the relevant monitoring tools for the problem area to obtain further data on the problem.

© SAP AG

E2E100

4-108

Dialog Step – Time Measurement Dialog Response Time Front-end PC

GUI

Net

Network

Roundtrip

Dispatcher Applicationserver

Wait

Roll-In

Network DB

Net

Load + Generate

Enqueue

Dispatcher

Processing time(1)

RollWait

RFC + CPIC Roll-Out (**)

Proc. (2)

Database request

RollWait

RolIIn(*)

RollOut

Proc. (3)

RolIIn(*)

DB processing

RollOut

DB

Network Disjunctive parts of Response Time

External system

RFC processing

CPU Time: Time consumed in CPU Other measured Times

© SAP AG 2006

Response time in milliseconds: Starts when a user request enters the dispatcher queue; ends when the next screen is returned to the user. Wait time in milliseconds: The time a user request sits in the dispatcher queue. It starts when user request is entered in the dispatcher queue; and ends when the request starts to be processed. Roll-in time in milliseconds: When a user is dispatched to a work process, "user context" data – the user's logon attributes, authorizations, and other relevant information – is transferred from the roll buffer, extended memory, or the roll file into the work process. This transfer (by copying or mapping as appropriate) of user context data into work process memory is the mechanism known as a "roll in". (*) Roll in happens only in the user context has been changed in roll memory of WP. It means that after RFC or round trip Roll-In does not necessarily happens. Load time in milliseconds: The time needed to load from the database and generate objects like ABAP source code, CUA, and screen information. Processing time: This is equivalent to response time minus the sum of wait time, database request time, load time, roll time, and enqueue time. Database request time: Starts when a database request is put through to the database interface; ends when the database interface has delivered the result. CPU time in milliseconds: CPU time is the amount of time during which a particular work process has active control of the central processing unit (CPU). This time includes CPU processing on all the steps. CPU is mostly used by Roll In / Roll Out / communication with DB interface and ABAP processing.

© SAP AG

E2E100

4-109

Roundtrips, Roll wait time, GUI time: During a roundtrip to the SAP GUI, the ABAP program is usually rolled out of the work process so that the WP could be used for another user. As a result of this you may observe a high roll wait time due to SAP GUI communication. This time is measured as GUI time. If the dialog step includes no RFC communication with an external system (only RFC communication to GUI), then the GUI time ≈ roll wait time. If the dialog step includes RFC communication with an external system, the GUI time may be less than the roll wait time, since roll wait time covers roll wait situations due to all RFCs, GUI time only covers the roll wait situations due to SAP GUI communication. See also SAP Note 8963. Roll-Out: When Dialog Step is ready then the WP is rolled out to the roll buffer (the parts that have been changed). Normally that is not part of the Dialog Response Time. (**) By Default when RFC is initiated Roll-Out happens after 500 ms.

© SAP AG

E2E100

4-110

Single Statistical Record per Dialog Step (STAD)

The average response time consists of: Wait time Database request time Roll time Load time Enqueue time Processing time is: ABAP processing time (part of total CPU time) Work process waiting for CPU Waiting for I/O


Time spent in R/3 work process = Response time - Wait time Processing time is the actual time required to process the dialog step inside the R/3 work process. Processing time = Response time - wait time - database request time - roll time - load time - enqueue time. Processing ABAP statements requires CPU. The more statements there are, the more CPU time is required. Ideally most of the processing time is used for processing ABAP statements. To achieve this you require: A low operating system overhead (the time that an R/3 work process waits for CPU) Short I/O wait times which occur when accessing external files or as a result of network time (RFC/CPIC) CPU time is also required for database access time, roll time, load time, and enqueue time.

© SAP AG

E2E100

4-111

ABAP Workload Analysis ST03N

Transaction ST03N, the Workload Monitor, accesses the statistics © SAP AG 2006

To access the Workload Monitor, use Transaction code ST03N. Select the required timeframe and by default the ‘Workload Overview’ view is displayed. In the “Administrator’s Mode” the default time frame of the workload analysis is the current day. Important for this training: Change to “Expert Mode”. Choose under “Detailed Analysis” -> “Last minutes load” During the workload exercise, restrict the time period to the time when the exercise is running. Under Analysis views, you can access, for example: Workload overview - Workload statistics according to work process type Transaction Profile - Workload statistics according to transaction. Time Profile - Workload statistics according to hour

© SAP AG

E2E100

4-112

Transaction Profile


In the Workload Monitor selecting Transaction Profile enables you to find out: The most-used transactions. Tuning these transactions results in the largest improvements in overall performance. The average response times for typical R/3 transactions.

© SAP AG

E2E100

4-113

Workload Monitor points us to the cause of the performance problem Problem symptoms: Large roll-wait time Performance problem?

Large load time

Check:

Yes All transactions affected?

Large database request times

Workload on all servers

(Average) Large CPU times No Program analysis

Processing time much larger than CPU time


If a problem is detected, the data in the Workload Monitor (Transaction ST03N) can be used as follows to identify the area of the system where the problem is located: First check for general performance problems affecting all transactions. Good general performance is normally indicated by: In the Workload Monitor, the following values normally indicate good performance: Wait time < 10% response time Average roll-in time < 20 ms Average roll wait time < 200 ms Average load (and generation) time < 10 % of response time (<50 ms) Average database request time < 40 % of (response time - wait time) Average CPU time < 40 % of (response time - wait time) Average CPU time – should be close to processing time Average response time - depends on your requirements – there is no general rule

© SAP AG

E2E100

4-114

Fact Sheet: Transaction ST03N Level 2 (must know)

Use Case Get workload information of your ABAP based components Analyze overall performance bottlenecks Provides

Key performance indicators for ABAP-based components Best Practice for Investigation / Rules of Thumb Large Roll-wait time: Communication problem with GUI or external system Large load time: Program buffer, CUA buffer, or Screen buffer too small Large database request times: CPU or memory bottleneck on database server, network problems, expensive SQL statements, database locks, missing indexes, missing statistics, small buffers Large CPU times: Expensive ABAP processing, for example, processing large tables, frequent accessing of R/3 buffers Processing time much larger than CPU time: CPU bottlenecks, network problems, communication problems © SAP AG 2006, Title of Presentation / Speaker Name / 1

© SAP AG

E2E100

4-115

ABAP Workload Analysis RoadMap

R/3 Workload Monitor (ST03N) Wait time > 10% of response time

? !

General performance problem ?

High database time: database time > 40% (response time - wait time) Database workload analysis

?

Processing time > CPU time x 2 Detailed analysis of hardware bottlenecks (ST06)

?

Load time > 50 ms Detailed analysis of R/3 memory configuration (is the program buffer too small?)

?

Roll-wait time or GUI time > 200 ms Detailed analysis of interfaces and GUI communication

Transaction Profile (transaction ST03N) sorted by Response time total ?

Programs with high CPU time: CPU time > 40% (response time - wait time)


© SAP AG

E2E100

4-116



? !



?


?


?





Lost Time = Av Processing Time – Av CPU time

© SAP AG

E2E100

4-117

ABAP Analysis Transaction ST06

CPU idle < 20%?

SAP work processes with high CPU utilization?


To check work processes with high CPU utilization open the ‘Detail analysis menu‘ in transaction ST06 and click on ‘Top CPU‘. Note: The Java process are named jlaunch.exe

© SAP AG

E2E100

4-118

ABAP Hardware Analysis Roadmap (1)

Operating System Monitor (transaction ST06) CPU idle < 30%?

? ?

Is there CPU capacity on other servers that is idle? Redistribute SAP work processes and users Operating System Monitor (transaction ST06): Top CPU Processes ?

SAP work processes with high CPU utilization? Work Process Overview (transaction SM50) or System wide Work Process Overview (transaction SM66) Detailed analysis of report or transaction

?

External processes with high CPU utilization? Stop or redistribute

?

High paging rate (> 20% of RAM per hour)?


This roadmap guides you in analyzing your hardware capacity. Call the Operating System Monitor ( transaction ST06 ). If the indicated CPU idle (%) is less than 20% there is a CPU bottleneck. In an optimal configuration more than 35% of CPU capacity is idle. If there is a CPU bottleneck: 1. If possible redistribute load to other servers. 2. Find out which processes are using the most CPU via the Operating System Monitor choose Detail analysis menu -> Top CPU processes. If the processes have high CPU utilization, proceed as follows: - For R/3 work processes (‘disp+work’): Using the process ID indicated in Top CPU processes identify the corresponding program name and user name in the Work Process Overview (transaction SM50). - For database processes (for example, "ORACLE8.0"): Identify corresponding long running SQL statements in the Database Process Monitor. To access this monitor, call transaction ST04 (Database Overview), and choose Detail analysis menu. Then choose, for example, Oracle Session. - For external processes, find out whether the process can be stopped or redistributed.

© SAP AG

E2E100

4-119



? ?



?


?



This roadmap guides you in analyzing your hardware capacity. First, call the Operating System Monitor (transaction ST06). If the indicated idle CPU is less than 20%, there is a CPU bottleneck. In an optimal configuration, more than 35% CPU capacity is idle. If there is a CPU bottleneck: 1. If possible, redistribute load to other servers. 2. To find out which processes are using the most CPU, in the Operating System Monitor choose Detail analysis menu -> Top CPU processes. If the processes have high CPU utilization, proceed as follows: - For R/3 work processes (‘disp+work’): Using the process ID indicated in Top CPU processes, identify the corresponding program name and user name in the Work Process Overview (transaction SM50). - For database processes (for example, ‘ORACLE8.0’): Identify corresponding long running SQL statements in the Database Process Monitor. To access this monitor, call transaction ST04 (Database Overview), and choose Detail analysis menu. Then choose, for example, Oracle Session. - For external processes, find out whether the process can be stopped or redistributed.

© SAP AG

E2E100

4-120

Fact Sheet: Transactions SM50 / SM66 Level 2 (must know)

Use Case Get overview of local and global work process activity Analyze overall performance bottlenecks Provides Live data on work process activity Best Practice for Investigation / Rules of Thumb

Check work process status and executed actions Check CPU Utilization during execution and compare it with WP time. If by refresh you observe that WP time increases at the same value as CPU time then it is likely ABAP processing. If you see by refresh no DB accesses but CPU time is increasing together with WP time then it is a good candidate for ABAP trace Check the reason in case of status ‘stopped’ Check details of work process to find out e.g. executed transaction / report name, memory consumed, etc.

© SAP AG 2006

Work process in status running -> Look in column Action/Reason Database actions in column Action/Reason for waiting - Sequential read, Direct read - Insert, Update, Delete, Commit R/3 actions in column Action/Reason for waiting - Load report, Load CUA - Roll in, Roll out No action in column Action/Reason for waiting - Most likely ABAP processing There should be always sufficient workprocesses in status waiting, otherwise this will lead to high wait times resp. high response times. To find out whether there are enough work processes configured, you can either check whether there is a high avg. wait time in ST03N or you can sort the process overview by the CPU time. In this case there should be always some dialog work processes with a CPU time close to 0 seconds. Please keep in mind that a high work process utilization usually is caused by other reasons than a too low number of configured work processes. If a work process is in status „stopped“ it waits for a certain event and cannot be used by other users -> Look in column Reason PRIV, DEBUG, CPIC, NUM, UPD, ADM, ENQ, SLEEP

© SAP AG

E2E100

4-121

SM50 – Work Process Overview


To display a snapshot of the current activities on the instance you are logged on to, use the Work Process Monitor. Call transaction SM50 The information displayed includes: type of work process operating system process-ID number current state and mode of a work process number of detected errors in the work process semaphore resource being used CPU time used by a work process in the dialog step that it is currently processing elapsed clock time used by a work process in the dialog step that it is currently processing name of the ABAP program running client used user holding the resource current action table being accessed

© SAP AG

E2E100

4-122

SM50 – Details

Via ‘Details‘ you get the number of database reads and the memory consumption of the work process.


To display the details of a work process mark the work process and use button ‘Details‘ You get the number of database reads and the memory consumption of the work process

© SAP AG

E2E100

4-123

Check for Workprocess configuration


To check whether there are enough dialog work processes configured: Use transaction SM50, display CPU time and sort the work processes by CPU consumption. There should be at least a few dialog workprocesses with a CPU time close to 0 seconds.

© SAP AG

E2E100

4-124

Global Work Process Monitor – SM66


Using transaction SM66 -> button Settings you find the option ‘Display only abbreviated information, avoid RFC’ Select this option (default setting) to run SM66 in ADM mode and show short information only - Each R/3 instance sends status information to the message server periodically (via ADM messages = special C calls). Running the SM66 reads this status information from the message server De-select this option (recommended for detailed analysis) to run SM66 in RFC mode and show full information - Running the SM66 in RFC mode means that SM66 schedules RFCs to all instances and collects the full information You can use the arrow buttons to display further information about the process.

© SAP AG

E2E100

4-125



? ?



?


?



This roadmap guides you in analyzing your hardware capacity. Call the Operating System Monitor (transaction ST06). If the indicated CPU idle (%) is less than 20%, there is a CPU bottleneck. In an optimal configuration, more than 35% CPU capacity is idle. If there is a CPU bottleneck: 1. If possible redistribute load to other servers. 2. To find out which processes are using the most CPU, in the Operating System Monitor choose Detail analysis menu -> Top CPU processes. If the processes have high CPU utilization, proceed as follows: - For R/3 work processes ("disp+work"): Using the process ID indicated in Top CPU processes, identify the corresponding program name and user name in the Work Process Overview (transaction SM50). - For database processes (for example, "ORACLE8.0"): Identify corresponding long running SQL statements in the Database Process Monitor. To access this monitor, call transaction ST04 (Database Overview), and choose Detail analysis menu. Then choose, for example, Oracle Session. - For external processes, find out whether the process can be stopped or redistributed.

© SAP AG

E2E100

4-126



?

… High paging rate (> 20% of RAM per hour)?

? ?

Memory capacity free on other servers? Redistribute SAP work processes and users

?

File system cache > 10% of RAM? Reduce file system cache Setups/Tune Buffers monitor (transaction ST02): Mode List ?

Programs / transactions with high memory consumption Detailed analysis of report or transaction


© SAP AG

E2E100

4-127

What is OS Paging?

PAGING: the exchange of data pages between the main memory of a host system and the overflow store in a paging file on the hard disk.

Operating System Paging File

Physical Memory RAM

On Unix systems there is a problem when there is a high Page Out on the server. A high Page Out means if the paging is more than 20% of RAM per hour. On Windows systems: A high Pages out value has no bearing on system utilization. Only the Pages In value is used as a measure of system utilization. If the limits are exceeded, the memory capacity of the machine is too small


© SAP AG

E2E100

4-128

Physical and Virtual Memory in ST06

Server

ST06

Virtual memory High paging rate (> 20% of RAM per hour)? Physical Memory RAM

Operating system Page File *

* The OS page file is not related to the R/3 page file


In the Operating System monitor (transaction ST06) note the amount of memory indicated beside Physical memory available. Compare this figure with the paging rate. To obtain the paging rate doubleclick Pages in/s. The paging rates for the last 24 hours are displayed in the columns Paged in [KB/h] and Paged out [KB/h]. If 20% of the total amount of physical memory is greater than the amounts indicated in these columns, you can normally be sure there is no memory bottleneck. If there is a memory bottleneck: 1. If possible redistribute load to other servers. 2. Check the size of the file system cache: see SAP Note 78498. If necessary, reduce file system cache to < 10% of the total physical memory. 3. To identify users and their programs with a high memory consumption, call the Mode List for the extended memory. To do this open the Setups/Tune Buffers monitor (transaction ST02) and choose Detail analysis menu -> SAP memory -> Mode List.

© SAP AG

E2E100

4-129

ABAP Memory Analysis: ST02

Programs / transactions with high memory consumption?

Doubleclick program / transaction for detailed analysis


© SAP AG

E2E100

4-130

SAP Memory Areas

Local Memory : always allocated precisely to one work process. Shared Memory : accessible to multiple work processes. Virtual Memory Shared Memory Roll Buffer

Local Memory Paging Buffer Heap Memory (temporary)

SAP Buffers

Roll File

Extended Memory Paging File

Individual workprocess

Virtual Address Space of a Work process 32-bit -> 2*32 = 4GB (In reality 2-3.7GB is usable) 64-bit -> 2*64 = 18,400,000,000 GB


© SAP AG

E2E100

4-131

ST02 Mode List


Call the Setups / Tune Buffers monitor (Transaction ST02), and choose Detail analysis menu −> SAP memory −> Mode list. There you can see the memory consumption of all user contexts.

© SAP AG

E2E100

4-132

Memory Problem: Determining how much RAM is required

How much memory will be needed for the R/3 system ? = EM (max used) + HEAP (max used) + Allocated memory in pools + wp‘s memory How much memory will be needed for the DB ? (oracle example) = Data Buffer Size + Shared Pool Size + (# shadow wp‘s) X (5 -10MB)

+

Note the highest 'Max used´ values for Heap and Extended Memory

≤ Physical RAM

© SAP AG 2006

© SAP AG

E2E100

4-133

Typical Memory Problems due to incorrect memory area configuration

SAP instance does not start because the operating system cannot provide the requested memory areas. Session terminations: A dialog window appears on the user’s screen with the error message “roll-out failure”. The session then disappears and the user is logged off. ABAP Dumps: Program terminations, you will find the logs for these errors (dumps) in Transaction ST22. Program error (endless loop) or used incorrectly – unnecessary amount of memory requested. SAP profile parameters set incorrectly Swap space on the operating system is not large enough Operating system limits are reached - incorrect OS parameters


The memory for the user context is used up: STORAGE_PARAMETERS_WRONG_SET SYSTEM_ROLL_IN_ERROR TSV_TNEW_BLOCKS_NO_ROLL_MEMORY, TSV_TNEW_PAGE_ALLOC_FAILED TSV_TNEW_NO_ROLL_MEMORY The allocated areas in the shared memory (EM and PXA) violate the OS specific restrictions. PXA_NO_SHARED_MEMORY Memory bottleneck during an operation in the database interface. DBIF_RTAB_NO_MEMORY DBIF_RSQL_NO_MEMORY SAP paging memory is used up. SYSTEM_NO_MORE_PAGING, TSV_TNEW_PAGE_ALLOC_FAILED If ztta/max_memreq_MB is set too low. SYSTEM_NO_ROLL

© SAP AG

E2E100

4-134

Memory Problems: Program Buffer too small? (1)

Call transaction ST02. Are these program buffer swaps really a problem?


© SAP AG

E2E100

4-135

Fact Sheet: Transaction ST02 Level 2 (must know)

Use Case Monitoring of Memory Management of ABAP-based systems Provides Live and historical data Best Practice for Investigation / Rules of Thumb

Check Roll /Paging Area and Extended Memory consumption Identify users and their programs with a high memory consumption, call the Mode List for the extended memory. To do this, in the Setups/Tune Buffers monitor (transaction ST02), choose Detail analysis menu -> SAP memory -> Mode List. Check if Program Buffer (PXA) is configured too small (If the daily PXA buffer swaps are > 10,000 AND PXA Buffer hitratio <= 95% AND Average Load + Gen time > 50ms it is an indication that the program buffer is too small) © SAP AG 2006, Title of Presentation / Speaker Name / 1

© SAP AG

E2E100

4-136

Fact Sheet: Transaction ST06 Level 2 (must know)

Use Case Analyze hardware performance bottlenecks

Provides Live and historical data of: Paging rates , CPU utilization, disk read/write (I/O) times, network response times

Best Practice for Investigation / Rules of Thumb CPU Utilization is more then 70% for a long time is an indication of CPU bottleneck Check if external processes are creating an unnecessary load on the CPU (it could be e.g. second instance or another R/3 System, external system / external database, looping processes, running backup tool, screen savers) Use other machines, extend the hardware (CPU), tune the processes if they are required to run R/3 system or terminate the processes if possible

Check if Paging Rate is more then 20% of Physical RAM: Memory bottleneck Identify users and their programs with a high memory consumption ( transaction ST02) Optimize program or train users to avoid high memory consumption If possible, redistribute load to other servers. © SAP AG 2006, Title of Presentation / Speaker Name / 1

Typical reasons for hardware Bottlenecks: Incorrect sizing (physical main memory, CPU) Incorrect workload distribution Expensive programs (SQL statements, external programs) Incorrect disk layout or slow disks Incorrect network topology In the Operating System monitor (transaction ST06), note the amount of memory indicated beside Physical memory available. Compare this figure with the paging rate. To obtain the paging rate, doubleclick Pages in/s. The paging rates for the last 24 hours are displayed in the columns Paged in [KB/h] and Paged out [KB/h]. If 20% of the total amount of physical memory is greater than the amounts indicated in these columns, you can normally be sure there is no memory bottleneck. To identify users and their programs with a high memory consumption, call the Mode List for the extended memory. To do this, in the Setups/Tune Buffers monitor (transaction ST02), choose Detail analysis menu -> SAP memory -> Mode List.

© SAP AG

E2E100

4-137



? !



?


?


?





© SAP AG

E2E100

4-138

Workload Analysis: Interfaces and GUI communication (I) The RFC time is summarized in Workload analysis transactions under specific TASK type „RFC“.

To see RFC Profiles, do not forget to switch to !

ST03N


© SAP AG

E2E100

4-139

Workload Analysis: Interfaces and GUI communication (2) The RFC time statistics can be analyzed in detail using the RFC Profiles section. Here you can see the statistics for client communication. - by RFC Destination - by physical hosts - internal communication to application servers

The summary can be grouped by the different sources of information.

ST03N

To see the summary for each group, double click on the name of group


© SAP AG

E2E100

4-140

Workload Analysis: Interfaces and GUI communication (3) Here you can see the summary of Server RFC calls

ST03N For the Server Profile ‘Function Module’ is the most important view

Standard functions: ARFC* - tRFC-Layer; TRFC* - qRFC-Layer a_rfc – aRFC calls

To see the details, double click on the line


sRFC – synchronous RFC call ABAP Statement: CALL FUNCTION DESTINATION Standard SAP protocol for external calls. In the most cases is named as RFC. aRFC – asynchronous RFC call ABAP Statement: CALL FUNCTION STARTING NEW TASK Start the function for asynchronous and parallel processing. tRFC – transactional RFC call ABAP Statement: CALL FUNCTION IN BACKGROUND TASK Save the function calls in one LUW and start the LUW afterwards asynchronously qRFC – queued RFC call ABAP Statement: CALL FUNCTION „TRFC_SET_QUEUE_NAME“ + CALL FUNCTION IN BACKGROUND TASK Save the function calls in one LUW for specified queue and start the LUW afterwards asynchronously

© SAP AG

E2E100

4-141

Workload Analysis: Interfaces and GUI communication (4)

Which Exernal Systems cause a high load ?

Which Function Modules were executed?

ST03N


© SAP AG

E2E100

4-142

General Component Analysis RoadMap


?


?


?



© SAP AG

E2E100

4-143

Database Analysis: ST04


© SAP AG

E2E100

4-144

Fact Sheet: Transaction ST04 Level 3 (Expert)

Use Case Database Analysis Provides Live and historical data

Best Practice for Investigation / Rules of Thumb Check SQL cache: sort by buffer gets and tune most expensive statements: An expensive SQL statement has a high number of Buffer gets compared to Total reads. If the ratio of Buffer gets to Total reads is greater or equal than 5%, the SQL statement is expensive Identify: Duration, Execution Path, Table accesed, Index used, Execution Cost Check Data buffer quality / Data cache hit ratio Check I/O performance © SAP AG 2006, Title of Presentation / Speaker Name / 1

Restriciton: Transaction code ST04 is same for all databases but depending on the underlying database the monitors look different and the interpretation of the database requires database specifc skills Oracle specific checks: Data buffer quality should be at least 94%. Ratio of Reads to Physical reads should be at least 10:1. Value of Buffer busy waits is less than 5% of the data buffer (Oracle Block) Reads. Shared Pool: DDCache quality > 80%, pinratio > 95%, reloads/pins <= 0,04. Calls: User/ Recursive calls >= 2 (in OLTP >=4). User Calls to Parses >= 4:1. Reads/User calls not higher as 50:1. Time statistics: Time/User call - values of more than 15ms show room for improvement. Busy wait time/CPU time 60:40 for a well-tuned system (higher values: e.g. 80:20 indicate perf. Problem). Check the key performance indicators as described in note 618868 question “How can I determine whether the general database performance can be optimized?” Monitor the wait situations (on system or session level) Check whether the database is mainly waiting using note 619188. If the focus is on the overall situation of the DB use v$system_event as an entrypoint. If a concrete action should be analyzed use v$session_event of the corresponding session. Depending on which wait events the most time is spent branch to the referenced notes. If the target of the monitoring is some specific steps perform an SQL Traces for those steps. Save the trace and the statement summary. MS SQL specific checks: Note how many physical memory and CPU available in database server

© SAP AG

E2E100

4-145

In “Detail Analysis” -> State on disk: Check how many data files are available (there should be at lest as much data files as number of CPU. Normally there should be more data files as nr. Of CPU to avoid wait situations) Prove if there are enough data files not completely filed up (Used MB column). Compare difference between Size (MB) and Used Size (MB). This difference should be normally similar for all the data files. Perform the same checks also for the temp DB (from top menu-> Other DB-> Change DB) In the “Detail Analysis” -> I/O per file check what is the value IO (ms) per read. It should be normally 50-100 ms/read. You should check if this value is similar for all the files. Please also note if workload distributed well enough (nr. of reads)

© SAP AG

E2E100

4-146

End-to-End Workload Analysis: Unit Summary

You should now be able to: Analyze a performance bottleneck using End-to-End Workload Analysis Analyze an issue with ABAP workload in detail Analyze an issue with SAP J2EE workload in detail Analyze operation system resources


© SAP AG

E2E100

4-147

© SAP AG

E2E100

4-148

Exercises Unit:

E2E Workload Analysis

Lesson:

Thread Dump Analysis

At the conclusion of this exercise, you will be able to: • Identify the cause of deadlocks that block threads by using the Thread Dump Analysis functionality of the Solution Manager.

1

Please connect to your Solution Manager and start the diagnostics part. Log In with user and password provided by the trainer. Choose the solution provided by the trainer and execute the Thread Dump Analysis tool. 1-1

Choose the application server node and trigger a dump.

1-2 Click the “Analyze” button. 1-3

Analyze the deadlock and answer the following questions: 1-3-1

How many threads are affected from the deadlock?

1-3-2

What is the name of the threads?

1-3-3

What is the name of the executed application?

Additional: How can you solve the deadlock situation?

© SAP AG

E2E100

4-149

© SAP AG

E2E100

4-150

Exercises Unit:


Lesson:

JAVA Memory analysis

At the conclusion of this exercise, you will be able to: • Use the Java Memory Analysis tool of SMD • Identify critical Java memory conditions

1

Please connect to your Solution Manager and start the diagnostics part. Log In with user and password provided by the trainer. 1-1

© SAP AG

Choose the solution provided by the trainer and execute the Java Memory Analysis tool. Please answer the following questions: 1-1-1

Can you identify a critical memory situation?

1-1-2

If “Yes” when was it?

1-1-3

What is the value for Max Heap size on this system?

1-1-4

What is the value for New Size?

1-1-5

How many full GC’s can you identify?

1-1-6

What is the average duration time over all full GC cycles?

E2E100

4-151

© SAP AG

E2E100

4-152

IT Reporting

Unit:


Lesson:


Exercises

At the conclusion of this exercise, you will be able to: • Identify the cause of a performance problem by using the Thread Dump Analysis functionality of the Solution Manager.

1

Please connect to your Solution Manager and start the diagnostics part. Log In with user and password provided by the trainer. Choose the solution provided by the trainer and execute the Thread Dump Analysis tool. 1-1

Choose the application server node and trigger a dump.

1-2 Click the “Analyze” button. 1-3

Identify your dump by timestamp and start to analyze it. Please answer the following questions: 1-3-4

Can you identify threads with a critical state?

1-3-5

What kind of critical state is it?

1-3-6

How many threads are in critical conditions?

1-3-7

What are the names of these threads?

1-3-8

What is the name of the lock holder thread?

1-3-9

What is the name of the problematic java package?

1-3-10 What is the name of the problematic application? Additional: How can the problem be solved?

© SAP AG

E2E100

4-153

© SAP AG

E2E100

4-154

Solutions

1-1

Unit:


Lesson:


Analyze a deadlock situation by following the steps as shown in the example below:

Now you should see a sreen similar to the one shown below. Please follow the steps accordingly.

Please choose ‘Open’ in the File Download window. The deadlock information will be displayed like shown below.

© SAP AG

E2E100

4-155

1-3-1

There are two threads affected.

1-3-2

The thread names are: SimpleDeadlockExercise_Thread1 and SimpleDeadlockExercise_Thread2.

1-3-3

The application name is: Exercise3.java Additional: To solve a deadlock situation, it is necessary to restart the application server node.

© SAP AG

E2E100

4-156

Solutions

1-1

Unit:


Lesson:

JAVA Memory analysis

After executing the Java Memory Analysis you should see a similar screen to the following one:

1-1-1

© SAP AG

You should identify a critical situation (marked in the picture above) where you have a lot of GC’s and a memory consumption close to the max heap size.

E2E100

4-157

© SAP AG

1-1-2

You can find the displayed time frame in the lower part of the screen (in the example above 10:00 – 10:30)

1-1-3

Click the button “Details View”. In the lower part of the new window, you will find a table with VM parameters. The max heap size is represented by the Xmx parameter.

1-1-4

You can find the value for New Size in the same table.

1-1-5

You can find the number of Full GC’s in table GC Statistics column Count.

1-1-6

You can find the value for Avarage Duration time in the same table.

E2E100

4-158

Solutions

1-1

Unit:


Lesson:


Analyze a deadlock situation by following the steps as shown in the example below:

Now you should see a screen similar to the one shown below. Please follow the steps accordingly.

The ‘Single Dump’ view will be displayed now. Please use the filter thread function to see all categories of threads. Now your screen should look similar to the example below.

© SAP AG

E2E100

4-159

Now click on the blocked threads to change to ‘Threads Detail’ view. Mark a blocked thread in the details view. The Java stack of this thread is shown in the lower part of the screen. Here is also the button ‘Find Lock Owner’. Please click this button. Now you should see a screen similar to the example below.

© SAP AG

E2E100

4-160

© SAP AG

1-3-1

Yes.

1-3-2

The threads are in state ‘waiting for monitor entry’.

1-3-3

Five threads are in critical conditions in the example above (can be manipulated from the trainer).

1-3-4

The threads are named ‘WaitingThread_For_ResourceShortageExercise’.

1-3-5

The lock holder thread is named ‘WorkerThread_For_ResourceShortageExercise’.

1-3-6

The java package name is com.sap.sup.e2e.exercises.

1-3-7

The application name is Exercise3.java.

E2E100

4-161

© SAP AG

E2E100

4-162

End-to-End Trace


© SAP AG


E2E100

5-1

End-to-End Trace

Contents: End-to-End Trace in SAP Solution Manager Client-Side Tracing with BMC AppSight SAP Web AS Java Tracing SAP Web AS ABAP Tracing


© SAP AG

E2E100

5-2

End-to-End Trace: Unit Objectives

After completing this unit, you will be able to: Perform an end-to-end trace to isolate a performance problem or error Trace a client-side application with BMC AppSight Trace an application on Web AS Java with Wily Introscope Transaction Tracer Trace an application on Web AS ABAP with SQL Trace (ST05) and Single Transaction Trace (ST12)


© SAP AG

E2E100

5-3



E2E Trace Analysis







JAVA Trace Analysis










Functional Problem

© SAP AG 2006

© SAP AG

E2E100

5-4

End-to-End Workload Analysis Roadmap

End-to-end workload analysis ?


?

Single transaction or client or network not performing End-to-End Trace


© SAP AG

E2E100

5-5

End to End Trace Analysis: Example End-to-End trace analysis example with an Employee Self-Service (ESS) scenario An employee complained about high system response time while using the Bank Information feature of an ESS scenario. The company support organization is informed about the problem and is starting the root cause analysis procedure using SAP Solution Manager. To begin analysis of the problem, end-to-end trace analysis will be used to locate the root cause.


The ESS scenario is distributed over physical hosts with installations of SAP Enterprise Portal and my SAP ERP Human Capital Management as back-end systems. The end-to-end trace analysis should be used to find if the problem is related to the client, network, or server (ABAP or Java).

© SAP AG

E2E100

5-6

End-to-End Trace Analysis Roadmap End-to-end trace ?



?


?


?


WAN Network

Presentation Logic

Enterprise Portal

Application Logic

Database

ECC


© SAP AG

E2E100

5-7

End-to-End Trace: From Browser to Disk Instrumented Browser

WAN Network

Presentation Logic

Application Logic

Enterprise Portal Correlation ID Trace Flags

1

Create Passport

ECC

Correlation ID Trace Flags

2

Activate Trace

Database

Correlation ID Trace Flags

3

Activate Trace

End-to-End Trace Display in SAP Solution Manager © SAP AG 2007, Title of Presentation / Speaker Name / 1

Traffic between client and server At the client, a client-independent proxy and a client-dependent plug-in are installed which provides unique correlators for scenario, click, and request to the traffic between client and server. Client-side trace data is transferred from client to the managing system via HTTP upload. Introscope-specific traffic Data recorded by the Introscope byte-code morphing agent are transferred directly to the Introscope Tracer database. An Introscope proprietary protocol is used which can be tunneled via HTTP. The Introscope trace summary is transferred to end-to-end trace analysis via the API . Currently Java and .NET components are covered by Introscope tracing capabilities. Traffic to retrieve trace summary data Non-Introscope trace data is written locally in database and file systems. Relevant trace summary data is retrieved by RFC protocol for ABAP-based components and P4/RMI protocol (used by Solution Manager diagnostics agent) for non-ABAP components.

© SAP AG

E2E100

5-8

End-to-End Trace Analysis Steps to create an end-to-end trace:

1 Download IE plug-in 2 Extract the IE plug-in .zip file on your local computer


The end-to-end trace plug-in is accessible via the SAP Note 1010428.

© SAP AG

E2E100

5-9

End-to-End Trace Creation 2

3

6 1

4

5

7


To create an end-to-end trace, follow these steps: Double-click on SAPIEPlugin.exe to start a new browser window including the plug-in. Navigate to the target url for analysis. Enter the name of the transaction and transaction step that you want to record. Choose a session trace level. Enter the Solution Manager Diagnostics host and HTTP port. Start recording the transaction. Execute the task that you want to trace. Stop the transaction. Enter a new transaction step name and choose New Step to continue tracing, or choose Exit to finish trace activities. If you choose Exit the recorded transaction.xml file will be uploaded automatically to your host.

© SAP AG

E2E100

5-10

End-to-End Trace Creation (2)

Trace Level

J2EE

SQL Trace

ABAP

Logging

None

IS Trace

SQL Trace (incl. RFC)

ABAP Trace

Enque ue/Buf fer

Authorization

X

X

X

Low

X

X

X

Medium

X

X

X

aggregated

High

X

X

X

Non aggregated

X

Summary displayed in Solution Manager diagnostics Not displayed in Solution Manager diagnostics


© SAP AG

E2E100

5-11

End-to-End Trace Analysis: User Interface

1 2

2

3 4 © SAP AG 2007, Title of Presentation / Speaker Name / 1

To evaluate a trace, follow these steps: Choose Traces End-to-End Trace analysis and enter the solution and system. Select the transaction in the table. It is also possible to upload a recorded transaction (BusinessTransaction.xml in the SAPIEPlugin folder logs Transaction name). After selecting a transaction the transaction steps are displayed in tab form. Choose the step you want to analyze. Click Display to analyze the transaction step.

© SAP AG

E2E100

5-12

End-to-End Trace Analysis: User Interface (2)

Is it possible to identify a problem area?


The first step is to analyze the accumulated time for the transaction step. Here it is possible to identify if the problem is related to the client, network, or server (including all involved back-end systems).

© SAP AG

E2E100

5-13

End-to-End Trace Analysis: Message Table

Filter the cached requests

Choose the request with the biggest time consumption


You can check for long-running server responses by choosing the Message table tab. Check the sRT (server response time) column for long-running responses and select this request for more detailed analysis.

© SAP AG

E2E100

5-14

End-to-End Trace Analysis: Message Table (2)

Communication flow from client to client


The next analysis step is it to check the HTTP header fields for possible manipulations (done by network components like proxy, router, and so on). In the upper-left corner you can see the request header as it was sent from client to server. On the lower left you can see the header as the server received it. On the lower right you can see the response header fields as they were sent from server to client, and on the upper right you see how the client received the response.

© SAP AG

E2E100

5-15


Display Introscope Transaction Tracer Display J2EE SQL Trace summary Display Log Viewer Display aggregated ABAP Trace Display ABAP SQL Trace summary © SAP AG 2007, Title of Presentation / Speaker Name / 1

The figure above shows request handling for request ID 29. The request was sent from the client to server via HTTP. The complete process time for this request was 1532 milliseconds. The J2EE processed the request, which took 1532 milliseconds; 952 milliseconds were spent in J2EE and about 21 milliseconds were used to establish five back-end connections. The RFC calls took 564 milliseconds; you can see the time for each RFC connection. You can analyze the request with Introscope Transaction Tracer, J2EE SQL Trace and Log Viewer for the Java side, and ABAP Trace and SQL Trace Summary for the ABAP side.

© SAP AG

E2E100

5-16



You can see the statistical record details by selecting a line from the End2End Trace Data table.

© SAP AG

E2E100

5-17

Fact Sheet: E2E Trace Analysis Level 2 (must know)

Use Case Identify the root cause of long running user requests Provides

Response times for each component involved in the request execution Best Practice for Investigation / Rules of Thumb Check the distribution of the response time over client, network and server Check for manipulations in the HTTP header (e.g. done by a proxy) Filter out cached requests since they are usually quite fast Check for requests which transfer large amounts of data


Limitations: Browser plugin is only available for Internet explorer Prerequisites: SAP Solution Manager 4.0 SP10; The Solution Manager host needs a network connection to the traced systems

© SAP AG

E2E100

5-18

End-to-End Trace Analysis: Exercise

Exercise End-to-End Trace Analysis


© SAP AG

E2E100

5-19



E2E Trace Analysis







JAVA Trace Analysis










Functional Problem

© SAP AG 2006

© SAP AG

E2E100

5-20

End-to-End Trace Analysis Roadmap (2) End-to-end trace ?



?


?


?


WAN Network

Presentation Logic

Enterprise Portal

Application Logic

Database

ECC


© SAP AG

E2E100

5-21

Client-Side Analysis: Example Example of Client analysis with BMC AppSight The company support organization completed the end-to-end trace analysis and it turned out that the performance problem is related to request processing on the browser side. To begin analysis of the problem, BMC AppSight will be used to identify the root cause.


© SAP AG

E2E100

5-22

BMC AppSight for Client Root Cause Analysis Network

Client

Server

Storage

Portal

CRM

Duet MW

SRM

Adobe

APO

Microsoft IE

Other browser

Desktop/ Laptop

SAP GUI

Forms

Office/Duet

ECC IO Subsystem

Router

Firewall

Switch

BI

XI

MDM

Load Balancer

AppSight © SAP AG 2007, Title of Presentation / Speaker Name / 1

© SAP AG

E2E100

5-23

Client-Side Issues at Support: Workflow Customer

AppSight Black Box

Black Box Log File

“Capture”

Customer reproduces the problem and traces it using Black Box with a recording profile provided by SAP

Support Upload of client log file to SAP by attaching it to the customer message on SAP Service Marketplace

AppSight Console “Identify”

Analysis of client log file offline by SAP Support consultant using AppSight System and AppSight Code


BMC AppSight has a Black Box and a Console. AppSight BlackBox is the agent on the client that records diagnostics data. It uses the generic hooks in the operating system and does not need modifications to the application on the client. Details to be recorded are defined in Recording Profiles. A movie of the activities on the user’s desktop can also be recorded. Appsight Console is a tool to analyze the client side log recorded by the Black Box. You can jump from the Appsight Console into Appsight Code which is used for an in-depth analysis of information from the execution of the application (e.g. when was a function called and how long did the function run).

© SAP AG

E2E100

5-24

Problem Analysis in AppSight System


The User View perspective in AppSight is the default. It is split into four areas where different panes are shown.

© SAP AG

E2E100

5-25

Visual Log

Following the Visual Log, the occurrence of the problem can be found.


This figure depicts how to navigate through the Visual Log to understand the end user‘s problem and to find the point of time in the recording where the problem occurred. You can let AppSight play the recorded screenshots like a slideshow or you can move back and forth manually.

© SAP AG

E2E100

5-26

Finding the Root Cause

Following the Performance Perspective, the expensive operation can be found.


The „System Performance“ pane you see the chart of the System Performance, which contains the consumption of physical memory and of CPU time. All metrics which you can obtain with „perfmon“ can be recorded with AppSight also. The advantage of AppSight is that it is correlated to the other operations by time, so that you can always check what was actually executed on the workstation at that moment. The System Performance Pane shows… System Performance Data The correlation to other activities on the system. The Performance Timeline shows… Which operations took place over the time. Different types of operations are marked in different colours. Gray areas indicate that no activity was recorded. White areas summarize many short operations. A horizontal red line shows the correlation to other panes. It is possible to set a filter for a specific time period. The „Performance Timeline“ is a timeline over the whole recording (or for a particular time frame if you have set an Advanced Filter on the time) that indicates what type of activity took place over the time. It is gray if either the system was waiting for user activity or if nothing was recorded at this time, because the type of operation was not enabled in the recording profile. During the white areas a lot of different and short running activity took place so that it can not be visualized in separate colours. Therefore the mix of the various small activities is displayed in white. © SAP AG

E2E100

5-27

The red line visualizes how long an activity which you have marked in another pane (e.g. the Log Operations pane) has taken on the timeline. Using an advanced filter you can limit the time frame to the particular period which you are interested in. With such a filter you avoid getting lost in too many log entries. Of course a prerequisite is to identify the relevant period in the recording via the Visual Log first. (You find this filter at „View“ -> „More Filters…“ -> „Advanced“.) The Application Performance Pane shows… Application operations categorized by type of operation. Can be sorted e.g. by duration Other panes are always kept in sync

© SAP AG

E2E100

5-28

Drill Down Analysis

The longest activity was the execution of „CScriptRuntime::Run“.


Now that we identifyed the most expensive operation can be analyzed in details In the „General“ tab in the „Application Performance“ pane we see that the longest running transaction was „CScriptRuntime::Run“. The „General“ tab in the „Application Performance“ pane shows the details per operation (in contrast to the „Code Operations“ tab on the next slide which shows the statistical summary for this method.)

© SAP AG

E2E100

5-29

Identifying the Performance Issue

A switch to the „Code Operations“ tab reveals that it is a method that is called very frequently. A double-click on „CScriptRuntime“ highlights the entry of this call in the „Log Operations“ pane (see next slide). © SAP AG 2007, Title of Presentation / Speaker Name / 1

The „Code Operations“ tab in the „Application Performance“ pane is the statistical view on (ONLY!) the code operations. It can tell you best, where you can and should optimize the performance of your coding. Also we can see here that „CScriptRuntime::Run“ was not only running for up to 21 seconds, but also that it was executed 210 times!

© SAP AG

E2E100

5-30

Code Operations in AppSight System

The Log confirms that this JavaScript is actually called very frequently!


As we are also recording Code Operations, we can now see the Code Operations in the „Log Operations“ pane now (above). But for a real code level analysis we better switch to AppSight Code via a right-click and „Show Event in AppSight Code“ for a deeper analysis.

© SAP AG

E2E100

5-31

Code Operations in AppSight Code

The functions that are called frequently are the candidates for a code review. © SAP AG 2007, Title of Presentation / Speaker Name / 1

In AppSight Code we see all the recorded calls to „CScriptRuntime::Run()“ und „SYM::InitFromSymbol()“ as well as which methods are called. Amongst other calls we find that the method „makeBusy“ is called frequently. (→ marked with orange arrow) Therefore we decide to have a look at the source code of the „makeBusy“ method.

© SAP AG

E2E100

5-32

Code Analysis

Analysis: The JavaScript functions identified in AppSight Code are analyzed. Result: These functions must be optimized!


As it is JavaScript, we can obtain the method from the HTML page. And actually we find expensive methods that explain the observed problem. In a real case you‘d have to discuss how these JavaScript functions can be optimized for more performance.

© SAP AG

E2E100

5-33

Fact Sheet: BMC AppSight for SAP Client Diagnostics An end user reports a problem with: A Windows fat client (e.g. SAP GUI, CRM Mobile Client, Duet Client) Internet Explorer

Use Case

Level 3 (Expert)

Trace the client side application Compare configuration between „good“ and „bad“ clients Provides Log with information on execution and exceptions of the running applications, configuration settings and a movie of activity on the host’s desktop. Best Practice for Investigation / Rules of Thumb Start with “User Actions” perspective Use “Visual Log” to navigate to Action to be analyzed Look for exceptions (lines in red) in “Log Operations” Pane Look for stack dumps from program crashes For performance analysis use “Performance” perspective © SAP AG 2007, Title of Presentation / Speaker Name / 1

Prerequisites Predefined Recording profile for Application must be available (e.g. from SAP) Limitations Only available for Windows 32Bit Platforms Has performance impact on recorded application (can be severe with heavy instrumentations) AppSight Code is used for an in-depth analysis of information from the execution of the application (e.g. when was which function called and how long did the function run).

© SAP AG

E2E100

5-34



E2E Trace Analysis







JAVA Trace Analysis










Functional Problem

© SAP AG 2006

© SAP AG

E2E100

5-35




?


?


?


WAN Network

Presentation Logic

Enterprise Portal

Application Logic

Database

ECC


© SAP AG

E2E100

5-36

Java Server Side Analysis: Example Java server analysis with Wily Introscope The company support organization finished the end-to-end trace Analysis and it turned out that the performance problem is related to the request processing inside a Java server. To begin analysis of the problem, the Wily Introscope Transaction Tracer tool will be used to identify the root cause.


© SAP AG

E2E100

5-37


Display Introscope Transaction Tracer Display J2EE SQL Trace summary Display Log Viewer Display aggregated ABAP Trace Display ABAP SQL Trace summary © SAP AG 2007, Title of Presentation / Speaker Name / 1

The figure above shows request handling for request ID 29. The request was sent from client to server via HTTP. The complete process time for this request was 1532 milliseconds. The J2EE processed the request, which took 1537 milliseconds; 952 milliseconds were spent in J2EE and about 21 milliseconds were used to establish five back-end connections. The RFC calls took 564 milliseconds; you can see the time for each single RFC connection. You can analyze the request with Introscope Transaction Tracer, J2EE SQL Trace and Log Viewer for the Java side, and ABAP Trace and SQL Trace Summary for the ABAP side.

© SAP AG

E2E100

5-38

Introscope WebView Transaction Tracer Viewer (3)

Trace result in Introscope WebView: Transaction Trace table

Transaction Trace table

Transaction Snapshot section © SAP AG 2007, Title of Presentation / Speaker Name / 1

The Transaction Trace table displays the transaction traces that exceeded the user-defined threshold execution time during the transaction trace session. Each transaction trace can be selected and its details can be analyzed in the Transaction Snapshot section. Each row represents a single transaction trace. Columns can be sorted by clicking on the column headers. If the table has been sorted new transactions will be inserted into the table in sorted order. To view a transaction trace as a Transaction Snapshot click on the row of the Transaction Trace table you wish to analyze. The transaction will be graphically represented in the Trace View section of the Transaction Snapshot area.

© SAP AG

E2E100

5-39


Component calling sequence

Transaction Snapshot: Trace view Component sequence over time Component

Tool Tip


Trace View shows the selected transaction in graphical format: Each component in the transaction as a bar The percentage of total transaction execution time for each component The calling relationships between components: The bars for components are displayed from top to bottom in calling order Transaction sequence over time: The placement of components from left to right indicates sequence. Relative wall clock time in milliseconds appears across the top of the Transaction Snapshot Errors within transactions (if ErrorDetector is installed): Red slices in the Transaction Snapshot represent errors within transactions In the Trace View of your workstation, you can: Hover your mouse over a component to open a tool tip Right-click a component to open the Investigator and view component metrics Select a component in the Trace View to open the Transaction Component Details pane

© SAP AG

E2E100

5-40


Transaction Snapshot: Tree view


The Tree view is a hierarchical view of the transaction’s components. Trace components that do not contribute a significant amount of time to the transaction are color-coded with a green icon.

© SAP AG

E2E100

5-41


Transaction Snapshot: Summary view


The Summary View shows metrics for the components in the selected transaction. Metrics include the path; number of calls; the length of the call in milliseconds; and the minimum, average, and maximum call times. You can double-click a metric to open it in the Investigator (Workstation only).

© SAP AG

E2E100

5-42


Transaction Snapshot – Transaction Details


Transaction Details shows information for the selected transaction: Host Agent name Domain Process User ID Start and end time Transaction duration in milliseconds

© SAP AG

E2E100

5-43

Wily Introscope Transaction Tracer

Exercise Wily Introscope Transaction Tracer


© SAP AG

E2E100

5-44

Fact Sheet: Introscope Transaction Tracer Level 2 (must know)

Use Case Identify the root cause of long running transactions Provides

Displays a list of all transactions in all managed Agents whose response times exceed a user-defined threshold Best Practice for Investigation / Rules of Thumb Set the threshold to a high value to really catch the slowest transactions In general the most suspect components will be those which are taking most of the parent component’s time but have no big time consuming children


Limitations: data can be just collected for the instrumented components Prerequisites: components have to be instrumented

© SAP AG

E2E100

5-45

Introscope WebView Transaction Tracer Viewer

Starting Transaction Tracer with Introscope WebView

1

3 Click

2

Choose New Trace


You can start a transaction trace from Introscope WebView, as well as from your Introscope Workstation. To use it from Workstation, choose Workstation Transaction Tracer New Session. To start a new transaction trace from Introscope WebView: Click on Transaction Viewer. Choose New Trace …. Choose the View button. The New Transaction Trace Session window appears.

© SAP AG

E2E100

5-46


Start tracing


In the Trace transactions section of the window, specify the threshold execution time. Select milliseconds or seconds from the dropdown list. Note that sub-second thresholds can have a negative impact on performance. To specify a transaction filter, select the checkbox to the left of the filter type list in the Trace transactions section, and select a type from the list: User ID: Enter an operator and a parameter value. Session ID: Enter an operator and a parameter value. URL, or URL Query: Enter an operator and a parameter value. Request Header: Enter a data type name, a condition, and a value. Request Parameter: Enter a data type name, an operator, and a parameter value. Session Attribute: Enter a data type name, an operator, and a parameter value. In the Trace Agents section, enter the length of the transaction trace session. In the Trace Agents section, select one or more agents for which to trace transactions: To trace all agents that support transaction tracing, choose Trace all supported Agents. This option traces supported agents that are currently connected, and any that connect during the trace session. To trace selected agents, choose Trace selected Agent(s) and select agents from the list (CTRL + click to select multiple agents). Click OK to start the transaction trace session.

© SAP AG

E2E100

5-47



E2E Trace Analysis







JAVA Trace Analysis










Functional Problem

© SAP AG 2006

© SAP AG

E2E100

5-48




?


?


?


WAN Network

Presentation Logic

Enterprise Portal

Application Logic

Database

ECC


© SAP AG

E2E100

5-49

ABAP Server Side Analysis: Example

ABAP server analysis with specific transactions The company support organization completed the end-to-end trace analysis and it turned out that the performance problem is related to the request processing inside an ABAP server. To begin analysis of the problem, the Single Transaction Trace (ST12) or ABAP SQL Trace (ST05) will be used to identify the root cause.


© SAP AG

E2E100

5-50

End-to-End Trace Analysis: Display ABAP Trace Summary

Display aggregated ABAP Trace © SAP AG 2007, Title of Presentation / Speaker Name / 1

© SAP AG

E2E100

5-51

St12: Collect ABAP trace – Select ABAP trace file

ST12

E2E Trace Analysis


© SAP AG

E2E100

5-52

ST12: Start Screen


Trace modes: User: Enter the name of the user you want to trace. The trace status symbol turns green when the user starts the transaction. There is also a button to stop the trace. Workprocess: Jump into an SM50 view where you can switch on the work process to be traced (for example a BTC work process where a batch job is already running). Current mode: Like the classic SE30, enter a transaction name. When a trace is started, a green light appears to the right of the Comment field, along with an end traces and collect button.

© SAP AG

E2E100

5-53

ST12: Tracing BSPs or many incoming RFCs

press

Choose ‚‘ if you do not know the logon server of the BSP user Performance (SQL) trace is switched on for the username Set the URL path filter if you want to trace only specific BSPs (start with ‚/‘, no trailing ‚/‘)

One click in a BSP page can result in many HTTP calls to the R/3 system for different screen elements. Each call gets its own ABAP trace. Enter here the max. number of ABAP trace activations per server.


Points to remember a) Trace incoming RFC by RFCUSER ‘’ waits for next user action systemwide When a trace is found to be active on one server trace requests on the other servers are cleared Do not leave SQL trace on systemwide! If you are not sure use ‚Utilities->Stop traces on all servers‘ to end the trace b) Trace BSP page or many incoming RFCs ‘Tasks & HTTP’ scenario for many trace activations one ABAP trace per screen element SQL/Performance trace is the same for all - for user & timeframe

© SAP AG

E2E100

5-54

ST12: Trace running a Batch Job


© SAP AG

E2E100

5-55

ST12: Start single transaction from ST12

The ABAP trace is activated dynamically during the transaction (this flag only works for the ABAP trace)


© SAP AG

E2E100

5-56

ST12: Gross time versus net time

Gross time is the summarized time over all call executions.

. . . SELECT . . . PERFORM . . . READ TABLE . . . . . .

Gross

Net time is the gross time minus the time when this mod.unit calls other mod.units minus the durations of simple statements that occur within this modularization unit AND that are explicitely measured.

Net


Gross time is the total time required to execute a statement or a modularization unit. Net time is the gross time minus the time taken by any modularization units and separate statements (which can be limited in the ABAP runtime initial screen using variants). The net time is the time that is not otherwise accounted for. If the gross time is the same as the net time, these times are not indicated separately. You can display times either as a percentage or as absolute times in microseconds. The addition 'explicitely measured' implies that net time can depend on the measurement scope. E.g. with trace scope 'with internal tables' the net times of mod.units can be lower because durations of statements on internal tables are also deducted. Like this the total sum of net time percentages always remains 100%. Gross times are used to get a top-down overview. Mod.unit names often give a good indication what happens below them, so that their gross time can be attributed to a certain functionality. Sorting by net times shows single expensive statements or mod.units that themselves consume much execution time.

© SAP AG

E2E100

5-57

1. Localized technical optimization / Net times

sort downward by column „Net time“(%)

© SAP AG 2006

Expensive database statements: Exact duration figures in % total response time and longer timeframes possible (30 min) than in SQL trace Expensive statments on internal tables: Optimize via Binary search techniques

© SAP AG

E2E100

5-58

Functionality Owner Identification


© SAP AG

E2E100

5-59

2. Optimization in the hierarchy above REPORT ZZBNDATRATEST . data: t1 type i, t2 type i.

Bottom-up call hierarchy Icon legend: Caller Called Call direction

define wait_ms. * * wait a certain number of milliseconds get run time field t1. do. get run time field t2. t2 = ( t2 - t1 ) / 1000. if t2 > &1. exit. endif. enddo. end-of-definition.

* * start of program start-of-selection. perform a1. perform a2. form a1. perform b. wait_ms 100. endform. form a2. perform b. wait_ms 100. endform. form b. perform c. wait_ms 300. endform. form c. wait_ms 800. endform.


© SAP AG

E2E100

5-60

Fact Sheet: Single Transaction Trace (ST12) Level 2 (must know)

Use Case Detailed analysis of a single transaction, program, Batch job, RFC

Provides “Gross”+”Net times” for units, Hierarchical top-down view, ABAP trace summary, short texts, ‘Time split hierarchy’, bottom-up call hierarchy

Best Practice for Investigation / Rules of Thumb Localized technical optimization = Net times tuning (~50% potential) binary search techniques for sorted standard tables buffering mechanisms sorted and hashed tables selectivity chain optimization

For aggregation by calls a bottom-up call hierarchy can be displayed Top-down gross time analysis (~30% potential) For functional analysis, you can display short description texts (titles of function modules, methods, programs, DB tables) To identify user exits or modifications, you can display the “Last changed by” information © SAP AG 2007, Title of Presentation / Speaker Name / 1

For optimization ABAP knowledge or developer is often required

© SAP AG

E2E100

5-61

End-to-End Trace Analysis: Display ABAP SQL Trace summary

Display ABAP SQL Trace summary © SAP AG 2007, Title of Presentation / Speaker Name / 1

The figure above shows request handling for request ID 29. The request was sent from client to server via HTTP. The complete process time for this request was 1532 milliseconds. The J2EE processed the request, which took 1537 milliseconds; 952 milliseconds were spent in J2EE and about 21 milliseconds were used to establish five back-end connections. The RFC calls took 564 milliseconds; you can see the time for each single RFC connection. You can analyze the request with Introscope Transaction Tracer, J2EE SQL Trace and Log Viewer for the Java side, and ABAP Trace and SQL Trace Summary for the ABAP side.

© SAP AG

E2E100

5-62

ABAP SQL Analysis (ST05)

ST05

E2E Trace Analysis © SAP AG 2007, Title of Presentation / Speaker Name / 1

© SAP AG

E2E100

5-63

ABAP SQL Analysis (ST05)


© SAP AG

E2E100

5-64

Detection Using SQL Trace (2)


Transaction ST05 allows you to activate and view SQL, Enqueue, RFC, and buffer traces. If an SQL Trace was performed, choose List trace to view the trace results. A screen is displayed like the one shown above indicating: Time and duration of the respective database operations Table accessed SQL statement used The lines with FETCH are part of the SQL statement in the previous row. For each FETCH the number of records transferred between application and database servers as a result of the SQL statement are indicated in the Rec column. To find which fields and indexes exist for the table accessed choose DDIC info. To find which index has been used by the database optimizer for the SQL choose Explain SQL.

© SAP AG

E2E100

5-65

Checking the Index Details Using “Table Detail”


© SAP AG

E2E100

5-66

Two Types of Expensive SQL Statements

1

Application server

Many records

2 ABAP program Database interface

Application Server

Few records

ABAP program Database interface

Records processed DBMS

DBMS

Database buffer

Many blocks

DBMS process

Database buffer

DBMS process

Buffer gets

Many blocks Disk reads Database

Database


There are two types of expensive SQL statements. Type I: The statement scans many data blocks in the database and is expensive because many records are transferred to the ABAP program. Database performance is optimal. Type II: The statement scans many data blocks in the database but transfers only a few records to the ABAP program. Database performance is not optimal, for example due to an inefficient search strategy.

© SAP AG

E2E100

5-67

Fact Sheet: SQL Trace (transaction ST05) Level 2 (must know)

Use Case Traces of single DB statements, Enqueues, RFCs Provides

“Net times” for single statements, aggregated “statement summary”, details on SQL request execution Best Practice for Investigation / Rules of Thumb Use Transaction GUID and time interval to select and display corresponding SQL trace Find SQL requests with longest execution time Analyze execution plan, indexes used

In “Summarized SQL Statements” view check for SQL request with high percentage of identical select Check how often they are executed, total duration, number of returned records Analyze SQL request © SAP AG 2007, Title of Presentation / Speaker Name / 1

Limitations: Trace has to be switched off. Trace file on server - can be overwritten Manual trace file switch

© SAP AG

E2E100

5-68

End-to-End Trace Analysis: Exercise

Exercise Single Transaction Trace (ST12)


© SAP AG

E2E100

5-69

End-to-End Trace: Unit Summary

You should now be able to: Perform an end-to-end trace to isolate a performance problem or error Trace a client-side application with BMC AppSight Trace an application on Web AS Java with Wily Introscope Transaction Tracer Trace an application on Web AS ABAP with SQL Trace (ST05) and Application Tracing (ST12)


© SAP AG

E2E100

5-70

Exercises Unit:

E2E Trace Analysis

Lesson:

E2E Trace Analysis

At the conclusion of this exercise, you will be able to: • Identify the cause of a long running user request by using the End 2 End Trace Analysis functionality of the Solution Manager.

1

Please connect to your Solution Manager and start the diagnostics part. Log In with user and password provided by the trainer. Choose the solution provided by the trainer and execute the E2E Trace Analysis tool (Traces -> E2E Trace Analysis). 1-1

Please mark ‘E2E_Trace_01’ in the Business Transactions table.

1-2

Please mark ‘BankInfo’ in the Transaction Steps table and click ‘Display’.

1-3 Analyze the transaction and answer the following questions: 1-3-1

How long was the total transaction duration time?

1-3-2

How was the time distribution over Client, Network and Server?

1-3-3

What is the ID of the longest running request?

1-3-4

How long was this request running?

1-3-5

What is the name of the slowest function module?

1-3-6

How much CPU time spent the slowest function module?

1-3-7

What is the instance name of the bad performing component?

1-3-8

How many bytes has sent the slowest function module?

1-3-9

What was the last modification date of the bad performing program part?

1-3-10 Who did the last modification of this program part?

© SAP AG

E2E100

5-71

© SAP AG

E2E100

5-72

Exercises Unit:

E2E Trace Analysis

Lesson:


At the conclusion of this exercise, you will be able to: • Use the Wily Introscope Transaction Tracer • Identify an application as root cause of critical performance issues.

1

Please connect to your Solution Manager and start the diagnostics part. Log In with user and password provided by the trainer. Choose the solution provided by the trainer and navigate to the E2E Trace Analysis menu. Open E2E Trace provided by trainer and find the longest request. 1-1

Open for this request Wily Introscope Transaction Trace. Choose the Transaction Viewer. Analyze the transaction and answer the following questions: 1-1-1

What is the name of the application that causes the bad performance?

1-1-2

How long was the duration time of this application?

1-1-3

What is the Java package name of the application? Additional: Try to get the same information from the “Tree View”.

© SAP AG

E2E100

5-73

© SAP AG

E2E100

5-74

IT Reporting

Unit:

E2E Trace Analysis

Lesson:


Exercises

At the conclusion of this exercise, you will be able to: • Use the Wily Introscope Transaction Tracer • Identify an DB query as root cause of critical performance issues.

1

Please connect to your Solution Manager and start the diagnostics part. Log In with user and password provided by the trainer. Choose the solution provided by the trainer and execute the Wily Introscope Web Viewer. Log in with user and password provided by the trainer. 1-1

Choose the Transaction Viewer and start a new trace. Trace transactions lasting longer than 20 seconds. The session should run for 10 minutes. Please choose the agent provided by the trainer. Start the session. Please stop tracing ones you caught a transaction. Analyze the transaction and answer the following questions: 1-1-1

Which SQL statement caused the problem?

1-1-2

How long was the query duration time? Additional: Try to find the SQL statement in the Investigator mode.

© SAP AG

E2E100

5-75

© SAP AG

E2E100

5-76

IT Reporting

Unit:

E2E Analysis of client-side issues

Lesson:

AppSight Analysis

Exercises

At the conclusion of this exercise, you will be able to: • Use BMC AppSight System for analysis • Identify a client-side configuration problem. Case description: An end user reports problems with SAPGUI. Sometimes a “Runtime Error” resp. “Frontend Error” occurs. The end user has made a recording of the problem with an AppSight BlackBox and a Recording Profile that you provided. Now you have to analyze the problem based on the AppSight Log that you received from the end user.

1

First you have to connect to the RKT Windows Terminal Server in the SAP Training Environment. Required: Citrix Client on your workstation. 1-1

If you are at a training site of SAP or on the SAP Intranet please open http://wts.wdf.sap.corp in Internet Explorer. If you can open this URL, then please proceed here. If you cannot access this URL please continue with 1.2. instead.

Click “EMEA”.

© SAP AG

E2E100

5-77

Click “Training Zone”.

Click “RKT Productive”. Then a Citrix session to the Windows Terminal Server where AppSight is installed will be opened without requiring a login or password.

1-2

Only if you are at a customer site or cannot access the above URL, then please connect to the RKT Windows Terminal Server from the Internet via http://mywts.sap.com.

Click “EMEA”.

© SAP AG

E2E100

5-78

Click “Training”.

Please enter the login and the password for accessing the WTS which you have received from your trainer.

Click “RKT Workshop”.

© SAP AG

E2E100

5-79

Click “RKT Productive”. Then a Citrix session to the Windows Terminal Server where AppSight is installed will be opened without requiring another login or password.

2

After you are successfully logged on to the WTS now, you have to open the AppSight Console. 2-1

In many cases AppSight has already been used on the WTS, so that you don’t have to make all the initial setup steps. Therefore please try the following first…

Open the “My Documents” folder. If you find a sub directory “AppSight” there, then double-click on the file “E2E100_AppSight_Exercise_01.asl”. If this successful and AppSight opens, then continue with number 3. directly. If the sub directory or the file is missing or AppSight does not open correctly, then please continue with number 2.2.

© SAP AG

E2E100

5-80

2-2

Initialize and set up the AppSight Exercise on the WTS. 2-2-1

Please navigate to “Identify AppSight” in the Start Menu and select “Initialize AppSight”.

If you get a warning, that there are existing AppSight Training files, please proceed and overwrite these files. (This script only copies the required files to your “My Documents” folder. It finishes without a message. You can continue with 2.2.2 immediately.) 2-2-2

Open “AppSight System” from the “Programs” → “Identify AppSight” folder in the Start Menu. (In case you were asked for a license, please choose “License Server” and name the host “appsight.wdf.sap.corp” as license server.)

2-2-3

Connect the the AppSight Server.

Select “Connect to AppSight Server …” from the “File” menu.

© SAP AG

E2E100

5-81

Enter the host name “tdcappsight.wdf.sap.corp” as AppSight server name. Afterwards you should see the following:

The host name can vary, but AppSight System should show that it is connected to this AppSight server. 2-2-4

Open the AppSight Log for analysis:

Select “Open AppSight Log…” from the “File” menu. Navigate to folder “My Documents\AppSight” and open the file “E2E100_AppSight_Exercise_01.asl”.

© SAP AG

E2E100

5-82

3

Analyze the AppSight Log and find out what caused the problem. Recommendation: 3-1

Make sure that the “Visual Log” displayed in AppSight System. If not, then please choose “View” → “Perspectives” → “User Actions”.

3-2

View the screen movie by replaying the “Visual Log”. (Be aware that after opening an AppSight Log the pointer is at the end of the recording, so that you have to navigate to the start, before you can play the screen movie.)

3-3

Navigate thru the “Visual Log” for understanding the user activity and for identifying the activity before and after the occurrence of the problem which define the time frame in which the error occurred.

3-4

Find the relevant information on the root cause of the problem by checking the entries in the “Log Operations” in the time frame you identified before.

© SAP AG

E2E100

5-83

© SAP AG

E2E100

5-84

Exercises Unit:

E2E Trace Analysis

Lesson:

ABAP Trace Analysis via Single Transaction Trace (ST12)

At the conclusion of this exercise, you will be able to: • Identify the cause of a long running function module by using the ST12 Single Transaction Trace

1

Please logon to the backend system (TT5) and start transaction ST12. 1-1

Record ABAP Trace. Since only one ABAP Trace could be taken at the same time two teams should work together. One Team executes ST12 and traces the other team. Team A: 1) Call Transaction ST12 2) Choose "Tasks& HTTP"scenario, Title "Team A and B Trace RFC " 3) Enter the user name of Team B, task type RFC 4) Set the flag "With internal tables" 5) Start Trace Team B: Execute report ZZST12_CALL_PERFORMANCE_SAMPL from SE38 Wait approx 15 sec until report finishes

© SAP AG

E2E100

5-85

Team A: 6) Stop Trace&Collect Team A&B: Display ABAP Trace Open ABAP trace for the report “ZZST12_PERFORMANCE_RFC”. 1-2

Use the default view "Per calling position" and sort descending by Net time. 1-2-1

What are the 4 most expensive statements?

1-3 Switch to the view "Per Mod.unit" and expand the modularization unit with highest net time. Look at the expensive "Read Table

© SAP AG

1-3-1

How much faster is the read with binary search per execution

1-3-2

How many lines approximately does the internal table LT_VBAP have? Hint: Look at the number of executions of the statements.

E2E100

5-86

Solutions

1-1

Unit:

E2E Trace Analysis

Lesson:

E2E Trace Analysis

Analyze an already recorded E2E Trace by following the steps as shown in the example below:

© SAP AG

E2E100

5-87

The requested information can be found in the marked areas.

1-3-1

The total transaction duration time, for the example was 1638 ms.

1-3-2

The time distribution is the following: Client time: 410 ms Network time: 461 ms Server time: 10865 ms Conclusion: The performance problem is related to the request processing on server side.

© SAP AG

E2E100

5-88

Please continue your analysis by changing to the ‘Messages Table’ view. The requested information can be found in the marked areas.

© SAP AG

1-3-3

The ID of the longest running request of this example is 29.

1-3-4

The request took 10735 ms.

1-3-5

The slowest function module is named ‘HRXSS_PER_INIT_PERNR’.

E2E100

5-89

Please continue your analysis by marking the row of the slowest function module. The requested information can be found in the marked areas.

1-3-6

The slowest function module spent 10090 ms CPU time.

1-3-7

The instance name of the bad performing function module is ‘Is165610_HNI_01’.

1-3-8

The function module sent 806 bytes.

Please continue your analysis by opening the aggregated ABAP trace. The requested information can be found in the marked areas.

1-3-9

The last modification date was 2007-02-06.

1-3-10 The program part was last changed by user E2EDEMO. © SAP AG

E2E100

5-90

Solutions

1

Unit:

E2E Transaction Trace

Lesson:

Wily Introscope Transaction Trace

You should open Wily Introscope Transaction Trace as follows: - Navigate to the Root Cause Analysis -> Traces -> E2E Trace Analysis and open E2E Trace provided by trainer:

© SAP AG

E2E100

5-91

After that please find the request with the longest execution time and display Introscope Summary for this request

Then please navigate to the “Detail”tab and start Introscope WebView from this tab:

© SAP AG

E2E100

5-92

After a transaction is displayed please analyze it. To analyze the transaction just mark it in the table. The requested information can be found in the areas which are marked in the example below

© SAP AG

1-1-1

The name of the application is: E2E_Exercises

1-1-2

The duration time is 33909 ms.

1-1-3

The Java package name is: com.sap.sup.e2e.exercises.wdp.

E2E100

5-93

© SAP AG

E2E100

5-94

Solutions

1-1

Unit:

E2E Transaction Trace

Lesson:

Wily Introscope Transaction Trace

You start a new trace by following the steps as shown in the example below:

After you choused “New Trace”, please click on “View” to do the trace specifications like shown below:

Click the “OK” button to start the trace session.

© SAP AG

E2E100

5-95

After a transaction is displayed, you can stop the trace. To analyze the transaction just mark it in the table. The requested information can be found in the areas which are marked in the example below

1-1-1

The name of the SQL statement is: SELECT * FROM ….

1-1-2

The duration time is 37804 ms. Additional: The Metric Path guides you to the tree structure of the Investigator mode.

© SAP AG

E2E100

5-96

Solutions

1

Unit:

E2E Analysis of client-side issues

Lesson:

AppSight Analysis

Following this guideline, you can identify the root cause of the error in SAPGUI, which you can see in the “Visual Log”. 1-1

Optimize your AppSight Desktop for viewing the Visual Log by choosing the “User Actions” “Perspective”:

(To best see the recorded user activity of the Visual Log, you might want to enlarge the “Visual Log” frame to the left and to the bottom by dragging the borders of the frame.) 1-2

© SAP AG

Rewind the Visual Log to the start, since it will be at the end of the recording, when you open the recorded log:

E2E100

5-97

1-3

Then play the screen movie:

1-4

Navigate thru the Visual Log to identify the activity before and after the occurrence of the problem which define the time frame in which the problem occurred. The best way to do this is by disabling all metrics in the “Filter” pane (left bottom frame) by unchecking “Operations” and then activating only the metric “GUI Operations”. Then you can navigate in the “Log Operations” window and the “Visual Log” is synchronized automatically.

1-5

© SAP AG

Set a filter by time which includes only this period from before the problem occurred until after the problem occurred. E2E100

5-98

To do this go to “View” → “Filter Log” → “More Filters…” and switch to tab “Advanced”.

Enter the relevant time frame here and click “OK”. (Now you will only see events within this time frame.) 1-6

Activate the other metrics in the “Filter” pane again by activating “Operations” and search for relevant errors in the “Log Operations” pane. If you double-click on an entry in the “Log Operations” pane, then the “Details” pane will show all details of that log entry.

1-7

You will find that the SAPGUI tried several times to load the library “SAPfewin.OCX” unsuccessfully.

The fact that the file “SAPfewin.OCX” was missing on the file system caused this error. © SAP AG

E2E100

5-99

2

How can you tell that the root cause was not the following error below, which is also in the time frame in which the error occured:

•

The exception due to the missing OCX file happens first and you can see in the screenshot above that before this missing log file should be opened, already many error messages were written to a SAPGUI error trace file. You can even check the text of these messages in the “Comment” column or – after you double-clicked on the line in the Log - in the “Details” pane.

•

Overall it is simply much more likely that a missing OCX file leads to such exception and not a missing log file.

•

But finally it is you who is doing the analysis and not the AppSight tool! You have to interpret what you find with AppSight.

© SAP AG

E2E100

5-100

Solutions

1-1

Unit:

E2E Trace Analysis

Lesson:

ABAP Trace Analysis via Single Transaction Trace (ST12)

Start transaction ST12 and analyze an already recorded ABAP Trace as shown in the example below: Team A - E2EDEMO Team B – E2E100-20 Team A:

© SAP AG

E2E100

5-101

Team B:

Team A:

© SAP AG

E2E100

5-102

Team A&B:

© SAP AG

E2E100

5-103

1-2-1

1-3

1-3-1

© SAP AG

E2E100

5-104

Display source code to find which request is executed with binary search

without binary search

with binary search

Compare the source code and ABAP trace. You can see that the calls in the ABAP trace are displayed in the same sequence as they have been executed.

without binary search

with binary search

© SAP AG

E2E100

5-105

Here you can see that the statement executed without binary search has taken 3.963.621 microseconds while the same statement executed with bnary search has taken only 14.359 microseconds 1-3-2

© SAP AG

If you compare coding and results of ABAP trace you can notice that at the very beginning of source code “APPEND LS_VBAP TO LT_VBAP.” Has been executed 14.430 times. Then the internal table LT_VBAP should have approximately 14.430 lines

E2E100

5-106

End-to-End Exception Analysis


© SAP AG


E2E100

6-1


Contents: End-to-End Exception Analysis in SAP Solution Manager Log Analysis in ABAP and non-ABAP Environments Dump Analysis in ABAP and Java environments


© SAP AG

E2E100

6-2

End-to-End Exception Analysis: Unit Objectives

After completing this unit, you will be able to: Perform an end-to-end exception analysis in SAP Solution Manager Analyze logs in ABAP and non-ABAP environments Analyze dumps in ABAP and Java environments


© SAP AG

E2E100

6-3

End-to-End Exception Analysis: Unit Overview

End-to-End Exception Analysis Lesson 1: End-to-End Exception Analysis Lesson 2: Web AS Java Exception Analysis Lesson 3: Web AS ABAP Exception Analysis


© SAP AG

E2E100

6-4



E2E Trace Analysis







JAVA Trace Analysis










Functional Problem

© SAP AG 2006

© SAP AG

E2E100

6-5

Root Cause Analysis Roadmap

Root Cause Analysis (Cross-Component) ?

Is it a Performance problem? E2E Workload Analysis Is it a Functional problem?

? ?

System or Application termination E2E Exception Analysis

?

Inconsistent data E2E Data Inconsistency Analysis

?

Incorrect results or error messages in application E2E Application Root Cause Analysis


© SAP AG

E2E100

6-6

End-to-End Exception Analysis: ABAP vs. Java

Exception Analysis Area

ABAP

Java

Solution-wide Exception Analysis


Program Termination

ABAP Runtime Error (ST22)

Log Viewer - applications*.log

Server Termination

System Log (SM21)

Log Viewer - std_server*.out

System Error

System Log (SM21)

Log Viewer -applications*.log -defaultTrace*.trc

Application Error

Analyse Application Log (SLG1)

Log Viewer -applications*.log

© SAP AG 2006

The table shows the different tools that can be used for exception analysis in the ABAP and Java (J2EE) environments. Although the same terms like dumps and application log exist in both environments, they do not always have the same meaning. While the application log is comparable, dumps are different in ABAP and Java. In ABAP, an unhandled exception of a program, for example, can cause an dump. The dump causes the program to end but will have no impact on other running programs in the ABAP system. In Java, an “out of memory” error for a server node can cause a cump. After the dump, the server process will restart, so it will impact all users and all programs running on this server node.

© SAP AG

E2E100

6-7

End-to-End Exception Analysis: Example

Exception Analysis example with an Employee SelfService (ESS) Scenario An employee of a company recently moved and wants to inform the HR department about the new address. The employee logs on to the corporate portal and navigates to Employee Self-Service. In the Personal Information area, the address data can be changed by the employee. When the employee clicks on the “Addresses” link, an error message is displayed in Internet Explorer.

To analyse the problem, the End-to-End Exception Analysis application will be used to locate the root cause.


The problem cannot be analyzed directly after the error occurred. First, log data from the different components needs to be collected by the Solution Manager. By default, the Scheduler in Solution Manager runs the “E2E Log Analysis Task” every 60 minutes.

© SAP AG

E2E100

6-8

Fact Sheet: E2E Exception Analysis Level 2 (must know)

Use Case Exception trend analysis Starting point for analysis of a specific exception Provides

Overview of the exceptions in different components of a solution

Best Practice for Investigation / Rules of Thumb Do a timeframe selection by day, week, or month to see the exception trend in a solution and its components. For analyzing a specific exception, select a timeframe of several minutes around the point in time when the exception occurred. Further investigate the components where exceptions occurred.


Prerequisites: The E2E log analysis task needs to run before a problem can be analysed with the E2E Exception Analysis application. By default, this task is executed every 60 minutes.

© SAP AG

E2E100

6-9

End-to-End Exception Analysis: Start Select component for detailed analysis

Select Timeframe or do a custom selection

Press to see all components of the solution in one list

Exception overview by component

Analysis tabs for errors and dumps © SAP AG 2006, Title of Presentation / Speaker Name / 1

To start an E2E Exception Analysis, log in to Solution Manager and execute transaction DSWP (or SOLUTION_MANAGER), then navigate to GoTo Start Solution Manager Diagnostics. In SMD, navigate to Root Cause Analysis -> Exceptions -> E2E Exception Analysis. Choose your solution and select all components. Click on Start to begin with your investigation. The picture above shows the entry screen with feature description. Here you get a good overview of the exceptions in your solution for the chosen timeframe. If you do a custom selection for the timeframe, press „Apply Filter“ to activate your selection.

© SAP AG

E2E100

6-10

Table Filter and Column Sort

Clear the filter settings

Set a filter on the table

Click on column header to sort the column


In the Exception Analysis tables, there is the following functionality to search for entries: Table Filter: If there is a table in the exception analysis screen of a component, you can search or resrict the displayed lines with a filter. In the example screen, a filter was used to display only the Java error messages from the location “com.sap.ic.ias.persist”. Column Sort: If there is a table in the exception analysis screen of a component, you can sort a column by clicking on the column header.

© SAP AG

E2E100

6-11

J2EE Component Java Errors The displayed information is the result of analyzing the J2EE application log and default trace files (applications*.log and defaultTrace*.trc)

Top 20 frequent errors

Top J2EE errors by location Top J2EE errors by user © SAP AG 2006, Title of Presentation / Speaker Name / 1

In the exception analysis screen of a J2EE component, there is a tab for Java Errors and Java Dumps. In the Java Errors tab you get an overview of the: Top 20 frequent errors Top J2EE errors by location Top J2EE errors by user

© SAP AG

E2E100

6-12

J2EE Component Java Errors: Details

Press to show full message text

Press to show error message in separate log viewer window


For further analysis of errors displayed in the J2EE component Java error top 20 list you have two possibilities: Press the “Details” button to see the full message text. In the example this shows the complete call stack. This might help give a better understanding of where the error occurred, as well as provide more information that can be used in an SAP Note search. Also, you can select the “Jump In” button. A new browser window will open which shows the error in the Log Viewer. Seeing the error message in the context of other messages in the error log might help to better understand the problem.

© SAP AG

E2E100

6-13

J2EE Component Java Errors: Details (2) Display unfiltered messages before and after the error

List all error messages matching the error pattern


Opening an error message from the “Top Frequent errors” list in a separate Log Viewer window provides additional information: List all errors that match the error pattern, so you can see when the errors occurred. Display unfiltered messages in the log before and after the error message, so you can see the error message in the context of other messages in the log.

© SAP AG

E2E100

6-14

J2EE Component Java Dumps The displayed information is the result of analyzing the J2EE standard output files (std_server*.out) Press to show the full thread dump


The J2EE component “Java Dump” tab provides an overview of when Java dumps occurred in the selected timeframe. As previously stated, Java dumps are different from ABAP dumps. If a Java dump (out of memory condition) occurs, the server node process will restart, and all users and programs running there are affected. By pressing the “Details” button, you will see the full thread dump written before the process restarts. There you can see which threads were active when the dump occurred. If Java dumps occurs, further analysis is necessary. Either the Java VM settings are wrong, or an application has a memory leak. More details are provided in the E2E Workload Analysis chapter.

© SAP AG

E2E100

6-15

ABAP Component ABAP Errors The displayed information is the result of analyzing the ABAP system log

ABAP errors in selected timeframe

Number of ABAP errors per user


In the exception analysis screen of an ABAP component there are two tabs, one for ABAP errors and one for ABAP dumps. The ABAP errors tab provide the following: The “ABAP Errors Report” table shows all errors from the system log in the selected timeframe. The “Top ABAP Errors by User” table shows which user caused most of the system log entries. The current version of the E2E ABAP exception analysis does not include the ABAP application log.

© SAP AG

E2E100

6-16

ABAP Component ABAP Errors: Details

Press to show the error details

Press to show jump in details to the ABAP system


To further analyze an ABAP error from the ABAP Error tab, there are two possibilities: When you mark an entry and press the “Details” button, a pop-up window will appear displaying all the information for this table row. For further analysis of the error in the ABAP system, you can press the “Jump In” button. A pop-up will appear showing all information required to find the error entry in the ABAP system. You need to manually login to the ABAP system and search for the error using the given information. Future releases of E2E Exception Analysis will allow users to directly jump to the ABAP system and display the error message there.

© SAP AG

E2E100

6-17

ABAP Component ABAP Dumps

ABAP dumps in selected timeframe

Number of ABAP dumps per user

Number of ABAP dumps per component


The ABAP dumps tab provides the following: The “ABAP Dumps Report” table shows all dumps in the system in the selected timeframe. The “Top ABAP Dumps by User” shows which user caused most of the dumps. The “Top ABAP Dumps by Component” shows in which ABAP component most of the dumps occurred.

© SAP AG

E2E100

6-18

ABAP Component ABAP Dumps (2)

Press to show the dump details

Press to show jump in details to the ABAP system


To further analyze a dump from the ABAP Dumps tab, there are two possibilities: When you mark an entry and press the “Details” button, a pop-up window will appear displaying all the information for this table row. For further analysis of the error in the ABAP system, you can press the “Jump In” button. A pop-up will appear showing all information required to find the error entry in the ABAP system. You need to manually login to the ABAP system and search for the dump using the given information. Future releases of E2E Exception Analysis will allow users to directly jump to the ABAP system and display the dump there.

© SAP AG

E2E100

6-19




© SAP AG

E2E100

6-20



E2E Trace Analysis







JAVA Trace Analysis










Functional Problem

© SAP AG 2006

© SAP AG

E2E100

6-21

Java Exception Analysis: Example

Exception analysis example with an Employee Self-Service (ESS) scenario (continued) The End-to-End Exception Analysis application showed Java errors which need to be further investigated.


© SAP AG

E2E100

6-22

Fact Sheet: Log Configuration Level 2 (must know)

Use Case No useful information is written to the log and trace files Performance problems of an application Provides Severity setting overview for categories and locations Best Practice for Investigation / Rules of Thumb

Check the severity setting of the Logging Categories and/or Tracing Locations you are interested in Check the minimum and maximum severity of the subtree to find wrong severity settings

© SAP AG 2006

You should know the following about logging and tracing in a SAP J2EE Engine: Logging: - “Classical” log messages, for particular problematic areas - Logs are written to categories Tracing - “Classical” trace messages, for coding problems - Traces are written to locations The following severities can be set for categories and locations: All, Debug, Path, Info, Warning, Error, Fatal, None The severity setting can be changed at runtime. No restart of the J2EE is required. The default severity setting for a productive system should be “Error”.

© SAP AG

E2E100

6-23

J2EE: Log Settings

Display of J2EE log settings in Solution Manager Diagnostics Select logging type categories or locations

Search for a category or location

Select row for further details

Filter options

Information about node subtree severity


To start Java Log Configuration: Log into Solution Manager and execute transaction DSWP (SOLUTION_MANAGER), then navigate to GoTo Start Solution Manager Diagnostics. In Diagnostics, navigate to Root Cause Analysis Configuration Java Log Configuration. Choose your solution and select one component. Click on Start to begin with your investigation. The picture above shows the initial screen with feature description. Here you can get an overview of the current log settings of the J2EE. The usability is better than the Visual Administrator’s, as you can: Easily see the severity setting of a subtree, whereas in the Visual Administrator you have to drilldown. Search for a category or location, which is not possible in the Visual Administrator. Filter by category, location, file name pattern and severity, which is not possible in the Visual Administrator

© SAP AG

E2E100

6-24

J2EE Log Settings: Category Details

Select row for category details

System configuration:

Per cluster node configuration:

Select node to see assigned destination © SAP AG 2006, Title of Presentation / Speaker Name / 1

By selecting a category node or entry, more details will be displayed. The log settings in a J2EE can be changed system-wide, or differently on each cluster node. Therefore the Detail view provides two tabs: System configuration Per cluster node configuration This enables you to easily get an overview of the log settings in the complete cluster.

© SAP AG

E2E100

6-25

J2EE Log Settings: Location Details

Select row for location details

System configuration:

Per cluster node configuration:

Select node to see assigned destination © SAP AG 2006, Title of Presentation / Speaker Name / 1

By selecting a location node or entry, more details will be displayed. The log settings in a J2EE can be changed system-wide, or differently on each cluster node. Therefore the Detail view provides two tabs: System configuration Per cluster node configuration This enables you to easily get an overview of the log settings in the complete cluster.

© SAP AG

E2E100

6-26

Fact Sheet: Log Viewer Level 2 (must know)

Use Case Further investigation of Java exception

Provides Detailed information of the exception from the defaultTrace.trc and applications.log Best Practice for Investigation / Rules of Thumb Display the merged defaultTrace.trc or applications.log files. Filter a timeframe of one or several minutes around the time when the exception occurred. Check the entries before and after the exception. Merging defaultTrace.trc or applications.log might give a better understanding of what happened. © SAP AG 2006, Title of Presentation / Speaker Name / 1

© SAP AG

E2E100

6-27

Log Viewer: Selection Features View selector

Log file selector Only available if Open Filter button was pressed

Merge logs option

Content filter


For further analysis of a Java error log message that is shown in E2E Exception Analysis, it might be necessary to see the available trace information in the trace file. By default the trace files are written with a formatter that is not human readable (at least the time stamp is not readable). The best way to analyze the trace files is to use the Log Viewer. To start the Log Viewer: Log in to Solution Manager and execute transaction DSWP (SOLUTION_MANAGER), then navigate to GoTo Start Solution Manager Diagnostics. In SMD navigate to Root Cause Analysis Exceptions Log Viewer. Choose your solution and select the component for which you want to analyze the trace files. Click on Start to begin with your investigation. The picture above shows the entry screen with feature description. View Selector: Here you can select predefined and custom views. A Log Browser selection can be stored as a new view and easily reused. Log Browser: In the Log Browser, you can be select any file known to the Log Viewer. When doing a wildcard selection, “Merge Logs if Possible” can be used to merge all files matching the wildcard selection criteria. A merge is only possible if the log files are written in the same format. Content Filter: Here you can filter on values in the different columns.

© SAP AG

E2E100

6-28

Log Viewer: Display Features Search in the displayed log files. Only available after pressing “Open Search”

Number of records to be displayed

Navigation buttons

Column customization to display/hide columns and arrange its sequence. Only available after pressing “Open Column Customization” button.


When the Log Browser log file selection and content filtering is done, log entries are displayed. In the Log Display, the Log Viewer has several useful features: Navigation buttons: These buttons are used to jump to the beginning or end of the displayed messages, go one page up or down, and go one message up or down. If “Filter by Content” is used, you can show unfiltered entries before and after the selected row. Number of records: Here you can specify how many messages are displayed on one page. You may adjust this according to your screen resolution, in order to avoid getting a scroll bar on the right side of the screen. Search: Here you can search in one of the columns for a value (including wildcards if necessary) using a logical expression for the search criteria.The search fields becomre visible after pressing the “Open Search” button. Column Customization: When you press “Open Column Customizing”, a frame appears on the left side of the screen with a list of all available columns in the displayed log. Here you can select which columns get displayed, and you can change the display sequence of the columns.

© SAP AG

E2E100

6-29

Log Viewer: Display Features (2) Downloads the content of the displayed log files

Shows details of the displayed log files in a pop-up window

Shows record details of selected row

Shows shows all record details


Open Log Details: This opens a pop-up window with a list of all displayed log files and detailed information about the log files (file location, log name, last modified date and time, and size in bytes). Download Content: Downloads all rows which are displayed according the “Log Browser” and “Filter by Content” settings into a comma separated file (*.csv). If the download is bigger than 10MB, it is split into several files, each with a size of 10 MB. Open Record Details: Shows a detailed view of the selected row. Open Full Record Information: This button is only available in the Record Detail view. It displays all available record information.

© SAP AG

E2E100

6-30

Log Viewer: Display Trace Files

Selection criteria to get traces files of a J2EE component displayed in the Log Viewer 1. Select “Custom View” “Log and trace files”

2. Select “Log file named” “as” “defaultTrace*” 3. Set flag at “Merge Logs if Possible”

4. Press the “Apply Logs” button


If you want to see trace information for further analysis of an error, follow the steps described in the slide to display the trace files in the Log Viewer. Use the filter or search functionality of the Log Viewer to find the entries of interest.

© SAP AG

E2E100

6-31




© SAP AG

E2E100

6-32



E2E Trace Analysis







JAVA Trace Analysis










Functional Problem

© SAP AG 2006

© SAP AG

E2E100

6-33

ABAP Exception Analysis: Example

Exception analysis example with a Employee Selfservice (ESS) scenario (continued) The End-to-End Exception Analysis application showed ABAP system log errors which need to be further investigated.


© SAP AG

E2E100

6-34

Fact Sheet: ABAP System Log (SM21) Level 2 (must know)

Use Case Further investigation of a ABAP system log entry Provides Details of the system log entry

Best Practice for Investigation / Rules of Thumb Check the details of the system log entry. Check the system log entries before and after the one directly related to the problem. Check the trace file of the worker process, in the event of a connection error.


Limitation: If the system log error displayed in the E2E Exception Analysis application is too old, you might not find it in transaction SM21 because it was overwritten.

© SAP AG

E2E100

6-35

ABAP System Log: Search Log Entry

Press after entering the selection criteria

Select timeframe when the error occurred

Select instance where the error occurred


To search for an ABAP System Log entry using the information shown in the “Jump In” pop-up window of E2E Exception Analysis, do the following: Logon to the ABAP system and start transaction SM21. In the menu, select “System log” “Choose” “Remote system log”. Select the correct instance name via F4 help according the information given in the pop-up (host and system number). Select a timeframe in which the error occurred. Press the “Reread system log” button to get the entries displayed which match your selection criteria. The ABAP system log is stored in a file on every application server. Every log entry has a fixed size. The file size is limited according to a profile parameter (rslg/max_diskspace/local), and the file is overwritten when the maximum length is exceeded. Therefore, only a limited number of log entries are available in the System Log. If you are searching for an error that occurred too long ago, you might not find it because it was overwritten by newer entries.

© SAP AG

E2E100

6-36

ABAP System Log Analysis

Press the “Details (F2)” button after selecting a log entry to see its details

Error message


When you find the System Log entry according to the “Jump In” information, press the “Details” button (or shortcut key F2) to display the detailed error message. As a first step, search for SAP Notes with the key words of the error message. In this example, a search for “ICM_HTTP_CONNECTION_FAILED” would make sense.

© SAP AG

E2E100

6-37

ABAP System Log Analysis (2)

The system log entries before and after the error may provide useful information


Even if the results of the SAP Note search do not provide a solution, they might give an idea about which area the problem is located. As a second step, extend the selection criteria to see the log entries before and after error. In many cases they will reveal the cause of the error. In this example, the messages after the error provide useful information.

© SAP AG

E2E100

6-38


Error message Process task and work process number (if available)

Transaction code and program name, if the log entry was written from ABAP

Module name, code line, error text, etc., if the log entry was written by the kernel

System log message documentation (if available)


Having a look at the detailed information of the log message is always a good idea. There is more useful information besides the error message text: Process task (e.g., Dialog [DIA], Background Processor [BTC], Dispatcher [DP], Gateway [RD], etc.) Work process number (according to work process number in transaction SM50) System log documentation for the error entry (if available) Transaction code and program name, if the log entry was written from ABAP. Module name, code line where error occurred, and further kernel information, if the log entry was written by the SAP kernel.

© SAP AG

E2E100

6-39


Select file and press the “Details” button (F2) to open it


There might be more useful information in the trace file of the kernel process (work process, gateway, etc.): Start transaction ST11. Double-click the corresponding trace file to open it. The file name is dev_. For example, “dev_rd” is the trace file of gateway process, and “dev_w5” is the trace file of work process number 5. There are also log files with the suffix “.old”, which are the trace files before the last system restart. Search the trace file for the timestamp when the error occurred.

© SAP AG

E2E100

6-40

Fact Sheet: ABAP Dump Analysis (ST22) Level 2 (must know)

Use Case Further investigation of an ABAP dump Provides Details of the ABAP dump Best Practice for Investigation / Rules of Thumb

Get the dump header (e.g., CALL_FUNCTION_CONFLICT_TYPE). Get the system environment (release, OS, etc.), as well as the user, transaction, etc. Read the error analysis and “How to correct the error” section. Check the frequency of the dump and search for common information, such as server, transaction, etc. Determine the affected program and the line of the source code. Search for SAP Notes. Conduct further analysis as needed, such as code analysis, field checks, analysis of the call stack, recreation of the problem, etc.


Limitation: If the ABAP dump displayed in E2E Exception Analysis is too old, you might not find it in transaction ST22 because it was deleted by the reorganization job.

© SAP AG

E2E100

6-41

ABAP Dump Analysis: Search Dump

Enter selection criteria according “Jump In” information

Press to find dumps matching selection criteria


To search for an ABAP dump with the information in the “Jump In” pop-up window of E2E Exception Analysis, do the following: Logon to the ABAP system and start transaction ST22. Enter selection criteria according the “Jump In” information (date, time, host, client, etc.) Press the “Start” button to get the ABAP dumps displayed which match your selection criteria Standard reorganization jobs should be scheduled regularly to clean up data which is no longer needed. For ABAP dump reorganization, job SAP_REORG_ABAPDUMPS (executing report RSSNAPDL) is scheduled daily. The standard values retention times and volumes for ABAP dumps are: SAP_BASIS 700: Maximum days to keep entries = 28 Maximum number of entries in table SNAP = 2.000.000 SAP_BASIS 640: Maximum days to keep entries = 7 Maximum number of entries in table SNAP = 100.000 SAP_BASIS 620: Maximum days to keep entries = 7 Maximum number of entries in table SNAP = 100.000 So it can happen that you do not find the dump in the ABAP system because it was too old and therefore already deleted by the reorganization job.

© SAP AG

E2E100

6-42

ABAP Dump Analysis

“lock” button to prevent the deletion of the dump

Press to get the details of the dump displayed


If you want to prevent the reorganization (deletion) of a dump, you can lock it: Select the dump you want to protect Press the “Lock” button. The value in the “Keep” column will change from “C” to “X”. To display the detailed dump information: Select the dump you want to analyze. Press the “Display Long Text” button (or shortcut key F2)

© SAP AG

E2E100

6-43

ABAP Dump Analysis

Navigation tree to easily navigate through the information


The most valuable information in the dump is found in these sections: Error analysis How to correct the error System environment User and Transaction Information on where terminated Source Code extract Contents of system fields Chosen variables Active Calls/Events Internal notes Active Calls in SAP Kernel and in the “Information for the customer” category: What happened? What can you do?

© SAP AG

E2E100

6-44

ABAP Dump Analysis

Systematic analysis of the dump

0

Get the dump header (e.g., CALL_FUNCTION_CONFLICT_TYPE)

1

Get the system environment (release, OS, etc.), as well as the user, transaction, etc.

2

Read the error analysis and “How to correct the error” section.

3

Check the frequency of the dump and search for common information, such as server, transaction, etc.

4

Determine the affected program and the line of the source code

5

Search for SAP Notes.

6

Conduct further analysis as needed, such as code analysis, field checks, analysis of the call stack, recreation of the problem, etc.


Some remarks on Point 3, above. (The other points are described in detail later.) If you start transaction ST22 and select all dumps of a particular runtime error (for example, CALL_FUNCTION_CONFLICT_LEN) for the last days, you get a list like this: Date Time Machine User Clt X Error ID 12.07.1999 11:47:01 pawdf025 FRANKM 000 C CALL_FUNCTI ... 12.07.1999 11:48:11 pawdf025 FRANKM 000 C CALL_FUNCTI ... 12.07.1999 11:51:17 pawdf025 FRANKM 000 C CALL_FUNCTI .... Check if all dumps occur on one machine or if only only one user is involved.

© SAP AG

E2E100

6-45

Typical ABAP Dumps

Questions: Is the number of ABAP dumps is much higher as the average? What are the types of dumps? Error

Possible Problem

TSV_TNEW_PAGE_ALLOC_FAILED LOAD_PROGRAM_LOST LOAD_PROGRAM_NOT_FOUND CALL_FUNCTION_OPEN_ERROR CONNE_IMPORT_WRONG_OBJECT_TYPE DYNPRO_NOT_FOUND CONTROL_NOT_FOUND DYNPRO_NOT_FOUND MESSAGE_TYPE_UNKNOWN SAPSQL_ARRAY_INSERT_DUPREC TIME_OUT MESSAGE_TYPE_X COMPUTE_INT_ZERODIVIDE TABLE_INVALID_INDEX DDIC_STRUCLENG_INCONSISTENT

Memory Configuration Problem? Transport Problems? To be analyzed, Syntax? Missing object due transport mistake? Syntax Problems Problems in Quality Assurance? Could result in data inconsistency. Update problem? Performance Problem? Background Execution?


Which type of dump? Syntactical errors may indicate problems in Quality Assurance - SAPSQL_INVALID_FIELDNAME; CONVT_NO_NUMBER , DDIC_STRUCLENG_INCONSISTENT , BCD_ZERODIVIDE

Problems in transports may indicate general mismatches in Change Management - LOAD_PROGRAM_LOST, GEN_PROGRAM_NOT_EXISTS

Problems in Memory Management may be the result of bottlenecks in hardware, insufficient parameters, or programs with too much memory consumption. Dumps of type TSV_TNEW_PAGE_ALLOC_FAILED can have two causes: 1. Wrong parameter settings 2. Large memory consumption Long-running dialog steps may indicate long-running statements or incorrect handling. Using background processing instead of online processing may be helpful. - TIME_OUT

© SAP AG

E2E100

6-46

ABAP Dump Analysis: Source Code

Information on where the error occurred and Source Code Extract current program

current event (form, function, etc.)

main program Head of the program group

current include and source line


Use transaction SE38 with the Include name to access the source code. With the known line of source code, you can navigate directly to the problematic statement. There you can set a breakpoint for debugging, or check the surrounding code. Use both the current program name and the include name if you search for SAP Notes. Also use the transaction code and/or name for an SAP Note search. Note that sometimes the highlighted line of source code is one statement before the one where the error actually occurred.

© SAP AG

E2E100

6-47

ABAP Dump Analysis: Call Stack

Active calls / events

event program include


The call stack shows the flow of the program. With this information you can display, for example, the include LCATREF20 line 655 in order to check how the form CALL_DYNAMIC_FUNCTION_UNIT was called. Please note that only the call stack is available. If a function is called and then the ABAP processor returns from this function and proceeds, the function will not be found in the call stack. If you have, for example, a CALL_FUNCTION_CONFLICT_TYPE, the called function will not be on the top of the stack because the call itself failed.

© SAP AG

E2E100

6-48

End-to-End Exception Analysis: Unit Summary

You should now be able to: Perform an end-to-end exception analysis in SAP Solution Manager Analyze logs in ABAP and non-ABAP environments Analyze dumps in ABAP and Java environments


© SAP AG

E2E100

6-49

© SAP AG

E2E100

6-50

Exercises Unit:


Lesson 1: E2E Exception Analysis application At the conclusion of this exercise, you will be able to: • Use the end to end exception analysis application in Solution Manager Diagnostics to identify problems in the different components of a solution. A user of the Employee Self-Service (ESS) scenario is getting errors in the Enterprise Portal page when he wants to change his bank data.

1-1

Logon to the Solution Manager Diagnostics and use the E2E exception analysis application to analyse the problem reported by the ESS end-user (E2EDEMO2). The trainer will provide the information in which Landscape (solution, system role) and at which point in time the problem occurred (do a custom selection for the timeframe). 1-1-1 What are the Top 20 java errors in the given timeframe? 1-1-2 How many Java Dumps do you see? 1-1-3 What kinds of ABAP errors (description column) are created by user E2EDEMO2? 1-1-4 What kinds of ABAP dumps (description column) are created by user E2EDEMO2?

© SAP AG

E2E100

6-51

© SAP AG

E2E100

6-52

Exercises Unit:


Lesson 2: Java Exception Analysis At the conclusion of this exercise, you will be able to: • Use the Log Viewer for detailed java exception analysis.

The error message written in the applications.log is not meaningful enough to identify the root cause of the problem. The Log Viewer is used for detailed analyses.

2-1

Use the Log Viewer to find more information about the root cause of the problem. Use in addition to the applications.log also the defaultTrace.trc. 2-1-1 What exception does you see in the defaultTrace.trc for user E2EDEMO2 when the error occurred. 2-1-2 Where do you further need to investigate the problem?

© SAP AG

E2E100

6-53

© SAP AG

E2E100

6-54

Exercises Unit:


Lesson 3: ABAP Exception Analysis At the conclusion of this exercise, you will be able to: • Use the ABAP System Log (SM21) and the ABAP Runtime Error transaction (ST22) for detailed analysis of ABAP errors and dumps. The defaultTrace shows an error, thrown in an RFC call, which indicates that the root cause might be in the ABAP system. Further investigations of the ABAP errors and dumps are necessary.

3-1

Logon to the ABAP system of the Solution. The trainer will provide the logon details. Use the information given in the jump in pop-up window to find the ABAP errors in the System Log (transaction SM21). 3-1-1 Check the Operating system call gethostbyname failed (error no. 0 ) log entry in case it occurred and search for the additional information in the trace of the related work process (transaction ST11). What information do you see in the work process trace? 3-1-2 Which method is mentioned in one of the log entries?

3-2

In the System Log you see also entries due to ABAP dumps. One is due to COMPUTE_INT_ZERODIVIDE. Check the details of this dump in transaction ST22. 3-2-1 Which coding line caused the dump? 3-2-2 How can the problem be fixed without changing the coding?

© SAP AG

E2E100

6-55

© SAP AG

E2E100

6-56

Solutions Unit:


Lesson 1: E2E Exception Analysis application

1-1

Navigate to the exception analysis screen via the top level navigation Root Cause Analysis Exceptions Select the Solution and System Role provided by the trainer and start the E2E Exception Analysis application in the left navigation panel. Select all components in the right frame and then press the Start button. In the timeframe drop down selection box select Custom Selection and specify the timeframe provided by the trainer. If you did it correct you will see an overview list like:

When you do the custom selection of the timeframe correct you will see a result like:

The number of errors and dumps might be different. 1-1-1

© SAP AG

You should see two exception entries from Category /Applications/Xss in the Top 20 error list:

E2E100

6-57

1-1-2

There are no java dumps in the system.

1-1-3

When you filter the ABAP error list for user E2EDEMO2 and sort the Description column you will easily find the different kinds of ABAP errors.

You should see the following errors: - wrong date format set in method IF_EX_HRXSS_PER_BEGDA~DEFAULT_DATE - Run-time error "" occurred - Perform rollback - Operating system call gethostbyname failed (error no. 0 ) 1-1-4

When you filter the ABAP dump list for user E2EDEMO2 and sort the Error ID column you will easily find the different kinds of ABAP dumps.

You should see the following dumps: - COMPUTE_INT_ZERODIVIDE - OBJECTS_OBJREF_NOT_ASSIGNED_NO

© SAP AG

E2E100

6-58

Solutions E2E

Unit:


Lesson 2: Java Exception Analysis

2-1

Start the Log Viewer application from the left navigation panel. Select the java component in the right frame and then press the Start button. Use the setting Show Custom View with the view Log and trace files and press the Open Filter button. In the Log Browser area select: Log file named as DefaultTrace* Log file named as applications* Set the flag Merge Logs If Possible and then press the Apply Logs button. In the Filter by Content area select the timeframe given by the trainer and then press the Apply Filters button. Your selection will look like the following:

© SAP AG

E2E100

6-59

2-1-1

In the defaultTrace.trc you will find an exception message directly before the message from the applications.log. It will look like the following:

You can see that exception is caused by an RFC_ERROR_SYSTEM_FAILURE due to a division by 0. 2-1-2

© SAP AG

Due to the exception thrown by an RFC error and the ABAP errors/dumps further investigations need to be done in the ABAP system.

E2E100

6-60

Solutions E2E

Unit:


Lesson 3: ABAP Exception Analysis

3-1

Logon to the ABAP system (logon details will be provided by the trainer). Start transaction SM21 and do the selection according the information given in the jump in pop-up of the E2E Exception Analysis application. If you select some minutes before and after you will see the following entries, which correspond to the problem:

© SAP AG

E2E100

6-61

3-1-1

Open the details of the Operating system call gethostbyname failed (error no. 0 ) log entry. There you will find the information in which work process and at what time the error occurred. Now start transaction ST11 in a new session and display the trace file of the work process in which the error occurred. Search in the trace file for the timestamp when the error occurred. You will the following information:

This error occurred due to an unknown host name. The ABAP program ZCL_IM_END_TO_END tried to call via the RFC destination E2EDEMO2 the function module Z_E2E_TEST_FM.

3-1-2

© SAP AG

In the second log entry the method name IF_EX_HRXSS_PER_BEGDA~DEFAULT_DATE is shown.

E2E100

6-62

3-2

Start transaction ST22 and do the selection according the information given in the jump in pop-up of the E2E Exception Analysis application. If you select some minutes before and after you will see the following dumps, which correspond to the problem:

3-2-1

Open the dump COMPUTE_INT_ZERODIVIDE and navigate to the source code extract. You will find the following:

The error was caused by the coding line: BEGDA = bdate3 / 0. 3-2-2

© SAP AG

As you can see in the coding extract of the dump, the “bad” coding is only executed in case the RFC call via the destination E2EDEMO2 failed. The system log showed that this RFC called failed due to an unknown host in the RFC connection. To fix the problem without a coding change the RFC connection E2EDEMO2 needs to be maintained correct. Please do not maintain the RFC connection by yourself. This should only be done by the trainer. E2E100

6-63

© SAP AG

E2E100

6-64



© SAP AG


E2E100

7-1


Contents: Procedure for Application Analysis Procedure for Data Inconsistency Analysis Tools for Detection and Resolution of Data Inconsistencies


© SAP AG

E2E100

7-2

Application and Data Inconsistency Analysis: Unit Objectives

After completing this unit, you will be able to: Explain the procedure for analysis on functional level Explain the procedure for data inconsistency analysis Use common tools to detect and resolve data inconsistencies


© SAP AG

E2E100

7-3



E2E Trace Analysis







JAVA Trace Analysis










Functional Problem

© SAP AG 2006

© SAP AG

E2E100

7-4


Functional Root Cause Analysis (Cross Component)

?

Program or update terminations E2E Log and Dump Analysis

?

Inconsistent data E2E Data Consistency Analysis

?



© SAP AG

E2E100

7-5

Reasons for Data Inconsistencies: Why can they occur User Level Data inconsistencies due to Real world operation ≠ system transaction Wrong Usage of transaction

Application Level Data inconsistencies within one system or between two systems due to Logical inconsistencies in application integration Errors in application programs Absence of error handling

Technology Level Data inconsistencies due to Absence of transactional security Data Loss


Inconsistencies can occur between different systems, between a system and the real world, and between two components of the same system. An inconsistency is present if two instances representing the same data have different values or are missing completely in one of both systems, for example a material stock can have a different value in the system than in the real world or information about the material may be missing altogether in the system. It is not possible without the context to identify which of these cases of inconsistencies are more severe. While a completely missing data set seems to be very serious at first glance it is usually easy to detect and easy to correct, while inconsistent content may be unobserved for a long time leading to a lot of follow up errors like incorrect dependent data. Differences are temporary inconsistencies. They will always occur between running, connected systems. RCA of inconsistencies and estimation of severity may lead to incorrect results if temporary inconsistencies are present.

© SAP AG

E2E100

7-6

Reasons for Data Inconsistencies Bank 1

The insert to the outbound queue does not happen in the same Logical Unit of Work as the insert of Account1, because a program is not transactionally secure

Inconsistency on Technical Level

Outbound queue

Bank 2

Inbound queue

Account 1

Account 2

Inconsistency on Application Level The update of Account2 is still in the inbound queue of Account2 due to performance problems in Bank2 queue processing. This temporary inconsistency will resolve over time.

Inconsistency on User Level

Bank customer deposits $1000 cash Bank clark credits $900 to customer account © SAP AG 2006, Title of Presentation / Speaker Name / 1

Example of an inconsistency on user level: Bank customer deposits $1000 cash. Bank clark credits $900 to customer account Example of a data inconsistency on application level: The update of Account2 is still in the inbound queue of Account2 due to performance problems in Bank2 queue processing. This temporary inconsistency will resolve over time. Example of a data inconsistency on technical level: The insert to the outbound queue does not happen in the same Logical Unit of Work as the insert of Account1, because a program is not transactionally secure

© SAP AG

E2E100

7-7


Functional Root Cause Analysis (Cross Component)

?

Program or update terminations E2E Log and Dump Analysis

?

Inconsistent data E2E Data Consistency Analysis

?



© SAP AG

E2E100

7-8

End-to-End Application Root Cause Analysis Roadmap E2E Application Root Cause Analysis ?

Is the user entering incorrect data or using transactions incorrectly? Train the users

?

Is the application throwing an error? Check the Application Log

? Are the results incorrect? ? Is the application specific customizing or the master data incorrect for the intended business process?

Change the customizing or master data Debugging


Often it is not possible to find the root cause of an application issue in a purely technical way, deep application knowledge is necessary. Before tracing and debugging it should be checked if the expectation is correct or if there is already for example a correction note available by searching for notes, reading the online documentation, the release information, the long text of the message, testing etc. Also analyse underlying data which might lead to the issue exactly.

© SAP AG

E2E100

7-9

Analysis of Unexpected System Behavior: Example

Expectation: The unplanned delivery costs of 3 EUR should be distributed to the items proportionally: 1 EUR should be added to the first item and 2 EUR should be added to the second item.


The following slides provide an example of unexpected results in the application invoice verification. It is provided as an example to show how to find the root cause of unexpected result by debugging. It is not intended to teach application knowledge like how the unplanned delivery costs should be distributed. The user enters an incoming invoice with transaction MIRO. The amount is posted directly to GL account, 10 EUR in the first item and 20 EUR in the second item. 3 EUR unplanned delivery costs are entered on header level on the detail tab. The expected system behaviour is that the unplanned delivery costs are distributed proportionally to the items: 1 EUR should be added to the first item and 2 EUR should be added to the second item. (If the customizing setting under „Materials Management -> Logistics invoice verification -> Incoming Invoice -> Configure how unplanned delivery costs are posted“ is „Distribute among invoice items“ for the used company code.

© SAP AG

E2E100

7-10

Expected Result


When pressing the Button Simulate the simulated financials follow-on document should have one item with a posting of 11 EUR to the GL account and one item with a posting of 22 EUR to the GL account.

© SAP AG

E2E100

7-11

Actual Result

Where does this result come from?


The actual result is different than the expected result. The next slides show a debugging session to find out why the result is different than the expected one.

© SAP AG

E2E100

7-12

Debugging: Creating a Breakpoint 1

Function module in which the delivery costs are distributed between the items

2 © SAP AG 2006, Title of Presentation / Speaker Name / 1

The simulate functionailty is debugged by entering /h in the command field, pressing simulate and subsequently putting a breakpoint on the function module which distributes the unplanned delivery costs between the items.

© SAP AG

E2E100

7-13

Debugging

Result of the function module as expected


The result of the function module can be checked in the internal table ptx_drseg for the invoice items. The portion of the unplanned delivery costs per item can be checked in the field bnkan_fw. The results are as expected.

© SAP AG

E2E100

7-14

Debugging: Creating a Watchpoint


As the results of the function module MRM_DISTRIBUTION_BNK are as expected and do not match the actual results, the values ptx_drseg[1]-bnkan_fw and ptx_drseg[2]-bnkan_fw are probably changed later on. For this reason watchpoints are put on these fields.

© SAP AG

E2E100

7-15

Debugging (2)


The watchpoints are reached in function module MRMBADI_UDC_DISTRIBUTE within a section with the comment ‚adopt the result of the BAdI call‘.

© SAP AG

E2E100

7-16

Calling the BAdI


A few lines above you can see the call of the badi.

© SAP AG

E2E100

7-17

Active Implementation of the BAdI


A breakpoint is put in the BAdI implementation and debugging is restarted.

© SAP AG

E2E100

7-18

Verification: Debugging the Implementation of the BADI


Restarting debugging confirms that the proportions of the unplanned delivery costs are indeed changed within the BAdI implementation to the actual values.

© SAP AG

E2E100

7-19

Fact Sheet: ABAP Debugger Level 3 (Expert)

Use Case

Clarification of functional errors which could not be solved by exception and trace analysis Provides Line-by-line variable inspection at runtime

Best Practice for Investigation / Rules of Thumb Set watchpoints/breakpoints on individual fields Display call stack Check for large internal tables


© SAP AG

E2E100

7-20

Application and Data Inconsistency Analysis: Unit Summary

You should now be able to: Explain the procedure for analysis on functional level Explain the procedure for data inconsistency analysis Use common tools to detect and resolve data inconsistencies


© SAP AG

E2E100

7-21

© SAP AG

E2E100

7-22

Exercises Unit:

E2E Data Consistency Analysis

Lesson:

Transactional Correctness Analysis

At the conclusion of this exercise, you will be able to: • Check transactional correctness by means of an SQL trace.

1

Logon to system TT5. 1-1

1-2

© SAP AG

Enter a sales order using transaction VA01 with the following data: Order Type:

OR

Sales Organization:

1000

Distribution Channel:

10

Division:

00

Sold-to-party:

2130

Material:

P-100

Order Quantity:

1

Sales Unit:

PC

While saving the sales order perform an SQL trace using transaction ST05 for your user. Display and analyze the SQL trace. Do you see any issue regarding transactional correctness? If yes, at which stage? ___________________________________________________________________ ___________________________________________________________________ ___________________________________________________________________ ___________________________________________________ E2E100

7-23

© SAP AG

E2E100

7-24

Solutions

1-2

Unit:

E2E Data Consistency Analysis

Lesson:

Transactional Correctness Analysis

While saving the sales order perform an SQL trace using transaction ST05 for your user. Display and analyze the SQL trace. Do you see any issue regarding transactional correctness? If yes, at which stage? You can see that data are inserted into database table ZTHOS and committed within the transaction. By pressing the corresponding button the call position in the ABAP program is displayed. It turns out that the INSERT and COMMIT is done within a user exit. (See corresponding slides in the presentation.) As later on the other data of the sales order will be committed this is a violation of the LUW principle. If for example because of an error message after the user exit the transaction is aborted the database table ZTHOS will be updated while the data of the sales order are not updated. This would be an inconsistency.

© SAP AG

E2E100

7-25

© SAP AG

E2E100

7-26

SAP EarlyWatch Alert Analysis Incident Management


SAP EarlyWatch Alert Analysis


Contents: Judge customer solution performance, stability and quality using SAP EarlyWatch Alert Architecture of SAP Early Watch Alert data provisioning


© SAP AG

E2E100

8-2

SAP EarlyWatch Alert Unit Objectives

After completing this unit, you will be able to: Evaluate SAP EarlyWatch Alert reports for different satellite systems Judge customer solution performance, stability and quality using SAP EarlyWatch Alert Explain the architecture of SAP EarlyWatch Alert for ABAP and Java satellite systems


© SAP AG

E2E100

8-3

Proving Stabilization of Solution Landscapes with SAP EarlyWatch Alert Customer situation BEFORE

Root Cause Analysis

EarlyWatch Alerts

Customer situation AFTER

EarlyWatch Alerts

weeks

EarlyWatch Alert (EWA) can be leveraged to prove the stabilization of a solution in a warroom © SAP AG 2006, Title of Presentation / Speaker Name / 1

SAP EarlyWatch Alert provides regular and automatic monitoring of the predefined Key Performance Indicators (KPI) of SAP components. It allows to evaluate current situation in areas of stability, performance, solution quality. Before a warrom starts the EarlyWatch Alerts for all the SAP components from customer solution are taken in order to obtain a „BEFORE-status of Solution Quality“. After the Root Cause Analysis and problem solving the EarlyWatch Alerts for all these SAP components are used in order to get „AFTER-status of Solution Quality“. Then the values of corresponding KPIs are compared in order to prove the stabilization of a solution in a warroom.

© SAP AG

E2E100

8-4

SAP EarlyWatch Alert: Unit Overview Diagram

SAP EarlyWatch Alert Lesson 1: SAP EarlyWatch Alert for ECC (Web AS ABAP) Lesson 2: EarlyWatch Alert for Enterprise Portal (Web AS Java) Lesson 3: Overview SAP EarlyWatch KPIs of SAP products


© SAP AG

E2E100

8-5

Example: Bad SAP ERP Performance Analysis Example with SAP EarlyWatch Alert for SAP ERP After Go Live with an SAP ERP System customer experiences a general performance problem Both SAP standard and customer-developed transactions are affected To stabilize the customer solution, the customer and SAP have agreed to open a a war room

To report on the solution quality before the war room starts, the EarlyWatch Alert for SAP ERP is leveraged …


In this example a performance problem is reported. Because of this only those EWA KPIs should be taken into account that are relevent for performance estimation.

© SAP AG

E2E100

8-6

EWA for SAP ERP - Important KPIs

Focus: Performance analysis

Important information for analysis:

Questions to be answered: Is there a performance problem in the system? If “yes”: is this problem general or specific? Is the performance problem caused by hardware bottlenecks or by long-running programs? Is the performance problem temporary or permanent? What are the possible causes for the problem?

Hardware Configuration Performance Overview Current Workload Transactions Total Workload Transactions by DB Load History of Response Times DB Load Profile CPU Utilization Paging Activity Database Growth Database Key Performance Indicators System Activity Response Times System Operation Hardware Capacity Active Users Avg. Response Time in Dialog Task Avg. Response Time at Peak Dialog Hour Avg. DB Request Time in Dialog Task DB Size


The first step of the performance analysis is to determine whether there are system bottlenecks or if there are long-running programs. If you find no system bottlenecks, but long-running programs, you cannot expect to improve the performance of the programs by system tuning (more hardware, configuration parameter recommendations etc.). In this case you have to analyse the programs in detail. If you find system bottlenecks, but the response times of the programs are OK, the system tuning recommendations are of minor importance. They are valid in a more long-term planning. In many cases, you will find system bottleneck as well as long-running, non-optimally implemented programs, e.g.: Programs with non-optimal database accesses (due to non-optimal database indexes) cause an CPU- or I/O-bottleneck on the database server. A memory area that is configured too small (e.g. program or table buffer) may cause unnecessary data reloads from the database and cause long runtimes. It is sometimes difficult to distinguish between the primary and secondary problems, find the reason for the problem and give recommendation that really helps, e.g. Buy more CPUs or start a detailed SQL statement analysis.

© SAP AG

E2E100

8-7

EWA for ERP - Key Performance Indicators (KPIs)

Question: What is the overall situation from the performance point of view (take the first impression)? – What is the Avg. Response in Dialog Task taking into account Active Users (dialog steps)? – What is the difference between Avg. Response and Avg. Response Time at Peak Dialog Hour? – What is the DB Request Time in Dialog Task? – What is the proportion between these two times? © SAP AG 2006, Title of Presentation / Speaker Name / 1

In the chapter “2 Performance Indicators” at the beginning you can see that the response time in the Dialog task is too high (1,8 seconds when anormal value should be around 1 sec) and the tendency is “up” . Also you can see that Avg. DB Request Time in Dialog Task is very high and almost 64% of Avg. Response Time in Dialog Task. That could indicate a database performance problems. In order to complete the first impression you can also note that the Last Month DB Growth is relative high it is almost 7% of the DB size (it could be important for example when a new development is tested on small data volumes and no performance problems have been detected. But by a large data volumes (e.g. by GoLive with a new land) a performance problem arise) If there is a large difference between Avg. Response and Avg. Response Time at Peak Dialog Hour it could be an indication of possible hardware bottleneck because hardware should be sized taking into account maximum workload on the system.

© SAP AG

E2E100

8-8

EWA for ERP– Current Workload , Performance Overview

All information in this section are based on data from transaction ST03

Question: Is there a general performance problem? – What is the total load to the system (Dialog Steps, Current Workload)? – What are the response times of the dialog task type? – What is the distribution of the response times for the different task types? © SAP AG 2006, Title of Presentation / Speaker Name / 1

In the chapter “5 Performance Overview” you can find clear indication on the performance problems. You can note that Avg. DB time is a high portion of Avg. Response time for DIALOG and BATCH task types. Also interesting to note high values of Avg. Wait Time for the UPDATE processes. It could be indication that there are not enough free work process configured for such a high number of updates performed in the system. Performance of SPOOL work processes could not be automatically rated since sppol is very customer specific. In case customer has business critical printing activities (e.g. printing of lables or invoices) then high wait times in Spool could have a negative business impact Workload in HTTP indicates that this system is accessed via Internet

© SAP AG

E2E100

8-9

EWA for ERP– Transaction Profile – Display Variant 1

Variant 1: sorted by total response time = load caused on the system by this transaction Question: Is the performance problem caused by system bottlenecks or by longrunning programs?

5.3.1 Transactions by Total Workload The total system workload can be defined as the sum of the total response times of all transactions. The following transaction profiles list the transactions that have the greatest share in the total workload and ar sorted by the percentage of total response times.

– Which transactions bring the most load on the system? – What are the most-used transactions. Tuning these transactions creates the greatest improvements in overall performance. – Are all the transactions affected or only specific ones? – What is the average response times for typical R/3 transactions?


In this chapter you can see the Z* transactions and transactions generated form customer coding (S_ALR_87012168) bring a very high load on the system. For example transaction ZEDK with only 600 dialog steps has Av. Res Time in 74 sec. Almost 98% of this time is access to DB. The standard SAP transactions like FBL5N, FBL1N, VA02 and FBL3N shows a very high DB time. During estimation of transaction profile take into account purpose of transactions. It could give you a hint for further investigation. For example in this case you can see that SESSION_MANAGER is on of the top transactions. On the other hand SESSION_MANAGER is responsible for processing of entry screens, user menus and authoritarians. If it appears in the top list it make sense to prove user menus. RSM13000 is responsible for update processing and should not be in the in the top transaction list.

© SAP AG

E2E100

8-10

EWA for ERP– Transaction Profile – Display Variant 2

Variant 2: sorted by DB time (= load caused on the DB by this transaction) Similar questions from the previous slide would help to find out the exact trouble maker in case of DB bottleneck.

5.3.2 Transactions by DB load The total database workload generated from the System can be defined as the sum of the total database access times of all transactions. The following transaction profiles list the transactions that have the greatest share in the database load, sorted by percentage of total database access times.


Here you can see that Z* transactions are causing a high load on database

© SAP AG

E2E100

8-11

EWA for ERP– Performance Evaluation

Which task types are affected? Does the performance problem occurs only during working hours or also during the night hours?


In the chapter “5.2 Performance Evaluation” you can see that the response times are very high during the working hours.

© SAP AG

E2E100

8-12

EWA for ERP - DB Load Profile

Question: Are there peak hours from the DB load point of view?

4.3 DB Load Profile The following graphics show the system DB load distribution at different times of day, as well as the typical DB load peaks. These statistics are given as a weekly average. The unit is 1 hour, except for the periods between an average value per hour. By comparing the load profiles for dialog and background activity, you have an overview of the amount of background activity during online working hours.


DB Load Profile: A RED rating is given, if in at least 2 time periods: ( 'DB Load DIA' + 'DB Load RFC' + 'DB Load BTC' ) is larger than ( 2 x number of CPUs on the database server ). Example: If you see a value of 10, that means that SM50 would on average show 10 processes running on the database. This value means that this requires 10 CPUs on the database server. However, in general the value should not be much larger (for example twice or higher) than the number of database CPUs From this section you can prove if customer has really 24 continuous Dialog operations. Also you can see if there is a possibility to separate background and dialog activity.

© SAP AG

E2E100

8-13

EWA for ERP - History of Activity and Response Time

What are the trends in Response Times? Can you observe some correlation between high values of Av. Response Time and other parts of Response Time? Is the Activity constant during reported interval?


In the “history of Response Times” diagram you can clearly see that the Av. Response time is mainly developed by Av. DB Time. The Av. CPU Time is almost a constant and Av. Response time is almost a copy of Av. DB Time. That indicates a DB problem

© SAP AG

E2E100

8-14

EWA for ERP – Hardware Capacity

Questions: Is there a hardware capacity problem? Is there a performance problem caused by hardware bottleneck?


A possible reason for that you can see in the “9 Hardware Capacity” chapter. It seems that a physical memory bottleneck is the reason for a high paging. In the Trend Analysis you can see that occured first as of 14.01.2007

© SAP AG

E2E100

8-15

EWA for ERP –Trend Analysis

Questions: Was the current system behavior only a temporary event, or has it happened before?


A possible reason for that you can see in the “9 Hardware Capacity” chapter. It seems that a physical memory bottleneck is the reason for a high paging. In the Trend Analysis you can see that occured first as of 14.01.2007

© SAP AG

E2E100

8-16

EWA for ERP - Workload from Application Modules

Indicator for workload from application modules CPU Load by Application Module

DB Load by Application

CPU Time in %

Module

DB Time in %

BC-ABA

28,1

Customer

26,3

Customer

19,8

BC

24,1

BC-FES

12,1

FI

18,7

FI

11,5

BC-FES

8,4

CO-OM

8,4

BC-ABA

5,7

BC

7,7

PM

4,1

CA

4,4

CO-OM

2,9

workload of more than 30% from customer programs may indicate necessity of performance tuning Performance issue? shows the importance of customer specific programs -> Upgrade issue? are these copies of former SAP programs modified afterwards? -> Upgrade Issue © SAP AG 2006, Title of Presentation / Speaker Name / 1

© SAP AG

E2E100

8-17

EWA for ERP - ABAP dumps

Questions: Is the number of ABAP dumps is much higher as the average? Which type? Error

Number

TSV_TNEW_PAGE_ALLOC_FAILED LOAD_PROGRAM_LOST LOAD_PROGRAM_NOT_FOUND DYNPRO_NOT_FOUND CALL_FUNCTION_OPEN_ERROR CONNE_IMPORT_WRONG_OBJECT_TYPE MESSAGE_TYPE_X COMPUTE_INT_ZERODIVIDE CONTROL_NOT_FOUND DYNPRO_NOT_FOUND TABLE_INVALID_INDEX MESSAGE_TYPE_UNKNOWN SAPSQL_ARRAY_INSERT_DUPREC DDIC_STRUCLENG_INCONSISTENT TIME_OUT

5 18 2 10 9 36 99 5 1 10 2 8 9 2 52

Memory Configuration Problem? Transport Problems? To be analyzed, Syntax? Missing object due transport mistake? Syntax Problems Problems in Quality Assurance?

Could result in data inconsistency. Update problem? Performance Problem? Background Execution?


Which type? Syntactical errors may indicate problems in quality Assurance - SAPSQL_INVALID_FIELDNAME; CONVT_NO_NUMBER , DDIC_STRUCLENG_INCONSISTENT , BCD_ZERODIVIDE Problems in transports may indicate general mismatches in Change Management - LOAD_PROGRAM_LOST, GEN_PROGRAM_NOT_EXISTS Problems in Memory Management maybe the result of bottlenecks in hardware, unsufficient parameters or programs with too much memory consumption - TSV_TNEW_PAGE_ALLOC_FAILED - Dumps of this type can have two causes: 1. Wrong parameter settings 2. Large memory consumption long running dialog steps may indicate long running statements or wrong handling (Background Processing would be useful) - TIME_OUT

© SAP AG

E2E100

8-18

EWA for ERP - Database Growth

Date

Current Size in GB

Monthly Growth in GB

31.07.2003

392,48

19,27

31.08.2003

409,47

16,99

30.09.2003

425,02

15,55

31.10.2003

440,33

15,31

30.11.2003

457,26

16,93

26.12.2003

467,36

10,10

History Are there strong differences to the average?

Prognosis Which hardware resources are necessary in future to fulfil the trend?

Archiving Is Archiving necessary? Does the archiving fulfil the expectations? Is that ‚normal‘ growth or is there the opportunity to avoid data? © SAP AG 2006, Title of Presentation / Speaker Name / 1

© SAP AG

E2E100

8-19




© SAP AG

E2E100

8-20

Example: Bad SAP Enterprise Portal Performance Stabilization of an Employee Self Service (ESS) Scenario Customer claims that regularly SAP Enterprise Portal crashes with “Out of memory” exception Customer experiences general perfomance problem in SAP Enterprise Portal To stabilize the customer solution, the customer and SAP have agreed to open a a war room

To report on the solution quality before the war room starts, the EarlyWatch Alert for SAP EP is leveraged …


For “Out of memory” and general performance problems the most relevant are Garbage Collection (GC) KPIs

© SAP AG

E2E100

8-21

EWA for SAP EP - Important KPIs for performance analysis

Focus: Performance Analysis


Questions to be answered: What is the Java VM Performance? What is the Java Application Performance? What is the Enterprise Portal Activity?

Garbage Collection Time Usage of Effective Old Space Fraction of Full Garbage Collections Allocation Rate Promotion Rate Top WebDynpro Applications Accumulated Time Top JCO Calls Response Time of JCO Calls Top SQL Selects Response Time of SQL Selects Accumulated Time Hourly User Distribution Most frequently visited iViews Most frequently visited pages Visits Requests Top Iviews Response Time of top Iviews Accumulated Time


© SAP AG

E2E100

8-22

EWA for SAP EP –Analysis of Java Memory Performance

What is the Average GC Time? What is the tendency in Av. GC Time development in comparison with the previous period?

Garbage Collection Time

What is the ratio of Full Garbage Collection in comparison to the Minor Garbage Collection?

Fraction of Full Garbage Collections

Old Space Growth Rate

What are the trends in growth rate of the memory used? How efficient is the Eden space used (What is the ratio between amount of memory allocated in the young generation area (Allocation Rate) and sent to the old space (Promotion Rate))?

Allocation / Promotion Rate


Garbage Collection (GC) Time Time spent in Garbage Collections (GC) prevents the Java virtual machine from executing any other task A high percentage of time spent in garbage collections has a directly measurable performance impact. Thus, if the average value in particular is large, this will require further analysis from a general performance perspective. Temporarily high values of the GC time are indicative of critical situations (in this example 62%). These specific situations should be analyzed in comparison with other Key Performance Indicators within this report to determine potential correlations. Fraction of Full Garbage Collections This KPI shows how often Full GC occurs in comparison to minor GC In general, minor garbage collections must occur more frequently than full garbage collections. A consecutive number of full garbage collections is a clear indicator that the Java Virtual Machine (JVM) is not removing objects from the old space. Old Space Growth Rate Critical memory situations in Java are often characterized by a sudden growth of the memory used. Rapid growth of memory used on old space is the most critical, since this area is required for longer living objects. A rapid growth rate quickly leads to a resource outage, and so high growth rates should be investigated further. The growth rate in the old space is calculated using a sliding window approach, in which the last 10 full garbage collections are analyzed. A linear trend analysis is performed on this interval. This approach is chosen so that short intervals of rapid growth in the old space can be detected. This kind of growth is typical of out of memory situations. © SAP AG

E2E100

8-23

The promotion rate quantifies the amount of memory transported to the old space per second. Due to the infant mortality of Java objects this value should always be much smaller than the Allocation Rate. One indicator of critical situations a high ratio of Promotion Rate to Allocation Rate as this implies that cannot make efficient use of the eden space.

© SAP AG

E2E100

8-24

EWA for SAP EP - Java Application Performance

Top WebDynpro Applications Top JCO Calls Top SQL Selects

Which WebDynpro Applications, JCO Calls and SQL requests have the most impact to the performance?


This section measures and reports the average response time over the reporting interval and the accumulated time of the identified WebDynpro application. Accumulated Time - The list can be used to determine which application effectively caused the greatest amount of load on the J2EE, since it is not only influenced by the average response time but also by the number of instances the application created at runtime. - Applications with a large accumulated time potentially have a larger overall performance impact. This may be worth further investigation.

© SAP AG

E2E100

8-25

EWA for SAP EP - Enterprise Portal Activity Reporting

Most frequently visited iViews Most frequently visited pages

Top Iviews


© SAP AG

E2E100

8-26

EWA for Non-ABAP components: Requirements

Required software components for Non-ABAP EWA 1. Solution Manager Diagnostics based on NW04s SPS10 or higher on Java Side

EWA is possible for the following Software components Netweaver 04 and Netweaver 2004s: Enterprise Portal XI Process Integration

2. Solution Manager 4.0 (ABAP side) with:

XI J2EE Adapter Engine

– ST-PI 2005_1_700 PL 002

AS Java

– ST 400 SPS5 – ST-SER 2006_2 SP1

3. Wily Introscope 7.1

SAP Note 976054


For monitored systems that comprise of both ABAP and non-ABAP components (e.g. XI) the implementation of note 1012503 may be necessary. Please note that ST-SER 2006_2 SP0 is technically sufficient, although the content of the EWA report generated with this version is reduced in comparison to ST-SER 2006_2 SP1 (which is the recommended version).

© SAP AG

E2E100

8-27




© SAP AG

E2E100

8-28

Business solutions have different KPIs

APO

R/3 Area

Indicators

Value

Trend

Value

System Performance

Active Users

1065

steady

10

Avg. Response Time in Dialog Task

638 ms

down

2862 ms

Max. Dialog Steps per Hour

73508

down

181

Avg. Response Time at Peak Dialog Hour

672 ms

down

91 ms

Avg. Availability per Week

100 %

steady

100 %

Avg. DB Request Time in Dialog Task

402 ms

down

214 ms

Avg. DB Request Time in Update Task

390 ms

down

214 ms

DB Size

880.98 GB

steady

88.81 GB

Last Month DB Growth

17.98 GB

up

0.07 GB

Database Performance

Database Space Management


Example: R/3 vs APO Active Users Normaly very high number in R/3, less in APO high number in dialog steps, lower number in APO Average Response Time in R/3 should be about 1,5 s as a rule. No rules for APO available, may differ because of very different business processes. In general it is higher as in R/3 Systems. Database Size R/3 databases are often big and rapidly increasing in general APO is smaller and growths slowly APO consists of APO DB and liveCache- in a specific manner two databases Specific know how of the solution is necessary to understand the meaning of technical indicators

© SAP AG

E2E100

8-29

EWA for BI - Important KPIs

Focus:


Performance Analysis of BI system Data Model analysis Workload of BI system Questions to be answered: Is there a general performance problem with BI? Is there a problem with Data Model design? Is there a problem with administration of BI?

Hardware Configuration Current Workload Transactions by DB Load DB Load Profile CPU Utilization Paging Activity DB Size Last Month DB Growth Avg. Total Runtime of the BW Queries Avg. Database Runtime of the BW Queries BW Workload Reporting and Upload Workload BW Runtime Statistics Top InfoProvider Frontend Distribution Query Profile Check Upload of transactional data Process Chains - Runtime Overview Database Parameters BW Application Checks Data Distribution Analysis of InfoProviders Analysis of Aggregates Recommendations for BW BI - KPIs


For SAP BI analysis the standard performance KPIs like Av. Dialog Response Time coud not be used for SAP BI performance estimation. The most important KPIs for SAP BI analysis are highlihted in the right side

© SAP AG

E2E100

8-30

EWA for SAP XI - Important KPIs


Focus: XI Analysis

Total Messages for Period Average Number of Messages/day Average Message Processing Time (seconds) Number of Active Interface Pairs XML Errors as % of Total Messages Weekly Load Profile Load Profile on Peak Day Top 20 Messages by Processing Time Workload Distribution by Business Interfaces Workload Distribution by Sending Interface Workload Distribution by Receiving Interface Messages With Errors in Previous Week

Questions to be answered: Are there messages with errors (correctness of XI operations)? What are the performance trends for the XI system? What is the workload distribution?

Covered by EWA

Adapter Framework

Integration Server (IS)

Adapter Framework

Message processing © SAP AG 2006, Title of Presentation / Speaker Name / 1

Up to now in the EarlyWatch Alert forSAP XI only Integration Server (IS) is covered

© SAP AG

E2E100

8-31

EWA for SAP APO/ SCM - Important KPIs


Focus: LiveCache Analysis Questions to be answered: Is there a general performance problem with LCA? What are the cause for the performance problems with LCA? Is there a hardware bottleneck? Is there a performance problem caused by LCA configuration? Are there RFC errors?

Hardware Configuration Current Workload Transactions Total Workload Transactions by DB Load DB Load Profile Database Key Performance Indicators CPU Utilization Paging Activity RFC Communication Transactional RFC Errors LiveCache Volumes Filling Level Size of LOG Volumes Memory consumption Object Version in Data Cache Expensive COM-Routines Standard COM Routines as Performance Indicators Update Statistics LiveCache APPID Multiple Connection LiveCache Parameters History of liveCache performance indicators Backup of liveCache Backup of LOG Volumes Transactional Simulations


For APO analysis the standard performance KPIs like Av. Dialog Response Time coud not be used for APO performance estimation. The most important KPIs for SAP SCM / APO analysis are highlihted in the right side

© SAP AG

E2E100

8-32

EWA for SAP CRM - Important KPIs

Focus: CRM Analysis depends strongly on the used CRM Scenarios


Questions to be answered: Which CRM scenarios are used? Are there the BDoc messages with the status "Error" or "Not processed fully"? Are the messages in the CRM application log? What is the status of Replication and Realignment Queue? Are there any problems with qRFC Administration?

Hardware Configuration Current Workload Transactions Total Workload Transactions by DB Load DB Load Profile Database Key Performance Indicators CPU Utilization Paging Activity Error messages in the CRM application log Unprocessed BDoc messages Replication and Realignment Queue Status Mobile Client Message Recovery qRFC Administration

What is the performance of the Internet Sales scenario in the system? Are the Errors / Warnings during the catalog replication

© SAP AG 2006

Exact recommendations depending on the CRM Scenario could be found in the Best Practice “Manage mySAP CRM 4.0”. You can download it from http://service.sap.com/solutionmanagerbp Solution Management Phase

© SAP AG

E2E100

8-33

EWA Reporting Source Data EWA data via SDCCN Data via Introscope, SMD agents

Output Documents SAP EarlyWatch Alert Report in Word/ HTML format

Advantages Implementation without expert knowledge

Restrictions One report per SAP System, Solution EarlyWatch Alert in development Standard Content (no customizing is possible) Standard alert thresholds (no customizing is possible) Standard frequency (no customizing is possible) Remote connection to SAP needed © SAP AG 2006, Title of Presentation / Speaker Name / 1

© SAP AG

E2E100

8-34

BI Reporting Source Data CCMS Data EWA Data Statistical Data Wily Data

Output Documents Reporting via Business Explorer Suite (BEx)

Advantages Data content and resolution is completely customizable Standard BI reporting functionality is possible (e.g. publishing in Customer Intranet site) Data aggregation could be done on BI side

Restrictions BI knowledge is required for implementation © SAP AG 2006, Title of Presentation / Speaker Name / 1

© SAP AG

E2E100

8-35

Appendix

EWA for SAP BI – Important KPIs

EWA for SAP XI – Important KPIs EWA for SAP APO/SCM – Important KPIs EWA for SAP CRM – Important KPIs

© SAP AG

E2E100

8-36

EWA for BI – Current Workload , BW Workload

Current Workload

BW Workload

Questions: What is the total workload to the system? What is the part of this workload is BW workload?


The BW workload you can preliminary estimate based on the ratio between Total Users and Users performing queries. Here you can possibly identify if SAP BW system is used for other activities

© SAP AG

E2E100

8-37

EWA for BI – BW Workload

Reporting and Upload Workload last week

Questions: If the times for data upload and reporting (Navigation Steps) are mixed?


Better is to perform data upload and data reporting at different time frames: e.g. during the day and night

© SAP AG

E2E100

8-38

EWA for BI – BW Runtime Statistics –Top Info providers

Question: What are the candidates for performance optimization? Which InfoProvider is the most expensive from the upload point of view? Which InfoProvider is mostly used by queries? Is it the same as the first one? What is the proportion between the Navigation Steps and Av. Runtime for different InfoProviders?


© SAP AG

E2E100

8-39

EWA for BI – BW Runtime Statistics- Query Profile Check Query Profile Check

Question: What are the candidates for performance optimization? What is the weak point from the performance point of view (OLAP, Database)? Which queries are the cause for this and should be optimized? Which objects should be buffered? © SAP AG 2006, Title of Presentation / Speaker Name / 1

© SAP AG

E2E100

8-40

EWA for BI – BW Runtime Statistics- Process Chains

Process Chains - Runtime Overview

Question: What are the candidates for performance optimization? Which Process Chains should be improved taking into account the Runtime and the number of Total Steps


© SAP AG

E2E100

8-41

EWA for BI – BW Data Model

Analysis of OLAP Cache

Data Distribution

How many requests are going to the OLAP Cache and how many are going to the Database? What is the Data Distribution?

Analysis of OLAP Cache

Question: What is the complexity of the BW Data Model?


© SAP AG

E2E100

8-42


Dimensions with line item and high cardinality flag

Question: For which InfoCubes the "High-Cardinality" indicator set?


Background: When this indicator is set, the database will create B*-tree indexes instead of bitmap indexes for this dimension. This can lead to a reduction in performance, as only bitmap indexes can be used for a Star-join access, which is the most effective way of selecting data.

© SAP AG

E2E100

8-43


Question: Are there InfoCubes with one or more dimensions that contain 30% or more entries compared to the number of records in the fact tables


Background: The ratio between the number of entries in the dimension tables and the number of entries in the fact table should be reasonable. If an InfoObject has almost as many distinct values as there are entries in the fact table, the dimension this InfoObject belongs to should be defined as a line item dimension. Instead of creating a dimension table that has almost as many entries as the fact table, the system then writes the data directly to the fact table.

© SAP AG

E2E100

8-44


Aggregates suggested for deactivation

Question: Are there aggregates that are not used during query execution?


© SAP AG

E2E100

8-45

EWA for BI – BI KPIs

BI-KPIs

Question: BW Key Performance Indicators exceed their reference value? Background: If some BW Key Performance Indicators exceed their reference value it indicates severe problems, or at least optimization potential for performance, data volumes, and administration .


© SAP AG

E2E100

8-46

Appendix

EWA for SAP BI – Important KPIs EWA for SAP XI – Important KPIs

EWA for SAP APO/SCM – Important KPIs EWA for SAP CRM – Important KPIs

© SAP AG

E2E100

8-47

EWA for SAP XI – XI Performance Checks

Performance Overview

XML Errors estimation: What is the total number of messages for the analysis period? Are there XML errors during this period and what is the percentage compared to the total number of messages?

Performance/Workload estimation: What is the average number of messages processed per day? What is the average message processing time?


You need also estimate the performance trends in the ABAP system By performance trend analysis it is important to compare the values with the previous analysis period taking into account which new interfaces have been connected and what are the changes in data volume. (In a SAP XI system normally the load caused by RFC task type is much larger as by Dialog). Normally Average Message Processing Time is within seconds. Here it is around 20 min. The requirements for displaying net processing times in service reports have not been met on this EWA for SAP XI. The requirements for displaying net times are described in SAP Note 970124. Gross times are shown where the terms 'processing time' or 'latency time' are used for XI messages. And in this EWA it is hard to determine whether the requirements for displaying net times are in place. For more information about these requirements, see SAP Note 970124.

© SAP AG

E2E100

8-48

EWA for SAP XI – XI Performance Checks

Load Profile estimation: Is there a peak load during the day? (If yes, it makes sense to analyse this period in more detail in the system)

Are there messages with high Latency time? What is the Latency time for the other messages?


The messages with the high Latency time could be indication that these messages are in error status for a long time or scheduled to be processed at nights

© SAP AG

E2E100

8-49

EWA for SAP XI – Workload Distribution

Which sending / receiving Interfaces placing most load on the system: Which sending / receiving interface is the most expensive from the resource point of view? Are there differences in workload distribution for certain interfaces in comparison with the previous weeks?


If there are differences in workload distribution for certain interfaces , it could be indication for not optimal time distribution or HW sizing Workload distribution graphics could be used by customer as a hint for cost estimation (e.g. which portion of the total costs should be paid by different interface owners). The information from the table “Workload Distribution by Business Interfaces” is interesting from a long term analysis point of view. For example to compare the workload distribution of the same interfaces with the same throughput for different weeks. For this purpose EWAs for other period of time are required.

© SAP AG

E2E100

8-50

Appendix

EWA for SAP BI – Important KPIs EWA for SAP XI – Important KPIs EWA for SAP APO/SCM – Important KPIs

EWA for SAP CRM – Important KPIs

© SAP AG

E2E100

8-51

EWA for APO/SCM – SCM Component Check

SCM Component check

Since some of the important problems are solved with new support packages it is important to get an overview of what are the current versions for different components What is the SCM version? What is the current COM build What is the current LCA release?


© SAP AG

E2E100

8-52

EWA for APO/SCM – Analysis LiveCache

Questions: What is the LiveCache Volumes Filling Level What is the Memory consumption What are the Expensive COM-Routines What is the Update Statistics What are the LiveCache Parameters What is the Backup of liveCache What is the Backup of LOG Volumes What is the Transactional Simulations © SAP AG 2006, Title of Presentation / Speaker Name / 1

Performance Overview is not so important for APO. For APO the normal threshold values for Dialog Response time is up to 20 sec -30 sec and if Dialog Response Time is more then 50 sec - the cause should be investigated. Hardware Capacity Since all the data for Live Cache should reside in the physical memeory high paging could be an indication that there are not enough memory and should be investigated in more details in the Live Cache Analysis chapter Remote Function Call Is important check since the errors in the RFC could lead to the functional or performance problems in other system. If RFC errors are reported here their cause should be cheked Analysis of liveCache is one of the most important checks LiveCache Volumes Filling Level Here it is necessary to check how good the memory is used. If the Used Data Space is more then 80 % it normally means that there is not enough physical memory and it should be enlarged. Memory Consumption Here is important to check the hit ratio of the Data Hit Ratio (OMS_DATA) is more then 99,8%. If it is lower then 99,8% it leads to perfomance problems. You need to check if enough memory available Classcontainers in Data Cache You need to check if the Classcontainers could reside in Data Cache. The situation could be critical if e.g. Sum of Classcontainers is larger then Data_Cache configured (pages) Largest Version You need to check if the size of the largest version you are using does fit in the configured data cache size © SAP AG

E2E100

8-53

Backup of liveCache SAP recommends performing a full liveCache backup after each working day. The more frequently you back up your database, the shorter the duration of a potential recovery.

© SAP AG

E2E100

8-54

EWA for APO/SCM – Analysis LiveCache – Rating Criteria


Detailed recommendations for APO/SCM analysis you woul find in the Best Practice: System Monitoring for SAP APO (3.x) / mySAP SCM (4.x/5.0) http://service.sap.com/solutionmanagerbp Topic Area

© SAP AG

E2E100

8-55

Appendix

EWA for SAP BI – Important KPIs EWA for SAP XI – Important KPIs EWA for SAP APO/SCM – Important KPIs EWA for SAP CRM – Important KPIs

© SAP AG

E2E100

8-56

EWA for SAP CRM - Important KPIs

Focus: CRM Analysis depends strongly on the used CRM Scenarios


Questions to be answered: Which CRM scenarios are used? Are there the BDoc messages with the status "Error" or "Not processed fully"? Are the messages in the CRM application log? What is the status of Replication and Realignment Queue? Are there any problems with qRFC Administration? What is the performance of the Internet Sales scenario in the system? Are the Errors / Warnings during the catalog replication

Hardware Configuration Current Workload Transactions Total Workload Transactions by DB Load DB Load Profile Database Key Performance Indicators CPU Utilization Paging Activity Error messages in the CRM application log Unprocessed BDoc messages Replication and Realignment Queue Status Mobile Client Message Recovery qRFC Administration

© SAP AG 2006

Exact recommendations depending on the CRM Scenario could be found in the Best Practice “Manage mySAP CRM 4.0”. You can download it from http://service.sap.com/solutionmanagerbp Solution Management Phase

© SAP AG

E2E100

8-57

EWA for SAP CRM – Field Sales and CIC

Questions: Which CRM scenarios are used?

Are there the BDoc messages with the status "Error" or "Not processed fully"?

Unprocessed BDoc messages


CRM Analysis depends strongly on the used CRM Scenarios. Because of this first please check the section “Used CRM Scenarios” to identify appropriate KPIs BDoc messages are used in your SAP CRM system as containers for the data that constitute a business process. In this example you can see that There are 38 unprocessed BDoc messages of type SALESDOCGEN_O_W remaining in intermediate states for a week There are 12 unprocessed BDoc messages of type BUPA_MAIN

© SAP AG

E2E100

8-58

EWA for SAP CRM: R&R Queues

Questions: What is the status of Replication and Realignment Queue?

Are the error messages in the CRM application log?


Many inbound and outbound R3A* queues in status SYSFAIL were detected in the system There are several errors and warnings in the analyzed replication logs for some of the latest product catalog replications

© SAP AG

E2E100

8-59

SAP EarlyWatch Alert: Unit Summary

You should now be able to: Explain the scope of IT Reporting and SAP EarlyWatch Alert Explain the architecture of SAP EarlyWatch Alert for ABAP and Java satellite systems Read SAP EarlyWatch Alert reports for different satellite systems Judge customer solution performance, stability and quality using SAP EarlyWatch Alert


© SAP AG

E2E100

8-60

Service Report SAP® EarlyWatch® Alert

SAP System ID

N/A

SAP Component

Release

Status

N/A

DB System

N/A N/A

© SAP AG

E2E100

8-61

Service Center Telephone Fax

Date of Session

00.00.0000

Date of Report

06.02.2007

Installation No.

Author

EarlyWatch Alert Service

Customer No.

Session No.

1234567

© SAP AG

E2E100

8-62

Reporting Interval Data displayed in this section of the report was collected for the time frame listed below. Start Time

End Time

Sun Dec 31 00:00:00 CET 2006

Sun Jan 07 00:00:00 CET 2007

1 Java VM Memory Performance 1.1 Garbage Collection Time Time spent in Garbage Collections (GC) prevents the Java virtual machine from executing any other task. At least one of the server nodes below spent in the reported interval in average more than 10% of the complete time with Garbage Collections. It is strongly recommended to analyze this behavior with help of other key performance indicators in this report.

Average GC Time (Cluster wide) [%] 12,00

The table above displays the cluster-wide weekly average of Garbage Collection (GC) time. Garbage collections themselves have a direct impact on system performance. Furthermore, they may also be indicative of more substantial performance issues that may require further analysis. Hostname

Cluster Element ID

wdfd00163713a.wdf.sap.corp

308065550

Average [%]

Max [%]

12,00

62,00

The table above lists the average and the maximum GC time for the server nodes calculated as defined above. You should investigate particularly high values since these can point to critical situations. As a starting point to analyze errors, you should analyze the log files of the particular server node for the given timeframe.

Recommendations: If a high percentage of time is spent in garbage collections, you can measure its impact on performance directly. Thus, if the average value is particularly large, you need to analyze this from a general performance perspective. Temporarily high values of the GC time indicate critical situations. You should analyze these specific situations in comparison with other key performance indicators in this report to determine potential correlations.

1.2 Old Space Growth Rate Critical memory situations in Java are often characterized by a sudden growth of the memory used. Rapid growth of memory used on old space is the most critical, since this area is required for longer living objects. A rapid growth rate quickly leads to a resource outage, and so high growth rates should be investigated further. The growth rate in the old space is calculated using a sliding window approach, in which the last 10 full garbage collections are analyzed. A linear trend analysis is performed on this interval. This approach is chosen so that short intervals of rapid growth in the old space can be detected. This kind of growth is typical of out of memory situations. The maximum values determined during the reporting interval for each server node are reported here. Hostname

Cluster Element ID

Max [KB/s]


308065550

3476.00

Thresholds:

Small, non-negative growth rates are not critical and are within the statistical fluctuations usually observed for this parameter. To assess the potential criticality, you have to compare this number to the available heap size, and in particular, the size of the old space.

Recommendation: Further analysis is required to determine the cause of rapid memory growth phases. In many cases, these are accompanied by increases in load, which in turn are often characterized by increases in HTTP sessions. The first stage is to analyze these correlations.

1.3 Usage of Effective Old Space The effective old space is defined as the part of the old space that triggers a full garbage collection if filled completely. In a simple definition, it can be regarded as the old space minus the new space, since a full garbage collection is triggered if the old space does not have sufficient space to receive the new space. Usage of Effective Old Space (Cluster wide)[%] 107,00

Hostname

Cluster Element ID


308065550

Average [%] 107,00

Thresholds: Although high resource utilization may be desirable, it leaves little room to cope with sudden high-load situations. The typical, non-critical value of effective old space usage is approximately 50%. The higher the value, the less leeway there is for high load situations. If the usage of old space is substantially larger than 50%, high-load situations can lead to rapid growth phases within the old space and therefore to sudden resource outages. As a result, average values that are higher than 80% can be regarded as indicative of critical resource utilization.

Recommendation: If persistently high values of effective old-space usage are observed, related parameters (such as the old-space growth rate) should be analyzed as a preliminary measure. If growth rates are observed within the cluster-high old-space, this may be due to high baseline resource utilization. In some cases, adding additional server nodes to make better use of available resources may be recommended in order to reduce the overall baseline usage. It may be advisable to enlarge the heap, depending on the type of application deployed and the operating system used. However, an increased heap size will also increase the average time required for a full garbage collection, and so any changes will have to be analyzed carefully using the parameters defined here.

1.4 Fraction of Full Garbage Collections In general, minor garbage collections must occur more frequently than full garbage collections. A consecutive number of full garbage collections is a clear indicator that the Java Virtual Machine (JVM) is not removing objects from the old space. If the dynamic behavior of the JVM is dominated by old space memory removal, insufficient resources are available to accept requests and their newly-created objects. In this case, the parameter below is analyzed. On a floating interval of all garbage collections, the fraction of full garbage collections is calculated. Hostname

Cluster Element ID

Max [%]


308065550

100.00

Thresholds: For typical, non-critical situations, the maximum value that should be observed is 5%. Therefore, values much larger than 10% can be regarded as critical.

© SAP AG

E2E100

8-2

Recommendation: High, full garbage collection ratios indicate consecutive periods of full garbage collections. The cause of this should be investigated further by comparing the other parameters calculated in this section. Such phases indicate critical situations.

1.5 Allocation Rate The allocation rate quantifies the amount of memory allocated on the young generation each second. This therefore quantifies the input to the Java VM. Particularly high values indicate high JVM throughput and so should always be analyzed in parallel with the user load or the number of HTTP sessions. Allocation Rate (Cluster wide)[KB/s] 4642

Hostname

Cluster Element ID


308065550

Average [KB/s] 4642

1.6 Promotion Rate The promotion rate quantifies the amount of memory transferred to the old space each second. Due to the infant mortality of Java objects, this value should always be much smaller than the allocation rate. One indicator of critical situations is a high ratio of promotion rate to allocation rate, since this implies efficient use of the Eden space cannot be made. Promotion Rate (Cluster wide)[KB/s] 575

Hostname

Cluster Element ID


308065550

Average [KB/s] 575

2 Java Application Performance 2.1 Top WebDynpro Applications This section measures and reports the average response time over the reporting interval and the accumulated time of the identified WebDynpro application.

2.1.1 Response Time The following table lists the top Web Dynpro applications sorted by the average response time. Note that the average was calculated across all application nodes of the SAP J2EE cluster. Rank Name

Average Response Time [ms]

1 2 3 4 5

1710 1412 963 603 401

sap.com/tc~sec~ume~wd~umeadmin/UmeAdminPortalApp sap.com/new_page/PageWizard sap.com/tc~lm~webadmin~mainframe~wd/WebAdminApp sap.com/pb/PageBuilder sap.com/new_iview/Applic

Recommendations: Investigate the tuning potential of Web Dynpro applications that have persistently high average response times.

© SAP AG

E2E100

8-3

2.1.2 Accumulated Time The following list is sorted by accumulated response time during the reporting interval. The list can be used to determine which application effectively caused the greatest amount of load on the J2EE, since it is not only influenced by the average response time but also by the number of instances the application created at runtime. Rank

Name

Accumulated Time [h]

1 2 3 4 5

sap.com/tc~lm~webadmin~mainframe~wd/WebAdminApp sap.com/new_page/PageWizard sap.com/pb/PageBuilder sap.com/tc~sec~ume~wd~umeadmin/UmeAdminPortalApp sap.com/new_iview/Applic

0.06 0.00 0.00 0.00 0.00

Recommendations: Web Dynpro applications with a large accumulated time potentially have a larger overall performance impact. This may be worth further investigation.

2.2 Top Servlets Both the average response time over the reporting interval as well as the accumulated time of the identified servlet is measured and reported here.

2.2.1 Response Time The following table lists the top servlet/JSP requests sorted by the average response time. Note that the average was calculated across all application nodes of the SAP J2EE cluster. Rank

Name


1 2 3 4 5 6 7

DispatcherServlet SAPMLogonServlet Dispatcher Gateway JSPServlet SoapServlet WebAdminRedirector

2616 136 60 40 5 1 0

Recommendations: The performance tuning potential of servlet/JSP requests with high average response times should be investigated.

2.2.2 Accumulated Time The following top list is sorted by the accumulated response time during the reporting interval. This is useful for determining which application placed the greatest load on the J2EE, as this is influenced by both the average response time and the number of times that the application is created at runtime. Rank

Name


1 2 3 4 5 6 7

DispatcherServlet Dispatcher Gateway SoapServlet SAPMLogonServlet JSPServlet WebAdminRedirector

0.19 0.12 0.04 0.01 0.00 0.00 0.00

Recommendations: Servlet/JSP requests with a high accumulated time potentially have a larger overall impact on performance that may be worth investigating further.

© SAP AG

E2E100

8-4

2.3 Top JCO Calls 2.3.1 Response Time The following table lists the top JCo calls sorted by the average response time. Note that the average was calculated across all application nodes of the SAP J2EE cluster. Rank Name


1 2 3 4 5 6 7 8 9 10

44 30 29 15 10 10 10 5 5 0

HNI-001@ls165610|JCO|RFC_GET_FUNCTION_INTERFACE HNI-001@ls165610|JCO|RFC_READ_TABLE HNI-001@ls165610|JCO|UWL_GET_ITEMS_PROXY HNI-001@ls165610|JCO|RFC_GET_SAP_SYSTEM_PARAMETERS SM6-001@p135809|JCO|RFC_PING HNI-001@ls165610|JCO|UWL_ASSIGN_USER_CHANNEL HNI-001@ls165610|JCO|DDIF_FIELDINFO_GET HNI-001@ls165610|JCO|RPY_EXISTENCE_CHECK_CLAS HNI-001@ls165610|JCO|MSS_GET_SY_DATE_TIME HNI-001@ls165610|JCO|DD_GET_UCLEN

Recommendations: JCo calls with high average response times should be investigated with respect to the performance of the associated back-end application or, if all JCo calls show persistently bad performance in general, with respect to potential network-related performance issues. The following top list is sorted by the accumulated response time during the reporting interval. Rank Name


1 2 3 4 5 6 7 8 9 10

0.05 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

HNI-001@ls165610|JCO|UWL_GET_ITEMS_PROXY HNI-001@ls165610|JCO|RFC_GET_FUNCTION_INTERFACE HNI-001@ls165610|JCO|DDIF_FIELDINFO_GET HNI-001@ls165610|JCO|RFC_GET_SAP_SYSTEM_PARAMETERS HNI-001@ls165610|JCO|RFC_READ_TABLE SM6-001@p135809|JCO|RFC_PING HNI-001@ls165610|JCO|UWL_ASSIGN_USER_CHANNEL HNI-001@ls165610|JCO|RPY_EXISTENCE_CHECK_CLAS HNI-001@ls165610|JCO|MSS_GET_SY_DATE_TIME HNI-001@ls165610|JCO|DD_GET_UCLEN

Recommendations: Investigate the potential overall performance impact of JCo calls with high accumulated times.

2.4 Top SQL Selects This section analyzes the top SQL select statements observed. These statements are presented as top lists and are sorted by both average response time and accumulated time.

2.4.1 Response Time The following table lists the top SQL select statements sorted by the average response time. Note that the average was calculated across all application nodes of the SAP J2EE cluster.

© SAP AG

E2E100

8-5

Rank Name


1

2839

2

3

4

5

6

7

jdbc%nwmss%sqlserver%//wdfd00163713a%1433;connectionretrycount=0;receivestringpara metertype=nvarchar;alternateservers=;databasename=f15;batchperformanceworkaround=fals e;authenticationmethod=auto;sendstringparametersasunicode=true;logintimeout=0;wsid=;res ultsetmetadataoptions=0;alwaysreporttriggerresults=false;useserversideupdatablecursors=fal se;selectmethod=direct;connectionretrydelay=3;insensitiveresultsetbuffersize=2048;maxpoole dstatements=0;codepageoverride=;netaddress=000000000000;programname=;loadbalancing =false;hostprocess=0|SQL|Prepared|Query|SELECT COUNT (*) FROM "EP_ATTR_HEADERS" jdbc%nwmss%sqlserver%//wdfd00163713a%1433;connectionretrycount=0;receivestringpara metertype=nvarchar;alternateservers=;databasename=f15;batchperformanceworkaround=fals e;authenticationmethod=auto;sendstringparametersasunicode=true;logintimeout=0;wsid=;res ultsetmetadataoptions=0;alwaysreporttriggerresults=false;useserversideupdatablecursors=fal se;selectmethod=direct;connectionretrydelay=3;insensitiveresultsetbuffersize=2048;maxpoole dstatements=0;codepageoverride=;netaddress=000000000000;programname=;loadbalancing =false;hostprocess=0|SQL|Prepared|Query|SELECT COUNT (*) FROM "EP_ATTR_VALUES" jdbc%nwmss%sqlserver%//wdfd00163713a%1433;connectionretrycount=0;receivestringpara metertype=nvarchar;alternateservers=;databasename=f15;batchperformanceworkaround=fals e;authenticationmethod=auto;sendstringparametersasunicode=true;logintimeout=0;wsid=;res ultsetmetadataoptions=0;alwaysreporttriggerresults=false;useserversideupdatablecursors=fal se;selectmethod=direct;connectionretrydelay=3;insensitiveresultsetbuffersize=2048;maxpoole dstatements=0;codepageoverride=;netaddress=000000000000;programname=;loadbalancing =false;hostprocess=0|SQL|Prepared|Query|SELECT "ID", "SUBTREE_ROOT_ID", "PARENT_ID", "NAME", "OBJECT_TYPE", "XFS_TYPE", "CLASS_TYPE", "NAMESPACE_ID", "DELTA_LINK_STATE" FROM "EP_OBJECTS" WHERE "NAMESPACE_ID" = ? AND ("SUBTREE_ROOT_ID" = "ID" OR "SUBTREE_ROOT_ID" < ?) AND "CLASS_TYPE" = ? AND NOT ("NAME" LIKE ? OR "NAME" LIKE ?) jdbc%nwmss%sqlserver%//wdfd00163713a%1433;connectionretrycount=0;receivestringpara metertype=nvarchar;alternateservers=;databasename=f15;batchperformanceworkaround=fals e;authenticationmethod=auto;sendstringparametersasunicode=true;logintimeout=0;wsid=;res ultsetmetadataoptions=0;alwaysreporttriggerresults=false;useserversideupdatablecursors=fal se;selectmethod=direct;connectionretrydelay=3;insensitiveresultsetbuffersize=2048;maxpoole dstatements=0;codepageoverride=;netaddress=000000000000;programname=;loadbalancing =false;hostprocess=0|SQL|Prepared|Query|SELECT COUNT (*) FROM "EP_OBJECTS" jdbc%nwmss%sqlserver%//wdfd00163713a%1433;connectionretrycount=0;receivestringpara metertype=nvarchar;alternateservers=;databasename=f15;batchperformanceworkaround=fals e;authenticationmethod=auto;sendstringparametersasunicode=true;logintimeout=0;wsid=;res ultsetmetadataoptions=0;alwaysreporttriggerresults=false;useserversideupdatablecursors=fal se;selectmethod=direct;connectionretrydelay=3;insensitiveresultsetbuffersize=2048;maxpoole dstatements=0;codepageoverride=;netaddress=000000000000;programname=;loadbalancing =false;hostprocess=0|SQL|Prepared|Query|SELECT "RID" FROM "UME_ACL_ACLENTRY" WHERE "HASH" IS NULL jdbc%nwmss%sqlserver%//wdfd00163713a%1433;connectionretrycount=0;receivestringpara metertype=nvarchar;alternateservers=;databasename=f15;batchperformanceworkaround=fals e;authenticationmethod=auto;sendstringparametersasunicode=true;logintimeout=0;wsid=;res ultsetmetadataoptions=0;alwaysreporttriggerresults=false;useserversideupdatablecursors=fal se;selectmethod=direct;connectionretrydelay=3;insensitiveresultsetbuffersize=2048;maxpoole dstatements=0;codepageoverride=;netaddress=000000000000;programname=;loadbalancing =false;hostprocess=0|SQL|Prepared|Query|SELECT "ID" FROM "UME_ACL_MANAGER" WHERE "APP" = ? AND "MANAGER" = ? jdbc%nwmss%sqlserver%//wdfd00163713a%1433;connectionretrycount=0;receivestringpara metertype=nvarchar;alternateservers=;databasename=f15;batchperformanceworkaround=fals e;authenticationmethod=auto;sendstringparametersasunicode=true;logintimeout=0;wsid=;res ultsetmetadataoptions=0;alwaysreporttriggerresults=false;useserversideupdatablecursors=fal se;selectmethod=direct;connectionretrydelay=3;insensitiveresultsetbuffersize=2048;maxpoole dstatements=0;codepageoverride=;netaddress=000000000000;programname=;loadbalancing =false;hostprocess=0|SQL|Prepared|Query|SELECT * FROM "UME_ACL_SUP_PERM" WHERE "OWNERID" = ?

© SAP AG

E2E100

2659

2068

922

296

266

211

8-6

Rank Name

8

9

10


jdbc%nwmss%sqlserver%//wdfd00163713a%1433;connectionretrycount=0;receivestringpara 170 metertype=nvarchar;alternateservers=;databasename=f15;batchperformanceworkaround=fals e;authenticationmethod=auto;sendstringparametersasunicode=true;logintimeout=0;wsid=;res ultsetmetadataoptions=0;alwaysreporttriggerresults=false;useserversideupdatablecursors=fal se;selectmethod=direct;connectionretrydelay=3;insensitiveresultsetbuffersize=2048;maxpoole dstatements=0;codepageoverride=;netaddress=000000000000;programname=;loadbalancing =false;hostprocess=0|SQL|Prepared|Query|SELECT "RID" FROM "UME_ACL_ACL" WHERE "HASH" IS NULL jdbc%nwmss%sqlserver%//wdfd00163713a%1433;connectionretrycount=0;receivestringpara 165 metertype=nvarchar;alternateservers=;databasename=f15;batchperformanceworkaround=fals e;authenticationmethod=auto;sendstringparametersasunicode=true;logintimeout=0;wsid=;res ultsetmetadataoptions=0;alwaysreporttriggerresults=false;useserversideupdatablecursors=fal se;selectmethod=direct;connectionretrydelay=3;insensitiveresultsetbuffersize=2048;maxpoole dstatements=0;codepageoverride=;netaddress=000000000000;programname=;loadbalancing =false;hostprocess=0|SQL|Prepared|Query|SELECT COUNT (*) FROM "EP_SHORT_STRS" jdbc%nwmss%sqlserver%//wdfd00163713a%1433;connectionretrycount=0;receivestringpara 156 metertype=nvarchar;alternateservers=;databasename=f15;batchperformanceworkaround=fals e;authenticationmethod=auto;sendstringparametersasunicode=true;logintimeout=0;wsid=;res ultsetmetadataoptions=0;alwaysreporttriggerresults=false;useserversideupdatablecursors=fal se;selectmethod=direct;connectionretrydelay=3;insensitiveresultsetbuffersize=2048;maxpoole dstatements=0;codepageoverride=;netaddress=000000000000;programname=;loadbalancing =false;hostprocess=0|SQL|Prepared|Query|SELECT "L"."QUALIFIEDNAME", "F"."VARIANT_TYPE", "F"."VARIANT_NAME", "F"."LAYER", "F"."DEPL_OBJ_PART", "F"."DEPL_OBJ_PART_TYPE", "F"."FILE_GUID", "T"."TYPE_NAME" FROM "TC_WDRR_MRO_FILES" "F", "TC_WDRR_MRO_LUT" "L", "TC_WDRR_TYPES" "T" WHERE "F"."DEPLOYABLE_OBJECT" = ? AND "F"."MRO_GUID" = "L"."GUID" AND "T"."GUID" = "L"."TYPEGUID"

Recommendations: Investigate the performance of the database associated with SQL select statements that have high average response times. If all statements show persistently bad performance in general, investigate potential networkrelated performance issues.

2.4.2 Accumulated Time The following top list is sorted by the accumulated response time during the reporting interval. Rank Name

1

2


jdbc%nwmss%sqlserver%//wdfd00163713a%1433;connectionretrycount=0;receivestringpara 0.00 metertype=nvarchar;alternateservers=;databasename=f15;batchperformanceworkaround=fals e;authenticationmethod=auto;sendstringparametersasunicode=true;logintimeout=0;wsid=;res ultsetmetadataoptions=0;alwaysreporttriggerresults=false;useserversideupdatablecursors=fal se;selectmethod=direct;connectionretrydelay=3;insensitiveresultsetbuffersize=2048;maxpoole dstatements=0;codepageoverride=;netaddress=000000000000;programname=;loadbalancing =false;hostprocess=0|SQL|Prepared|Query|SELECT "FBLOB" FROM "J2EE_CONFIGENTRY" WHERE "CID" = ? AND "NAMEHASH" = ? AND "NAME" = ? AND "ISFILE" = ? jdbc%nwmss%sqlserver%//wdfd00163713a%1433;connectionretrycount=0;receivestringpara 0.00 metertype=nvarchar;alternateservers=;databasename=f15;batchperformanceworkaround=fals e;authenticationmethod=auto;sendstringparametersasunicode=true;logintimeout=0;wsid=;res ultsetmetadataoptions=0;alwaysreporttriggerresults=false;useserversideupdatablecursors=fal se;selectmethod=direct;connectionretrydelay=3;insensitiveresultsetbuffersize=2048;maxpoole dstatements=0;codepageoverride=;netaddress=000000000000;programname=;loadbalancing =false;hostprocess=0|SQL|Prepared|Query|SELECT "CPATH", "CID", "CACHEMODE", "CTYPE" FROM "J2EE_CONFIG" WHERE "PARENTCID" = ?

© SAP AG

E2E100

8-7

Rank Name


3

0.00

4

5

6

7

8

jdbc%nwmss%sqlserver%//wdfd00163713a%1433;connectionretrycount=0;receivestringpara metertype=nvarchar;alternateservers=;databasename=f15;batchperformanceworkaround=fals e;authenticationmethod=auto;sendstringparametersasunicode=true;logintimeout=0;wsid=;res ultsetmetadataoptions=0;alwaysreporttriggerresults=false;useserversideupdatablecursors=fal se;selectmethod=direct;connectionretrydelay=3;insensitiveresultsetbuffersize=2048;maxpoole dstatements=0;codepageoverride=;netaddress=000000000000;programname=;loadbalancing =false;hostprocess=0|SQL|Prepared|Query|SELECT COUNT (*) FROM "EP_ATTR_HEADERS" jdbc%nwmss%sqlserver%//wdfd00163713a%1433;connectionretrycount=0;receivestringpara metertype=nvarchar;alternateservers=;databasename=f15;batchperformanceworkaround=fals e;authenticationmethod=auto;sendstringparametersasunicode=true;logintimeout=0;wsid=;res ultsetmetadataoptions=0;alwaysreporttriggerresults=false;useserversideupdatablecursors=fal se;selectmethod=direct;connectionretrydelay=3;insensitiveresultsetbuffersize=2048;maxpoole dstatements=0;codepageoverride=;netaddress=000000000000;programname=;loadbalancing =false;hostprocess=0|SQL|Prepared|Query|SELECT "ID", "ATTR_NAME_NUM", "DELTA_LINK_TYPE", "SUBTREE_ROOT_ID", "ORDERED", "ATTR_VAL_TYPE", "INHERITANCE_TYPE", "STORAGE_TYPE", "WRITE_PERM_TYPE", "VALUE_COUNT", "VALUE_LENGTH", "INT_VALUE", "DOUBLE_VALUE", "STR_SHORT_VALUE", "TEXT_TYPE", "LOCALE_LANGUAGE", "LOCALE_COUNTRY", "STR_LARGE_VALUE" FROM "EP_ATTR_HEADERS" WHERE "ID" = ? jdbc%nwmss%sqlserver%//wdfd00163713a%1433;connectionretrycount=0;receivestringpara metertype=nvarchar;alternateservers=;databasename=f15;batchperformanceworkaround=fals e;authenticationmethod=auto;sendstringparametersasunicode=true;logintimeout=0;wsid=;res ultsetmetadataoptions=0;alwaysreporttriggerresults=false;useserversideupdatablecursors=fal se;selectmethod=direct;connectionretrydelay=3;insensitiveresultsetbuffersize=2048;maxpoole dstatements=0;codepageoverride=;netaddress=000000000000;programname=;loadbalancing =false;hostprocess=0|SQL|Prepared|Query|SELECT "ID", "ATTR_NAME_NUM", "DELTA_LINK_TYPE", "SORT_INDEX", "SUBTREE_ROOT_ID", "VALUE_LENGTH", "INT_VALUE", "DOUBLE_VALUE", "STR_SHORT_VALUE", "LOCALE_LANGUAGE", "LOCALE_COUNTRY", "STR_LARGE_VALUE" FROM "EP_ATTR_VALUES" WHERE "ID" = ? ORDER BY ?, ?, ? jdbc%nwmss%sqlserver%//wdfd00163713a%1433;connectionretrycount=0;receivestringpara metertype=nvarchar;alternateservers=;databasename=f15;batchperformanceworkaround=fals e;authenticationmethod=auto;sendstringparametersasunicode=true;logintimeout=0;wsid=;res ultsetmetadataoptions=0;alwaysreporttriggerresults=false;useserversideupdatablecursors=fal se;selectmethod=direct;connectionretrydelay=3;insensitiveresultsetbuffersize=2048;maxpoole dstatements=0;codepageoverride=;netaddress=000000000000;programname=;loadbalancing =false;hostprocess=0|SQL|Prepared|Query|SELECT COUNT (*) FROM "EP_ATTR_VALUES" jdbc%nwmss%sqlserver%//wdfd00163713a%1433;connectionretrycount=0;receivestringpara metertype=nvarchar;alternateservers=;databasename=f15;batchperformanceworkaround=fals e;authenticationmethod=auto;sendstringparametersasunicode=true;logintimeout=0;wsid=;res ultsetmetadataoptions=0;alwaysreporttriggerresults=false;useserversideupdatablecursors=fal se;selectmethod=direct;connectionretrydelay=3;insensitiveresultsetbuffersize=2048;maxpoole dstatements=0;codepageoverride=;netaddress=000000000000;programname=;loadbalancing =false;hostprocess=0|SQL|Prepared|Query|SELECT "NAME", "ISFILE", "DTYPE", "VBIGINT", "VDOUBLE", "VSTR", "VBYTES" FROM "J2EE_CONFIGENTRY" WHERE "CID" =? jdbc%nwmss%sqlserver%//wdfd00163713a%1433;connectionretrycount=0;receivestringpara metertype=nvarchar;alternateservers=;databasename=f15;batchperformanceworkaround=fals e;authenticationmethod=auto;sendstringparametersasunicode=true;logintimeout=0;wsid=;res ultsetmetadataoptions=0;alwaysreporttriggerresults=false;useserversideupdatablecursors=fal se;selectmethod=direct;connectionretrydelay=3;insensitiveresultsetbuffersize=2048;maxpoole dstatements=0;codepageoverride=;netaddress=000000000000;programname=;loadbalancing =false;hostprocess=0|SQL|Prepared|Query|SELECT "ID", "SUBTREE_ROOT_ID", "PARENT_ID", "NAME", "OBJECT_TYPE", "XFS_TYPE", "CLASS_TYPE", "NAMESPACE_ID", "DELTA_LINK_STATE" FROM "EP_OBJECTS" WHERE "NAMESPACE_ID" = ? AND ("SUBTREE_ROOT_ID" = "ID" OR "SUBTREE_ROOT_ID" < ?) AND "CLASS_TYPE" = ? AND NOT ("NAME" LIKE ? OR "NAME" LIKE ?)

© SAP AG

E2E100

0.00

0.00

0.00

0.00

0.00

8-8

Rank Name

9

10


jdbc%nwmss%sqlserver%//wdfd00163713a%1433;connectionretrycount=0;receivestringpara 0.00 metertype=nvarchar;alternateservers=;databasename=f15;batchperformanceworkaround=fals e;authenticationmethod=auto;sendstringparametersasunicode=true;logintimeout=0;wsid=;res ultsetmetadataoptions=0;alwaysreporttriggerresults=false;useserversideupdatablecursors=fal se;selectmethod=direct;connectionretrydelay=3;insensitiveresultsetbuffersize=2048;maxpoole dstatements=0;codepageoverride=;netaddress=000000000000;programname=;loadbalancing =false;hostprocess=0|SQL|Prepared|Query|SELECT "INT_VALUE", "VALUE_LENGTH", "BIN_VALUE" FROM "EP_ATTR_HEADERS" WHERE "ID" = ? AND "ATTR_NAME_NUM" = ? AND "DELTA_LINK_TYPE" = ? jdbc%nwmss%sqlserver%//wdfd00163713a%1433;connectionretrycount=0;receivestringpara 0.00 metertype=nvarchar;alternateservers=;databasename=f15;batchperformanceworkaround=fals e;authenticationmethod=auto;sendstringparametersasunicode=true;logintimeout=0;wsid=;res ultsetmetadataoptions=0;alwaysreporttriggerresults=false;useserversideupdatablecursors=fal se;selectmethod=direct;connectionretrydelay=3;insensitiveresultsetbuffersize=2048;maxpoole dstatements=0;codepageoverride=;netaddress=000000000000;programname=;loadbalancing =false;hostprocess=0|SQL|Prepared|Query|SELECT "XMLVALUE" FROM "BC_DDDBTABLERT" WHERE "NAME" = ?

Investigate the potential overall performance impact of SQL statements with high accumulated times.

3 Enterprise Portal Activity Reporting 3.1 Top Iviews 3.1.1 Response Time Rank Name


1 2 3

6950 6950 5115

4 5 6 7

8

9

10

com.sap.portal.appdesigner.pageeditor.LayoutJsp| com.sap.portal.layouts.default.doubleTEqualWidth| com.sap.portal.templateSelection|pcd-portal_content/com.sap.pct/admin.templates/ iviews/editors com.sap.km.PortalFavorites|pcd-portal_content/every_user/general/defaultDesktop/ frameworkPages/frameworkpage/com.sap.portal.innerpage com.sap.portal.masthead|pcd-portal_content/every_user/general/defaultDesktop/fra meworkPages/frameworkpage com.sap.portal.pageToolbar|pcd-portal_content/every_user/general/defaultDesktop/ frameworkPages/frameworkpage com.sap.portal.lockadmin|pcd-portal_content/administrator/super_admin/super_admi n_role/com.sap.portal.system_administration/com.sap.portal.system_admin_ws/com.s ap.portal.monitoring/portal com.sap.portal.usersInLast3Hours|pcd-portal_content/administrator/super_admin/su per_admin_role/com.sap.portal.system_administration/com.sap.portal.system_admin_ ws/com.sap.portal.monitoring/portal/com.sap.portal.activityReports com.sap.portal.10mostActiveUsers|pcd-portal_content/administrator/super_admin/su per_admin_role/com.sap.portal.system_administration/com.sap.portal.system_admin_ ws/com.sap.portal.monitoring/portal/com.sap.portal.activityReports com.sap.portal.statusOfActivityReporting|pcd-portal_content/administrator/super_ admin/super_admin_role/com.sap.portal.system_administration/com.sap.portal.syste m_admin_ws/com.sap.portal.monitoring/portal/com.sap.portal.activityReports

2724 1551 1505 1242

550

540

410

3.1.2 Accumulated Time Rank Name


1 2

0.03 0.02

com.sap.portal.heartbeats.PortalHeartbeat| com.sap.km.PortalFavorites|pcd-portal_content/every_user/general/defaultDesktop/ frameworkPages/frameworkpage/com.sap.portal.innerpage

© SAP AG

E2E100

8-9

Rank Name


3 4

0.01 0.01

5

6 7 8 9 10

com.sap.portal.navigation.portallauncher.default| com.sap.portal.templateSelection|pcd-portal_content/com.sap.pct/admin.templates/ iviews/editors com.sap.portal.contentCatalogTreeContent|pcd-portal_content/administrator/super_ admin/super_admin_role/com.sap.portal.content_administration/com.sap.portal.cont ent_admin_ws/com.sap.portal.portal_content com.sap.portal.masthead|pcd-portal_content/every_user/general/defaultDesktop/fra meworkPages/frameworkpage com.sap.portal.pageToolbar|pcd-portal_content/every_user/general/defaultDesktop/ frameworkPages/frameworkpage com.sap.portal.appdesigner.pageeditor.LayoutJsp| com.sap.portal.layouts.default.doubleTEqualWidth| com.sap.portal.httpconnectivity.urliviews.editor|

© SAP AG

E2E100

0.00

0.00 0.00 0.00 0.00 0.00

8-10


SAP System ID

ERP

SAP Component

ERP

Release

2004

Status

Productive

DB System

ORACLE 9.2.0.6.0

© SAP AG

E2E100

8-11


Date of Session

23.01.2007

Date of Report

23.01.2007

Installation No.

Author


Customer No.

Session No.

© SAP AG

E2E100

8-12

1 Service Summary The EarlyWatch Alert Service has detected some severe problems that may cause you to lose business. We recommend that you take corrective action immediately.

This EarlyWatch Alert session has been rated RED due to critical problems. The overall session is rated RED if one of the following areas is rated RED. o DB Performance o Database Administration: - DB growth - Missing indexes - Critical database problems The table below lists all of your system alerts, including those critical area(s) that have led to the overall RED rating. Note: Only sections mentioned above and that are rated RED in the following table, can set the overall rating to RED. Note: All recommendations provided in this report are based on our general experience only. We advise you to test our recommendations before using them in your production system. Also note that EarlyWatch Alert is an automatic service. Section Overview

Rating

Checked Area System Configuration System Performance Workload Distribution SAP System Operating Database Settings Database Administration Database server load from expensive SQL statements Security Checks

Note: For more information about EarlyWatch Alert, a sample EarlyWatch Alert report with explanations is available in the Media Library on SAP Service Marketplace (service.sap.com/ewa). This provides an overview of the check rating strategy and the KPIs that trigger the EWA alerts.

Alert Message Overview

Priority

Description

New Alert

High

Performance problems exist or are expected.

New

High

Upgrade your SAP R/3 plug-in to the latest release available.

New

Medium

There are security issues in your system.

New

Medium

We detected several ABAP dumps in your system.

New

Medium

Your database is growing very rapidly.

New

Note: If you need help investigating the alerts listed, order an EarlyWatch contract by contacting your local support organization, or by creating a customer message in component XX-SER-TCC. If you already have such a contract, a detailed analysis can be performed during the next Service Session. Please address the topic during the preparation of the session. Note: If you experience serious performance problems, create a customer message on component XX-SER-TCC with priority 'high' or 'very high'.

2 Performance Indicators The following table shows the relevant performance indicators in various system areas.

Area

Indicators

Value

Trend

System Performance

Active Users

134

down


1800 ms

up


10581

down


1356 ms

up


99 %

steady


1154 ms

up


489 ms

up

DB Size

283.34 GB

steady


18.85 GB

steady



3 Recommended Service The analysis of your SAP EarlyWatch Alert service session has shown that your installation has a high optimization potential. Therefore, SAP would highly recommend you to perform following service:

Service

Issue

Scope

Identifies and eliminates basic SAP Remote System performance problems that are Performance performance is not already clearly located and Optimization Service satisfactory identified.

© SAP AG

E2E100

Benefit

Average Consulting Days

Optimized response times and throughput

1

8-2

4 System Configuration To ensure optimal support of your SAP system, the SAP components, database, and operating system used must all be supported. Detailed information about SAP's platform support is available on SAP Service Marketplace at service.sap.com/platforms.

Rating

Check Hardware Configuration Software Configuration SAP Service Preparation

4.1 Hardware Configuration The table below shows the detected hardware configuration: HARDWARE CONFIGURATION

Server Type

Server

OS

OS Version

Number of CPUs

Memory [MB]

Database Server

h80062

WIN-NT

5.2

4

9727

DATABASE SYSTEM

Database Server

Database System

Current Version

h80062

ORACLE

09.2.0

4.2 Software Configuration The table below shows the installed software components. The "Rating" column is filled only for the SAP R/3 PlugIn (PI) and SAP Basis Plug-In (PI_BASIS), and is based on the maintenance schedule. COMPONENTS

Component

Component Release

Patch Level

EA-APPL

500

0005

EA-DFPS

500

0005

EA-FINSERV

500

0005

EA-GLTRADE

500

0005

EA-HR

500

0005

EA-IPPE

300

0004

EA-PS

500

0005

EA-RETAIL

500

0005

PI

2004_1_500

0004

© SAP AG

E2E100

Rating

8-3

Component

Component Release

Patch Level

PI_BASIS

2004_1_640

0009

SAP_ABA

640

0012

SAP_APPL

500

0006

SAP_BASIS

640

0013

SAP_BW

350

0012

SAP_HR

500

0006

ST-A/PI

01H_ECC500

0000

ST-PI

2005_1_640

0004

Rating

The table above shows a RED rating for your current SAP Plug-In because it is out of maintenance. This means that SAP no longer provides Support Packages for this component. You may also encounter problems if you want to integrate new products, since these may not be supported with this SAP Plug-In release. Recommendation: We strongly recommend that you upgrade your SAP Plug-In to the latest available release. Detailed information on the release strategy and the recommended upgrade path is available on SAP Service Marketplace at: service.sap.com/R3-Plug-In and in the subfolders (for example, Upgrade => R/3 Plug-In Upgrade Matrix).

4.2.1 SAP Kernel Release Your SAP kernel release (release 640, patch level 144) is not up to date. Recommendation: Ensure that you are using the latest SAP kernel. See SAP Notes 19466 and 138704. (These notes refer to further SAP Notes, depending on your system characteristics.)

4.3 SAP Service Preparation For detailed information about the required steps, see SAP Note 91488 (SAP Support Services: Central preparation note). Please ensure that all the functionality mentioned in the SAP Note is up to date, especially connections, collectors, and service tools.

4.3.1 SAP Service Tools Before we can deliver any services, the latest version of the SAP Service tools must be available in your system. The last run of report RTCCTOOL was 03.07.2006. Recommendation: You should run report RTCCTOOL, as described in the SAP Note 91488, to update tool release information. In case of missing transports detected by RTCCTOOL, implement the recommendations given by the tool.

© SAP AG

E2E100

8-4

5 Performance Overview The performance of your system was analyzed with respect to the average response times and total workload. We have detected severe problems that could have a serious financial impact on your business. You should take corrective action immediately. For further information, please contact SAP Support.

Rating

Check Performance Evaluation

The following table shows the average response times for various task types:

Task type

Dialog Steps

Avg. Resp. Time in ms

Avg. CPU Time in ms

Avg. Wait Time in ms

Avg. Load Time in ms

Avg. DB Time in ms

DIALOG

431438

1799,8

177,5

95,5

10,7

1154,0

RFC

114869

963,6

64,6

55,6

6,7

401,2

UPDATE

87008

4247,0

62,6

3541,8

6,0

489,2

UPDATE 2

71086

2075,3

28,2

1966,5

2,6

57,4

BATCH

123012

5363,8

1433,3

21,9

10,1

3169,0

SPOOL

30425

983,4

77,1

74,5

0,1

78,3

1238

1671,8

27,7

119,0

50,6

503,7

HTTP

Avg. GUI Time in ms 117,7

5.1 Current Workload The following table lists the number of current users (measured from our workload analysis) in your system.

Users

Low Activity

Medium Activity

High Activity

Total Users

Measured in System

65

98

36

199

5.2 Performance Evaluation For the task types DIALOG, UPDATE, and RFC, the measured average response times and the measured average database request times were compared with reference values. (Note: If the number of transaction steps are rather small, for example less than 20,000, an overall rating for this check is not reasonable.) The table below shows that a performance problem is expected for the task that is rated RED.

Task Dialog Steps Dia

431438

Upd

87008

RFC

114869

© SAP AG

Application Server Performance

E2E100

Database Server Performance

8-5

Rating Task Time Steps Avg. Response Time (ms)

Avg. CPU Time (ms) Avg. Database Time (ms)

Dia

08-9 34811 1416

346

873

Dia

09-10 50968 1293

144

917

Dia

10-11 47122 2573

106

1208

Dia

11-12 48353 1874

124

1473

Dia

12-13 32964 3744

116

1718

Dia

13-14 38377 1720

182

1046

Dia

14-15 52904 1356

110

1105

Dia

15-16 42438 1567

216

1187

Dia

16-17 36865 1386

215

1011

72

1543

Upd 12-13 3131

46936

For further information about the Performance section rating strategy, please refer to the EWA Sample Report at: service.sap.com/ewa -> Media Library -> Sample Report with explanations.

5.3 Transaction Profile Check The following tables show the response times and the number of dialog steps for the transactions that cause the highest workload in your system. NOTE: A business transaction normally consists of more than one dialog step. As a rough estimate, the actual number of business transactions that are executed in your system is approximately one third of the total dialog steps shown. (Transaction CPIC/RFC is not taken into account.)

5.3.1 Transactions by Total Workload The total system workload can be defined as the sum of the total response times of all transactions. The following transaction profiles list the transactions that have the greatest share in the total workload and are sorted by the percentage of total response times.

WORKLOAD BY TRANSACTION (DIALOG/UPDATE)

Transaction

Typ Dialog e Steps

Total

Total Resp. Avg. Resp. Time in % Time in ms

Avg. CPU Avg. DB Time in ms Time in ms

Avg. GUI Time in ms

519835

100,0

2215,0

159,2

1046,0

117,8

RSM13000

UPD

39579

11,1

3216,3

91,4

484,5

0,0

ZMRG

DIA

3919

10,3

30284,3

62,6

29987,1

157,6

SESSION_MANAGE DIA R

25984

7,0

3098,7

188,8

1432,6

293,7

FB01

UPD

2850

4,1

16421,2

19,3

487,7

0,0

S_ALR_87012168

DIA

1092

3,9

40849,7

437,2

40331,6

91,5

ZEDK

DIA

600

3,9

74194,0

186,5

73711,4

310,4

FBL3N

DIA

25403

3,6

1641,3

93,8

1298,1

125,9

VA02

UPD

1768

3,4

21948,3

66,5

199,4

0,0

FBL5N

DIA

37499

3,4

1030,7

88,1

590,1

38,1

© SAP AG

E2E100

8-6

Transaction

Typ Dialog e Steps

FBL1N

DIA


28703

2,6

Avg. CPU Avg. DB Time in ms Time in ms

1024,9

56,4

Avg. GUI Time in ms

437,0

90,7

WORKLOAD BY TRANSACTION (BATCH/SPOOL)

Total Avg. Resp. Resp. Time in ms Time in %

Typ Dialog e Steps

Transaction Total

Avg. CPU Time in ms

Avg. DB Time in ms

122850

100,0

5398,5

1454,2

3182,6

BTC

1328

28,3

141423,4

247,8

141150,8

Z_SD_EXTERNAL_INVOICE_CO BTC PY

67

14,1

1396575,7

1122932,2

177240,8

BTC

117

10,6

599855,3

514719,9

38798,7

RSAL_BATCH_TOOL_DISPATCH BTC ING

165

8,5

343281,6

2306,8

341552,6

SBIE0001

BTC

166

7,7

309299,3

49711,3

160508,4

SAPF150S2

BTC

9

4,1

3026382,2

15415,1

1106438,0

RSNAST00

BTC

732

3,6

32967,0

12321,6

21323,5

RBDAPP01

BTC

6732

3,6

3548,2

206,5

3077,4

SAPF110S

BTC

279

2,9

68388,0

2703,8

29186,3

RFUMSV00

BTC

3

2,3

5108803,0

3694916,7

1565258,3

RBDMIDOC

Z_FI_KREDITLIMIT

5.3.2 Transactions by DB load The total database workload generated from the System can be defined as the sum of the total database access times of all transactions. The following transaction profiles list the transactions that have the greatest share in the database load, sorted by percentage of total database access times. DATABASE LOAD BY TRANSACTIONS (DIALOG/UPDATE)

Transaction

Type Dialog Steps

Total

Total DB Time in %

Avg. DB Time in ms

519835

100,0

1046,0

ZMRG

DIA

3919

21,6

29987,1

ZEDK

DIA

600

8,1

73711,4

S_ALR_87012168

DIA

1092

8,1

40331,6

SESSION_MANAGER

DIA

25984

6,8

1432,6

FBL3N

DIA

25403

6,1

1298,1

FBL5N

DIA

37499

4,1

590,1

RSM13000

UPD

39579

3,5

484,5

VF03

DIA

5142

3,4

3644,3

FBL1N

DIA

28703

2,3

437,0

WE02

DIA

674

1,9

15654,1

© SAP AG

E2E100

8-7

DATABASE LOAD BY TRANSACTIONS (BATCH/SPOOL)

Transaction

Type Dialog Steps

Total

Total DB Time in % Avg. DB Time in ms

122850

100,0

3182,6

BTC

1328

47,9

141150,8

RSAL_BATCH_TOOL_DISPATCHING BTC

165

14,4

341552,6

SBIE0001

BTC

166

6,8

160508,4

RBDAPP01

BTC

6732

5,3

3077,4

RSNAST00

BTC

732

4,0

21323,5

Z_SD_EXTERNAL_INVOICE_COPY

BTC

67

3,0

177240,8

SAPRSLOG

BTC

7

2,8

1590072,0

SAPF150S2

BTC

9

2,5

1106438,0

SAPF110S

BTC

279

2,1

29186,3

RFUMSV00

BTC

3

1,2

1565258,3

RBDMIDOC

5.4 History of Activity and Average Response Times The following graphics or tables show the system activity, which is the number of dialog steps and the average response times for the past 20 weeks.

© SAP AG

E2E100

8-8

The top table or graphic compares the total system activity to the dialog activity in the time window specified. The bottom table shows the average CPU, DB, and wait times as parts of the average response time for dialog tasks in the time window specified.

6 Workload Distribution The performance of your system was analyzed with respect to workload distribution. We did not detect any major problems that could affect the performance of your SAP System.

Rating

Check Workload by Application Module DB Load Profile

6.1 Workload by Application Module The following graphics or tables show how each application module contributes to the total system workload. Two workload aspects are shown: - CPU time: Total CPU load on all servers in the system landscape - Database time: Total database load generated by the application The #Customer# section contains all programs and transactions that start with Y or Z. The #Others# section contains all programs that cannot be assigned to an SAP standard application module (for example, an industry solution or a third party add-on development). CPU LOAD BY APPLICATION

Module

CPU Time in %

Customer

52,5

BC

12,8

FI

12,1

© SAP AG

E2E100

8-9

Module

CPU Time in %

SD-BIL

9,0

SD

4,1

CA

3,5

LO

2,2

Others

1,6

CO

1,4

DB LOAD BY APPLICATION

Module

DB Time in %

BC

40,1

Customer

20,0

FI

14,3

CA

9,1

Others

7,5

SD-BIL

4,1

SD

2,4

CO

1,3

6.2 DB Load Profile The following table or diagram shows the DB load caused by Dialog, RFC, and Background tasks over different time frames. The data given in the table represents the average number of database processes occupied by each task type in the database during the given time frames. These statistics are calculated as a weekly average, which means the average values over six working days with a unit of one hour. Periods between 00:00-06:00 and 21:00-24:00 contain an average value per hour. By comparing the load profiles for dialog and background activity, you have an overview of the amount of background activity during online working hours. DB LOAD PROFILE FOR DIALOG, RFC AND BATCH

Start_End Time

DB Load (DIA)

DB Load (RFC)

DB Load (BTC)

00--06

0,01

0,09

0,36

06--07

0,03

0,00

0,02

07--08

0,27

0,12

0,20

08--09

1,41

0,19

1,26

09--10

2,17

0,32

1,27

10--11

2,64

0,13

1,26

11--12

3,30

0,12

2,49

12--13

2,63

0,20

1,04

© SAP AG

E2E100

8-10

Start_End Time

DB Load (DIA)

DB Load (RFC)

DB Load (BTC)

13--14

1,86

0,09

1,26

14--15

2,71

0,07

1,92

15--16

2,33

0,05

1,41

16--17

1,73

0,09

1,32

17--18

1,50

0,02

0,93

18--19

0,27

0,03

0,16

19--20

0,10

0,01

0,26

20--21

0,02

0,03

0,20

21--24

0,02

0,05

0,30

7 SAP System Operating Your system was analyzed with respect to daily operation. We detected some problems that may impair system operation and stability. To ensure system stability, you should take corrective action as soon as possible.

Rating

Check System Availability Update Errors Table Reorganization ABAP Dumps

7.1 Update Errors In a system running under normal conditions, only a small number of update errors should occur. To set the rating for this check, the number of active users is also taken into consideration. We did not detect any problems.

7.2 Table Reorganization When analyzing your database, we found large or rapidly growing tables or indexes. Recommendation: Implement the SAP Notes listed below to reduce the size of some of these tables or indexes. Background: More information about Data Volume Management is available on the SAP Service Marketplace at service.sap.com/dvm

Table / Index Name

Size of Table / Index [MByte]

TST03

© SAP AG

Recommended SAP Note 8075,0 48400, 130978, 16083

E2E100

8-11

Table / Index Name


Recommended SAP Note

VRPMA~0

7815,0 332677, 44595

7.3 Transports The following graphics contain information about the number of objects which were transported to the SAP system. The three graphics/tables show objects in total, programs and table (content). TRANSPORTED OBJECTS

Date

Total Number of Transported Objects

SAP Objects

Customer Objects

Objects in Support Package

15.01.200 7

0

0

0

0

16.01.200 7

6

0

6

0

17.01.200 7

8

0

8

0

18.01.200 7

11

0

11

0

19.01.200 7

10

0

10

0

20.01.200 7

0

0

0

0

21.01.200 7

0

0

0

0

22.01.200 7

15

0

15

0

TRANSPORTED PROGRAMS

Date

Total Number of Transported Programs

SAP Programs

Customer Programs

Programs in Support Packages

15.01.200 7

0

0

0

0

16.01.200 7

2

0

2

0

17.01.200 7

2

0

2

0

18.01.200 7

3

0

3

0

19.01.200 7

4

0

4

0

20.01.200 7

0

0

0

0

21.01.200 7

0

0

0

0

22.01.200

3

0

3

0

© SAP AG

E2E100

8-12


Date

SAP Programs

Customer Programs

SAP Tables

Customer Tables


7 TRANSPORTED TABLES

Date

Total Number of Transported Tables

Tables in Support Packages

15.01.2007

0

0

0

0

16.01.2007

0

0

0

0

17.01.2007

0

0

0

0

18.01.2007

0

0

0

0

19.01.2007

0

0

0

0

20.01.2007

0

0

0

0

21.01.2007

0

0

0

0

22.01.2007

1

0

1

0

7.4 Program Errors (ABAP Dumps) 31 ABAP dumps have been recorded in your system since 17.01.2007. ABAP dumps are deleted after 7 days by default. To view the ABAP dumps in your system, call transaction ST22 and choose “Selection”. Then select a timeframe. ABAP DUMPS PER DAY

Date

Quantity of Dumps 17.01.2007

26

18.01.2007

32

19.01.2007

19

20.01.2007

0

21.01.2007

0

22.01.2007

3

DETECTED ABAP DUMPS

Error

Number Server (e.g.) User (e.g.)

Date (e.g.)

Time (e.g.)

DBIF_RSQL_INVALID_RSQL

1 h80062

FAKHIR

17.01.2007

08:11:20

COMPUTE_BCD_OVERFLOW

1 h80062

FAKHIR

17.01.2007

15:27:52

12 h80062

MERKEL

18.01.2007

11:09:57

TIME_OUT EXPORT_TABLE_UPDATE_CONFLICT

1 h80062

DDIC

18.01.2007

11:30:45

SPOOL_I/O_ERROR

1 h80062

GRASSA

18.01.2007

11:51:54

ROLLBACK_IN_POSTING

3 h80062

MERKEL

18.01.2007

12:54:34

© SAP AG

E2E100

8-13

Error


Date (e.g.)

Time (e.g.)

DBIF_RSQL_SQL_ERROR

10 h80062

BATCH_BC

18.01.2007

13:00:32

DBIF_REPO_SQL_ERROR

1 h80062

WITSCHKE

18.01.2007

13:10:53

LOAD_PROGRAM_NOT_FOUND

1 h80062

SLODOWSK

18.01.2007

15:51:30

8 Security Checks Critical security issues were detected in your system. We determined that several users have critical authorizations for certain transactions. See the information in the following checks. SAP offers a complete Security Optimization service for your system, which covers all of R/3 Basis and other SAP products. We recommend that you make use of this service. For more information, see SAP Service Marketplace at www.service.sap.com/SOS.

Rating

Check Users Authorized to Display all Tables Users Authorized to Start all Reports Users Authorized to Debug / Replace Users Authorized to Display Other Users Spool Request Users Authorized to Administer RFC Connections Users Authorized to Reset/Change User Passwords Password of One or More Standard Users is not Changed

8.1 Password of One or More Standard Users is not Changed Evaluated Risk - High Recommendation: Change the standard password for users SAP* DDIC, SAPCPIC, and EARLYWATCH in all clients. Do not delete the SAP* user because this enables the use of the hard-coded SAP* user. For more information, see the SAP Security Guides on SAP Service Marketplace at: www.service.sap.com/securityguide.

Client

User

000

SAPCPIC

001

SAPCPIC

100

SAPCPIC

© SAP AG

E2E100

8-14

9 Hardware Capacity We have checked your solution for possible CPU or memory bottlenecks and found that the hardware resources have been exhausted. This could have a negative impact on the overall performance of your system. We therefore recommend analyzing this issue in more detail during an SAP EarlyWatch Check.

Server

Max. CPU load [%]

h80062

Date

Rating

RAM [MB]

Max. Paging [% of RAM] Date 9727

30 18.01.2007

Rating

452 21.01.2007

10 Database Settings We have detected some problems with the settings of the database. These settings may affect performance.

Rating

Check Missing Indexes Database Key Performance Indicators Setup of the Temporary Tablespace Database Parameters Optimizer Statistics

10.1 Load per user The following table provides an overview of the load caused by different database users. Listed for each database user is the: (a) Percentage of executed statements (b) Number of block reads from disk (indicative for the I/O load caused by this database user) (c) Number of buffers accessed (d) Number of records returned as obtained from the shared pool (Oracle view V$SQL) at the time of the download The first line with the name TOTAL indicates the total number of (a), (b), (c) and (d) for all the user names.

10.1.1 User name

Executions

Disk reads

Buffer gets

Rows processed

TOTAL (not a user name!)

126769957

115183858

2292230513

468801521

© SAP AG

E2E100

8-15

User name

Executions

Disk reads

Buffer gets

Rows processed

SYS

1%

1%

0%

0%

SYSTEM

0%

0%

0%

0%

SAPERP

99%

98%

95%

99%

OPS$H80062\SAPSERVICEE

0%

1%

4%

1%

10.2 Database Parameters We noticed that the following database parameters, which are currently active in your system, do not follow our standard recommendations. These standard recommendations cannot take into account possible special aspects for your system. Deviations from those standard recommendations may be necessary in your case. We therefore advise you to discuss theses parameter settings in your next Service session. NOTE FOR ORACLE PARAMETER SETTINGS

Note number

Description

124361

Oracle DB parameters for R/3 Release >= 4.x

132503

Update of threshold values of database system check

Parameters

Current value

Description

Recommended value

_optimizer_join_sel_sanity_chec Checks the order of join k conditions

Please check SAP note 0124361

filesystemio_options


Filesystem I/O options

Parameter Description s

Current value

Recommended value

processes

Maximum number of database shadow/background processes

80


sessions

Maximum number of concurrent database sessions

96


To provide special functionality or control system behavior, Oracle has a number of events. These can be activated in the configuration file and/or dynamically in the server manager or sqlplus. In general, these events and parameters should only be activated on request. When they are no longer needed, they should be removed. However, there might be the need to activate some events during normal database operation. The following table displays the values for the parameter "event" as found in your system:

Parameter

Current Value

event

10183 trace name context forever, level 1 10191 trace name context forever, level 1 10028 trace name context forever, level 1

© SAP AG

E2E100

8-16

The following table lists events that may have to be activated. Please check the mentioned SAP Notes for details.

Event

Note number for Event

Description

10027

596420

System downtime during deadlock (ORA-60)

38043

960633

Merge Join Cartesian in complex join

11 Database Administration

In the checks performed, problems regarding the administration of your database were found.

Rating

Check Database Growth Backup Frequency Archive Frequency Freespace in Tablespaces Space Critical Objects brconnect -f check (sapdba -check) schedule

11.1 Database Growth The following figure shows the development of the size of your database in GB. DATABASE SIZE

Date

© SAP AG

DB Size used 31.01.2006

63,53

28.02.2006

63,95

31.03.2006

65,22

30.04.2006

69,17

31.05.2006

75,52

30.06.2006

97,08

31.07.2006

115,40

31.08.2006

147,99

30.09.2006

174,66

31.10.2006

207,38

30.11.2006

239,54 E2E100

8-17

Date


258,39

23.01.2007

283,34

An overview of the freespace development of your database in GB is shown here. DATABASE FREE SPACE

Date

DB Size free 31.01.2006

12,52

28.02.2006

12,48

31.03.2006

12,48

30.04.2006

12,18

31.05.2006

11,04

30.06.2006

6,77

31.07.2006

11,93

31.08.2006

10,88

30.09.2006

18,23

31.10.2006

13,32

30.11.2006

11,43

31.12.2006

14,89

23.01.2007

64,56

The following table shows you the current size and the monthly growth of your database in GB.

Date

© SAP AG

Current Size in GB


28.02.2006

63,95

0,42

31.03.2006

65,22

1,27

30.04.2006

69,17

3,95

31.05.2006

75,52

6,35

30.06.2006

97,08

21,56

31.07.2006

115,40

18,32

31.08.2006

147,99

32,59

30.09.2006

174,66

26,67

31.10.2006

207,38

32,72

30.11.2006

239,54

32,16

31.12.2006

258,39

18,85

23.01.2007

283,34

E2E100

8-18

Your database is growing at a rate of more than 30 GB of data per month. Recommendation: Analyze the top growing tables and check that the table growth is really necessary for your business process. Set up a archiving project that allows you to archive data periodically to keep the database at a manageable size. Note: Data archiving is particularly important before SAP Basis Release upgrades, conversion, or migration projects.

11.2 Oracle Release In general, we recommend that you install the most current Oracle patch set as soon as possible (see SAP Note 415900). Therefore, check SAP Note 539921 from time to time for the latest patch set for Oracle 9.2.0. Note that in addition to the patch set, further bug fixes might be available and recommended. See the following SAP Notes: For patch set 9.2.0.3 - 610445: Merge fix for Oracle 9.2.0.3 For patch set 9.2.0.4 - 695080: Merge fix for Oracle 9.2.0.4 For patch set 9.2.0.5 - 755629: KKO merge fix for Oracle 9.2.0.5 For patch set 9.2.0.6 - 834100: Merge fix for Oracle 9.2.0.6 For patch set 9.2.0.7 - 896903: Optimizer Merge fix for Oracle 9.2.0.7

11.3 Desupport of multibyte character sets as of Oracle 10g An important SAP Note is available concerning systems that run with an Oracle database and that make use of setting the database code page to one of the following multibyte character sets: JA16SJIS Japanese KO16KSC5601 Korean ZHS16CGB231280 Simplified Chinese ZHT16BIG5 Traditional Chinese This SAP Note is: Note 858869: Desupport of multibyte character sets as of Oracle 10g. (see also related note 695899) The database settings as shown above have been set by many Asian customers to allow third-party tools to connect to their databases. With Oracle 10 clients or kernel release 7.00, the settings above are no longer supported and a Unicode conversion of the systems prior to an upgrade becomes mandatory for these systems.

11.4 Important Notes for Oracle To optimize system performance and to prevent data loss or system downtime, implement the following SAP Notes as soon as possible.

Note number

Description

858869

Desupport of multibyte character sets as of Oracle 10g

871455

Performance when accessing DBA_SEGMENTS

© SAP AG

E2E100

8-19

12 Database server load from expensive SQL statements The expensive SQL statements that we have identified on your system cause such a heavy load, that they might overload the database server. Since the database server is a single resource, severe performance problems may affect all users.

12.1 Expensive SQL Statements DATABASE LOAD FROM EXPENSIVE STATEMENTS

CPU & memory Load [%] I/O Load [%] Startup Date Analysis Date Comment 34.0 -> 38.1

55.4 -> 63.0

12.01.2007

23.01.2007

The following table lists the objects that have the highest load due to expensive SQL statements. The load of the statement is evaluated against the current load in the database and the total load since startup. If the statement was first executed at a time close to the database startup or a time close to the analysis, then the lower or higher value will be more appropriate. A more technical explanation is available in SAP Note 551646. The database administrator should conduct a detailed analysis, or you may prefer to order one of the remote or onsite optimization services from the SAP Service Marketplace at service.sap.com/bppo -> SQL. EXPENSIVE STATEMENTS

Object Name

CPU & memory load [%]

I/O Load [%]

Instance Details Program Details

VBRP

6,80

0,00 01 (erp)

/1BCDWB/SAPLSF0... (6331)

S130

6,70

0,00 01 (erp)

LWRPLF03 (00050)

BDCPV

6,40

16,50 01 (erp)

LBD01F03 (00112)

ARFCSDATAVALU ES

5,60

0,30 01 (erp)

LARFCU03 (00412)

USER_INDEXES

5,50

0,10 01 (erp)

CL_SQL_STATEMEN... (642)

VBRP/VBRK

5,00

19,30 01 (erp)

Z_SD_EDK (149)

VBRK

1,90

22,70 01 (erp)

ZMONREGU (1168)

BDCPS

0,20

4,10 01 (erp)

RBDMONI_CCMS_IDOC (828)

Some statement loads are given within a range. If we state the load is 'up to' a certain value, the load ranges from very low value up to the maximum value (which is given after the '->'). You should cross reference the 'first load' date of each statement to the 'database startup' date, to see if the minimum load value or the maximum load value is most appropriate for a statement. If the database has not been active for at least one day when the analysis is done, the information given may not be accurate. Note: If an object name in the table above contains the character '/' it may be a statement to a join. If no object exists in the ABAP dictionary (transaction SE12) with the object name, then check for each individual table of the join, which are separated by '/'.

© SAP AG

E2E100

8-20

12.1.1 VBRP SQL Statement and Explain STATEMENT DETAILS

Operation

Object Type

SELECT

TABLE

Total Executions

Disk Reads

353748

Buffer Gets 6

Records Processed

155981797

353747

SELECT SUM( "FKIMG" ) FROM "VBRP" WHERE "MANDT" = :A0 AND "VBELN" = :A1 AND "UEPOS" = :A2 AND "FBUDA" = :A3 AND "KONDM" = :A4& Execution Plan SELECT STATEMENT ( Estimated Costs = 4 , Estimated #Rows = 0 ) 3 SORT AGGREGATE Estim. Bytes: 32 2 TABLE ACCESS BY INDEX ROWID VBRP ( Estim. Costs = 4 , Estim. #Rows = 1 ) Estim. Bytes: 32 1 INDEX RANGE SCAN VBRP~Z2 ( Estim. Costs = 6 , Estim. #Rows = 300 ) Search Columns: 2

12.1.2 S130 SQL Statement and Explain STATEMENT DETAILS

Operation

Object Type

SELECT

TABLE

Total Executions

Disk Reads

108475

21055

Buffer Gets 152790309

Records Processed 0

SELECT "SPMON" , "SPTAG" , "SPWOC" , "SPBUP" , "UMSMNG" FROM "S130" WHERE "MANDT" = :A0 AND "SSOUR" = :A1 AND "VRSIO" = :A2 AND "SPWOC" BETWEEN :A3 AND :A4 AND "MATNR" = :A5 AND "KUNAG" = :A6& Execution Plan SELECT STATEMENT ( Estimated Costs = 80 , Estimated #Rows = 0 ) 3 FILTER 2 TABLE ACCESS BY INDEX ROWID S130 ( Estim. Costs = 80 , Estim. #Rows = 1 ) Estim. Bytes: 56 1 INDEX RANGE SCAN S130~VAB ( Estim. Costs = 95 , Estim. #Rows = 15.765 ) Search Columns: 2

© SAP AG

E2E100

8-21

12.1.3 BDCPS SQL Statement and Explain STATEMENT DETAILS

Operation

Object Type

SELECT

TABLE

Total Executions

Disk Reads 816

Buffer Gets

4719410

Records Processed

4858122

816

SELECT COUNT(*) FROM "BDCPS" WHERE "PROCESS" = :A0 AND "MANDT" = :A1& Execution Plan SELECT STATEMENT ( Estimated Costs = 870 , Estimated #Rows = 0 ) 2 SORT AGGREGATE Estim. Bytes: 4 1 TABLE ACCESS FULL BDCPS ( Estim. Costs = 870 , Estim. #Rows = 713.852 ) Estim. Bytes: 2.855.408

12.1.4 BDCPV SQL Statement and Explain STATEMENT DETAILS

Operation

Object Type

SELECT

TABLE

Total Executions

Disk Reads

7553

Buffer Gets

19012931

146195136

Records Processed 33717

SELECT "MANDT" , "CPIDENT" , "MESTYPE" , "PROCESS" , "TABNAME" , "TABKEY" , "FLDNAME" , "CRETIME" , "ACTTIME" , "USRNAME" , "CDOBJCL" , "CDOBJID" , "CDCHGNO" , "CDCHGID" FROM "BDCPV" WHERE "MANDT" = :A0 AND "MESTYPE" = :A1 AND "PROCESS" = :A2 AND "CRETIME" <= :A3 AND "CDOBJCL" = :A4 ORDER BY "MANDT" , "CPIDENT"& Execution Plan SELECT STATEMENT ( Estimated Costs = 385 , Estimated #Rows = 0 ) 5 MERGE JOIN ( Estim. Costs = 385 , Estim. #Rows = 12.098 ) Estim. Bytes: 1.488.054 1 INDEX RANGE SCAN BDCPS~1 ( Estim. Costs = 1.112 , Estim. #Rows = 118.975 ) Search Columns: 3 Estim. Bytes: 2.379.500 4 SORT JOIN ( Estim. Costs = 274 , Estim. #Rows = 11.918 )

© SAP AG

E2E100

8-22

Estim. Bytes: 1.227.554 Estim. Temp. Space: 3.220.000 Last WorkArea Mem kB: 212.992 3 TABLE ACCESS BY INDEX ROWID BDCP ( Estim. Costs = 75 , Estim. #Rows = 11.918 ) Estim. Bytes: 1.227.554 2 INDEX RANGE SCAN BDCP~POS ( Estim. Costs = 96 , Estim. #Rows = 1 ) Search Columns: 2

12.1.5 VBRP/VBRK SQL Statement and Explain STATEMENT DETAILS

Operation

Object Type

SELECT

JOIN

Total Executions

Disk Reads

17414

22156290



SELECT T_00 . "KONDA_AUFT" , T_00 . "NETWR" , T_00 . "VBELN" FROM "VBRP" T_00 , "VBRK" T_01 WHERE ( T_01 . "MANDT" (+) = :A0 AND T_00 . "VBELN" = T_01 . "VBELN" (+) ) AND T_00 . "MANDT" = :A1 AND T_00 . "ZZVGBEL" = :A2&

12.1.6 VBRK SQL Statement and Explain STATEMENT DETAILS

Operation

Object Type

SELECT

TABLE

Total Executions

Disk Reads 866

26110649



SELECT * FROM "VBRK" WHERE "MANDT" = :A0 AND ( "XBLNR" = :A1 OR "ZUONR" = :A2 )& Execution Plan SELECT STATEMENT ( Estimated Costs = 3.613 , Estimated #Rows = 0 ) 2 TABLE ACCESS BY INDEX ROWID VBRK ( Estim. Costs = 3.613 , Estim. #Rows = 161 ) Estim. Bytes: 58.604 1 INDEX RANGE SCAN VBRK~LOC ( Estim. Costs = 3.259 , Estim. #Rows = 560.984 ) Search Columns: 1

© SAP AG

E2E100

8-23

12.1.7 USER_INDEXES SQL Statement and Explain STATEMENT DETAILS

Operation

Object Type

SELECT

TABLE

Total Executions

Disk Reads 180

52944



SELECT index_name, table_name, uniqueness FROM user_indexes ORDER BY index_name& Execution Plan SELECT STATEMENT 24 SORT ORDER BY Last WorkArea Mem kB: 2.416.640 23 NESTED LOOPS OUTER 20 NESTED LOOPS OUTER 17 NESTED LOOPS 14 NESTED LOOPS OUTER 11 NESTED LOOPS 8 NESTED LOOPS OUTER 5 NESTED LOOPS 2 TABLE ACCESS BY INDEX ROWID OBJ$ 1 INDEX RANGE SCAN I_OBJ2 Search Columns: 1 4 TABLE ACCESS BY INDEX ROWID IND$ 3 INDEX UNIQUE SCAN I_IND1 Search Columns: 1 7 TABLE ACCESS BY INDEX ROWID OBJ$ 6 INDEX UNIQUE SCAN I_OBJ1 Search Columns: 1 10 TABLE ACCESS BY INDEX ROWID OBJ$ 9 INDEX UNIQUE SCAN I_OBJ1 Search Columns: 1 13 TABLE ACCESS CLUSTER USER$ 12 INDEX UNIQUE SCAN I_USER# Search Columns: 1 16 TABLE ACCESS CLUSTER USER$ 15 INDEX UNIQUE SCAN I_USER# Search Columns: 1 19 TABLE ACCESS CLUSTER SEG$ 18 INDEX UNIQUE SCAN I_FILE#_BLOCK# Search Columns: 3 22 TABLE ACCESS CLUSTER TS$

© SAP AG

E2E100

8-24

21 INDEX UNIQUE SCAN I_TS# Search Columns: 1

12.1.8 ARFCSDATAVALUES SQL Statement and Explain STATEMENT DETAILS

Operation

Object Type

INSERT

TABLE

Total Executions

Disk Reads

67861

Buffer Gets

307324

Records Processed

128711178

813074

INSERT INTO "ARFCSDATA" VALUES( :A0 , :A1 , :A2 , :A3 , :A4 , :A5 , :A6 , :A7 , :A8 , :A9 , :A10 , :A11 , :A12 , :A13 )&

12.2 Network Load from Expensive SQL statements EXPENSIVE SQL STATEMENTS DUE TO NETWORK TRAFFIC

Operation

Object Type

Object Name

Executions

Records

Program

SELECT

TABLE

D010SINF

19193761

16529389

LSEUTABLU04

SELECT

TABLE

AGR_HIER

52492

41785360

LSMTR_NAVIGATIO...

SELECT

TABLE

D010INC

143005

19193761

LSEUTABLU04

SELECT

TABLE

VAPMA

13

11942519

SDV03V02

SELECT

TABLE

CROSS

11

16387717

RSCROSS10

SELECT

TABLE

ZMM_TALON

3

6515977

LRSA3U03

SELECT

TABLE

REPOSRC

11

16221672

RSCROSS10

SELECT

TABLE

ZMM_TALON

1

1203538

/1BCDWB/DBZMM_T...

The table above contains details of the top statements that cause a high network load in your system. The table is provided for information only. Statements are listed if they are executed very often, if they return a high number of records for each execution, or both. They therefore do not have a technical problem or solution. Any investigation should focus on the application or program logic.

13 Trend Analysis This section contains the trend analysis for key performance indicators (KPIs). Diagrams or tables are built week by week after activating the EarlyWatch Alert Service.

13.1 System Activity The following diagrams or tables contain data about system activity: - Total Activity: Transaction steps performed each week (in thousands) - Dialog Activity: Transaction steps performed in dialog task each week (in thousands) - Peak Activity: Transaction steps in hour during the peak hour; this peak hour is calculated as the hour with the maximum dialog activity in the ST03 time profile divided by 5 (work days) - Total Users: Users that logged on in the week - Active Users: Users that have performed more than 400 transaction steps in the week SYSTEM ACTIVITY

Date

Total Activity

Dialog Activity

Peak Activity

18.06.2006

475

169

12

© SAP AG

E2E100

8-25

Date

Total Activity

Dialog Activity

Peak Activity

02.07.2006

750

161

12

08.07.2006

511

200

8

16.07.2006

509

224

14

23.07.2006

530

196

5

30.07.2006

478

174

4

06.08.2006

576

207

5

13.08.2006

626

276

8

20.08.2006

557

244

7

27.08.2006

593

271

8

03.09.2006

959

274

8

10.09.2006

922

300

8

17.09.2006

677

285

7

24.09.2006

672

281

7

01.10.2006

876

347

9

08.10.2006

910

278

7

15.10.2006

850

379

10

22.10.2006

706

314

9

29.10.2006

573

279

8

05.11.2006

473

232

7

12.11.2006

832

397

12

19.11.2006

969

370

10

25.11.2006

771

413

10

03.12.2006

1167

377

10

10.12.2006

1030

467

12

17.12.2006

798

395

10

31.12.2006

305

174

4

07.01.2007

949

502

13

14.01.2007

1433

521

13

21.01.2007

874

431

11

USERS

Date

Logged on

Active

18.06.2006

104

64

02.07.2006

108

70

08.07.2006

115

73

© SAP AG

E2E100

8-26

Date

Logged on

Active

16.07.2006

110

76

23.07.2006

108

73

30.07.2006

110

70

06.08.2006

122

84

13.08.2006

130

88

20.08.2006

121

88

27.08.2006

120

88

03.09.2006

125

88

10.09.2006

142

99

17.09.2006

141

98

24.09.2006

151

98

01.10.2006

145

89

08.10.2006

141

98

15.10.2006

159

106

22.10.2006

156

102

29.10.2006

171

100

05.11.2006

146

91

12.11.2006

170

113

19.11.2006

178

121

25.11.2006

173

118

03.12.2006

177

120

10.12.2006

188

127

17.12.2006

182

130

31.12.2006

129

76

07.01.2007

181

124

14.01.2007

191

143

21.01.2007

199

134

13.2 Response Times The following diagram or table show the development of the response times: - Average response time in dialog task (total, DB and CPU parts) - Average response times for database requests - Average response times for the top 5 transactions SYSTEM PERFORMANCE

Date

Dialog Response Time [ms]

18.06.2006 467 © SAP AG

Dialog DB Response Time [ms]

Dialog CPU Time [ms]

100

97 E2E100

8-27

Date




02.07.2006 1100

532

135

08.07.2006 547

130

175

16.07.2006 529

164

120

23.07.2006 730

264

124

30.07.2006 496

131

129

06.08.2006 701

206

111

13.08.2006 593

226

116

20.08.2006 532

179

87

27.08.2006 349

90

89

03.09.2006 466

188

92

10.09.2006 517

230

94

17.09.2006 371

125

108

24.09.2006 452

176

115

01.10.2006 376

124

110

08.10.2006 364

124

94

15.10.2006 526

286

108

22.10.2006 466

205

102

29.10.2006 389

166

92

05.11.2006 446

221

80

12.11.2006 666

375

114

19.11.2006 633

309

126

25.11.2006 578

224

133

03.12.2006 916

512

159

10.12.2006 959

628

136

17.12.2006 863

522

124

31.12.2006 693

471

85

07.01.2007 485

245

130

14.01.2007 1467

934

232

21.01.2007 1800

1154

178

DATABASE PERFORMANCE

Date

02.07.200 6 © SAP AG

Time per Direct Read in Dialog task [ms] 0,2

Time per Seq. Read in Dialog task [ms] 2,6

Time per Change in Dialog task [ms]

Time per Direct Read in Background task [ms]

6,5

0,1 E2E100

Time per Seq. Time per Change Read in in Background Background task task [ms] [ms] 1,6

8,5 8-28

Date

Time per Direct Read in Dialog task [ms]

Time per Seq. Read in Dialog task [ms]



Time per Seq. Time per Change Read in in Background Background task task [ms] [ms]

08.07.200 6

0,1

0,5

3,6

0,2

2,6

6,0

16.07.200 6

0,1

0,4

5,3

0,4

1,3

3,6

23.07.200 6

0,3

0,9

11,3

0,2

1,6

12,7

30.07.200 6

0,1

0,6

1,3

0,2

0,6

3,4

06.08.200 6

0,2

0,7

8,3

0,2

1,2

5,9

13.08.200 6

0,1

0,8

4,3

0,2

2,0

3,6

20.08.200 6

0,2

0,3

7,0

0,0

0,6

4,9

27.08.200 6

0,1

0,2

3,5

0,2

0,5

5,0

03.09.200 6

0,1

0,5

10,4

0,1

1,8

4,4

10.09.200 6

0,1

1,3

7,7

0,3

1,9

3,4

17.09.200 6

0,1

0,3

2,0

0,1

0,5

2,2

24.09.200 6

0,1

0,5

4,1

0,2

1,0

0,9

01.10.200 6

0,1

0,5

3,7

0,2

0,5

2,1

08.10.200 6

0,1

0,7

3,6

0,2

0,5

1,8

15.10.200 6

0,2

0,5

6,5

0,1

1,0

4,0

22.10.200 6

0,1

0,6

4,1

0,1

1,3

2,1

29.10.200 6

0,1

0,5

3,6

0,1

0,9

1,9

05.11.200 6

0,1

0,8

2,6

0,2

1,1

2,0

12.11.200 6

0,2

1,3

5,8

0,2

1,7

2,5

19.11.200 6

0,1

0,4

4,2

0,1

1,2

1,8

25.11.200 6

0,1

0,7

6,1

0,1

1,0

2,3

© SAP AG

E2E100

8-29

Date






03.12.200 6

0,2

2,0

6,9

0,1

2,3

2,0

10.12.200 6

0,2

2,1

3,8

0,0

1,6

1,1

17.12.200 6

0,2

1,2

4,8

0,1

2,7

1,9

31.12.200 6

0,1

3,2

2,9

0,0

6,9

1,0

07.01.200 7

0,1

1,0

4,5

0,1

2,6

1,1

14.01.200 7

0,4

1,8

3,8

0,4

2,9

1,9

21.01.200 7

0,2

2,8

10,2

0,2

4,1

2,1

TOP 5 TRANSACTIONS

Date

ZMRG

SESSION_MANAGER

S_ALR_87012168

ZEDK

19.11.2006

4481

917

28062

389

25.11.2006

2082

972

57062

621

03.12.2006

4259

1564

77630

642

10.12.2006

14555

1317

187308

899

17.12.2006

8035

1116

116034

800

31.12.2006

5687

754

101765

496

07.01.2007

4016

546

124364

466

14.01.2007

15507

1660

118802

763

21.01.2007

30284

3099

40850

74194

Transaction Code

Load (%)

ZMRG

10.3

SESSION_MANAGER

7.0

S_ALR_87012168

3.9

ZEDK

3.9

FBL3N

3.6

© SAP AG

E2E100

FBL3N

1641

8-30

13.3 System Operation The following diagram or table shows important KPIs for system operation. Note: The column 'Availability' is filled for systems with SAP Basis 4.6 and higher. SYSTEM OPERATION

Date

Availabity (%)

Program Errors (ABAP)

Update Errors

18.06.2006

100

196

0

02.07.2006

100

285

0

08.07.2006

100

16

1

16.07.2006

100

47

0

23.07.2006

100

23

1

30.07.2006

100

10

0

06.08.2006

100

4

0

13.08.2006

100

45

0

20.08.2006

96

378

1

27.08.2006

100

88

0

03.09.2006

93

94

0

10.09.2006

98

42

0

17.09.2006

100

83

1

24.09.2006

100

16

0

01.10.2006

100

53

0

08.10.2006

100

12

0

15.10.2006

94

50

0

22.10.2006

100

33

0

29.10.2006

97

95

1

05.11.2006

100

53

3

12.11.2006

100

28

0

19.11.2006

100

11

0

25.11.2006

100

13

0

03.12.2006

97

17

1

10.12.2006

98

42

2

17.12.2006

100

17

0

31.12.2006

100

6

5

07.01.2007

100

49

3

14.01.2007

95

356

2

21.01.2007

99

31

0

© SAP AG

E2E100

8-31

13.4 Hardware Capacity HARDWARE CAPACITY

Date

Database Server CPU Max. Load (%)

Appl. Server CPU Max. Load (%)

18.06.2006

35

0

02.07.2006

70

70

08.07.2006

39

39

16.07.2006

70

70

23.07.2006

92

92

30.07.2006

53

53

06.08.2006

61

61

13.08.2006

100

100

20.08.2006

85

85

27.08.2006

73

73

03.09.2006

90

90

10.09.2006

100

100

17.09.2006

24

24

24.09.2006

30

30

01.10.2006

45

45

08.10.2006

43

43

15.10.2006

72

72

22.10.2006

22

22

29.10.2006

30

30

05.11.2006

18

18

12.11.2006

28

28

19.11.2006

52

52

25.11.2006

46

46

03.12.2006

43

43

10.12.2006

31

31

17.12.2006

41

41

31.12.2006

22

22

07.01.2007

27

27

14.01.2007

66

66

21.01.2007

30

30

© SAP AG

E2E100

8-32

Exercises Unit:

EarlyWatch Alert

Lesson 3: EWA for SAP BI At the conclusion of this exercise, you will be able to: • Get familiar with important KPIs in Early Watch Alert for SAP BI • Analyze EWA for BI

1

Please open the example Early Watch Alert for SAP BI provided by trainer 1-1

© SAP AG

Please investigate this EWA based on the slides from APPENDIX and answer the following questions: •

Is there a general performance problem with BI?

•

If “Yes” what is the cause for this?

•

Is there a problem with Data Model of BI?

•

Is there a problem with workload distribution?

E2E100

8-33

© SAP AG

E2E100

8-34

IT Reporting

Unit:

Exercises

EarlyWatch Alert

Lesson 3: SAP EWA for SAP SCM / APO At the conclusion of this exercise, you will be able to: • Get familiar with important KPIs in Early Watch Alert for SAP SCM • Analyze EWA for SCM

1

Please open the example Early Watch Alert for SAP SCM / APO provided by trainer 1-1

© SAP AG


Is there a general problem with LCA?

•

If “Yes” what is the cause for this?

•

Is there a hardware bottleneck?

•

Is there a performance problem caused by LCA configuration?

•

Are there RFC errors?

E2E100

8-35

© SAP AG

E2E100

8-36

IT Reporting

Unit:

Exercises

EarlyWatch Alert

Lesson 3: SAP EWA for SAP XI At the conclusion of this exercise, you will be able to: • Get familiar with important KPIs in Early Watch Alert for SAP XI • Analyze EWA for SAP XI

1

Please open the example Early Watch Alert for SAP XI provided by trainer 1-1


© SAP AG

What is the average number of messages processed per day?

•

What is the average message processing time?

•

What is the tendency in XI performance development?

•

Are there the messages with errors (correctness of SAP XI operations)?

•

What is the workload distribution?

E2E100

8-37

© SAP AG

E2E100

8-38

IT Reporting

Unit:

Exercises

EarlyWatch Alert

Lesson 3: EWA for CRM (ISA, Field Sales, CIC) At the conclusion of this exercise, you will be able to: • Get familiar with important KPIs in Early Watch Alert for SAP CRM • Analyze EWA for CRM

1

Please open the example Early Watch Alert for SAP CRM provided by trainer 1-1

© SAP AG

Please investigate this EWA and answer the following questions depending on the scenario used: •

Which CRM scenarios are used?

•

Are there the BDoc messages with the status "Error" or "Not processed fully"?

•

Are the messages in the CRM application log?

•

What is the status of Replication and Realignment Queue?

•

Are there any problems with qRFC Administration?

•

What is the performance of the Internet Sales scenario in the system?

•

Are the Errors / Warnings during the catalog replication

E2E100

8-39

© SAP AG

E2E100

8-40


SAP System ID

IWP

SAP Component

SEM

Release

3.5

Status

Productive

DB System

ORACLE 9.2.0.7.0

© SAP AG

E2E100

8-41


Date of Session

09.10.2006

Date of Report

09.10.2006

Installation No.

Author


Customer No.

Session No.

© SAP AG

E2E100

8-42


This EarlyWatch Alert session has been rated RED due to critical problems. The overall session is rated RED if one of the following areas is rated RED. o DB Performance o Database Administration: - DB growth - Missing indexes - Critical database problems The table below lists all of your system alerts, including those critical area(s) that have led to the overall RED rating. Note: Only sections mentioned above and that are rated RED in the following table, can set the overall rating to RED. Note: All recommendations provided in this report are based on our general experience only. We advise you to test our recommendations before using them in your production system. Also note that EarlyWatch Alert is an automatic service. Section Overview

Rating

Checked Area System Configuration System Performance Workload Distribution SAP System Operating Database Settings Database Administration BW Checks Security Checks

Note: For more information about EarlyWatch Alert, a sample EarlyWatch Alert report with explanations is available in the Media Library on SAP Service Marketplace (service.sap.com/ewa). This provides an overview of the check rating strategy and the KPIs that trigger the EWA alerts. Alert Message Overview

Priority

Description

New Alert

Medium

We detected several ABAP dumps in your system.

New

Medium

Your database is growing very rapidly.

New

Note: If you need help investigating the alerts listed, order an EarlyWatch contract by contacting your local support organization, or by creating a customer message in component XX-SER-TCC. If you already have such a contract, a detailed analysis can be performed during the next Service Session. Please address the topic during the preparation of the session. Note: If you experience serious performance problems, create a customer message on component XX-SER-TCC with priority 'high' or 'very high'.


Area

Indicators

Value

Trend

System Performance

Active Users

357

up

Hardware Capacity

Max. CPU Utilization on DB Server

0%

-

Max. CPU Utilization on Appl. Server

55 %

up

DB Size

2326.34 GB

steady


171.31 GB

steady

Avg. Total Runtime of the BW Queries

16.4 s

down

Avg. Database Runtime of the BW Queries

12.1 s

down


Query Performance


Scope

Benefit


Service

Issue

SAP Remote Performance Optimization Service

Analyzes and optimizes Increased user Long runtime of critical business productivity and system 1 critical transactions processes acceptance

3.1 SAP Remote Performance Optimization Higher Return on Investment - Accelerate your core business processes. - Increase user productivity and system acceptance. - Make optimal use of your hardware resources. - Train your in-house staff to expertly monitor and maintain performance. With SAP Remote Performance Optimization, you use SAP's experience to save time, reduce costs, and keep your SAP solution running optimally. More information about the service is available on the SAP Service Marketplace at service.sap.com/smo.

© SAP AG

E2E100

8-2


Rating



Server Type

Server

OS

OS Version

Number of CPUs

Memory [MB]

Others

server1

HP-UX

N/A

32

65472

Database Server

db_server

N/A

N/A

32

N/A

Others

server2

WIN-NT

5.2

8

3711

Others

server3

WIN-NT

5.2

8

3711

Others

server4

WIN-NT

5.2

8

3711

Others

server5

WIN-NT

5.2

8

3711

Others

server6

WIN-NT

5.2

8

3711

Others

server7

WIN-NT

5.2

8

3711

Others

server8

WIN-NT

5.2

8

3711

Others

server9

WIN-NT

5.2

4

3711

Others

server10

WIN-NT

5.2

8

3711

Others

server11

WIN-NT

5.2

8

3711

Others

server12

WIN-NT

5.2

8

3711

Others

server13

WIN-NT

5.2

8

3711

Others

server14

WIN-NT

5.2

4

8023

Others

server15

WIN-NT

5.2

4

8023

Others

server16

WIN-NT

5.2

4

12119

Others

server17

WIN-NT

5.2

N/A

N/A

Others

server18

WIN-NT

5.2

4

12119

Others

server19

WIN-NT

5.2

4

12119

© SAP AG

E2E100

8-3

Server Type

Server

OS

OS Version

Number of CPUs

Memory [MB]

Others

server20

WIN-NT

5.2

4

12119

DATABASE SYSTEM

Database Server

Database System

Current Version

iwpdb

ORACLE

09.2.0

4.2 Software Configuration The table below shows the installed software components. The "Rating" column is filled only for the SAP R/3 PlugIn (PI) and SAP Basis Plug-In (PI_BASIS), and is based on the maintenance schedule. COMPONENTS

Component

Component Release

Patch Level

BI_CONT

330

0018

FINBASIS

200

0019

PI_BASIS

2005_1_620

0006

SAP_ABA

620

0055

SAP_BASIS

620

0055

SAP_BW

310

0023

SEM-BW

350

0019

ST-A/PI

01G_BCO620

0000

ST-PI

2005_1_620

0003

WP-PI

600_620

0000000000

Rating

4.2.1 SAP Kernel Release Your SAP kernel release (release 640, patch level 129) is not up to date. Recommendation: Ensure that you are using the latest SAP kernel. See SAP Notes 19466 and 138704. (These notes refer to further SAP Notes, depending on your system characteristics.)


4.3.1 SAP Service Tools Before we can deliver any services, the latest version of the SAP Service tools must be available in your system. The last run of report RTCCTOOL was 23.08.2006. Recommendation: You should run report RTCCTOOL, as described in the SAP Note 91488, to update tool release information. In case of missing transports detected by RTCCTOOL, implement the recommendations given by the tool.

© SAP AG

E2E100

8-4

5 Performance Overview The performance of your system was analyzed with respect to the average response times and total workload. We did not detect any major problems that could affect the performance of your system.


Task type DIALOG + RFC

Dialog Steps


Avg. CPU Time in ms

Avg. Wait Avg. Load Time in ms Time in ms

Avg. DB Time in ms

1600554

4243,0

839,4

261,5

5,1

2277,1

UPDATE

1317

517,7

25,9

111,5

11,5

167,4

UPDATE2

334

137,2

35,4

0,1

7,4

85,7

BATCH

310819

34845,7

3567,6

28,9

16,3

15935,5

SPOOL

12

1742,6

56,1

7,8

0,0

1250,1

HTTPS

129810

2970,3

645,5

9,3

0,2

130,3

Avg. GUI Time in ms 15,0


Users

Low Activity

Medium Activity

High Activity

Total Users

Measured in System

2813

304

53

3170

5.2 BW Workload 5.2.1 Workload per User and Navigation Steps This overview takes into account the following: - The number of users who execute queries (grand total) - This number is grouped according to InfoConsumer, Executive, and Power User (totals), depending on their number of navigation steps - The InfoConsumer is divided again according to the number of navigation steps (subtotals). - The timeframe is the last complete week from Monday to Sunday.

User/Consumer

Number

Grand total: Users performing queries

2461

Total: Info Consumer [1 - 400 Nav Steps/ week]

2435

...Sub total: Info Consumer 1-10 Nav Steps/ week

1393


738


163


85

© SAP AG

E2E100

8-5

User/Consumer

Number


36


20

Total: Executive [401 - 1200 Nav Steps/ week]

23

Total: Power User [> 1200 Nav Steps/ week]

3


© SAP AG

E2E100

8-6


6 BW Runtime Statistics The performance of your queries and upload was analyzed with respect to average runtime and total workload. The following table provides an overview of your system activity and performance from the BW point of view. Note: All queries using the 'Read API' of your system (such as from connected SAP-APO or SAP-SEM systems) are named 'RSDRI_QUERY,' so you cannot locate them in your BW system. Please note that the following chapters only contain queries/InfoCubes for which the statistics indicators are set. The following table lists the top five InfoProviders based on the number of query hits. TOP INFOPROVIDERS PER NUMBER OF QUERIES

InfoProvide Navigation r Steps

Avg. Runtime Avg. time runtime [s] [%] OLAP [s]

Avg. time Avg. Frontend Avg. # DB [s] time [s] Frontend cells

PROLFOS2

21171

17,90

51

0,50

14,20

3,00

51

PCRMSPIO P

8146

23,20

26

1,50

15,00

5,40

773

PCOPA_MU L

2074

29,70

8

1,00

26,80

1,30

2

PPROC_TN E

1523

36,20

7

0,80

26,30

8,70

12

PSIBW_MA C

342

157,20

7

6,00

120,70

29,70

603

6.1 Frontend Distribution

# Queries

BEX 1080

© SAP AG

WEB 202

ODBO 803

E2E100

API 38

37

8-7

The table above provides an overview of the front-end distribution. It contains the total number of queries executed over the last complete week (Monday to Sunday) and the number of queries executed from the different front ends.

6.2 Query Profile Check 6.2.1 Queries The table below gives an overview of the query runtimes in regard to the use of the OLAP Cache. This means the line 'Cache Queries' summarizes all the queries which used the OLAP Cache, whereas 'DB Queries' directly accessed the database. Please note that all queries using the 'Read API' (Queries named 'RSDRI_QUERY') cannot use the OLAP Cache. Therefore these queries are listed separately. If there were no queries in the last seven days which used the OLAP Cache, accessed the database or used the 'Read API', the corresponding summary line is not displayed.

Task Type

Nav. Steps

Runtime > 20 seconds [%]

Avg. Runtime [s]

Avg. Time OLAPINIT [s]

Avg. Time OLAP [s]

Avg. Time DB [s]

All Queries

89121

13

16,39

0,22

0,64

12,09

Cache Queries

38189

1

2,44

0,18

0,33

0,00

DB Queries

36433

30

29,57

0,27

1,22

21,69

RSDRI Queries

14499

3

20,02

0,20

0,00

19,82

The following table provides a summary of query runtimes and distinguishes between the different front ends. (BEX, WEB, ODBO and API). If no queries were started with one of the given options over the last seven days, the corresponding summary line is not displayed.

Task Type

Nav. Steps

Runtime > 20 seconds [%]

Avg. Runtime [s]

Avg. Time OLAPINIT [s]

Avg. Time OLAP [s]

Avg. Time DB [s]

All Queries

89121

13

16,39

0,22

0,64

12,09

WEB Queries

58103

16

14,71

0,22

0,69

11,12

BEX Queries

4899

35

51,67

0,75

3,26

22,70

RSDRI Queries

14499

3

20,02

0,20

0,00

19,82

ODBO Queries

11620

5

5,39

0,00

0,11

2,84

6.2.2 Top Time Queries by Total Workload The total workload caused by queries is defined as the sum of the total runtimes of all queries. The following query profile lists the queries, as a percentage of total runtime, that contribute the greatest amount to the total workload.

Query name

InfoCube

Total ZROLF_PAC_1000_2006 © SAP AG

PROLFOS2

Avg. Avg. Avg. Avg. Avg. # Nav. Runtime OLAP runtime DB time Frontend Frontend Steps [%] time [s] [s] time [s] cells [s] 20100

100

25,53

21,99

0,53

2,83

88

6769

25

19,28

15,99

0,35

2,81

20

E2E100

8-8

Avg. Avg. Avg. Avg. Avg. # Nav. Runtime OLAP runtime DB time Frontend Frontend Steps [%] time [s] [s] time [s] cells [s]

Query name

InfoCube

RSDRI_QUERY

PCRMSPIO P

73

ZROLF_PAC_1001_2006

PROLFOS2

ZROLF_PAC_6000_2006

19 1319,79 1319,79

0,00

0,00

0

3757

10

14,02

9,85

0,42

3,63

45

PROLFOS2

3347

9

13,92

11,37

0,49

1,95

2

ZCOPA_IBU10_2006

PCOPA_MU L

900

8

43,16

40,50

1,57

0,91

3

RSDRI_QUERY

0PCA_C01

15

6 2110,96 2110,96

0,00

0,00

0

ZROLF_PAC_4006_2007

PROLFOS2

1827

6

17,22

14,50

0,41

2,17

0

ZBUDG_COPA_001_05_PC

PBUDGCL3 4

3150

6

9,68

6,23

0,52

2,48

439

PSIBW_CE_PL_DPC_MA_ PSIBW_MA 02 C

256

6

112,74

87,46

4,74

20,11

314

PBUDGCL1 5

6

5 4286,40 4286,40

0,00

0,00

0

RSDRI_QUERY

6.2.3 Top Time Queries by DB Load The total database workload generated by the BW system is the sum of the total database access times of all queries. The following query profile lists the queries, as percentages of total database access time, that make up the largest part of the database load.

Query name

InfoCube

Total

Navigations

Db load in [%]

Avg. DB time [s] Avg. Runtime [s]

20100

100

21,99

25,53

6769

24

15,99

19,28

73

22

1319,79

1319,79

ZROLF_PAC_1000_2006

PROLFOS2

RSDRI_QUERY

PCRMSPIOP

ZROLF_PAC_6000_2006

PROLFOS2

3347

9

11,37

13,92

ZROLF_PAC_1001_2006

PROLFOS2

3757

8

9,85

14,02

ZCOPA_IBU10_2006

PCOPA_MU L

900

8

40,50

43,16

RSDRI_QUERY

0PCA_C01

15

7

2110,96

2110,96

ZROLF_PAC_4006_2007

PROLFOS2

1827

6

14,50

17,22

RSDRI_QUERY

PBUDGCL1 5

6

6

4286,40

4286,40

PSIBW_CE_PL_DPC_MA_0 PSIBW_MA 2 C

256

5

87,46

112,74

PBUDGCL3 4

3150

4

6,23

9,68


© SAP AG

E2E100

8-9

6.2.4 Top Time Queries by Average Runtime The ten queries whose average runtimes have the highest optimization potential are listed here.

Query name

Avg. Avg. Avg. Avg. DB Runtime OLAP Frontend time [s] [s] time [s] time [s]

InfoCube

Total

Avg. # Frontend cells

2546,86 2518,24

2,44

21,42

12058

RSDRI_QUERY

TSERVINC2

5790,21 5790,21

0,00

0,00

0

RSDRI_QUERY

PBUDGCL1 5

4286,39 4286,40

0,00

0,00

0

RSDRI_QUERY

PPAPAC02

3085,76 3085,76

0,00

0,00

0

RSDRI_QUERY

PPAPAC01

3013,31 3013,32

0,00

0,00

0

RSDRI_QUERY

PMKISMA0 2

2948,99 2948,99

0,00

0,00

0

RSDRI_QUERY

PCRMSMB WI

2527,64 2527,64

0,00

0,00

0

RSDRI_QUERY

0PCA_C01

2110,96 2110,96

0,00

0,00

0

RSDRI_QUERY

PPROC_CH K

1735,73 1735,73

0,00

0,00

0

RSDRI_QUERY

PPSPA_C02

1671,02 1671,03

0,00

0,00

0

124,45

1092,55

614957

ZCATS_GLOBAL_TOOL_ACTUAL_D PCATS_06 AYS

1517,60

57,97

6.3 BEX - Queries by Total Workload (TOP 10 per Runtime%) The table below contains data about the 10 queries that consumed the most time, with regard to runtime. Note that this table contains data about single executions of your queries. This means the data is not summarized, and the name of a query may appear several times.

Query name

InfoCube

Total

Avg. Avg. Nav. Runtim DB runtim time Steps e [%] e [s] [s]

Avg. OLA P time [s]

Avg. Avg. # Fronten Fronten d time d cells [s]

1061

100 113,30 47,72

3,79

60,84

5359

PSIBW_CE_PL_DPC_MA_02

PSIBW_MA C

175

19 133,84 98,32

6,64

28,38

460

PPROC_TNE_TPEXEC

PPROC_TN E

229

13

1,25

45,00

17

ZPOOL_SIBW_CE_PL_DPC_MA_ PSIBW_MA 02 C

12

13 1288,65 918,87 59,67

308,63

2125

YCRMS_GLSTAND_SO_01

PCRMSPIO P

261

10

ZROLF_PAC_1000_2006

PROLFOS2

120

9

ZROLF_PAC_1001_2006

PROLFOS2

56

© SAP AG

E2E100

69,27 22,51

46,71

4,68

2,02

38,92

6446

93,86 30,64

1,14

61,55

1131

9 185,39 76,27

1,49

106,77

3043

8-10

Avg. OLA P time [s]

Avg. Avg. DB Nav. Runtim runtim Steps e [%] time e [s] [s]

Query name

InfoCube

YCRMS_GLSTAND_SO_03

PCRMSPIO P

25

8 370,77

ZCONS11Q6_2006

PCONS11

52

7 153,00 64,55

ZROLF_PAC_4001_2006

PROLFOS2

97

6

ZROLF_PAC_4000_2006

PROLFOS2

34

Avg. Avg. # Fronten Fronten d time d cells [s]

7,04 18,46

338,81

92735

3,03

82,94

20626

77,77 19,62

3,39

54,39

196

6 201,61 77,51

4,85

117,60

5214

6.4 WEB - Queries 6.4.1 WEB Templates (TOP 10 per Runtime) The following table lists the top 10 templates that contribute the greatest amount to the total workload.

Template Name (Page ID)

Avg. # # runtime Runtime Navigation Executions for one Steps [s] [Sessions] NavStep [Queries] [s]

Max. runtime for one NavStep [s]

Avg. Template prepare time [s]

Avg. Template render time [s]

Total

373530,6 1

23032

23063

16,20 3827,44

1,58

0,22

ZROLF_PAC_1000_2006

118636,9 2

6621

6632

17,89 3827,44

1,62

0,11

ZROLF_PAC_6000_2006

45464,68

3316

3323

13,68

308,34

1,68

0,08

ZROLF_PAC_1001_2006

42255,41

3699

3700

11,42

315,00

1,98

0,09

ZCOPA_IBU10T_2006

38535,91

894

896

43,01 1138,66

0,63

0,19

ZROLF_PAC_4006_2007

31460,64

1827

1827

17,22

212,12

2,08

0,09

ZBUDG_COPA_001T_05_PC

24524,71

2778

2781

8,82

430,29

1,25

0,38

YCRMS_GLSTAND_SO_RA W_T

19486,69

1655

1660

11,74

476,88

1,18

1,15

PCAM2T_0102_2004

18243,46

200

200

91,22

893,43

1,73

0,05

ZCOPA_IBU1T_2006

17677,89

871

872

20,27

922,16

1,11

0,23

ZROLF_PAC_4000_2006

17244,30

1171

1172

14,71

162,96

1,45

0,11

6.4.2 WEB - Queries by Total Workload (TOP 10 per Runtime%) The table below contains data about the 10 queries that consumed the most time, with regard to runtime. Note that this table contains data about single executions of your queries. This means the data is not summarized, and the name of a query may appear several times.

© SAP AG

E2E100

8-11

Query name

InfoCube

Total

Avg. Avg. Avg. Avg. OLA Templat DB Nav. Runtim runtim P e Steps e [%] time e [s] time prepare [s] [s] [s]

Avg. Templat e render [s]

2221 9

100

16,90 13,99

0,71

1,59

0,24

ZROLF_PAC_1000_2006

PROLFOS2

6649

32

17,94 15,73

0,34

1,64

0,12

ZROLF_PAC_6000_2006

PROLFOS2

3323

12

13,68 11,33

0,49

1,67

0,08

ZROLF_PAC_1001_2006

PROLFOS2

3701

11

11,43 8,84

0,40

1,98

0,09

ZCOPA_IBU10_2006

PCOPA_MU L

896

10

43,01 40,44

1,57

0,63

0,19

ZROLF_PAC_4006_2007

PROLFOS2

1827

8

17,22 14,50

0,41

2,08

0,09


PBUDGCL34 2836

7

8,90 6,37

0,45

1,26

0,41

YCRMS_GLSTAND_SO_RAW

PCRMSPIOP

1655

5

11,72 3,33

3,70

1,18

1,15

ZPCAM2_0102_2004

PPCAMMOS 2

200

5

91,22 88,65

0,66

1,73

0,05

PPROCIMW_WORKLOAD_HISTO PPROCIMW RY

260

5

68,65 64,44

1,18

1,36

1,56

PCOPA_MU L

872

5

20,27 17,37

0,50

1,11

0,23

ZCOPA_IBU1_2006

6.4.3 WEB Templates - Multiple Query/View usage TOP 10: NUMBER OF DIFFERENT DATA PROVIDERS PER QUERY/VIEW

Template

Query / View

# of Data Providers

PPROC_RU_EMAIL_LEAN_3

PPROC_CPI_REPORT_CUST_DELIV_H

7

ZROLF_APA_LOB_SALES_2006

ZROLF_APA_LOB_2005_2006

6

CH_MOCP_06

ZPOOL_CRMSCH_BP_01

3

PCCI

ZBUDG000DV_ABC03_04_2

3

PPAPA_HC_COCD_COMPARISON_T0 PPAPA_HC_COCD_COMPARISON 1

3

PPAPA_HC_CO_COMPARISON_T01

PPAPA_HC_CO_COMPARISON_V01

3

PPAPA_HC_ORG_COCD

PPAPA_HC_ORG_COCD_V01

3

PPROC_CPI_RU_NC_CUR_AN

PPROC_CPI_PROJECT_INFO_NUMBERS

3

YCRMSOSPS04

YCRMSOOR01

3

YCRMS_001

YCRMS_0102004_V

3

Several Web items can use the same data provider if there is no need for independent navigation in a template. Based on our experience, independent navigation is not required in most cases, and you can therefore avoid the multiple query execution by assigning a query or view to just one data provider, which is used by all web items. Recommendation:

© SAP AG

E2E100

8-12

Check for the templates above if you require independent navigation. The names of the data providers are contained in the table below. Note that this table contains only the templates with the highest number of data providers for each query or view, but we recommend that you check all of your templates. If possible, reduce the number of data providers in the Web Application Designer to avoid multiple queries being executed unnecessarily. Using BW 3.x, old data providers that are no longer used must be deleted manually from the HTML code in the Application Designer. As of the BI in SAP NetWeaver 2004, there is a delete function in the data provider's context menu. DATA PROVIDERS PER QUERY/VIEW

Template

Query / View

Data Provider



DATAPROVIDER_3



DATAPROVIDER_4



DATAPROVIDER_5



DATAPROVIDER_6



DATAPROVIDER_7



DATAPROVIDER_8



DATAPROVIDER_9



DATAPROVIDER_1



DATAPROVIDER_2



DATAPROVIDER_3



DATAPROVIDER_4



DATAPROVIDER_5



DATAPROVIDER_6

CH_MOCP_06

ZPOOL_CRMSCH_BP_01

DATAPROVIDER_1

CH_MOCP_06

ZPOOL_CRMSCH_BP_01

DATAPROVIDER_2

CH_MOCP_06

ZPOOL_CRMSCH_BP_01

DATAPROVIDER_3

6.4.4 WEB Templates - Design Statistics TEMPLATE COMPLEXITY REGARDING DIFFERENT QUERIES/VIEWS

# of Queries / Views

# of Web Templates

01

4359

02

570

03

163

04

181

05

40

06

65

07

23

08

10

09

20

© SAP AG

E2E100

8-13

# of Queries / Views

# of Web Templates

10

45

> 10

35

WEB TEMPLATES WITH HIGHEST NUMBER OF DIFFERENT DATA PROVIDER

Template

# of Data Providers # of Queries # of Views # of Web Items

ZSBRT_HR_0030

32

32

0

34

ZMKISSRL_TZ_DA_COPY

23

3

20

23

PPROC_RU_EMAIL_REP

22

22

0

34

ZMKISSRL_TZ_DA

22

3

19

22

ZMKISSRL_TZ_DA_PUBLIC

22

3

19

22

ZMKISSRL_TZ_DA_PUBLIC_ROLE

22

0

22

22

ZMKISSRL_TZ_DA_PUBLIC_ROLE_2

22

0

22

22

ZMKISSRL_TZ_DA_PUBLIC_ROLE_IW P

22

0

22

22

ZPOOL_SBRT_HR_0020

21

21

0

24

PPROC_RU_OV_REPORT_PA_HI

20

20

0

23

WEB TEMPLATES WITH HIGHEST NUMBER OF WEB ITEMS

Template

# of Web Items

# of Data Providers

# of Queries

# of Views

ZAGR_TAB_VIEW4

42

2

0

2

ZAGR_TAB_VIEW3

40

2

0

2

YCRM_OPP_78

37

1

0

1

YCRM_OPP_78B_1

36

1

0

1

YCRM_OPP_78A_1

36

1

0

1

YCRM_OPP_04A_2

35

1

0

1

YCRM_OPP_04A

35

1

0

1

YCRM_OPP_78A

35

1

0

1

YCRM_OPP_07

34

1

0

1

YCRM_OPP_04

34

1

0

1

6.4.5 WEB Templates - Unused Data Providers TOP10 NUMBER OF UNUSED DATA PROVIDERS PER TEMPLATE

Template

# unused Data Providers

ZMKISSRL_TZ_DA_PUBLIC_ROLE_2

22

ZMKISSRL_TZ_DA_PUBLIC_ROLE_IWP

22

PPROC_RU_RKT_REPORT_PA_HI

18

© SAP AG

E2E100

8-14

Template

# unused Data Providers

PPROC_RU_OV_REPORT_PA_HI

18

PPROC_RU_OV_REPORT_PA

17

PPROC_RU_REPORT_PA_ALL

7

PPROC_RU_MENDO_PILOT_REP

6

PPROC_RU_SUPP_REPORT_PA

6

PPROC_RU_SUPP_REPORT_PA_HI

6

PPROC_CPI_RU_REPORT_PA_ALL_IWP

5

TOTAL NUMBER OF UNUSED DATA PROVIDERS

Total number of unused Data Providers 456

Recommendation: Unused data providers consume OLAP initialization runtime and can cause transport problems. In particular, if you have several templates that contain unused data providers, we recommend that you delete them to obtain an optimized and clean template implementation. Using BW 3.x, unused data providers must be deleted manually from the HTML code in the Application Designer. As of the BI in SAP NetWeaver 2004, there is a delete function in the data provider's context menu.

6.5 ODBO - Queries by Total Workload (TOP 10 per Runtime%) The table below contains data about the 10 queries that consumed the most time, with regard to runtime. Note that this table contains data about single executions of your queries. This means the data is not summarized, and the name of a query may appear several times.

Query name

Avg. Avg. Avg. Avg. OLA ODB DB Navigatio Runtim runtim P O n steps e [%] time e [s] time Time [s] [s] [s]

InfoCube

Total

7831

100

Avg. Prepar e Axis [s]

7,22 3,92

0,14

0,07

0,92

YPSPA_C06_CD1

PPSPA_C0 6

223

32

80,71

16,7 5

1,15

2,19

31,21

ZBSC_MFG_CALL_RATE

TSERVINC 2

107

17

91,83

89,6 8

0,84

0,00

0,00

ZBSC_MFG_FI_001

ZBSCCOP A

4570

15

1,85 0,71

0,04

0,00

0,00

ZPCAM2_0902_2005

ZBSCSWR 1

403

10

11,4 1

0,81

0,00

0,00

ZBSC_MFG_FI_003

ZBSCCOP A

1685

8

2,56 1,38

0,03

0,01

0,00

ZBSC_OP_MFG_006

ZBSCCSS

419

4

5,99 4,75

0,00

0,00

0,00

ZBSC_MFG_FI_002

ZBSCREV

209

4

10,27 7,48

0,11

0,00

0,00

ZBSC_SCA_ORDER_ENTRY

ZBSCSCA

108

4

18,79 17,2

0,13

0,00

0,00

© SAP AG

E2E100

14,64

8-15

Query name

InfoCube

Avg. Avg. Avg. Avg. OLA ODB DB Navigatio Runtim runtim P O n steps e [%] time e [s] time Time [s] [s] [s]

Avg. Prepar e Axis [s]

4 TSERVML TI

106

3

ZCATS_GLOBAL_TOOL_ACTUAL_ PCATS_06 DAYS

1

3

ZBSC_MFG_CP_007

17,40

16,1 7

0,00

1517,6 57,9 124,4 0 7 5

0,00

0,00

0,00 225,59

6.6 API - Queries per InfoCube (TOP 10 per Runtime%) The table below contains data about the 10 queries that consumed the most time, with regard to runtime. Note that this table contains data about single executions of your queries. This means the data is not summarized, and the name of a query may appear several times.

Infoprovider # Executions Total

Runtime [%]

Avg. Runtime Avg. OLAPINIT [s] Time [s]

Avg. DB Time QDBTRAN [s] S

6958

100

35,74

0,15

PCRMSPIOP

73

39

1319,79

0,00

1319,79

7601745

0PCA_C01

15

13

2110,96

0,00

2110,96

51366659

PBUDGCL1 5

6

10

4286,40

0,00

4286,40

9088854

PMKISMA0 2

6

7

2948,99

0,00

2948,99

9086476

PCRMSMB WI

7

7

2527,64

0,00

2527,64

550484

PROLFFI01

3997

7

4,36

0,12

4,24

3419066

PPROC_CH K

10

7

1735,73

0,00

1735,73

14417538

PMKISMA0 1

443

5

25,55

0,34

25,21

8394222

PROLFHR01

2399

3

3,10

0,17

2,94

150351

2

2

3013,32

0,00

3013,32

2131079

PPAPAC01

35,59 106206474

6.7 Analysis of MultiProvider Queries During parallel processing of MultiProvider queries, subprocesses that process the partial accesses to the InfoProviders in the MultiProvider are separated from a parent process. The parent process provides a synchronization point where the overall result is collected. In contrast to sequential processing, which permits partial results to reach the OLAP processor, parallel processing requires that the overall result first be collected at the synchronization point. To avoid an overflow of the process memory, parallel processing is cancelled as soon as the collected result contains 30,000 rows or more and there is at least one subprocess not finished. The MultiProvider query is then restarted automatically and processed sequentially.

© SAP AG

E2E100

8-16

This strategy explains why sequential processing can appear to be faster than parallel processing in the case of queries like these. What appears to be parallel processing corresponds to sequential processing plus the preceding phase of parallel processing up to the termination. Recommendation: Please check whether parallel processing is deactivated. Otherwise, consider deactivating parallel processing for those queries that are MultiProvider queries and have large result sets. Furthermore, starting with BW 3.0B Support Package 14 (BW 3.1 Support Package 8), you can set parameters to define the size of the maximum permissible interim overall result, that is, you can change the default valid value of 30,000 rows. For further details, please refer to SAP Notes 622841, 607164, and 630500. Please note that it is generally recommended that the number of base InfoProviders of a MultiProvider is not more than 10. A larger number of base InfoProviders is likely to result in a scenario where there are a lot more base InfoProviders than there are available dialog processes, resulting in limited parallel processing and many pipelined subqueries. Details can be found in SAP Note 629541.

6.8 Upload of transactional data We could not detect any uploads of transactional data from 02.10.2006 to 09.10.2006. To display an overview of the runtime during an upload, we require statistical information from the different requests. Recommendation: Check whether the "BW statistics" for uploading data are activated. Implementation: Call the Adminstrator Workbench (Transaction: RSA1) and from the menu choose Tools -> BW Statistics for Infocubes. Activate the WHM indicator for all productive Infocubes.

6.8.1 Number Range Buffering for BW Objects For each characteristic and dimension, BW uses a number range to uniquely identify a value (SIDs and DIM-IDs). If the system creates a high amount of new IDs periodically, the performance of a data load may decrease. To avoid the high number of accesses to the NRIV table, activate the number range buffering for these BW objects (Main Memory Number Range Buffering). For more detailed information, see SAP Notes 504875, 141497, and 179224. The tables provide an overview of the number range buffering, setting of dimensions, and InfoObjects based on the highest number range level ("Level"). The purpose of this information is to identify the candidates for activating the number range main memory buffer. Recommendation: Activate number range buffering for all dimensions and InfoObjects, with a high number of rows, based on the rules mentioned in Note 857998. Note: A package dimension must not be buffered! TOP10 UNBUFFERED DIMENSIONS WITH HIGHEST NUMBER RANGE LEVEL

InfoCube

Dimension

103895

103895P

1 BID0129861

446000002

0

103890

103890P

1 BID0129861

446000002

0

103886

103886P

1 BID0129861

446000002

0

104152

104152P

1 BID0129861

446000002

0

103881

103881P

1 BID0129861

446000002

0

PSCA1BDT

PSCA1BDTP

25 BID0129861

446000002

0

103901

103901P

1 BID0129861

446000002

0

103882

103882P

1 BID0129861

446000002

0

103900

103900P

1 BID0129861

446000002

0

104149

104149P

1 BID0129861

446000002

0

© SAP AG

Rows

Object

E2E100

Level

Buffered

Buffer Size

8-17

TOP10 BUFFERED DIMENSIONS WITH HIGHEST NUMBER RANGE LEVEL

InfoCube

Dimension

Rows

Object

Level

Buffered

Buffer Size

0BWTC_C02

0BWTC_C021

141048 BID0034902

1881846329

3000000

PSCA1BDT

PSCA1BDT6

66720 BID0129868

426000328

1000000

PSCA1BDT

PSCA1BDT3

114577 BID0129865

409008165

1000000

PSCA1BDT

PSCA1BDT5

103654 BID0129867

385017154

1000000

PSCA1BDT

PSCA1BDT4

56926 BID0129866

361001781

1000000

PCONS05

PCONS053

223570 BID0037648

352243868

500000

PCONS05

PCONS051

89230 BID0037646

350706170

500000

PSCA1BDT

PSCA1BDT7

111097 BID0129869

346100000

1000000

PCONS05

PCONS052

23814 BID0037647

339095000

500000

PSCA1BDT

PSCA1BDTT

6405 BID0129862

236002033

1000000

TOP10 UNBUFFERED INFOOBJECTS WITH HIGHEST NUMBER RANGE LEVEL

InfoObject

SID Table

PBWTC_BPG

/BIC/SPBWTC_BPG

17781582 BIM0150176

242777835

0

PBWTC_TCS

/BIC/SPBWTC_TCS

18061731 BIM0150181

217418925

0

PCRMSTAOP

/BIC/SPCRMSTAOP

608480 BIM0115020

56791242

0

0CATSBELNR

/BI0/SCATSBELNR

56141500 BIM0023692

56159431

0

0TCTSTAUID

/BI0/STCTSTAUID

45236252 BIM0001162

45680791

0

PBWTC_TCG

/BIC/SPBWTC_TCG

3040892 BIM0150180

34606004

0

0TCTNAVUID

/BI0/STCTNAVUID

33076374 BIM0001173

33421491

0

TASKCOUNT

/BIC/STASKCOUNT

28802500 BIM0034605

31152891

0

PSIBWDNO

/BIC/SPSIBWDNO

19643250 BIM0070350

19755764

0

2381439 BIM0037891

18636768

0

0TEL_NUMBER /BI0/STEL_NUMBER

Rows

Object

Level

Buffered Buffer Size

No InfoObject is number range buffered.

6.9 Process Chains - Runtime Overview The Process Chain Runtime Analysis is based on the last 7 days before the download. The table contains statistical information of all chains which aren't started by another local process chain. This includes process chains which are started by the Service API Interface or remotely by a chain from another system. Please note that only the top 20 chains having the longest runtimes are displayed. The columns '# Total Subchains', '# Total Steps' represent the summarized values of the main chain and their subchains. The runtimes are the range from the start of the main chain up to the end of the last process type executed within the main chain and their subchains. This means the real runtime of the main chain and their subchains is displayed here.

© SAP AG

E2E100

8-18

Total Avg. Med. Avg. #Total #Total #Runs Runtime Runtime Runtime Proc.Type Subchains Steps [min] [min] [min] Runt. [min]

Main Chain COPA_AGGREGATE_CHECK

0

47

7

5927

847

789

6456

PCA_AGGR_CHECK_PPCA_H 01

0

12

8

4537

568

590

1487

YCRMATSAP_TRANSACT_D AILY

3

136

7

3355

480

552

1698

YCRMATSAP_MASTER_DAIL Y1

1

89

7

3238

463

250

764

ISM_1

0

15

8

2456

307

341

396

MARKETING_LOAD

0

56

6

2343

391

329

375

ZMKIS_PC_SURV_ACTIV

0

21

7

2152

308

328

334

ST03_UPLOAD

0

14

7

2100

300

327

299

PPAPA_META_DAILY

2

88

6

2095

350

418

417

PCA_AGGR_CHECK_0PCA_C0 1

0

18

8

2057

258

276

831

ZBUDG_TD_OPA_DAILY

0

10

6

1937

323

370

319

13

140

6

1851

309

187

312

ZBUDG_TD_CCA_DAILY

0

9

6

1652

276

320

271

YCRMAUTH

0

25

7

1577

226

243

224

CPR_TRANSACTION_DATA

2

65

6

1487

248

238

402

CPR_MASTER_DATA

1

35

6

1367

228

113

130

PBROLF

0

27

8

1363

171

186

275

PPAPA_MASTERDATA

0

34

6

1333

223

107

317

DAILY_BWTC_UPLOADS

0

6

6

1313

219

242

218

ZPE_XXXX_08

0

63

1

1197

1197

1197

1197

ZYPRM_PRMATSAP_START

The following table displays the statistical information based on the actual defined chain hierarchy. The columns '# Subchains' and '# Steps' are based on the subchain itself only. Only the top three main process chains with the highest number of executions are displayed. For each main process chains only the first ten subchains with the highest runtimes are shown.

Parent Chain

Total Avg. # # # Runti Runti Subchai Ste Ru me me ns ps ns [min] [min]

Lev Chain el

0

YCRMATSAP_TRANSA CT_DAILY

Avg. Med. Proc.T Runti ype me Runt. [min] [min]

3 114

7

3355

480

552

1698


1 YCRMATSAP_CPR_TD

0

7

7

944

135

130

134

YCRMATSAP_TRANSA

1 YCRMATSAP_SPLIT_D

0

6

7

130

19

21

18

© SAP AG

E2E100

8-19

Total Avg. # # # Runti Runti Subchai Ste Ru me me ns ps ns [min] [min]

Lev Chain el

Parent Chain

CT_DAILY

Avg. Med. Proc.T Runti ype me Runt. [min] [min]

ELTA


YCRMATSAP_MASTER _DAILY1

1 YCRMATSAP_SPLIT

0

9

7

41

6

5

6

0

YCRMATSAP_MASTER _DAILY1

1

58

7

3238

463

250

764

1

YCRMATSAP_BPARTN ER

0

31

7

2757

394

425

1622

2

3

6

2095

350

418

417

0 PPAPA_META_DAILY PPAPA_META_DAILY

1

PPAPA_TRANSACTION _B_DAILY

0

42

6

1697

283

417

374

PPAPA_META_DAILY

1

PPAPA_TRANSACTION _A

0

43

6

1381

231

204

222

6.10 Change-Run Analysis Total maintenance time of all change-runs within one week.

Week

# Change-Runs

Total Runtime [min]

40.2006

58

598

39.2006

67

684

38.2006

43

605

37.2006

55

334

36.2006

59

384

35.2006

59

463

34.2006

84

505

33.2006

93

486

32.2006

86

560

31.2006

72

740


© SAP AG

E2E100

8-20

Rating


7.1 Workload by Application Module The following graphics or tables show how each application module contributes to the total system workload. Two workload aspects are shown: - CPU time: Total CPU load on all servers in the system landscape - Database time: Total database load generated by the application The #Customer# section contains all programs and transactions that start with Y or Z. The #Others# section contains all programs that cannot be assigned to an SAP standard application module (for example, an industry solution or a third party add-on development). CPU LOAD BY APPLICATION

Module

CPU Time in %

BC

53,9

BW-WHM

33,8

BC-BW

8,6

Others

3,3

DB LOAD BY APPLICATION

Module

DB Time in %

BW-WHM

52,1

BC

41,5

Others

3,1

BC-BW

2,5

7.2 DB Load Profile The following table or diagram shows the DB load caused by Dialog, RFC, and Background tasks over different time frames. The data given in the table represents the average number of database processes occupied by each task type in the database during the given time frames. These statistics are calculated as a weekly average, which means the average values over six working days with a unit of one hour. Periods between 00:00-06:00 and 21:00-24:00 contain an average value per hour. By comparing the load profiles for dialog and background activity, you have an overview of the amount of background activity during online working hours. DB LOAD PROFILE FOR DIALOG, RFC AND BATCH

Start_End Time

DB Load (DIA)

DB Load (RFC)

DB Load (BTC)

00--06

0,18

8,32

11,20

06--07

0,24

3,26

20,66

07--08

0,34

3,53

16,75

© SAP AG

E2E100

8-21

Start_End Time

DB Load (DIA)

DB Load (RFC)

DB Load (BTC)

08--09

0,56

4,05

19,50

09--10

0,54

9,87

14,72

10--11

0,67

8,62

11,23

11--12

0,33

6,73

11,59

12--13

0,29

5,75

12,76

13--14

0,37

4,97

7,71

14--15

0,37

5,34

10,26

15--16

0,37

5,68

2,51

16--17

0,36

6,72

7,74

17--18

0,77

4,14

3,93

18--19

0,24

3,48

6,48

19--20

0,13

4,94

5,08

20--21

0,18

4,58

2,55

21--24

0,17

10,20

2,88


Rating



8.2 Table Reorganization When analyzing your database, we found large or rapidly growing tables or indexes.

© SAP AG

E2E100

8-22

Recommendation: Implement the SAP Notes listed below to reduce the size of some of these tables or indexes. Background: More information about Data Volume Management is available on the SAP Service Marketplace at service.sap.com/dvm

Table / Index Name


Recommended SAP Note

BALDAT

8168,0 195157

EDI40

42496,0 40088


Date


78

03.10.2006

69

04.10.2006

99

05.10.2006

78

06.10.2006

85

07.10.2006

31

08.10.2006

55

DETECTED ABAP DUMPS

Error OBJECTS_TABLES_NOT_COMPATIBLE CALL_FUNCTION_NOT_REMOTE

Numbe Server (e.g.) User (e.g.) r

Date (e.g.) Time (e.g.)

3 server4

D034752

02.10.2006

15:46:35

30 server3

D046722

03.10.2006

11:18:16

TABLE_HASH_NO_MEMORY

2 server17

C5082443

04.10.2006

06:13:48

ASSIGN_LENGTH_0

1 server3

D032907

04.10.2006

17:52:35

TSV_TNEW_OCCURS_NO_ROLL_MEMOR Y

3 server15

I030875

05.10.2006

09:29:18

PARAMETER CONVERSION ERROR

1 server3

D020319

05.10.2006

09:43:15

TSV_TNEW_BLOCKS_NO_ROLL_MEMOR Y

2 server14

D019118

05.10.2006

10:39:46

MESSAGE_TYPE_X_TEXT

7 server17

D043794

05.10.2006

11:00:28

TYPELOAD_LOST

3 server16

C5064668

05.10.2006

13:09:43

LOAD_PROGRAM_NOT_FOUND

2 server3

D028520

05.10.2006

15:37:23

TSV_TABH_POOL_NO_ROLL_MEMORY

2 server14

I005017

05.10.2006

16:55:30

76 server11

D028685

05.10.2006

17:22:57

RFC_NO_AUTHORITY © SAP AG

E2E100

8-23

Numbe Server (e.g.) User (e.g.) r

Error

Date (e.g.) Time (e.g.)

DBIF_RSQL_TABLE_UNKNOWN

2 server8

C5068423

06.10.2006

05:15:03

DYNPRO_NOT_FOUND

1 server3

I012934

06.10.2006

09:19:03

TEXTENV_KEY_INVALID

9 server4

-:b

06.10.2006

09:24:09

CALL_FUNCTION_NO_RECEIVER

1 server4

C5067472

06.10.2006

09:24:14

CALL_FUNCTION_OPEN_ERROR

2 server4

I037366

06.10.2006

11:08:38

UC4_ADMIN

06.10.2006

11:46:07

SYSTEM_CANCELED

11 server11

SYSTEM_CORE_DUMPED

5 server11

D046423

06.10.2006

11:50:46

RFCGUI_UNEXPECTED_DATA

2 server11

D031185

06.10.2006

11:50:58

CALL_FUNCTION_SEND_ERROR

9 server3

D046367

06.10.2006

12:28:49

DYNPRO_SEND_IN_BACKGROUND

10 server11

SAPSYS

06.10.2006

12:31:28

IMPORT_ALIGNMENT_MISMATCH

16 server5

SAPSYS

06.10.2006

12:51:27

1 server15

I035063

06.10.2006

15:11:29

32 server14

I039699

07.10.2006

10:11:21

2 server16

I009883

07.10.2006

19:36:34

SYSTEM_NO_MEMORY OBJECTS_OBJREF_NOT_ASSIGNED GETWA_NOT_ASSIGNED UNCAUGHT_EXCEPTION

59 server3

I800861

08.10.2006

01:48:24

SAPSQL_INVALID_TABLENAME

12 server4

C5064658

08.10.2006

02:06:13

TSV_TNEW_PAGE_ALLOC_FAILED

71 server3

C5082444

08.10.2006

17:30:58

C5064658

08.10.2006

22:14:38

REMOTE_PRP 08.10.2006

22:44:50

TYPELOAD_NEW_VERSION

7 server20

MESSAGE_TYPE_X

61 server19

9 Hardware Capacity We have checked your system for potential CPU or memory bottlenecks and found that the hardware of your database and application servers is sufficient for the current workload.

Server

Max. CPU load [%]

Date

Rating

RAM [MB]

Max. Paging [% of RAM]

Date

server2

7 05.10.2006

3711

3 02.10.2006

server3

12 06.10.2006

3711

12 05.10.2006

server4

16 05.10.2006

3711

4 05.10.2006

server5

10 08.10.2006

3711

9 05.10.2006

server6

13 08.10.2006

3711

0 03.10.2006

© SAP AG

E2E100

Rating

8-24

Server

Max. CPU load [%]

Date

Rating

RAM [MB]


Date

server7

8 05.10.2006

3711

1 04.10.2006

server8

12 05.10.2006

3711

0 05.10.2006

server9

19 05.10.2006

3711

1 02.10.2006

server10

6 05.10.2006

3711

0 02.10.2006

server11

15 05.10.2006

3711

28 06.10.2006

server12

0 02.10.2006

3711

0 02.10.2006

server13

0 02.10.2006

3711

0 08.10.2006

server14

55 02.10.2006

8023

0 06.10.2006

server16

15 03.10.2006

12119

0 06.10.2006

server17

6 06.10.2006

0

0 06.10.2006

server20

10 05.10.2006

12119

0 02.10.2006

Rating

10 Database Settings We have detected major problems with the settings of your database system. These settings can potentially lead to performance problems.

Rating

Check Database Key Performance Indicators Locally managed Temp Tablespace Database Parameters Optimizer Statistics


© SAP AG

E2E100

8-25

10.1.1 User name

Executions

Disk reads

Buffer gets

Rows processed

TOTAL (not a user name!)

602070129

101457229

5765653588

6768815579

SYS

1%

2%

0%

0%

SYSTEM

0%

0%

0%

0%

SAPR3

99%

98%

100%

100%

OPS$IWPADM

0%

0%

0%

0%

10.2 Database Parameters We noticed that the following database parameters, which are currently active in your system, do not follow our standard recommendations. These standard recommendations cannot take into account possible special aspects for your system. Deviations from those standard recommendations may be necessary in your case. We therefore advise you to discuss theses parameter settings in your next Service session. NOTE FOR ORACLE PARAMETER SETTINGS

Note number

Description

332696

ORA-00600 [2865], star_transformation_enabled

632556

Oracle 9.2.0.* database parameter settings for BW

Parameters

Description

Current value

_optimizer_or_expansio control or expansion approach n used filesystemio_options

Filesystem I/O options

Recommended value Please check SAP note 0124361 and note 358921

asynch

Please check SAP note 0124361 and note 358921

Parameters

Description

Current value

Recommended value

max_dump_file_size

Maximum size of the optimizer trace file

unlimited


open_cursors

Max. Nr. of concurrently open cursors per database session

10000


shared_pool_reserved_size

Part of Shared Pool for large statements

536870912


workarea_size_policy

Controls the mode in which working areas are tuned

MANUAL


To provide special functionality or control system behavior, Oracle has a number of events. These can be activated in the configuration file and/or dynamically in the server manager or sqlplus. In general, these events and parameters should only be activated on request. When they are no longer needed, they should be removed. However, there might be the need to activate some events during normal database

© SAP AG

E2E100

8-26

operation. The following table displays the values for the parameter "event" as found in your system:

Parameter

Current Value

event

10171 trace name context forever, level 1 10191 trace name context forever, level 1 38019 trace name context forever, level 1


Event


Description

10027

596420


10028

596420


38068

176754

Problems with CBO and RBO

38069

904184

ORA-600 [expcmo_strdef1] with star transformation

The following table lists events we found to be activated in your system, but which should only be activated in special circumstances. Please check whether there are special reasons to set these events. If they are no longer needed, they should be removed.

Event 10171 38019

10.3 Optimizer Statistics The Oracle Cost-Based Optimizer requires current information about the distribution of the database data in order to find the best search path. If this information is not up-to-date, or is not available at all, the Cost-Based Optimizer may choose the wrong query path, which can result in severe performance problems. The "data distribution" database statistics are not updated in accordance with the SAP strategy for generating optimizer statistics with the 1-phase strategy using BRCONNECT. For optimal performance, perform a statistical update with BRCONNECT regularly. We analyzed the log file entries for the optimizer runs executed before the 09.10.2006, and noticed the following problems: There are no successful brconnect (sapdba) runs related to the recommended 1-phase strategy. Recommendation: Schedule the "Update Statistics" task to run weekly. For detailed information about the SAP strategy for generating optimizer statistics in parallel, including updating the data distribution statistics of your database, see "Update Statistics for the Oracle Cost-Based Optimizer" at service.sap.com/dbaora -> Oracle Database Administration -> Media Library -> Performance. Note: If your users or developers are authorized to create their own tables or indexes, we recommend that you schedule the creation of statistics for tables with missing statistics to run each day. Otherwise, since Oracle does not create statistics for newly created objects, these objects will remain without valid statistics until the next update run for the optimizer statistics.

© SAP AG

E2E100

8-27

Note number

Description

132861

CBO: Weekly scheduling with SAPDBA

184513

CBO: Parallel generation of optimizer statistics

93098

Changes to the upgrade to 4.0 - CBO Oracle

93256

CBO: Changes for installation of 4.0

535986

MONITORING for BW fact tables under Oracle

403704

Enhanced BRCONNECT 6.10 functions


In the checks performed, problems regarding the administration of your database were found.

Rating

Check Database Growth Backup Frequency Archive Frequency Freespace in Tablespaces Space Critical Objects brconnect -f check (sapdba -check) schedule


Date

© SAP AG


1014,71

30.11.2003

847,67

17.12.2003

704,35

23.08.2004

1112,25

31.08.2005

2439,16

19.09.2005

2583,77

30.11.2005

2134,51

16.12.2005

2279,15

E2E100

8-28

Date


2624,47

31.08.2006

2086,66

30.09.2006

2257,97

06.10.2006

2326,34


Date


314,97

30.11.2003

504,00

17.12.2003

655,32

23.08.2004

829,76

31.08.2005

605,34

19.09.2005

668,67

30.11.2005

813,01

16.12.2005

820,33

18.01.2006

666,97

31.08.2006

913,84

30.09.2006

900,75

06.10.2006

894,36


Date

© SAP AG

Current Size in GB


30.11.2003

847,67

-167,04

17.12.2003

704,35

-143,32

23.08.2004

1112,25

407,90

31.08.2005

2439,16

1326,91

19.09.2005

2583,77

144,61

30.11.2005

2134,51

-449,26

16.12.2005

2279,15

144,64

18.01.2006

2624,47

345,32

31.08.2006

2086,66

-537,81

30.09.2006

2257,97

171,31

06.10.2006

2326,34

E2E100

8-29

Your database is growing at a rate of more than 30 GB of data per month. Recommendation: Analyze the top growing tables and check that the table growth is really necessary for your business process. Set up a archiving project that allows you to archive data periodically to keep the database at a manageable size. Note: Data archiving is particularly important before SAP Basis Release upgrades, conversion, or migration projects.

11.2 Backup Frequency When we checked the backup log files, we detected that your backup strategy does not follow the SAP backup recommendations. In the time period from 12.09.2006 to 09.10.2006 , we noticed the following problems: - There was no successful backup on Thursday 05.10.2006 - There was no successful backup on Wednesday 04.10.2006 - There was no successful backup on Tuesday 03.10.2006 - There was no successful backup on Monday 02.10.2006 There are 4 working days without successful backup this week. - There was no successful backup on Wednesday 27.09.2006 - There was no successful backup on Tuesday 26.09.2006 - There was no successful backup on Monday 25.09.2006 There are 3 working days without successful backup this week. - There was no successful backup on Friday 22.09.2006 - There was no successful backup on Wednesday 20.09.2006 - There was no successful backup on Tuesday 19.09.2006 There are 3 working days without successful backup this week. - There was no successful backup on Friday 15.09.2006 - There was no successful backup on Tuesday 12.09.2006 There are 2 working days without successful backup this week. Recommendation: Back up the entire database after each workday. If the available backup window is too short for backing up the entire database, back up part of the database on a daily basis. The duration of a potential recovery will be shorter the more you backup your database. Additionally, we recommend a backup cycle of at least four weeks and a verification of a backup at least once in the backup cycle.

11.3 Check brconnect -f check (sapdba -check) schedule When we checked the brconnect -f check schedule, we detected that it is not scheduled as recommended In the analysis ending the 09.10.2006, we noticed the following problems: Brconnect -f check is not scheduled at all. BRCONNECT with the option -f check (or SAPDBA with the option -check) offers you an easy way to check for database problems specific to the SAP environment. Recommendation: Schedule BRCONNECT -f check (or SAPDBA -check) to run daily during periods of low system activity, using either the DBA Planning Calendar (transaction DB13) or by entering command BRCONNECT -f check (or sapdba -check) at the command prompt. After each run, check the log information written by the BRCONNECT -f check (SAPDBA -check). Check SAP Note 403704 for information about SAPDBA and BRCONNECT.

11.4 Oracle Release In general, we recommend that you install the most current Oracle patch set as soon as possible (see SAP Note 415900). Therefore, check SAP Note 539921 from time to time for the latest patch set for Oracle 9.2.0. Note that in addition to the patch set, further bug fixes might be available and recommended. See the following SAP Notes: For patch set 9.2.0.3 - 610445: Merge fix for Oracle 9.2.0.3 For patch set 9.2.0.4 - 695080: Merge fix for Oracle 9.2.0.4 For patch set 9.2.0.5 - 755629: KKO merge fix for Oracle 9.2.0.5

© SAP AG

E2E100

8-30

For patch set 9.2.0.6 - 834100: Merge fix for Oracle 9.2.0.6 For patch set 9.2.0.7 - 896903: Optimizer Merge fix for Oracle 9.2.0.7

11.5 Desupport of multibyte character sets as of Oracle 10g An important SAP Note is available concerning systems that run with an Oracle database and that make use of setting the database code page to one of the following multibyte character sets: JA16SJIS Japanese KO16KSC5601 Korean ZHS16CGB231280 Simplified Chinese ZHT16BIG5 Traditional Chinese This SAP Note is: Note 858869: Desupport of multibyte character sets as of Oracle 10g. (see also related note 695899) The database settings as shown above have been set by many Asian customers to allow third-party tools to connect to their databases. With Oracle 10 clients or kernel release 7.00, the settings above are no longer supported and a Unicode conversion of the systems prior to an upgrade becomes mandatory for these systems.


Note number

Description

858869


871455


12 Business Information Warehouse Severe problems were detected, that may have serious impact on your system's performance and stability. You should take corrective action immediately.

Rating

Check BW Application Checks Data Distribution Analysis of Infoproviders Analysis of Aggregates Source System Settings Recommendations for BW

© SAP AG

E2E100

8-31

Rating

Check Frontend Servers

12.1 BW Application Checks for IWP The BW application was analyzed with respect to design and data distribution. We did not detect any major problems that could affect the performance of your system.

12.1.1 Analysis of OLAP Cache The OLAP Cache is used for duplicated storing of often used data, whereby this data can be accessed quickly. In the tables below, you can find information regarding the size and the usage of the OLAP Cache.

Cache usage of queries Defined Queries The OLAP Cache can buffer results from queries and can provide them again for different users and similar queries (that is, same queries or real subsets of them). The OLAP Cache reduces the workload of the DB server. The OLAP Cache can store the query results with their navigation statuses in the memory of the application server; the data can also be stored in database tables and files. When the buffer (Export/Import Shared Memory) memory overruns, it stores the displaced data - depending on the persistence mode - on the database server. The following OLAP Cache modes exist: 0 Cache is Inactive 1 Main Memory Cache Without Swapping 2 Main Memory Cache With Swapping 3 Cluster / Flat File Cache for Each Application Server 4 Cluster / Flat File Cache Across Each Application Server What is the best cache mode? This greatly depends on the requirements. The fastest cache mode is always the usage of the main memory. However, if several application servers are in use, the cache has to be filled for each query on each application server. In this case, it might be better to use the cross-application-server cache mode (flat file/cluster table). An alternative might be to use logon groups and to ensure that users executing the same/similar queries work on the same application server. MODE 0 - Cache is Inactive All data is read from the relevant InfoProvider. MODE 1 - Main Memory Cache without Swapping New data is stored in the export/import SHM buffer until this memory area is full. If then new data has to be added to the buffer, an LRU mechanism is applied. Least recently used data is DISPLACED from the buffer and lost. If this data is needed again by a query, it must access the relevant InfoProvider on the DB server. MODE 2 - Main Memory Cache with Swapping This works in a similar fashion to MODE 1. HOWEVER in this scenario if the memory is full, data is not DISPLACED and lost but written to a cluster table/flat file. Then if this data is needed again by a query, it can be read from this file, which is still quicker than reading this from the relevant InfoProvider on the DB server. MODE 3 - Cluster/Flat File Cache for Each Application Server The cache data is kept persistently in a cluster table or in flat files for each application server. The overall data quantity is only restricted by the database or file system. Swapping does not occur in the same way as with the cache main memory mode. MODE 4 - Cluster/Flat File Cache Application Server-Extensive This mode is the same as the mode described above (cluster/flat file for each application server), the only difference being that the cache entries of all of the application servers in a system are used together. NOTE: FOR MODE 1 and 2, remember that MEMORY is INSTANCE DEPENDENT.

© SAP AG

E2E100

8-32

NUMBER OF QUERIES PER CACHE MODE

Cache Mode

# Queries

Total

10359

OFF

44

1

9924

2

0

3

0

4

391

NUMBER OF INFOCUBES PER CACHE MODE

Cache Mode

# Infocubes

Total

871

OFF

1

1

870

2

0

3

0

4

0

Executed Queries

Task type All Queries

Nav. Steps 89121

Accessed DB [%]

Accessed Cache [%] 41

RSDRI Queries [%] 43

16

There are two types of caches: The local cache and the transactional cache (OLAP Cache). The local cache belongs to a query session and therefore cannot be used by other sessions. The OLAP Cache can store query data on the application server and can have a swap file or use a swap cluster table.

© SAP AG

E2E100

8-33

The OLAP Memory Cache is located in the Export/Import buffer SHM (parameter rsdb/esm/buffersize_kb). As the global cache size is a logical value and the Export/Import SHM gives a physical limit, we recommend that you do not set the global cache parameter to a higher value than the Export/Import SHM buffer. Note: The OLAP Cache was optimized in SAP BW 3.0B SP19, SAP BW 3.1C SP13, and SAP BW 3.5 SP02. For more information, see SAP Note 683194.

Rating

Description

Current Value

Cache active

Active

Recommendation

Cache Persistence Mode Flat File Name Comprehensive Flat File Name for AppServer Local Cache Size (MB)

50

Global Cache Size (MB)

100

12.2 Data Distribution Please note that we could not perform a complete download in order to avoid generating a short dump. Therefore, we were not able to collect complete data regarding the object sizes of the object types mentioned in the following table. Please note that for these object types, larger objects may exist than the ones mentioned in the following chapters.

missing objects InfoCubes Aggregates Master data SID-tables Master data X-tables Master data Y-tables Hierarchy tables (I-tables) Validity tables (L-tables)

12.2.1 Largest ODS tables ODS Name

ODS Table name

PSIBWO4A

/BIC/APSIBWO4A00

38914506

PCATS_09

/BIC/APCATS_0900

31218358

PPROCIMW

/BIC/APPROCIMW00

30445835

PSIBWOD4

/BIC/APSIBWOD400

23880612

PSCA0024

/BIC/APSCA002400

20508677

PCATS_07

/BIC/APCATS_0700

18349213

PCONSSID

/BIC/APCONSSID00

15782481

PSIBWOD1

/BIC/APSIBWOD100

14651792

© SAP AG

No of records

E2E100

8-34

ODS Name

ODS Table name

No of records

PSIBWODG

/BIC/APSIBWODG00

14508845

PDMPAR02

/BIC/APDMPAR0200

12459098

Large ODS Objects can have a negative impact on reporting and upload performance. Please pay attention to more detailed recommendations in subsequent sections of this report. With BW release 3.x parallel ODS Data Activation can fail on large ODS Objects with a deadlock on the Oracle database. Recommendation: Please follow the implementation description in SAP Note 634458 "ODS object: Activation fails - DEADLOCK" and 84348 "Oracle deadlocks, ORA-00060".

12.2.2 Largest InfoCubes InfoCube Name

No of records

PPROC_LOC

308368617

PBWTC_ST3

185514631

PPROCST0N

164428431

PPSPA_Y02

83291016

PPROC_TNE

75501680

PPAPASK01

70516570

PPROCSCOV

68678106

PPCA_H01

50115656

PCATS_01

48796643

PPROCCHKO

40541763

Large InfoCubes require high effort in administration and have a long runtime during maintenance activities. Recommendation: Please consider splitting a large InfoCube into small InfoCubes (such as separated by a time characteristic like 0CALYEAR). You can use MultiProviders as query targets to obtain all data in your query result.

12.2.3 Largest Aggregates Infocube

© SAP AG

Aggregate Name

No of records

102872

22001977

105480

19890898

106610

16109419

103234

14245391

102569

13489047

103041

12313697

103634

11575470

101952

11407240

E2E100

8-35

Infocube

Aggregate Name

No of records

103767

10678500

106061

9855240

Large aggregates need high runtime for maintenance like change runs and rollup of new data. Recommendation: Please check section: "Analysis of Aggregates" for further hints.

12.3 Analysis of InfoProviders 12.3.1 InfoProvider Distribution The following section provides an overview of the distribution of your InfoProviders. Note that the following overview table takes into account only objects that can currently be used for reporting.

Virtu Remot ODS Info InfoProvider MultiProvide Aggregate Transaction InfoSet BasisCubes al e Object Object s rs s al Cubes s Cubes Cubes s s 2380

577

234

1082

1

1

59

164

249

13

Aggregates The table below displays the top 3 InfoCubes regarding the number of their aggregates.

InfoCube

# Aggregates

PCORPPLAN

55

0PCA_C01

51

PCOPA0004

48

ODS Objects The following table gives a summary of the ODS objects. Description: BEx Indicator With the BEX indicator, you can determine whether the ODS object is immediately available for BEX queries. If the ODS object is not used for reporting, but only as a data container, you can delete the BEx Reporting indicator. The advantage is that no SIDs are generated, which leads to a better activation performance. If your ODS object reporting requirements are very "simple," in other words, if you are only selecting a limited number of records, you can use InfoSets for reporting on ODS objects even if the BEx reporting indicator is switched off. Please note that the loaded values are not checked for their correctness if SIDs are not generated. If you update the data of the ODS object in other data targets (DataMart functionality), SIDs are generated during this update should the BEx reporting indicator not be set. Additionally, it is possible to set the BEx reporting indicator even if the ODS object already contains data. In this case, the system generates the SIDs. Unique Data Records If each key field combination is only loaded once, you can set the indicator 'Unique Data Records,' which leads to a performance enhancement during activation. The system then does not need to check the existence of a record in the active data table and therefore a mass insert into the active table can be executed directly (otherwise, checks are first carried out to establish whether the datasets are already available in the active data table and then the dataset is marked either for update or insert). Additionally the data is not sorted. Attention: If the indicator 'Unique Data Records' is activated, the BW system does not guarantee unique records by deleting duplicates with regard to the ODS object key fields. Therefore the DataSource has to make sure that unique records are always delivered. If this is not the case, the ODS data activation will terminate.

© SAP AG

E2E100

8-36

# ODS Objects

# ODS Objects with BEX flag 397

# ODS Objects with unique flag 164

# Transactional ODS Objects 20

62

12.3.2 Dimensions with line item and high cardinality flag Whilst downloading information about InfoCubes that contain dimensions with the "High-Cardinality" indicator set, we had to stop the download to avoid generating a TIME OUT short dump. Therefore, the information in this chapter may not be complete. When the "High-Cardinality" indicator is set, the database will create B*-tree indexes instead of bitmap indexes for this dimension. This can lead to a reduction in performance, since only bitmap indexes can be used for a Star-join access, which is the most effective way of selecting data. Recommendation: Remove all "High-Cardinality" indicators from the InfoCube definition. In rare cases, the time used to recreate the indexes can increase. In this case, set the indicator again. Note: Setting the "Line-Item" indicator is useful for reporting and upload performance, but it can have a negative influence on the performance of input help functions.

12.3.3 InfoCube Design of Dimensions INFOCUBE PSDINACT

NO OF ROWS IN CUBE

MAX % ENTRIES IN DIMS COMPARED TO F-TABLE

114329

53430

PCRMSMBWI

73706

15048

PECPCADEL

226492

423

PSCA0601

370067

209

PSBR_PIP

268241

168

PCONS_SD1

11071996

158

PQIDX_CA0

170421

138

PBWTC_IT1

2136796

128

PCONSGLVC

932273

128

PBUDGCL15

1333576

120

PPROCNOBJ

1174985

117

PMKIS_15

964024

111

PCRMCEIS

125289

106

PPROCBSSR

760134

103

81523

100

PSDDBGMD

2410100

100

PQIDX_CA1

176447

100

PMKISMA2X

1454892

100

PPSPA_CS1

62671

100

PBWTC_IT3

2081135

96

PCATS_04

8634316

94

PSCA0001

1749363

94

PPIPE_2

© SAP AG

E2E100

8-37

INFOCUBE

NO OF ROWS IN CUBE


PMKISMA02

1332989

93

PCONSPEBC

451332

91

PBWTC_I51

72479

90

CPE_C01

2376988

89

PMKISBPRC

1807818

86

PPROCTRPR

4875718

84

PPAPATR01

2634134

81

PSCA0021

2388656

78

55499

78

PSCA01AG

101544

71

PPROCST22

2455853

65

PPROCTRPO

24067406

64

PCONSCBA

576095

64

PPSPA_C11

66063

64

PPSPA_CA5

60336

61

PPSPA_C09

217114

61

PSCA02MD

65344

60

PCRMSBP6

1847081

56

PCRMSOPCL

1330640

55

PITSC_SLA

922325

53

PCRMSOPSO

681936

52

PCRMSACPL

1152660

52

PCRMSBP1

1186825

52

PSCA0041

3185448

51

396990

51

11726737

50

0SEMBPSPF

100020

50

PCRMSBP4

1679694

50

PCRMSBP2

1441668

50

PCRMSBP3

1406174

50

PCRMSBP5

1378308

50

PCRMSLEAC

146710

48

PPROCSCN1

73208

48

198020

47

PEMCOCHKH

PCRMSOPAC PFIAR0005

0CPR_C02

© SAP AG

E2E100

8-38

INFOCUBE

NO OF ROWS IN CUBE


PCRMSCP2

496577

47

PCRMSCPR

496082

47

PPROCSCN2

73208

47

PCRMSLEPP

152440

46

PPROCSC_H

77110

46

PDMPAR12

4119587

45

PMKISBPR3

11937275

45

PMKIS_09

3001733

44

PMKISBPR2

4792488

41

PPICCACCL

53643

36

PFIAR0006

14337422

35

PPROCSKU

3215596

34

397226

34

PCONS07

79894

32

PPSPA_C10

62543

31

PPROCWP2

211838

31

PPROCSYSM

While downloading information regarding InfoCubes with one or more dimensions containing 30% or more entries as compared to the number of records in the fact table, we had to stop the execution of the download in order to avoid generating a TIME OUT short dump. Therefore, the information in this chapter may not be complete. Please verify if there are further InfoCubes where the number of records in the dimension tables should be limited. Explanation: The ratio between the number of entries in the dimension tables and the fact table should be reasonable. If an InfoObject has almost as many distinct values as there are entries in the fact table, the dimension this InfoObject belongs to should be defined as a line item dimension. Instead of creating a dimension table that has almost as many entries as the fact table, the system then writes the data directly to the fact table. On the other hand, if there are several dimension tables with very few entries (for example, less than 10), then those small dimensions should be combined into only one dimension. In order to obtain this information for your InfoCubes: - Call transaction RSRV. - Choose 'All Elementary Tests' - 'Database.' - Double-click the line 'Database Information about InfoProvider Tables.' - In the right window, choose 'Database Information about InfoProvider Tables.' - Enter the InfoCube name and choose 'Execute Tests.' - After the analysis finishes, choose 'Display Messages' and open the analysis tree with the correct time stamp. If the percentages seem to be unreasonable, this might be due to the fact that the optimizer statistics are not up to date. Recommendation: Try to limit the number of records in the dimension tables. Use the following guidelines: 1. If an InfoObject has almost as many distinct values as there are entries in the fact table, define the dimension of the InfoObject as a line item dimension. If this is defined in this manner, the system will write the data directly to the fact table (a field with the data element RSSID, which immediately shows the SID table of the InfoObject, is written in the fact table) instead of creating a dimension table that has almost as many entries as the fact table. 2. Only group related characteristics into one dimension. Unrelated characteristics can use too much disk space and cause performance problems (for example, 10,000 customers and 10,000 materials may result in 100,000,000 records). 3. Avoid characteristics with a high granularity, that is, many distinct entries compared with the number of

© SAP AG

E2E100

8-39

entries in the fact table. 4. If you cannot avoid characteristics with a high granularity and most of your queries do not use these characteristics, create an aggregate that stores summarized information. Do not use characteristics with a high granularity in this aggregate. Implementation: Regarding Line Item Dimension, when creating the dimensions as part of InfoCube maintenance, flag the relevant dimension as a line item. You can assign this dimension to exactly one InfoObject. Please note that the LineItem indicator can have a negative performance impact on input help usage for this characteristic in the query.

12.3.4 InfoCube partitioning The check for your range partitioning validity status could not be performed since you have not applied at least STA/PI 01H*.

F-Fact table partitioning Table names

No. of Partitions

/BIC/F103304

1511

/BIC/FPSDSPUSAG

886

/BI0/F0BBP_C01

826

/BIC/FPBWTC_CPU

823

/BI0/F0BWTC_C02

593

/BIC/FPPROCNATR

472

/BIC/FPMKISWSC1

464

/BIC/FPBWTC_ST3

423

/BIC/F105243

417

/BIC/F105241

417

/BIC/F105240

416

/BIC/F105242

416

/BIC/FPBUDGCL13

372

/BIC/FPPROCQRES

289

/BIC/FPCATS_04

254

/BIC/FPCONS_SD1

200

/BI0/F0PCA_C01

196

/BIC/FTSERVBLH

183

/BIC/FPMKISMA01

176

/BIC/FPBWTC_I41

174

/BIC/FPPROCTABA

164

/BI0/F0CPR_C02

159

/BIC/FPBUDGCL12

149

/BIC/FPMKISTMON

141

/BIC/FPMKISBPR2

140

© SAP AG

E2E100

8-40

Table names

No. of Partitions

/BIC/FPMKISBPR3

140

/BIC/FPMKISBPRC

140

/BIC/FPPROC_IMH

135

/BIC/F105511

124

/BIC/F105683

124

/BIC/FPBWTC_I21

114

/BIC/FPBWTC_I31

112

/BIC/FPBWTC_I01

110

/BI0/F0BWTC_C05

108

/BI0/F0BWTC_C04

108

/BIC/FPBWTC_I12

107

/BIC/FTSERVINC2

106

/BIC/FTSERVSOL

104

/BIC/FPPROCRUTH

104

/BIC/FPPSPA_CA3

102

/BIC/FPPORM_C05

102

/BIC/FPPROCRU_H

102

/BIC/FPBUDGSL02

99

/BI0/F0BBP_C02

98

/BIC/F105666

98

/BIC/FPPROCTRPO

98

/BIC/F105664

98

/BIC/FPPROC_CHK

97

/BIC/FZLGDSRCOP

94

/BIC/FPPROCSKU

93

/BIC/FPPROC13H

93

/BIC/F105512

91

/BIC/F105684

91

/BIC/F105788

91

/BIC/F105789

91

/BIC/F105725

91

/BIC/F105497

90

/BIC/F106800

90

/BIC/F106799

90

© SAP AG

E2E100

8-41

Table names

No. of Partitions

/BIC/F105506

90

/BIC/FPPROCST22

88

/BIC/FPPROCRZ20

87

……. Further 200 entries from this table have been deleted from this table in order not to print further 4 pages

The tables listed in the table 'F-Fact tables' have more than 30 partitions. Recommendation: To avoid performance issues when these tables are read, we recommend that you use compression on the InfoCubes of these F-Fact tables. Please see SAP Note 590370 for further details.

Empty F-Fact table Partitions F-fact table

No.Empty Partitions

/BIC/FPBWTC_CPU

181

/BI0/F0BBP_C01

129

/BI0/F0PCA_C01

104

/BIC/F105506

82

/BIC/F105512

82

/BIC/F105684

82

/BIC/F106800

75

/BIC/FPPROC_TNE

72

/BI0/F0BWTC_C02

62

/BIC/FPBUDGCL13

60

The #Empty Partitions# table contains up to ten F-fact tables with the highest number of empty partitions. There are 314 F-fact tables in total that contain empty partitions in your system. To check for tables with empty partitions, use report SAP_DROP_EMPTY_FPARTITIONS. See SAP Note 430486 for the appropriate recommendations for an overview or repair of F-fact tables in a BW InfoCube. Note: If the values shown in the table above do not seem reasonable when compared to report SAP_DROP_EMPTY_FPARTITIONS, the database statistics are probably not up to date and we were therefore not able to collect the most current data. To fix this, refresh the statistics.

E-Fact-Table partitioning (Search for E Fact tables with many parts ) InfoCube

# Partitions

# empty Partitions

PartChar

Time From

Time To

0PCA_C01

133

95

0CALMONTH

200001

201012

PPCA_P01

133

89

0CALMONTH

200001

201012

PPCA_SBR

133

132

0CALMONTH

200001

201012

PSBR_CONS

133

132

0CALMONTH

200001

201012

PSBR_PIP

133

132

0CALMONTH

200001

201012

The tables listed in the "Partitions of E-Facts" table have over 100 partitions.

© SAP AG

E2E100

8-42

Recommendation: To avoid performance issues, we recommend that you do not create more than 100 partitions. For more information, see SAP Note 590370. The "#empty Partitions" column contains the number of empty partitions for this InfoCube (E-fact table).

12.4 Analysis of Aggregates Aggregates only improve performance when they are used by your queries and when they summarize the data of the structure they are built from (the InfoCube or another aggregate). Unused or incorrect aggregates consume space on your database and increase the time needed for the roll-up and the change run procedure. For this reason, you should create proper aggregates in your system and regularly check that you are using the proper aggregates. We offer the following training courses for performance optimization: TEWA50 SAP BW Query tuning with aggregates. For more details about training, refer to the Internet address service.sap.com/empoweringworkshops

12.4.1 Aggregates suggested for building We checked for queries that spent a large part of their runtime on the database and selected more data on the database than necessary. The time frame for this analysis was the past seven days. We checked at the InfoCube level (we summarized all queries belonging to one InfoCube) or at the query level if aggregates were missing for your queries. The table(s) below list the largest InfoCubes and top 10 queries that spent more than 20% of the runtime on the database and that selected at least five times more data on the database than actually necessary. Recommendation: Create aggregates to increase the performance of your queries. Implementation: Use transaction RSDDV (Aggregate Maintenance Screen) to create aggregates for the InfoCubes listed in the table below. Use transactions RSDDV, RSRT, and RSRTRACE to find suitable aggregates for the queries listed in the table below. SAP Note 166433 describes how to use transactions RSDDV, RSRT, and RSRTRACE. Note: You should continuously check how aggregates can improve query performance, but you should also consider that aggregates consume space on your database and increase the time needed for the roll-up and the change-run procedure. AGGREGATES SUGGESTED ON CUBE LEVEL

Cube name

Time on DB [sec]

Time on DB / Rows selected / Total Rows selected Total time transferred from transferred Rows on time [sec] on DB [%] DB DB

PBWTC_ST3

5169,1

5200,5

99,4

24013673

618843

38,8

PCSSU_SU3

2916,6

4044,8

72,1

9641964

526228

18,3

PPROC_LOC

1253,1

1694,4

74,0

128511

694

185,2

PPROCCHK O

1143,6

1164,5

98,2

27854

2734

10,2

PPROCST0N

882,2

940,0

93,8

17094397

142368

120,1

PPROCTRPR

736,5

808,5

91,1

91515

148

618,3

PBWTC_I41

697,3

1065,3

65,5

1379244

22594

61,0

PBWTC_I01

664,1

1643,8

40,4

4134284

9398

439,9

PPROCTRPO

371,5

407,6

91,1

486171

34343

14,2

PBWTC_I12

246,9

373,6

66,1

136750

1531

89,3

PCMIMMC0 2

245,7

425,8

57,7

13071697

692091

18,9

PPROCPAT

172,6

190,7

90,5

44928

611

73,5

PPROCNOBJ

126,0

164,0

76,8

13404

95

141,1

© SAP AG

E2E100

8-43

Cube name

Time on DB [sec]

Time on DB / Rows selected / Total Rows selected Total time transferred from transferred Rows on time [sec] on DB [%] DB DB

PPROCST22

75,1

115,4

65,1

69725

1097

63,6

PPROCSPN1

58,8

66,0

89,2

859226

12186

70,5

PBWTC_DB S

22,8

80,2

28,4

38969

1876

20,8

PBWTC_IT1

13,2

14,8

89,3

101344

535

189,4

12.4.2 Aggregates suggested for deactivation We have found aggregates that are not used during query execution ("Number of calls" is equal to zero), or that are almost as large as the parent aggregate they are built from (the "Average reduce factor" is too low). Recommendation: Consider deactivating or deleting the aggregates listed below. Note: Do not deactivate an aggregate that has been created to improve the performance of master data activation. This kind of aggregate (Basis aggregate) does not contain any navigational attributes or hierarchies. This means it is not affected during master data activation. Aggregates that must be recalculated during master data activation can access the Basis aggregate instead of the InfoCube during master data activation.

Cube name

Aggr.-cube

0BWTC_C02

103234

0BWTC_C02

# calls

Created at

Last Call

14309315 2

20

29.03.2006

04.10.2006

102569

13547305 1

65

29.03.2006

29.09.2006

0BWTC_C02

103041

12358152 2

1438

29.03.2006

08.10.2006

0BWTC_C02

103632

7382295 1

0

29.03.2006

..0

0BWTC_C02

103609

22168 2

4

29.03.2006

29.09.2006

0BWTC_C02

103835

2477 14243

0

29.03.2006

..0

0BWTC_C02

103201

00

0

29.03.2006

..0

0PCA_C01

106657

12834626 1

5361

22.07.2006

09.10.2006

0PCA_C01

106924

8884296 1

20

28.09.2006

09.10.2006

0PCA_C01

105025

5247656 2

936

22.07.2006

05.10.2006

0PCA_C01

106699

5185747 1

115

23.07.2006

04.10.2006

0PCA_C01

104913

5165571 1

15218

22.07.2006

09.10.2006

0PCA_C01

104962

4755517 1

84

26.07.2006

07.10.2006

0PCA_C01

104963

4137757 1

141

26.07.2006

07.10.2006

0PCA_C01

104964

3999705 1

56

26.07.2006

06.10.2006

0PCA_C01

106880

3866679 1

147

11.09.2006

09.10.2006

0PCA_C01

106095

3865331 1

9936

22.07.2006

09.10.2006

0PCA_C01

106677

3433775 2

12506

22.07.2006

09.10.2006

0PCA_C01

106905

3433411 2

1017

11.09.2006

09.10.2006

0PCA_C01

104971

2820145 1

181

26.07.2006

25.09.2006

0PCA_C01

106675

2739154 1

2113

23.07.2006

09.10.2006

© SAP AG

# entries

Average reduce factor

E2E100

8-44

Cube name

Aggr.-cube

0PCA_C01

106894

0PCA_C01

# calls

Created at

Last Call

2726140 1

25

11.09.2006

06.10.2006

104965

2618217 2

150

26.07.2006

06.10.2006

0PCA_C01

106072

2062084 1

772

23.07.2006

05.10.2006

0PCA_C01

106902

2047286 2

0

11.09.2006

..0

0PCA_C01

106075

2033016 1

6626

23.07.2006

09.10.2006

0PCA_C01

106898

2018377 2

81

11.09.2006

07.10.2006

0PCA_C01

106906

1880476 1

14288

11.09.2006

09.10.2006

0PCA_C01

106673

1557564 2

1571

23.07.2006

08.10.2006

0PCA_C01

106887

1541938 2

300

11.09.2006

06.10.2006

0PCA_C01

106696

1528984 2

4095

23.07.2006

08.10.2006

0PCA_C01

106576

1368587 1

613

08.06.2006

08.10.2006

0PCA_C01

106876

1152062 1

0

11.09.2006

..0

0PCA_C01

106094

1151761 1

238

22.07.2006

09.10.2006

0PCA_C01

106081

956756 2

836

22.07.2006

09.10.2006

0PCA_C01

106903

956651 2

153

11.09.2006

09.10.2006

0PCA_C01

106078

591748 2

2867

23.07.2006

06.10.2006

0PCA_C01

106891

587702 2

340

11.09.2006

07.10.2006

0PCA_C01

106687

516569 1

4199

23.07.2006

08.10.2006

0PCA_C01

105641

484862 2

30

26.07.2006

12.09.2006

0PCA_C01

106272

124931 2

420

22.07.2006

06.10.2006

0PE_C01

106377

85308 1

35

08.08.2006

05.10.2006

0PE_C01

106378

85192 1

0

08.08.2006

..0

0PE_C01

106375

76055 1

0

08.08.2006

..0

0PE_C01

106373

75119 1

0

08.08.2006

..0

0PE_C01

106372

69193 1

0

08.08.2006

..0

0PE_C01

106363

68369 1

0

08.08.2006

..0

0PE_C01

106358

63307 1

1

08.08.2006

19.09.2006

0PE_C02

101409

1938630 2

0

12.06.2006

..0

0PE_C02

104783

1686945 2

692

10.02.2006

06.10.2006

0PE_C02

102573

1686508 2

0

10.02.2006

..0

0PE_C02

102572

1322211 2

0

10.02.2006

..0

0PE_C02

104851

408495 7

0

12.06.2006

..0

0PE_C02

103564

408495 1

0

12.06.2006

..0

0PE_C02

102911

178047 1

0

10.02.2006

..0

© SAP AG

# entries


E2E100

8-45

Cube name

Aggr.-cube

0PE_C02

106396

0PE_C02

# calls

Created at

Last Call

149211 3

0

10.02.2006

..0

106404

75724 3

0

10.02.2006

..0

0PE_C02

106397

49521 8

0

10.02.2006

..0

0PE_C02

106395

36927 10

0

12.06.2006

..0

0PE_C02

106405

28456 14

0

10.02.2006

..0

0PE_C02

106357

203 4

0

12.06.2006

..0

0PE_C03

103598

8437773 1

46

17.01.2006

06.10.2006

0PE_C03

106338

2205794 1

1

17.01.2006

09.03.2006

0PE_C03

106337

2202939 1

30

17.01.2006

02.10.2006

0PE_C03

106335

1294718 1

46

17.01.2006

05.10.2006

0PE_C03

106336

716166 1

0

24.06.2006

..0

0PE_C03

106334

716166 1

0

17.01.2006

..0

CPE_C00

104784

282634 1

1525

16.01.2006

06.10.2006

CPE_C00

106398

20306 1

30

27.01.2006

27.03.2006

CPE_C00

106403

19915 1

0

27.01.2006

..0

CPE_C00

106401

19879 1

0

27.01.2006

..0

CPE_C00

106402

16928 3

0

18.01.2006

..0

CPE_C00

106400

11291 5

0

18.01.2006

..0

CPE_C01

106280

127880 5

0

17.01.2006

..0

CPE_C02

106333

17300 1

121

12.06.2006

06.10.2006

CPE_C02

101313

2403 55

0

12.06.2006

..0

CPE_C02

106273

161 3

0

12.06.2006

..0

CPE_C02

106332

150 2

134

12.06.2006

08.10.2006

CPE_C06

106330

177 3

0

16.01.2006

..0

CPE_C07

106345

693452 1

0

13.08.2006

..0

CPE_C07

106367

693452 1

13

13.08.2006

05.10.2006

CPE_C07

106346

692598 1

0

13.08.2006

..0

CPE_C07

106370

692598 1

8

13.08.2006

29.09.2006

CPE_C07

106359

691351 1

0

13.08.2006

..0

CPE_C07

106364

691351 1

0

13.08.2006

..0

CPE_C07

106344

690887 1

0

13.08.2006

..0

CPE_C07

106348

690887 1

4

13.08.2006

19.09.2006

PBUDGAL01

103637

743174 2

0

10.10.2005

..0

PBUDGAL01

104636

738094 2

55

10.10.2005

29.08.2006

© SAP AG

# entries


E2E100

8-46

Cube name

Aggr.-cube

PBUDGAL01

103271

PBUDGAL01

# entries


# calls

Created at

Last Call

738070 1

63

10.10.2005

20.09.2006

103225

736062 1

6697

10.10.2005

06.10.2006

PBUDGAL04

103386

455693 1

2349

07.10.2004

20.09.2006

PBUDGAL04

103226

455549 1

103064 07.10.2004

06.10.2006

PBUDGCL10

104612

1551488 2

54

10.10.2005

29.08.2006

PBUDGCL10

103366

1551480 1

68

10.10.2005

20.09.2006

PBUDGCL10

103073

1543726 1

6968

10.10.2005

06.10.2006

PBUDGCL10

103640

1541954 2

0

10.10.2005

..0

PBUDGCL11

106342

421153 2

0

17.01.2006

..0

PBUDGCL12

103634

12812998 3

0

09.10.2006

..0

PBUDGCL12

103767

10665637 1

0

09.10.2006

..0

PBUDGCL12

105196

2424002 15

0

09.10.2006

..0

PBUDGCL12

103768

1833452 7

0

09.10.2006

..0

PBUDGCL12

103633

1548760 8

0

09.10.2006

..0

PBUDGCL12

103642

551251 23

0

09.10.2006

..0

PBUDGCL12

103635

551251 1

0

09.10.2006

..0

PBUDGCL12

106793

72869 1

0

09.10.2006

..0

PBUDGCL12

103769

00

0

09.10.2006

..0

PBUDGCL13

105069

2462517 2

1

09.10.2006

09.10.2006

PBUDGCL13

106796

1140421 2

0

09.10.2006

..0

PBUDGCL13

105093

318042 2

0

09.10.2006

..0

PBUDGCL13

105092

100949 1

0

09.10.2006

..0

PBUDGCS02

103007

277156 2

4267

07.10.2004

07.10.2005

PBUDGCS02

103214

181410 1

34487

07.10.2004

02.03.2005

PBUDGCS02

103376

118011 1

89270

07.10.2004

06.10.2006

PBUDGDV10

104673

69657 1

3429

07.10.2004

29.08.2006

PBUDGDV10

103350

69657 1

2366

07.10.2004

20.09.2006

PBUDGDV10

103227

69651 1

89482

07.10.2004

06.10.2006

PBUDGHLH1

104994

403148 1

9118

16.12.2005

08.10.2006

PBUDGHLH1

105091

202707 2

20631

16.12.2005

09.10.2006

……. Further 600 entries from this table have been deleted from this table in order not to print further 11 pages

12.4.3 Aggregates Maintenance The tables below show an overview of maintenance costs for your aggregates in the last six weeks. The first table shows the sum of all aggregates for each InfoCube. The second table shows up to 20 aggregates

© SAP AG

E2E100

8-47

that required the most total maintenance time. In addition, we analyzed the three aggregates with the most "total time" in more detail.

CUBES [28.08.2006 07:50:53 09.10.2006 04:02:40]

InfoCube

Nr. of Aggregates

Total

659

0PCA_C01

79

Total time [sec]

% Rollup % (Re)creation % Delta Change- Records time time run time inserted

558427,7

738603337

57356,1 97

0

3

6492527

PBUDGCL12 9

51816,3 11

88

0

84684041

PCONSPEB 10 D

45732,3 5

95

0

196416329

PCOPA0004 48

37222,8 90

2

8

2679760

PCONSPIO2 38

37191,9 0

100

0

84781216

PCONS09B

16

32136,7 71

19

10

27349256

PROLFFI01

22

25460,6 69

24

7

26265048

PBWTC_ST3 4

24194,7 100

0

0

13664

PBUDGCL13 7

22568,3 20

79

0

46424714

PFIAR0005

16

18550,9 99

0

1

8498832

PSIBWPREP 18

18465,4 57

43

0

39858195

PPCA_H01

16599,4 0

84

16

881430

PSIBWHDIT 37

15727,4 84

0

16

3579233

0PE_C02

21

13053,0 62

0

38

6010025

ZLGDSRPIP 11

12666,4 26

74

0

42945512

PCONS05

23

12191,7 65

30

5

15534384

0PE_C03

6

11935,2 85

0

15

26362362

PPROC_LOC 1

10907,3 100

0

0

4155254

PMKISMA01 27

9619,4 100

0

0

26284

PCONS07

9354,5 71

28

1

907102

PCATS_042 9

8328,4 100

0

0

698908

PSIBWAGG 14

7864,3 86

0

14

2015302

ZLGDSRINV 2

6903,0 57

6

37

6361509

ZLGDSRCO 4 P

5882,9 15

47

37

18107933

CPE_C07

8

5537,7 61

0

39

16541346

PCATS_01

4

5341,8 16

84

0

17828803

0BWTC_C02 8

4809,8 100

0

0

176026

PPAPAC01

3764,5 30

70

0

25325392

© SAP AG

6

27

5

E2E100

8-48

InfoCube

Nr. of Aggregates

0PE_C01

7

3402,7 61

0

39

1343336

PMKIS_15

17

3247,3 87

7

5

164146

CPE_C00

8

3176,8 59

0

41

672291

PPROCCHK 1 O

2478,7 20

80

0

9526244

PSDINACT

7

1744,5 68

32

0

1981449

CPE_C02

7

1728,0 93

3

4

143835

PCATS_03

3

1184,3 59

41

0

261663

PCSSU_SU3 5

937,1 37

63

0

2387129

PCONS05A

4

908,6 0

2

98

9240

PPAPAC02

3

906,0 80

20

0

1831028

PPSPA_C02 1

851,5 100

0

0

133230

PCONS09A

4

831,5 0

39

61

2982252

PSDIN0001

8

585,8 100

0

0

278033

PBUDGCL11 1

541,1 100

0

0

2064509

PCORPPLA N

529,6 4

0

96

96366

PPROCST0N 1

506,2 100

0

0

204147

CPE_C01

1

461,8 88

0

12

3637

PCONS09

5

444,6 66

23

11

612001

PROLFHR01 1

394,3 100

0

0

107548

PCONSCBA 1

282,0 100

0

0

3126

PROLFCS01 1

254,3 100

0

0

82811

PCMIAC10

237,0 49

51

0

55632

PPAPAC02A 1

226,1 0

100

0

1214619

PROLFSW01 1

197,1 100

0

0

71124

PPSPA_C07 1

189,7 37

19

44

104099

PBUDGSL02 2

148,5 100

0

0

73900

PSIBWFC01 1

125,9 100

0

0

165767

CPE_C06

125,8 100

0

0

5

PPAPAC01A 3

121,3 0

100

0

1095123

PSDDBGMD 2

120,0 67

33

0

6816

PFIAR0001

107,4 0

0

100

0

0

86

2214

11

2

1

8

PBUDGHLP 5 1 © SAP AG

Total time [sec]


95,7 14

E2E100

8-49

InfoCube

Nr. of Aggregates

Total time [sec]


PSIBWFC02 1

62,9 100

0

0

3730

PBUDGHLH 1 1

54,9 0

0

100

1900

PCONS02

1

30,9 0

0

100

0

PBUDGHLS 1 1

5,1 0

0

100

0

0,0 0

0

0

0

51

AGG. [28.08.2006 07:50:53 09.10.2006 04:02:40]

InfoCube

Aggregate

Total

Total time [sec]

% Rollup time

% (Re)creation time

% Delta Changerun time

558427,7

Records inserted 738603337

PBUDGCL1 103634 2

23206,8 4

96

0

6918472

PCONSPEB 106065 D

21485,3 3

97

0

88046007

PCONSPEB 106061 D

18188,9 3

97

0

87664215

PBUDGCL1 103767 2

13565,1 8

92

0

53282739

PPCA_H01

13168,0 0

100

0

480998

10907,3 100

0

0

4155254

8174,9 100

0

0

3477145

PBUDGCL1 105069 3

7684,6 15

85

0

13374809

PCONS09B

7441,9 93

7

0

275778

PBWTC_ST3 105240

6843,6 100

0

0

10094

PCONS09B

6201,0 95

5

0

213714

PBWTC_ST3 105242

6116,4 100

0

0

1640

PBWTC_ST3 105243

5740,3 100

0

0

1264

PBWTC_ST3 105241

5494,4 100

0

0

666

PCONS09B

105227

4758,4 41

35

25

12602406

PROLFFI01 106925

4637,9 14

80

6

6259579

PBUDGCL1 106794 3

4545,1 20

80

0

11997861

PCATS_01

101952

4357,2 10

90

0

11829199

104971

4021,1 100

0

0

147828

4006,1 18

82

0

1698194

106926

PPROC_LO 106801 C PFIAR0005

102435

106416

106418

PBUDGCL1 105196 © SAP AG

E2E100

8-50

InfoCube

Aggregate

Total time [sec]

% Rollup time

% (Re)creation time

% Delta Changerun time

Records inserted

2

12.5 Source System Settings Upload customizing BW (SBIW) In every SAP source system, part of the ROIDOCPRMS table controls the data transfer from the source system to the system. The table contains the following information: MAXSIZE - Maximum size of a data package in KB STATFRQU - Number of packets that are transferred before statisticalinformation is sent MAXLINES - Maximum number of records sent in one data package MAXPROCS - Maximum number of dialog work processes for each upload request used to send the data to the system To ensure the performance and stability of the upload process, it is important that this table is set up correctly. We have checked the setup of this table for all source systems connected to the system. Recommendation: To avoid potential extraction problems, change the entries in table ROIDOCPRMS in the different source systems as explained in the table below. Consequences: If the setting for MAXSIZE is too large, data packets will also be very large. A system timeout may occur for work process(es) on the server assigned to these packets, or there may be problems with the memory. If the setting for MAXLINES is too large, the high number of records for each data packet might lead to dumps in the source system or memory problems on the server. If the values are too small, several small data packets are created, which increases the upload time. Note that you do not have to reduce the values if you have enough resources available (memory, work processes) and your parameter settings are tested carefully. Implementation: To implement those settings you have two possibilities: a) Modify all source systems centrally using the Administration Workbench of Call transaction RSA1, click a source system using the secondary mouse button, and choose #Customizing for Extraction#. The system navigates directly to the selected source system in transaction SBIW. Choose General Settings -> Maintain Control Parameters for the Data Transfer. b) Maintain the settings in the different R/3 source systems. Call transaction SBIW and choose General Settings ->Maintain Control Parameters for the Data Transfer. Note: If table ROIDOCPRMS is empty, the systems use default values during runtime. The defaults are 10,000 for MAXSIZE and 100,000 for MAXLINES. The default for MAXSIZE is small whereas the default for MAXLINES is reasonable for most systems. In some cases, it may be necessary to set the values for some of the parameters outside our recommended range. One example of this is the MAXPROCS parameter (maximum number of dialog work processes for each upload request). This parameter limits the use of resources during an upload in order to prevent a resource bottleneck occurring in the system. If you reduce the setting for this parameter, the upload performance may suffer even though there are enough system resources available. If you use your as a DataMart, you have to maintain this table in the system as well.

System name

Parameter name

AE6CLNT100

maxsize

000000 20000

AE6CLNT100

statfrqu

00 10

AE6CLNT100

maxprocs

00 3

CRP_MDT200

maxsize

000000 20000

CRP_MDT200

statfrqu

00 10

CRP_MDT200

maxprocs

00 3

CSRCLNT001

maxlines

500000 100000

© SAP AG

Current value

E2E100

Recommended value

8-51

System name

Parameter name

Current value

Recommended value

CSRCLNT001

maxprocs

ECACLNT000

maxsize

000000 20000

ECACLNT000

statfrqu

00 10

ECACLNT000

maxprocs

00 3

GTPCLNT001

maxsize

000000 20000

GTPCLNT001

statfrqu

00 10

GTPCLNT001

maxprocs

00 3

IBPCLNT001

maxsize

000000 20000

IBPCLNT001

statfrqu

00 10

IBPCLNT001

maxprocs

00 3

ICPCLNT001

maxsize

010000 20000

IFPCLNT001

maxsize

000000 20000

IFPCLNT001

statfrqu

00 10

IFPCLNT001

maxprocs

00 3

IWACLNT001

maxlines

500000 100000

IWACLNT001

maxprocs

05 4

IWPCLNT001

maxsize

010000 20000

IWPCLNT001

maxlines

500000 100000

IWPCLNT001

statfrqu

01 10

IXPCLNT001

maxsize

000000 20000

IXPCLNT001

statfrqu

00 10

IXPCLNT001

maxprocs

00 3

ORPCLNT001

maxsize

000000 20000

ORPCLNT001

statfrqu

00 10

ORPCLNT001

maxprocs

00 3

PRPCLNT001

maxsize

000000 20000

PRPCLNT001

statfrqu

00 10

PRPCLNT001

maxprocs

00 3

QAPCLNT200

maxsize

000000 20000

QAPCLNT200

statfrqu

00 10

QAPCLNT200

maxprocs

00 3

05 4

12.5.1 Important Notes for Source System Upload Settings To optimize upload performance and to prevent any loading problems for the R/3 source systems, check and implement the following SAP Notes as soon as possible.

© SAP AG

E2E100

8-52

Note Number

Description

327828

Performance of BW supply systems

385741

Collective runs are not deleted

351936

Problems with different collective runs

12.6 Recommendations for BW 12.6.1 General Parameters The table shown here provides an overview of the Customizing parameters of the table 'RSADMIN' of which some or all are considered INVALID. Each parameter is assigned a rating in the column 'Invalid,' which details the reason for the parameters validity. Additionally, a check is made as to whether a duplicate exists. Only parameters that are invalid/duplicates and/or have an OSS note available will be displayed. The validity rating is explained below, sequence 0-5 : Invalid 0 = This means the parameter is valid and used correctly. Invalid 1 = This parameter is invalid and you should reconsider the setting, because it is either not recognized as existing or your current release does not support the use of this parameter. Invalid 2 = This parameter is invalid and you should reconsider the setting, because a value is mandatory for this parameter but none was found. Invalid 3 = This parameter is invalid and you should reconsider the setting, because it is not relevant for your current database. Invalid 4 = This parameter is invalid and you should reconsider the setting, because your current support package does not support its use. Invalid 5 = This parameter is invalid and you should reconsider the setting, because your SAPI release is not sufficient. Recommendation: Please verify if the parameters showing as invalid can be removed or have to be changed. If duplicates exist, please remove the incorrect version.

PARAM_NAME

VALUE

OSS_NOTE

INVALI DUPLICATE D

COMPNOMERGE

X

613701

0

CPE_C05

NOPARALLEL

449477

0

CR_MAXWPC

10

534630

0

CR_RFCGROUP

CHANGERUN

534630

0

705011

2

CSV_MIMETYPE DBMS_STATS

1

351163

0

DEBUG_RSDRI

OFF

302937

0

766432

2

IS_MO_AUTH_CHECK MDX_LARGE_INTERVAL_KEY

X

671672

0

MPRO_MAX_RESULT

10240

630500

0

ORA_LOGGING

ON

442763

0

ORA_PARALLEL_DEGREE

10

544521

0

PCOPA_MUL

NOPARALLEL

449477

0

PSIBW_AC1

NOPARALLEL

449477

0

© SAP AG

E2E100

8-53

PARAM_NAME

VALUE

OSS_NOTE

INVALI DUPLICATE D

PSIBW_MAC

NOPARALLEL

449477

0

709141

2

REUSE_XL_INSTANCE RSDM_F4_NO_XYTAB_JOIN

X

581079

0

RSDRH_BUFFER_SIZE

3000

584216

0

RSMD_SELECT_SINGLE_LIMIT

1000

657106

0

RSPC_EXECUTE_USER

X

640746

0

SM21LOG

X

548871

0

TRACEPAK01

DAILY UPDATE (DELTA) 308319,

0

12.6.2 Important notes for BW The table below lists important SAP Notes for BW that deal with performance and data security.

Note Number

Description

0309824

BW problem message / Support Packages, Patches

0166433

Options to find aggregates

0423099

"Low Speed Connection" Parameter and BEx Analyzer

0558994

General information on BW front-end patches

0550669

Compressed transfer of BW Web Applications

0567747

Composite note BW 3.x performance: Extraction

0567746

Composite note BW 3.x performance: Query

0567745

Composite note BW 3.x performance: DB-specific

DATA CONSISTENCY

Note Number

Description

Area

598691

Check: InfoObjects with ALPHA and fields without ALPHA

Check Tools

323140

DUPREC short dump when loading master data

Data Loading

529789

BW extraction/extractor checker differences

Data Loading

487393

Incorrect values with star_transformation_enabled=true

Database

534295

Incorrect values in the BW environment

Database

586163

Composite SAP note on SAP R/3 Inventory Management

Non-Cumulatives

565086

Composite SAP note: Incorrect data in DataMart or Query

Query

12.6.3 BI - KPIs There are BW Key Performance Indicators that exceed their reference value. This indicates severe problems, or at least optimization potential for performance, data volumes, and administration.

© SAP AG

E2E100

8-54

Based on these key figures, we recommend that you order the corresponding services for further analysis. The table below shows the KPIs, their observed value, and the reference limit. The last column contains the Service Code for finding the appropriate service in the second table, "Available Services". RECOMMENDATIONS

Recommended Services

KPI

Description

Observed Reference Rating

Active Rep. Auth. (#)

Total number of active reporting authorization objects

4902

500

YELLO E2 W

Avg.Runtime MP

Average runtime of manual planning (s)

53

30

YELLO O3 W

DB Growth (GB)

Database Growth per month (GB) 171.31

25

YELLO O6 W

DB Size (GB)

Database Size (GB)

2326.34

500

YELLO O6 W

Defined process chains Total number of defined Process (#) Chains

372

200

YELLO O1, O5 W

Meta Chain executions Total number of Meta Process last week Chain executions last week (#)

916

250

YELLO O1, O5 W

NrAggr0Call (#)

Nr of aggregates with no calls

339

150

RED

NrBuildAggrCube (#)

Nr of aggregates recommended to 17 build - cube level

15

YELLO E1 W

NrDelAggr (#)

Nr of Aggregates recommended to delete

200

RED

698

E1

E1

AVAILABLE SERVICES

Service Code

Service Name

G1

General link to support services

Description

Link service.sap.com/supportservices

SAFEGUARDING S1

EarlyWatch Service

Remote analysis of system parameters and technical performance tuning

service.sap.com/safeguarding

OPTIMIZATION O1

BPPO - Extraction and Loading

Optimization of critical load path

service.sap.com/bppo

O2

BPPO - Query Tuning

Optimization of query execution


O3

BPPO - Planning and Simulation

Optimization of Planning and Simulation Application


O4

SAP Remote Performance Optimization

service.sap.com/rpo

O5

SAP System Administration Knowledge transfer for optimal for BI business process administration

service.sap.com/sysadmin

O6

SAP Data Volume Management for SAP BW

service.sap.com/dma

© SAP AG

Analysis of disk space saving potential

E2E100

8-55

Service Code

Service Name

Description

Link

Official classroom training TEWA50

EMPOWERING E1

TEWA50 - Query tuning with aggregates

Workshop - Efficient Aggregate Control and Query Tuning

E2

PDEBW2 - BI Reporting Authorizations

Workshop - Reporting Authorization Official classroom training Setup and Optimization PDEBW2

12.6.4 Nametab inconsistencies NAMETAB INCONSISTENCIES

Table

# Total # View 01 # View 02 # View 03 # View 04 # View 05 # View 06

DDNTT

65374

DBDIFF

1319

3111

28905

0

0

32039

241022 0

4016

29215

26619

181172

0

RSDD_TMPNM_ADM

38118

2060

4016

1

1

1

32039

DBA_VIEWS

0

0

0

0

0

0

0

There are several entries in tables DDNTT and DDNTF that cannot be found in tables DBDIFF and RSDD_TMPNM_ADM (or in the DB_VIEWS of Oracle). This means that these temporary entries are obsolete and no longer used. SAP Note: 449891 Temporary database objects in BW 3.x Actions: 1. If there are too many entries in DBDIFF: After a change run, obsolete entries may still appear in table DBDIFF. This table has buffer mode FULL => for example: one day it grew up to 247MB. The problem is solved in the following Support Packages: - BW 3.0B Support Package 22 - BW 3.10 Support Package 16 - BW 3.50 Support Package 4 Report SAP_UPDATE_DBDIFF clears these obsolete entries from table DBDIFF. The problem may occur also in systems containing the correct Support Package if the correction report has not yet been run. SAP Note: 736845 After Change Run entries in DBDIFF resist 2. If there are several entries for 03 views in DBDIFF: Use report SAP_UPDATE_DBDIFF to clean DBDIFF. The report checks temporary objects in table RSDD_TMPNM_ADM and deletes any unnecessary ones. 3. If there are several entries for 06 views in RSDD_TMPNM_ADM: SAP Note: 698280 Reduce number of 06-er Tables in the database 4. If there are too many entries for 01 views in RSDD_TMPNM_ADM:

© SAP AG

E2E100

8-56

SAP Note: 811354 Usage of complex queries reduces the performance of the database 5. If there are too many entries for 03 views in RSDD_TMPNM_ADM: Use the function module RSDDTMPTAB_CLEANUP as described in SAP Note: 867000 Inconsistencies in temporary tables of TYPE 06 6. If there are too many entries for 03 views in DDNTT and DDNTF compared to the number of entries in DBDIFF/RSDD_TMPNM_ADM. Several obsolete entries in the dictionary tables that are not cleaned Use the SAP_DROP_TMPTABLES report as described in SAP Note: 883843 "BW instances can no longer be started" Final result: Run SAP_DROP_TMPTABLES, SAP_UPDATE_DBDIFF or RSDDTMPTAB_CLEANUP depending on the situation to clean obsolete temporary entries.

12.7 Frontend Servers To ensure the best performance and stability on your frontend servers, you must install the most recent frontend server release and patch level. From frontend server 2.0B patch level 6 onwards, the installation data of the frontend server is written to the BW server table RSFEC. During this session, we downloaded and checked the most recent information (in a time frame of two weeks) with respect to the frontend server release and patch level used and the installation of a maximum of 100 frontend servers. Even though we checked up to 100 frontend servers, other frontend servers may still have severe problems. We cannot check frontend servers with an older release or patch level than frontend server 2.0B patch level 6. The following table gives an overview of the results. SUMMARY OF CHECKED PCS

Rating

Number of PCs checked

Green

65

Yellow

5

Red

30

For more information about the problems we detected, please refer to the following sections.

12.7.1 PCs with outdated Patch Level The frontend patch installed on some of your frontend servers is not current. Recommendation: Immediately update the frontend servers listed in the following table to the newest frontend patch available.

PC Name

Current Frontend Release

Current Frontend Patch

Recommended Frontend Release

Recommended Frontend Patch

Ratin g

BLRN50050945A

3.50

7

3.50

9

Yello w

© SAP AG

E2E100

8-57

PC Name





Ratin g

BLRN50056302A

3.50

7

3.50

9

Yello w

DUBN2127000000 3.50 A

5

3.50

9

Red

I801725A

3.50

0

3.50

9

Red

IGB117287

3.50

6

3.50

9

Red

LONN60131582A

3.50

3

3.50

9

Red

LONN60131635A

3.50

6

3.50

9

Red

LONN60140448A

3.50

6

3.50

9

Red

LONN60142337A

3.50

3

3.50

9

Red

LONN60143507A

3.50

6

3.50

9

Red

LONN60143526A

3.50

1

3.50

9

Red

MHGN00154761A 3.50

5

3.50

9

Red

MILD00010057A

3.50

7

3.50

9

Yello w

P103216

3.0B

9

3.0B

20

Red

P104641

3.0B

9

3.0B

20

Red

P108559

3.50

4

3.50

9

Red

P109827

3.50

6

3.50

9

Red

P113204

3.0B

9

3.0B

20

Red

P121534

3.50

5

3.50

9

Red

P123428

3.50

6

3.50

9

Red

P123605

3.50

7

3.50

9

Yello w

P125807

3.50

7

3.50

9

Yello w

P129443

3.50

3

3.50

9

Red

P129574

3.0B

10

3.0B

20

Red

P133061

3.50

4

3.50

9

Red

P134203

3.0B

13

3.0B

20

Red

P138896

3.50

3

3.50

9

Red

P6121649

3.50

4

3.50

9

Red

P78355

3.50

5

3.50

9

Red

WDFD3000137281 3.50 A

3

3.50

9

Red

WDFN00104975A 3.50

5

3.50

9

Red

© SAP AG

E2E100

8-58





Ratin g

WDFN00110820A 3.50

3

3.50

9

Red

WDFN00123447A 3.50

3

3.50

9

Red

WDFN00133879A 3.50

6

3.50

9

Red

WDFN00144016A 3.50

3

3.50

9

Red

PC Name

12.7.2 PCs with old SAP GUI installation Available SAP GUI releases for your BW release

SAP GUI version support timeline

6.20

SAP GUI 6.20 support is planned to be "long term".

6.40

SAP GUI 6.40 support ends on December 31, 2007.

13 SEM Checks Use the information in the tables below to monitor the performance of the SEM application on a continuous basis. The analyzed period is always one complete month up to the date of the session, as this is a typical length of a planning cycle in SEM.

13.1 Planning Activities PLANNING ACTIVITIES

Date

Manual Planning Calls

Planning Function Calls

API Calls

Web Accesses

02.10.2006 27

144

0

1609

03.10.2006 22

40

0

2414

04.10.2006 46

75

1

3029

05.10.2006 59

180

0

2509

06.10.2006 28

139

0

2329

07.10.2006 1

4

0

1599

08.10.2006 1

4

0

1711

13.2 Planning Performance Overview In the following tables you can see an overview on the performance/throughput of your self-defined planning functions and planning layouts in SEM-BPS. The information is aggregated on a planning function type level. The key figure 'Records/Second' shows the throughput of a particular planning function type. The higher this value, the better the performance. (Planning functions triggered from web interfaces are not considered!) BENCHMARK: PLANNING FUNCTIONS

Planning Function Type Executions Runtime / Execution [ms] Records / Execution Records / Second 1

9

212590

69119

472

3

143

7648

217

70

© SAP AG

E2E100

8-59

Planning Function Type Executions Runtime / Execution [ms] Records / Execution Records / Second 5

3

74666

0

0

6

141

10911

1080

133

7

150

6927

3388

906

10

144

1418

133

93

Above, the technical description is used for the planning function types (e.g. Type 07 for a FORMULA function). Below you can find the description for a selection of the most frequently used planning function types. PLANNING FUNCTION TYPES (SELECTION)

Planning Function Type

Description

01

Copy

02

Repost

05

Delete

06

Exit Function

07

Formula

08

Revaluation

10

Currency Translation

12

Repost (Characteristic Relationships)

14

Copy to Several Target Objects

36

Valuation

40

Allocation

42

Currency Translation (Account-Based)

BENCHMARK: MANUAL PLANNING

Executions

BW Time Runtime [ms] / Time ABAP Time to Open Transfer Time Cells / [ms] / BW Execution [ms] / Execution [ms] / Execution [ms] / Execution Execution Call

185

52419

6565

8711

8454

32882

1723

Legend: - Runtime/Execution(ms): Average runtime in milliseconds for a single execution of a planning function respectively the initial start of a planning layout. - Records/Execution: Average no. of records processed (Records Read from planning function + Records Changed + Records Created) for a single execution of a planning function - Records/Second: Average no. of processed records that a particular planning function can process within one second. The higher this number, the better the performance/throughput of the function. Only the ABAP processing time of the function is considered here, not the BW read access. - BW Time(ms)/BW Call: Average read time in milliseconds for a single BW Access of a planning layout. - TimeABAP(ms)/Exec: Average time in milliseconds for the planning processor to build up the logical screen of the planning layout

© SAP AG

E2E100

8-60

- TimeToOpen(ms)/Execution: Average time to open the EXCEL viewer (this time component is not measured for ALV layouts). This is part of launching a planning layout. - TransferTime(ms)/Execution: Average time to transfer the cells from the application server to the EXCEL frontend (this time component is not measured for ALV layouts). - Cells/Execution: Average number of cells transferred to the planning layout. This number indicates the size of a planning layout (this key figure is not measured for ALV layouts).

13.3 TOP 15 Planning Functions In the following tables, performance critical planning functions are listed with different perspectives on performance. In the table below the planning functions with the highest runtime for a single execution are displayed. The BW Read Time is part of the overall runtime ('RunTime(ms)'). You can identify whether the long runtime is caused from an expensive BW access (long BW read time) and whether a big number of records were processed from the planning function. TOP 15 PLANNING FUNCTIONS BY SINGLE RUNTIME

Typ Planning e Area

Planning Level

Planning Package

Planning Function

Parameter Group

Run Time [ms]

BW Read Records Time [ms] Processed

01

ROLFMU01 ROLFVC10 0-ADHOC ROLFVC10

ROLFVC10 715188

112052

235262

01


ROLFVC20 333484

90342

142975

06

ROLFMU01 ROLFCR10 0-ADHOC ROLFCR20

ROLFCR21

47

0

07


ROLFVC20 115900

0

235262

03

BUDGMA0 BUDGMAH 0-ADHOC BUDGMAF8 BUDGMAF8 112966 7 7

101123

206

01

ROLFMU01 ROLFGC19 0-ADHOC ROLFGC20

ROLFGC20 110952

81608

27801

07


ROLFVC41 106860

56641

34908

05


ROLFVC21 85610

82782

0

01


ROLFGC14 77155

24078

22258

06

BUDGMA0 BUDGMAU 0-ADHOC BUDGMAF5 BUDGMAF5 74985 7 8

9078

22899

01


ROLFVC19 71671

25765

27703

03

BUDGMA0 BUDGMAU 0-ADHOC BUDGMAF8 BUDGMAF8 60203 7 7

58375

1285

07

BUDGMA0 BUDGMAU 0-ADHOC BUDGMA09 BUDGMAF9 58766 7 7

56859

29

01


ROLFGC11 47984

34796

5095

07

ROLFMU01 ROLFSW38 0-ADHOC ROLFSW01 ROLFSW01 45500

15343

8230

172367

Legend: - Records Processed: Records read from planning function + records changed + records newly created. In the table below planning functions with the highest average runtime per execution are displayed. In comparison to the table above, statistical outliers can be eliminated by considering the number of calls as well. The number of calls indicates how often this function has been executed in the last month.

© SAP AG

E2E100

8-61

TOP 15 PLANNING FUNCTIONS BY AVERAGE RUNTIME

Typ Planning e Area

Planning Level

Planning Planning Package Function

Parameter Group

Avg Call Run s Time [ms]

Avg BW Avg Read Records Time Processed [ms]

01

ROLFMU0 0ROLFVC10 1 ADHOC

ROLFVC10 ROLFVC10 3

413261

52657

132032

01

0ROLFMU0 ROLFVC20 ADHOC 1

ROLFVC20 ROLFVC20 1

333484

45171

142975

06

ROLFMU0 0ROLFCR10 1 ADHOC

ROLFCR20 ROLFCR21 4

113014

47

0

01

ROLFMU0 0ROLFGC19 1 ADHOC

ROLFGC20 ROLFGC20 1

110952

40804

27801

07


ROLFVC41 ROLFVC41 1

106860

18880

34908

05

0ROLFMU0 ROLFVC21 ADHOC 1

ROLFVC21 ROLFVC21 3

74666

71770

0

01


ROLFVC19 ROLFVC19 1

71671

12883

27703

07


ROLFVC20 ROLFVC20 3

57873

0

132032

01

0ROLFMU0 ROLFGC13 ADHOC 1

ROLFGC14 ROLFGC14 2

54718

11387

11201

01

ROLFMU0 0ROLFGC10 1 ADHOC

ROLFGC11 ROLFGC11 1

47984

17398

5095

07

ROLFMU0 0ROLFSW38 1 ADHOC

ROLFSW01 ROLFSW01 1

45500

7672

8230

06


ROLFCR20 ROLFCR23 1

22906

31

0

07


ROLFVC42 ROLFVC42 1

19329

0

34908

06


ROLFCR20 ROLFCR25 2

17961

39

0

03

BUDGMA0 BUDGMAH 07 7 ADHOC

BUDGMAF BUDGMAF 32 8 8

17384

5500

361

The table below shows planning functions that contributed significantly to the overall SEM workload (from last month) on your system. The relative runtime describes the percentage of the overall total runtime (runtime of all planning functions) caused by this particular function. TOP 15 PLANNING FUNCTIONS BY TOTAL RUNTIME

Typ Planning e Area

Planning Level


Parameter Group

Rel. Avg Run Call Run Time s Time [ms] [%]

Avg Records Processed

01

ROLFMU0 ROLFVC10 0-ADHOC ROLFVC10 ROLFVC10 3 1

413261

21

132032

06

BUDGMA0 BUDGMAU 0-ADHOC BUDGMAF BUDGMAF 98

9030

15

1550

© SAP AG

E2E100

8-62

Typ Planning e Area 7

Planning Level


8

5

Parameter Group

Rel. Avg Run Call Run Time s Time [ms] [%]

Avg Records Processed

5

07

BUDGMA0 BUDGMAU BUDGMAF 0-ADHOC BUDGMA09 98 7 7 9

5682

9

230

03

BUDGMAF BUDGMAF BUDGMA0 BUDGMAH 32 0-ADHOC 8 8 7 7

17384

9

361

03

BUDGMA0 BUDGMAU BUDGMAF BUDGMAF 0-ADHOC 97 7 7 8 8

5415

9

199

06

ROLFMU0 ROLFCR10 0-ADHOC ROLFCR20 ROLFCR21 4 1

113014

8

0

01


333484

6

142975

05


74666

4

0

07


57873

3

132032

07

BUDGMAF BUDGMA0 BUDGMAH 32 0-ADHOC BUDGMA09 9 7 7

3853

2

350

01

ROLFMU0 ROLFGC19 0-ADHOC ROLFGC20 ROLFGC20 1 1

110952

2

27801

01


54718

2

11201

07


106860

2

34908

10

BUDGMA0 BUDGMAU BUDGMAF BUDGMAF 0-ADHOC 98 7 2 7 7

1067

2

77

10

BUDGMA0 BUDGMAH BUDGMAF BUDGMAF 0-ADHOC 32 7 7 7 7

2941

2

350

The table below shows planning functions that contributed significantly to the overall BW load on your system (caused from SEM). You can see the overall BW read time for a particular number of BW calls (a single planning function can cause no, one or several BW calls to read the required data) and how many records were read from BW (column 'Total BW Recs Read'). TOP 15 PLANNING FUNCTIONS BY TOTAL BW READ TIME

Typ Planning e Area

Planning Level

Planning Package

Planning Function

Parameter Group

Avg BW BW Read Calls Time [ms]

Rel. BW Avg BW Read Recs Time Read [%]

07

BUDGMAF BUDGMA0 BUDGMAU 56 0-ADHOC BUDGMA09 9 7 7

7150

17

1763

03

BUDGMA0 BUDGMAU BUDGMAF 0-ADHOC BUDGMAF8 55 7 7 8

7251

17

250

06


6606

16

1426

© SAP AG

E2E100

8-63

Typ Planning e Area

Planning Level

Planning Package

Planning Function

Parameter Group

Avg BW BW Read Calls Time [ms]

Rel. BW Avg BW Read Recs Time Read [%]

01


52657

14

66016

03

BUDGMA0 BUDGMAH BUDGMAF 0-ADHOC BUDGMAF8 44 7 7 8

5500

10

1148

05


71770

9

0

01


45171

4

73069

01


40804

4

27752

07


18880

2

28738

01


11387

2

5601

01


17398

1

2548

01


12883

1

13852

07

ROLFMU0 ROLFSW38 0-ADHOC ROLFSW01 ROLFSW01 2 1

7672

1

5523

07

BUDGMA0 BUDGMAU BUDGMAF 0-ADHOC BUDGMA09 14 7 4 9

423

0

39

03


414

0

6

13.4 TOP 15 Planning Layouts In the following tables, performance critical planning layouts are listed with different perspectives on performance. These planning layouts have been executed either from the planning workbench (transaction BPS0) or from planning folders. Web Layouts are currently not being analyzed. Certain time components (Open Excel Time, Data Transfer Time, Number of Cells) are not measured for ALV based layouts. In the table below the planning layouts with the highest runtime for a single execution are displayed. The BW Read Time and other time components (like 'ABAP Time') are part of the overall runtime ('RunTime(ms)'). You can identify whether a long runtime is caused for e.g. from an expensive BW access (long BW Read Time) or whether a layout has a considerable size (high number of cells). TOP 15 PLANNING LAYOUTS BY SINGLE RUNTIME

Planning Area

Planning Level

Planning Parameter Package Group

Run Time [ms]

BW Read Time [ms]

Open ABAP Excel Time Time [ms] [ms]

Data Number Transfer of Cells Time [ms]

BUDGMA0 BUDGMAU 07 9 ADHOC

BUDGMAT 4100025 485 2

187

2281

4095604

42

BUDGMA0 BUDGMAU 0-

BUDGMAT 590165

2032

2110

569196

1521

© SAP AG

E2E100

13765

8-64

Planning Area

Planning Level

Planning Parameter Package Group

7

9

ADHOC

Run Time [ms]

BW Read Time [ms]

Open ABAP Excel Time Time [ms] [ms]

Data Number Transfer of Cells Time [ms]

2


BUDGMAT 398760 2

15547

8140

258700

105218

6018


BUDGMAT 216767 2

7313

1375

3766

201642

1518


BUDGMAT 155001 2

7516

4796

6234

135423

30


BUDGMAT 153048 2

12860

1624

10985

125907

204


BUDGMAT 97827 2

12750

16578 18359

28047

11436


BUDGMAT 91061 2

15422

1172

10453

62421

147


BUDGMAT 82485 2

5188

625

66172

8703

225


BUDGMAF 68470 3

10062

12079 2547

27610

49619


BUDGMAT 67796 2

39859

3485

14078

5765

1047


BUDGMAT 67749 2

12312

3031

41077

9547

108


BUDGMAT 61610 2

15813

7030

5156

20063

9636


BUDGMAF 61452 5

468

157

38562

9406

39


BUDGMAT 59235 2

10047

578

42969

4250

225

In the table below planning layouts with the highest average runtime per execution are displayed. In comparison to the table above, statistical outliers can be eliminated by considering the number of calls as well. The number of calls indicates how often this function has been executed in the last month. TOP 15 PLANNING LAYOUTS BY AVERAGE RUNTIME

Planning Area

Parameter Group

Call Avg. Run s Time [ms]

Avg. BW Read Avg. No. Time [ms] Cells

BUDGMA07 BUDGMAU9 0-ADHOC

BUDGMAT2

118 73830

8121

914


BUDGMAT3

1

37483

4109

444

MKISKPI2

PMKISLY2

5

29763

1309

64


BUDGMAF1

3

26604

901

9


BUDGMAF3

29

15692

5358

7149

ROLFEPS

ROLFEPS5

4

10133

1802

81

© SAP AG

Planning Level

PMKISK21

ROLFEPS1

Planning Package

0-ADHOC

0-ADHOC

E2E100

8-65

Planning Area

Planning Level

Planning Package

Parameter Group


Avg. BW Read Avg. No. Time [ms] Cells

ROLFEPS

ROLFEPS1

0-ADHOC

ROLFEPS1

4

9485

3039

225

ROLFEPS

ROLFEPS1

0-ADHOC

ROLFEPS2

1

9140

750

81


BUDGMAF5

17

8855

2361

81

ROLFEPS

ROLFEPS4

3

8750

2367

27

ROLFEPS1

0-ADHOC

The table below shows planning layouts that contributed significantly to the overall SEM workload on your system. The relative runtime describes the percentage of the overall total runtime (runtime of all planning layouts) caused by this particular layout. The number of cells indicates the size of the layouts. TOP 15 PLANNING LAYOUTS BY TOTAL RUNTIME

Planning Area

Planning Level

Parameter Group


Rel. Run Time [%]

Avg. No. Cells


BUDGMAT2

118 73830

90

914


BUDGMAF3

29

15692

5

7149


BUDGMAF5

17

8855

2

81

MKISKPI2

PMKISLY2

5

29763

2

64


BUDGMAF1

3

26604

1

9

ROLFEPS

ROLFEPS1

0-ADHOC

ROLFEPS5

4

10133

0

81

ROLFEPS

ROLFEPS1

0-ADHOC

ROLFEPS1

4

9485

0

225


BUDGMAT3

1

37483

0

444

ROLFEPS

ROLFEPS1

0-ADHOC

ROLFEPS4

3

8750

0

27

ROLFEPS

ROLFEPS1

0-ADHOC

ROLFEPS2

1

9140

0

81

PMKISK21

Planning Package

0-ADHOC


14.1 System Activity The following diagrams or tables contain data about system activity: - Total Activity: Transaction steps performed each week (in thousands) - Dialog Activity: Transaction steps performed in dialog task each week (in thousands) - Peak Activity: Transaction steps in hour during the peak hour; this peak hour is calculated as the hour with the maximum dialog activity in the ST03 time profile divided by 5 (work days) - Total Users: Users that logged on in the week - Active Users: Users that have performed more than 400 transaction steps in the week SYSTEM ACTIVITY

Date

Total Activity

Dialog Activity

Peak Activity

16.04.2006

1994

535

0

23.04.2006

1555

275

0

© SAP AG

E2E100

8-66

Date

Total Activity

Dialog Activity

Peak Activity

30.04.2006

1351

276

0

07.05.2006

1605

269

0

14.05.2006

1927

463

0

21.05.2006

1568

223

0

28.05.2006

1431

183

0

04.06.2006

1738

286

0

11.06.2006

1631

302

0

17.06.2006

1782

464

0

18.06.2006

1889

464

0

23.06.2006

1402

307

0

25.06.2006

1707

329

0

02.07.2006

1910

375

0

09.07.2006

2002

474

0

16.07.2006

2391

628

0

23.07.2006

1590

278

0

30.07.2006

1557

261

0

06.08.2006

1364

245

0

13.08.2006

1721

318

0

19.08.2006

1406

236

0

20.08.2006

1638

270

0

27.08.2006

1579

245

0

02.09.2006

1298

254

0

03.09.2006

1473

269

0

10.09.2006

1831

380

0

17.09.2006

2066

565

0

24.09.2006

1874

394

0

30.09.2006

1661

374

0

08.10.2006

2179

569

0

USERS

Date

Logged on

Active

16.04.2006

2721

356

23.04.2006

2569

209

30.04.2006

2645

220

07.05.2006

2789

233

© SAP AG

E2E100

8-67

Date

Logged on

Active

14.05.2006

3048

332

21.05.2006

2765

220

28.05.2006

2540

162

04.06.2006

2895

219

11.06.2006

2753

228

17.06.2006

2813

354

18.06.2006

2830

354

23.06.2006

3049

224

25.06.2006

3084

228

02.07.2006

3268

282

09.07.2006

3167

337

16.07.2006

3199

394

23.07.2006

2754

242

30.07.2006

2937

231

06.08.2006

2966

221

13.08.2006

2808

285

19.08.2006

2763

207

20.08.2006

2780

212

27.08.2006

2854

187

02.09.2006

2861

218

03.09.2006

2886

218

10.09.2006

3018

298

17.09.2006

3033

395

24.09.2006

2985

291

30.09.2006

3088

280

08.10.2006

3170

357


Date



16.04.2006 4420

2940

727

23.04.2006 5627

3685

1009

© SAP AG


E2E100

8-68

Date




30.04.2006 4468

2503

896

07.05.2006 7047

4842

1042

14.05.2006 4870

3168

798

21.05.2006 4904

3282

806

28.05.2006 5865

4111

978

04.06.2006 6889

4586

1046

11.06.2006 6642

4250

1011

17.06.2006 6850

4766

721

18.06.2006 6787

4668

741

23.06.2006 6194

3841

939

25.06.2006 5702

3419

879

02.07.2006 5732

3270

969

09.07.2006 3502

1927

746

16.07.2006 3809

2205

719

23.07.2006 4149

2396

884

30.07.2006 4100

2351

1036

06.08.2006 4435

2534

1045

13.08.2006 4390

2495

1040

19.08.2006 4269

2420

1011

20.08.2006 5082

2704

1103

27.08.2006 5344

2758

1272

02.09.2006 5275

2870

1053

03.09.2006 5792

2870

1062

10.09.2006 5378

2477

1070

17.09.2006 4028

2051

793

24.09.2006 4553

2174

706

30.09.2006 4543

2251

865

08.10.2006 4243

2277

839

TOP 5 TRANSACTIONS

Date

RSPC

RSA1

SESSION_MANAGER

SAPMSYST

27.08.2006

25

1789

1764

4597

843

02.09.2006

24

1877

2116

6670

570

03.09.2006

23

2059

2322

10323

571

10.09.2006

182

1911

1905

2706

© SAP AG

E2E100

8-69

Date

RSPC

RSA1

SESSION_MANAGER

SAPMSYST

17.09.2006

205

2045

1955

7509

24.09.2006

1555

1949

2893

5264

30.09.2006

265

2295

2655

3728

4642

08.10.2006

742

2256

2579

5554

50538

Transaction Code

Load (%)

50.2

RSPC

14.6

RSA1

11.1

SESSION_MANAGER

6.8

SAPMSYST

1.8


Date

Availabity (%)


Update Errors

16.04.2006

100

267

0

23.04.2006

90

398

0

30.04.2006

90

323

0

07.05.2006

100

341

0

14.05.2006

99

493

0

21.05.2006

100

270

0

28.05.2006

94

1932

0

04.06.2006

90

731

0

11.06.2006

97

852

0

17.06.2006

100

653

0

18.06.2006

93

373

0

23.06.2006

100

674

0

25.06.2006

100

313

0

02.07.2006

100

236

0

09.07.2006

100

276

0

16.07.2006

91

266

0

23.07.2006

96

442

0

© SAP AG

E2E100

8-70

Date

Availabity (%)


Update Errors

30.07.2006

96

248

0

06.08.2006

85

272

0

13.08.2006

97

227

0

19.08.2006

100

219

0

20.08.2006

92

216

0

27.08.2006

94

390

0

02.09.2006

89

10

0

03.09.2006

83

777

0

10.09.2006

95

273

0

17.09.2006

96

257

0

24.09.2006

86

255

0

30.09.2006

80

275

0

08.10.2006

98

445

0


Date



09.04.2006

35

21

16.04.2006

35

20

23.04.2006

24

14

30.04.2006

26

15

07.05.2006

30

23

14.05.2006

39

18

21.05.2006

0

13

28.05.2006

0

14

04.06.2006

0

32

11.06.2006

0

15

17.06.2006

0

29

18.06.2006

0

29

23.06.2006

37

37

25.06.2006

37

37

02.07.2006

34

34

09.07.2006

26

26

16.07.2006

33

33

© SAP AG

E2E100

8-71

Date



23.07.2006

26

26

30.07.2006

32

32

06.08.2006

24

24

13.08.2006

35

35

19.08.2006

23

23

20.08.2006

23

23

27.08.2006

19

19

02.09.2006

16

16

03.09.2006

16

16

10.09.2006

28

28

17.09.2006

25

25

24.09.2006

0

27

30.09.2006

0

19

08.10.2006

0

55

© SAP AG

E2E100

8-72

Solutions Unit:

EarlyWatch Alert

Lesson 3: SAP EWA for SAP BI

After After analysis of Example EWA report for BI you can find the following problems 1-1

Design Problem of BI system 1-1-1

Design problem. In the chapter 12.3.3 you would see a large number of the InfoCubes that have a tremendous amount of entries in the dimension tables in comparison to the fact tables. The reasonable ratio between the number of entries in the dimension tables and the fact table is normally 30 %. In this report there are too much dimension tables containing too much entries. That indicates that the design of Infocubes and InfoProvides is not done in accordance with ASP rouls. It could lead to a very large Database and not optimal performance of BI. In the chapter “12.3.4 InfoCube partitioning” you can see too many F-Fact table containing a lot of partitions

1-1-2

Administration problem In the chapter “12.4.2 Aggregates suggested for deactivation” you would see a list of aggregates that are not used during query execution ("Number of calls" is equal to zero), or that are almost as large as the parent aggregate they are built from (the "Average reduce factor" is too low). In the chapter “12.6.3 BI – KPIs” you would see some reference thresholds for the BI KPIs. The “Nr of aggregates with no calls” and “Nr of Aggregates recommended to delete” indicates that an administration problem (the design of BI data model is not reviewed regularly). Because large number of not used aggregates leads to the large size of BI database.

© SAP AG

E2E100

8-73

RECOMMENDATIONS

Recommended Services

KPI

Description

Observed Reference Rating

Active Rep. Auth. (#)

Total number of active reporting authorization objects

4902

500

YELLO E2 W

Avg.Runtime MP

Average runtime of manual planning (s)

53

30

YELLO O3 W

DB Growth (GB)

Database Growth per month (GB) 171.31

25

YELLO O6 W

DB Size (GB)

Database Size (GB)

2326.34

500

YELLO O6 W

Defined process chains Total number of defined Process (#) Chains

372

200

YELLO O1, O5 W

Meta Chain executions Total number of Meta Process last week Chain executions last week (#)

916

250

YELLO O1, O5 W

NrAggr0Call (#)

Nr of aggregates with no calls

339

150

RED

NrBuildAggrCube (#)

Nr of aggregates recommended to 17 build - cube level

15

YELLO E1 W

NrDelAggr (#)

Nr of Aggregates recommended 698 to delete

200

RED

© SAP AG

E2E100

E1

E1

8-74

Solutions Unit:

EarlyWatch Alert

Lesson 3: SAP EWA for SCM / APO

1-1

After analysis of Example EWA report for SAP SCM / APOyou can find the following problems 1-1-1

In the system the RFC Problems occurred (page 15)

1-1-2

Physical Memory is not enough and that results in bad high hit ratio of OMS_DATA Page 17: Hit ratio for OMS_DATA is 99,25%. The reason for this is that the memory consumption of Data Objects (OMS DATA) is larger than the configured DATA CACHE (compare Sum of Classcontainers (pages) and Data_Cache configured (pages)) Solution in this case should be increase of physical memory

1-1-3

© SAP AG

LiveCache backup strategy is not in accordance with SAP recommendations (page 18)

E2E100

8-75

© SAP AG

E2E100

8-76

Solutions Unit:

EarlyWatch Alert

Lesson 3: SAP EWA for SAP XI

After analysis of Example EWA report for SAP XI you can find the following problems 1-1

In the section “11 XI Performance of XIP” you would see that the average message processing time during normal throughput is 1459.39 seconds. The average processing time at peak is 1063.04 seconds. These values are rather high (it is around 20 min, while normally it should be within seconds). The reason for this could be for example that customer decided to postpone message processing for night hours. Then it is not a problem. In case it is normal processing time, then tt needs to be investigated further.

1-2

In the section “11.1.1.3 Top 20 Messages by Processing Time” you would see that a lot of messages have a very high values of Latency. The messages with the high Latency time could be an indication that these messages are in error status for a long time or scheduled to be processed at nights. So, further information is required

1-3

The information from the table “Workload Distribution by Business Interfaces” is interesting from a long term analysis point of view. For example to compare the workload distribution of the same interfaces with the same throughput for different weeks. For this purpose EWAs for other period of time are required.

1-4

In the section “12.1 Messages With Errors in Previous Week” you would find several messages with error status. But here it is important to note meaning of error codes. Messages with error code 021 are canceled manually and generally could be ignored since it indicates that customer is aware of the problem. Messages with error code 014 should be investigated further to find the reason of XML error. Integration Engine

Date

Error Code

Integration Server is.00.mnxi3p

22.01.2007

014

3

23.01.2007

014

4

27.01.2007

021

187

Error Code

Description

014

System Error - Manual Restart Possible

021

Canceled Manually

© SAP AG

E2E100

No. of Errors

8-77

© SAP AG

E2E100

8-78

Solutions Unit:

EarlyWatch Alert

Lesson 3: EWA for CRM (ISA, Field Sales, CIC)

1-1

1-2

2-1

In the page 25 you would find that the used CRM Scenarios are: •

E-Commerce

•

Field Sales or Field Service

•

Interaction Center

•

CRM Online

After analysis of Example EWA report for CRM you can find the following problems 1-2-1

From a general perspective you can note that the response time of CIC transactions is relative good. Then you can also observe rather good status for Conntrans Success Statistics (no Errors and good ratio between finished and interrupted Conntans). Then you can note that customer has two Conntranses but they are not loaded equally.

1-2-2

There are several BDoc messages with technical error status, validation error status and intermediate state (pages 29-30).

1-2-3

In the page 31 you would find several Mobile Client queues that contains entries are several months old (you can detect it from the time stamp). It normally means that the Mobile Clients are not in use. In this case these Mobile Clients should be deactivated and the corresponding entries deleted from the corresponding table.

1-2-4

Several inbound and outbound R3A* queues in status SYSFAIL could be found in the page 31. If R3A* queues are in status SYSFAIL it could lead to a data inconsistency between CRM and Backend System. Because of this these queues should be restarted.

After analysis of section for CRM ISA you can find that 2-1-1

© SAP AG

There are several errors and warnings in the analyzed replication logs for some of the latest product catalog replications (page 28). It seems to no problem since that are only warnings.

E2E100

8-79

© SAP AG

E2E100

8-80


SAP System ID

TEC

SAP Component

SCM

Release

5.0

Status

Productive

DB System

MAXDB

© SAP AG

E2E100

8-81


Date of Session

13.11.2006

Date of Report

13.11.2006

Installation No.

Author


Customer No.

Session No.

© SAP AG

E2E100

8-82

4 Download Problem We were unable to perform a correct analysis because essential parts of the download data are missing. This is most likely due to a problem with data collection or transfer. Recommendation: Check the log for this session in the Service Data Control Center (execute transaction SDCC or the new SDCCN in your system). If an error is listed, perform the Service Preparation Check. Implementation: To perform the Service Preparation Check, execute transaction SDCC and choose Information -> Service prep. check, or if you are using SDCCN, create a new "Service Preparation Check" task.


This EarlyWatch Alert session has been rated RED due to critical problems. The overall session is rated RED if one of the following areas is rated RED.

o DB Performance o Database Administration: - DB growth - Missing indexes - Critical database problems

The table below lists all of your system alerts, including those critical area(s) that have led to the overall RED rating.

Note:

Only sections mentioned above and that are rated RED in the following table, can set the overall rating to RED.

Note:

All recommendations provided in this report are based on our general experience only. We advise you to test our recommendations before using them in your production system. Also note that EarlyWatch Alert is an automatic service.

Section Overview

© SAP AG

E2E100

8-83

Rating

Checked Area System Configuration System Performance Workload Distribution SAP System Operating Database Administration Remote Function Call Analysis of liveCache APO Basis APO Application Analysis Security Checks

Note:

For more information about EarlyWatch Alert, a sample EarlyWatch Alert report with explanations is available in the Media Library on SAP Service Marketplace (http://service.sap.com/ewa). This provides an overview of the check rating strategy and the KPIs that trigger the EWA alerts.

Alert Message Overview Priority

Description

New Alert

High High High High Medium Medium

Classcontainer Largest version does not fit into the configured Review your LiveCache Logdevspace archiving strategy. Review your liveCache backup strategy. Your liveCache DATA_CACHE is full. There are security issues in your system. We detected several ABAP dumps in your system.

New New New New New New

Note:

If you need help investigating the alerts listed, order an EarlyWatch contract by contacting your local support organization, or by creating a customer message in component XX-SER-TCC. If you already have such a contract, a detailed analysis can be performed during the next Service Session. Please address the topic during the preparation of the session.

Note:

If you experience serious performance problems, create a customer message on component XX-SERTCC with priority 'high' or 'very high'.

6 Performance Indicators The following table shows the relevant performance indicators in various system areas. Area

Indicators

Value

System Performance

Active Users Avg. Response Time in Dialog Task Max. Dialog Steps per Hour Avg. Response Time at Peak Dialog Hour Avg. Availability per Week Avg. DB Request Time in Dialog Task Avg. DB Request Time in Update Task DB Size Last Month DB Growth

0 3266 ms 0 0 ms 100 % 1021 ms 2433 ms 37 GB 10 GB

Database Performance Database Space Management

© SAP AG

E2E100

Trend

8-84

7 Recommended Service The analysis of your SAP EarlyWatch Alert service session has shown that your installation has a high optimization potential. Therefore, SAP would highly recommend you to perform following service: Service

Issue

Scope

Benefit

SAP Security Optimization Service

Some users have critical authorizations in certain transactions

Increase of security in your SAP System

Avoids severe security 1 problems in your SAP solution

© SAP AG

E2E100


8-85

8 System Configuration To ensure optimal support of your SAP system, the SAP components, database, and operating system used must all be supported. Detailed information about SAP's platform support is available on SAP Service Marketplace at http://service.sap.com/platforms.

Rating



Server Type

Server

OS

OS Version

Number of CPUs

Memory [MB]

Optimizer Server Optimizer Server Database Server

pserver1 pserver1.company pserver2

WIN-NT WIN-NT SOLARIS

N/A N/A 5.10

N/A N/A 2

N/A N/A 4096

DATABASE SYSTEM

Database Server

Database System

pserver2

ADABAS

Current Version

ADDITIONAL APO SERVER

APO Server Type

Server

Server ID

Server Version

LiveCache server CTM01 Optimizer DPS01 Optimizer MMP01 Optimizer SEQ01 Optimizer SNP01 Optimizer SNP02 Optimizer VSR01 Optimizer

pserver2 pserver1 pserver1 pserver1 pserver1.company pserver1.company pserver1.company pserver1.company

COE N/A N/A N/A N/A N/A N/A N/A

7.6.00.29 5.0_REL SP04, 356375, Jun 22 2006 02:50: 5.0_REL SP04, 356375, Jun 22 2006 02:48: 5.0_REL SP04, 356375, Jun 22 2006 02:49: 5.0_REL SP04, 356375, Jun 22 2006 02:49: 5.0_REL SP04, 356375, Jun 22 2006 02:49: 5.0_REL SP04, 356375, Jun 22 2006 02:49: 5.0_REL SP04, 356375, Jun 22 2006 02:49:

8.2 Software Configuration The table below shows the installed software components. The "Rating" column is filled only for the SAP R/3 Plug-In (PI) and SAP Basis Plug-In (PI_BASIS), and is based on the maintenance schedule.

© SAP AG

E2E100

8-86

COMPONENTS

Component

Component Release

Patch Level

BI_CONT

702

0004

EA-IPPE

400

0004

LCAPPS

2005_700

0002

PI_BASIS

2005_1_700

0007

QIE

200

0004

SAP_ABA

700

0007

SAP_AP

700

0004

SAP_BASIS

700

0007

SAP_BW

700

0007

SCM

500

0005

SCM_BASIS

500

0005

ST-A/PI

01I_SCM500

0000

ST-PI

2005_1_700

0002

Rating

8.2.1 SAP Kernel Release Your system is running on SAP kernel release 700, patch level 75 .

8.2.2 SCM Component check The table below shows the SCM support package and the liveCache release. To get the most recent compatible combinations of APO support packages and liveCache releases, please visit the Matrix overview in Service Marketplace at http://service.sap.com/scm; with the following path: mySAP SCM Technology -> Availibility of APO support packages/COM Builds/LiveCache Builds. Product release

Current Support Package

Current liveCache release

Current COM Build

Current Patch level

Current Changelist

5.0

5

7.6.00.29

N/a

N/a

N/a

8.2.3 Not Productive System ? There is no production client maintained in table T000. Therefore, your system is recognized as 'not productive'. Recommendation: Check with your development department whether this status is correct. If this system is a production system, check if the Customizing of this system can be changed so that table T000 contains a production client. Note: Some security mechanism are missing when the status is 'not in production operation'.


8.3.1 SAP Service Tools Before we can deliver any services, the latest version of the SAP Service tools must be available in your system. During the checks, no missing transports were detected. All tools are up to date. The last activation of report RTCCTOOL was 09.11.2006. © SAP AG

E2E100

8-87


The following table shows the average response times for various task types: Task type

Dialog Steps


Avg. CPU Time in ms



Avg. DB Time in ms

Avg. GUI Time in ms

DIALOG RFC UPDATE UPDATE2 BATCH SPOOL

5420 9024 39 5 40459 10076

3265,6 7796,5 6554,0 7706,0 4928,2 262,5

188,2 1008,1 56,9 204,0 252,5 6,4

74,1 41,5 1409,9 1009,6 55,3 49,6

342,4 70,5 154,2 186,8 41,6 3,2

1021,1 1779,2 2432,8 667,8 1099,4 74,5

97,6

9.1 Current Workload The following table lists the number of current users (measured from our workload analysis) in your system. Users

Low Activity

Medium Activity

High Activity

Total Users

Measured in System

0

0

0

0

9.2 Performance Evaluation For the task types DIALOG, UPDATE, and RFC, the measured average response times and the measured average database request times were compared with reference values. (Note: If the number of transaction steps are rather small, for example less than 20,000, an overall rating for this check is not reasonable.) Task

Dialog Steps

Dia

0

Upd

0

RFC

0




© SAP AG

E2E100

8-88

9.3.1 Transactions by Total Workload The total system workload can be defined as the sum of the total response times of all transactions. The following transaction profiles list the transactions that have the greatest share in the total workload and are sorted by the percentage of total response times.

According to our analysis, the main system load is caused by background jobs. The EarlyWatch Alert Service can only detect performance problems for dialog, update and RFC response times. This service does not evaluate performance problems for background jobs.

9.3.2 Transactions by DB load The total database workload generated from the System can be defined as the sum of the total database access times of all transactions. The following transaction profiles list the transactions that have the greatest share in the database load, sorted by percentage of total database access times.


History of Activity

[steps/1000]

100 80 60

Total Activity

40

Dialog Activity

20 0 08.10.2006

15.10.2006

22.10.2006

29.10.2006

05.11.2006

12.11.2006

Week

[ms]

History of Response Times 7000 6000 5000 4000 3000 2000 1000 0

Avg. Response Time Avg. DB Time Avg. Wait Time Avg. CPU Time Avg. GUI Time 08.10.2006

15.10.2006

22.10.2006

29.10.2006

05.11.2006

12.11.2006

Week


© SAP AG

E2E100

8-89


Rating


10.1 Workload by Application Module The following graphics or tables show how each application module contributes to the total system workload. Two workload aspects are shown: - CPU time: Total CPU load on all servers in the system landscape - Database time: Total database load generated by the application The #Customer# section contains all programs and transactions that start with Y or Z. The #Others# section contains all programs that cannot be assigned to an SAP standard application module (for example, an industry solution or a third party add-on development).

10.2 DB Load Profile The following table or diagram shows the DB load caused by Dialog, RFC, and Background tasks over different time frames. The data given in the table represents the average number of database processes occupied by each task type in the database during the given time frames. These statistics are calculated as a weekly average, which means the average values over six working days with a unit of one hour. Periods between 00:00-06:00 and 21:00-24:00 contain an average value per hour. By comparing the load profiles for dialog and background activity, you have an overview of the amount of background activity during online working hours.

© SAP AG

E2E100

8-90


Rating

Check System Availability Update Errors ABAP Dumps


11.2 Program Errors (ABAP Dumps) 147 ABAP dumps have been recorded in your system since 06.11.2006. ABAP dumps are deleted after 7 days by default. To view the ABAP dumps in your system, call transaction ST22 and choose “Selection”. Then select a timeframe.

ABAP DUMPS PER DAY

Date

Quantity of Dumps

06.11.2006 07.11.2006 08.11.2006 09.11.2006 10.11.2006 11.11.2006 12.11.2006

4 19 0 1 0 0 0

DETECTED ABAP DUMPS

Error CONVT_NO_NUMBER DBIF_DSQL2_CONNECTERR CONVT_OVERFLOW DBIF_RSQL_INVALID_RSQL SYSTEM_CORE_DUMPED LOAD_PROGRAM_LOST

© SAP AG

Number Server (e.g.) 2 6 133 3 1 2

pserver2_TEC_20 pserver2_TEC_20 pserver2_TEC_20 pserver2_TEC_20 pserver2_TEC_20 pserver2_TEC_20

E2E100

User (e.g.)

Date (e.g.)

Time (e.g.)

NEUMANNHE PETRENZ NEUMANNHE STAPI_ADMIN STAPI_ADMIN RICHTERN

06.11.2006 07.11.2006 07.11.2006 07.11.2006 07.11.2006 09.11.2006

16:38:43 01:15:48 17:00:56 22:15:05 22:30:08 09:27:38

8-91

12 Security Checks Critical security issues were detected in your system. We determined that several users have critical authorizations for certain transactions. See the information in the following checks. SAP offers a complete Security Optimization service for your system, which covers all of R/3 Basis and other SAP products. We recommend that you make use of this service. For more information, see SAP Service Marketplace at http://www.service.sap.com/SOS.

Rating


12.1 Password of One or More Standard Users is not Changed Evaluated Risk - High Recommendation: Change the standard password for users SAP* DDIC, SAPCPIC, and EARLYWATCH in all clients. Do not delete the SAP* user because this enables the use of the hard-coded SAP* user. For more information, see the SAP Security Guides on SAP Service Marketplace at: http://www.service.sap.com/securityguide.

Client

User

066 000 800 802 066

SAP* SAPCPIC SAPCPIC SAPCPIC EARLYWATCH

© SAP AG

E2E100

8-92

13 Hardware Capacity We have checked your solution for possible CPU or memory bottlenecks and found that the hardware resources have been exhausted. This could have a negative impact on the overall performance of your system. We therefore recommend analyzing this issue in more detail during an SAP EarlyWatch Check.

Server pserver2

© SAP AG

Max. CPU load [%]

Date Rating

30 08.11.2006

RAM [MB] 4096

E2E100


Date Rating

73 07.11.2006

8-93

14 Database Administration Severe problems regarding administration of your SAP DB database were found. Check section 'Database Administration' ASAP.

Rating

Check Cache Hit Ratios Database Growth Top growing Tables Database and Log Backup Frequency DB system performance SAP System Parameters relevant for SAP DB / MaxDB

14.1 Cache Hit Ratios CACHE HIT RATIOS

Data Cache

Catalog Cache

99

81

The general database performance depends to some extent on the hit ratios of the different caches, that is the data cache, the converter cache, and the catalog cache.

The performance of your database may decrease if the data cache hit ratio is below 98% or the catalog cache hit ratio is below 86%.

Before increasing the size of the data cache, you should check memory conditions and limits of the server and operating system. A high number of intensive reading and long running SQL statements could also decrease the data cache hit ratio. In this case, it is necessary to analyze the SQL efficiency of programs, that is, carry out a performance analysis on that server.

The converter cache is not analyzed any more, since for SAP DB / MaxDB releases higher than 6.2, the size of the converter cache is set automatically.

The size of the catalog cache is determined by the parameter CAT_CACHE_SUPPLY.

14.2 Database Growth No problems with database free space have been found.

© SAP AG

E2E100

8-94

13.11.2006

12.11.2006

11.11.2006

10.11.2006

09.11.2006

08.11.2006

07.11.2006

06.11.2006

05.11.2006

04.11.2006

03.11.2006

02.11.2006

01.11.2006

31.10.2006

30.10.2006

29.10.2006

28.10.2006

27.10.2006

26.10.2006

25.10.2006

24.10.2006

23.10.2006

22.10.2006

21.10.2006

20.10.2006

19.10.2006

18.10.2006

17.10.2006

16.10.2006

15.10.2006

40 30 20 10 0 14.10.2006

Size [GB]

Database Growth

Date

14.3 Top growing Tables This check could not be performed because of missing data in the download. Please follow the instructions in SAP Notes 352081 and 362542.

14.4 Database and Log Backup Frequency According to our check, your log backup strategy is not sufficient. Log backups have not been performed daily or have finished with errors. The table below shows periods longer than a day when no log backups took place. PERIODS WITHOUT LOG BACKUP

From

To

11.11.2006 09.11.2006 07.11.2006 04.11.2006 27.10.2006 25.10.2006 23.10.2006 20.10.2006 18.10.2006 16.10.2006 14.10.2006

13.11.2006 11.11.2006 09.11.2006 06.11.2006 29.10.2006 27.10.2006 25.10.2006 23.10.2006 20.10.2006 18.10.2006 16.10.2006

14.5 DB system performance The table below shows the methods used to access tables in the database, together with their efficiency in percent. Efficiency of a method is defined as the ratio of the qualified rows over read rows for the given access method. A low efficiency value may indicate some problems with index definitions or misleading table statistics.

If you notice low efficiencies, please check database response times in transaction ST03. If the transaction shows bad database response times, we recommend analysis of SQL performance. Transaction DB50 offers tools for such an analysis. © SAP AG

E2E100

8-95

Access method

Efficiency (%)

Primary key access Primary key range access Index access Index range access Isolated index access Isolated index range access Table scan Isolated index scan

90.6 31.5 2.1 79.1 100.0 51.0 27.0 87.7

14.6 SAP System Parameters relevant for SAP DB / MaxDB From the data collected in your system, the following parameter value should be adapted as follows:

rsdb/min_in_blocking_factor = rsdb/max_in_blocking_factor = 50

For more information, refer to SAP Notes 819324, 48230, and 48244.

To improve the analysis of performance issues, an option for calling up ABAP coding from SQL monitors of the database could be helpful.

For this purpose, it is necessary to store ABAP call-points and report names with SQL commands. We recommend that you activate this feature by setting the following instance profile parameter:

dbs/ada/register_appl_info = 1

Please refer to SAP Note 216208 for more information.

© SAP AG

E2E100

8-96

15 Remote Function Call 15.1 qRFC Administration 15.1.1 qRFC Queues qRFC Outbound Queues qRFC outbound queues in error status qRFC outbound queues with error statuses were detected in the qRFC Monitor. The table below lists the queues with the highest number of entries for each error status found. Recommendation: Analyze the errors according to SAP Note 378903. Monitor the qRFC outbound queues daily using transaction SMQ1.

Queue Name

Status

CFIP000100002346 CFIP000100002347 CFIP000100002348

CPICERR CPICERR CPICERR

Date of first entry 11.08.2006 11.08.2006 11.08.2006

Entries Error Message 1 / LB: Unexpected error in LG Layer. 1 / LB: Unexpected error in LG Layer. 1 / LB: Unexpected error in LG Layer.

qRFC Inbound Queues qRFC unprocessed inbound queues Some unprocessed qRFC inbound queues were detected in the qRFC Monitor. The table below lists the queues with the highest number of entries for each status found. Recommendation: Analyze the queues according to SAP Note 378903. Monitor the qRFC inbound queues daily using transaction SMQ2.

Queue Name

Status

BASIS_TEST_Q1 STV_TEST_Q1 XBTB0000 XBTR0012 XBTR0016

NOEXEC NOEXEC READY READY READY

© SAP AG

E2E100

Date of first entry

Entries

17.01.2006 17.01.2006 29.03.2006 14.03.2006 13.03.2006

4 1 2 1 1

8-97

16 Analysis of liveCache We have checked your liveCache and found some issues that might have a negative impact on your overall system performance. Review the report carefully and implement our recommendations. Furthermore, we recommend analyzing this issue in more detail during an EarlyWatch Session.

Rating

Check Memory consumption Largest Version in Classcontainers liveCache Parameters Backup of liveCache Backup of Log devspace Transactional Simulations Update Statistics

A check without rating means either the download data is missing for this particular check or the data is not representative enough for further analysis. In case of missing data, please ensure that you have the latest service tools necessary by running program RTCCTOOL to check if everything is set up correctly.

16.1 Volumes 16.1.1 LiveCache Volumes Filling Level The table below concerns the liveCache Volumes filling level. Description Total size Used area Permanent used area Temporary used area

Pages

Kbytes

Percentage

512001 278526 278503 23

4096008 2228208 2228024 184

100,00 54,40 54,40 0,00

16.1.2 Size of LOG Volumes Below is the size of LOG Volumes table, showing the size of the LOG area and LOG segment configured in the system. Descriptions

in Pages

in KBytes

Percentage (%)

59937 21333

479496 170664

100,00 35,59

LOG Size LOG Segment Size

Description

Number

Number of Log Segments

3

The LOG Volumes size in your system is too small. © SAP AG

E2E100

8-98

Recommendation: Configure LOG Volumes to at least 2 GB.

16.2 Performance COE 16.2.1 Memory consumption The hit ratio is below 99.50%. This could lead to performance problems in LiveCache.

The tables contain data concerning your liveCache Memory Consumption. HEAP_USAGE

Description OMS_HEAP_LIMIT RESERVED_HEAP Current Used HEAP HEAP_USAGE by the COM-Object

Heap (kb)

Percentage (%)

524288 231288 4250 0

100,00 44,11 0,81 0,00

ACCESSES

Cache OMS_DATA OMS_HISTORY/UNDO SQL_DATA

Accesses

Successful

Unsuccessful

Percentage (%)

81635718 16892066 132533728

81020714 16869480 132522525

615004 22586 11203

99,25 99,87 99,99

Size (Pages)

Size (kb)

Percentage

52429 50840 50049 0 791

419432 406720 400392 0 6328

100,00 96,97 95,46 0,00 1,51

USAGE OF DATA_CACHE

Cache CACHE_SIZE DATA_CACHE_USED OMS_DATA OMS_HISTORY/UNDO SQL_DATA

16.2.2 Classcontainers in Data Cache The memory consumption of Data Objects (OMS DATA) is larger than the configured DATA CACHE. The use of Planning Areas might be the reason for this. This situation is not critical if mostly one Planning Area is being used currently, and the others were last used a month ago. The situation becomes critical if all Planning Areas are used at the same time. In this case you have to expect I/O and a large decrease of performance. The hardware of the system is not enough to handle the volume of data if it is used concurrently. Please contact your hardware partner to resize the system if this is the situation. CLASSCONTAINERS VS DEVSPACES

Sum of Classcontainers (pages)

Used Perm Devspaces (pages)

Data_Cache Usage (%)

268849

278503

97,00

© SAP AG

E2E100

8-99

CLASSCONTAINERS VS DATA_CACHE

Sum of Classcontainers (pages)

Data_Cache configured (pages)

268849

52429

16.2.3 Largest Version The size of the largest version you are using does not fit in the configured data cache size. Recommendation: If the data cache cannot be increased due to physical memory being too small, contact your hardware partner.

LARGEST VERSION

Name

Size (in Pages)

Size of configured DataCache

% compared to size of Data Cache

216222

52429

412,41

800000

16.2.4 LiveCache Parameters Parameters

Description

MAXCPU

max. number of CPU assigned to liveCache Segmentation of the liveCache heap Max time a db session waits for a locked table row, 325384

OMS_HEAP_COUNT REQUEST_TIMEOUT

Current value

Recommended value

2 Please refer to note 719652 2 Please refer to note 719652 180 Please refer to note 719652

We detected that some of the liveCache parameter settings are not in accordance with standard SAP recommendations. Carefully review the recommended SAP Notes. In case there is a doubt on a specific recommendation (due to special circumstances), please open a customer message and/or investigate this further in an EarlyWatch session.

16.3 Administration of the liveCache COE 16.3.1 Backup of liveCache Your liveCache backup strategy is not in accordance with SAP recommendations. Please review your backup strategy as soon as possible. Recommendation: SAP recommends performing a full liveCache backup after each working day. The more frequently you back up your database, the shorter the duration of a potential recovery. Implementation: Use report RSLVCBACKUP to back up your liveCache according to SAP Note 455154. For further details, see service.sap.com -> Alias 'SCM' -> 'MySAP SCM Technology' -> 'Backup and Recovery' -> 'SAP liveCache 7.4: Backup, Recovery and Administrational News.'

© SAP AG

E2E100

8-100

Note: A consistency check (VERIFY) of a liveCache instance should be carried out regularly for backing up the structural data consistency, or for the concatenation of the object data. Please see SAP Note 521870 for further details.

16.3.2 Backup of LOG Volumes You have activated AutoSave Log mode but you do not perform a daily backup. Recommendation: SAP recommends that you perform a daily archive of log Volumes with AutoSave Log mode activated. To ensure that you perform a daily backup of log Volumes, decrease the LOG_SEGMENT_SIZE liveCache parameter by a factor of two. Adjust the log segments size further if this is not sufficient.

© SAP AG

E2E100

8-101

17 SAP SCM Basis 17.1 SAP SCM Frontend Checks Recommendation: The SAP SCM frontend requires a minimum of 64 MB of main memory. However, SAP recommends using at least 128 MB main memory. Response time will be negatively influenced by OS paging on frontend servers that do not meet these minimum requirements. The SAP SCM frontend requires at least a 133 MHz processor or an equivalent. However, SAP recommends having at least a 350 MHz processor or an equivalent. Check SAP Note196998 for details. The speed of the network connection between the SAP SCM application servers and the frontend servers also considerably influences the response time for SAP SCM application users.

17.2 Important SAP Notes for SAP SCM Systems To optimize system performance and to prevent data loss or system downtime, implement the following SAP Notes as soon as possible. NOTES RELEVANT FOR SAP SCM SYSTEMS

Note Description number 202344 384077 458077 425825 719652

Setting up DB connection in OSS APO: Optimizing CIF Communication For All Entries: Performance Problems in APO Demand Planning Consistency Checks 'Setting initial parameters for liveCache 7.5 or 7.6

NOTES RELEVANT FOR LIVECACHE

Note Description number 386714 487972 458221 496318 205220 516661 521870 608608 749904 872157

© SAP AG

SQLSTUDIO + DBMGUI Installation from sapserv Operating System Parametrization of the liveCache server Network problems between LiveCache and Application server Parameter Values as of LiveCache 7.4.2 Minimum size: MAXUSERTASKS in the liveCache 7.4.2 liveCache performance problems with OMS_HEAP_COUNT Consistency check (verify) of a liveCache Creating the Optimizer statistics in the liveCache mount options for liveCache/SAPDB/MAXDB on Solaris with VxFS No history data in DB50

E2E100

8-102


18.1 System Activity The following diagrams or tables contain data about system activity: - Total Activity: Transaction steps performed each week (in thousands). - Dialog Activity: Transaction steps performed in dialog task each week (in thousands). - Peak Activity: Transaction steps in thousands during the peak hour; this peak hour is calculated as the hour with the maximum dialog activity in the ST03 time profile over 5 working days per week. - Total Users: Users that logged on in the week - Active Users: Users that have performed more than 400 transaction steps in the week

Activity (Steps/1000)

System Activity 80 70 60 50 40 30 20

Total Activity Dialog Activity Peak Activity

10 0 10.11.2006

Week

Users 1

Users

0,8 0,6

Logged on

0,4

Active

0,2 0 10.11.2006

Week

18.2 Response Times The following diagram or table show the development of the response times: - Average response time in dialog task (total, DB and CPU parts) - Average response times for database requests - Average response times for the top 5 transactions

Avg. Response Time (ms)

System Performance 3500 3000 2500


2000


15 0 0


10 0 0 500 0 10 .11.2 0 0 6

Week

Database Performance Avg. Request Time (ms)

16


14 12


10 8


6 4


2 0

Time per Seq. Read in Background task [ms]

10 .11.2 0 0 6

Time per Change in Background task [ms]

Week


Top 5 Transactions 1,2 1 0 ,8 0 ,6 0 ,4 0 ,2 0 10 .11.2 0 0 6

Week

© SAP AG

E2E100

8-2

18.3 System Operation The following diagram or table shows important KPIs for system operation. Note: The column 'Availability' is filled for systems with SAP Basis 4.6 and higher.

System Operation 160 140 120 100 80 60 40

Availabity (%) Program Errors (ABAP) Update Errors

20 0 10.11.2006

Week

18.4 Hardware Capacity

[%]

Hardware Capacity 35 30 25 20 15 10 5 0

Database Server CPU Max. Load (%) Appl. Server CPU Max. Load (%)

10.11.2006

Week

© SAP AG

E2E100

8-3

© SAP AG

E2E100

8-4


SAP System ID

XIP

SAP Component

NW

Release

04

Status

Productive

DB System

ORACLE 9.2.0.5.0

© SAP AG

E2E100

8-5

Date of Session

30.01.2007

Date of Report

30.01.2007

Installation No.

Author


Customer No.

Session No.

© SAP AG

E2E100

8-6


This EarlyWatch Alert session has been rated RED due to critical problems. The overall session is rated RED if one of the following general areas is rated RED. o DB Performance o Database Administration: - DB growth - Missing indexes - Critical database problems The table below lists all of your system alerts, including the critical area(s) that have led to the overall RED rating. Note: All recommendations provided in this report are based on our general experience only. We advise you to test our recommendations before using them in your production system. Also note that EarlyWatch Alert is an automatic service. Section Overview

Rating

Checked Area System Configuration System Performance SAP System Operating Database Settings Database Administration Database server load from expensive SQL statements Security Checks XI Checks

Note: For more information about EarlyWatch Alert, a sample EarlyWatch Alert report with explanations is available in the Media Library on SAP Service Marketplace (service.sap.com/ewa). This provides an overview of the check rating strategy and the KPIs that trigger the EWA alerts. Alert Message Overview

Priority

Description

New Alert

Medium

There are security issues in your system.

New

Note: If you need help investigating the alerts listed, order an EarlyWatch contract by contacting your local support organization, or by creating a customer message in component XX-SER-TCC. If you already have such a contract, a detailed analysis can be performed during the next Service Session. Please address the topic during the preparation of the session.

Note: If you experience serious performance problems, create a customer message on component XX-SER-TCC with priority 'high' or 'very high'.


Area

Indicators

Value

Trend

System Performance

Active Users

8

steady


1459 ms

down


31

down


632 ms

down


93 %

steady


560 ms

down


704 ms

up

DB Size

85.96 GB

steady


5.16 GB

up




Scope

Benefit


Service

Issue


Analyzes and optimizes Increased user Long runtime of critical business productivity and system 1 critical transactions processes acceptance

3.1 SAP Remote Performance Optimization Higher Return on Investment - Accelerate your core business processes. - Increase user productivity and system acceptance. - Make optimal use of your hardware resources. - Train your in-house staff to expertly monitor and maintain performance. With SAP Remote Performance Optimization, you use SAP's experience to save time, reduce costs, and keep your SAP solution running optimally. For more information about the service, see SAP Service Marketplace at service.sap.com/safeguarding

© SAP AG

E2E100

8-2


Rating



Server Type

Server

OS

OS Version

Number of CPUs

Memory [MB]

Others

MNXI3P

Windows 2K

N/A

8

7975

Others

MNXITEST

Windows 2K

N/A

8

7975

Database Server

mnxi3p

WIN-NT

5.0

8

7975

DATABASE SYSTEM

Database Server

Database System

Current Version

mnxi3p

ORACLE

09.2.0

4.2 Software Configuration 4.2.1 Plug-In release Configuration The table below shows the software components installed. The 'Rating' column is filled only for the SAP R/3 Plugin (PI) and SAP Basis Plug-In (PI_BASIS) and is based on the maintenance schedule. COMPONENTS

Component

Component Release

Patch Level

PI_BASIS

2005_1_640

0006

SAP_ABA

640

0015

SAP_BASIS

640

0015

SAP_BW

350

0015

ST-A/PI

01I_BCO640

0000

ST-PI

2005_1_640

0005

© SAP AG

E2E100

Rating

8-3

4.2.2 SAP Kernel Release Your SAP kernel release (release 640, patch level 122) is not up to date. Recommendation: Ensure that you are using the latest SAP kernel. See SAP Notes 19466 and 138704. These SAP Notes refer to further SAP Notes, depending on your system characteristics.

4.2.3 Implemented Notes via Notes Assistant The following SAP notes were implemented using SAP Notes Assistant.

Note

Versio Component n

Description

0000699589

0001

BC-BMT-WFM

Events: parallel creation of instance linkages

0000708338

0003

BC-XI-IS-BPE

XI 3.0: Generating object proxies for XI interfaces

0000761197

0002

KM-KW

SR13 Default-Flag für PlainFile und HelpFile

0000777776

0005

BC-SRV-PMICEN

PMI: Performance problems due to the SPI_REORG_CS job

0000778272

0004

BC-BW

BW generation tool: "Object R3TR CLASS GP... " popup

0000778276

0004

BC-DWB-WSABA

Upgrade problems during generation of virtual interfaces

0000778712

0003

BC-SRV-PMI

PMI: Incorrect job periods during reorganization

0000781931

0005

BC-SRV-PMI

PMI: Performance improvements of the reorganization

0000808254

0002

BC-XI-IS-BPE

XI Cache: Fehler bei Anlage der Korrelationen

0000811086

0001

BC-XI-IS-IEN

Potential loss of data - check report

0000820149

0002

BC-XI-IS-IEN

Deletion of manually changed and terminated messages

0000824374

0004

BC-XI-IS-IEN

XI messages are not archived or deleted

0000831429

0003

BC-XI-IS-IEN

XI acknowledgment messages cannot be deleted

0000834651

0001

BC-XI-IS-IEN

Messages without Commit are not deleted

0000842187

0005

BC-XI-IS-IEN

Switch: deleting logically deleted message physically in adv

0000844559

0001

BC-SRV-PMI

Performance problems during end-to-end monitoring

0000859521

0004

BC-SRV-PMI

Assembler terminates

0000862587

0002

BC-XI-IS-IEN

Delete job terminates due to missing history entries

0000866896

0002

BC-XI-IS-IEN

XI messages are not archived and/or deleted

0000868656

0002

BC-BMT-WFM

Performance of CL_SWF_EXP_SERVICE>ELEMENT_GET_ATTRIBUTE

0000873474

0001

BC-MID-ICF

HTTP Client Test: Problems with SET_CDATA / GET_CDATA

0000873603

0001

BC-TWB-TSTECA

Error in Parameter Transfer from TCD to Rerecord

0000890760

0001

BC-BMT-WFM

Error accessing a message payload

0000894193

0003

BC-XI-IS-BPE

BPE-RUN: find messages that are no longer used and delete

0000898312

0002

BC-XI-IS-BPE

BPE-RUN: TIME_OUT when processing ends

© SAP AG

E2E100

8-4

Note

Versio Component n

Description

0001003363

0001

Changes in the function moduel for note test

BC-UPG-NA


4.3.1 SAP Service Tools Before we can deliver any services, the latest version of the SAP Service tools must be available in your system. During the checks, no missing transports were detected. All tools are up to date. The last activation of report RTCCTOOL was 02.01.2007.



Task type

Dialog Steps

DIALO G


Avg. CPU Time in ms



Avg. DB Time in ms

Avg. GUI Time in ms

625

1458,6

543,6

159,0

13,3

559,7 124.0

1395097

181,2

32,4

2,4

4,1

115,6 N/A

UPDAT E

1

15648,0

109,0

9297,0

1130,0

704,0 N/A

BATCH

19322

11007,3

181,2

6,5

8,8

2210,9 N/A

1936

272,6

54,6

63,4

7,5

82,7 N/A

RFC

HTTP


Users

Low Activity

Medium Activity

High Activity

Total Users

Measured in System

4

3

5

12

5.2 Transaction Profile Check The following tables show the response times and the number of dialog steps for the transactions that cause the highest workload in your system. NOTE: A business transaction normally consists of more than one dialog step. As

© SAP AG

E2E100

8-5

a rough estimate, the actual number of business transactions that are executed in your system is approximately one third of the total dialog steps shown. (Transaction CPIC/RFC is not taken into account.)

5.2.1 Transactions by Total Workload The following transaction profiles list the transactions that have the greatest share in the total workload, sorted by the percentage of total response times. WORKLOAD BY TRANSACTION (DIALOG/UPDATE)

Transaction

Type

Dialog Steps

Total


Avg. CPU Time in ms

Avg. DB Avg. GUI Time in ms Time in ms

2562

100,0

568,0

173,9

199,3

124,0

SAPMHTTP

HTT P

1936

36,3

272,6

54,6

82,7

0,0

SXI_MONITOR

DIA

199

34,2

2503,4

1033,6

1029,5

296,8

SXMB_ADMIN

DIA

92

16,4

2601,1

1272,2

1130,3

92,1

DIA

8

6,6

12079,0

23,6

8,9

0,0

SESSION_MANAGE DIA R

49

2,7

797,1

138,5

459,5

137,3

RSM13000

UPD

1

1,1

15648,0

109,0

704,0

0,0

SWF_XI_SWPR

DIA

24

0,5

288,3

102,8

54,5

72,0

ZCONTA

DIA

1

0,4

5217,0

609,0

4747,0

0,0

RSABAPPROGRAM DIA

104

0,3

46,3

16,8

23,4

0,5

12

0,3

382,7

156,3

91,6

59,8

SXI_CACHE

DIA

0.4% of the "Total Resp. Time" in the above table is caused by customer transactions. WORKLOAD BY TRANSACTION (BATCH/SPOOL)

Transaction

Total Avg. Resp. Resp. Time in ms Time in %

Typ Dialog e Steps

Total

Avg. CPU Time in ms

Avg. DB Time in ms

19264

100,0

10997,1

176,7

2186,2

SPI_PS_ASSEMBLE_DSP

BTC

152

63,5

884835,2

1902,2

29165,3

SPI_REORGANISATION_CS

BTC

4

12,9

6806350,8

35773,5

2806520,0

RSDBAJOB

BTC

11

7,4

1418624,2

55,5

423,9

SPI_CS_COLLECT

BTC

57

1,9

69994,3

3471,4

61237,3

RSXMB_ARCHIVE_MESSAGES

BTC

1

1,7

3693975,0

817344,0

2822139,0

ZRSWWWIDE_TOPLEVEL

BTC

5

1,6

665844,4

34865,6

564793,8

SXMS_REFRESH_ADAPTER_STAT BTC US

12

1,4

254592,1

12351,7

243390,3

SXMS_PF_AGGREGATE

BTC

114

1,4

25912,4

86,6

25220,4

SPI_REORGANISATION_PS

BTC

27

1,2

94813,1

1266,3

92763,4

RSBTCRTE

BTC

5013

1,2

501,0

29,4

74,6

© SAP AG

E2E100

8-6

1.6% of the "Total Resp. Time" in the above table is caused by customer transactions. According to our analysis, the main system load is caused by background jobs. The EarlyWatch Alert Service can only detect performance problems for dialog, update and RFC response times. This service does not evaluate performance problems for background jobs.

5.2.2 Transactions by DB Load The following transaction profiles list the transactions that have the greatest share in the database load, sorted by percentage of total database access times. DATABASE LOAD BY TRANSACTIONS (DIALOG/UPDATE)

Transaction

Type Dialog Steps

Total

Total DB Time in % Avg. DB Time in ms

2562

100,0

199,3

199

40,1

1029,5

1936

31,4

82,7

SXI_MONITOR

DIA

SAPMHTTP

HTTP

SXMB_ADMIN

DIA

92

20,4

1130,3

SESSION_MANAGER

DIA

49

4,4

459,5

ZCONTA

DIA

1

0,9

4747,0

RSABAPPROGRAM

DIA

104

0,5

23,4

RSXMB_RESET_ADAPTER_STATUS DIA

8

0,4

274,1

SMQR

DIA

53

0,4

34,1

DB02

DIA

3

0,3

444,3

SWF_XI_SWPR

DIA

24

0,3

54,5

0.9% of the "Total DB Time" in the above table is caused by customer transactions. DATABASE LOAD BY TRANSACTIONS (BATCH/SPOOL)

Transaction

Type Dialog Steps Total DB Time in % Avg. DB Time in ms

Total

19264

100,0

2186,2

SPI_REORGANISATION_CS

BTC

4

26,7

2806520,0

SPI_PS_ASSEMBLE_DSP

BTC

152

10,5

29165,3

SPI_CS_COLLECT

BTC

57

8,3

61237,3

SXMS_REFRESH_ADAPTER_STATUS BTC

12

6,9

243390,3

SXMS_PF_AGGREGATE

BTC

114

6,8

25220,4

ZRSWWWIDE_TOPLEVEL

BTC

5

6,7

564793,8

RSXMB_ARCHIVE_MESSAGES

BTC

1

6,7

2822139,0

SPI_REORGANISATION_PS

BTC

27

5,9

92763,4

(B)OTHER

BTC

9256

3,4

153,9

RSAL_BATCH_TOOL_DISPATCHING

BTC

58

3,0

21789,2

6.7% of the "Total DB Time" in the above table is caused by customer transactions.

© SAP AG

E2E100

8-7



© SAP AG

E2E100

8-8

6 SAP System Operating Your system was analyzed with respect to daily operation problems. We did not detect any major problems that could affect the operation of your SAP System.

Rating



6.2 Transports The following graphics contain information about the number of objects which were transported to the SAP system. The three graphics/tables show objects in total, programs and table (content). TRANSPORTED OBJECTS

Date

Total Number of Transported Objects

SAP Objects

Customer Objects

Objects in Support Package

22.01.200 7

0

0

0

0

23.01.200 7

0

0

0

0

24.01.200 7

0

0

0

0

25.01.200 7

0

0

0

0

26.01.200 7

0

0

0

0

27.01.200 7

0

0

0

0

28.01.200 7

0

0

0

0

29.01.200 7

2

0

2

0

© SAP AG

E2E100

8-9

TRANSPORTED PROGRAMS

Date


SAP Programs

Customer Programs


22.01.200 7

0

0

0

0

23.01.200 7

0

0

0

0

24.01.200 7

0

0

0

0

25.01.200 7

0

0

0

0

26.01.200 7

0

0

0

0

27.01.200 7

0

0

0

0

28.01.200 7

0

0

0

0

29.01.200 7

0

0

0

0

TRANSPORTED TABLES

Date

Total Number of Transported Tables

SAP Tables

Customer Tables

Tables in Support Packages

22.01.2007

0

0

0

0

23.01.2007

0

0

0

0

24.01.2007

0

0

0

0

25.01.2007

0

0

0

0

26.01.2007

0

0

0

0

27.01.2007

0

0

0

0

28.01.2007

0

0

0

0

29.01.2007

1

0

1

0


Date

© SAP AG


2

25.01.2007

2

26.01.2007

3 E2E100

8-10

Date


2

28.01.2007

2

29.01.2007

2

DETECTED ABAP DUMPS

Error

Number Server (e.g.)

TIME_OUT

1 mnxi3p

EXSORT_NOT_ENOUGH_MEMORY

12 mnxi3p

User (e.g.)

Date (e.g.)

Time (e.g.)

XISUPER

26.01.2007

02:31:18

ADMIN

29.01.2007

20:30:41

It is important that you monitor ABAP dumps using transaction ST22 on a regular basis. If ABAP dumps occur, you should determine the cause as soon as possible. Based on our analysis, we expect no serious problems at the moment.

7 Security Checks Critical security issues were found in your system. Multiple users with critical authorizations for certain transactions were found. See the information in the following checks. SAP offers a Security Optimization Self Service. This service is provided free of charge in the current version of SAP Solution Manager. We recommend that you make use of this service. For more information, see SAP Service Marketplace at www.service.sap.com/SOS.

Rating


7.1 Password of One or More Standard Users is not Changed Evaluated Risk - High Recommendation: Change the standard password for the users SAP* DDIC, SAPCPIC, and EARLYWATCH in all clients. Do not delete the SAP* user because this enables the hard-coded SAP* user to be used. For more information, see the SAP Security Guides on SAP Service Marketplace at www.service.sap.com/securityguide.

Client

User

100

SAPCPIC

© SAP AG

E2E100

8-11

8 Hardware Capacity We have checked your system for potential CPU or memory bottlenecks and found that the hardware is sufficient for the current workload.

9 Database Settings We have detected some problems with the settings of the database. These settings may affect performance.

Rating

Check Missing Indexes Database Key Performance Indicators Setup of the Temporary Tablespace Database Parameters Optimizer Statistics


User name

Executions

Disk reads

Buffer gets

Rows processed

total (not a user name!)

9022893

12595221

225266605

38032147

SYS

5%

2%

1%

1%

SYSTEM

0%

0%

0%

0%

SAPXIP

91%

92%

95%

92%

© SAP AG

E2E100

8-12

User name

Executions

Disk reads

Buffer gets

Rows processed

OPS$MONTENEGRO\SAPSERV

0%

0%

3%

1%

SAPXIPDB

4%

6%

1%

5%

9.2 Database Key Performance Indicators Performance Indicator

Description

Observed Value

Reference Value

log file sync

av. wait time for a commit

41

< 40

This table summarizes one or more key parameters for the performance of the database, based on statistics from the dynamic performance views v$system_event and v$sysstat, collected since the last database startup. Any values that are not within the mentioned limit indicate severe problems.

9.2.1 High Average Response Time per Commit The commit time of the processes in your system is too high. The system has to wait for the redo entry to be flushed to disk, which increases the database response time of your system. See 'log file sync' in the table above. Recommendation: Check the hardware components in your system that are part of the I/O related to the online redo log files. For example: - Check for heavily used data files on the same disks as the online redo log files. - Use fast disks or/and disk striping over more than one disk. - Check whether RAID technology with limited WRITE performance is used for the online redo log files. - Check the order in which the redo log files are used: To avoid contention, two successive redo log files should not be located on the same disk. Other possible reasons for high commit times are: - A slow database writer can cause the logwriter to wait. In this case, the message 'checkpoint not complete' is written to the Oracle alert file. - If the Oracle parameter log_buffer is set too high, this can increase the wait time. If you need further assistance, contact your hardware vendor.

Note number

Description

0124361

Oracle database parameter setting for R/3 4.x

9.3 Database Parameters We noticed that the following database parameters, which are currently active in your system, do not follow our standard recommendations. These standard recommendations cannot take into account possible special aspects for your system. Deviations from those standard recommendations may be necessary in your case. We therefore advise you to discuss theses parameter settings in your next Service session.

Current value

Parameters

Description

_b_tree_bitmap_plans

Enable use of bitmap plans for tables w. only B-tree indexes


_eliminate_common_subexpr

Elimination of common sub expression

Please check SAP notes 814514 and 822638.

_sort_elimination_cost_ratio

Efficient index with FIRST_ROWS and ORDER BY


_optimizer_join_sel_sanity_chec Checks the order of join conditions k


filesystemio_options

Please check SAP note

© SAP AG

Filesystem I/O options E2E100

Recommended value

8-13

Parameters

Current value

Description

Recommended value 0124361

For parameter filesystemio_options, see the following SAP Notes for restrictions: LINUX: See SAP Note 914177

Parameters

Description

Current value

_optim_peek_user_binds

Bind value peeking

Recommended value Please check SAP note 0124361

To provide special functionality or control system behavior, Oracle has a number of events. These can be activated in the configuration file and/or dynamically in the server manager or sqlplus. In general, these events and parameters should only be activated on request. When they are no longer needed, they should be removed. However, there might be the need to activate some events during normal database operation. The following table displays the values for the parameter "event" as found in your system:

Parameter

Current Value

event

10183 trace name context forever, level 1: 10191 trace name context forever, level 1


Event


Description

10027

596420


10028

596420


The following table lists events we found to be activated in your system, but which should only be activated in special circumstances. Please check whether there are special reasons to set these events. If they are no longer needed, they should be removed.

Event 10183

9.4 Order of Redo Log Files Using the V$LOG and V$LOGFILE Oracle views, we determined the name and the order of the redo log files as listed in the following table. Note that the files are ordered by "sequence number" as found in V$LOG. This is because the sequence number indicates the actual order in which the redo log files are used, and NOT the group number.

GROUP#

SEQUENCE#

File Name

3

4223

C:\ORACLE\XIP\ORIGLOGA\LOG_G13M1.DBF

3

4223

C:\ORACLE\XIP\MIRRLOGA\LOG_G13M2.DBF

2

4224

E:\ORACLE\XIP\ORIGLOGB\LOG_G12M1.DBF

2

4224

E:\ORACLE\XIP\MIRRLOGB\LOG_G12M2.DBF

© SAP AG

E2E100

8-14

GROUP#

SEQUENCE#

File Name

1

4225

C:\ORACLE\XIP\ORIGLOGA\LOG_G11M1.DBF

1

4225

C:\ORACLE\XIP\MIRRLOGA\LOG_G11M2.DBF

4

4226

E:\ORACLE\XIP\ORIGLOGB\LOG_G14M1.DBF

4

4226

E:\ORACLE\XIP\MIRRLOGB\LOG_G14M2.DBF


In the checks performed, no problems regarding the administration of your database were found.

Rating

Check Database Growth Freespace in Tablespaces Space Critical Objects brconnect -f check (sapdba -check) schedule Multibyte Character Sets


Date

© SAP AG


55,53

28.02.2006

58,84

31.03.2006

61,43

30.04.2006

51,87

31.05.2006

55,82

30.06.2006

58,66

31.07.2006

58,85

31.08.2006

62,84

30.09.2006

70,57

31.10.2006

71,45

30.11.2006

83,76

E2E100

8-15

Date


80,80

29.01.2007

85,96


Date


53,44

28.02.2006

50,15

31.03.2006

47,55

30.04.2006

57,19

31.05.2006

46,41

30.06.2006

43,57

31.07.2006

44,39

31.08.2006

42,83

30.09.2006

38,10

31.10.2006

41,70

30.11.2006

33,14

31.12.2006

42,39

29.01.2007

40,97


Date

© SAP AG

Current Size in GB


28.02.2006

58,84

3,31

31.03.2006

61,43

2,59

30.04.2006

51,87

-9,56

31.05.2006

55,82

3,95

30.06.2006

58,66

2,84

31.07.2006

58,85

0,19

31.08.2006

62,84

3,99

30.09.2006

70,57

7,73

31.10.2006

71,45

0,88

30.11.2006

83,76

12,31

31.12.2006

80,80

-2,96

29.01.2007

85,96

5,16

E2E100

8-16

10.2 Oracle Release We recommend that you install the latest Oracle patch set as soon as possible (see SAP Note 415900). Check SAP Note 539921 regularly for the latest patch set for Oracle 9.2.0. Note that in addition to the patch set, further bug fixes may be available and recommended. Refer to the following SAP Notes: For patch set 9.2.0.3, 610445 Merge fix for Oracle 9.2.0.3. For patch set 9.2.0.4, 695080 Merge fix for Oracle 9.2.0.4. For patch set 9.2.0.5, 755629 KKO merge fix for Oracle 9.2.0.5. For patch set 9.2.0.6, 834100 Merge fix for Oracle 9.2.0.6. For patch set 9.2.0.7, 896903 Optimizer Mergefix for Oracle 9.2.0.7.


Note number

Description

858869


871455


11 XI Performance of XIP Performance issues have been found which may affect productive operation of your XI installation. Check the relevant sections below for more details.

RATING OVERVIEW

Rating

Check Integration Server Performance Overview XML Errors

PERFORMANCE INDICATORS

Description

Value

Period Analysed

22.01.2007 - 24.01.2007

Total Messages for Period

43486

Average Number of Messages/day

6212.10

Average Message Processing Time (seconds)

1459.39

Number of Active Interface Pairs

147

XML Errors as % of Total Messages

0.45

© SAP AG

E2E100

8-17

Note: The XI runtime distinguishes between net and gross processing times when specific Basis Support Packages are installed. This is useful in scenarios where delayed message processing is configured. The "delay" time is removed and the remaining processing time is shown. The requirements for displaying net processing times in service reports have not been met on XIP. The requirements for displaying net times are described in SAP Note 970124. Gross times are shown where the terms 'processing time' or 'latency time' are used for XI messages. and we could not determine whether the requirements for displaying net times are in place. For more information about these requirements, see SAP Note 970124.

11.1 Integration Server is.00.mnxi3p 11.1.1 Integration Server Performance Overview XI message processing performance is not within expectations. Please see the sections below for more detail. The average message processing time during normal throughput is 1459.39 seconds. The average processing time at peak is 1063.04 seconds.

11.1.1.1 Weekly Load Profile The message volume per day during the week 22.01.2007 - 24.01.2007 is shown in the graphic below. The average number of messages processed in a day is 6212.1.

11.1.1.2 Load Profile on Peak Day The graphic below shows the number of messages per hour on 24.01.2007, which was the peak day of the week analyzed. The peak processing hour was from 09:00 to 10:00. Throughput during that period reached 5580 messages. The average processing time per message during the peak hour was 1063.04 seconds.

© SAP AG

E2E100

8-18

11.1.1.3 Top 20 Messages by Processing Time

Message ID

Latency (sec)

Timestamp

Interface ID

AEA61CF367F14746837CCA1E37408F57

172794

20070127133150

1

A0EE1FAE3BB27C4A83A164A151FED2DC

172758

20070127133151

1

2A80372E84E01E4E8AA9387112AD58A8

172479

20070127133151

1

B2393D1FC06F834DA420597BF9246A32

172403

20070127133151

1

27A215CF2C34C549A395B47CC602B796

172331

20070127133152

1

11D750E684F9044BA4DDA1B2E174DC88

172281

20070126072911

1

23042EA4AC61D740997B84010102252D

172263

20070127133152

1

BBA0385BCC63B24BA9A108294AC6B305

172223

20070126072912

1

© SAP AG

E2E100

8-19

Message ID

Latency (sec)

Timestamp

Interface ID

64C4748FE8656D48903ED10631280487

172210

20070127133152

1

0DCE2791B641A443B957C32B144730CE

172122

20070127133152

1

B0DBE19A38A4F64E94DF1EFB28F5A282

172119

20070126072912

1

A1CF16DB1A70C34A8EC69378DB45DA4B

172083

20070127133153

1

E505B949943B5849846921C61AF13DAF

172052

20070126072912

1

16BE57DF7785E745BC51277C44FBA124

171912

20070127133153

1

FF92FB83CE812D41914D50CB2C2180D7

171621

20070127133153

1

382FD0C9B489EB43B8878CDF99F3CEC6

171518

20070126072912

1

29BFA186C2FDF34CB604D5E09F7D77AB

171483

20070126072913

1

FCA08570AD5E65499848D255E8180221

171334

20070126072913

1

C219B72EE17E874381DBEB9CF3CFC24E

171308

20070127133154

1

AC6A03CC11F65748B15919737FB0FAE3

171298

20070126072913

1

SENDER INTERFACE DETAILS

ID

Sender System

Sender Interface Name

1

PRD_200_XIP

DEBMAS.DEBMAS06.ZDEBMAS06

RECEIVER INTERFACE DETAILS

ID

Receiver System

Receiver Interface Name

1

Workload Distribution by Business Interfaces WORKLOAD DISTRIBUTION BY BUSINESS INTERFACES

Sender

Number Total of Processin Message g Time s

Receiver

Averag Average e Processin Messag g Time e Size (Seconds) (Bytes)

ZXI_TABAFCS_SISINFO.ZXISI dw01afcs_to_Sisinfo S

4776 5.566.872

1165,6

916

MI_bpmMovOrdini

MI_movOrdini

4298

43.608.05 5

10146,1

16994

ORDRSP.ORDERS05

MI_bpmMovOrdini

3877

657.320

169,5

24742

MI_bpmReqChekOrderHold

MI_verifyIfOrderHold

3876

4.077

1,1

17604

MI_exportSalesOrder

MI_exportSalesOrdersAbstract

1830

23.902

13,1

48813

MI_AddSalesOrderAbs

MI_AddSalesOrder

1702

39.167

23,0 143765

MI_AddSalesOrderAbsResponse MI_exportSalesOrderAck

1702

47.483

27,9

18486

MATMAS.MATMAS05

1381

661.080

478,7

12418

© SAP AG

AnagArticoliToQUALIWARE

E2E100

8-20


Sender

Receiver

MATMAS.MATMAS05

AnagMateriali_to_SISINFO

1198 1.397.658

MATMAS.MATMAS05

MainDataMaterial_to_SM1

1196

MATMAS.MATMAS05

MI_Materiali_to_FIBO

MI_MainDataOrdersAbs

Averag Average e Processin Messag g Time e Size (Seconds) (Bytes) 1166,7

12925

653.723

546,6

12904

1158

846.239

730,8

13178

MI_MainDataOrders

1151

116.779

101,5

3925

INVOIC.INVOIC02.INVOIC02

MI_bmpINVOIC

1149

51.194

44,6

93312

MI_bpmReqCheckInvoicHold


1149

2.144

1,9

4286

DESADV.DELVRY03.DELVRY MI_SendDelivery 03

1094

19.518

17,8

34248

MATMAS.MATMAS05

MI_Materiali_to_CABL

1083

844.432

779,7

13262

MI_bmpINVOIC

MovFatture_to_SISINFO

1078

99.271

92,1

97722


MI_Invoice2OrdersAbs

1065

50.383

47,3

52348

MI_exportInvoicesAbs

MI_exportInvoices

1064

32.841

30,9

4236

MI_requestMgmSalesOrderAbs

MI_MgmSalesOrder

699

2.092

3,0

21348

ZXI_TABELLE_SM1.ZXISM1

MainDataArtDecode

650

94.460

145,3

737

MATMAS.MATMAS05

MI_AnagMateriali2Ozzano

543

267.320

492,3

15202

SHPADV.SHPMNT05

MI_SHIPMENTAbs

448

10.790

24,1

98365

DEBMAS.DEBMAS06.ZDEBM MI_chkOrdgcDebmasABS AS06

397

6.481

16,3

6509

MI_chkOrdgcDebmasABS

394

6.648

16,9

7973

DEBMAS.DEBMAS06.ZDEBM AnagClienti_to_SISINFO AS06

394 4.197.791

10654,3

6562

MATMAS.MATMAS05

MI_AnagMateriali2Dolzago

339

121.271

357,7

17028

SHPADV.SHPMNT05

OrdiniPrelievo_to_TNT

335

35.007

104,5

93353

ORDERS.ORDERS05.ORDERS0 rcvBpmListeOdA 5

294

4.327

14,7

7403

MI_ConsegneOutbound

276

2.612

9,5

12661

MI_LookupConsegneAbstract_UP MI_DeliveryUpdate DA

273

894

3,3

35234

MI_LookupConsegneAbstract

MI_DeliveryUpdate

273

688

2,5

35076

MI_requestQryT100

MI_bpmQryT100

245

3.469

14,2

4065

MI_bpmQryT100

Mail

156

685

4,4

5464

MI_lookupSalesOrderResponseAb MI_exportSalesOrderAck s

127

3.610

28,4

1179

MATMAS.MATMAS05

MI_Materiali

127

26.335

207,4

14353

MI_requestLockSalesOrderAbs

MI_LockSalesOrder

120

772

6,4

1537

© SAP AG

MainDataCustomer_to_SM1

MI_ConsegneAbstract

E2E100

8-21



Sender

Receiver

MI_callBAPI_ACC

MI_rcvCallZBAPI_ACC_GL_PO STING

91

99

1,1

6943

MI_bpmRequestQryBKPF

MI_excQryBKPF

91

849

9,3

2157

MainDataCustomer_from_SM1

MainDataCustomerAbstract_from _

86

579

6,7

8383

MI_EnhancDataCustomerAbs

DEBMAS.DEBMAS06.ZDEBM AS06

86

537

6,2

2961

MI_getEnhancDataCustomer

MI_readEnhancDataCustomer

86

89

1,0

6870

MI_sndBpmZRFC_FI_VERSAG MI_rcvCallZRFC_FI_VERSAGE E_SM1 _SM1

66

349

5,3

5813

MI_sendVersamAgentiSM1

MI_rcvBpmVersamAgenti

66

214

3,2

8592

ZCE1ARGM_MESS.ZCE1ARG M

MI_Tab_Fatturato

60

23.244

387,4

167644 8

MI_bpmReqExc_PraticheLegUpg MI_exc_PraticheLegUpdFlowR3 Flo

49

60

1,2

835

DEBMAS.DEBMAS06.ZDEBM AnagFiliali_to_SISINFO AS06

47

624.497

13287,2

4370

WMMBXY.WMMBID02

MovimentiToQUALIWARE

42

744

17,7

1843

MI_bmpINVOIC

MovResiMerce_to_SISINFO

40

1.025

25,6

67223

ZEDI_INVOIC.INVOIC02.INVO MI_Invoice2EDI IC02

32

5.405

CREMAS.CREMAS04

AnagFornitoriToQUALIWARE

31

183

5,9

3209

MI_ConsegneResponseAbstract

Mail

30

131

4,4

10612

MI_bpmChkZSD_ESITI

MI_ChkZSD_ESITI

29

70

2,4 257316

MI_bpmUpdZSD_ESITI

MI_UpdZSD_ESITI

27

134

5,0 232415

MI_bmpINVOIC

MovNoteDC_to_SISINFO

25

213

8,5

MI_File_EC2SAP

MI_bpmFile_EC

23

123

5,3 126903

MI_bpmLoadTemp_Table_EC

MI_LoadTemp_Table_EC

21

40

1,9 198966

MI_bpmFile_EC

MI_File_EC2NextProcess

21

195

9,3 124317

MI_bpmexcProc_xi_ec_load

MI_excProc_xi_ec_load

21

30

1,4 124701

MI_bpmLoadTemp_Table_PC

MI_LoadTemp_Table_PC

20

25

1,2

3699

MI_bpmexcProc_xi_pc_load

MI_excProc_xi_pc_load

20

21

1,0

2836

ZXI_DELIVERY.ZXIDLRY

MI_DeliveryRollback

19

124

6,5

696

sndBpmListeOdAfinale

rcvListeOdAfinale

18

90

5,0

29477

MI_File_EC_CABL

MI_bpmFile_EC

18

789

43,8

9431

© SAP AG

E2E100

168,9 167088

32834

8-22



Sender

Receiver

ZXI_END.ZXI_END

rcvBpmEndTransfer

18

48

2,7

602


rcvListeOdAfinaleCopiaLocale

18

89

5,0

29477

MI_File_PC_CABL

MI_bpmFile_PC

17

781

45,9

2050

MI_NoteSpese

MI_bpmNoteSpese_wrkAbs

10

30

3,0

19981

SHPADV.SHPMNT05

OrdiniPrelievo_to_CABL

10

76

7,6

14150

MI_bpmResponseBAPI_ACCfor Mail MailA

10

29

2,9

2333

MI_bpmResponseUpdZSD_ESITI Mail

9

33

3,6

10361

ZPARTITE2SM1.ZPARTITE2S M1

MI_PartiteContab2SM1

9

1.087

120,7

309933 0


MI_DeliveryUpdate

7

19

2,7

4307

MI_Detdoc2leg

7

10

1,4

0

DESADV.DELVRY03.DELVRY OrdiniReso_to_TNT_CopiaLocal 03 e

7

46

6,6

9060

ZGELPRATICHE.ZGELPRATIC MI_Detdoc2leg HE

7

32

4,6

2735

MI_ConsegneOutbound

7

390

55,8

1574

DESADV.DELVRY03.DELVRY OrdiniReso_to_TNT 03

7

48

6,9

9060

MI_Pratica2leg

6

14

2,4

0

ZGELPRATICHE.ZGELPRATIC MI_Cntrparti2leg HE

6

16

2,7

2841

MI_bpmexcProc_xi_esiti_ok

6

383

63,8

1453

MI_Lookup_EntrataMerci_Abstra MI_GoodsMovementCreate c

6

11

1,8

34228

MI_bpmCallUpdLvlMovContab

MI_CallUpdLvlMovContab

6

7

1,1

1823

MI_bpmCallZDATI_FATTURA

MI_CallZDATI_FATTURA

6

5

0,8

4677

ZFIDO2SM1.ZFIDO2SM1

MI_Fidi

6

86

14,4 153791

MI_EsitiOBLOGISTICA

MI_bpmFile_EC

6

26

4,3 174237

MI_EntrataMerci_Outbound

MI_EntrataMerci_Abstract

6

37

6,1

5339

ZGELPRATICHE.ZGELPRATIC MI_Pratica2leg HE

6

19

3,2

2841

MI_bmpINVOIC

6

24

4,0

24344

MI_Cntrparti2leg

6

16

2,6

0

ZXI_TABELLE_SISINFO.ZXISI dw01acpg_to_Sisinfo S

5

178.168

© SAP AG

MI_ConsegneAbstract

MI_excProc_xi_esiti_ok

MovResiContab_to_SISINFO

E2E100

35633,6 117261

8-23

Sender


Receiver


ZXI_TABELLE_SISINFO.ZXISI dw01aspr_to_Sisinfo S

5

178.142

35628,3 101163

ZXI_TABELLE_SISINFO.ZXISI dw01agrp_to_Sisinfo S

5

178.138

35627,6

71020

ZXI_TABELLE_SISINFO.ZXISI dw01afcn_to_Sisinfo S

5

178.182

35636,5

11829

ZXI_TABELLE_SISINFO.ZXISI dw01afcp_to_Sisinfo S

5

178.184

35636,8 316511

ZXI_TABELLE_SISINFO.ZXISI dw01avet_to_Sisinfo S

5

178.160

35631,9

57093

MI_bpmDETDOC

MI_Detdoc2leg

5

6

1,1

1780

ZXI_FATSM1.ZXIFAT

MI_fatturePassive2SM1

5

138.955

27791,0

27539

FIDCC2.FIDCCP02.FIDCCP02

MI_bpmDETDOC

5

14

2,7

6156

ZXI_TABELLE_SISINFO.ZXISI dw01alp2_to_Sisinfo S

5

178.129

35625,7

39781

ZXI_TABELLE_SISINFO.ZXISI dw01acpr_to_Sisinfo S

5

178.176

35635,2 138665

ZXI_TABELLE_SISINFO.ZXISI dw01attp_to_Sisinfo S

5

178.141

35628,1

41757

ZXI_TABELLE_SISINFO.ZXISI dw01adpr_to_Sisinfo S

5

178.168

35633,7

21995

ZXI_TABELLE_SISINFO.ZXISI dw01afpr_to_Sisinfo S

5

178.185

35637,0 224909

ZXI_TABELLE_SISINFO.ZXISI dw01acmp_to_Sisinfo S

5

178.180

35636,0 277329

ZXI_TABELLE_SISINFO.ZXISI dw01ascl_to_Sisinfo S

5

178.185

35637,0 230992

ZXI_TABELLE_SISINFO.ZXISI dw01aumi_to_Sisinfo S

5

178.158

35631,7 235877

ZXI_TABELLE_SISINFO.ZXISI dw01asgr_to_Sisinfo S

5

178.138

35627,7

40195

ZXI_TABELLE_SISINFO.ZXISI dw01asfp_to_Sisinfo S

5

178.140

35628,0

59533

ZXI_SEMAFORO.ZXISEM

4

120.934

30233,6

705

MI_ZWsDeliveryUpdate2_Abstra MI_ZWsDeliveryUpdate2 ct

4

13

3,2

16242

MI_bpmexcProc_xi_dc_load

4

4

1,0

1668

MI_EntrataMerciReso_Outbound MI_EntrataMerciReso_Abstract

4

34

8,6

5362

MI_bpmLoadTemp_Table_DC

4

5

1,1

2009

© SAP AG

MI_Semaforo

MI_excProc_xi_dc_load

MI_LoadTemp_Table_DC E2E100

8-24

Sender

Receiver

MI_File_DC_CABL

MI_bpmFile_DC


ZXI_TABELLE_SISINFO.ZXISI dw01azve_to_Sisinfo S

4

26

4

178.263

Averag Average e Processin Messag g Time e Size (Seconds) (Bytes) 6,4

714

44565,8 833761

MI_filepraticheLegUpdFlowR3

MI_bpm_filePraticheLegUpdFlo wR

3

20

6,8

4376

MI_GiacenzeFromCABL

MI_GiacenzeToSAP_CABLB20

3

28

9,4

50977

DESADV.DELVRY03.DELVRY ListeIngresso_to_TNT 03

3

13

4,4

22026

MI_File_OK_CABL

MI_bpmFile_OK

3

31

10,3

244

MI_GiacenzeFromTNT

MI_GiacenzeToSAP

3

51

17,1

383547 1

MI_GiacenzeFromCABL


3

31

10,2

50977


MainDataT053_to_SM1_new

3

1.640

546,8

243404 9

MI_GiacenzeFromCABL

MI_GiacenzeToSAP_CABLM10

3

35

11,8

50977

DESADV.DELVRY03.DELVRY ListeIngresso_to_TNT_CopiaLoc 03 a

3

14

4,6

22026

COND_A.COND_A02

XmlListDettToSM1

3

4

1,2

1474

MI_EsitiGRENDI

MI_bpmFile_EC

3

15

4,9 248990

MI_File_EC_EURO

MI_bpmFile_EC

2

14

7,0

11329

2

3

1,4

1194

MI_lookupSalesOrderResponseAb MI_exportSalesOrderAck s Z_HYDRA.ZHYDRA

MovimentiAccise_to_HYDRA_S Lazz

2

20

9,9 643792

Z_HYDRA.ZHYDRA

MovimentiAccise_to_HYDRA_D olza

2

19

9,5 135686

ZEDI_DESADV.DELVRY03.DE MI_Desadv2EDI LVRY03

2

7

3,3

21282

MI_ZBapiGoodsmvtCreate_Abstr MI_GoodsMovementCreate ac

2

3

1,3

5066

MI_File_PC_EURO

2

13

6,5

847

MI_EntrataMerciReso_Response_ Mail A

2

6

2,8

8675

MI_bpmFile_EC

Mail

2

30

14,9

82563

MI_File_OK_EURO

MI_bpmFile_OK

2

17

8,4

216

MI_bpmRequestDocumentFlowSa MI_FindDocumentFlowSalesOrd les er

2

1

0,5

1399

MI_requestFindSalesOrderAlignS MI_FindSalesOrderAfterError

2

2

0,9

1522

© SAP AG

MI_bpmFile_PC

E2E100

8-25



Sender

Receiver

MI_File_EC_FIBO

MI_bpmFile_EC

1

18

18,0

240624 7

MI_exportSalesOrderAlignSM1

MI_multipleSalesOrderAlignSM1 A

1

1

0,9

356

MI_EntrataMerci_Response_Abst Mail r

1

7

6,6

738

MI_Semaforo

Acknowledgment

1

1

1,2

705

MI_File_DC_FIBO

MI_bpmFile_DC

1

13

13,3

44254

MI_File_OK_FIBO

MI_bpmFile_OK

1

7

6,6

227

MI_bpm_responsePraticheLegUp Mail dF

1

2

2,2

668

MI_File_PC_FIBO

1

10

10,0

1151

MI_bpmFile_PC

MESSAGE VOLUME BY BUSINESS INTERFACE

Average Total Volume Message Size (Bytes) (Bytes)

Sender

Receiver

MI_AddSalesOrderAbs

MI_AddSalesOrder

244688113

143.765


MI_bmpINVOIC

107215178

93.312

MI_bmpINVOIC

MovFatture_to_SISINFO

105344741

97.722

ZCE1ARGM_MESS.ZCE1ARGM

MI_Tab_Fatturato

100586871

1.676.448

ORDRSP.ORDERS05

MI_bpmMovOrdini

95926333

24.742

MI_exportSalesOrder

MI_exportSalesOrdersAbstract

89328368

48.813

MI_bpmMovOrdini

MI_movOrdini

73042122

16.994



68233830

17.604


MI_Invoice2OrdersAbs

55750671

52.348

SHPADV.SHPMNT05

MI_SHIPMENTAbs

44067692

98.365

DESADV.DELVRY03.DELVRY03

MI_SendDelivery

37466867

34.248

MI_AddSalesOrderAbsResponse

MI_exportSalesOrderAck

31462528

18.486

SHPADV.SHPMNT05

OrdiniPrelievo_to_TNT

31273099

93.353

ZPARTITE2SM1.ZPARTITE2SM1

MI_PartiteContab2SM1

27893967

3.099.330

MATMAS.MATMAS05

AnagArticoliToQUALIWARE

17148750

12.418

MATMAS.MATMAS05


15483615

12.925

MATMAS.MATMAS05


15432791

12.904

MATMAS.MATMAS05

MI_Materiali_to_FIBO

15260390

13.178

© SAP AG

E2E100

8-26


Sender

Receiver

MI_requestMgmSalesOrderAbs

MI_MgmSalesOrder

14922020

21.348

MATMAS.MATMAS05

MI_Materiali_to_CABL

14362742

13.262

MI_GiacenzeFromTNT

MI_GiacenzeToSAP

11506413

3.835.471

MI_LookupConsegneAbstract_UPDA MI_DeliveryUpdate

9618977

35.234


MI_DeliveryUpdate

9575633

35.076

MATMAS.MATMAS05

MI_AnagMateriali2Ozzano

8254634

15.202

MI_bpmChkZSD_ESITI

MI_ChkZSD_ESITI

7462162

257.316


MainDataT053_to_SM1_new

7302146

2.434.049

MI_bpmUpdZSD_ESITI

MI_UpdZSD_ESITI

6275205

232.415

MATMAS.MATMAS05

MI_AnagMateriali2Dolzago

5772461

17.028

ZEDI_INVOIC.INVOIC02.INVOIC0 MI_Invoice2EDI 2

5346800

167.088

MI_bpmReqCheckInvoicHold


4924517

4.286


MI_MainDataOrders

4517705

3.925

MI_exportInvoicesAbs

MI_exportInvoices

4507573

4.236

ZXI_TABAFCS_SISINFO.ZXISIS

dw01afcs_to_Sisinfo

4374816

916

MI_bpmLoadTemp_Table_EC

MI_LoadTemp_Table_EC

4178278

198.966

MI_ConsegneOutbound

MI_ConsegneAbstract

3494513

12.661

ZXI_TABELLE_SISINFO.ZXISIS

dw01azve_to_Sisinfo

3335044

833.761

MI_chkOrdgcDebmasABS

MainDataCustomer_to_SM1

3141334

7.973

MI_File_EC2SAP

MI_bpmFile_EC

2918770

126.903

MI_bmpINVOIC

MovResiMerce_to_SISINFO

2688920

67.223

MI_bpmexcProc_xi_ec_load

MI_excProc_xi_ec_load

2618717

124.701

MI_bpmFile_EC

MI_File_EC2NextProcess

2610666

124.317

DEBMAS.DEBMAS06.ZDEBMAS0 AnagClienti_to_SISINFO 6

2585598

6.562

DEBMAS.DEBMAS06.ZDEBMAS0 MI_chkOrdgcDebmasABS 6

2584247

6.509

MI_File_EC_FIBO

MI_bpmFile_EC

2406247

2.406.247

ORDERS.ORDERS05.ORDERS05

rcvBpmListeOdA

2176587

7.403

MATMAS.MATMAS05

MI_Materiali

1822883

14.353


dw01afcp_to_Sisinfo

1582555

316.511


dw01acmp_to_Sisinfo

1386645

277.329

Z_HYDRA.ZHYDRA

MovimentiAccise_to_HYDRA_SLaz z

1287584

643.792

© SAP AG

E2E100

8-27


Sender

Receiver


dw01aumi_to_Sisinfo

1179385

235.877


dw01ascl_to_Sisinfo

1154960

230.992


dw01afpr_to_Sisinfo

1124545

224.909

MI_EsitiOBLOGISTICA

MI_bpmFile_EC

1045422

174.237

MI_requestQryT100

MI_bpmQryT100

996018

4.065

ZFIDO2SM1.ZFIDO2SM1

MI_Fidi

922748

153.791

MI_bpmQryT100

Mail

852315

5.464

MI_bmpINVOIC

MovNoteDC_to_SISINFO

820858

32.834

MI_EsitiGRENDI

MI_bpmFile_EC

746970

248.990

MainDataCustomer_from_SM1

MainDataCustomerAbstract_from_

720946

8.383


dw01acpr_to_Sisinfo

693325

138.665

MI_callBAPI_ACC

MI_rcvCallZBAPI_ACC_GL_POSTI NG

631806

6.943

MI_getEnhancDataCustomer

MI_readEnhancDataCustomer

590838

6.870


dw01acpg_to_Sisinfo

586305

117.261

MI_sendVersamAgentiSM1

MI_rcvBpmVersamAgenti

567052

8.592


rcvListeOdAfinaleCopiaLocale

530592

29.477


rcvListeOdAfinale

530592

29.477


dw01aspr_to_Sisinfo

505815

101.163


MainDataArtDecode

479050

737

MI_sndBpmZRFC_FI_VERSAGE_S MI_rcvCallZRFC_FI_VERSAGE_S M1 M1

383660

5.813


dw01agrp_to_Sisinfo

355100

71.020

MI_ConsegneResponseAbstract

Mail

318363

10.612


dw01asfp_to_Sisinfo

297665

59.533


dw01avet_to_Sisinfo

285465

57.093

Z_HYDRA.ZHYDRA

MovimentiAccise_to_HYDRA_Dolza

271371

135.686

MI_EnhancDataCustomerAbs

DEBMAS.DEBMAS06.ZDEBMAS0 6

254656

2.961


dw01attp_to_Sisinfo

208785

41.757

DEBMAS.DEBMAS06.ZDEBMAS0 AnagFiliali_to_SISINFO 6

205374

4.370

MI_Lookup_EntrataMerci_Abstrac

MI_GoodsMovementCreate

205367

34.228


dw01asgr_to_Sisinfo

200975

40.195

MI_NoteSpese

MI_bpmNoteSpese_wrkAbs

199810

19.981

© SAP AG

E2E100

8-28


Sender

Receiver


dw01alp2_to_Sisinfo

198905

39.781

MI_bpmRequestQryBKPF

MI_excQryBKPF

196285

2.157

MI_requestLockSalesOrderAbs

MI_LockSalesOrder

184470

1.537

MI_File_EC_CABL

MI_bpmFile_EC

169752

9.431

MI_bpmFile_EC

Mail

165125

82.563

MI_GiacenzeFromCABL


152931

50.977

MI_GiacenzeFromCABL


152931

50.977

MI_GiacenzeFromCABL

MI_GiacenzeToSAP_CABLM10

152931

50.977

MI_lookupSalesOrderResponseAbs


149760

1.179

MI_bmpINVOIC

MovResiContab_to_SISINFO

146064

24.344

SHPADV.SHPMNT05

OrdiniPrelievo_to_CABL

141498

14.150

ZXI_FATSM1.ZXIFAT

MI_fatturePassive2SM1

137696

27.539


dw01adpr_to_Sisinfo

109975

21.995

CREMAS.CREMAS04

AnagFornitoriToQUALIWARE

99474

3.209

MI_bpmResponseUpdZSD_ESITI

Mail

93248

10.361

WMMBXY.WMMBID02

MovimentiToQUALIWARE

77409

1.843

MI_bpmLoadTemp_Table_PC

MI_LoadTemp_Table_PC

73973

3.699


ListeIngresso_to_TNT_CopiaLoca

66077

22.026


ListeIngresso_to_TNT

66077

22.026

MI_ZWsDeliveryUpdate2_Abstract

MI_ZWsDeliveryUpdate2

64968

16.242


OrdiniReso_to_TNT

63417

9.060


OrdiniReso_to_TNT_CopiaLocale

63417

9.060


dw01afcn_to_Sisinfo

59145

11.829

MI_bpmexcProc_xi_pc_load

MI_excProc_xi_pc_load

56721

2.836

MI_File_DC_FIBO

MI_bpmFile_DC

44254

44.254

ZEDI_DESADV.DELVRY03.DELV MI_Desadv2EDI RY03

42564

21.282

MI_bpmReqExc_PraticheLegUpgFlo MI_exc_PraticheLegUpdFlowR3

40915

835

MI_File_PC_CABL

MI_bpmFile_PC

34853

2.050

MI_EntrataMerci_Outbound

MI_EntrataMerci_Abstract

32034

5.339

FIDCC2.FIDCCP02.FIDCCP02

MI_bpmDETDOC

30782

6.156


MI_DeliveryUpdate

30151

4.307

MI_bpmCallZDATI_FATTURA

MI_CallZDATI_FATTURA

28064

4.677

23326

2.333

MI_bpmResponseBAPI_ACCforMail Mail © SAP AG

E2E100

8-29

Sender


Receiver

A MI_File_EC_EURO

MI_bpmFile_EC

22658

11.329

MI_EntrataMerciReso_Outbound

MI_EntrataMerciReso_Abstract

21448

5.362

ZGELPRATICHE.ZGELPRATICHE MI_Detdoc2leg

19145

2.735

MI_EntrataMerciReso_Response_A

17350

8.675

ZGELPRATICHE.ZGELPRATICHE MI_Cntrparti2leg

17046

2.841

ZGELPRATICHE.ZGELPRATICHE MI_Pratica2leg

17046

2.841

ZXI_DELIVERY.ZXIDLRY

MI_DeliveryRollback

13221

696

MI_filepraticheLegUpdFlowR3

MI_bpm_filePraticheLegUpdFlowR

13128

4.376

MI_ConsegneOutbound

MI_ConsegneAbstract

11019

1.574

MI_bpmCallUpdLvlMovContab

MI_CallUpdLvlMovContab

10938

1.823

ZXI_END.ZXI_END

rcvBpmEndTransfer

10836

602

MI_ZBapiGoodsmvtCreate_Abstrac

MI_GoodsMovementCreate

10132

5.066

MI_bpmDETDOC

MI_Detdoc2leg

8902

1.780

MI_bpmexcProc_xi_esiti_ok

MI_excProc_xi_esiti_ok

8718

1.453

MI_bpmLoadTemp_Table_DC

MI_LoadTemp_Table_DC

8034

2.009

MI_bpmexcProc_xi_dc_load

MI_excProc_xi_dc_load

6670

1.668

COND_A.COND_A02

XmlListDettToSM1

4422

1.474

MI_requestFindSalesOrderAlignS

MI_FindSalesOrderAfterError

3044

1.522

MI_File_DC_CABL

MI_bpmFile_DC

2854

714

ZXI_SEMAFORO.ZXISEM

MI_Semaforo

2820

705

MI_bpmRequestDocumentFlowSales MI_FindDocumentFlowSalesOrder

2797

1.399

MI_lookupSalesOrderResponseAbs


2388

1.194

MI_File_PC_EURO

MI_bpmFile_PC

1694

847

MI_File_PC_FIBO

MI_bpmFile_PC

1151

1.151

MI_EntrataMerci_Response_Abstr

Mail

738

738

MI_File_OK_CABL

MI_bpmFile_OK

733

244

MI_Semaforo

Acknowledgment

705

705

MI_bpm_responsePraticheLegUpdF

Mail

668

668

MI_File_OK_EURO

MI_bpmFile_OK

432

216

MI_exportSalesOrderAlignSM1

MI_multipleSalesOrderAlignSM1A

356

356

MI_File_OK_FIBO

MI_bpmFile_OK

227

227

0

0

Mail

MI_Detdoc2leg

© SAP AG

E2E100

8-30

Sender


Receiver

MI_Pratica2leg

0

0

MI_Cntrparti2leg

0

0

Workload Distribution by Sending Interface

WORKLOAD DISTRIBUTION BY SENDER

Sender

Number of Messages Total Processing Time Average Processing Time

MATMAS.MATMAS05

7025

4.818.057

685,8

ZXI_TABAFCS_SISINFO.ZXIS IS

4776

5.566.872

1165,6

© SAP AG

E2E100

8-31

Sender

Number of Messages Total Processing Time Average Processing Time

MI_bpmMovOrdini

4298

43.608.055

10146,1

ORDRSP.ORDERS05

3877

657.320

169,5


3876

4.077

1,1


2214

101.577

45,9

MI_exportSalesOrder

1830

23.902

13,1

MI_AddSalesOrderAbsResponse

1702

47.483

27,9

MI_AddSalesOrderAbs

1702

39.167

23,0


1151

116.779

101,5

Workload Distribution by Receiving Interface

© SAP AG

E2E100

8-32

WORKLOAD DISTRIBUTION BY RECEIVER

Receiver

Number of Messages

Total Processing Time

Average Processing Time (seconds)


5025

6.221

1,2

dw01afcs_to_Sisinfo

4776

5.566.872

1165,6

MI_movOrdini

4298

43.608.055

10146,1

MI_bpmMovOrdini

3877

657.320

169,5


1831

51.096

27,9

MI_exportSalesOrdersAbstrac t

1830

23.902

13,1

MI_AddSalesOrder

1702

39.167

23,0

AnagArticoliToQUALIWAR E

1381

661.080

478,7


1198

1.397.658

1166,7


1196

653.723

546,6

12 XML Errors

© SAP AG

E2E100

8-33

12.1 Messages With Errors in Previous Week

A high number of messages were found with "error" status. The table below lists these errors in detail.

Integration Engine

Date

Error Code

Integration Server is.00.mnxi3p

22.01.2007

014

3

23.01.2007

014

4

27.01.2007

021

187

Error Code

Description

014

System Error - Manual Restart Possible

021

Canceled Manually

No. of Errors

13 Database server load from expensive SQL statements The expensive SQL statements that we have identified did not lead to performance problems. As reference, you find the load overview listed in the table below and further details of the most expensive statements at the end of the section. For further details on the rating of this check please refer to SAP Note 551646.

13.1 Expensive SQL Statements The analysis of your system's SQL statement cache found no significant problems. DATABASE LOAD FROM EXPENSIVE STATEMENTS

CPU Load [%]

I/O Load [%]

30,5

18,4

Startup Date

Analysis Date

27.01.2007

Comment

30.01.2007 System was not yet very active.

The following table lists the objects that have the highest load due to expensive SQL statements. The load of the statement is evaluated against the current load in the database and the total load since startup. If the statement was first executed at a time close to the database startup or a time close to the analysis, then the lower or higher value will be more appropriate. A more technical explanation is available in SAP Note 551646. The database administrator should conduct a detailed analysis, or you may prefer to order one of the remote or onsite optimization services from the SAP Service Marketplace at service.sap.com/bppo -> SQL. EXPENSIVE STATEMENTS

Object Name

CPU Load [%]

I/O Load [%] Instance Details

Program Details

SSPICSTRHEAD

3,5

3,5 01 (xip)

CL_SPI_CS_DB===... (307)

VSPICSTR

2,1

13,2 01 (xip)

CL_SPI_AS_CENTR... (215)

© SAP AG

E2E100

8-34

Object Name

CPU Load [%]

I/O Load [%] Instance Details

Program Details

TRFCQIN

17,2

1,2 01 (xip)

LQIWKU06 (00537)

TRFCQIN

7,7

0,5 01 (xip)

LIRFCU34 (00047)

If the database has not been active for at least one day before the analysis is done, the information given may not be accurate. Note: If an object name in the table above contains the character '/' it may be a statement to a join. If no object exists in the ABAP dictionary (transaction SE12) with the listed object name, check for each part of the join (items separated by '/').

13.1.1 SSPICSTRHEAD SQL Statement and Explain STATEMENT DETAILS

Operation

Object Type

Total Executions

Disk Reads

Buffer Gets

Records Processed

DELETE

TABLE

650

441.865

7.790.567

421.195

DELETE FROM "SSPICSTRHEAD" WHERE "CLIENT" = :A0 AND "TRGUID" = :A1& Execution Plan DELETE STATEMENT ( Estimated Costs = 1 , Estimated #Rows = 0 ) 2 DELETE 1 INDEX UNIQUE SCAN SSPICSTRHEAD~0 ( Estim. Costs = 2 , Estim. #Rows = 1 ) Search Columns: 2 Estim. Bytes: 93

13.1.2 VSPICSTR SQL Statement and Explain STATEMENT DETAILS

Operation

Object Type

Total Executions

Disk Reads

Buffer Gets

Records Processed

SELECT

TABLE

836

1.653.488

4.822.372

1.013.588

SELECT * FROM "VSPICSTR" WHERE "CLIENT" = :A0 AND "SIGNATURE" = :A1 AND "INSTAMP" >= :A2 AND "TIMESTAMP" >= :A3 AND "FLDNAME" = :A4& Execution Plan SELECT STATEMENT ( Estimated Costs = 2.444 , Estimated #Rows = 0 ) 5 NESTED LOOPS ( Estim. Costs = 2.444 , Estim. #Rows = 8.932 ) Estim. Bytes: 2.224.068

© SAP AG

E2E100

8-35

2 TABLE ACCESS BY INDEX ROWID SSPICSTRHEAD ( Estim. Costs = 625 , Estim. #Rows = 9.097 ) Estim. Bytes: 1.837.594 1 INDEX RANGE SCAN SSPICSTRHEAD~S02 ( Estim. Costs = 2.275 , Estim. #Rows = 9.155 ) Search Columns: 4 4 TABLE ACCESS BY INDEX ROWID SSPICSTRPL ( Estim. Costs = 1 , Estim. #Rows = 1 ) Estim. Bytes: 47 3 INDEX UNIQUE SCAN SSPICSTRPL~0 ( Estim. Costs = 1 , Estim. #Rows = 1 ) Search Columns: 3

13.1.3 TRFCQIN SQL Statement and Explain STATEMENT DETAILS

Operation

Object Type

Total Executions

Disk Reads

Buffer Gets

Records Processed

SELECT

TABLE

14.373

146.291

38.733.626

1

SELECT /*+ FIRST_ROWS */ "QCOUNT" FROM "TRFCQIN" WHERE "MANDT" = :A0 AND "QNAME" = :A1 AND "QCOUNT" < :A2 AND ROWNUM <= :A3& Execution Plan SELECT STATEMENT ( Estimated Costs = 1 , Estimated #Rows = 0 ) 2 COUNT STOPKEY 1 INDEX RANGE SCAN TRFCQIN~0 ( Estim. Costs = 2 , Estim. #Rows = 1 ) Search Columns: 3 Estim. Bytes: 82

13.1.4 TRFCQIN SQL Statement and Explain STATEMENT DETAILS

Operation

Object Type

Total Executions

Disk Reads

Buffer Gets

Records Processed

SELECT

TABLE

249.377

65.142

17.272.798

25.075

SELECT DISTINCT "MANDT" , "QNAME" , MIN( "QCOUNT" ) "FQCOUNT" , COUNT(*) "QDEEP" FROM

© SAP AG

E2E100

8-36

"TRFCQIN" WHERE "MANDT" = :A0 AND "QNAME" LIKE :A1 GROUP BY "MANDT" , "QNAME"& Execution Plan SELECT STATEMENT 2 SORT GROUP BY 1 INDEX RANGE SCAN TRFCQIN~4 Search Columns: 2

13.2 Network Load from Expensive SQL statements EXPENSIVE SQL STATEMENTS DUE TO NETWORK TRAFFIC

Operation

Object Type

Object Name

Executions

Records

Program

SELECT

TABLE

XI_AF_MSG_AUDIT

1

1.741.392

JDBC Thin Client

SELECT

TABLE

SXMSCLUP

83.288

83.289

CL_XMS_PERSIST=...

INSERT

TABLE

SXMSCLUR

98.609

98.604

CL_XMS_PERSIST=...

INSERT

TABLE

SXMSPFRAWH

11.996

11.996

CL_XMS_RUNTIME_...

The table above contains details of the top statements that cause a high network load in your system. The table is provided for information purposes only. Statements are listed if they are executed very often, return a high number of records for each execution, or both. Thus they do not have a technical problem or solution. Any investigation should focus on the application or program logic.


14.1 System Activity The following diagrams or tables show data about system activity: - Total Activity: Transaction steps performed each week (in thousands) - Dialog Activity: Transaction steps performed in dialog task each week (in thousands) - Peak Activity: Transaction steps (in thousands) during the peak hour; this peak hour is calculated as the hour with the maximum dialog activity in the ST03 time profile over 5 working days each week. - Total Users: Users that logged on in the week - Active Users: Users that have performed more than 400 transaction steps in the week. SYSTEM ACTIVITY

Date

Total Activity

Dialog Activity

Peak Activity

23.04.2006

971

2

0

29.04.2006

1943

1

0

06.05.2006

1525

0

0

12.05.2006

1271

0

0

20.05.2006

1517

1

0

© SAP AG

E2E100

8-37

Date

Total Activity

Dialog Activity

Peak Activity

27.05.2006

1501

1

0

02.06.2006

1519

1

0

10.06.2006

1488

1

0

17.06.2006

1492

0

0

23.06.2006

1517

0

0

01.07.2006

1519

0

0

08.07.2006

1513

1

0

14.07.2006

1513

1

0

22.07.2006

1482

0

0

29.07.2006

1502

0

0

15.09.2006

1382

0

0

23.09.2006

1383

0

0

29.09.2006

1380

1

0

14.10.2006

839

0

0

21.10.2006

1380

0

0

27.10.2006

1378

1

0

04.11.2006

1385

1

0

09.12.2006

1409

0

0

16.12.2006

881

1

0

22.12.2006

1380

0

0

29.12.2006

1407

0

0

06.01.2007

1380

1

0

12.01.2007

1373

0

0

20.01.2007

1310

0

0

27.01.2007

1420

1

0

USERS

Date

Logged on

Active

23.04.2006

11

9

29.04.2006

12

8

06.05.2006

11

8

12.05.2006

12

8

20.05.2006

11

8

27.05.2006

11

8

02.06.2006

11

8

© SAP AG

E2E100

8-38

Date

Logged on

Active

10.06.2006

10

8

17.06.2006

10

8

23.06.2006

10

8

01.07.2006

10

8

08.07.2006

10

8

14.07.2006

11

8

22.07.2006

10

8

29.07.2006

10

8

15.09.2006

11

8

23.09.2006

10

8

29.09.2006

12

8

14.10.2006

10

6

21.10.2006

12

8

27.10.2006

10

8

04.11.2006

10

8

09.12.2006

10

8

16.12.2006

11

8

22.12.2006

11

8

29.12.2006

12

8

06.01.2007

12

8

12.01.2007

12

8

20.01.2007

11

8

27.01.2007

12

8


Date



23.04.2006 912

347

116

29.04.2006 1052

321

193

06.05.2006 779

279

253

12.05.2006 713

306

168

20.05.2006 833

217

241

© SAP AG


E2E100

8-39

Date




27.05.2006 671

231

203

02.06.2006 1248

209

326

10.06.2006 929

496

285

17.06.2006 1093

461

345

23.06.2006 1047

422

310

01.07.2006 987

369

300

08.07.2006 2673

1934

220

14.07.2006 1023

512

241

22.07.2006 985

358

245

29.07.2006 585

212

226

15.09.2006 1212

502

412

23.09.2006 10102

9648

166

29.09.2006 913

368

347

14.10.2006 360

225

82

21.10.2006 1300

553

412

27.10.2006 583

248

166

04.11.2006 929

263

325

09.12.2006 3236

2704

210

16.12.2006 606

293

148

22.12.2006 4754

4283

225

29.12.2006 842

313

277

06.01.2007 2581

1115

323

12.01.2007 4821

4524

153

20.01.2007 769

383

201

27.01.2007 1459

560

544

DATABASE PERFORMANCE

Date






23.04.200 6

1,1

3,9

8,2

2,7

17,6

20,9

29.04.200 6

0,8

2,0

4,6

2,1

10,8

29,3

06.05.200 6

0,6

1,5

4,9

1,7

9,8

23,0

© SAP AG

E2E100

8-40

Date






12.05.200 6

1,2

2,9

8,4

2,0

9,9

54,5

20.05.200 6

0,6

1,4

4,7

1,2

10,4

37,9

27.05.200 6

0,4

1,8

4,6

1,2

11,0

44,6

02.06.200 6

0,3

0,8

2,7

1,7

12,0

23,4

10.06.200 6

1,1

0,8

6,7

1,6

10,4

49,5

17.06.200 6

0,4

1,8

5,5

1,3

11,3

54,8

23.06.200 6

0,5

1,3

5,9

1,3

9,2

36,8

01.07.200 6

0,4

1,3

5,8

2,2

8,2

31,3

08.07.200 6

0,6

1,9

255,2

1,0

8,3

47,3

14.07.200 6

0,7

3,1

5,4

1,5

7,8

40,8

22.07.200 6

0,6

2,6

2,4

2,0

9,0

44,7

29.07.200 6

0,7

1,5

2,8

2,2

9,2

38,5

15.09.200 6

0,4

1,2

4,8

1,3

10,5

36,0

23.09.200 6

0,4

3,9

1132,8

1,1

10,8

36,6

29.09.200 6

0,4

1,2

4,3

2,4

12,1

45,4

14.10.200 6

0,5

4,6

13,5

1,0

10,7

36,7

21.10.200 6

0,7

1,7

6,2

2,0

10,8

49,9

27.10.200 6

0,6

2,1

7,5

2,4

11,7

45,6

04.11.200 6

0,4

1,8

4,2

1,6

11,6

38,1

09.12.200 6

1,5

22,7

7,8

2,1

13,7

31,1

16.12.200 6

1,3

2,7

6,6

2,3

18,7

26,8

© SAP AG

E2E100

8-41

Date






22.12.200 6

1,1

2,4

559,2

1,7

13,7

36,0

29.12.200 6

0,5

1,5

4,5

1,7

1,1

27,8

06.01.200 7

1,3

3,1

1,7

2,4

13,1

19,7

12.01.200 7

0,5

2,2

896,1

1,0

10,4

39,2

20.01.200 7

0,7

2,9

7,4

2,4

13,2

33,2

27.01.200 7

0,7

1,7

3,4

1,8

13,2

39,9

TOP 5 TRANSACTIONS

Date

SXI_MONITOR SXMB_ADMIN
SESSION_MANAGER

SWF_XI_SWPR

29.09.2006

1281

29

453

249

14.10.2006

631

30

432

21.10.2006

2585

630

19

649

608

27.10.2006

1167

2351

64

489

734

04.11.2006

933

347

593

3557

09.12.2006

614

1888

34

969

604

16.12.2006

1813

2389

835

1187

527

22.12.2006

894

1013

16

948

446

29.12.2006

1881

751

26

644

274

06.01.2007

1318

1322

13980 692

767

12.01.2007

649

1070

23

590

397

20.01.2007

918

618

23

943

311

27.01.2007

2503

2601

12079 797

288

2399

Transaction Code

Load (%)

SXI_MONITOR

53.7

SXMB_ADMIN

25.8

10.4

SESSION_MANAGER

4.2

SWF_XI_SWPR

0.7

© SAP AG

E2E100

8-42


Date

Availabity (%)


Update Errors

23.04.2006

88

5910

0

29.04.2006

100

105

0

06.05.2006

100

292

0

12.05.2006

96

197

0

20.05.2006

100

0

0

27.05.2006

100

2

0

02.06.2006

100

0

0

10.06.2006

100

0

0

17.06.2006

100

0

0

23.06.2006

100

0

0

01.07.2006

100

0

0

08.07.2006

100

0

0

14.07.2006

100

0

0

22.07.2006

100

1

0

29.07.2006

100

0

0

15.09.2006

93

12

0

23.09.2006

93

14

0

29.09.2006

93

13

0

14.10.2006

89

17

0

21.10.2006

93

12

0

27.10.2006

93

12

0

04.11.2006

93

12

0

09.12.2006

93

13

0

16.12.2006

88

186

0

22.12.2006

93

49

0

29.12.2006

93

12

0

06.01.2007

93

68

0

12.01.2007

93

12

0

20.01.2007

89

13

0

27.01.2007

93

13

0

© SAP AG

E2E100

8-43


Date



14.04.2006

31

0

23.04.2006

29

0

29.04.2006

35

0

06.05.2006

28

0

12.05.2006

30

0

20.05.2006

31

0

27.05.2006

30

0

02.06.2006

38

0

10.06.2006

35

0

17.06.2006

34

0

23.06.2006

0

0

01.07.2006

39

39

08.07.2006

40

40

14.07.2006

34

34

22.07.2006

35

35

29.07.2006

38

38

15.09.2006

27

27

23.09.2006

31

31

29.09.2006

31

31

14.10.2006

50

50

21.10.2006

0

0

27.10.2006

0

0

04.11.2006

46

46

09.12.2006

44

44

16.12.2006

47

47

22.12.2006

33

33

29.12.2006

51

51

06.01.2007

29

29

12.01.2007

32

32

20.01.2007

36

36

27.01.2007

0

0

© SAP AG

E2E100

8-44


SAP System ID

CM3

SAP Component

CRM/BBP

Release

4.0

Status

Productive

DB System

DB2 UDB for Unix and Windows 8.2.0

© SAP AG

E2E100

8-45

Date of Session

16.11.2006

Date of Report

17.11.2006

Installation No.

Author


Customer No.

Session No.

© SAP AG

E2E100

8-46

19 Service Summary During this EarlyWatch Alert Session, we detected potential problems concerning your system. We recommend that you take corrective action as soon as possible.

Note:

All recommendations provided in this report are based on our general experience only. We advise you to test our recommendations before using them in your production system. Also note that EarlyWatch Alert is an automatic service.

Section Overview Rating

Checked Area System Configuration System Performance Workload Distribution SAP System Operating DB Performance DB Administration Remote Function Call CRM Checks Security Checks

Note:

For more information about EarlyWatch Alert, a sample EarlyWatch Alert report with explanations is available in the Media Library on SAP Service Marketplace (http://service.sap.com/ewa). This provides an overview of the check rating strategy and the KPIs that trigger the EWA alerts.

Alert Message Overview Priority

Description

New Alert

Medium Medium Medium Medium Medium Medium

BDoc messages with statuses "Error" or "Not processed fully" were Performance problems exist or are expected. Some tablespaces have reached a critical fill level or a critical size. There are security issues in your system. Upgrade your SAP R/3 plug-in to the latest release available. We found many ABAP dumps in your system.

New New New New New New

Note:

If you need help investigating the alerts listed, order an EarlyWatch contract by contacting your local support organization, or by creating a customer message in component XX-SER-TCC. If you already have such a contract, a detailed analysis can be performed during the next Service Session. Please address the topic during the preparation of the session.

Note:

If you experience serious performance problems, create a customer message on component XX-SERTCC with priority 'high' or 'very high'.

© SAP AG

E2E100

8-47

20 Performance Indicators The following table shows the relevant performance indicators in various system areas. Area

Indicators

Value

System Performance

Active Users Average resp. time for RFC Max. number of RFCs per hour Average RFC response time during peak work hours Avg. Availability per Week Average DB time for RFC Avg. DB Request Time in Update Task DB Size Last Month DB Growth

65 185 ms 42057 123 ms 100 % 81 ms 85 ms 156.23 GB 3.30 GB

Database Performance Database Space Management

Trend

21 Recommended Service The analysis of your SAP EarlyWatch Alert service session has shown that your installation has a high optimization potential. Therefore, SAP would highly recommend you to perform following service: Service

Issue

Scope

Benefit


Long runtime of Analyzes and critical transactions optimizes critical business processes


Increased user 1 productivity and system acceptance

21.1 SAP Remote Performance Optimization Higher Return on Investment - Accelerate your core business processes. - Increase user productivity and system acceptance. - Make optimal use of your hardware resources. - Train your in-house staff to expertly monitor and maintain performance. With SAP Remote Performance Optimization, you use SAP's experience to save time, reduce costs, and keep your SAP solution running optimally. For more information about the service, see SAP Service Marketplace at service.sap.com/safeguarding

© SAP AG

E2E100

8-48

22 System Configuration To ensure optimal support of your SAP system, the SAP components, database, and operating system used must all be supported. Detailed information about SAP's platform support is available on SAP Service Marketplace at http://service.sap.com/platforms.

Rating



Server Type

Server

OS

OS Version

Number of CPUs

Memory [MB]

Database Server

aix014e

AIX

5.3

16

12288

DATABASE SYSTEM

Database Server

Database System

Current Version

aix014e

DB2 UDB for UNIX and Windows

8.2.0

22.2 Software Configuration 22.2.1 Plug-In release Configuration The table below shows the software components installed. The 'Rating' column is filled only for the SAP R/3 Plug-in (PI) and SAP Basis Plug-In (PI_BASIS) and is based on the maintenance schedule. COMPONENTS

Component

Component Release

Patch Level

BBPCRM

400

0008

PI_BASIS

2004_1_620

0008

SAP_ABA

620

0050

SAP_BASIS

620

0050

ST-A/PI

01F_CRM400

0000

ST-PI

2005_1_620

0003

Rating

The plug-in shown with a YELLOW rating in the table above will run out of maintenance in the foreseeable future. Recommendation: We recommend that you upgrade your plug-in software to the latest available release. Detailed information about the release strategy and the recommended upgrade path is available on SAP Service Marketplace at: http://service.sap.com/r3-plug-in[http://service.sap.com/r3-plug-in], http://service.sap.com/basis-plug-in[http://service.sap.com/basis-plug-in].

© SAP AG

E2E100

8-49

22.2.2 SAP Kernel Release Your SAP kernel release (release 640, patch level 99) is not up to date. Recommendation: Ensure that you are using the latest SAP kernel. See SAP Notes 19466 and 138704. These SAP Notes refer to further SAP Notes, depending on your system characteristics.

22.2.3 Implemented Notes via Notes Assistant The following SAP notes were implemented using SAP Notes Assistant. Note

Version

Component

Description

0000191908 0000375929 ………

0051 0004

BC-FES-CTL CRM-BTX-ACT

Sammelhinweis: HTML Viewer Control Bei Statusänderung behält Activitygrund die alten Werte


22.3.1 SAP Service Tools Before we can deliver any services, the latest version of the SAP Service tools must be available in your system. The last run of report RTCCTOOL was 13.10.2006. During the check missing transports were detected. Recommendation: Run report RTCCTOOL, as described in the SAP Note 91488, and implement the recommendations given by the tool.

© SAP AG

E2E100

8-50

23 Performance Overview The performance of your system was analyzed with respect to average response times and total workload. We have detected some problems that may impair system performance. To ensure adequate performance in your core business transactions, you should take corrective action as soon as possible. For further information, please SAP Support.

Rating

Check Performance Evaluation

The following table shows the average response times for various task types: Task type

Dialog Steps


Avg. CPU Time in ms



DIALOG RFC UPDATE UPDATE2 BATCH SPOOL HTTP

73506 2193568 17757 142008 41714 3 5744

1899,7 184,9 120,0 342,6 1285,2 8349,7 858,7

242,7 32,3 22,9 5,0 284,3 130,7 171,6

7,1 1,5 0,8 310,4 1,4 0,0 2,1

6,0 1,6 2,7 1,0 5,8 27,7 4,6

Avg. DB Avg. GUI Time in ms Time in ms 1122,1 80,7 85,3 22,2 424,2 237,7 355,4

270.6 N/A N/A N/A N/A N/A N/A

23.1 Current Workload The following table lists the number of current users (measured from our workload analysis) in your system. Users

Low Activity

Medium Activity

High Activity

Total Users

Measured in System

100

53

12

165

23.2 Performance Evaluation The measured times are compared against reference times to provide a rating. - If the number of dialog steps in an hour is less than 1000 this hour is not considered. - If the total number of transaction steps is less than 24000, the rating for the task is not performed and a blue line is put in the table. The table below shows that a performance problem is expected for the task that is rated YELLOW. Task

Steps

Dia

73506

Upd

17757

HTTP



5744

Rating Task Time Steps


Avg. CPU Time (ms)

Avg. Database Time (ms)

Dia

07-08 2726

1718

186

1096

Dia

08-09 5696

2101

236

1343

Dia

09-10 9894

2312

214

1231

Dia

10-11 10682 1939

234

1249

© SAP AG

E2E100

8-51

Rating Task Time Steps


Avg. CPU Time (ms)

Avg. Database Time (ms)

Dia

11-12 9413

1943

226

1334

Dia

12-13 5992

1731

282

1051

Dia

13-14 5315

1393

203

839

Dia

14-15 8157

2188

227

1063

Dia

15-16 7738

1569

250

932

Dia

16-17 5489

1691

357

861

Dia

17-18 1969

1436

331

662

Only bad dialog response times will cause the check to go red. The reference times are in the table below. Task


Avg. DB time (ms)

1200 2400 1200

600 1200 600

Dia Upd HTTP


23.3.1 Transactions by Total Workload The following transaction profiles list the transactions that have the greatest share in the total workload, sorted by the percentage of total response times. WORKLOAD BY TRANSACTION (DIALOG/UPDATE)

Transaction

Type

Total CRMD_BUS2000120 CIC0 COMM_PCAT_IMS_INIT SAPMHTTP SESSION_MANAGER RSM13000 !XYZ CRMD_ORDER S_AE2_89000019 CRMD_MKTSEG

DIA DIA DIA HTTP DIA UPD DIA DIA DIA DIA

Dialog Total Steps Resp. Time in % 97006 38637 21983 15 5744 1967 17455 2361 254 57 899

Avg. Resp. Avg. CPU Time in ms Time in ms

100,0 62,3 18,7 4,5 3,4 2,1 1,4 1,4 1,2 0,7 0,7

1512,3 2367,3 1249,4 438504,1 858,7 1530,5 117,4 848,6 7153,5 18800,9 1179,9

198,3 212,6 323,1 235,5 171,6 98,5 22,9 252,2 260,8 145,9 321,9

Avg. DB Avg. GUI Time in ms Time in ms 886,9 1604,7 532,0 116810,9 355,4 389,8 82,9 248,6 6547,4 18224,9 544,1

270,6 275,9 258,7 6399,1 0,0 843,5 0,0 238,6 173,4 352,1 195,0

0.0% of the "Total Resp. Time" in the above table is caused by customer transactions. WORKLOAD BY TRANSACTION (BATCH/SPOOL)

Transaction

Type

Total RSDBAJOB SMO6_REORG

BTC BTC

© SAP AG

Dialog Steps

Total Resp. Time in %


Avg. CPU Time in ms

Avg. DB Time in ms

41221 13 7

100,0 20,2 15,7

1246,2 798740,7 1149268,0

264,0 76,8 459447,6

417,2 346,0 831356,3

E2E100

8-52

Transaction

Type

Dialog Steps

Total Resp. Time in %


Avg. CPU Time in ms

Avg. DB Time in ms

GN_GENERATE_CHECK RSAL_BATCH_TOOL_DISPATCHING Z_BW_DOWNLOAD SMWP_BATCH ZBC_STAD COM_PCAT_EXP RSCONN01 RSTRFCIC

BTC BTC BTC BTC BTC BTC BTC BTC

7 168 7 175 7 15 1004 279

12,0 7,2 6,2 5,6 5,4 3,8 3,1 3,0

877351,3 21931,9 455141,3 16579,7 399444,9 130282,4 1563,7 5585,1

4764,3 2003,2 192377,6 7275,7 342632,7 40893,8 136,9 40,9

4150,6 10563,9 169954,7 9587,3 42684,4 47924,9 1267,8 63,3

11.6% of the "Total Resp. Time" in the above table is caused by customer transactions.

23.3.2 Transactions by DB Load The following transaction profiles list the transactions that have the greatest share in the database load, sorted by percentage of total database access times. DATABASE LOAD BY TRANSACTIONS (DIALOG/UPDATE)

Transaction

Type

Total CRMD_BUS2000120 CIC0 SAPMHTTP COMM_PCAT_IMS_INIT CRMD_ORDER RSM13000 S_AE2_89000019 SESSION_MANAGER !XYZ CRMD_MKTSEG

DIA DIA HTTP DIA DIA UPD DIA DIA DIA DIA

Dialog Steps

Total DB Time in %

Avg. DB Time in ms

97006 38637 21983 5744 15 254 17455 57 1967 2361 899

100,0 72,1 13,6 2,4 2,0 1,9 1,7 1,2 0,9 0,7 0,6

886,9 1604,7 532,0 355,4 116810,9 6547,4 82,9 18224,9 389,8 248,6 544,1

0.0% of the "Total DB Time" in the above table is caused by customer transactions. DATABASE LOAD BY TRANSACTIONS (BATCH/SPOOL)

Transaction

Type

Dialog Steps

Total DB Time in %

Avg. DB Time in ms

Total SMO6_REORG RSAL_BATCH_TOOL_DISPATCHING SMWP_BATCH RSCONN01 Z_BW_DOWNLOAD COM_PCAT_EXP RSPPFPROCESS (B)OTHER ZMW_MAP_TAETIGKEITEN_BW SAPRSLOG

BTC BTC BTC BTC BTC BTC BTC BTC BTC BTC

41221 7 168 175 1004 7 15 1003 19189 7 7

100,0 33,8 10,3 9,8 7,4 6,9 4,2 3,6 3,6 2,3 2,1

417,2 831356,3 10563,9 9587,3 1267,8 169954,7 47924,9 617,4 32,1 57552,6 51373,6

9.2% of the "Total DB Time" in the above table is caused by customer transactions.

23.3.3 Transaction !XYZ Transaction '!XYZ' is within the top 10 transactions list by workload on your system. In the statistics collector, specific transaction codes have been filtered and replaced by '!XYZ'. This happens to transaction names with only two characters.

© SAP AG

E2E100

8-53

Recommendation: Apply SAP Note 507255 to solve this issue.



© SAP AG

E2E100

8-54

24 CRM Checks We analyzed the performance of CRM related functions and several administrative aspects. We have detected some problems that require your attention.

Rating

Check Selected RFC Calls Catalog Replication Error messages in the CRM application log Unprocessed BDoc messages Replication and Realignment Queue Status qRFC Administration

24.1 Used CRM Scenarios We detected that the following CRM scenarios are active in your system. Scenario E-Commerce Field Sales or Field Service Interaction Center CRM Online

24.2 CRM Performance Data Mobile Client ConnTrans session statistics Communication Station on host HEI-COS03 Number of Conntrans per day of the week The graphic below show the number of 'ConnTrans' executions by day of the week. If the time interval analyzed exceeds a week, the average for each day of the week is calculated.

© SAP AG

E2E100

8-55

Conntrans Success Statistics The following table displays the number of 'ConnTrans' executions finished successfully versus aborted executions and those with errors. Finished successfully

With errors

Interrupted

1801

0

14

Communication Station on host HEI-COS04 Number of Conntrans per day of the week The graphic below show the number of 'ConnTrans' executions by day of the week. If the time interval analyzed exceeds a week, the average for each day of the week is calculated.

Conntrans Success Statistics The following table displays the number of 'ConnTrans' executions finished successfully versus aborted executions and those with errors. © SAP AG

E2E100

8-56

Finished successfully

With errors

Interrupted

4

0

0

Communication Session Statistics The following table provides an overview on the number of 'ConnTrans' executions from different mobile clients performed within the time interval specified above. Legend: Number of sites : Number of different Mobile Clients that performed a 'ConnTrans' execution within the time interval specified above. Number of sessions: Number of 'ConnTrans' executions that were performed within the time interval specified above. Average session time : Average time a 'ConnTrans' execution was active

Number of sites

Number of sessions

Average time per session [sec]

224

1805

91,8

The following tables displays the 10 mobile clients with the highest upload and download response times. Recommendation: Please verify whether the response times of the 'ConnTrans' executions meet your expectations or whether mobile clients suffer from unacceptable 'ConnTrans' response times due, for example, to high amount of data or bad network transfer rates.

Top ten sites by average CRM Server Download time The following table displays the ten mobile clients with the highest average download time from the CRM server. Mobile Client Site Queue Name CRM_SITE_000000000002525 CRM_SITE_000000000002466 CRM_SITE_000000000002501 CRM_SITE_000000000002701 CRM_SITE_000000000001751 CRM_SITE_000000000001763 CRM_SITE_000000000002641 CRM_SITE_000000000001901 CRM_SITE_000000000002657 CRM_SITE_000000000001717

No. of sessions per site 1 1 1 1 1 1 1 1 1 1

Avg. Data Avg. CRM Server quantity per download response time session [kBytes] per session [ms] 4282 92785 87684 185366 16630 12286 8402 11776 6006 8424

425127 189454 169547 109709 87590 75092 70389 69249 56499 55112

CRM Server transfer rate [kBytes/s] 10 490 517 1690 190 164 119 170 106 153

Top ten sites by average CRM Server Upload time The following table displays the ten mobile clients with the highest average upload time to the CRM Server. Mobile Client Site Queue Name CRM_SITE_000000000002341 CRM_SITE_000000000001590 CRM_SITE_000000000001589

© SAP AG

No. of sessions per site

Avg. Data quantity per session [kBytes]

Avg. CRM Server upload response time per session [ms]

CRM Server transfer rate [kBytes/s]

1 1 2

13578 6174 3777

44750 16001 23844

303 386 158

E2E100

8-57

Mobile Client Site Queue Name CRM_SITE_000000000002283 CRM_SITE_000000000001751 CRM_SITE_000000000001805 CRM_SITE_000000000002151 CRM_SITE_000000000001731 CRM_SITE_000000000001804 CRM_SITE_000000000002701



Avg. CRM Server upload response time per session [ms]

CRM Server transfer rate [kBytes/s]

1 1 2 2 2 1 1

3596 16630 2992 2597 3567 6143 185366

15173 87590 7101 15001 22423 39406 109709

237 190 421 173 159 156 1690

Top ten sites by average Network Download time The following table displays the ten mobile clients with the highest average network time spent during the download. Mobile Client Site Queue Name CRM_SITE_000000000002701 CRM_SITE_000000000001685 CRM_SITE_000000000002595 CRM_SITE_000000000001608 CRM_SITE_000000000001621 CRM_SITE_000000000001736 CRM_SITE_000000000002659 CRM_SITE_000000000001657 CRM_SITE_000000000001751 CRM_SITE_000000000001734



Avg. network download response time per session [ms]

Network transfer rate [kByte/s]

1 7 3 3 1 1 8 1 1 4

185366 14942 18472 8189 11132 10550 19615 6842 16630 3627

1528167 683568 325988 253427 241208 211731 210960 198746 198171 175184

121 22 57 32 46 50 93 34 84 21

Top ten sites by average Network Upload time The following table displays the ten mobile clients with the highest average network time spent during the upload. Mobile Client Site Queue Name CRM_SITE_000000000002646 CRM_SITE_000000000001616 CRM_SITE_000000000002341 CRM_SITE_000000000002701 CRM_SITE_000000000001654 CRM_SITE_000000000001731 CRM_SITE_000000000002383 CRM_SITE_000000000001805 CRM_SITE_000000000002283 CRM_SITE_000000000001667

No. Avg. Data quantity sessions per per session site [kBytes] 4 5 1 1 6 2 1 2 1 5

725 2700 13578 185366 3116 3567 5869 2992 3596 2360

Avg. network upload Network transfer response time per rate [kBytes/s] session [ms] 5677 118806 132234 1528167 120804 61474 103470 55019 57722 81070

128 23 103 121 26 58 57 54 62 29

E-Selling - Selected Function Calls The performance of the Internet Sales scenario in your system was analyzed with respect to average call times and workload created by Internet Sales specific remote function calls between your Internet Transaction © SAP AG

E2E100

8-58

Server(s) / In-Q-My Application Server(s) and your CRM Server. The following table shows statistics about selected Internet Sales function calls (a function call is only rated if it is called more than 10 times). Rating Incoming function call

Calls

CRM_ISA_BASKET_CHANGEITEMS

Total call time, %

Average call time, ms

KBytes sent

KBytes received

546

0,06

1051,79

2716,41

1583,77

57

0,01

1169,20

279,88

59,55

CRM_ISA_BASKET_SAVE CRM_ISA_BASKET_CREATE

4 158

0,00 0,01

1677,58 521,80

0,75 68,26

1,46 319,79

CRM_ISA_SOLDTO_ADD_TO_BASKET

156

0,01

383,28

34,01

80,34

59

0,01

1603,66

15,22

23,07

229

0,00

27,34

850,51

144,77

CRM_ISA_SALESDOC_GETLIST

CRM_ISA_BASKET_ORDER CRM_ISA_BASKET_GETHEAD

Catalog Replication Product Catalog, Variant CATALOG_1 CATALOG_2 CATALOG_3 CATALOG_4 CATALOG_5

Replication Start date mode

Initial Initial

06.11.2006 13.11.2006 16.10.2006 23.10.2006 30.10.2006

Start time

End date

End time

Duratio n

07:16:21 07:27:39 06:25:24 06:48:13 07:45:46

06.11.2006 13.11.2006 16.10.2006 23.10.2006 30.10.2006

09:11:10 09:19:08 08:06:08 08:46:09 09:31:16

01:54:49 01:51:29 01:40:44 01:57:56 01:45:30

No of No of Log Number Error War s nings 1 2 1 1 1

00000000000003447837 00000000000003460813 00000000000003412965 00000000000003421986 00000000000003439029

There are several errors and warnings in the analyzed replication logs for some of the latest product catalog replications. Recommendation: Please repeat the replication process for the listed product catalogs. If the replication process will finish unsuccessfully after the second replication, we recommend creating a customer message in the component "CRM-MD-PCT". For future replications, we recommend monitoring the replication logs after every replication and making sure that the replication was successful.

Unprocessed BDoc messages BDoc messages are used in your SAP CRM system as containers for the data that constitute a business process. To ensure that the business process is working and that business data in different systems remain in a consistent state it is necessary to monitor BDoc messages for errors or for BDoc messages, which do not reach a final state within a certain amount of time. BDoc messages in error state or BDoc messages which were not processed completely were found in your system. Please refer to the following section to analyze and solve the problem(s). Note: This analysis only takes into account BDoc messages updated during the time interval specified in the following table.

Startdate

Enddate

06.11.2006

12.11.2006

BDoc messages in error state BDoc messages that could not be fully processed due to errors were found in your system. Please refer to the following section to see which errors occurred and which corrective measures are necessary. © SAP AG

E2E100

8-59

Note: SAP Note 526853 contains some tips on how to deal with BDoc messages in error state.

Technical Error The processing of one or several BDoc messages ended abnormally due to a technical error. Recommendation:then open a SAPNet message on component CRM-MW. Please refer to the recommendation text in the table. Monitoring -> Message Flow -> Display BDoc Messages or call transaction SMW01. Enter "E01" into field "BDoc State" and press button "Execute" or "Enter". Select the appropriate BDoc message and press button "Errors" to get a list of errors.

BDoc type

Flow Context

# of BDoc messages

MKTPROF_MSG

MO1 - mBDoc Notification

7

Partially send, receivers have errors The processing of one or several BDoc messages ended abnormally due to errors returned by the receivers of the BDoc message information. Recommendation: Analyze the BDoc messages that have an error status and try to resolve the error condition. If you are not able to fix the problem then open a SAPNet message on component CRM-MW. Note: BDoc messages with receiver errors can be monitored as follows: Follow the Menu path Middleware -> Monitoring -> Message Flow -> Display BDoc Messages or call transaction SMW01 Enter "E02" into field "BDoc State" and press button "Execute" or "Enter". Select the appropriate BDoc message and press button "Receivers" to get a list of Receivers. Press button "Errors" to get a list of error messages returned by the receivers.

BDoc type

Flow Context

# of BDoc messages

BUPA_MAIN BUPA_REL

MO2 - mBDoc Notification Multiple MO2 - mBDoc Notification Multiple

1 1

BDoc validation error The processing of one or several BDoc messages ended abnormally due to errors returned by the CRM Validation Service. Recommendation: Analyze the BDoc messages that have an error status and try to resolve the error condition. If you are not able to fix the problem then open a SAPNet message on component CRM-MW. Note: BDoc messages with validation errors can be monitored as follows: Follow the Menu path Middleware -> Monitoring -> Message Flow -> Display BDoc Messages or call transaction SMW01. Enter "E04" into field "BDoc State" and press button "Execute" or "Enter". Select the appropriate BDoc message and press button "Errors" to get a list of errors.

© SAP AG

E2E100

8-60

BDoc type

Flow Context

# of BDoc messages

PRODUCT_MAT BUPA_MAIN BUPA_REL

MI0 - mBDoc Validate MI0 - mBDoc Validate MI0 - mBDoc Validate

35 27 1

BDoc messages in intermediate state BDoc messages remaining in intermediate states since last week were found in your system. BDoc messages in intermediate state for a short period of time are normal but if the intermediate state lasts longer then problems may occur. Please refer to the following section for information on which BDoc types are in intermediate state and error analysis.

BDoc stored before update task One or several BDoc messages were not processed completely and remain in intermediate state I04 (BDoc stored before update task) for a long time. Recommendation: Analyze the BDoc messages that remain in intermediate state and try to resolve the error condition. If you are not able to fix the problem, then open a customer message on component CRM-MW.

Note: BDoc messages in intermediate state "I04" can be monitored as follows: Follow the Menu path Middleware -> Monitoring -> Message Flow -> Display BDoc Messages or call transaction SMW01. Enter "I04" into field "BDoc State" and press button "Execute" or "Enter".

BDoc type

Flow Context

# of BDoc messages

BUS_TRANS_MSG

MO1 - mBDoc Notification

1

Sent to receivers (not all have confirmed) One or several BDoc messages are waiting for confirmation from receiver systems (e.g. OLTP R/3 backend systems) for a long time and therefore cannot be processed completely. Recommendation: Analyze the BDoc messages that are waiting for confirmation from receivers for a long time and try to resolve the error condition. If you are not able to fix the problem then open a SAPNet message on component CRM-MW.

Note: BDoc messages in state "Sent to receivers (not all have confirmed)" can be monitored as follows: Follow the Menu path Middleware -> Monitoring -> Message Flow -> Display BDoc Messages or call transaction SMW01. Enter "O01" into field "BDoc State" and press button "Execute" or "Enter". Select the appropriate BDoc message and press button "Receivers" to get a list of Receivers. To analyze the error search for stuck queues, shortdumps or other error indicators on the receiver system.

BDoc type

Flow Context

BUPA_MAIN BUPA_MAIN BUPA_REL

MO1 - mBDoc Notification MO2 - mBDoc Notification Multiple MO2 - mBDoc Notification Multiple

© SAP AG

# of BDoc messages

E2E100

2 2 1

8-61

qRFC Administration qRFC Queues qRFC Outbound Queues Mobile Client Site Queues We found many Mobile Client queues in your system that either have a high number of entries or are more than one month old. A high number of qRFC queue entries may cause severe performance problems, for example long-running SQL statements on table ARFCSSTATE. Recommendation: Make sure that the respective mobile client users regularly perform Conntrans. If the queues are for non-existent mobile clients delete all subscriptions for these queues and delete the queues afterwards.

Queue Name CRM_SITE_000000000002426 CRM_SITE_000000000002471 CRM_SITE_000000000002453 CRM_SITE_000000000002389 CRM_SITE_000000000001781 CRM_SITE_000000000001807 CRM_SITE_000000000001585 CRM_SITE_000000000001603 CRM_SITE_000000000002714 CRM_SITE_000000000002142

Date of first entry

Entries

25.04.2006 23.05.2006 01.06.2006 09.06.2006 16.06.2006 25.07.2006 18.08.2006 28.09.2006 06.10.2006 13.10.2006

124990 113801 111531 93001 59370 43633 56992 38182 32334 33689

Outbound Queues to R/3 Many R3A* queues in status SYSFAIL were detected in your system. As a result of this, changes in the CRM system have not been updated on the R/3 system Recommendation: Take administrative action to solve the errors.

Queue Name

Status

R3AUBUPA0056035261 R3AUBUPA0077098518

SYSFAIL SYSFAIL

Date of first entry

Entries

07.11.2006 07.11.2006

3 1

qRFC Inbound Queues Inbound Queues from R/3 Many R3A* queues in status SYSFAIL were detected in your system. As a result of this, changes in the R/3 system have not been updated on the CRM system Recommendation: Take administrative action to solve the errors.

Queue Name

Status

R3AD_MATERIA000 R3AD_MATERIA003

SYSFAIL SYSFAIL

© SAP AG

E2E100

Date of first entry

Entries

06.11.2006 15.11.2006

6571 88

8-62


Rating


25.1 Workload by Application Module The following graphics or tables show how each application module contributes to the total system workload. Two workload aspects are shown: - CPU time: Total CPU load on all servers in the system landscape - Database time: Total database load generated by the application The 'unknown' section contains all programs that cannot be assigned to a standard SAP application module (for example, customer programs, an industry solution, or a third-party add-on development).

Module

CPU Time in %

BC CRM-BTX CRM-CIC CRM-MW Unknown CRM-MD CRM-MKT

© SAP AG

73,3 8,5 6,8 5,9 3,8 0,9 0,4

E2E100

8-63

Module

DB Time in %

BC CRM-BTX CRM-CIC CRM-MW Unknown CRM-MD CRM-MKT

65,7 23,6 4,2 3,6 1,6 1,0 0,3

25.2 DB Load Profile The following table or diagram shows the DB load caused by Dialog, RFC, and Background tasks over different time frames. The data given in the table represents the average number of database processes occupied by each task type in the database during the given time frames. These statistics are calculated as a weekly average, which means the average values over six working days with a unit of one hour. Periods between 00:00-06:00 and 21:00-24:00 contain an average value per hour. By comparing the load profiles for dialog and background activity, you have an overview of the amount of background activity during online working hours.

© SAP AG

E2E100

8-64

The graph above indicates the CPUs will handle the database load so the check is rated green.

© SAP AG

E2E100

8-65


Rating

Check System Availability Update Errors ABAP Dumps

26.1 Update Errors In a system running under normal conditions, only a small number of update errors should occur. To set the rating for this check, the number of active users is also taken into consideration. The following table contains the number of update errors detected.

We did not detect any problems.

26.2 Transports The following graphics contain information about the number of objects which were transported to the SAP system. The three graphics/tables show objects in total, programs and table (content).

© SAP AG

E2E100

8-66

© SAP AG

E2E100

8-67


Date

Quantity of Dumps

10.11.2006 11.11.2006 12.11.2006 13.11.2006 14.11.2006 15.11.2006

12 1 2 10 17 18

DETECTED ABAP DUMPS

Error


CALL_FUNCTION_NOT_FOUND SAPSQL_ARRAY_INSERT_DUPREC TSV_TNEW_OCCURS_NO_ROLL_MEMORY MESSAGE_TYPE_X

2 1 6 51

aix014e aix014e aix014e aix014e

ALBERT.W COMMSTAT EDINGER KOECK

Date (e.g.) Time (e.g.) 10.11.2006 12.11.2006 15.11.2006 15.11.2006

09:48:44 13:17:11 02:08:18 15:49:37

27 Security Checks Critical security issues were found in your system. Several users with critical authorizations for certain transactions were found. See the information in the following checks. SAP offers a Security Optimization Self Service. This service is included into the current version of the SAP Solution Manager for free. We recommend that you make use of this service. For more information, see the SAP Service Marketplace http://www.service.sap.com/SOS.

Rating


27.1 Users Authorized to Display all Tables Unauthorized access to sensitive data is possible if too many users have this authorization. The mentioned number of users for each client has the checked authorization. Client

User Count

100

102

Evaluated Risk - High © SAP AG

E2E100

8-68

Recommendation: Use the Profile Generator (PFCG) to correct roles and/or transactions SU02 (Maintain Profiles) / SU03 (Maintain Authorizations) to correct profiles and authorizations, depending on your environment. With the authorization info system (SUIM), you can check the results. For this check, you should take a look at the roles or profiles that include the authorization objects listed below.

Authorization objects: Object 1: S_TCODE with TCD=SE16, TCD=SE16N, TCD=SE17, TCD=SM30, or TCD=SM31 Object 2: S_TABU_DIS with ACTVT = 03 or 02 and DICBERCLASS = *

27.2 Users Authorized to Start all Reports With this authorization, critical functions and reports that do not contain own authorization checks can be executed. The mentioned number of users for each client has the checked authorization. Client

User Count

100

104

Evaluated Risk - High Recommendation: Use the Profile Generator (PFCG) to correct roles and/or transactions SU02 (Maintain Profiles) / SU03 (Maintain Authorizations) to correct profiles and authorizations, depending on your environment. With the authorization info system (SUIM), you can check the results. For this check, you should take a look at the roles or profiles that include the authorization objects listed below.

Authorization objects: Object 1: S_TCODE with TCD=SE38 or TCD=SA38 or TCD=SC38 Object 2: S_PROGRAM with P_ACTION=SUBMIT P_GROUP=*

27.3 Users Authorized to Display Other Users Spool Request This authorization allows unauthorized access to sensitive data contained in spool requests. The mentioned number of users for each client has the checked authorization. Client

User Count

100

104

Evaluated Risk - High Recommendation: Use the Profile Generator (PFCG) to correct roles and/or transactions SU02 (Maintain Profiles) / SU03 (Maintain Authorizations) to correct profiles and authorizations, depending on your environment. With the authorization info system (SUIM), you can check the results. For this check, you should take a look at the roles or profiles that include the authorization objects listed below. Authorization objects: Object 1: S_TCODE with TCD = SP01 or SP01O Object 2: S_ADMI_FCD with S_ADMI_FCD = SP01 or SP0R Object 3: S_SPO_ACT with SPOACTION = BASE and DISP and SPOAUTH = * or __USER__

27.4 Users Authorized to Administer RFC Connections If too many users have this authorization, two problems can occur: - Unauthorized access to other systems - Malfunction of interfaces if invalid connection data is entered © SAP AG

E2E100

8-69

The mentioned number of users for each client has the checked authorization. Client

User Count

100

104

Recommendation: Use the Profile Generator (PFCG) to correct roles and/or transactions SU02 (Maintain Profiles) / SU03 (Maintain Authorizations) to correct profiles and authorizations, depending on your environment. With the authorization info system (SUIM), you can check the results. For this check, you should take a look at the roles or profiles that include the authorization objects listed below.

Authorization objects: Object 1: S_TCODE with TCD=SM59 Object 2: S_ADMI_FCD with S_ADMI_FCD = NADM

27.5 Users Authorized to Reset/Change User Passwords The following users are allowed to change and reset the passwords of all users. This is very risky because all these users could change the password and log on themselves with any user. The only consequence is that the "real user" would no longer be able to log on, because the password was changed. However, this normally results in the password being reset, because there is a chance that the "real user" might have forgotten the correct password. Client

User Count

100

89

Evaluated Risk - High Recommendation: Use the Profile Generator (PFCG) to correct roles and/or transactions SU02 (Maintain Profiles) / SU03 (Maintain Authorizations) to correct profiles and authorizations, depending on your environment. With the authorization info system (SUIM), you can check the results. For this check, you should take a look at the roles or profiles that include the authorization objects listed below.

Authorization objects: Object 1: S_TCODE with TCD=SU01 or TCD=OIBB or TCD=OOUS or TCD=OPF0 or TCD=OPJ0 or TCD=OVZ5 Object 2: S_USER_GRP with ACTVT=05

27.6 Password of One or More Standard Users is not Changed Evaluated Risk - High Recommendation: Change the standard password for the users SAP* DDIC, SAPCPIC, and EARLYWATCH in all clients. Do not delete the SAP* user because this enables the hard-coded SAP* user to be used. For more information, see the SAP Security Guides on the Service Marketplace http://www.service.sap.com/securityguide.

Client

User

066 000 100 066

SAP* SAPCPIC SAPCPIC EARLYWATCH

© SAP AG

E2E100

8-70

28 DB Performance In the checks performed, no problems regarding the performance of your database were found.

Rating

Check Locking Average Read and Write Times Missing Indexes

28.1 Average Read and Write Times We checked the average read and write times for each tablespace and did not detect any tablespaces above a critical threshold. However, as this check is performed at the tablespace level, you cannot exclusively rely on it to identify an I/O bottleneck.

29 DB Administration In the checks performed, problems regarding the administration of your database were found.

Rating

Check Tablespaces with a critical size Log Retention Mode

29.1 Database Storage These two figures show your database growth and freespace development over the past few months.

© SAP AG

E2E100

8-71


© SAP AG

Date

Current Size (GB)

Monthly Growth (GB)

01.12.2005 01.01.2006 01.02.2006 01.03.2006 01.04.2006 01.05.2006 01.06.2006 01.07.2006 01.08.2006 01.09.2006 01.10.2006 01.11.2006

114,33 128,88 127,74 128,70 134,53 137,38 143,68 147,25 149,88 150,22 152,93 156,23

0,00 14,55 0,00 0,96 5,83 2,85 6,30 3,57 2,63 0,33 2,71 3,30

E2E100

8-72

29.2 Tablespaces with a critical Size Tablespaces with a critical Used Size CM3#BTABD CM3#STABD CM3#STABI CM3#POOLD CM3#POOLI

Partition

Used Size (%)

0 0 0 0 0

92 92 91 93 93

The "Tablespaces with a critical Used Size" report table shows the DMS tablespaces with regular content that have reached a critical fill level. Recommendation: Extend the tablespaces that have reached a critical fill level and are still growing. Implementation: Use the DB2 Control Center or the DB2 command line. Note: The containers of a single tablespace should be the same size. If you add a container you should take into account that the new tablespace size is not immediately available because of the DB2 rebalancing process. The new freespace is only available after the DB2 rebalancing process is finished. You can check whether the rebalancing process was successful in DB2's db2diag.log. As of DB2 UDB version 7, existing containers can be increased in size.

© SAP AG

E2E100

8-73


30.1 System Activity The following diagrams or tables show data about system activity: - Total Activity: Transaction steps performed each week (in thousands) - Dialog Activity: Transaction steps performed in dialog task each week (in thousands) - Peak Activity: Transaction steps (in thousands) during the peak hour; this peak hour is calculated as the hour with the maximum dialog activity in the ST03 time profile over 5 working days each week. - Total Users: Users that logged on in the week - Active Users: Users that have performed more than 400 transaction steps in the week.

30.2 Response Times The following diagram or table show the development of the response times: - Average response time in dialog task (total, DB and CPU parts) - Average response times for database requests - Average response times for the top 5 transactions

© SAP AG

E2E100

7-2

Transaction Code

Load (%)

CRMD_BUS2000120 CIC0 COMM_PCAT_IMS_INIT SESSION_MANAGER !XYZ

64.5 19.4 4.6 2.1 1.4

30.3 System Operation The following diagram or table shows important KPIs for system operation. Note: The column 'Availability' is filled for systems with SAP Basis 4.6 and higher.

© SAP AG

E2E100

7-3

30.4 Hardware Capacity

© SAP AG

E2E100

7-4

E2E100 - End to End Root Cause Analysis

Recommend Documents