SAP Startup Troubleshooting Guide for Netweaver Application Server This document was designed to help SAP customers to troubleshoot and resolve the majority of SAP system startup problems own their own, minimizing the business impacts and costs attached to a standard support ticket life cycle. As time goes by, new topics and startup error scenarios will be added to this guide. Stay tuned! This document will not (at first) approach start-up problem caused by cluster related configurations. NEWS from December 2016: 2016: if SAP is running on AIX 7.1 or 7.2 and you are unable to start SAP after updating the AIX, read the SAP note 2402207.
01. PRELIMINARY CHECKLIST: This initial checklist will approach mandatory settings that have proven to be root cause of startup issues in several occasions. With this in mind, please *DO NOT* skip any phases.
1.1: IS DATABASE UP AND RUNNING? This is *absolute* prerequisite for SAP SYSTEM to start. In order to test this, operating system command "R3trans -d" can be executed (as SIDadm user). Any return code different than 0 means that DB cannot be reached by DIALOG work processes, and so SAP system will not initiate. The following return indicates that database is up, running and reachable by SAP work processes:
:SIDadm > R3trans -d This is R3trans version X.xx (release 742 - 18.11.14 - 20:14:09). unicode enabled version R3trans finished (0000).
*** There is NO POINT in moving the analysis further while the return code is different than zero. *** In case the return code is different than zero, please involve i nvolve your DBA/Network/OS teams.
1.2: IS SAP STARTUP SERVICE CORRECTLY DEFINED? UNIX: File "/usr/sap/sapservices" must contain the correct service definitions.
LD_LIBRARY_PATH=/usr/sap//exe:$LD_LIBRARY_PATH; export LD_LIBRARY_PATH; /usr/sap///exe/sapstartsrv pf=/usr/sap//SYS/profile/__/SYS/profi le/__ ame> -D -u
WINDOWS: I. Run "services.msc"; II. Look for the relevant service SAP_; III. Check the command line assigned to it:
:\usr\sap\\\exe\sapstartsrv pf=:\usr\sap\\SYS\profile\ pf=:\usr\sap\\SYS\profile\
1.3: [START] PROFILES: In order to correctly start the required processes such as message server, enqueue server and the disp+work, the SAP startup agent (sapstartsrv) needs to find a correct scenario for that. This scenario consists on a list of "Start_Program_" commands that should be present an the "START" profile (for all kernel releases up to 740) or at the "INSTANCE" profile (kernel release 740 onwards).
*** These "start" entries are mandatory in the profile being used by sapstartsrv process. You can use the process list l ist on UNIX and the service definition at WINDOWS to find out what is the exactly profile currently assigned to sapstartsrv process.
[EXAMPLES:]
Entries needed for starting the dispatcher + work processes:
_DW=dw.sap$(SAPSYSTEMNAME)_$(INSTANCE_NAME) _DW=dw.sap$(SAPSYSTEMNAME)_$(INSTANCE_NAME) Execute_0X=local rm -f $(_DW) Execute_0X=local ln -s -f $(DIR_EXECUTABLE)/disp+wo $(DIR_EXECUTABLE)/disp+work$(FT_EXE) rk$(FT_EXE) $(_DW) Start_Program_0X=local $(_DW) pf=$(_PF)
Example of message server start entries:
_MS = ms.sap$(SAPSYSTEMNAME)_$(INSTANCE_NAME) ms.sap$(SAPSYSTEMNAME)_$(INSTANCE_NAME) Execute_0X = local rm -f $(_MS) Execute_0X = local ln -s -f $(DIR_EXECUTABLE)/msg_server$(FT_EXE) $(_MS) Restart_Program_0X = local $(_MS) pf=$(_PF)
Enqueue server related start commands:
_EN = en.sap$(SAPSYSTEMNAME)_$(INSTANCE_NAME) en.sap$(SAPSYSTEMNAME)_$(INSTANCE_NAME) Execute_0X = local rm -f $(_EN) Execute_0X = local ln -s -f $(DIR_EXECUTABLE)/enserver$( $(DIR_EXECUTABLE)/enserver$(FT_EXE) FT_EXE) $(_EN) Restart_Program_0X = local $(_EN) pf=$(_PF)
1.4: LOGGING IN OS LEVEL DIRECTLY AS USER: It is mandatory that SIDadm user should be able to log on at operating system level *WITHOUT ANY* other prompts such as:
1. 2. 3. 4.
Company policies; User policies; Next scheduled downtimes/System news; Logging in as another user and SUdoing to SIDadm;
This can cause several problems throughout different scenarios, but mainly on upgrade procedures. Further details on SAP note 1301712. #1301712 - UNIX sapstartsrv: Determining the environment
1.5: IS SAP STARTUP AGENT (SAPSTARTSRV) RUNNING? There must be *ONLY ONE* sapstartsrv process running for each instance. Examples for each platform below: UNIX: As an example, we will use a centralized system with the following 03 instances running on the same host
(ASCS, PAS and DI). The following output should return for the operating system level command: [ ps -ef | grep | grep sapstartsrv ]
(...) /usr/sap//ASCS00/exe/ /usr/sap//ASCS00/exe/sapstartsrv sapstartsrv pf=/usr/sap//SYS/profile/_ASCS01_ -D pf=/usr/sap//SYS/profile/_ASCS01_ (...) /usr/sap//DVEBMGS01/e /usr/sap//DVEBMGS01/exe/sapstartsrv xe/sapstartsrv pf=/usr/sap//SYS/profile/_DVEBMGS00_ pf=/usr/sap//SYS/profi le/_DVEBMGS00_ -D (...) /usr/sap//D02/exe/sap /usr/sap//D02/exe/sapstartsrv startsrv pf=/usr/sap//SYS/profile/_D02_ pf=/usr/sap//SYS/profi le/_D02_ -D
Profile pattern names returned with the command can be such as "START__" in older releases. There can be another sapstartsrv process (as below) but it will play no parts in the startup of the instance. When sapstartsrv service is not running you could run manually the above command for the desired instance If the service doesn't start and nothing it's written in "sapstartsrv.log" it might happen the binaries are damaged in the local exe folder. In this case has to be replaced
(...) /usr/sap/hostctrl/exe/sapstartsrv pf=/usr/sap/hostctrl/exe/host_profile -D
WINDOWS: I. You shoud first configure task manager output so it displays the command line related to the selected process: Execute task manager >> Switch to tab "Processes" >> Menu "view" >> Select "columns" >> Select column 'PID' and 'COMMAND LINE'; II. Then using task manager tab "Processes", look for the instance sapstartsrv process with similar:
:\usr\sap\\\exe\sapstartsrv pf=:\usr\sap\\SYS\profile\ pf=:\usr\sap\\SYS\profile\
1.6: IS ANY SAP PROCESS ALREADY RUNNING? After trying to start the instance (either through the SAP MMC, the "startsap" command or using "sapcontrol ...
-function Start"), no error is returned. However, the SAP processes (Dispatcher - ABAP; jstart - Java) do not start. No trace file is updated either. Verify whether any process is already in "GREEN" or "YELLOW" status (e.g., the IGS).
hostname:sidadm 40> sapcontrol -nr 33 -function GetProcessList 12.10.2015 20:15:45 GetProcessList OK name, description, dispstatus, textstatus, starttime, elapsedtime, pid msg_server, MessageServer, GRAY, Stopped, , , 1234 disp+work, Dispatcher, GRAY, Stopped, , , 1111 igswd_mt, IGS Watchdog, GREEN, Running, 2015 07 31 11:59:15, 1:21:09, 2323 hostname:sidadm 41>
All processes must have the "GRAY" status. Otherwise, the SAP Startup Agent (sapstartsrv) will "ignore" the start command. Stop the instance first, so all processes are stopped. Then, start the instance again.
1.7: PERMISSIONS: UNIX: I. "/TMP" DIRECTORY: "/tmp" directory's permissions must match the following. The final 't' for "temporary" is of ultimate relevance and should not be overlooked:
drwxrwxrwt
927637 II. SAPSTARTSRV.LOG:
22 root
root
4096 Apr
6 20:16 tmp
"/usr/sap///work/" "Unable to open trace file sapstartsrv.log. (Error 13 Permission denied)" III. SAPUXUSERCHECK FILE:
-rwsr-x
1 root
sapsys
1293660 Apr 06 2015
sapuxuserchk
There is one SAPUXUSERCHECK file per instance (/usr/sap///exe) and the permissions and ownership should look alike for all files. The following command can help tracking the files:
find /usr/sap//* -name sapuxuserchk
1.8: IP / HOSTNAME RESOLUTION: LOCALHOST: startup commands from sapcontrol will be using "localhost" implicitly (when no "-host" option is explicitly informed) and for such the loopback IP "127.0.0.1" should be resolved for it. What is i s the output for "niping for "niping -v -H localhost" ? The loopback IP should be the only one returned. If not, please involve your network team.
Lookup of hostname: localhost (NiHostToAddr) --> IP-Addr.: 127.0.0.1 Lookup of IP-Addr.: 127.0.0.1 (NiAddrToHost) --> Hostname: localhost
1476386 - Wrong localhost name resolution setting
HOSTNAME: What IP is resolved for the local hostname (output of OS level command niping -v -H )? There is a known error described in SAP note #1054467 where the loopback IP 127.0.0.1 is wrongly resolved for it.
1054467 - Local host name refers to loopback address
02. IDENTIFYING AFFECTED STAGE: This troubleshooting (starting from here) *considers TRUE* that all points from the initial checklist are done. Please keep in mind that all of above steps were taken from real incidents and they were, in several of those cases, the required fix the for the issue resolution. The very first question that needs to be answered is: which component/process has failed? Looking at the image below, it is possible to find out the sequence is which the logs will be written and therefore where the specific component information can be found.
The main approach is to begin for the latest updated files, based on this image's timeline. As an example, in case the work processes traces (dev_w##) are not updated after the latest failed start attempt, then the dispatcher trace (dev_disp) needs to be verified and so on.
03. DISPATCHER (dev_disp): Dispatcher is commonly the first and most important trace file analyzed for such problem scenarios. Most of the times the reason for the startup failure will be here. In the dispatcher trace, we'll initially look for critical error situations, which directly contributed to the startup fail. If such straightforward occurrences cannot be found, then we look for any situation which would, indirectly, lead to the same situation. As an example, another component has crashed, therefore dispatcher has to shut itself down. --> Example Example of error where it's possible possible to see the cause right away:
***LOG Q0K=> DpMsAttach, mscon ( ) [dpMessageSer 1652] *** DP_FATAL_ERROR => DpMsAttach: local hostname '' is resolved to loopback address (cf. SAP note 1054467 for details) *** DISPATCHER EMERGENCY SHUTDOWN ***
--> Example of dispatcher startup error error caused by the crash of all work processes. The very next step in this case is to check the "dev_w#" traces and find out what is killing the disp+work processes and, ultimately, what is the cause of the instance failure:
*** ERROR => DpHdlDeadWp: W64 (pid 1898) died (severity=0, status=65280) [dpxxwp.c 1739] DpTraceWpStatus: child (pid=1898) exited with exit code 255 *** DP_FATAL_ERROR => DpWPCheck: no more work processes *** DISPATCHER EMERGENCY SHUTDOWN ***
*** IMPORTANT: --> Whenever Whenever suspicious about a dispatcher crash and the following following entries are found:
DpSigInt: caught signal 2 DpHalt: shutdown server >____ umber> < (normal) DpModState: change server state from ACTIVE to SHUTDOWN DpHalt: stop work processes
Please keep in mind that a manual shut down happened. This can be proved by the "signal 2" entry and also by the "< (normal)" sign, next to the shut-down sign.
3.1: MOST COMMON DISPATCHER CRASH SITUATIONS: In this list we bring the most common causes for a "dispatcher crash", along with its resolution:
I. "SI_EPORT_INUSE": These error entries are preventing the dispatcher from initialize and they are not caused by SAP but network inconsistencies. At the time dispatcher is trying to start and bind its default port 32 it finds
that there is already a process binding this port, thus, it aborts logging below entries:
***LOG Q0I=> NiIBindSocket: bind (10013: WSAEACCES: Permission denied) [nixxi.cpp 3740] < Problem... *** ERROR => NiIBindSocket: SiBind failed for hdl 9/sock 1292 (SI_EPORT_INUSE/10013; I4; ST; 0.0.0.0:3202) [nixxi.cpp 3740] *** ERROR => DpCommInit: NiBufListen (rc=-4) [dpxxdisp.c 10223] *** DP_FATAL_ERROR => DpSapEnvInit: DpCommInit *** DISPATCHER EMERGENCY SHUTDOWN ***
RESOLUTION: You must involve your network team and find out what process is listening on port 32 and ensure that the dispatcher port is free at the time it is starting up. As an example, in UNIX you can try the OS command "netstat -an | grep 32" when searching for a process listening on that port.
II. PLATFORM PATCHNO... BUT MY PATCHNO IS: Following error is seen for scenarios where the path release of dispatcher and message server is not the same. This can happen for couple of reasons but the major number of occurrences is related to issues on SAPCPE which failed to copy newest files from the GLOBAL EXE folder to the INSTANCE EXE folder, resulting in mixed releases.
*** *** *** ***
ERROR => platform patchno is xxx [dpxxdisp.c 17170] ERROR => but my patchno is yyy [dpxxdisp.c 17171] DP_FATAL_ERROR => Release inconsistent with running system DISPATCHER EMERGENCY SHUTDOWN ***
RESOLUTION: You must ensure that all instances are running with the same kernel release. As a potential potential fix, you can copy manually the files from the global "EXE" folder to the instance "EXE" folder.
III. BAD MEMORY PARAMETRIZATION:
*** ERROR => SHM2_EsCreateShared: esSHMAddSharedSegment(,userID=268641024) returned 2 [esuxshm2.c 2451] *** ERROR => DpEmInit: EmInit (1) [dpxxdisp.c 10864] *** ERROR => DpMemInit: DpEmInit (-1) [dpxxdisp.c 10778] *** DP_FATAL_ERROR => DpSapEnvInit: DpMemInit *** DISPATCHER EMERGENCY SHUTDOWN ***
OR
*** ERROR => SHM2_EsInit: invalid argument ES/SHM_MAX_SHARED_SEGS=2. Valid range is 1 ... 1d [esuxshm2.c 1381] *** ERROR => DpEmInit: EmInit failed (1) [dpInit.c 1475] *** ERROR => DpMemInit: DpEmInit (-1) [dpInit.c 1339] *** DP_FATAL_ERROR => DpSapEnvInit: DpMemInit *** DISPATCHER EMERGENCY SHUTDOWN ***
RESOLUTION: There are a few interconnected memory memory parameters parameters that must reach each other requisites. Mainly, following SAP notes have to be consulted: # 1998233 - IBM IBM i: SHM HM2_ 2_E EsIni sInitt: inva invali lid d argu argum ment ent ES/S ES/SH HM_ M_M MAX_S AX_SHA HARE RED_ D_SE SEG GS # 1014010 - EG memory up to 32 GB (em/global_area_MB up to 32000) And mainly those two rules should be reviewed:
ES/SHM_MAX_SHARED_SEGS <= ES/SHM_PROC_SEG_COUNT - ES/SHM_MAX_PRIV_SEGS ES/SHM_PROC_SEG_COUNT >= ES/SHM_MAX_SHARED_SEGS + ES/SHM_MAX_PRIV_SEGS
IV. SI_EADDR_NAVAIL: This error is commonly commonly seen when "localhost" cannot be resolved as "127.0.0.1".
***LOG Q0I=> NiIBindSocket: bind (68: Can't assign requested address) [nixxi.cpp 3239] *** ERROR => NiIBindSocket: SiBind failed for hdl 0 / sock 7 (SI_EADDR_NAVAIL/68; I4; DG; xx.y.zzz.ww:3200) [nixxi.cpp 3239] *** ERROR => DpCommInit: NiDgHdlBindName failed: -16 [dpxxdisp.c 10263] *** DP_FATAL_ERROR => DpSapEnvInit: DpCommInit *** DISPATCHER EMERGENCY SHUTDOWN ***
RESOLUTION: See PRELIMINARY CHECKLIST above, topic [1.7] and ensure that "localhost" is resolved as loopback IP "127.0.0.1" and vice-versa.
V. NO MORE WORK PROCESSES: Dispatcher is shutting itself down because there are no more work processes available for processing of requests. In order to pinpoint the the cause, work processes traces have to be analyzed. analyzed. Just follow the initialization entries and soon the ERROR entries with the root cause will be seen.
*** DP_FATAL_ERROR => DpWPCheck: no more work processes *** DISPATCHER EMERGENCY SHUTDOWN ***
RESOLUTION: Whenever this error is the root cause for a dispatcher dispatcher crash, it means that in this very same same trace you'll see see lots of entries related to the crash of every single work process. These entries look like this: -----------------------------*** ERROR => DpHdlDeadWp: W0 (pid 21104) died (severity=0, status=65280) [dpxxwp.c DpTraceWpStatus: child (pid=21104) exited with exit code 255 ------------------------------
1727]
We will then then search in the dispatcher trace for the very first occurrence of work process that crashed with entries similar to the one above. There we see that the work process 0 (zero) died. Then we need to change change the investigation investigation to the the work process trace. In this case it will be the "dev_w0", located at the work directory of the instance. Check the "usr/sap////work/dev_w#" >/work/dev_w#" trace to find out why work processes are being killed. Most common cause are problems to reach/connect to database. (FAIL TO SPOT TOPIC [1.1] FROM PRELIMINARY CHECKLIST)
VI: DISPATCHER NOT ALLOWED TO CONNECT TO MESSAGE SERVER: *** ERROR => not allowed to connect to message server via port XXXX [dpxxdisp.c 12175] *** ERROR => Please check your configuration (profile parameter rdisp/msserv_internal) [dpxxdisp.c 12176] DpHalt: shutdown server >PTSAPT01_QS1_03< (normal)
RESOLUTION: SAP note # 1285956 - Not allowed to connect to message server
VII: "GATEWAY COULD NOT BE STARTED": Straightforward. Dispatcher shut down as the gateway reader could not be started.
*** ERROR => gateway (pid 29081) died [dpxxdisp.c 16872] DpGwCheck: appc_adm.stat = 2 *** DP_FATAL_ERROR => Gateway could not be started - I better exit now
RESOLUTION: Gateway trace file "dev_rd" should be checked checked for the the rootcause. A very common cause is the unavailability unavailability of the gateway's default port 33 during instance startup. If this this is the case, the following following entries are seen in "dev_rd":
***LOG Q0I=> NiIBindSocket: bind (226: Address already in use) [nixxi.cpp 3738] *** ERROR => NiIBindSocket: SiBind failed for hdl 1/sock 11 (SI_EPORT_INUSE/226; I4; ST; 0.0.0.0:3310) [nixxi.cpp 3738]
*** In this case case the instance number number was 10, hence hence "0.0.0.0:3310". "0.0.0.0:3310". If you face the same situation, you have to involve your network/OS teams and find out what process has already bound the port.
VIII. Es2IResCreateShm: shmget failed: This issue is mainly seen on SAP systems systems based on HP-UX servers.
*** ERROR => Es2IResCreateShm: shmget failed (size= 17980981248. Check OS kernel parameter shmmax. (22: Invalid Invalid argument) [es2ux.c 219] *** ERROR => Es2ResCreateFiles: Es2IResCreate failed [es2xx.c 1324] *** ERROR => Es2ResCreate: Es2ResCreateFiles failed [es2xx.c 1507] *** ERROR => DpEmInit: Es2ResCreate (1) [dpInit.c 1642] *** ERROR => DpMemInit: DpEmInit (-1) [dpInit.c 1417] *** DP_FATAL_ERROR => DpSapEnvInit: DpMemInit *** DISPATCHER EMERGENCY SHUTDOWN ***
RESOLUTION: You must correctly set the OS kernel parameters as per below SAP note: # 172747 - SAP on HP-UX: OS kernel parameter recommendations Below SAP notes can provide additional info as well: # 180251 - HP-UX 64 Bit Kernel, Startup schlaegt fehlt # 1827388 - Syst System em cann cannot ot star startt due due to Es2I Es2IRe ResC sCre reat ate e - shmg shmget et fail failed ed - File File exis exists ts erro errorr
IX. MULTICAST ERROR: This error is present only as of kernel 740. The SAP Kernel release 740 uses a multicast address at the network 224.0.0.0/24 to dispatch tasks to the work processes. When required settings related to multicast are not in place, the instance will not start and in the "dev_disp.new" trace file, the following entries are logged:
*** ERROR => NiISetSockOpt: SiSetSockOpt failed for hdl 1/sock 6 (SI_EADDR_NAVAIL/19; I4; DG; IPPROTO_IP-IP_ADD_MEMBERSHI IPPROTO_IP-IP_ADD_MEMBERSHIP) P) [nixxi.cpp 9154] *** ERROR => NiIDgHdlGroupAdm: failed to join 224.0.0.129 on interface 0.0.0.0 [nixxi.cpp 9572] *** ERROR => RqQInit: NiDgHdlJoinGroup failed (rc=-10) [dpQueue.c 1559] *** ERROR => DpIPCInit2: RqQInit [dpxxtool2.c 514] *** DP_FATAL_ERROR => DpSapEnvInit: DpIPCInit2 *** DISPATCHER EMERGENCY SHUTDOWN ***
RESOLUTION: --> For customers with kernel 740, the following notes guide you on how to set up multicast multicast in a correct way: # 1932555 - Dispatcher startup issue with SAP Kernel 740 # 1931675 - IP mult ulticas icastt confi onfigu gura rattion ion for for disp dispat atch cher er wa wake keup up mecha echani nism sm --> For customers with kernel equal of higher than 741 PL 47, SAP does reccommend the usage of a new implementation based on events. This feature is enabled with the following parameter (which only exists as of kernel 741 PL 47): rdisp/queue/use_events_for_dispatching=on --> This parameter's default value value is ON starting starting from from SAP kernel 742.
X. APPLICATION SERVER'S SHARED MEMORY CORRUPTION:
*** ERROR => e=22 semop(3145913,(0,-1,6144),1) (22: Invalid argument) [semux.c 577] *** ERROR ERROR => DpITmSlotAllocate: SemRq [dpxxtool2.c [dpxxtool2.c 4333] *** ERROR => e=22 semop(3145913,(0,-1,6144),1) (22: Invalid argument) [semux.c 577] *** DP_FATAL_ERROR => DpTmDisconnect: SemRq *** DISPATCHER EMERGENCY SHUTDOWN ***
RESOLUTION: As these entries are related to a crash other than failed startup attempts, the resolution would be to start the instance once again. The easiest way to reproduce this scenario (and most common cause of the crash) is by removing the instance's IPC keys on runtime wich, by obvious reasons, *SHOULD NOT BE DONE*. # 1641668 - Various semop errors in workprocess traces
XI. WRONG EG (EXTENDED GLOBAL MEMORY) CONFIGURATION: *** ERROR => EgInit: requested segment size (105327362048 bytes) too large.Max size = 34359738367 bytes [egxx.c 244] *** ERROR => DpEmInit: EmInit (1) [dpxxdisp.c 10757] *** ERROR => DpMemInit: DpEmInit (-1) [dpxxdisp.c 10671] *** DP_FATAL_ERROR => DpSapEnvInit: DpMemInit *** DISPATCHER EMERGENCY SHUTDOWN ***
RESOLUTION: The instance will fail to start and log the above above entries in case the EG configuration is not maintained as per below SAP notes: # 785365 - Ext Extende ended d Globa loball Mem emor ory y (EG) EG) inc increas reased ed from from 2 GB to 8 GB # 1014010 - EG memory up to 32 GB (em/global_area_MB up to 32000) Hint By following this logic, any dispatcher related probem can be handled, even for different logged errors.
Hint There are SAP documentation for most of the possible cases. Research on them.
04. SAPCONTROL'S INSTANCE MANAGEMENT COMMANDS SAPSTARTSRV (through SAPCONTROL SAPCONTROL interface) offers a variety of web methods that can be used to monitor/manage the instances of the system. They are specially useful for handling startup error scenarios. The "must-know" ones are listed below and also the general pattern for executing them:
sapcontrol -nr -function
4.1: "GetSystemInstanceList": This one returns returns the list of active instances for the system, along with component list and the the current status. GetSystemInstanceList OK hostname, instanceNr, httpPort, httpsPort, startPriority, features, dispstatus testserver001, 0, 50013, 50014, 3, ABAP|GATEWAY|ICMAN|IGS, GREEN testserver001, 1, 50113, 50114, 1, MESSAGESERVER|ENQUE, GREEN testserver001, 2, 50213, 50214, 3, ABAP|GATEWAY|ICMAN|IGS, GREEN
EXPECTED OUTPUT: - One entry per instance of the system, including ASCS, SCS INSTANCES; - GREEN values for all entries on column "dispstatus". - For all instances running on the same host (column hostname), there must be a differente value on "instanceNr".
4.2: "GetProcessList": GetProcessList web method will bring the component list currently in place on any of the instances seen in the first command. It will come in handy as soon as the web method "GetSystemInstanceList" returns one instance reporting as YELLOW. This will happen when at least one component is presenting some kind of probem, so now it is possible to check that affected process with more details:
GetProcessList OK name, description, dispstatus, textstatus, starttime, elapsedtime, pid disp+work, Dispatcher, GREEN, Running, 2015 03 23 23:51:35, 263:27:26, 25549 igswd_mt, IGS Watchdog, GREEN, Running, 2015 03 23 23:51:35, 263:27:26, 263 :27:26, 25550 gwrd, Gateway, GREEN, Running, 2015 03 23 23:51:36, 263:27:25, 25590 icman, ICM, GREEN, Running, 2015 03 23 23:51:36, 263:27:25, 25591
4.3: "GetVersionInfo": Great for kernel consistency checks, this output will return the current kernel release + patch for each and every of its components within the instance informed as argument in the command line. Excluding the DB* libraries, all the remainning components of the instance must always share the same kernel and patch release on a consistent scenario.
GetVersionInfo OK Filename, VersionInfo, Time /usr/sap/WSO/DVEBMGS00/exe/sapstartsrv, /usr/sap/WSO/DVEBMGS00/exe/s apstartsrv, 742, patch 28, changelist 1540128, RKS compatibility level 0, optU (Nov 26 2014, 19:45:41), linuxx86_64, 2014 11 26 19:29:07 /usr/sap/WSO/DVEBMGS00/exe/disp+work, /usr/sap/WSO/DVEBMGS00/exe/d isp+work, 742, patch 28, changelist 1540128, RKS compatibility level 0, optU (Nov 26 2014, 19:45:41), linuxx86_64, 2014 11 26 19:36:32 /usr/sap/WSO/DVEBMGS00/exe/gwrd, /usr/sap/WSO/DVEBMGS00/exe/g wrd, 742, patch 28, changelist 1540128, RKS compatibility level 0, optU (Nov 26 2014, 19:45:41), linuxx86_64, 2014 11 26 18:48:09 /usr/sap/WSO/DVEBMGS00/exe/msg_server, /usr/sap/WSO/DVEBMGS00/exe/m sg_server, 742, patch 28, changelist 1540128, RKS compatibility level 0, optU (Nov 26 2014, 19:45:41), linuxx86_64, 2014 11 26 18:48:09 /usr/sap/WSO/DVEBMGS00/exe/dboraslib.so, /usr/sap/WSO/DVEBMGS00/exe/d boraslib.so, 742, patch 5, changelist 1503525, RKS compatibility level 0, dbgU (Jun 13 2014, 23:23:51), linuxx86_64, 2014 12 05 17:53:41 /usr/sap/WSO/DVEBMGS00/exe/dbmssslib.so, /usr/sap/WSO/DVEBMGS00/exe/d bmssslib.so, 742, patch 28, changelist 1540128, RKS compatibility level 0, optU (Nov 26 2014, 19:45:41), linuxx86_64, 2014 11 26 18:47:06 /usr/sap/WSO/DVEBMGS00/exe/dbdb2slib.so, /usr/sap/WSO/DVEBMGS00/exe/d bdb2slib.so, 742, patch 28, changelist 1540128, RKS compatibility level 0, optU (Nov 26 2014, 19:45:41), linuxx86_64, 2014 11 26 18:47:03 /usr/sap/WSO/DVEBMGS00/exe/dbdb4slib.so, /usr/sap/WSO/DVEBMGS00/exe/d bdb4slib.so, 742, patch 28, changelist 1540128, RKS compatibility level 0, optU (Nov 26 2014, 19:45:41), linuxx86_64, 2014 11 26 19:29:51 /usr/sap/WSO/DVEBMGS00/exe/dbdb6slib.so, /usr/sap/WSO/DVEBMGS00/exe/d bdb6slib.so, 742, patch 28, changelist 1540128, RKS compatibility level 0, optU (Nov 26 2014, 19:45:41), linuxx86_64, 2014 11 26 18:47:06 /usr/sap/WSO/DVEBMGS00/exe/dbsybslib.so, /usr/sap/WSO/DVEBMGS00/exe/d bsybslib.so, 742, patch 28, changelist 1540128, RKS compatibility level 0, optU (Nov 26 2014, 19:45:41), linuxx86_64, 2014 11 26 18:47:08 /usr/sap/WSO/DVEBMGS00/exe/enserver, /usr/sap/WSO/DVEBMGS00/exe/e nserver, 742, patch 28, changelist 1540128, RKS compatibility level 0, optU (Nov 26 2014, 19:45:41), linuxx86_64, 2014 11 26 18:47:48 /usr/sap/WSO/DVEBMGS00/exe/icman, /usr/sap/WSO/DVEBMGS00/exe/i cman, 742, patch 28, changelist 1540128, RKS compatibility level 0, optU (Nov 26 2014, 19:45:41), linuxx86_64, 2014 11 26 18:48:12 /usr/sap/WSO/DVEBMGS00/exe/sapwebdisp, /usr/sap/WSO/DVEBMGS00/exe/s apwebdisp, 742, patch 28, changelist 1540128, RKS compatibility level 0, optU (Nov 26 2014, 19:45:41), linuxx86_64, 2014 11 26 18:48:07 /usr/sap/WSO/DVEBMGS00/exe/jcontrol, /usr/sap/WSO/DVEBMGS00/exe/j control, 742, patch 28, changelist 1540128, RKS compatibility level 0, optU (Nov 26 2014, 19:45:41), linuxx86_64, 2014 11 26 18:49:16 /usr/sap/WSO/DVEBMGS00/exe/jlaunch, /usr/sap/WSO/DVEBMGS00/exe/j launch, 742, patch 28, changelist 1540128, RKS compatibility level 0, optU (Nov 26 2014, 19:45:41), linuxx86_64, 2014 11 26 19:08:36 /usr/sap/WSO/DVEBMGS00/exe/jstart, /usr/sap/WSO/DVEBMGS00/exe/j start, 742, patch 28, changelist 1540128, RKS compatibility level 0, optU (Nov 26 2014, 19:45:41), linuxx86_64, 2014 11 26 19:28:59
EXPECTED OUTPUT:
Hint There are hundreds of web methods available for the most diverse cases. They can be listed by typing just "sapcontrol" command as SIDadm user.
Hint These instance management commands can help to quickly find out version and specific component details.
CONCERNING THIS DOCUMENT... The original concept of this document is that it'll be never finished. Instead, constantly improved, based on the incoming feedback. For this reason, you are invited to share whether or not the information present here helped you to solve a startup problem and, even more importantly, in case it didn't. In that case you are welcome to share your own scenario and inform what was the solution applied, constraints, specifics.
COMING SOON... *** STARTUP issues faced during upgrade procedure...
