KPI Overview LTE Optimization Principles [RL40] Module 02
1
RA47042EN40GLA0
Course Outlook 1. 2. 3. 4. 5. 6. 7. 8.
3
RF Measurement Quantities KPI Overview Physical RF Optimization Call Setup Optimization Call Drop Analysis Mobility and Layering Capacity Management Peak Throughput Optimization
RA47042EN40GLA0
Module Objectives After completing this module, you will be able to: • Distinguish different classes of KPIs: network and e2e • Name the documentation for detailed definitions of NSN KPIs • Explain how to derive network KPIs based on performance counters • Explain the measurement of e2e KPIs • List the most important KPIs and their target values
4
RA47042EN40GLA0
Index • NSN counter-based KPIs • NSN field test KPIs
5
RA47042EN40GLA0
RL40 Measurement Examples MEAS ID M51120 M51121 M51122 M51123 M51125 M51126 M51127 M51128 M51129 M51130 M51131 M51132 M51136 M8000 M8001 M8004 M8005 M8006
6
Measurement LTE_IP_Stats LTE_PHB_Stats LTE_TOP_Stats LTE_Ethernet_Link LTE_IPSec LTE_IP_Filtering LTE_VLAN_IP_Stats LTE_VLAN_PHB_Stats VLAN_stats LTE_ETHIF_stats LTE_L2SWI_Stats LTE_TWAMP_Stats LTE_TAC_Stats LTE_S1AP LTE_Cell_Load LTE_Transport_Load LTE_Pwr_and_Qual_UL LTE_EPS_Bearer
RA47042EN40GLA0
MEAS ID M8007 M8008 M8009 M8010 M8011 M8012 M8013 M8014 M8015 M8016 M8017 M8018 M8019 M8020 M8021 M8022 M8023
Measurement LTE_Radio_Bearer LTE_RRC LTE_Intra_eNB_HO LTE_Pwr_and_Qual_DL LTE_Cell_Resource LTE_Cell_Throughput LTE_UE_State LTE_Inter_eNB_HO LTE_Neighb_Cell_HO LTE_Inter_Sys_HO LTE_ISYS_HO_UTRAN_NB LTE_eNB_Load LTE_ISYS_HO_GSM_NB LTE_Cell_Avail LTE_Handover LTE_X2AP LTE_UE_and_ServDiff
Accessibility E-UTRAN E-RAB Setup Success Ratio The KPI describes the setup success ratio of the elementary E-RAB setup procedure used to setup the E-RAB between MME and UE. It indicates the E-UTRAN contribution to network accessibility for the enduser, not the whole end-to-end service accessibility.
Logical formula E-RAB SSR=(E-RAB setup successes / ERAB setup attempts)*100%
7
Summarization formula (PI ID)
Summarization formula (Abbreviation)
100*sum([M8006C1]) / sum([M8006C0])
100*sum([EPS_BEARER_SETUP_COMPLETIONS]) / sum([EPS_BEARER_SETUP_ATTEMPTS])
RA47042EN40GLA0
Retainability E-RAB Normal Release Ratio, User perspective This KPI describes the ratio of normally released E-RABs from user perspective. This KPI is corresponding to a Connection Completion Ratio.
Logical formula
Summarization formula (PI ID)
100*sum([EPC_EPS_BEARER_REL_REQ_NORM]+ EPC_EPS_BEARER_REL_REQ_DETACH]+ [ENB_EPSBEAR_REL_REQ_RNL_REDIR]+ [ENB_EPS_BEARER_REL_REQ_RNL_INA]) / sum([EPC_EPS_BEARER_REL_REQ_NORM]+ [EPC_EPS_BEARER_REL_REQ_DETACH]+ [EPC_EPS_BEARER_REL_REQ_RNL]+ [EPC_EPS_BEARER_REL_REQ_OTH] + [ENB_EPSBEAR_REL_REQ_RNL_REDIR]+ [ENB_EPS_BEARER_REL_REQ_RNL_INA] + [ENB_EPS_BEARER_REL_REQ_RNL_RLF]+ [ENB_EPS_BEARER_REL_REQ_TNL]+ [ENB_EPS_BEARER_REL_REQ_OTH])
E-RAB NRR UP=(normal E-RAB releases user perspective / all E-RAB releases)*100%
8
Summarization formula (Abbreviation)
RA47042EN40GLA0
Mobility E-UTRAN HO Success Ratio, intra eNB This KPI describes the success ratio for the handover execution phase, when the source eNB receives information that the UE successfully is connected to the target cell within own eNB.
Summarization formula (PI ID)
Logical formula Intra HO SR =(number of successful intra eNB HOs) / (number of intra eNB HO attempts)*100%
9
RA47042EN40GLA0
Summarization formula (Abbreviation) 100*sum([SUCC_INTRA_ENB_HO]) / sum([ATT_INTRA_ENB_HO])
Mobility E-UTRAN HO Success Ratio, inter eNB X2 based This KPI describes the success ratio for the inter eNB X2 based handover execution phase, when the source eNB receives information that the UE successfully is connected to the target cell within target eNB.
Summarization formula (PI ID)
Logical formula Inter X2 based HO SR =(number of successful inter eNB X2 based HOs) / (number of inter eNB X2 based HO attempts)*100%
10
RA47042EN40GLA0
Summarization formula (Abbreviation) 100*sum([SUCC_INTER_ENB_HO]) / sum ([ATT_INTER_ENB_HO])
KPIs followed and reported from live LTE networks (by Technical Support)
11
RA47042EN40GLA0
Index • NSN counter-based KPIs • NSN field test KPIs
12
RA47042EN40GLA0
Field KPI Attach Time With Attach, the mobile terminal registers at the LTE network. At the end of the procedure the UE is authenticated, and a default (nGBR) bearer is established. The Attach Time is the interval between the connection request and the acknowledgement of the positive response by the UE.
Mean value and 95% from all measured samples Formula : Attach Time [ms] = tAttach Complete – tAttach Request Related E-UTRAN Counters & KPIs • None Target KPI values (ms) Initial Attach Re-Attach (w/o Authentication)
Min Mean Unloaded Load A* 241 271
95%-ile Load A* 362
110
145
125
*Target Load, or Load A. 60% of the available resources 13
RA47042EN40GLA0
Field KPIs Detach Time With an explicit Detach request the UE informs the LTE network that it does not want to access the EPS any longer. At the end of the procedure all EPS bearers of the UE are released. The Detach Time is the interval between the Detach Request and the reception of a Detach Accept message by the UE. No Detach Accept is sent by the network if the cause for Detach is switching the UE off. Formula : Detach Time [ms] = tDetachRequest – tDetachAccept Related E-UTRAN Counters & KPIs • None Target KPI values (ms) Detach Time
14
RA47042EN40GLA0
Min Mean Unloaded Load A 46 66
95%-ile Load A 88
Field KPIs Attach Success Rate The Attach Success Rate is defined as the ratio between the number of successful registrations and the number of all requests. This is the probability that a user can attach to the LTE network at any moment of time. The calculated success ratio figure excludes attach requests, which are rejected by authentication failures. On the other hand, network attach requests which are terminated by timer expiry (due to the unavailability of some LTE resource) are considered as unsuccessful registrations. If the success rate is calculated on the eNB by counting incoming RRC requests, RRC CONNECTION REQUEST retries are to be excluded, since they would increase the overall number of establishment attempts, and thus reduce the success ratio. number _ of _ successfull _ attachments LTENwAttSR 100 % Formula: number _ of _ successfull _ attachments Target KPI values: 95% Related E-UTRAN Counters & KPIs •PM counters: LTE EPS Bearer - M8006; •KPIs: Initial E-RAB Setup Success Ratio (ISSR) - LTE_5112a 15
RA47042EN40GLA0
Field KPIs Service Request Time: UE Initiated It is the time taken by the LTE network to setup an EPS bearer on request by the UE. The EPS bearer can be new (dedicated), or an existing one (e.g. the default EPS bearer). The latter is needed to reassign Uu radio and S1 bearer resources to the existing EPS bearer of a previously Idle UE. The EPS bearer has to be created or activated before IP packets can be exchanged, i.e. the Service Request creates the IP link dynamically over the mobile access (LTE) network. The UE Initiated EPS Bearer Setup Time is the interval between the submission of the message RRC CONNECTION REQUEST (see References 26. [3GPP36.331]) carrying a NAS: Service Request message in its body, and the reception of the RRC CONNECTION RECONFIGURATION response by the UE. The Bearer Setup procedure is based on the Service Request procedure, see References 3. [3GPP23.401] and 20. [3GPP36.300]. 16
RA47042EN40GLA0
Field KPIs Service Request Time: UE Initiated Continued Mean value and 95% from all measured samples Formula: Service Request Time [ms] = tRRC_Reconfig – tRRC_Request Target KPI values (ms): Min Mean 95%-ile Unloaded Load A Load A UE initiated 95 122 110
Related E-UTRAN Counters •Mean EPS Setup Time: SAEB.EstabTimeMean.QCI •Max EPS Setup Time: SAEB.EstabTimeMax.QCI
17
RA47042EN40GLA0
Field KPIs Service Request Time: Network Initiated •It is the time taken by the LTE network to set up an EPS bearer on request by the P-GW. The EPS bearer has to be created before IP packets can be sent (DL) to the UE if the UE has no proper EPS bearer for the given IP packet flow. The network initiated Service Request •Time includes a Paging Time (ref. to Paging Time) if the UE is idle. The EPS Bearer Setup procedure is based on the Paging and Service Request procedures according to References 26. [3GPP36.331] and 3.[3GPP23.401]. Mean value and 95% from all measured samples Formula: Service Request Time [s] = tRRC_Reconfig – tRRC_Request
Target KPI values (ms): Min Mean 95%-ile Unloaded Load A Load A NW initiated (w/o Paging) 172 145 212
Related E-UTRAN Counters (same as UE Initiated) •Mean EPS Setup Time: SAEB.EstabTimeMean.QCI •Max EPS Setup Time: SAEB.EstabTimeMax.QCI 18
RA47042EN40GLA0
Field KPIs Service Request Success Rate •This KPI is defined as the ratio between successfully established EPS bearers compared to the overall number of EPS bearer establishment attempts. It corresponds to the probability that a user or the LTE network can establish an EPS bearer at any moment in time. •Requests that are terminated by timer expiry (due to the non-accessibility of some LTE resource) are considered as unsuccessful attempts. Authentication errors (requests rejected by the MME) are included in the total number of failures. •Only the first RRC CONNECTION REQUEST is to be considered, since counting retries of the same message would increase the overall number of bearer establishment attempts, and thus reduce the success ratio.
19
RA47042EN40GLA0
Field KPIs Service Request Success Rate Continued Formula: EPSSR
number_of (RRC_CONN_ RECONFIGUR ATION_COMP LETE) 100 % number_of (RRC_CONNE CTION_REQU EST)
Target KPI values: 98% Related E-UTRAN Counters & KPIs • Group of related PM counters: LTE EPS Bearer - M8006 • KPI: Additional E-RAB Setup Success Ratio (ASSR) LTE_5113a
20
RA47042EN40GLA0
Field KPIs Service Drop Rate It is the ratio between abnormally released bearers and the overall number of established EPS bearers. An abnormal release is defined as any EPS bearer termination that was not triggered by the mobile user (from UE side). Thus, it reflects the probability that an established bearer is aborted due to insufficient network resources. Dropping the bearer becomes visible to the end-user if an application service is actively using it. If the application automatically re-establishes the bearer, it remains unnoticed by the user. Formula: number_of (dropped_c alls) EPSBearerD R
number_of (successfu ll_calls)
Target KPI values: < 2%
21
RA47042EN40GLA0
100 %
Field KPIs Testing methods for network wide testing Target: • Few selected KPIs that can be post-processed automatically from drive test logs. • Network wide drive testing executable with one drive (=one test case/script) per cluster. Different test methods: 1. Each session started from EMM-DEREGISTERED state 2. Each session started from RRC-IDLE EMM-REGISTERED state 3. Each session started from RRC-CONNECTED state
22
RA47042EN40GLA0
Field KPIs Testing methods 1. Each session started from EMM-DEREGISTERED state • • •
•
•
Attach – Application data – Detach Several data applications can be included in the same session (FTP DL/UL, HTTP, Ping) Possible to measure Success Rates/times: Attach, ERAB setup, RRC setup, Application setup, HO Not reflecting end user behavior, as users typically start sessions from EMMREGISTERED state. sesssion1
Example script • • • • • •
Attach FTP DL (5MB) FTP UL (3MB) HTTP page DL (x5) Ping 32-1024B (x10) Detach
Session 1 stop
23
RA47042EN40GLA0
Time 13:56:34.094 13:56:34.094 13:56:34.097 13:56:34.127 13:56:34.132 13:56:34.366 13:56:34.368 13:56:34.369 13:56:34.390 13:56:34.410 13:56:34.414 13:56:34.416 13:56:34.416 13:56:34.424 13:56:34.458 13:56:34.458 13:56:34.459 13:56:35.464 13:56:56.606 13:56:56.607 13:56:56.662 13:56:56.663 13:56:56.665 13:56:57.142 13:56:57.142
Subchannel Direction Uplink Uplink CCCH Uplink CCCH Downlink DCCH Uplink DCCH Downlink DCCH Uplink DCCH Downlink DCCH Uplink DCCH Downlink DCCH Uplink Downlink Downlink DCCH Uplink Uplink Uplink DCCH Uplink DCCH Uplink Uplink DCCH Uplink DCCH Downlink DCCH Downlink Downlink BCCH_BCH Downlink BCCH Downlink
Message ATTACH_REQUEST PDN_CONNECTIVITY_REQUEST RRCConnectionRequest RRCConnectionSetup RRCConnectionSetupComplete SecurityModeCommand SecurityModeComplete UECapabilityEnquiry UECapabilityInformation RRCConnectionReconfiguration RRCConnectionReconfigurationComplete ATTACH_ACCEPT ACTIVATE_DEFAULT_EPS_BEARER_CONTEXT_REQUEST MeasurementReport ATTACH_COMPLETE ACTIVATE_DEFAULT_EPS_BEARER_CONTEXT_ACCEPT ULInformationTransfer MeasurementReport DETACH_REQUEST ULInformationTransfer DLInformationTransfer RRCConnectionRelease DETACH_ACCEPT SYSTEM_INFORMATION_BCH MASTER_INFORMATION_BLOCK
Field KPIs Testing methods 2. Each session started from RCC IDLE, EMM-REGISTERED state • • • •
RCC setup - Application data – Wait – Release due to inactivity Several data applications can be included in the same session (FTP DL/UL, HTTP, Ping) Possible to measure Success Rates/times: Service request, ERAB setup, RRC setup, Application setup, HO Challenges in practical measurement setup, especially with inactivity timer >10s. Requires specific firewall settings to measurement laptop to block all outgoing packages during wait period. sesssion1
•
Example script • • • • • •
RRC setup/service request FTP DL (5MB) FTP UL (3MB) HTTP page DL (x5) Ping 32-1024B (x10) Wait until released by eNB
session2
session3
24
RA47042EN40GLA0
Time 14:25:52.537 14:25:52.540 14:25:52.574 14:25:52.576 14:25:52.577 14:25:52.581 14:25:52.594 14:25:53.634 14:26:06.540 14:26:17.957 14:26:18.021 14:26:18.023 14:26:18.059 14:26:18.061 14:26:18.061 14:26:18.066 14:26:18.077 14:26:19.117 14:26:57.044 ….
Subchannel CCCH DCCH DCCH DCCH DCCH DCCH DCCH DCCH DCCH CCCH CCCH DCCH DCCH DCCH DCCH DCCH DCCH DCCH DCCH
Direction Downlink Uplink Downlink Uplink Downlink Uplink Uplink Uplink Downlink Uplink Downlink Uplink Downlink Uplink Downlink Uplink Uplink Uplink Downlink
Message RRCConnectionSetup RRCConnectionSetupComplete SecurityModeCommand SecurityModeComplete RRCConnectionReconfiguration RRCConnectionReconfigurationComplete MeasurementReport MeasurementReport RRCConnectionRelease RRCConnectionRequest RRCConnectionSetup RRCConnectionSetupComplete SecurityModeCommand SecurityModeComplete RRCConnectionReconfiguration RRCConnectionReconfigurationComplete MeasurementReport MeasurementReport RRCConnectionRelease
Field KPIs Testing methods 3. Each session started from RCC Connected, EMMREGISTERED state • • • •
Application data only Several data applications can be included in the same session (FTP DL/UL, HTTP, Ping) Possible to measure Success Rates/times: Application setup, HO Only mobility and re-establishment/drop related signaling is seen.
•
Example script • • • •
25
FTP DL (5MB) FTP UL (3MB) HTTP page DL (x5) Ping 32-1024B (x10)
RA47042EN40GLA0
Field KPIs Testing methods Example list of KPIs that could be measured with different methods: KPIs Attach SR Attach time Session setup SR from Deregistetered Session setup time from Deregistered Session setup time from RRC idle Session setup SR from RRC idle LTE intra freq HO SR LTE inter freq HO SR Call/Session drop rate # drops/minute Re-establishment success rate FTP/HTTP/PING SR RTT PING FTP DL throughput FTP UL throughput HTTP page download time
1 x x x x
x x x x x x x x x x
2
x x x x x x x x x x x x
3
Comment Attach request -> Attach complete Attach request -> Attach complete RRCC request -> ULinformation transfer (Attach complete) RRCC request -> ULinformation transfer (Attach complete) RRCC request -> RRCC reconf. Complete RRCC request -> RRCC reconf. Complete
x x x x x x x x x
Recommended method for mass testing, closest match to user experience Could be used, if Attach success rate measurements are mandatory
26
RA47042EN40GLA0
