Nokia Siemens LTE E2E Field Network Performance

Nokia Siemens Networks LTE Radio Access, Rel. RL15, Operating Documentation Prerelease, Issue 01

LTE E2E Field Network Performance Definitions of Key Performance Indicators DN0972412 Issue 01 DRAFT APPROVED Approval Date 2010-06-22

LTE E2E Field Network Performance - Definitions of Key Performance Indicators

The information in this document i s subject to change without notice and descri bes only the product defined in the introduction of this documentation. This documentation is in tended for the use of Nokia Siemens Networks c ustomers only for the purposes of the agreement under which the document is submitted, and no part of it may be used, reproduced, modified or transmitted in any form or means without the prior written permission of Nokia Siemens Networks. The documentation has been prepared to be used by pr ofessional and properly trained personnel, and the customer assumes full responsibility when using it. Nokia Siemens Networks welcomes customer comments as part of the pr ocess of continuous development and improvement of the documentation. The information or statements given in this documentation concerning the suitability, capacity, or performance of the mentioned hardware or so ftware products are given "as is" and all liability arising in connection with such hardware or software products shall be defined conclusively and finally in a separate agreement between Nokia Siemens Networks and the customer. However , Nokia Siemens Networks has made all reasonable efforts to ensure that the instructions contained in the document are adequate and free of m aterial errors and omissions. N okia Siemens Networks will, if deemed necessary by Nokia Siemens Networks, explain issues whi ch may not be covered by the document. Nokia Siemens Networks will correct errors in this documentation as soon as possible. IN NO EVENT WILL Nokia Siemens Networks BE LIABLE FOR ERRORS IN THIS DOCUMENTATION OR FOR ANY DAMAGES, INCLUDING B UT NOT LIMITED TO SPECIAL, DIRECT, INDIRECT, INCIDENTAL OR CONSEQUENTIAL OR ANY LOSSES, SUCH AS BUT NOT LIMITED TO LOSS OF PROFIT, REVENUE, BUSINESS INTERRUPTION, BUSINESS OPPORTUNITY OR DATA,THAT MAY ARISE FROM THE USE OF THIS DOCUM ENT OR THE INFORMATION IN IT. This documentation and the product it desc ribes are considered protected by copyrights and other intellectual property rights according to the applicable laws. The wave logo is a trademark of Nokia Siemens Networks Oy. Nokia is a registered trademark of Nokia Corporation. Siemens is a registered trademark of Siemens AG. Other product names mentioned in this document may be trademarks of their respective owners, and they are mentioned for identification purposes only. Copyright © Nokia Siemens Networks 2010. All rights reserved

f

Important Notice on Product Safety Elevated voltages are inevitably present at specific points in this electrical equipment. Some of the parts may also have elevated operating temperatures. Non-observance of these conditions and the saf ety instructions can result in personal injury or in property damage. Therefore, only trained and qualified personnel may install and maintain the system . The system complies with the standard EN 6 0950 / IEC 60950. All equipment connected has to comply with the applicable safety standards.

The same text in German: Wichtiger Hinweis zur Produktsicherheit In elektrischen Anlagen stehen zwangsläufig bestimm te Teile der Geräte unter Spannung. Einige Teile können auch eine hohe Betri ebstemperatur aufweisen. Eine Nichtbeachtung dieser Situation und der Warnungshinweise kann zu Körperverletzungen und Sachschäden führen. Deshalb wird vorausgesetzt, dass nur geschultes und qualifiziertes Personal die Anlagen installiert und wartet. Das System entspricht den Anforderungen der EN 60950 / IEC 60950. Angeschlossene Geräte müssen die zutreffenden Sicherheitsbestimmungen erfüllen.

2

Id:0900d805807a32c5

DN0972412 Issue 01 DRAFT APPROVED


The information in this document i s subject to change without notice and descri bes only the product defined in the introduction of this documentation. This documentation is in tended for the use of Nokia Siemens Networks c ustomers only for the purposes of the agreement under which the document is submitted, and no part of it may be used, reproduced, modified or transmitted in any form or means without the prior written permission of Nokia Siemens Networks. The documentation has been prepared to be used by pr ofessional and properly trained personnel, and the customer assumes full responsibility when using it. Nokia Siemens Networks welcomes customer comments as part of the pr ocess of continuous development and improvement of the documentation. The information or statements given in this documentation concerning the suitability, capacity, or performance of the mentioned hardware or so ftware products are given "as is" and all liability arising in connection with such hardware or software products shall be defined conclusively and finally in a separate agreement between Nokia Siemens Networks and the customer. However , Nokia Siemens Networks has made all reasonable efforts to ensure that the instructions contained in the document are adequate and free of m aterial errors and omissions. N okia Siemens Networks will, if deemed necessary by Nokia Siemens Networks, explain issues whi ch may not be covered by the document. Nokia Siemens Networks will correct errors in this documentation as soon as possible. IN NO EVENT WILL Nokia Siemens Networks BE LIABLE FOR ERRORS IN THIS DOCUMENTATION OR FOR ANY DAMAGES, INCLUDING B UT NOT LIMITED TO SPECIAL, DIRECT, INDIRECT, INCIDENTAL OR CONSEQUENTIAL OR ANY LOSSES, SUCH AS BUT NOT LIMITED TO LOSS OF PROFIT, REVENUE, BUSINESS INTERRUPTION, BUSINESS OPPORTUNITY OR DATA,THAT MAY ARISE FROM THE USE OF THIS DOCUM ENT OR THE INFORMATION IN IT. This documentation and the product it desc ribes are considered protected by copyrights and other intellectual property rights according to the applicable laws. The wave logo is a trademark of Nokia Siemens Networks Oy. Nokia is a registered trademark of Nokia Corporation. Siemens is a registered trademark of Siemens AG. Other product names mentioned in this document may be trademarks of their respective owners, and they are mentioned for identification purposes only. Copyright © Nokia Siemens Networks 2010. All rights reserved

f

Important Notice on Product Safety Elevated voltages are inevitably present at specific points in this electrical equipment. Some of the parts may also have elevated operating temperatures. Non-observance of these conditions and the saf ety instructions can result in personal injury or in property damage. Therefore, only trained and qualified personnel may install and maintain the system . The system complies with the standard EN 6 0950 / IEC 60950. All equipment connected has to comply with the applicable safety standards.

The same text in German: Wichtiger Hinweis zur Produktsicherheit In elektrischen Anlagen stehen zwangsläufig bestimm te Teile der Geräte unter Spannung. Einige Teile können auch eine hohe Betri ebstemperatur aufweisen. Eine Nichtbeachtung dieser Situation und der Warnungshinweise kann zu Körperverletzungen und Sachschäden führen. Deshalb wird vorausgesetzt, dass nur geschultes und qualifiziertes Personal die Anlagen installiert und wartet. Das System entspricht den Anforderungen der EN 60950 / IEC 60950. Angeschlossene Geräte müssen die zutreffenden Sicherheitsbestimmungen erfüllen.

2

Id:0900d805807a32c5



Table of Contents This document has 79 pages. Summary of Changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2 2.1 2. 1 2.2 2. 2

Measuremen entt Methods and Reference Condit itiion ons s . . . . . . . . . . . . . . . . 11 Int ntro rodu duct ctio ion n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Refer eren enc ce Con ond diti tio ons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.2.1 2.2. 1 2.2. 2. 2.2 2 2.2. 2. 2.3 3

Distin Dist ingu guis ishe hed d Cel Celll Po Posi siti tion ons. s. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Chan Ch anne nell Mo Mode dels. ls. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Temp Te mpla late te fo forr the the De Desc scri ript ptio ion n of Re Refe fere renc nce e Co Cond ndit itio ions ns . . . . . . . . . . . . . 14

2.3 2.3 2.3. 2. 3.1 1 2.3.1. 2.3 .1.1 1

Mea eas sur ure emen entt Scen ena ari rio os . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Thro Th roug ughp hput ut Me Meas asur urem emen ents ts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Cell Ce ll Thr Throug oughp hput ut . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.3.1.2 2.3.1. 2 2.3 2. 3.2 2.3.2. 2.3 .2.1 1

Peak Use Peak Userr Dat Data a Rate. Rate. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 Late La tenc ncy y Me Meas asur urem emen ents ts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 Roun Ro und d Tri Trip p Time Time Me Meas asur ureme ement nt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.3.2.2 2.3.2. 2 2.3.2. 2.3 .2.3 3 2.3. 2. 3.3 3

Signal Sign aling ing an and d Medi Media a Del Delay ay Me Meas asur ureme ement nts s . . . . . . . . . . . . . . . . . . . . . . 21 Refer Re ferenc ence e Cond Condit itio ions ns fo forr Late Latenc ncy y Mea Measu sure remen ments. ts. . . . . . . . . . . . . . . . . 22 Serrvi Se vic ce Acc Acces essi sibi bili lity ty Me Meas asur urem emen ents ts . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

2.3.4 2.3. 4 2.3.4. 2.3 .4.1 1 2.3.4. 2.3 .4.2 2

Reliab Reli abil ilit ity y Mea Measu sure reme ment nts. s. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Tran Tr ansp spor ortt Erro Errorr Rat Rates es . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Servic Ser vice e Drop Drop Rate Rates s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

2.3.5 2.3. 5 2.4 2. 4 2.4. 2. 4.1 1

Mobili Mobi lity ty (H (Han ando dove ver) r) Me Meas asur urem emen ents. ts. . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 Mea eas sur ure emen entt Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 Fiel Fi eld d Net Netwo work rk Cl Clus uste ter. r. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

2.4.2 2.4. 2 2.4. 2. 4.3 3 2.4.3. 2.4 .3.1 1

Triall Lab. Tria Lab. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 Meas Me asur urem emen entt Dat Data a Co Coll llec ecti tion on . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 Subsc Sub scrib riber er and Equ Equipm ipment ent Tr Trac aces es.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

2.4.3.2 2.4.3. 2 2.5 2. 5

PM Cou Count nter ers s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Mea eas sur ure emen entt Report rtiing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3

Defin iniitions of Ke Key y Pe Perrfo forrman anc ce Indica cattors (KP KPIIs) . . . . . . . . . . . . . . . . . 34

3.1 3.1 3.1. 3. 1.1 1 3.1 3. 1.2

Ove verv rviiew . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 Appl Ap plic icat atio ion n Ser Servi vice ces s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 LTE LT E Net Netwo work rk Se Serv rvic ices es . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

3.1.2.1 3.1.2. 1 3.1.2. 3.1 .2.2 2 3.2 3. 2

LTE E2E LTE E2E Netw Network ork Ser Servi vice ce . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 Radi Ra dio o Bear Bearer er Ser Servi vice ces s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 KPI Def efiini nittio ion n Te Temp mpllate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

3.3 3.3 3.3. 3. 3.1 1 3.3.1. 3.3 .1.1 1

App ppllicati tio on Se Serv rviice ces s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 Datta Dow Da Downl nlo oad / Upl Uploa oad d (FT (FTP) P) Ser ervi vice ce KP KPIIs . . . . . . . . . . . . . . . . . . . . 39 (FTP (F TP)) Serv Servic ice e Acce Access ssibi ibili lity ty Rat Ratio. io. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

3.3.1.2 3.3.1. 2 3.3.1. 3.3 .1.3 3 3.3.1. 3.3 .1.4 4

(FTP)) C (FTP Comp omple lete ted d Ses Sessi sion on Rat Ratio io . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 (FTP (F TP)) Ses Sessi sion on Ti Time. me. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 (FTP (F TP)) Ser Servic vice e Acc Acces ess s Tim Time e . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

3.3.1.5 3.3.1. 5 3.4 3. 4

(FTP)) Use (FTP Userr Data Data Ra Rate te . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 LTE Ne Networ ork k Se Serrvi vic ces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46


Id:0900d805807a32c5

3


4

3.4.1

LTE E2E Network KPIs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

3.4.1.1 3.4.1.2 3.4.1.3

Attach Time. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 Detach Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 Attach Success Rate. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

3.4.1.4 3.4.1.5 3.4.1.6

Service Request Time (EPS), UE Initiated . . . . . . . . . . . . . . . . . . . . . . . 53 Service Request (EPS) Time, Network Initiated . . . . . . . . . . . . . . . . . . . 57 Service Request (EPS) Success Rate . . . . . . . . . . . . . . . . . . . . . . . . . . 58

3.4.1.7 3.4.1.8 3.4.1.9

Service (EPS Bearer) Drop Rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 Handover Procedure Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 Handover Success Rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

3.4.1.10 3.4.1.11 3.4.1.12

Paging Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 Paging Failure Rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 (LTE) Round Trip Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

3.4.1.13 3.4.1.14 3.4.1.15

(LTE) User Data Rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 (LTE) Packet Loss Rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 (LTE) Service Interrupt Time (HO) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

3.4.2 3.4.2.1 3.4.2.2

Radio Bearer KPIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 (RB) Packet Loss Rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 (RB) User Data Rate. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

3.4.2.3

Cell Throughput . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

4

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

Id:0900d805807a32c5



List of Figures Figure 1

LTE Bearer Service Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Figure 2 Figure 3 Figure 4

Distribution of Users for Cell Throughput Measurements . . . . . . . . . . . 17 Reference Cluster for Field Performance Verification Tests . . . . . . . . . 30 KPIs for Packet Switched Data Services . . . . . . . . . . . . . . . . . . . . . . . . 43


Attach Procedure w/ Initial EPS Bearer Establishment (3GPP 23.401). 48 UE-Initiated Detach Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 EPS Bearer Setup Procedure, Part 1 . . . . . . . . . . . . . . . . . . . . . . . . . . 55


EPS Bearer Setup Procedure, Part 2 . . . . . . . . . . . . . . . . . . . . . . . . . . 56 Handover Procedure: Intra MME/S-GW, Inter eNB via X2 . . . . . . . . . . 63 Handover Procedure: Intra MME/S-GW, Inter eNB via S1 . . . . . . . . . . 64

Figure 11

Paging Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68


Id:0900d805807a32c5

5


List of Tables

6

Table 1

Channel Models and Doppler Frequencies . . . . . . . . . . . . . . . . . . . . . . 13

Table 2 Table 3 Table 4

Description of Reference Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 KPI Categories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Reference Conditions for Cell Throughput Measurements . . . . . . . . . . 18


Reference Conditions for User Data Rate Measurements . . . . . . . . . . . 20 Reference Conditions for Latency Measurements . . . . . . . . . . . . . . . . . 23 Reference Conditions for Success Rate Measurements . . . . . . . . . . . . 24


Reference Conditions for Packet Loss Rate Measurements . . . . . . . . . 26 Reference Conditions for Service Drop Rate Measurements . . . . . . . . 28 Reference Conditions for Handover Measurements . . . . . . . . . . . . . . . 29


Overview of Application Service KPIs . . . . . . . . . . . . . . . . . . . . . . . . . . 35 Overview of LTE E2E Network Service KPIs . . . . . . . . . . . . . . . . . . . . . 36 Overview of LTE Radio Bearer Service KPIs . . . . . . . . . . . . . . . . . . . . 37

Table 14

KPI Definition Template . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

Id:0900d805807a32c5



Summary of Changes

Summary of Changes Issue History

Issue number

Date of issue

01 DRAFT

2010-06-14


Reason for update

First issue for RL15 pre-release

Id:0900d805807a32af

7

Summary of Changes

8


Id:0900d805807a32af



Introduction

1 Introduction The current document provides the following information: •

Definitions of End-to-End Key Performance Indicators ( E2E KPI s) for Long-Term Evolution (LTE )

•

Descriptions of measurement methods for field and lab trials

KPIs are basic network quality indicators for offer processing and field acceptance tests. The KPIs specified in this document are E2E related KPIs; they h elp network operators and Nokia Siemens Networks to determine the quality of deployed LTE networks. In order to prove that the performance of an operating network matches the targeted quality, field acceptance tests are carried out. As multiple factors may influence the performance of an operational network, detailed conditions, under which network KPIs are valid, must be specified and agreed upon. These include system performance, network planning and dimensioning, measurement campaign planning and parameterization. These conditions will be reffered to as "reference conditions" in the sequel. LTE system architecture was designed to meet the following major goals. These are also the constant focus of lab and field performance tests (for more details, see References 7. [3GPP25.913]): •

Improved system capacity & coverage

•

High user data rates (peak upload and download rates)

•

Reduced latency (one-way delay and RTT)

•

Simplified architecture, IP based transport infrastructure

•

Superior user experience

•

Seamless connection to legacy networks (GSM, UMTS, 3GPP2/CDMA)

These improvements can be assessed with K PI measurements as defined in this document. The KPIs are defined for different service levels (see References 20. [3GPP 36.300], and Figure 1 LTE Bearer Service Architecture): a) Application Services b) LTE E2E Network Service c) Radio Bearer ( RB) Service d) IP based LTE Bearer Services: X2, S1, S5/S8 and External

The KPIs of group B (LTE E2E Network Service) characterize the quality of the LTE network as IP transport service for applications. They are in the center of field performance investigations, because they characterize the overall LTE network performance. The Application Service KPIs of group A (Application Services) show the user-perceived quality of application services as impacted by the LTE network. The KPIs of this group are service specific. The following services are of common interest of mobile network operators: •

Web Browsing

•

Data Upload / Download

• Audio Video Streaming •

Conversational Audio Video (VoIP and Video Telephony)

This paper concentrates on the Data Upload / Download (File Transfer Protocol (FTP )/Transmission Control Protocol ( TCP)) application service, because it is the


Id:0900d805807a32b1

9

Introduction


support of this service that stands in focus of the first commercial NSN LTE product release. In addition, KPIs for the radio link (RB Service, group C - Radio Bearer Service) are also specified, because the radio link is the most critical component of the E2E transport path. The other bearers, the E-UTRAN Radio Access Bearer ( E-RAB) and Evolved Packet System ( EPS) bearers are composed of the RB Service and of the IP based bearer services (of group D - IP based LTE Bearer Services: X2, S1, S5/S8 and External). Monitoring the performance of all bearer services is necessary to fully understand the E2E transport behavior of the LTE network and to optimize the end-user perceived quality of application services running on top of it.

Figure 1

LTE Bearer Service Architecture

While the KPI definitions (see Definitions of Key Performance Indicators (KPIs)) are independent of LTE product releases, Measurement Scenarios focuses on the first product release in order to restrict the number of possible scenario combinations (e.g. by not considering scenarios with Guaranteed Bit Rate ( GBR ) bearers).

10

Id:0900d805807a32b1



Measurement Methods and Reference Conditions

2 Measurement Methods and Reference Conditions 2.1

Introduction The measurement method to be applied for field performance verification depends on the specifics of the measured Key Performance Indicator ( KPI ). End-user perceived KPIs are measured differently from network performance KPIs, and the conditions for delay measurements are different from those for acc essibility KPIs. For this reason, this Section discusses measurement methods according to KPI categories. Since the end-user experience of application services is mostly dependent on throughput and latency, these aspects together with Handover ( HO) scenarios constitute the core part of field performance verification measurements. While the scope of KPI definitions follows the performance goal s of the LTE system (for details see References 7. [3GPP25.913]), the selected methods and conditions of field performance tests have to ensure that typical conditions in operational networks are covered. Additionally, the measurement conditions also include relevant impairment factors of the radio link, because the radio link has the most influence on E2E performance. Especially, those characteristics of the Orthogonal Frequency-Division Multiplexing (OFDM) encoding technique, which have side effects for radio network performance are considered in the selection of conditions, e.g.: •

Sensitivity to Doppler shift

•

Sensitivity to frequency synchronization problems

•

Sensitivity to multipath effects and cell area (urban, sub-urban, rural)

•

Dependency on used bandwidth and frequency range

•

Dependency on used multiple antenna techniques - Multiple Inputs Multiple Outputs (MIMO) (transmit diversity, spatial multiplexing)

These aspects have a major bearing on the design of measurement scenarios. For example, investigating the impact of Doppler shift on network performance, e.g. on user and cell data rates, on delay and packet loss KPIs requires that high speed movement becomes part of the measurement scenario. Another aspect would be the multipath reception, which decreases the impact of the Doppler shift. Thus, the measurement setup should also include space diversity if worst-case conditions are being discussed measurements with these conditions have to show, how the investigated KPI figure depends of the velocity and terminal category of the receiving user. Some of the measurement scenarios might be much too costly to implement in live network environments, or it may be impossible to ensure the desired radio conditions during the whole duration of the test. In such cases, lab trials with simulated radio links are acceptable as replacement of live network tests. The combination of all different options for reference con ditions and measurement scenarios would result in a very large number of test cases, which is simply n ot feasible. It will be the task of test specifications to select options and scenarios out of the outlined possibilities for a given LTE performance verification campaign considering the available resources and time constraints for test execution.


Id:0900d805807a32b3

11


2.2

Reference Conditions Performance figures can only be interpreted correctly if the exact conditions under which they have been acquired are known. This chapter describes t he "reference conditions" under which LTE field performance tests should be executed. They are formulated on an abstract level.

2.2.1

Distinguished Cell Positions Operators are keen to know the distribution of throughput and latency KPI figures (e.g. peak user data rate, cell UL/DL throughput, network delay, etc.) across the cells of a "reference cluster". However, it is impossible or would be associated with prohibitive high costs to measure these KPIs in a live network environment in so many cell positions that a Cumulative Distribution Function ( CDF ) can be calculated for 5%, 50%, or 95% of the cell area. For this reason, it is proposed to replace the dist ribution function by KPI figures measured on three characteristic positions of the cell (in the sequel, the term "cell" will be used to denote both omni-directional and directional cells): a) Very Good radio conditions (line of sight close to cell center within 50m, low path loss) b) Medium radio conditions (BTS distance of ~1/2 cell radius, channel with medium noise conditions, medium path losses) c) Cell Edge conditions (cell overlapping area, receive power from neighbor cells at about the same level).

The radio conditions are formulated in terms of Signal to Interference and Noise Ratio (SINR) and receive power levels. The measured KPI figures in Good, M edium and Cell Edge positions replace their distribution across the LTE cell and will be considered as their Peak, Average and Minimum values, respectively. The three variants are given by mean and 95% values calculated from all measurement samples, where the number of samples is to be derived from the required confidence level. If no suitable cell positions are available in the reference cluster, the tests can be executed in the laboratory, where the radio conditions are to be simulated. If automated test execution is possible, even the calculation of CDF function values at 5%, 50% and 95% distribution is an option. This shows that often the full set of results can only be produced by combined field and lab trials. In certain cases, even simulations might become necessary to accomplish (incomplete) measurement results. For this reason, it is best practise to measure all E2E field network KPIs in vendor lab before going into field tests with the operator. The description of test conditions (see Table 2 Description of Reference Conditions, row "Setup") specifies which test methods are recommended.

12

Id:0900d805807a33b4



2.2.2

Channel Models Several factors of the radio link have influence on the E2E performance of application, or LTE network services. Their importance is dependent on the category of the KPI to be measured. The following parameters are checked for each scenario: •

UE category

• Antenna systems on UE and eNB •

Position and distribution of terminals in the cell, terminal movement

•

Load conditions in the measured cell (UL/DL, data volume)

•

Receive power level, interference and noise conditions, diversity and fading

•

Cell area, or clutter type (urban, sub-urban, rural)

In order to reduce the number of possible combinations, mobile terminal speed and radio propagation conditions for different environments and distances were combined in "channel models". The relevant channel models for LTE are described in References 30. Annex B of [3GPP36.521-1]. The fo llowing three channel models are defined: •

EPA - Extended Pedestrian A

•

EVA - Extended Vehicular A

•

ETU - Extended Typical Urban

In addition, 3GPP proposes some combinations of channel models and Doppler frequencies, which are complemented in Table 1 Channel Models and Doppler Frequencies with scenarios for •

stationary UEs (EPA0, EVA0 and ETU0), and

•

high speed train (ETU250)

The high speed train scenario with app. 250 km/h uses a carrier frequency of 900 MHz (former GSM band) since it is assumed that initial LTE deployments will use this frequency in rural areas. If the assumption is not true for a given customer t rial, the Doppler frequency will change proportionally to the ap plied carrier frequency (500Hz @ 1.8 GHz, 583 Hz @ 2.1 GHz, 722 Hz @ 2.6 GHz). The current document uses o nly these models as reference models. Model

Source

Maximum Doppler Frequency

EPA0

0 Hz (stationary UE)

EVA0


ETU0


EPA5

3GPP36.521-1, Annex B; see 30.

5Hz

EVA5


5Hz

EVA70


70Hz

ETU70


70Hz

ETU250 ETU300 Table 1


250 Hz (with 900 MHz carrier frequency) 3GPP36.521-1, Annex B; see 30.

300Hz

Channel Models and Doppler Frequencies

Id:0900d805807a33b4

13


Channel model conditions can be set in lab trials with simulated radi o links precisely, but they cannot be ensured in field network tests. In field trials, radio conditions with best suit to the channel model are used to calibrate trial lab results.

2.2.3

Template for the Description of Reference Conditions Table 2 Description of Reference Conditions includes the channel model together with other parameters to summarize the essential conditions for measurement scenarios. The column "Reference Condition" gives textual descriptions and possible settings as abstract values only. The meaning of rows is as follows: •

"Antenna" identifies the number, type and modulation of antennas used on the UE and on the eNB side. The reference UE configuration comprises two receive antennas in DL and one antenna in UL. (However, it may not be fr eely selectable in reference clusters of operational networks.)

•

"The "Application" parameter identifies the test application as a real application, or an equivalent replacement (test script), which allows the collection of information necessary to calculate / extrapolate the KPI figure.

•

The "Setup" parameter identifies the environment , where the KPI can best be m easured. It is not always the reference cluster of the live network. Some conditions, e.g. SINR are easier to set in the lab. – The "Reference Cluster" denotes a selected area of the ope rator´s live network performance verification tests. In the cluster typical load conditions apply, as generated by real users -stationary or mobile - and the radio conditions are determined by cell location (rural, or city area, high, or low buildings, etc.). The conditions are given and cannot be modified during the trial. – The "Trial Lab" is either a vendor or op erator lab used for executing performance tests, where exact measurement conditions can be en sured. The lab allows the background traffic mix, load levels, and ra dio conditions to be precisely set. The simulation of mobile users of high velocity is also possible. – "Simulation" results are applied where field and even lab trials are expensive or cannot be executed in the given time or budget.

14

•

"Concurrent Load" describes the load which is to be generated in addition (i.e. background traffic) to the traffic produced by active test terminals in the measured cell. Load conditions in neighbor cells are the operational load in field network clusters, and are set to medium load in the lab.

•

"Drive test "Routes" are defined for mobile terminals in live LTE environments. In trial labs, terminal movement and hand-over are simulated (by varying the channel conditions).

•

"Backhaul Capacity" defines the necessary throughputs of transport links if their values deviate from default and are important for test execution, e.g. to prevent a transport link becoming bottleneck in user data rate, or cell throughput measurements. It also defines other parameters, like Packet Delay (PD), Packet Delay Variation (PDV ), Packet Loss Rate ( PLR) and their expected values. These parameters are observed in field trials, and are set to their maximal allowed values in the lab.

Id:0900d805807a33b4



Parameter

Reference Condition

Terminal (UE)

Type

Mobile or smart phone, laptop with LTE card, or w/ modem

Category

UE categories 1-5 according to [3GPP36.306]; see 21.

Antenna

Type: SISO, MIMO1*2, MIMO2*2, MIMO4* 2, etc. Modulation: QPSK, 16QAM, 64QAM

Position

Good, medium and cell edge (for each terminal)

Distribution

Equal, or non-equal distribution of "1 to n" UEs

Network Environment

Channel Model

EPA, EVA, ETU with different Doppler frequencies

Backhaul Capacity

Capacities of X2, S1-U, S1-MME transport links.

QoS Configuration

Link weight and priority assigned to the flow(s)

Concurrent Load

Unloaded, or concurrent UL/DL data volumes

Application

Application service or traffic generation for LTE E2E, EPS, E-UTRAN, EPC bearers.

Traces

Trace points & data to be collected

Setup

Trial Lab, Reference Cluster, or Simulation

Route

In reference cluster, drive test route description.

Table 2

Description of Reference Conditions

The number of test executions (sample size) is not defined as a parameter in the template, but it is to be defined for each measurement scenario so that the calculated KPI figure has the required statistical confidence.


Id:0900d805807a33b4

15


2.3

Measurement Scenarios Measurement Scenarios describes the measurement methods and recommended scenarios according to KPI categories summarized in Table 3 KPI Categories. The first column indicates the respective QoS terminology defined by the ITU [ITU-T.E800], and referenced by 3GPP in References 12. [3GPP32.410] and 17. [3GPP32.450] for LTE related KPIs. The third and f ourth columns indicate the selected terminology and explanation for its usage in the current paper. ITU QoS Category

3GPPKPI Category

Terminology Used

Meaningfor LTE Field Network Performance

Accessibility

Accessibility

Accessibility

Service accessibility, e.g. Attach, or EPS Bearer Setup Request success rates.

Reliability

Retainability

Reliability

Once established, the reliability of the service, e.g. VoIP call, or EPS bearer drop rates.

Integrity

Throughput, Latency

Throughput, Latency

Promise to deliver the service with a certain quality, level of conformance. E.g. cell throughput, or user data rates, packet delay.

Availability

Availability

not used

Relationship between service up and down times, e.g. cell, or LTE network availability

not defined

Mobility

Mobility

Different mobility related metrics, e.g. hand-over success or drop rates, latencies.

Table 3

KPI Categories

Since the KPI category is part of all KPI definitions (see KPI Definition Template ), it is easy to lookup the corresponding measurement method to each KPI.

2.3.1

Throughput Measurements Throughput is the primary metric for the characterization of the LTE radio technology (Orthogonal Frequency Division Multiple Access ( OFDMA) in DL, Single-Carrier Frequency Division Multiple Access ( SC-FDMA) in UL). It shows the data ra te that can be provided to users among different radio conditions. The throughput of an LTE cell is expressed as a function of the number of concurrently active users and the amount of aggregate traffic that the cell can still support. The throughp ut the LTE system can offer depends on many factors: • •

16

Channel environment (e.g. stationary or mobile, speed) and fading conditions. Reception conditions impaired by traffic load levels, and by interference between the cells, in short by the user's SINR.

•

Network layout, type of antenna.

•

Position of users in the cell (implies e.g. path loss and fading).

Id:0900d805807a33b6



Thus, it is obvious that the location of the mobile terminal in the cell will strongly influences the data rate that can be experienced by the user. For this reason, the capacity of an LTE cell is characterized by two distinguished metrics, one viewing capacity from the operator's point of view (How many users c an be supported by the cell?), the other from the end-user perspective:

2.3.1.1

•

Cell Throughput, and

•

Peak User Data Rate

Cell Throughput The metric "Cell Throughput" shows the sustainable aggregate cell capacity available to a number of "n" users. In order to approximate a typical operating environment, the users are distributed uniformly in the cell and use a typical mix of applications concurrently. The Figure 2 Distribution of Users for Cell Throughput Measurements shows how users are arranged for cell throughput measurements. Each user position is characterized by different combinations of path loss, receive signal, and neighbor cell interfere nce strengths.

Figure 2

Distribution of Users for Cell Throughput Measurements

The measurement is done with stationary terminals to avoid the impact of movement and Hand-Over (HO). The latter implies that for those mo biles that are located at the cell edge, HO must be suppressed on the UE or by the network. The proposed reference conditions for cell throughput measurements are summarized in Table 4 Reference Conditions for Cell Throughput Measurements.


Id:0900d805807a33b6

17


Parameter

Reference Condition

Terminal

UE Type

Laptop with LTE card

UE Category

Different categories, KPI specific

Antenna

Types: SISO, MIMO 1*2, 2*2, …;

Position

Good, medium, cell edge (see Distinguished Cell Position)

Distribution

"1-n" terminals, equal distribution in the cell

Network Environment

Channel Model

Stationary outdoor - EPA0, EVA0, ETU0 with different SINRs

Backhaul Capacity


QoS Configuration

Non-real-time QoS, equal weights and priorities

Concurrent Load

Unloaded (no concurrent load)

Application

Data upload / download (FTP); separate UL/DL and concurrent transmission.

Traces

UE, Uu, eNB, S1-U, … FTP Server

Setup

Reference cluster in live network

Route

N.a.

Table 4

Reference Conditions for Cell Throughput Measurements

The default backhaul capacity of S1-U transport links are to be controlled and modified if necessary to prevent this link becoming a bottleneck during the measurement. The modified value should become default for subsequent measurements. Cell throughput measurements should be executed with an increasing number of UEs starting with 1. While the first test case correspond s to the peak user data rate measurement, the subsequent ones illustrate the dependency of this KPI on the number of active

18

Id:0900d805807a33b6



users. The results (cell throughput and peak user data rate) are to be illustrated together in a diagram like the example below:

2.3.1.2

Peak User Data Rate The "Peak User Data Rate " metric describes the data capacity that is available to one user in a cell. Data rate is the most important single factor which influences the end-user experience of non-real-time application services, like web browsing, email, File Transfer Protocol (FTP) up/downloads, and interactive gaming, due to its impact on (signaling and data) transfer delays. The goal of field performan ce measurements is to investigate the sensitivity of this metric to LTE impairment factors. The realistically achievable data rates under concurrent load (i.e. with concurrently active users in the cell) in stationary case are shown with cell throughput measurements as described in Cell Throughput. Since the available peak user data rate is influenced by any or all of the following conditions, •

Channel environment (mobility, stationary)

•

Radio conditions (signal power, path loss, SINR)

• Aggregate cell load (i.e. intra-cell and inter-cell traffic) •

Restriction of user data rates (e.g. by terminal category)

•

Link sharing weights (Quality of Service ( QoS) configuration)

•

Backhaul capacity

the measurement scenarios include variations of these parameters. User data rates are also measured in cell edge conditions, even including hand-over. The reference condi-


Id:0900d805807a33b6

19


tions for user data rate tests are summarized in Table 5 Reference Conditions for User Data Rate Measurements. Parameter

Reference Condition

Terminal

UE Type

Smart phone and laptop with LTE card

UE Category

Corresponding UE categories

Antenna

Types: SISO, MIMO 1*2, 2*2, etc.

Position

Good, medium, and cell edge (see Distinguished Cell Position)

Distribution

One single terminal

Network Environment

Channel Model

EPA, EVA stationary and mobile w/ 3, 50 km/h

Backhaul Capacity


QoS Configuration

Non-real-time QoS, equal weights and priorities

Concurrent Load

Unloaded

Application

FTP upload, download; separate

Traces

UE, eNB, FTP Server; Uu, S1-U, S1-MME, X1, ...

Setup

Reference cluster in live network

Route

Drive test route passing good and medium cell positions, and crossing cell borders.

Table 5

Reference Conditions for User Data Rate Measurements Ideally, the results of field or lab trial measurements of this KPI would be a full distribution of Peak User Data Rate values (a Cumulative Distribution Function ( CDF)) across the cell. For practical reasons, however, it is sufficient to produce the values in selected cell positions as described in Distinguished Cell Position. Radio positions (good, average and poor SINR) that are best suited to the channel model of stationary users is selected. Mobility tests with up to 50 km/h can be execute d in field reference clusters, but higher speeds, e.g. high speed train with ~350 km/h are to be executed in simulated a lab environment.

2.3.2 2.3.2.1

Latency Measurements Round Trip Time Measurement The benchmark measurement for finding the E2E latency of LTE access networks for data applications is the Round Trip Time ( RTT ) measurement. RTT is measured with the Ping application of the UE´s operating system or with a comparable measurement tool. It records the time difference between sending an Internet Control Message Protocol (ICMP ) Echo Request to an IP host, and the reception of the corresponding ICMP Echo Reply message. The measurement can be executed bet ween a mobile terminal and an IP host, which is configured to respond to ICMP Echo Requests. This way, round trip delays can be measured between UE - eNB (LTE Uu), UE - S/P-GW (E-UTRAN Radio Access

20

Id:0900d805807a33b6



Bearer(E-RAB), EPS) and UE - AS (LTE E2E), respectively. The measurement can (and should) be executed in both directions. The Ping application sends successive ICMP Echo Requests • •

either upon reception of the ICMP Echo Reply to the previous request, or after a predefined period of time (in the range of ~100ms to 1s)

This is done to ensure that only one Ping request/response is under way at any given time. This way, the delay time will have no bias caused by queuing, or schedul ing delays along the transport path. The total number of measurements per cycle should be chosen so that results have the required statistical confidence level. The recommended number of measurements per IP host is ≥ 100. If the measurement is executed over the radio link (LTE Uu), the first IP packet might suffer additional delay caused by the dynamic allocation of radio resou rces to non-realtime (non-GBR) bearers. For this r eason, the round trip delay of the first packet s hould not be considered in statistic calculations. On the other hand, the first ping measurement can be used to determine UE state t ransition delays if the UE starts from non-registered or idle states (from non-registered to registered, or from idle to connected states). The periodicity of Pings needs to be selected so that the once allocated radi o bearer is kept over the whole duration of the measurement. The test is to be done for stationary and mobile end users. The measurement with stationary UE should be performed under average radio link conditions with normal SINR. For moving UEs, two different positions are to be selected, one in average radio link conditions and normal SINR, and another in the cell overlapping area. It is important to make sure that handovers between cells are avoided. The reference conditions should be monitored during the test us ing UE performance measurements; also, relevant trace data can be collected from the network. In lab trials, the measurements need to be executed with and w/o concurrent load. From the measurement results (samples) mean figures, standard deviation and 95% delays need to be calculated.

2.3.2.2

Signaling and Media Delay Measurements If the bandwidths of UL and DL bearers in RTT measurements are identical, the benchmark value of UL and DL latencies can be given as 1/2 RTT. However, latency KPIs of signaling procedures on the network or application level (e.g. Radio Access Bearer (RAB ) service setup, or VoIP call setup flows) are often complex and include the exchange of more than one messages of different sizes. In addition, the link capacities used in UL and DL are often different, e.g. in case of the application services data download, gaming or video streaming. In these cases, the RTT or 1/2 RTT figures are not sufficient to characterize the latencies incurred by the mobile access network. Here, the time synchronization of network elements is the central problem, since the start and stop triggers of the measurement lie on different Network Elements ( NE) and interfaces. In case of measuring latencies over the radio link, the time needed to a ttach the mobile to the network and to setup the necessary radio resources is a significant component. In these cases, the measurement should be made with different initial states of the mobile, e.g. power off, idle, connected and active states. LTE State Models lists the dif-


Id:0900d805807a33b6

21


ferent states of Mobile Terminations ( MT) in 3GPP LTE specifications, see References 3. [3GPP23.401]. In field tests, latencies should be measured with the typical initial state of the mobile. The initial state will depend on the measurement scenario, e.g. Idle for network attach, or Connected for EPS bearer setup. Depending on the location of trigger points, end-user visible delays, LTE network delays, different bearer (RB, E-RAB and EPS) delays, transport and NE processing delays can be described similarly. The following aspects need to be specified for latency measurements:

2.3.2.3

•

Start and stop triggers of the media or signaling flow. The trigger point is defined by the message type, NE and interface; it identifies where time stamps are to be recorded.

•

In case of measuring one-way delays, the method of time synchronization.

•

Trace points and analysis tools. In addition to trace points in the network, traces may be needed on mobile terminals when measuring end-u ser visible latencies, or E2E LTE network delays.

Reference Conditions for Latency Measurements Field measurements of latencies are requeste d under reference radio conditions (equal to cell medium), which are specified by the acceptable ranges of UL/DL SINR and of receive power levels. In addition, the following variations of conditions are planned (see also Table 6 Reference Conditions for Latency Measurements):

22

•

Unloaded versus loaded cell conditions

•

Stationary UE versus mobile on the move

•

Drive route for measurements with velocity from medium to cell edge position including hand-over

Id:0900d805807a33b6



Parameter

Reference Condition

Terminal

UE Type

Mobile phone, smart phone, laptop with LTE card

UE Category

UE category 1-5 according to [3GPP36.306]; see References 21.

Antenna

One antenna: e.g. SISO

Position

Medium, overlapping area

Distribution

One terminal

Network Environment

Channel Model

EVA0, EVA70 (stationary, mobile w/ ~50 km/h)

Backhaul Capacity Capacities of X2, S1-U transport links. QoS Configuration Real-time and non-real-time QoS, the latter with equal weights and priorities Concurrent Load

unloaded and loaded with concurrent UL/DL data of medium volume (below cell target capacity)

Application

Data upload download and real-time over EPS bearer.

Traces

Trace points, send & receive packet counts

Setup

Field Reference Cluster, Trial Lab

Route

From radio position with reference condition (medium) to overlapping area including hand-over.

Table 6

Reference Conditions for Latency Measurements

Those samples that were taken in measurement s during which the specified conditions (reference, or explicitly modified ones) were not met, have to be excluded from the statistic calculations.

2.3.3

Service Accessibility Measurements Accessibility KPIs characterize the accessibility of a service, or service element to its users. They are expressed as a ratio of successful service requests to the total number of attempts. (This group of KPIs corresponds to the category of "service accessibility KPIs" in References 12. [3GPP32.410].) The KPI value is calculated as:

Success Rate = (# successful service requests / # total attempts) * 100 [%]

For this test to be meaningful, a significant number of samples need to be collected. E.g. to get the success rate with 0.2% granularity, at least 500 samples are needed. Service request attempts that are rejected due to insufficient access rights, bad user authentication, or errors in request parameters are not considered and should be taken out of the samples used for calculating the metric. The measurements should be executed with the following variations of reference conditions:


Id:0900d805807a33b6

23


• Application is selected according to the mobile network service procedure that is tested. Test scripts are used for the automated execution of service requests, for the evaluation of responses and for the calculation of the KPI figure. •

Mobile terminals in stationary and mobile use.

•

The mobile use includes drive routes among good and medium receive conditions, as well as a route that crosses cell overlapping areas. In the latte r case, hand-over is executed. The frequency of handovers follows the assumptions of the NSN reference traffic model for LTE networks.

•

Unloaded and loaded cell conditions. Concurrent load is to be generated in lab trials with low and medium levels (as a certain percentage of the cell target load). In field network tests, where a certain background load i s given, load levels are to be tra ced to allow for scaling (calibrating) of lab trial results.

The reference conditions are summarized in Table 7 Reference Conditions for Success Rate Measurements. Parameter

Reference Condition

Terminal

UE Type

Laptop with LTE card, or modem.

UE Category

UE category according to [3GPP36.306]; see References 21.

Antenna


Position

Good, medium and cell edge area.

Distribution

n.a. (one terminal only)

Network Environment

Channel Model

EVA0, EVA70: stationary, mobile w/ ~50 km/h

Backhaul Capacity Capacities of X2, S1-U, S1-MME transport links. QoS Configuration n.a. Concurrent Load

Unloaded or concurrent UL/DL data volumes of low and medium levels.

Application

Procedure dependent test script for automated execution of different applications.

Traces

Data collected by test script. Monitoring of online statistic counters.

Setup

Field Network Cluster and Trial Lab.

Route

Real drive test routes (in Cluster), or simulated with varying channel models (in Lab).

Table 7

Reference Conditions for Success Rate Measurements

Due to the high number of test executions, the measurements can be executed in lab environment with simulated air links. At least the stationary tests should be executed in field network cluster. Automated test execution is needed in both environments. In addition to the KPI figures calculated by the test script, test results should include online performance measurement statistics, which are gained by monitoring the performance counters of LTE network elements (i.e. on the eNB, MME, S-GW, P-GW). Please

24

Id:0900d805807a33b6



note, however, that the success ratio figures calculated from eNB counters will not be identical to the success ratio measured by the test UE.

2.3.4

Reliability Measurements Due to similarities in their measurements, the following types of KPIs are discussed together in this chapter. •

Service Drop Rates (Register, Attach, Service Request, etc.)

•

Transport Error Rates (Block Error Rate ( BLER ), Packet Loss Ratio ( PLR), Frame Error Rate (FER), etc.)

The service drop rate KPIs are categorized as Retainability by 3GPP, and Reliability by the ITU. The transport error rate KPIs are categorized as Integrity KPIs by 3GPP, and by the ITU.

2.3.4.1

Transport Error Rates These KPIs put the number of erroneous or lost data u nits to the overall number of data units sent into relation. Depending on the payload of the transmitted data unit, the ratio is given as BLER for radio blocks, FER for voice and video frames, and Packet Loss Rate for IP packets. The chapter deals with the method of measuring PLR in the field. Measurements of BLER and FER are planned for lab tests only; their methods will be provided elsewhere. E2E IP PLR is measured with the IP flow of an application service over a single EPS bearer between the UE and the Application Ser ver (AS) connected directly to the P-GW (first hop IP router). The RAB is selected with appropriate QoS handling (i.e. nGBR bearers for data applications and GBR bearers for real-time flows). The measurement equipment for PLR is placed on the UE side and on the AS side to measure PLR in both directions (UL and DL). It can either be a professional measurement tool, or a simple test script. PLR KPIs are measured across the following variations of conditions: • Application service: data upload / download, and real-time (e.g. video streaming) •

Mobile terminal in stationary and mobile use

•

The mobile use includes drive routes among good and medium reception conditions, as well as a route that crosses cell overlapping areas. In the latter case, a handover is executed.

•

Operational load in source and destination cells of the f ield network cluster. Concurrent load generated in lab t rials according to the reference traffic mix at low, medium and high levels.


Id:0900d805807a33b6

25


Parameter

Reference Condition

Terminal

UE Type


UE Category

UE category tbd. according to [3GPP36.306]; see 21.

Antenna


Position

Good and medium in originating cell, drive across hand-over area.

Distribution


Network Environment

Channel Model


Backhaul Capacity n.a. QoS Configuration n.a. Concurrent Load

Loaded & unloaded in source & destination cells

Application

Data download (UDP) and video streaming.

Traces

UE and eNB, mobility events

Setup

Reference Cluster, Trial Lab.

Route

Drive test route passing best and medium cell positions, and crossing cell borders.

Table 8

Reference Conditions for Packet Loss Rate Measurements

One KPI value (measured sample) is calculated for each scenario as:

Packet Loss Rate = (# not received packets / # sent packets) * 100 [%]

The PLR is given as mean value over a ll samples of the same measurement scenario. For this test to be meaningful, a significant number of samples needs to be collected. For this reason, the test should run for at least one hour in the lab. It is not necessary to make a detailed analysis of the reasons behind packet losses, as it is difficult to store extremely large amoun ts of trace data. However, if professional test equipment is available for this analysis, the measurement setup may include it. Online performance statistics on PLR can be collected during the measurement if the corresponding Performance Management ( PM ) counters are available on LTE network elements (i.e. on the P-GW). Please n ote, however, that the PLR figures collected may not be identical to the PLR measured on the test UE and on the AS.

26

Id:0900d805807a33b6



2.3.4.2

Service Drop Rates Attach, Service Request, Register, etc. are examples of service requests on different service levels; service requests establish a relationship between the UE and the network or application. The relationship i s normally terminated with a "Release Request" by the UE. In case of network errors (e.g. handover failures), or insufficient radio resources the relationship can be terminated unexpectedly. The ratio of such "abnormal" terminations to the number of all established relationships is given by "drop rate" KPIs. For certain services, like bearer creation or VoIP call, the service drop rate is also related to the duration of the session. This is motivated by the fact that the possibility of network errors, e.g. hand-over failures is higher if the service duration is longer. In such cases, the service usage time should also be measured, and the drop ratio expressed in [1/s] units instead of [%]. The current version of this document does not consider session time for service drop rates. It will be added in the next update of this document. In order to measure the ratio, the given service request has to be executed in a sufficiently large number of times to assume statistical significance. E.g. each test scenario should be executed ≥ 100 times. After the relationship has been established, common operations of the service are to be executed for a typical period of service usage time. The LTE traffic model can deliver input for this. Due to the complexity of the test setup and the long time needed to execute the tests, the measurements are better execute d in a lab environment with simulated air links and automated service execution. The measurements are to be executed under the following varying conditions: •

Mobile terminal in stationary and mobile use

•

The mobile use includes drive routes among good and medium reception conditions, as well as routes that cross cell o verlapping areas. In the latter case, a handover is executed. If executed in the trial lab, these conditions are set in simulated radio links, i.e. the "drive route" is simulated.

•

Unloaded and loaded cell conditions. Concurrent load is to be generated up to full (or target) load level of the cell.


Id:0900d805807a33b6

27


Parameter

Reference Condition

Terminal

UE Type


UE Category

UE category tbd. according to [3GPP36.306]; see References 21.

Antenna


Position

Good, medium and cell edge area.

Distribution


Network Environment

Channel Model


Backhaul Capacity Capacities of X2, S1-U, S1-MME transport links. QoS Configuration Link weight and priority assigned to the flow(s) Concurrent Load

Unloaded or concurrent UL/DL data volumes up to full cell load.

Application

n.a.

Traces

Trace points & data to be collected

Setup

Trial Lab with simulated air link conditions.

Route

The drive test routes are simulated using different channel models and varying radio conditions (receive level, SINR).

Table 9

2.3.5

Reference Conditions for Service Drop Rate Measurements

Mobility (Handover) Measurements Mobility KPIs from a group because all of them are related to hand-over procedures. They could, however, be discussed in the Latency and Accessibility groups, as well, because HO-related KPIs are either service interrupt time, or success ratio type KPIs. A successful hand-over is understood to be any successfully performed make-beforebreak procedure. For its measurement, a DL User Datagram Protocol ( UDP ) data stream is generated as the only load during the whole measurement. The test is performed in an overlapping area between two cells. The overlapping region of a cell is defined by the level of radio conditions (DL SINR and receive power levels) which are identical for neighboring cells. It can be expressed as the difference between receive sensitivity levels of neighboring cells. I.e. at the border line of t he overlapping region the relative receive sensitivity level is 0 dB. Drive routes need to be specified so that they begin in good radio conditions of the originating cell and proceed directly through the hand-off area and end in high signal strength and good SINR in the other cell. Drive routes should proceed directly through the hand-off area in order to prevent multiple handoffs caused by stop lights, stop signs, traffic congestion as well as to prevent extending of the time period in the hand-over region. Routes must be carefully selected to avoid areas where handoff is driven through lack of coverage, which leads to breaks in connection. This is the most common cause of so-called "false failures".

28

Id:0900d805807a33b6



In order to measure handover KP Is, the logging of mobility events is necessary in mobile terminals. A successful handover is marked by an entry in the log, which carries the identifier of the new cell (destination). The HO KPIs are measured under the following variations of conditions ( Table 10 Reference Conditions for Handover Measurements): •

Unloaded versus loaded cell conditions for Intra RAT, ideal conditions in target cell for Inter Radio Access Technology (RAT) (GSM, UMTS) hand-over scenarios.

•

Mobile terminals with vehicular speed.

•

Drive routes from good reception condition (in the originating cell) over hand-over area to a good position on the destination cell. Parameter

Reference Condition

Terminal

UE Type

Smart phone, laptop with LTE card

UE Category

UE category tbd. according to [3GPP36.306]; see References 21.

Antenna


Position

Good/medium in originating cell directly across hand-over area to good/medium position in new cell.

Distribution

n.a.

Network Environment

Channel Model


Backhaul Capacity n.a. QoS Configuration n.a. Concurrent Load

Unloaded and loaded destination cell in UL/DL.

Application

UDP data stream in DL with hand-offs acc. to reference traffic model.

Traces

MT, Source and Target eNBs, mobility events

Setup

Reference Cluster, Trial Lab

Route

In reference cluster, drive test route description.

Table 10

Reference Conditions for Handover Measurements

The drive test has to be repeated at least 20 times to get t he required level of statistical confidence. All reasons for handover failures (no resources in target cell, service not supported, etc.) should be excluded.


Id:0900d805807a33b6

29


2.4 2.4.1

Measurement Setup Field Network Cluster For field performance verification tests of KPIs to be executed in live network environments, the following LTE Reference Cluster is recommended (see also Figure 3 Reference Cluster for Field Performance Verification Tests): •

Sufficient number and positioning of UEs as specified by the measurement scenario. All UEs are inside of the cells of the reference cluster.

•

eUTRAN including at least three eNBs, one MME, and an integrated S-GW/P-GW.

•

Each eNB supporting three LTE cells, each with predefined minimum radio conditions at cell borders.

•

Provisioning of network equipment and setting of configuration parameters according to the guidelines of the LTE network operator and agreed with the vendor.

The recommendation considers constraints of the NSN LTE product release. The Reference Cluster environment must not change during the test campaign (no changes to network configuration and parameter values). Traces are to be activated on the UE, and on t he network elements MME and S/P-GW as required by the measurement scenario and reference conditions of the given KPI. Trace data that can be collected on the interfaces LTE-Uu, S1, X1 and S11 are described in References 14. [3GPP32.423]. Traces should be taken such that the investigated KPIs (e.g. latencies) are not impacted. If this cannot be avoided, the measurements should be repeated without traces.

Figure 3

Reference Cluster for Field Performance Verification Tests

In addition to Figure 3 Reference Cluster for Field Performance Verification Tests, the final diagram of a test specification should contain indications of: • At least three eNBs with three cells each

30

•

Trace points on all NE interfaces

•

Monitoring station or test equipment with adequate protocol stack (e.g. K12xx) to decode IP and radio network layer protocol messages

Id:0900d805807a33b8



2.4.2

•

Test workstations ( WS) to generate background load

•

WS for post-processing of traces

•

Operations, Administrations and Maintenance Performance Management ( OAM PM ) for monitoring counters, displaying of statistics

Trial Lab A description of the Trial Lab should be given with the performance verification test specifications.

2.4.3

Measurement Data Collection The following ways of performance data collection are possible in KPI measurements: •

Collection of traces from terminals, network elements and network interfaces

•

Collection and evaluation of counters from network elements by OAM PM

The preferred way of collecting performance data in a live network is via PM counters. It does not add additional load to the network and is a permanent source of information that can be monitored all the times. However, PM counters do not exist for all categories and for all individual KPIs. Typically, n o counters are defined, or can be implemented f or latency KPIs. For this reason, terminal, server and network traces are also collected. Since tracing puts additional processing load on these elements, traces are activated only on operator demand and for the time of test execution. On the other hand, traces include very detailed information on call level, which allows performance analyses of individual mobiles and operations. With traces, it is possible to go further in monitoring and optimization operations. For more details on LTE MMS and E-UTRAN traces, see References 13. [3GPP32.421] and 14. [3GPP32.423].

2.4.3.1

Subscriber and Equipment Traces Subscriber and UE Traces should be activat ed on the UE, and on the network elements MME and S/P-GW of the Reference Cluster, as required by the scenario description (see Reference Conditions) of the given KPI measurement. Collected measurement data is transferred to an external server for post-processing and for evaluation. This data is necessary to check if the test run was executed w/o failure (e.g. dropped call analysis) and if the measured figure can be considered for statistic calculations and statements. Moreover, it helps to verify if the reference conditions were met (e.g. RF coverage and capacity) during the whole measurement. Traces deliver additional information for tuning and optimizing the Reference Cluster itself. The capability to log data on any interface at call level for a specific user (e.g. International Mobile Subscriber Identity ( IMSI)) or mobile type (e.g. International Mobile Equipment Identity ( IMEI)), or service initiated by a user makes it possible to get information which cannot be deduced from Performance Measurements such as the end-user perception of Quality of Service ( QoS ) during a call (e.g. requested QoS vs. provided QoS), and correlation between protocol messages and RF measurements. Moreover, as opposed to OAM Performance Measurements, which provide values aggregated on an observation period (i.e. interval statistics), Subscriber a nd UE traces give instantaneous values for a specific event (e.g. call, location update, etc.).


Id:0900d805807a33b8

31


2.4.3.2

PM Counters Performance measurements are mand atory in all field test scenarios. This provides performance related information similar to operational environments. The PM counters of special interest are given with the KPI description (see Overview). This level of detail is provided in accordance with the availability of PM counters in the latest LTE product release and in 3GPP specifications.

32

Id:0900d805807a33b8



2.5

Measurement Reporting Some measurement scenario descriptions already contain hints about the form of the expected measurement reports (e.g. in Peak User Data Rate). A complete description will be provided with the field verification test plan.


Id:0900d805807a33ba

33

Definitions of Key Performance Indicators (KPIs)


3 Definitions of Key Performance Indicators (KPIs) 3.1

Overview Key Performance Indicators (KPIs) are defined for application and for LTE network services (see Figure 1 LTE Bearer Service Architecture): a) Application Services b) LTE E2E Network Service c) Radio Bearer Services: E-RAB, RB d) Transport Bearer Services: X2, S1, S5/S8 and External e) LTE Network Elements

The Application KPIs are service specific. The following services are considered: •

Web Browsing

•

Data Upload / Download

• Audio Video Streaming •

Conversational Voice (VoIP)

The current paper focuses on the application service “Data Download/ Upload (FTP) Service KPIs", and on “LTE E2E Network Service" KPIs in line with the feature scope of the LTE release. The performance indicators of the categories D and E are useful for the decomposition of E2E performance figures. Their output can help to locate potential performance bottlenecks in the field, and to optimize the overall performance of the LTE network. E.g. the KPI values of category D can be the basis for Service Level Objectives (SLOs) for IP backhaul networks if these are operated b y third party ISPs. Because SLOs are v ery important, maximum acceptable values are specified for them as assumptions for the achievement of performance targets for application and LTE network service KPIs. KPIs are distinguished according to the measurement setup, which should be used for their verification. They are denoted "Lab" KPIs, if field network trials are not recommended with them due to high costs or technical limitations. These KPIs are measured in vendor or operator laboratories only. The other category of KPIs is called "Field" KPIs, because they should be measured in Reference Network Clusters of operational networks. However, also "Field" KPIs have to be verified in the laboratory before measurements are executed in the field. Lab results serve as benchmark figures and will be used to calibrate (scale) field network measurement results. After the overview of all KPI categories, detailed descriptions are given in Application Services and LTE Network Services.

34

Id:0900d805807a32b6



3.1.1


Application Services The full set of application service KPIs is summarized in Table 11 Overview of Application Service KPIs. The following IP based Packet Switched ( PS) data services are characterized with a common set of KPIs: •

Data Download / Upload (FTP)

•

Web browsing (HTTP)

•

Email (POP/SMTP)

The Data Download / Upload service is selected for detailed investigations and verification in the LTE network, because of the Transmission Control Protocol ( TCP) windowing mechanism. #

KPI Name

KPI Category

Trial Network

Chapter

PS Data Services (FTP, HTTP, etc.)

1

Service Accessibility Ratio [%]

Accessibility

Field

(FTP) Service Accessibility Ratio

2

Completed Session Ratio [%]

Reliability

Field

(FTP) Completed Session Ratio

3

Service Access Time [s]

Latency

Field

(FTP) Service Access Time

4

Session Time [s]

Latency

Field

(FTP) Session Time

5

User Data Rate [Mbps]

Throughput

Field

(FTP) User Data Rate

Table 11

Overview of Application Service KPIs


Id:0900d805807a32b6

35


3.1.2 3.1.2.1


LTE Network Services LTE E2E Network Service The full set of LTE E2E network service KPIs is summarized in Table 12 Overview of LTE E2E Network Service KPIs. Some of the KPIs are defined as "Lab", which means that they are verified in in-house (or vendor) labs, but not in the field.

#

KPI Name

KPI Category

Trial Network

Chapter

LTE E2E Network Service, Control Plane

6

Attach Time [ms]

Latency

Field

Attach Time

7

Detach Time [ms]

Latency

Lab

Detach Time

8

Attach Success Rate [%]

Accessibility

Field

Attach Success Rate

9

Service Request (EPS) Time [ms]

Latency

Field

Service Request Time (EPS), UE Initiated, Service Request (EPS) Time, Network Initiated

10

Service Request (EPS) Success Rate [%]

Accessibility

Field

Service Request (EPS) Success Rate

11

Service (EPS) Drop Rate [%]

Reliability

Field

Service (EPS Bearer) Drop Rate

12

Handover Procedure Time [ms]

Mobility

Lab

Handover Procedure Time

13

Handover Success Rate [%]

Mobility

Field

Handover Success Rate

14

Paging Time [ms]

Latency

Lab

Paging Time

15

Paging Failure Rate [%]

Accessibility

Lab

Paging Failure Rate

LTE E2E Network Service, User Plane

16

(LTE) Round Trip Time (RTT) [ms]

Latency

Field

(LTE) Round Trip Time

17

(LTE) User Data Rate [Mbps]

Throughput

Field

(LTE) User Data Rate

18

(LTE) Packet Loss Rate (PLR) [%]

Reliability

Field

(LTE) Packet Loss Rate

19

(LTE) Service Interrupt Time (HO) [ms]

Mobility

Lab

(LTE) Service Interrupt Time (HO)

Table 12

36

Overview of LTE E2E Network Service KPIs

Id:0900d805807a32b6



3.1.2.2 #


Radio Bearer Services

KPI Name

KPI Category

Trial Network

Chapter

Radio Bearer Service, User Plane

20

(RB) Packet Loss Rate UL / DL [%]

Reliability

Lab

(RB) Packet Loss Rate

21

(RB) User Data Rate [Mbps]

Throughput

Field

(RB) User Data Rate

22

Cell Throughput [Mbps]

Throughput

Field

Cell Throughput

Table 13

Overview of LTE Radio Bearer Service KPIs


Id:0900d805807a32b6

37


3.2

KPI Definition Template For each KPI, relevant information i s provided according to the structure shown in Table 14 KPI Definition Template. The fields in the KPI definition are marked as mandatory, or optional according to References 12. [3GPP32.410].

KPI Name

M Mandatory long name of the KPI.

KPI Type, Category, Object, Unit

M Type: Ratio, Mean, CDF, etc. [3GPP32.814]; see References 19. Category: Accessibility, Reliability, Latency, Mobility, etc. Object: Application Service, E2E, E-RAB, RB, X1, etc. Unit: %, ms, Mbps, Erlang, etc.

Definition

M Description of the KPI. End-user or network view. Basic information about begin and end trigger points.

Methodology of measuring

O

Test environment: field or lab test, or simulation.Measurement method: according to KPI Category, plus specifics for the KPI (if any).

Assumptions and pre-conditions

O

Important information for measurement execution, i.e. scenario description including terminal category, channel model, load conditions, etc.

Formula (Logical)

M Provided for KPIs, e.g. Success Rate & Drop Rate which are calculated from some other metrics (measured)

Message flow, trigger points

O

Provided for latency KPIs, for monitoring of procedure correctness, and for success rate KPIs to indicate trigger points where PM counters are incremented.

Related KPIs

O

Reference to related KPIs

Related PM counters

O

Reference to related domain specific PM counters with indication of how to aggregate them to the KPI.

Table 14

KPI Definition Template The "KPI Object" field identifies the service, or network element to be characterized: •

Services: AS or one of the 3GPP bearer services: E2E network (LTE), EPS, E- RAB, RB, X2, S1, S5/S8 and External. The bearer services are uniquely defined by their end-points, i.e. by the network interfaces of the LTE architecture.

•

Network Elements: eNB, MME, S/P-GW.

The following "KPI Categories" are used: Accessibility, Reliability (Retainability), Latency, Throughput, Mobility and Quality. The PM measurements and KPIs of special interest are given with indication of their relationship to the field network KPI, e.g. if and how to calculate the field network KPI value out of them.

38

Id:0900d805807a3c4c



3.3 3.3.1

Application Services Data Download / Upload (FTP) Service KPIs Download of reference files with different sizes from the Internet, e.g. SW images, pictures, or video, as well as upload of the same files by FTP. For FTP protocol details, see References 33. [RFC979].

3.3.1.1



KPI Category, Object, Type, Unit

M Category: Accessibility, Object: AS (UE, FTP server), Type: Ratio, Unit: [%]

Definition

M The service accessibility ratio denotes the probability that the user can establish the necessary bearer (EPS) and access the FTP service successfully (see References 39. [ETSI102.250-2]). Service access covers starting the FTP client on the UE, setting up mobile access and creating a TCP connection to the FTP server.

Measurement method

O

Accessibility measurement, see Service Accessibility Measurements. Field and lab trial.

Assumptions, pre-conditions

O

Conditions for accessibility measurements see Service Accessibility Measurements

Formula (logical)

M FtpCmdSR

=

number_of(successful_ftp_commands) --------------------------------------------------------------------------------------------------------- ⋅ 100% number_of(total_ftp_commands)

Related KPIs

O


Related PM counters

O

No PM counters exist for this KPI


O

Trigger points:


•

Start: ftp get / put command issued on the UE

•

Stop: first data byte sent / received by the UE, or ftp indicates network error (e.g. timeout expired)

Id:0900d805807a3c21

39


3.3.1.2




M Category: Availability, Object: AS (UE, FTP server), Type: Ratio, Unit: [%]

Definition

M The completed session ratio is the proportion of completed FTP sessions and sessions that were starte d successfully (see References 39. [ETSI102.250-2]). The session is not completed if a predefined timer expires, or if a network failure has occurred that could not be repaired by automatic restarts.

Measurement method

O

Availability measurement, see Service Accessibility Measurements. Field and lab trial.


O

Conditions for availability measurements see Service Accessibility Measurements. The size of the file used for data upload / download, or the session duration time is to be specified for a concrete requirement o n the target value of this KPI.

Formula (logical)

M FtpSessionSR

=

number_of(completed_sessions) ------------------------------------------------------------------------------------------------------------------- ⋅ 100% nu mb er_of(successfully_started_sessions)

Related KPIs

O


Related PM counters

O

No PM counters exist for this KPI.


O

Trigger points:

40

•

Start: ftp get / put command issued on the UE.

•

Stop: after error free execution of the command, ftp returns with a prompt and indicates the number of bytes transmitted.

Id:0900d805807a3c21



3.3.1.3

(FTP) Session Time



M Category: Latency, Object: AS (UE, FTP server), Type: Mean and max (95%) values, Unit: [s]

Definition

M It is the overall duration of the download or upload of reference files from / to the FTP server (ref. to [ETSI102.250-2]). The elapsed time is measured between start and end trigger points.

Measurement method

O

Latency measurement, see Latency Measurements


O

Conditions for latency measurements see Reference Conditions for Latency Measurements

Formula (logical)

M Ft pS essi onTim e [s]

=

t sessionend

–

t sessionstart

Related KPIs

O

(FTP) Service Access Time, Service Request (EPS) Time

Related PM counters

O



O

Trigger points UI: •

Start: ftp client started on the UI.

•

Stop: after error free execution of the "ftp get / put" command, receiving prompt with the number of bytes transmitted.

Trigger points MT / R if (Download): •

Start: First [SYN] sent to setup a TCP connection.

•

Stop: Reception of the last data packet containing content.

Trigger points MT / R if (Upload):


•

Start: First [SYN] sent to setup a TCP connection.

•

Stop: Receive [FIN, ACK] to last data packet sent with file content.

Id:0900d805807a3c21

41


3.3.1.4

(FTP) Service Access Time Figure 4 KPIs for Packet Switched Data Services shows the phases of a data download / upload (FTP) service session using GPRS access network as example (see References 39. [ETSI102.250-2]). In case of an LTE network, GPRS would be replaced by LTE and an EPS bearer would correspond to the "PDP-Context". The full data download / upload session is subdivided into the phases Service Access and Data Transfer. In this document, the Service Access Time is defined as independent KPI. It corresponds to the phase "IP Service Access" on the Figure 4 KPIs for Packet Switched Data Services. (The EPS bearer setup time is defined as LTE E2E network KPI in Service Request Time (EPS), UE Initiated.) The trigger points for the Service Access Time KPI are defined on two levels: •

end-user interface ( UI ), i.e. on command line level, and

•

TE - MT interface, i.e. on UE R interface level.

On the command line level, the end of the mobile network access phase i s recognized by the ftp> prompt. By this time, the FTP client is initialized, the terminal is attached to the LTE network and the EPS bearer has been created. The Session is started by issuing the "ftp get / put" command and terminates with the display of the results of the data transfer (e.g. the # of bytes transmitted.). After the "get / put" command has been be issued, a TCP connection to the FTP server is created and the data transfer phase begins. Trigger events of the Service Access Time KPI cannot be seen on the terminal UI. In order to find them, it is necessary to analyze the message flows on the R interface level. The Session (and the IP Service Access phase, see Figure 4 KPIs for Packet Switched Data Services) begins with setting up the TCP connection to the FTP server and terminates with the transmission (send, or receive) of the last IP packet with file content payload. The phases Service Access and IP Service A ccess end, and the Data Transfer phase begins with the transmission (send, or receive) of the first IP packet with file content payload.

42

Id:0900d805807a3c21



Figure 4


KPIs for Packet Switched Data Services

Id:0900d805807a3c21

43


(FTP) Service Access Time


M Category: Latency, Object: AS (UE, FTP server), Type: Mean and max (95%) values, Unit: [s]

Definition

M It is the time period needed to access the FTP service successfully, from starting the ftp client to the point of time when the first d ata packet is sent or received (see References 39. [ETSI102.250-2]). It is assumed that the UE is already attached to the mobile network and an EPS bearer exists.

Measurement method

O

Latency measurement, see Latency Measurements.


O

Conditions for latency measurements see Reference Conditions for Latency Measurements.

Formula (logical)

M F tp Se rv ic eA cc es sT im e [s]

=

t co nte nt

sent or received

Related KPIs

O

(FTP) Session Time, Service Request (EPS) Time

Related PM counters

O



O

Trigger points DL/UL:

44

–

t ftp command started

•

Start: "ftp get / put" command issued on the UI

•

Stop: first data packet containing file content received / sent on the R interface.

Id:0900d805807a3c21



3.3.1.5

(FTP) User Data Rate

(FTP) User Data Rate UL / DL


M Category: Throughput, Object: AS (UE, FTP server), Type: Min (CDF 5%), Mean (CDF 50%), Max (CDF 95%) values, Unit: [Mbps]

Definition

M After the connection to the FTP server has been successfully established, the parameter describes the average data transfer rate measured over the data transfer phase (see Figure 4 KPIs for Packet Switched Data Services). This means that successful Service Access (availability of an EPS bearer and connection to the FTP service) is a prerequisite to the data t ransfer. The data transfer phase should also conclude successfully. The data rate is measured on application level in UL/DL direction.

Measurement method

O

Throughput measurement, see Peak User Data Rate. Single user measurement both in field and lab trials. In order to eliminate the impact of service startup times, the measurement begins with the transmission of first IP packets with file content payload. Since the data rates can vary strongly from session to session, a sufficiently large number of measurement executions is needed (>10) to get statistical confidence. The size of the large reference file should be selected such that the data transfer phase does not take less then thirty seconds. Option ally, a small reference file can be measured in addition, but the number of executions should be the same. The measured User Data Rates should be displayed in diagrams as a function of time to show the impact of FTP slow start.


O

Conditions for capacity measurements see Peak User Data Rate. The FTP server will be connected directly to the SGi interface. Additional delays or bottlenecks between the S/P-GW and the server should be excluded. The initial status of UE before starting the ftp application: EMM-REGISTERED, ECM-CONNECTED. The FTP application will be executed in binary mode, and the file to be transferred will also be binary.

Formula (logical)

M FtpMeanDataRateUL/DL

=

transf erred_data_volume_UL/DL[bytes] ⋅ 8 -------------------------------------------------------------------------------------------------------------------transfer_time [s]

Related KPIs

O

(FTP) Service Access Time, (FTP) Session Time

Related PM counters

O



O

The average throughput is measured from the opening of the data connection to the end of the successful transfer of the content (file, e-mail or web page). The trigger points for the Data Transfer phase can be seen in Figure 4 KPIs for Packet Switched Data Services. For more details see References 39. [ETSI102.250-2], chapters 4.6.1.7 and 4.6.2.7.

It is to be noted that the measurement of the User Data Rate (FTP) KPI (named after [ETSI102.250-2]; see References 39.) is executed as a single user measurement without concurrent load in the cell, since the goal is to find the maximum available FTP data rate for one user.


Id:0900d805807a3c21

45


3.4

LTE Network Services This chapter includes detailed KPI definitions for LTE Network Services.

3.4.1 3.4.1.1

LTE E2E Network KPIs Attach Time

Attach Time, Control Plane


M Category: Latency, Object: End-user, LTE (UE R interface), Type: Mean and max (95%) value, Unit: [ms]

Definition

M With Attach, the mobile terminal registers at the LTE network. At the end of the procedure the UE is authenticated, and a default (nGBR) bearer is established. The Attach Time is the interval between the connection request and the acknowledgement of the positive response by the UE.

Measurement method

O

Latency measurement, see Latency Measurements. Field and lab trial. Stationary users only.


O

Conditions for latency measurements, see Reference Conditions for Latency Measurements. UE status before measurement: EMMDEREGISTEREDMAP inter-working according to [3GPP29.002], see References 10., is assumed between MME and HSS (no Diameter).

Formula (logical)

M tbd.: mean value and 95% from all measured samples At ta ch Time [ms]


Related KPIs

O

O

=

t At tac h Complete

–

t At ta ch

Request

Trigger points on UE R interface (see Figure 5 Attach Procedure w/ Initial EPS Bearer Establishment (3GPP 23.401), References 3. [3GPP23.401]): •

Begin: 1. RRC CONNECTION REQUEST sent by UE (see also References 26. [3GPP36.331]) carrying NAS: Attach Request in its body.

•

End: 21. RRC DIRECT TRANSFER message with NAS: Attach Complete in its body sent by UE (see References 4. [3GPP24.008]).

a) Attach Success Rate. b) No 3GPP defined KPI for Attach Time found (exists). c) Attach Setup Time KPI in [ETSI102 .250-2], see References39. chapter 4.6.3.2.

Related PM counters

46

O

Attach Time related measurements are not defined in 3GPP Release 8 documents. However, the same trigger events for counting Attach requests and successful attachment completions can be used as Begin and End triggers for the measurement of elapsed time on the eNB.

Id:0900d805807a3c48



For initial Attach, the following specifics of the message flow in Figure 5 Attach Procedure w/ Initial EPS Bearer Establishment (3GPP 23.401), have to be considered: • A signaling bearer (SRB1) is to be established, before an A ttach Request can be sent, i.e. the UE is expected execute a contention based Rand om Access (RA) procedure. •

Step 3. is not executed, si nce the IMSI is queried from the UE directly using Step 4. The optional Identity Request / Response messages in step 5.b do not have to be exchanged if the MEI is already returned by the UE using the Security Mode Complete message as part of step 5.a.

•

Steps 7-11 are not executed, no EPS bearers are to be deleted.


Id:0900d805807a3c48

47


Figure 5

48

Attach Procedure w/ Initial EPS Bearer Establishment (3GPP 23. 401)

Id:0900d805807a3c48



3.4.1.2

Detach Time

Detach Time, Control Plane


M Category: Latency, Object: LTE (UE R interface) Type: Mean and max (95%-ile) values, Unit: [ms]

Definition

M With an explicit Detach request the UE informs the LTE network that it does not want to access the EPS any longer. At the end of the procedure all EPS bearers of the UE are released. The Detach Time is the interval between the Detach Request and the reception of a Detach Accept message by the UE. No Detach Accept is sent by the network if the cause for Detach is switching the UE off.

Measurement method

O Latency measurement, see Latency Measurements Lab trial. Stationary users only.

Assumptions, O Conditions for latency measurements see Reference Conditions for Latency Measurements . pre-conditions UE status before measurement: EMM-REGISTERED . Formula (logical)

M


O Trigger points on UE R interface (see Figure 6 UE-Initiated Detach Procedure, [3GPP23.401]):

Related KPIs

De ta ch Time [ms]

=

t DetachRequest

–

t DetachAccept

•

Begin: 1. RRC UL Information Transfer sent by UE carrying NAS: Detach Request in its body.

•

End: 11. RRC DL Information Transfer message with NAS: Detach Accept in its body received by the UE.

O a) Attach Time b) No 3GPP defined KPI for Detach Time found (exists).

Related PM counters

O Detach Time related measurements are not defined in 3GPP Release 8 documents.


Id:0900d805807a3c48

49


Figure 6

50

UE-Initiated Detach Procedure

Id:0900d805807a3c48



3.4.1.3

Attach Success Rate

Attach Success Rate, Control Planee


M Category: Accessibility, Object: LTE (UE R, eNB Uu interface), Type: Ratio, Unit: [%]

Definition

M The Attach Success Rate is defined as the ratio between the number of successful registrations and the number of all requests. This is the probability that a user can attach to the LTE network at any moment of time. The calculated success ratio figure excludes at tach requests, which are rejected by authentication failures. On the other hand, network attach requests which are terminated by timer expiry (due to the unavailability of some LTE resource) are considered as unsuccessful registrations. If the success rate is calculated on the eNB by counting incoming RRC requests, RRC CONNECTION REQUEST retries are to be excluded, since they would increase the overall number of establishment attempts, and thus reduce the success ratio.

Measurement method

O Accessibility measurement, see . Field and lab trial. Stationary users only. For defining the success ratio, the same series o f "Attach Time" measurements can be used. The KPI cannot be measured as end-user perceived ratio on certain mobiles, which do not indicate network attachment status to the end-user.

Assumptions, O Conditions for accessibility measurements see Service Accessibility Measurements. pre-conditions See also conditions in Attach Time. Formula (logical)

M


O Trigger events on UE R / eNB Uu interface (see Figure 5 Attach Procedure w/ Initial EPS Bearer Establishment (3GPP 23.401)):

LTENwAttSR

•

=

number_of_successful_attachments -------------------------------------------------------------------------------------------------- ⋅ 100% number_of_all_attempts

Success: 21. RRC DIRECT TRANSFER message with L3 NAS: Attach Complete in its body (see References 4. [3GPP24.008]) sent by the UE / received by eNB

• Attempts: 1. RRC CONNECTION REQUEST messages (with cause Attach Request) sent by UE / received by eNB.

Related KPIs

O a) Attach Time, Service Request (ESP) Success Rate. b) E-RAB Accessibility" KPI in [3GPP32.450], see References17. chapter 6.1.1. Since an SRB and S1 connection, as well as initial EPS bearer(s) have to be created during the Attach procedure (see Figure 5 Attach Procedure w/ Initial EPS Bearer Establishment (3GPP 23.401)), its success rate is dependent of the success rates of t hose component bearers. The success rate of initial EPS bearers is given by the following expression (for “additional” EPS bearers, refer to the Service Request (EPS) Success Rate):

c) See "Attach Failure Ratio" KPI in References39. [ETSI102.250-2], Chapter 4.6.3.1 defined as the inverse probability of successful network attachment.


Id:0900d805807a3c48

51


Related PM counters

O a) Related 3GPP measurements are defined in [3GPP32.425],see References15., for eNB: Signaling Radio Bearer ( SRB) creation: • Attempts: RRC.ConnEstabAtt.Cause = Attach •

Success: RRC.ConnEstabSucc.Cause = Attach

• Failures: RRC.ConnEstabFail.Cause = Attach UE associated logical S1 connection (S1-AP): • Attempts: S1SIG.ConnEstabAtt • Success: S1SIG.ConnEstabSucc Initial EPS setups during Attach: • Attempts: SAEB.EstabInitAttNbr.QCI •

Success: SAEB.EstabInitSuccNbr.QCI where QCI identifies the SAE Bearer level quality of service class.

•

Failures: SAEB.EstabInitFailNbr.Cause where Cause identifies the cause resulting in the setup failure.

b) The corresponding eNB counters are: • Attempts: SIG_CON_EST_ATT due to Attach •

Successful attachments: SIG_CON_EST_SUCC

• Failures: SIG_CON_EST_FAIL For the calculation of success, or failure ratios, it is necessary to have identical Causes for _ATT, _SUCC and _FAIL types of counter groups. Since the EPS_SETUP_ATT/_SUCC counters do not differentiate between initial and subsequent EPS bearer setups, they cannot be set in dire ct relation to the Attach SR KPI.

52

Id:0900d805807a3c48



3.4.1.4

Service Request Time (EPS), UE Initiated

Service Request Time (EPS Bearer Setup), UE Initiated, Control Plane


M Category: Latency, Object: EPS (UE R, eNB Uu interface), Type: Mean and max (95%) values, Unit: [ms]

Definition

M It is the time taken by the LTE network to setup an EPS bearer on request by the UE. The EPS bearer can be new (dedicated), or an existing one (e.g. the default EPS bearer). The latter is needed to reassign Uu radio and S1 bearer resources to the existing E PS bearer of a previously Idle UE. The EPS bearer has to be created or activated before IP packets can be exchanged, i.e. the Service Request creates the IP link dynamically over the mobile access (LTE) network. The UE Initiated EPS Bearer Setup Time is the interval between the submission of the message RRC CONNECTION REQUEST (see References 26. [3GPP36.331]) carrying a NAS: Service Request message in its body, and the reception of the RRC CONNECTION RECONFIGURATION response by the UE. The Bearer Setup procedure is based on the Service Request procedure, see References 3. [3GPP23.401] and 20. [3GPP36.300].

Measurement method

O

Latency measurement, see Latency Measurements. Field and lab trial. Stationary users only. Dedicated, or default EPS bearer setup. The response time can be measured on the UE R interface, or on the eNB Uu interface. The latter metric shows the d elay component of the E-UTRA network only. End-user delays cannot be measured, since terminals do not indicate the corresponding state change toward the end-user. The setup times of the RB and S1 bearers are part of the EPS setup time and can be determined using the same sequence of measurements.


O

Conditions for latency measurements see Reference Conditions for Latency Measurements. UE status before measurement: EMM-REGISTERED , but otherwise different initial states (idle, dormant or active).

Formula (logical)

M Mean value and 95% from all measured samples.Service Request Service Request Time [ms]



O

=

t RRC_Reconfig

–

t RRC_Request

The message flow diagram is shown in Figure 7 EPS Bearer Setup Procedure, Part 1 and Figure 8 EPS Bearer Setup Procedure, Part 2. The user initiated EPS bearer setup time is measured between the trigger points BUE and EUE: •

Begin: RRC CONNECTION REQUEST sent by the UE carrying a NAS: Service Request message in its body.

•

End: corresponding RRC CONNECTION RECONFIGURATION COMPLETE message or first PDU in UL sent by the UE (see References 4. [3GPP24.008]).

Id:0900d805807a3c48

53


Related KPIs

O

a) Attach Time, network initiated Service Request (EPS Bearer Setup) Time. b) No 3GPP defined KPI for EPS bearer setup found (exists).

Related PM counters

O

The eNB delay budget of the overall Service Request Time can be measured using two counters of [3GPP32.425], see References 15. •

Mean EPS Setup Time: SAEB.EstabTimeMean.QCI

•

Max EPS Setup Time: SAEB.EstabTimeMax.QCI

The latency is measured between the trigger points T1, T2 as can be seen in Figure 8 EPS Bearer Setup Procedure, Part 2.

54

Id:0900d805807a3c48



LTE-Uu UE

eNodeB

S6a

S11

S1- MME MME

S-GW / P-GW

UE EMM-REGISTERED and ECM-IDLE

BUE

RRC CONNECTION

DL data

D Paging Procedure

L2 SCTP: DL DATA NOTIFICATION

HSS

BN

Begin measurement

REQUEST RRC CONNECTION Setup SRB1

SETUP RRC CONNECTION SETUP COMPLETE (L3 NAS: SERVICE REQUEST)

S1-AP: INITIAL UE MESSAGE (L3 NAS: SERVICE REQUEST)

L2 SCTP: Create Bearer Request L2 SCTP: Create Bearer Response

If authentication vectors are not available in MME, authenticate UE S1-AP: UL NAS TRANSPORT RRC: DL INFORMATION (L3: AUTHENTICATION AND CIPHERING TRANSFER REQUEST) (L3: AUTHENTICATION AND CIPHERING REQUEST)

Activate EPS bearer setup in ePC

MAP: Send Authentication Info MAP: Send Authentication Info Ack

If authentication timer has expired, setup security association for user plane RB

RRC: UL INFORMATION TRANSFER (L3: AUTHENTICATION AND CIPHERING RESPONSE)

Figure 7


S1-AP: DL NAS TRANSPORT (L3: AUTHENTICATION AND CIPHERING RESPONSE)

EPS Bearer Setup Procedure, Part 1

Id:0900d805807a3c48

55


LTE-Uu UE

S1- MME eNodeB

T1

S11 MME

S1-AP: INITIAL CONTEXT

S6a S-GW / P-GW

HSS

PM Counter: EPS_SETUP_ATT

SETUP REQUEST S1-AP: INITIAL CONTEXT SETUP FAILURE RRC SECURITY MODE COMMAND RRC CONNECTION RECONFIGURATION

Setup user plane RB

RRC SECURITY MODE COMMAND COMPLETE EUE

RRC CONNECTION RECONFIGURATION

End measurement

COMPLETE

Means “Delay DL Packet Notification Request” to stop/ enable DL data transfer

First UL PDU

T2

S1-AP: INITIAL CONTEXT

PM Counter: EPS_SETUP_SUCC

SETUP COMPLETE L2 SCTP: Update Bearer Request L2 SCTP: Update Bearer Response

EN

First DL PDU

Figure 8

EPS Bearer Setup Procedure, Part 2

The service request times for UE and for network initiated EPS bearer setups are measured between the trigger points BUE and EUE, and BN and EN, respectively. These trigger points are colored green in the diagram. The trigger points T1 and T2 denote events, which increment the PM counters EPS_BEARER_ATT and EPS_BEARER_SUCC on the eNB. They are colored yellow. These counters can be used on the PM server to calculate EPS bearer setup success rates as seen by the eNB. Please note that this value will be different from that of the Service Request Success Rate KPI defined in this document (see Service Request (EPS) Success Rate). The eNB counters calculate the RB (and not the EPS) setup rate.

56

Id:0900d805807a3c48



3.4.1.5

Service Request (EPS) Time, Network Initiated

Service Request (EPS Bearer Setup) Time, Network Initiated, Control Plane


M Category: Latency, Object: EPS (S-GW S11 interface), Type: Mean and max (95%) values, Unit: [ms]

Definition

M It is the time taken by the LTE network to set up an EPS bearer on request by the P-GW. The EPS bearer has to be created before IP packets can be sent (DL) to the UE if the UE has no prop er EPS bearer for the given IP packet flow. The network initiated Service Request Time includes a Paging Time (ref. to Paging Time) if the UE is idle. The EPS Bearer Setup procedure is based on the Paging and Service Request procedures according to References 26. [3GPP36.331] and 3. [3GPP23.401].

Measurement method

O

Latency measurement, see Latency Measurements. Lab trial and stationary users only. Default EPS bearer setup. The setup times of ERAB and RB bearers, as well as the Paging time are part of the EPS setup time and can be calculated from the same sequence of measurements by taking appropriate traces.


O

Conditions for latency measurements see Reference Conditions for Latency Measurements. UE status before measurement: EMM-REGISTERED, but otherwise different initial states (idle, dormant or active).

Formula (logical)

M Mean value and 95% from all measured samples.Service Request Service Request Time [s]


O

=

t RRC_Reconfig

–

t RRC_Request

The message flow of the Service Request procedure is shown in Figure 7 EPS Bearer Setup Procedure, Part 1 and Figure 8 EPS Bearer Setup Procedure, Part 2. The network initiated service request time is measured between the trigger points BN and EN on the S-GW S11 interface: • •

Begin: L2 SCTP: DL DATA NOTIFICATION sent to MME End: L2 SCTP: Update Bearer Response sent to MME

If the service request is triggered by the arrival of a DL packet, the interrupt time of the packet stream is measured on the S-GW as the latency between: •

Begin: arrival of the DL packet at the S-GW toward the idle UE,

•

End: forwarding of the same packet to the UE connected.

Related KPIs

O

Attach Time, Paging Time, E-RAB and RB Setup Times.

Related PM counters

O

The eNB delay budget of the overall Service Request (ESP) Time is identical to that of the UE initiated procedures. See related PM counters and measurements there.

The network initiated Service Request Time is not described f rom the UE point of view, because the delay component seen on the UE is not the complete time the downlink data packet is delayed before it can be fo rwarded from the P-GW to the UE. The network initiated EPS bearer setup time is dependent on the mobility state of the mobile terminal. If the terminal is idle, paging becomes necessary.


Id:0900d805807a3c48

57


3.4.1.6

Service Request (EPS) Success Rate

Service Request (EPS) Success Rate, Control Plane

KPI Category,Object, Type, Unit

M Category: Accessibility, Object: EPS (UE R, eNB Uu), Type: Ratio, Unit: [%]

Definition

M This KPI is defined as the ratio between successfully established EPS bearers compared to the overall number of EPS bearer establishment attempts. It corresponds to the probability that a user or the LTE network can establish an EPS bearer at any moment in time. Requests that are terminated by timer expiry (due to the unaccessibility of some LTE resource) are considered as unsuccessful attempts. Au thentication errors (requests rejected by the MME) are included in the total number of failures. Only the first RRC CONNECTION REQUEST is to be considered, since counting retries of the same message would increase the overall number of bearer establishment attempts, and thus reduce the success ratio.

Measurement method

O

Accessibility measurement, see Service Accessibility Measurements. Field and lab trial. Stationary users only. For calculating the success ratio, the same series of "Service Request (EPS) Time" measurements can be used.


O

Conditions for accessibility measurements see Service Accessibility Measurements. See additional conditions in Service Request (EPS) Time, Network Initiated .

Formula (logical)

M EPSSR


O

=

nu mbe r_of(RRC_CONN_RECONFIGURATION_COMPLETE) ----------------------------------------------------------------------------------------------------------------------------------------------------------------------⋅ 100% number_of(RRC_CONNECTION_REQUEST)

The message flow diagram is shown in Figure 7 EPS Bearer Setup Procedure, Part 1 and Figure 8 EPS Bearer Setup Procedure, Part 2. The trigger points for UE initiated EPS bearer setup success ratio on the R interface are: •

Success: RRC CONNECTION RECONFIGURATION COMPLETE message sent by the UE, see EUE in Figure 8 EPS Bearer Setup Procedure, Part 2.

•

Total: an attempt is made to send an RRC CONNECTION REQUEST with appropriate establishment cause by the UE, see BUE in Figure 7 EPS Bearer Setup Procedure, Part 1. Trigger points on the eNB Uu interface:

•

Success: RRC CONNECTION RECONFIGURATION COMPLETE message received by the eNB.

•

Total: an attempt is made to receive an RRC CONNECTION SETUP COMPLETE MESSAGE by the eNB with L3 NAS: SERVICE REQUEST in its body.

The measurement on the eNB side is an approximation of the success ratio as perceived by the end-user, since it does not include SRB setup failures. The UE side delivers more precise results.The exact calculation of network initiated EPS bearer setup success ratios uses the triggers BN and EN, which are also shown in Figure 7 EPS Bearer Setup Procedure, Part 1 and Figure 8 EPS Bearer Setup Procedure, Part 2.

58

Id:0900d805807a3c48



Related KPIs

O

Service Request (EPS) Time, user and network initiated.The "E-RAB Accessibility" KPI of [3GPP32.450], see References 17. chapter 6.1.1 corresponds well with this KPI, considering explicitly created "additional" EPS bearers:

For initial EPS bearers, see KPI Attach Success Rate. Related PM counters O

a) 3GPP defines the following measurements for additional EPS bearers: h. Additional EPS setup attempts: SAEB.NbrAttEstabAdd.QCI i. Successful setups: SAEB.EstabAddSuccNbr.QCI. j. Failures: SAEB.EstabAddFailNbr.Cause For initial EPS bearer setup measurements, see Attach procedure in Attach Success Rate. b) The corresponding eNB counters are: •

EPS_SETUP_ATT, EPS_SETUP_SUCC, EPS_SETUP_FAIL each counts both initial and additional EPS setups (i.e. no differentiation is made). See also trigger points T1 & T2 in Figure 8 EPS Bearer Setup Procedure, Part 2.

Using online PM measurements on the eNB to count the total number of EPS bearer setup requests ( EPS_SETUP_ATT ) and the successfully terminated ones (EPS_SETUP_SUCC ), an EPS setup success rate can be calculated (see trigger points T1 and T2). This ratio is, howe ver, not identical to the user perceived success rate, since the latter includes additional failure causes, like failures in sett ing up the SRB signaling bearer and the ePC part of the EPS bearer, as well as UE authentication errors.


Id:0900d805807a3c48

59


3.4.1.7

Service (EPS Bearer) Drop Rate

Service (EPS Bearer) Drop Rate, Control Plane


M Category: Availability, Object: EPS (UE R, eNB Uu), Type: Ratio, Unit: [%]

Definition

M It is the ratio between abnormally released bearers and the overall number of established EPS bearers. An ab normal release is defined as any EPS bearer termination that was not triggered by the mobile user (from UE side). Thus, it reflects the probability that an established bearer is aborted due to insufficient network resources. Dropping the bearer becomes visible to the end-user if an application service is actively using it. If the application automatically re-establishes the bearer, it remains unnoticed by the user.

Measurement method

O

Availability measurement, see Service Accessibility Measurements. Field and lab trial.


O

Conditions for availability measurements see Service Accessibility Measurements.The ratio is calculated on the same series of Service Request (EPS) Time measurements (see Service Request Time (EPS), UE Initiated and Service Request (EPS) Time, Network Initiated).

Formula (logical)

M EPSBearerD R


O

=

number_of(dropped_calls) ------------------------------------------------------------------------------ ⋅ 100% number_of(successful_calls)

Trigger points on UE R / eNB Uu interfaces: •

Success: submission of the message RRC CONNECTION RECONFIGURATION COMPLETE by the UE, see References26. [3GPP36.331], or reception by the eNB.

•

Drop: successful establishments minus terminations by the user, i.e. the UE submitting a L3DETACH REQUEST (see References 4. [3GPP24.008]) carried within the RRC message UL NAS TRANSPORT (see References 26. [3GPP36.331]), or the eNB receiving this message.

Measurements on the eNB side produce eq uivalent results to measurements on the UE side. The drop ratio calculated this way is valid for both UE initiated EPS bearers. For the calculation of a network initiated EPS bearer drop rate, ePC initiated L3 DETACH REQUEST -s needs to be considered.

60

Id:0900d805807a3c48



Related KPIs

O

a) Service Request (EPS) Success Rate. b) The "E-RAB Retainability" KPI of [3GPP32.450], see References 17. in chapter 6.2.1, is an abnormal EPS release rate, which is related to the above Drop Rate definition, but also considers the session duration (its unit is [1/s]):

∑ SAEB.RelAct.[QCI] R2

=

QC I -------------------------------------------------------------

SAEB.SessionTimeUE

The necessary measurements are defined by 3GPP (see below). Related PM counters

O

a) The following online PM counters of the eNB can be used to calculate the Service Drop Rate: EPS_SETUP_SUCC, EPS_REL_EPC_INI, EPS_REL_ENB_INI. b) 3GPP defines the following measurements for an abnormal EPS bearer release: SAEB.RelAct.QCI, SAEB.SessionTimeUE and SAEB.SessionTimeQCI.QCI

The EPS bearer release rate determined by online PM counters on the eNB is equivalent to the value of the Service (EPS) Drop Rate KPI.


Id:0900d805807a3c48

61


3.4.1.8

Handover Procedure Time

Handover Procedure Time, Control Plane


M Category: Mobility (Latency), Object: LTE (UE R interface), Type: Mean and max (95%) value, Unit: [ms]

Definition

M It denotes the total time needed for the hand-over procedure as seen by the UE. It begins by receiving a Handover Command from the SeNB and ends by sending the Handover Confirm response to the TeNB by the UE. Its relevance is the discontinuity of the IP packet f low in the user plane that is implied by it (also called service interruption). The value of the HO Procedure Time KPI depends on the hand-over scenario. The following HO scenarios are distinguished (though not directly seen by the UE): •

intra LTE intra- and inter-frequency mobility

•

inter RAT mobility (LTE ↔ 2G/3G)

•

intra vs. inter eNB, the latter via X2, or S1 interface

•

intra vs. inter MME/S-GW

Remark: The value of the HO Procedure Time itself does not depend on the direction of an ongoing data transfer (UL, or DL). It is the LTE Service Interrupt Time (HO) which will be different. Measurement method

O

Assumptions, pre-conditions O Formula (logical)

Mobility (HO) measurement, see Mobility (Handover) Measurements. Lab and field trials, mobile user scenarios only, crossing of cell borders included. The handover time should be measured on the UE R interface. However, measurements on the source eNBs also provide good approximations of the handover time as perceived by the UE. They even have some advantage, since the SeNB sees the full handover procedure, the handover preparation phase included. End-user delays can be measured with test equipment only, since terminals do not indicate cell changes toward the end-user. Conditions for handover measurements see Mobility (Handover) Measurements. UE status before measurement: registered and connected.

M Mean value and 95% from all measured samples. Handover Procedure Time [ms]


Related KPIs

O

O

=

t HO_Confirm

–

tHO_Command

The HO procedure time is seen by the UE as the elapsed time between (see message flow in Figure 9 Handover Procedure: Intra MME/S-GW, Inter eNB via X2): •

BUE: UE receives Handover Command in the body of an RRC Connection Reconfiguration request from source eNB (SeNB).

•

EUE: UE sends Handover Confirm to the target eNB (TeNB) encoded in an RRC Connection Reconfiguration Complete message.

a) (LTE) Service Interrupt Time, (RB) Service Interrupt Time, HO Success Rate. b) No 3GPP defined KPI for HO procedure time found.

Related PM counters

62

O

HO Procedure Time related measurements are not defined in 3GPP Release 8 documents.

Id:0900d805807a3c48



Figure 9


Handover Procedure: Intra MME/S-GW, Inter eNB via X2

Id:0900d805807a3c48

63


Figure 10

64

Handover Procedure: Intra MME/S-GW, Inter eNB via S1

Id:0900d805807a3c48



3.4.1 .4.1.9 .9

Hand Handov ove er Suc Succe cess ss Rate Rate

Handover Procedure Time, Control Plane


M Category: Category: Mobility Mobility (Accessibili (Accessibility), ty), Object: Object: eNB (X2 (X2 interfaces interfaces), ), Type: Ratio, Unit: [%]

Definition

M The Handover Success Rate is the ratio between successfully executed (committed) HO procedures and the number of all Handover attempts.

Meas easurem uremen entt meth method od

O

Mobi Mobili lity ty (HO) (HO) measu easure reme ment nt,, see see Mobility (Handover) Measurements. Measurements . Field and lab trial. For calculating the success ratio, the same series of "Service Interrupt Time" measurements can be used. For restrictions in measurement scenarios, refer to the description of the KPI "(LTE) Service Interrupt Time". The measurement is to be executed for different HO scenarios (see description with the KPI Handover Procedure Time). Time ). If practical limitations make it difficult carry out a sufficient number of performance tests with terminals, eNB online statistics can be used as approximations (as described below). In case of Inter-RAT scenarios, t he eNB statistics cover the LTE leg of the hand-off only. HO failures in the 2G/3G leg have to be collected with corresponding statistics of the BTS/NodeB.

O

Conditi Conditions ons for for mobil mobility ity measure measuremen ments ts see see Mobility (Handover) Measurements. Measurements . See additional conditions at the KPI “(LTE) Service Interrupt Time".

Assumptions, pre-conditions Formula (logical)

M HOSR


O

=

number_of(Handover_Confirm) -------------------------------------------------------------------------------------- ⋅ 100% number_of(Handover_Request)

The success success rate rate perceived perceived by end-use end-users rs is approxim approximated ated with with the success success rate as seen on the source eNB. Trigger points on the eNB X2 interface: • •

Total #: Handover Request sent by SeNB to TeNB (X2). Success #: UE Context Release message received by SeNB from TeNB.

In case of Intra eNB HO procedures, no Handover Request is sent, both trigger points are to be counted by the (Source) eNB internally. Related KPIs

O

a) Handover Procedure Time b) “E-UTRAN Mobility Success Rate" in [3GPP32.450], see References17. Chapter 6.5.1, is a success rate, which applies to all Intra- and Inter-RAT mobility scenarios:

The KPI is calculated by multiplying the success rates for HO preparation and execution phases. The relationship of variables in this expression and the HO measurements defined in [3GPP32.425],see References 15. 15.,, is still to be clarified.


Id:0900d805807a3c48

65


Related PM counters

O

a) 3GPP defines measurements / c ounters on eNB to support HO success ratio calculations (see References 15. 15. [3GPP32.425] [3GPP32.425] chapter 4.3). Examples for selected scenarios: Intra-RAT, intra-eNB handovers: •

# HO attempts: HO.IntraENBOutAtt.Cause • # HO success: HO.IntraENBOutSucc.Cause where Cause identifies the cause for handover Intra-RAT, inter-eNB handovers: •

# HO attempts: HO.InterENBOutPrepAtt

•

# HO attempts: HO.InterENBOutAtt.Cause

• # HO success: HO.InterENBOutSucc.Cause Handover measurements on neighbor cell basis: •

# HO attempts: HO.OutAttTarget.Cause • # HO success: HO.OutSuccTarget.Cause Inter-RAT handovers (LTE -> 2G/3G): •

# HO attempts: HO.IartOutAtt.Cause

•

# HO success: HO.IartOutSucc.Cause

b) LTE product related counters on eNB which support the calculation of handover success rates: Intra-RAT, intra-eNB handovers: •

# HO attempts: INTRA_HO_PREP

• # HO success: INTRA_HO_SUCC Intra-RAT, iner-eNB handovers: •

# HO attempts: INTER_HO_PREP

• # HO success: INTER_HO_SUCC Inter-RAT handovers (LTE -> 2G/3G): •

# HO attempts: INTER_RAT_HO_PREP

•

INTER-RAT_HO_SUCC # HO success: INTER-RAT_HO_SUCC

The calculation of HO Success Rate does not differentiate between different causes of a negative hand-over decision. All rejected HO attempts are considered as unsuccessful handovers.

66

Id:0900d805807a3c48



3.4.1.10 .10

Pag Paging ing Time

Paging Time, Control Plane

KPI KPI Cate Categor gory, y, Obje Object, ct, Type, Type, Uni Unitt

M Cate Categor gory: y: Late Latenc ncy, y, Obje Object: ct: LTE LTE (MME, (MME, S-G S-GW; W; S11, S11, S1-U S1-U), ), Type: Mean and max (95%) value, Unit: [s]

Definition

M Paging is initiated : •

By the arrival of a DL packet pa cket at the S-GW for an UE w hose location is not known, i.e. its location information does not exist in the S-GW, or

•

By network initiated EPS bearer setup requests (ISR flag is set)

It denotes the total procedure time from starting the paging request in DL and terminating it with the subsequent service request (EPS bearer setup) of the UE after it has been located. Thus, the paging time is per definition the difference between network and UE initiated Service Request Times. Trigger events can be defined on the MME and S-GW as initiators of the paging request, either on the S11 (MME - S-GW) or on the S1-U (UE - S-GW) interfaces, see "Trigger points" below. Measurement method

O

Latency measurement, see Latency Measurements. Measurements. Lab trial with mobile user scenario only, crossing of cell borders included. The paging time can be measured on the S11 (MME - S-GW), or S1-U (UE - S-GW) interfaces. End-user delays cannot be measured, since paging is always initiated by the network.


O

Conditions for latency measurements see Reference Conditions for Latency Measurements. Measurements. UE status before measurement: registered, but idle (not connected).

Formula (l (logical)

M The pa paging time is calculated as th the di difference be between ne network initiated service request and UE initiated service request times. P ag ag i ng ng Time [s]


O

=

t SRT network initiated

–

tSR T UE initiated initiated

See message flow in Figure 11 Paging Procedur Procedure e.The paging time can be approximated on the MME S11 with: •

Start: L2 SCTP: DL Data Notification received for an UE

•

Stop: L3 NAS: SERVICE REQUEST received received from the same UE

On S-GW S1-U, latency is measured between: •

Start: DL packet arrives for an idle UE

•

Stop: DL packet is forwarded to the connected UE

The latter latency is a good metric of DL "service interrupt time" for an IP packet stream caused by paging. The paging time has to be calculated as the measured latency minus the UE initiated Service Request Time (EPS). Related KPIs

O

a) Service Request Time (EPS), UE and network initiated. b) No 3GPP Release 8 defined KPIs for Paging Time found.


Id:0900d805807a3c48

67


Related PM counters

O

Paging Time related measurements are neither defined in 3GPP Release 8, nor in LTE reference documents.

LTE-Uu UE

eNodeB

S6a

S11

S1- MME MME

S-GW / P-GW

HSS

DL Packet

BN D e l a y

UE EMM-REGISTERED and ECM-IDLE

Paging Procedure [3GPP TS 36.413]

L2 SCTP: DL Data Notification

Paging Attempt T1 S1-AP: DL NAS TRANSPORT

RRC: DL INFORMATION TRANSFER

(L3 NAS: PAGING)

To all cells of all TAs

(L3 NAS: PAGING) Retry (optionally)

L2 SCTP: DL Data Notification Ack T i m e r

Paging Failure T2

L2 SCTP: DL Data Notification Reject

UE Triggered Service Request Procedure [3GPP TS 23.401]

UE in EMM-REGISTERED and ECM-CONNECTED

L2 SCTP: Stop Paging Forward DL Packet to UE

Figure 11

68

EN

Paging Procedure

Id:0900d805807a3c48



3.4.1.11

Paging Failure Rate

Paging Failure Rate, Control Plane


M Category: Accessibility, Object: MME (S1-AP, S11), Type: Ratio, Unit: [%]

Definition

M The Paging Failure Rate is the ratio between unsuccessful paging requests and the number of all paging attempts initiated by the MME. Retries of the same paging request by the MME are not counted as new attempts. Similarly, multicasting the same request to more than one eNBs (in the UE´s tracking areas) is considered as one attempt.

Measurement method

O

Accessibility measurement, see Service Accessibility Measurements. Field and lab trials. For calculating the failure ratio, the same series of "Service Request (EPS) Success Rate" measurements can be used, h owever, restricted to the network initiated bearer setup scenario. For restrictions in measurement scenarios, see also the KP I “Service Request Success Rate". The existence of online performance counters for paging on the MME can help to avoid the execution of performance tests repeatedly. Suitable counters are indicated in the message flow in Figure 11 Paging Procedure (trigger points T1 and T2).


O

Conditions for accessibility measurements, see Service Accessibility Measurements. See additional conditions at the KPI "Service Request Success Rate".

Formula (logical)

M PagingFR


Related KPIs

O

O

=

number_of(Paging_Failures) -------------------------------------------------------------------------------- ⋅ 100% number_of(Paging_Attempts)

Trigger points on MME (see T1: Attempt and T2: Failure on the Figure 11 Paging Procedure): •

Total #: different L3 NAS Paging Requests sent to an eNB (S1-AP).

•

Failure #: DL Data Notification Reject messages sent to S-GW (S11)

a) Paging Time, Service Request Success Rate - network initiated. b) No Paging related KPI is defined in 3GPP Release 8 documents.

Related PM counters

O

Directly applicable trigger events are T1 and T2 as shown in Figure 11 Paging Procedure. a) 3GPP defined counter (see References15. [3GPP32.425] Chapter 4.7) on eNB is: •

Discarded Paging Records: PAG.DiscardedNbr

b) LTE product related counters defined for eNB in: • •

Requests: RRC_PAGING Responses: SIG_CON_EST_ATT.Cause = PagingResponse

Using these counters, the Paging Failure Rate can be calculated as: Paging FR =( 1 - SIG_CON_EST_ATT.Cause = PagingResponse / RRC_PAGING *100%)


Id:0900d805807a3c48

69


3.4.1.12

(LTE) Round Trip Time

Round Trip Time (RTT), User Plane


M Category: Latency, Object: LTE (UE R, P-GW Gi interfaces), Type: min (CDF 5%), mean (CDF 50%), max (CDF 95%) values, Unit: [ms]

Definition

M RTT in UL is the interval between sending a datagram by the UE & receiving the corresponding reply from an IP peer entity connected to the Gi interface of the P-GW. RTT in DL is the interval between sending a datagram to the UE & receiving the corresponding reply by the IP host (peer entity).

Measurement method

O

Latency measurement, Latency Measurements. Field and lab trials. Stationary users in different cell positions, which are uniformly distributed across the cell. RTT is measured with the Ping application between the UE and an IP host (peer entity). As average RTT figure, the output of the Ping application is used (e.g. " MS-DOS >ping -n count -l size host_name "). Size of ICMP packets: 32, …, MTU, 2x MTU. Ping should be executed at least 100 times to reduce the impact o f the first ICMP message, which triggers the setup of radio bearer establishments in UL and DL directions.


O

Conditions for latency measurements see Reference Conditions for Latency Measurements. UE status before measurement: registered and connected. The IP peer entity should be located as close as possible to the SGi interface. Additional delays between the P-GW and the IP host should be avoided or minimized.

Formula (logical)

M Mean value and 95% from all measured samples. R o un d Trip Time [ms]


O

•

O

t ICMP Echo Reply

–

t IC MP

Echo Request

Trigger points both from UE, and from IP peer entity view: •

Related KPIs

=

Start: ICMP ECHO REQUEST Stop: ICMP ECHO REPLY

a) Network Delay UL / DL KPI for LTE, RB and IP based transport bearers X2, S1, S5/S8) b) No 3GPP defined KPI for LTE RTT found. c) See "{Service} Round Trip Time" KPI definition for Ping service in [ETSI102.250-2], see References 39. Chapter 4.6.3.6.

Related PM counters

70

O

Measurements of round trip times are neither defined in 3GPP nor in LTE reference documents.

Id:0900d805807a3c48



3.4.1.13

(LTE) User Data Rate

User Data Rate UL / DL, User Plane


M Category: Throughput, Object: LTE (UE R if, IP peer entity) Type: Min (CDF 5%), Mean (CDF 50%) Max (CDF 95%) values, Unit: [Mbps]

Definition

M The metric describes the data speed available to one user of the LTE network on UDP/IP level. It is given as the maximum (95%-ile) value that can be observed over a short period of time (e.g. of 1s) and as a mean value that characterizes longer data transfer periods (minutes). Its value distribution over the radio cell is given as a function of the SINR. The maximum value is often referred to in the literature as instantaneous "Peak Throughput" that is achieved in optimal radio conditions. The user data rate can be given for a single user active in the cell (single user data rate), or to one of several concurrently active users.

Measurement method

O

Throughput measurement, see Throughput Measurements. Stationary and mobile user. Cell positions: best, medium, cell edge, including different HO scenarios. The data rate is measured in UL/DL direction with UDP/IP traffic over a time period of several minutes. Signaling delays, e.g. initial radio bearer setup has no impact on the result, because transient times at the beginning and end of the data transfer are not considered.


O

Conditions for throughput measurements see Peak User Data Rate. The IP host used as traffic generator (e.g . with Iperf) should be connected directly to the SGi interface. Additional delays and bandwidth restrictions between the S/P-GW and the server need to be excluded.

Formula (logical)

M UserDataRate


O

=

tr ans fer re d_data_volume [bytes] ⋅ 8 --------------------------------------------------------------------------------------------------tran sfe r_time [s] ⋅ 1000

Trigger points on the UE R interface: •

UL: UE sending UDP/IP packets to the IP peer entity.

•

DL: IP peer entity sending UDP/IP packets to the UE.

Related KPIs

O

(RB) User Data Rate, (FTP) User Data Rate.

Related PM counters

O

Measurements of user data rate on LTE level (R <-> Gi) are neither defined in 3GPP, nor LTE reference documents.

The User Data Rate KPI has b een defined on E2E network level to verify if the capacity of large non-GBR EPS bearers (QoS profile with large maximum bit rates) can in fact be exploited by IP based services. The same holds for the E-RAB and RB bearers being part of the EPS. Especially, the ma ximum achievable data rate of the RB bearer is interesting, since it is the critical resource of the LTE network. For this reason, the Peak Data Rate is defined as a separate KPI on the RB level (see (RB) Packet Loss Rate), too. Because of this relationship (containment) between EPS, E-RAB and RB, user dat a rate KPIs can be verified with the same series of measurements. The KPIs a re interesting as values calculated over the full duration of long data transfers (~minutes). Of equal interest are peak values observed for a short time period only (~seconds). Please note that data rate KPIs have also been defined for application services, e.g. User Data Rate KPI of the Data Download / Upload service, but these do not characterize the capability of the LTE network alone, but also depend on the application service and its protocols.


Id:0900d805807a3c48

71


3.4.1.14

(LTE) Packet Loss Rate

(LTE) Packet Loss Rate UL / DL, User Plane


M Category: Availability, Object: LTE (UE R interface, IP peer entity) Type: Ratio, Min (CDF 5%), Mean (CDF 50%) Max (CDF 95%) values Unit: [%]

Definition

M This is the ratio between the numbers of lost or corrupted IP packets, and of all IP packets sent. Corrupted IP packets are those that contain bit errors in their headers or i n their payload.Packets with "residual", i.e. undetected errors are not counted as lost.

Measurement method

O

Availability measurement, see Service Accessibility Measurements. The EPS bearer is to be configured such that corrupted IP packets are not delivered to the application. Measured in loaded and unloaded network, under different radio conditions. Stationary and mobile users. The packet loss rate should be mea sured in test scenarios, where handovers occur. The number of handovers should be oriented to ward the NSN reference traffic model, or, in operator t rials, toward the operator´s traffic model.


O

Conditions for availability measurements, see Service Accessibility Measurements. Measured in loaded and unloaded network, under different radio conditions. Stationary and mobile user including HO scenarios. The ratio can be calculated on a series of User Data Rate (see Peak User Data Rate) measurements.

Formula (logical)

M PL R


Related KPIs

O

O

=

number_of(lost_corrupted_packets) ------------------------------------------------------------------------------------------------ ⋅ 100% number_of(all_packet_sent)

UL / DL: •

Total: packages sent at UE R if / IP peer entity

•

Lost: Total minus packets received at IP peer entity / UE R if.

a) (RB) Initial Radio Block Error Rate (BLER) b) No PLR KPI (LTE level) is defined in 3GPP and ETSI documents.

Related PM counters

72

O

Measurements of packet loss rate on LTE level (R <-> Gi) are neither defined in 3GPP nor in LTE reference documents.

Id:0900d805807a3c48



3.4.1.15

(LTE) Service Interrupt Time (HO)

(LTE) Service Interrupt Time (HO), User Plane


M Category: Latency, Object: LTE (UE R if) Type: Mean and max (95%-ile) value, Unit: [ms]

Definition

M The Service Interrupt Time is the interval between the last sent/received IP packet of a continuous UL/DL data stream in the old cell and the first sent/received user IP packet in the new cell measured on the UE (also called "user plane break"). The value of the KPI depends of the handover scenario. See HO scenarios at the KPI Handover Procedure Time.

Measurement method

O

Mobility (HO) measurement, see Mobility (Handover) Measurements. Only mobile user scenarios including cell border crossing. The test application is an FTP upload/download service, or an isochronous UP/IP flow (iperf) maintained during handover.


O

Conditions for availability measurements, see Mobility (Handover) Measurements. In inter eNB "HO via X2" scenariosthe X2 interface shall be enabled between the source and target eNBs.

Formula (logical)

M Service Interrupt Time [ms]

=

t fir st

packet to/from TeNB

–

tla st

packet to/from SeNB

Mean value and 95% from all measured samples. Message flow, trigger points

O

The trigger points for service interrupt time (from UE point of view): •

BUE: UE sends/receives last packet to/from source eNB

•

EUE: UE sends/receives first packet to/from target eNB.

See Figure 9 Handover Procedure: Intra MME/S-GW, Inter eNB via X2 Related KPIs

O

a) HO Procedure Time, Voice Interrupt Time (HO) b) No related KPI definitions found in 3GPP Rel. 8 documents.

Related PM counters


O

Measurements of service interrupt times caused by HO are neither defined in 3GPP nor in LTE reference documents.

Id:0900d805807a3c48

73


3.4.2 3.4.2.1

Radio Bearer KPIs (RB) Packet Loss Rate

(RB) Packet Loss Rate UL / DL, User Plane


M Category: Availability, Object: RB (UE R interface, eNB) Type: Ratio, Min (CDF 5%), Mean (CDF 50%) Max (CDF 95%) values Unit: [%]

Definition

M This is the ratio between the numbers of lost or corrupted IP packets, and of all IP packets sent. Corrupted IP packets are those that contain bit errors in their headers or in their payload. Packets with "residual", i.e. undetected errors are not counted as lost.

Measurement method

O

The KPI is not measured on its own. Traces are to be taken on the UE and eNB during (LTE) Packet Loss Rate measurements, see (RB) Packet Loss Rate. Lab trials with results for internal use.


O

See comment above.

Formula (logical)

M RBPLR


O

=

number_of(lost_corrupted_packets) ------------------------------------------------------------------------------------------------ ⋅ 100% number_of(all_packet_sent)

UL / DL: •

Total #: packages sent on UE R / eNB

•

Lost #: Total minus packets received on eNB / UE R

Measured at the upper SAPs of the L2 PDCP. Related KPIs

O

a) Initial Radio Block Error Rate (BLER) b) No PLR KPI (RB level) is defined in 3GPP and ETSI documents.

Related PM counters

O

a) The following measurements are defined in LT E for the number of discarded (dropped) PDUs on the eNB in DL: •

Discarded PDCP SDUs: PDCP_SDU_DISCARD

•

Discarded RLC blocks: SDU_DISCARD_DL_DTCH

b) The following measurements are defined by 3GPP in [3GPP32.425], see References 15. chapters 4.4.3-4, for packet (PDCP SDU) loss and drop rates: •

Drop rate on eNB in DL: DRB.PdcpSduDropRateDl.QCI

•

Loss rate on Uu in DL: DRB.PdcpSduAirLossRateDl.QCI

• Loss rate on Uu in UL: DRB.PdcpSduLossRateUl.QCI A packet drop rate on eNB in UL is not defined.The loss rate values may refer to the L2 protocols PDCP, RLC or MAC (see [3GPP32.314] chapter 4.1.5.1-3)

74

Id:0900d805807a3c48



3.4.2.2

(RB) User Data Rate

(RB) User Data Rate UL / DL, User Plane


M Category: Throughput, Object: RB, eNB, Type: Min (CDF 5%), Mean (CDF 50%), Max (CDF 95%) values, Unit: [Mbps]

Definition

M The metric describes the UDP/IP data rate achievable by one user. It can be given as single user data rate if only one user is active in the cell, or as multi-user data rat e for a given number of concurrently active users. The user data rate distribution over the cell is given as function (CDF) of the radio conditions characterized by the SINR. The time variation of the KP I value in a given cell position is given with its maximum (95%) (also referred to as instantaneous "Peak User Data Rate"), and with its average (mean) value. The Peak User Data Rate corresponds to the Cell Throughput KPI on UDP/IP level, when the number of active users in the cell is n=1 and best radio conditions are observed. The User Data Rate is also called "User Throughput" in the literature.

Measurement method

O

Throughput measurement, see Throughput Measurements. Stationary and mobile user. Cell positions: best, medium, cell edge, including different HO scenarios. Field and lab trials. The eNB assigns all resources (radio blocks) to this user, except for the radio blocks used for signaling and controlling. Measured on UDP/IP level for all UE categories (1-5). With the help of trace analysis, PDCP, RLC, MAC and PHY layer throughputs can also be defined. With and w/o concurrency in UL / DL traffic. The mean user data rate is calculated over a few minutes, peak data rate values are averaged over short periods of time (e.g. 1s).


O

Formula (logical)

M

Conditions for capacity measurements, see Peak User Data Rate. All transport bearers between the UE and IP peer entity should have higher capacity than the radio link in order to avoid bandwidth bottlenecks.

RBUserDataRate

=

transf erred_data_volume [bytes] ⋅ 8 ------------------------------------------------------------------------------------------------6 transfer_time [s] ⋅ 10 –


O

Trigger events on the eNB: •

UL: eNB receiving PDUs

•

DL: eNB sending PDUs

Measured at the upper SAPs of the protocols PDCP, RLC, MAC or PHY. Related KPIs

O

a) (LTE) User Data Rate, Cell Throughput. b) See 3GPP defined KPIs in Cell Throughput.

Related PM counters


O

See 3GPP defined measurements with the KPI in Cell Throughput.

Id:0900d805807a3c48

75


3.4.2.3

Cell Throughput

Cell Throughput UL / DL, User Plane


M Category: Throughput, Object: UE, RB Uu, Type: Min (CDF 5%), Mean (CDF 50%), Max (CDF 95%) values, Unit: [Mbps]

Definition

M The metric shows the sustainable aggregate throughput of the cell (in UL/DL) available to "n" stationary users distributed uniformly in the cell and running a typical mix of applications. The "cell throughput" is the sum of all bits transported in all radio blocks carrying PDUs (i.e. bits in UL-SCH / DL-SCH transport blocks) during one second. The cell capacity is also given as pea k value (called peak cell capacity, or throughput), which is defined as the aggregate throughput of "n" users all located in best radio conditions. The cell throughput value is defined here on PHY level, but could be given for other protocol levels (UDP/IP, PDCP, RLC, MAC), too. When the (peak, average) cell throughput is expressed on UDP/IP level, it corresponds to the (peak, mean) user data rate value at comparable radio conditions.

Measurement method

O

Throughput measurement, Throughput Measurements. Stationary users uniformly distributed in the cell. UE categories according to application mix. Peak value measured with one user in LoS cell position using UDP/IP load to approach the full buffer condition. Field trial without target value (only to learn the available capacity). Measured with UDP/IP traffic on PHY layer (UL-SCH / DL-SCH transport block bits) over several minutes. With the help of trace analysis PDCP, RLC and MAC layer values can also be given.

Assumptions, pre-conditions O

Conditions for throughput measurements see Attach Success Rate. All IP transport bearers (S1, S5/S8, SGi) between the UE and IP peer entity should have higher bandwidth than the cell capacity to avoid bandwidth bottlenecks.

Formula (logical)

M CellThroughput

=

transf erred_data_volume [bytes] ⋅ 8 ------------------------------------------------------------------------------------------------6 tra nsfe r_time [s] ⋅ 10 –

Message flow, trigger points O

Related KPIs

O

Trigger points on the eNB Uu interface: •

UL: eNB receiving PDUs / RLC blocks / transport data blocks

•

DL: eNB sending PDUs / RLC blocks / transport data blocks

a) (RB) User Data Rate b) 3GPP defines the KPI "E-UTRAN IP Throughput" in [3GPP32.450], see References 17. chapter 6.3.1.

76

Id:0900d805807a3c48



Related PM counters

O

a) The following measurements are available on eNB in L TE reference document, which are related to the Cell Throughput KPI: •

CELL_THROUGHPUT_PDCP (separate counters for min, mean and peak values, both for UL and DL on Uu interface)

•

PDCP_SDU (same for one cell)

b) The following measurements are defined by 3GPP in [3GPP32.425] Chapters 4.4.1 for cell data (PDCP SDU) rates: • Average data rate in DL: DRB.PdcpSduBitrateDl.QCI •

Maximal data rate in DL: DRB.PdcpSduBitrateDlMax

• Average data rate in UL: DRB.PdcpSduBitrateUl.QCI •

Maximal data rate in UL: DRB.PdcpSduBitrateUlMax

The data rate values are measured on the eNB at the upper SAP of L2 PDCP and refer to SDU bits. SDUs that are forwarded over X2/S1 to another eNB are subtracted from the overall bit count.


Id:0900d805807a3c48

77

References


4 References 1. [3GPP21.905] 3GPP TR 21.905; v 8.x.x; Vocabulary for 3GPP Specification 2. [3GPP23.203] 3GPP TS 23.203; V8.7.0 (2009-09); Policy and charging control architecture 3. [3GPP23.401] 3GPP TR 23.401; v.8.x.x; General Packet Radio Service (GPRS) enhancements for Long Term Evolution (LTE) access 4. [3GPP24.008] 3GPP TR 24.008; V8.5.0 (2009-03); Mobile Radio Interface Layer 3 Specification; Core network protocols; Stage 3 5. [3GPP24.228] 3GPP TR 24.228; V5.15.0 (2006-10); Signaling flows for the IP multimedia call control based on Session Initiation Protocol (SIP) and Session Description Protocol (SDP);Stage 3 (Release 5) 6. [3GPP24.229] 3GPP TR 24.229; V8.8.0 (2009-06);IP multimedia call control protocol based on Session Initiation Protocol (SIP) and Session Description Protocol (SDP);Stage 3 (Release 8) 7. [3GPP25.913] 3GPP TR 25.913; V8.0.0 (2008-12); Requirements for Evolved UTRA (E-UTRA) and Evolved UTRAN (E-UTRAN) 8. [3GPP25.943] 3GPP TR 25.943; V8.0.0 (2008-12); Deployment Aspects (Release 8) 9. [3GPP26.114] 3GPP TS 26.114 V8.2.1 (200 9-03); IP Multimedia Subsystem (IMS); Multimedia Telephony; Media handling and interaction (Release 8) 10. [3GPP29.002] 3GPP TS 29.002, V8.x.x; Mobile Application Part (MAP) 11. [3GPP29.060] 3GPP TS 29.060; v 8.x.x; General Packet Radio Service (GPRS); GPRS Tunneling Protocol (GTP) across the Gn and Gp interface 12. [3GPP32.410] 3GPP TS 32.410; V8.0.0 (2009-03); Telecommunication management; Key Performance Indicators for UMTS and GSM (Release 8) 13. [3GPP32.421] 3GPP TS 32.421, V8.4.0 (2008-12); Telecommunication management; Subscriber and equipment trace; Trace concepts and requirements 14. [3GPP32.423] 3GPP TS 32.423, V8.0.0 (2009-03); Telecommunication management; Subscriber and equipment trace, Trace data definition & management 15. [3GPP32.425] 3GPP TS 32.425, V8.1.0 (2009-06); Telecommunication management; PM Performance Measurements, E-UTRAN (Release 8) 16. [3GPP32.426] 3GPP TS 32.426, V8.0.0 (2009-03); Telecommunication management; Performance Management (PM); Performance measurements, Evolved Packet Core network (EPC) 17. [3GPP32.450] 3GPP TS 32.450, V8.1.0 (2009-06); Telecommunication management; Key Performance Indicators for E-UTRAN: Definitions (Release 8) 18. [3GPP32.451] 3GPP TS 32.451, V8.0.0 (2009-03); Telecommunication management; Key Performance Indicators for E-UTRAN: Requirements (Release 8) 19. [3GPP32.814] 3GPP TS 32.814, V7.0.0 (2007-03); Telecommunication management; UTRAN and GERAN Key Performance Indicators (KPI), (Release 7) 20. [3GPP36.300] TS 36.300; v 8.x.x; Evolved Universal Terrestrial Radio Access (EUTRA) and Evolved Universal Terrestrial Radio Access Network (E-UTRAN); Overall description; Stage 2 21. [3GPP36.306] TS 36.306; V8.3.0 (2009-03); Evolved Universal Terrestrial Radio Access (E-UTRA); User Equipment (UE) Radio Access Capabilities 22. [3GPP36.211] TS 36.211; V8.7.0 (2009-05); Evolved Universal Terrestrial Radio Access (E-UTRA); Physical Channels and Modulation

78

Id:0900d805807a32b8


Nokia Siemens LTE E2E Field Network Performance

Recommend Documents