13 Dec 2016
LTE SSV REPORT CHECKING GUIDELINE
RF OPTIMIZATION DEPARTMENT MCI – CREATED BY MEHRAN HADDADI (
[email protected] )
1
MCI LTE SSV REPORT CHECKING GUIDELINE VERSION NO. V 1.0 AUTHOR: MEHRAN HADDADI DEC 2016
© Copyright 1999 — 2016 MCI.IR All rights reserved. This document was produced by MCI. The document is to be used for training purposes only and it is strictly prohibited to copy, reproduce, disclose or distribute it in any manner without the express written consent from MCI.
2
TABLE OF CONTENTS 1. INTRODUCTION ............................................................................................................ 4 2. EVALUATION OF SHEETS AND TABLES OF SSV REPORTERROR! BOOKMARK NOT DEFINED.5 3. COVER SHEET ......................................................................................................................... 8 4. SITE INFO SHEET ........................................................................................................... 9 5. PHYSICAL CELL & ATOLL & ALARM SHEET .................................................................... 10 6. DT METHODOLOGY SHEET .......................................................................................... 13 7. LTE DT PLOT ............................................................................................................... 14 8. LTE DT KPI SHEET ........................................................................................................ 15 9. INTRODUCTION ABOUT SSV CHECKING IN ACTIX ANALYZER ........................................ 16 9.1 Binning Mode ..............................................................................................................16 9.2 CellRef Section .............................................................................................................18 10. RSRP ........................................................................................................................ 37 11. SINR ......................................................................................................................... 62
3
12. CSFB WITH RWR ....................................................................................................... 68 13. CSFB CALL SETUP TIME ............................................................................................. 79 14. SMS ......................................................................................................................... 89 15. RRC SETUP SUCCESS RATE ..................................................................................... 103 16. E-RAB SETUP SUCCESS RATE ................................................................................... 114 17. ERAB DROP RATE.................................................................................................... 127 18. LATENCY (RTT IN MS).............................................................................................. 168 19. WEB PAGE LOAD TIME ............................................................................................ 184 20. DL THROUGHPUT (MBPS) ....................................................................................... 200 21. UL THROUGHPUT (MBPS) ....................................................................................... 215 22. INTRA LTE SITE HO .................................................................................................. 228
4
1. INTRODUCTION
Single site verification (SSV) is an audit method, where we need to check the entire KPI (Key performance indicator), coverage and quality for a single radio base station
This document is prepared to provide guidelines and analysis processes for the approval of the SINGLE SITE VERIFICATION reports by NPO for the MCI Network.
The studies, preparations and assumptions made in order to check and approve the SSV reports are explained in details.
5
2. EVALUATION OF SHEETS AND TABLES OF SSV REPORT
For every QA passed LTE Site, an SSV report will be submitted on related FTP server. This report includes 13 sheets and each sheet contains a number of tables. All the contents should be checked by the responsible engineer in detail according to their schedules. The validation process would be to compare the report data with submitted relevant drive test logs.
Sheets
Tables
Evaluation Steps
Cover
Information,Status,Teams
Make sure that general information is correct (Site ID, Site Name, RF Planning Cluster ID, …) Make sure that general information is correct
Site Info (compare it with CDD , EP & IP Plan) Physical Cell & Atoll & Alarm
Physical Cell Information
Make sure that the physical parameters (Azimuth, Electrical Tilt, Mechanical tilt, Antenna height, RET,…) are implemented based on the delivered plans.
6
Cell Information
Check the validity of all items using related log file.
Alarms (VSWR, major alarms)
Check 2 snapshots that shows site has no alarms during ready to SSV & Time of DT.
Antenna Swapping
Check the validity of all items using related log file
Atoll Update
Check Atoll update for each Node. All the conditions for the performed tests have been mentioned here. During the log file analysis, it is very important to check if the conditions met.
DT Methodology
DT Methodology, SRVCC & VoLTE
Except CSFB logs, all log files should be locked on LTE. Otherwise DT should be done again. 2 Snapshots should be delivered in the report to show that SRVCC feature is activated & VoLTE configuration is Done.
LTE DT Plot
RSRP, SINR, PCI, MIMO, Intra LTE site HO Plots should be checked.
LTE DT KPI
Check the validity of all DT KPIs using related log files.
7
LTE OSS KPI
Check the validity of all OSS KPIs in FARAZ.
2G OSS KPI(BTSAntenna Change)
In case of antenna change, 2G OSS KPI should be delivered.
3G OSS KPI(Antenna Change)
In case of antenna change, 3G OSS KPI should be delivered.
4G OSS KPI(Antenna Change)
In case of antenna change, 4G (Band 7 = 2600 MHz) OSS KPI should be delivered.
Calculation attribute in Actix
This sheets guide you through related attribute for each DT KPI.
3G OSS KPI formula (FARAZ)
This sheets guide you through related formula in FARAZ for 3G OSS KPI.
LTE OSS KPI formula (FARAZ)
This sheets guide you through related formula in FARAZ for LTE OSS KPI.
8
Cover Sheet In cover sheet, make sure that general information is corrected based on EP. Also site status & Team tables should be filled.
9
Site Info Sheet Check these items from MCI EP. (Site ID,Site Name,Longtitude,Latitude,Number of cells,Phase) Check these items from MCI CDD. (eNodeB type,BSC Name,RNC Name,MME Name,MIMO) * RS Power (dBm) should be 21.2 for 5 MHz bandwidth
10
Physical Cell & Atoll & Alarm Sheet
11
For opening TEMS LTE Serving Cell window, follow below route: 1- Open log file in TEMS 2- Go to Presentation/LTE/Serving Cell
12
13
DT Methodology Sheet In this sheet, below items should be checked by the responsible engineer: 1-DT methodology should be same as MCI SSV template (V1.6-20161211) 2-UT type should be reported 3-2 snapshots related SRVCC & VoLTE should be reported.
14
LTE DT Plot In this sheet plots should be reported based on below attribute & MS. Otherwise, report should be revised. Plots RSRP SINR PCI MIMO Intra LTE site HO
UE MS1 MS1 MS1 MS2 MS2
Attribute in actix LTE_UE_RSRP LTE_UE_SINR LTE_UE_PCI LTE_UE_SpatialRank LTE_RRCConnection_HO_OK/LTE_RRCConnection_HO_Init
e.g. RSRP plot should be reported like below snapshot: MS1-LTE_UE_RSRP (dBm)
>= -140.00 to < -110.00 (0) 0.0%
>= -110.00 to < -100.00 (0) 0.0%
>= -100.00 to < -90.00 (0) 0.0%
>= -90.00 to < -80.00 (17) 1.2%
>= -80.00 to < -70.00 (484) 33.3%
Above -70.00 (951) 65.5%
15
LTE DT KPI Sheet This sheet is the most important one in SSV report. There are 13 DT KPIs as below table: Except 2 of DT KPIs (CSFB Call Setup Time & Web Page Load Time), other 11 DT KPIs has target. SSV approved when all 11 KPIs passed their target. you can find comprehensive tutorial to how calculate each DT KPIs in next pages. No
DT KPI
Methodology
MS
Technology
Attribute in Actix
1 2 3
RSRP SINR CSFB with RWR CSFB Call Setup Time SMS (MO & MT) RRC Setup success rate E-RAB Setup success rate ERAB Drop Rate Latency (RTT in ms) Web Page Load Time DL Throughp ut (Mbps) UL Throughp ut (Mbps) Intra LTE site HO
DT-Idle DT-Idle DT-Stationary
MS1 MS1 MS1
Lock On LTE Lock On LTE Unlock
LTE_UE_RSRP LTE_UE_SINR LTE_Event_CSFB_CallAttempt/LTE_Event_CSFB_CallSetupOK
DT-Stationary
MS1
Unlock
LTE_CSFB_Call_SetupTime
DT-Stationary
MS1/ MS2 MS1
Lock On LTE
TEMSSMSSent/TEMSSMSReceived
Lock On LTE
LTE_RRCConnectionAttempt/LTE_RRCConnectionSetupOK
DT-Short callStationary
MS1
Lock On LTE
LTE_Event_ActDefaultEPSBearerContextAttempt/LTE_Event_ActDefaultEPSBearerContextC omplete
DT-Short callStationary DT-Stationary
MS1
Lock On LTE
TEMSEvent_NAS_Signalling_Failure
MS1
Lock On LTE
Ping_RTT
DT-Stationary
MS1
Lock On LTE
Task_Duration & Task_Application_Type=1
DT-Stationary
MS1
Lock On LTE
LTE_UE_Throughput_PDCP_DL
DT-Stationary
MS1
Lock On LTE
LTE_UE_Throughput_PDCP_UL
DT-Long Call
MS2
Lock On LTE
LTE_RRCConnection_HO_OK/LTE_RRCConnection_HO_Init. Also check LTE_UE_PCI
4 5 6 7
8 9 10 11 12 13
DT-Short callStationary
16
Introduction about SSV Checking in Actix Analyzer Binning Mode: Binning is a method of grouping and averaging the data loaded from a log file. The data to be binned can be partitioned according to one of several criteria:
Time
Geographic Location
Distance traveled
Message number
SSV agreed binning mode is Message (count :1) Set binning mode as below: Home menu/binnning mode Message (count :1)
17
18
CellRef Section In Analyzer, network configuration information (such as the names, IDs, and locations of the cell sites, etc.) is stored in a text file that is called the CellRefs file. This section provides information about this file and the configuration file, which defines its structure and format. You can upload CellRefs file by following below steps:
Step1 :
19
Step 2:
20
Step 3:
21
Step 4:
22
Step 5:
23
Step 6
24
Step 7
25
Step 8
26
Step 9
27
Step 10
28
Step 11
29
Step 12
30
Step 13
31
Step 14
32
Step 15
33
Step 16
34
Step 17
35
Step 18
36
Step 19
37
RSRP For checking this KPI, you have to follow below step: 123456-
Open Actix analyzer Select related log files After loading files finished, you have to Superstream all logs. First check related MS for coverage & name Superstream log as Coverage. Open LTE_UE_RSRP attribute in Coverage Superstream log. Change the legend based on the SSV target Compare the value with delivered report
38
Step 1
39
Step 2:
40
Step 3:
41
Step 4: "Wait until loading is finished"
42
Step 5:
43
Step 6:
44
Step 7:
45
Step 8:
46
Step 9:
47
Step 10:
48
Step 11:
49
Step 12:
50
Step 13:
51
Step 14:
52
Step 15:
53
Step 16:
54
Step 17:
55
Step 18:
56
Step 19:
57
Step 20:
58
Step 21:
59
Step 22:
60
Step 23:
61
Step 24:
62
SINR For checking this KPI, you have to follow below step: 123456-
Open Actix analyzer select related log files (coverage) after loading files finished, you have to Superstream all logs. First check related MS for coverage & name Superstream log as Coverage. Open LTE_UE_SINR attribute in Coverage Superstream log. Change the legend based on the SSV target Compare the value with delivered report
63
Step 1:
64
Step 2:
65
Step 3:
66
Step 4:
67
Step 5:
68
CSFB with RWR For checking this KPI, you have to follow below step: 12345-
Open Actix analyzer select related log files (refer to the report- LTE DT KPI Sheet/ MS No& Log file name) Minimum 5 attemps should be done in this DT scenario. Less than 5 attempts, Re-Drive is necessary. Open LTE_Event_CSFB_CallAttempt/LTE_Event_CSFB_CallSetupOK attributes in a table. Compare the value with delivered report
69
Step 1:
70
Step 2:
71
Step 3:
72
Step 4:
73
Step 5:
74
Step 6:
75
Step 7:
76
Step 8:
77
Step 9:
78
Step 10:
79
CSFB Call Setup Time For checking this KPI, you have to follow below step: 1234-
Open Actix analyzer select related log files (refer to the report- LTE DT KPI Sheet/ MS No& Log file name) Open LTE_CSFB_Call_SetupTime attributes in a table. Compare the value with delivered report
P.S. there is no need to lock on LTE in this DT scenario.
80
Step 1:
81
Step 2:
82
Step 3:
83
Step 4:
84
Step 5:
85
Step 6:
86
Step 7:
87
Step 8:
88
Step 9:
89
SMS For checking this KPI, you have to follow below step: 123456-
Open Actix analyzer select related log files (refer to the report- LTE DT KPI Sheet/ MS No& Log file name) Check Technology_Mode attribute for MO & MT (UE should be locked on this DT scenario) Minimum 2 attemps should be done in this DT scenario. Less than 2 attempts, Re-Drive is necessary. Open TEMSSMSSent/TEMSSMSReceived attributes in a table. Compare the value with delivered report
90
Step 1:
91
Step 2:
92
Step 3:
93
Step 4:
94
Step 5:
95
Step 6:
96
Step 7:
97
Step 8:
98
Step 9:
99
Step 10:
100
Step 11:
101
Step 12:
102
Step 13:
103
RRC Setup success rate For checking this KPI, you have to follow below step: 12345-
67-
Open Actix analyzer select related log files (refer to the report- LTE DT KPI Sheet/ MS No& Log file name) Check Technology_Mode attribute (UE should be locked on this DT scenario) Minimum 5 attemps should be done in this DT scenario. Less than 5 attempts, Re-Drive is necessary. UE should download 30 second & delay 15 second in this scenario (check throughput to be sure about short call scenario). CP active (Cyclic Prefix) is not accepted. (in CP active, just DT man turn data on & off) Open LTE_RRCConnectionAttempt/LTE_RRCConnectionSetupOK attributes in a table. Compare the value with delivered report
104
Step 1:
105
Step 2:
106
Step 3:
107
Step 4:
108
Step 5:
109
Step 6:
110
Step 7:
111
Step 8:
112
Step 9:
113
Step 10:
114
E-RAB Setup success rate
For checking this KPI, you have to follow below step: 12345-
6-
7-
Open Actix analyzer Select related log files (refer to the report- LTE DT KPI Sheet/ MS No& Log file name) Check Technology_Mode attribute (UE should be locked on this DT scenario) Minimum 5 attemps should be done in this DT scenario. Less than 5 attempts, Re-Drive is necessary. UE should download 30 second & delay 15 second in this scenario. CP active ( Cyclic Prefix) is not accepted. (in CP active, just DT man turn data on & off) Open LTE_Event_ActDefaultEPSBearerContextAttempt/LTE_Event_ActDefaultEPSBearerContextCom plete attributes in a table. Compare the value with delivered report
115
Step 1:
116
Step 2:
117
Step 3:
118
Step 4:
119
Step 5:
120
Step 6:
121
Step 7:
122
Step 8:
123
Step 9:
P.S. for example as you see in above snapshot, number of attempts are 4. It should beRe-drive.
124
Step 10:
125
Step 11:
126
Step 12:
127
ERAB Drop Rate The generic formula for ERAB drop rate is (Number of abnormal ERAB release/Number of ERAB established) *100% We can find the number of ERAB established from counting how many EPS bearer contexts completed successfully. Now for ERAB abnormal release, we have only one related event in TEMS and Actix and they are as follows. Actix attribute: TEMSEvent_NAS_Signalling_Failure/ LTE_Event_ActDefaultEPSBearerContextComplete TEMS events: EPS NAS Signaling Failure /EUTRAN Attach Complete
EPS NAS Signalling Failure:
128
EUTRAN Attach Complete:
For checking this KPI, you have to follow below step: 12345-
67-
Open Actix analyzer select related log files (refer to the report- LTE DT KPI Sheet/ MS No& Log file name) Check Technology_Mode attribute (UE should be locked on this DT scenario) Minimum 5 attemps should be done in this DT scenario. Less than 5 attempts, Re-Drive is necessary. UE should download 30 second & delay 15 second in this scenario. CP active ( Cyclic Prefix) is not accepted. (in CP active, just DT man turn data on & off) Open TEMSEvent_NAS_Signalling_Failure/LTE_Event_ActDefaultEPSBearerContextComplete attributes in a table. Compare the value with delivered report
129
Step 1:
130
Step 2:
131
Step 3:
132
Step 4:
There is no ERAB abnormal release
133
Step 5:
134
Step 6:
135
Step 7:
136
Step 8:
137
Step 9:
138
Step 10:
139
Step 11:
140
Step 12:
141
Step 13:
142
Step 14:
143
Step 15:
144
Step 16:
145
Step 17:
146
Calculation in TEMS investigation for ERAB DROP Rate Step 1:
147
Step 2:
148
Step 3:
149
Step 4:
150
Step 5:
151
Step 6:
152
Step 7:
153
Step 8:
154
Step 9:
155
Step 10:
156
Step 11:
157
Step 12:
158
Step 13:
159
Step 14:
160
Step 15:
161
Step 16:
162
Step 17:
163
Step 18:
164
Step 19:
165
Step 20:
166
Step 21:
167
Step 22:
168
Latency (RTT in ms)
For checking this KPI, you have to follow below step: 123456-
Open Actix analyzer select related log files (refer to the report- LTE DT KPI Sheet/ MS No& Log file name) Check * Ping_Technology attribute (UE should be locked on LTE in this DT scenario) Open Ping_RTT attributes in a table. Compare the mean value with delivered report if ping is more than 50 ms should provide evidence
* Ping_Technology = Gives the used technology during a Ping test.
169
Step 1:
170
Step 2:
171
Step 3:
172
Step 4:
173
Step 5:
174
Step 6:
175
Step 7:
176
Step 8:
177
Step 9:
178
Step 10:
179
Step 11:
180
Step 12:
181
Step 13:
182
Step 14:
183
Step 15:
184
Web Page Load Time
For checking this KPI, you have to follow below step: 12345-
Open Actix analyzer select related log files (refer to the report- LTE DT KPI Sheet/ MS No& Log file name) Check Technology_Mode attribute (UE should be locked on LTE in this DT scenario) Open Task_Duration & Task_Server-URL attributes in a table. (Web Page Load Time should be done both for google.com & mci.ir Compare the mean value with delivered report
185
Step 1:
186
Step 2:
187
Step 3:
188
Step 4:
189
Step 5:
190
Step 6:
191
Step 7:
192
Step 8:
193
Step 9:
194
Step 10:
195
Step 11:
196
Step 12:
197
Step 13:
198
Step 14:
199
Step 15:
200
DL Throughput (Mbps)
For checking this KPI, you have to follow below step: 123456-
Open Actix analyzer select related log files (refer to the report- LTE DT KPI Sheet/ MS No& Log file name) Check Technology_Mode attribute (UE should be locked on LTE in this DT scenario) Open LTE_UE_Throughput_PDCP_DL attributes in a table. Compare the Max value with delivered report per sector Compare average of all 3 sector value with MCI Target. (Average of 3 sectors >= 18 Mbps (Bandwidth: 5M)
201
Step 1:
202
Step 2:
203
Step 3:
204
Step 4:
205
Step 5:
206
Step 6:
207
Step 7:
208
Step 8:
209
Step 9:
210
Step 10:
211
Step 11:
212
Step 12:
213
Step 13:
214
Step 14: Average of 3 sector values should be over 18 Mbps (bandwidth= 5M)
215
UL Throughput (Mbps)
For checking this KPI, you have to follow below step: 12345-
Open Actix analyzer select related log files (refer to the report- LTE DT KPI Sheet/ MS No& Log file name) Check Technology_Mode attribute (UE should be locked on LTE in this DT scenario) Open LTE_UE_Throughput_PDCP_UL attributes in a table. Compare the Max value with delivered report per sector
216
Step 1:
217
Step 2:
218
Step 3:
219
Step 4:
220
Step 5:
221
Step 6:
222
Step 7:
223
Step 8:
224
Step 9:
225
Step 10:
226
Step 11:
227
Step 12:
228
Intra LTE site HO
For checking this KPI, you have to follow below step: 1-
Open Actix analyzer
2-
select related log files (refer to the report- LTE DT KPI Sheet/ MS No& Log file name)
3-
Check Technology_Mode attribute (UE should be locked on LTE in this DT scenario)
4-
Open LTE_RRCConnection_HO_OK/LTE_RRCConnection_HO_Init/LTE_UE_PCI attributes in a table.
5-
Min 1 handover from Cell A to B, B to C, C to A, should be success. Otherwise, it should be Redrive.
.
229
Step 1:
230
Step 2:
231
Step 3:
232
Step 4:
233
Step 5:
234
Step 6:
235
Step 7:
236
Step 8:
237
Step 9:
238
Step 10:
239
Step 11:
240
Step 12:
241
Step 13: in below snapshot. It shows that Intra LTE HO is Ok from PCI 396 to 397.
Intra LTE HO is Ok from PCI 396 to 397
242
Step 14: Check each sector PCI to be sure that Intra LTE HO is correctly done for all sectors
243
Step 15:
