GSM Radio Network Performance Evaluation Delivery Report (Template)
For internal use only
Huawei Technologies Co., Ltd. All Rights Reserved 2012-7-21
Revision Record Date
Version Description
2010-9-28
2.2
2012-07-21
Initial
HUAWEI Confidential
Review
Author
Lishucheng
2010-9-28
Page2, Total25
For internal use only
Content 1 Introduction...................................................................7 2 Workflow.......................................................................8 3 Evaluation information collection....................................9 1.1 Complaint Information Collection.................................................................................. ........ ..........9 ..9 1.2 Statistic data Collection........................................................................ Collection......................................................................................................... ................................. ..10
4 Subscriber complaint ...................................................11 1.3 Various complaints percentages...............................................................................................11 percentages...............................................................................................11
5 DT preparation.............................................................12 1.4 Specific drive test measurement tools................................................................................... ..12 1.5 External Antenna and Attenuator Requirement.......................................................................12 Requirement.......................................................................12 1.6 Cluster definition........................................................................ definition.................................................................................................................. .......................................... ...13 1.7 Route definition.......................................................................... definition.................................................................................................................... .......................................... ...15 1.8 Test Methodology....................................................................................................................16
6 Drive Test.....................................................................17 7 DT data procession.......................................................18 1.9 Coverage KPI......................................... KPI................................................................................................................ ....................................................................... ........ ..........18 ..18 1.9.1 Rx-lev log file........................................................................................................... file.................................................................................................................... .........18 18 1.9.2 Rx-lev Distribution.................................. Distribution............................................................................................................18 ..........................................................................18 1.9.3 Rx-lev Coverage...................................... Coverage................................................................................................................18 ..........................................................................18 1.9.4 Rx-Qual log file.................................................................................................................19 file.................................................................................................................19 1.9.5 Rx-Qual Distribution............................................................ Distribution...................................................................................................... .......................................... ...19 1.9.6 Rx-Qual Coverage................................................. Coverage.......................................................................................................... ......................................................... ...19 1.10 Accessibility KPI...................................................................................................................19 1.10.1 Call Setup Success Rate Rate of MS Origination log file.............................................. ......... .........19 19 1.10.2 Origination Call Call Setup Success Success Rate...............................................................................20 Rate...............................................................................20 1.10.3 Call Setup Success Rate Rate log file......................................................................................20 file......................................................................................20
2012-07-21
HUAWEI Confidential
Page3, Total25
For internal use only
1.10.4 Call Setup Setup Success Success Rate ........................................................................................ ......... .........20 20 1.11 Retainability KPI.............................. KPI.............................................................................................................. ................................................................................ .....21 1.11.1 Call-Drop Rate log log file............................................................................................. file............................................................................................. .......21 1.11.2 Call-Drop Rate.......................................................................... Rate.................................................................................................................21 .......................................21 1.12 Handover...............................................................................................................................24 1.12.1 Handover Statistics log file.................................................................. file............................................................................ .................. ................. .........24 24 1.12.2 Handover Statistics................................................... Statistics.......................................................................................................... ....................................................... 24
8 Summary......................................................................24
2012-07-21
HUAWEI Confidential
Page4, Total25
For internal use only
Figures Workflow of Network Evaluation........................................9 Complaints percentage....................................................11 Top view for install Ext ANT..............................................13 Cluster border ................................................................14 Test Route...................................................................... Route.......................................... .............................16 .16 Rx-lev Coverage..............................................................18 Rx-Qual Coverage............................................................19 Drop Call Location-1.......................................................23 Drop Call Reason-2.........................................................24
2012-07-21
HUAWEI Confidential
Page5, Total25
For internal use only
Tables Timelines of Performance Evaluation..................................7 Complaint Information.......................................................9 Pre-swap statistic data....................................................10 Cluster sites list..............................................................14 Origination Call Setup Success Rate.................................20 Call Setup Success Rate..................................................20 Call Drop Rate.................................................................21 Handover Statistics.........................................................24 Drive Test KPI summary...................................................24
2012-07-21
HUAWEI Confidential
Page6, Total25
For internal use only
1
Introduction
Before a radio network swapping, a well evaluation of the original network should be carried carried out. The The evaluation evaluation result will will be the benchmark benchmark of the the swapped swapped network. network. This report will discuss about methodology of evaluation and give an example to represent real status of the network. Evaluation will be done in 3 aspects as follows:
‐ Complain status evaluation ‐ Traffic Statistic evaluation ‐ Drive Test evaluation For the previous 2 tasks the main work are collect information from operator and classify them into tables that can be reference after the swapping. Thus the key task in this report is the last one “Drive Test evaluation”, it will be discuss in detail. Scope of This Report:
‐
Evaluation information collection
‐ DT preparation ‐ Drive Test ‐ DT data procession ‐ Summary Performance Evaluation Schedule: Table 1.1 Timelines of Performance Evaluation No.
Items
1
Up date Ra Rad io En Eng in eering Pa Paramet er
0.2 week
2
Collect Co mplain t Info rmat ion
0.4 week
3
Coll ollect ect Per Perfform ormance ance Stat Statiistic stic bef befor oree Sw Swap
0.4 week eek
4
Rev iew Exis tin g Sit es Co o rdin at es
0.4 week
5
Drive Tes t
7 week
6
DT Data Pos t Pro ces s
6 week
7
Gen erat e Ev alu at io n Repo rt
0.6we ek
2012-07-21
weekl y progres progresss
Duration
HUAWEI Confidential
1st 2nd 2nd 3rd 3rd 4th 4th 5th 5th 6th 6th 7th 7th 8th 8th
Page7, Total25
For internal use only
2
Workflow
The workflow of Performance Evaluation as below: Existing Radio Network Network Informati Information on
Legend START
Data-In-001
Huawei 001
Operator
Collect Existing Radio Network Information
Huawei & Operator
Doc-Out-001 002
NOC Review Existing Sites Coordinates
OS
003
Drive Test
OS/ RD
004
DT Data Data Post Process Process
005
Generate Generate Evaluati Evaluation on Report
Doc-Out-002
OS
Out Sourcing
RD
Remote Delivery
END
2012-07-21
HUAWEI Confidential
Page8, Total25
For internal use only
Figure 1.1 Workflow of Network Evaluation
3
Evaluation information collection
Before Before commenci commencing ng swap swap proces process, s, there there is a group group of informa information tion need to be collect collected ed from from the origina originall radio radio networ network. k. These These informa information tion will will become become the baseline baseline for evaluate evaluate the swapped swapped network network performanc performancee in the future. future.
1.1 Complaint Information Collection Complai Complaint nt inform informatio ation n can be get from the operato operatorr call call centre, centre, it record record the compla complaint int history history informa information tion befor beforee the swap. swap. This This informa information tion can indica indicate te existent problems in the original network that can be the reference for swapped network. It used to evaluate the feeling of subscriber in the network before and after the swap. It should include recent subscriber’s cases which are still open.(problems, areas, analysis and follow up action items) The recent period means during last 2 weeks or 1 month before the swap. If swap is delayed and overdue past 1 month, this information needed to re-collect. Complaint information showing as the table below: Table 1.1 Complaint Information NO
Problem
Problem
Problem
Related
Related
Detail
Area
time
RNC ID
CELL ID
Related
Related
Trouble
Analysis
Solution
REMAR
SITE
CLUSTER
shooting
conclusion
& result
K
Missing
Adding
neighbor
neighb
NAME
1
Call
xx
hh:mm,
drop/Ca
buildin
dd-
INTHRA
ll setup setup
g,
mm-
EXPRESS
xx
2012-07-21
10
1001
RAM
data
1
HUAWEI Confidential
DT data & Trace Trace
or
Page9, Total25
For internal use only
fail/Low
street,
yyy y
WAY
c e ll s
cells,
through
xx
the
put
district,
proble
xx
m
city/xx
disapp
cross
eared.
road, xx district, xx city
This This table table need need keep keep same same with with the the “Com “Compla plain intt Info Inform rmat ation ion”” shee sheett of the < GRFSDTXXXXX-Radio GRFSDTXXXXX-Radio Network complaint Information>
1.2 Statistic data Collection Pre-swap statistic data need to be collected as the benchmark criterion for swapped network. The statistic data should include all the items that negotiated in KPI Mapping phase. The data should be collected in the last 2 weeks or 1 month before the swap. If swap is delay and overdue past 1 month, this data need re-collect to make sure the statistic will represent real time performance of the original network. Statistic data for example as the table below: Pre-swap statistic data Table 1.2 Pre-swap 2010 2010-1 -100-1 11.
Drop
Call
2010010-10 10-1 -12 2
2010 2010-1 -100-13 13
20 2010-1 10-10 0-14 -14
2010 2010-1 -10 0-15 -15
2010 2010-1 -100-16 16
2010 2010-1 -100-1 17
1%
Rate
Call
Setup
97%
Success Rate
Soft
Hand
98%
Over Over Succ Success ess Rate
2012-07-21
HUAWEI Confidential
Page10, Total25
For internal use only
The KPI items here are just for example, the exact ones need negotiated with operator in the KPI Mapping phase.
4
Subscriber complaint
1.3 Various complaints percentages
Figure 1.1 Complaints percentage
2012-07-21
HUAWEI Confidential
11, Total25 Page11,
For internal use only
5
DT preparation
1.4 Specific drive test measurement tools
Probe V2R3 for voice and data tests
External Antenna 0dB gain with cable and connector
GPS with external antenna
Laptop loaded with Probe V2R3 or TEMS
2 Mobiles and 1 data card
In case of some KPI acceptance items are special in other vendor’s test and analyze tool, validate whether them can be realized in Huawei’s tool also. If not TEMS/Pilot Fleet Fleet will will be reco recomm mmend ended ed or need need choo choose se other other tool, tool, that that mean meanss DT tool tool purchasing purchasing flow flow need to be started. started. Pay Pay attention attention the operation operation guide of of the test tools tools from other vendor should be considered also. Setup: MS: CS Calls using a National National Roaming disabled SIM. MS is set as GSM mode only and locked on operator network (2G->3G cell reselection is not allowed) Data Card for PS, SIM card special requirement:
Using a National Roaming disabled SIM and locked on operator GSM
Support EGPRS
SIM cards provided and prepared by operator
FTP Download Server provided by operator
CS Calls Answering Machine provided by operator
Vehicle
DC to AC converter
Requirement 1.5 External Antenna and Attenuator Requirement The external antenna of MS will be used in Drive Test, and the antenna will be mounted on rooftop of drive test car. The following figure shows the external antenna installation. It is necessary that 1 Lambda displacement between Antenna and the roof edge. 2012-07-21
HUAWEI Confidential
Page12, Total25
For internal use only
Roof luggage rack is not allowed.
G
S
P
A
N
TM
+
S +
A
N
T
+
V
G
S
P
A
A
M N
N
TS
A
N
T
+
Figure 1.1 Top view for install Ext ANT
1.6 Cluster definition The clusters shall be defined based on the principles determined both huawei and custome customerr. Princi Principle pless please please refer refer to section 2.2. There is a cluster definition example below:
2012-07-21
HUAWEI Confidential
Page13, Total25
For internal use only
Figure 1.2 Cluster border
List of cluster sites: Table 2.1 Cluster sites list SiteID
NodeB Name
Cell Name
10350
IESSW08
IESSX
1
10357
25
10350
IESSW08
IESSY
1
10358
06
10350
IESSW08
IESSZ
1
10359
57
10360
MALLSURW08
MALLSURX
1
10367
18
10360
MALLSURW08
MALLSURY
1
10368
09
10360
MALLSURW08
MALLSURZ
1
10369
20
10390
SAIBAW08
SAIBAX
1
10397
38
10390
SAIBAW08
SAIBAY
1
10398
3
10390
SAIBAW08
SAIBAZ
1
10399
17
11070
CGUEVARAW08
CGUEVARAX
1
11077
60
11070
CGUEVARAW08
CGUEVARAY
1
11078
1
11070
CGUEVARAW08
CGUEVARAZ
1
11079
52
2012-07-21
HUAWEI Confidential
Cluster
Cell ID
Frequenc y
Page14, Total25
For internal use only
Total number of sites for the cluster is: 10 Percentage Sites Radiating: 100%
1.7 Route definition The route will be defined by Huawei and sent to customer for confirmation. If customer agree on it, Huawei will use it for the cluster route otherwise changes will be made. made. All sectors in every site in the cluster should be covered by the drive route. Routes Routes should should cover cover Busines Businesss center centers, s, shoppin shopping g center centers, s, tourist tourist attract attraction ionss and railway stations. Major roads: Motorways in both directions. The routes should be laid out to gain a clear footprint of each cell. Routes should avoid roads with restricted access, pedestrian areas or secure areas or those requiring special access conditions. Routes should be captured in MapInfo format or something similar, i.e. Google Earth. At least 200 samples for PS and CS have to be collected. Below is a sample of test route.
2012-07-21
HUAWEI Confidential
Page15, Total25
For internal use only
Figure 1.3 Test Route
1.8 Test Methodology Cluster drive tests shall perform 2 call scenarios: Voice short calls, VP short calls with 120 sec duration with 10 sec interval. PS functionality tests were performed by downloading a huge file from an FTP server with CQT (Call Quality Test).
2012-07-21
HUAWEI Confidential
Page16, Total25
For internal use only
6
Drive Test
During the DT a Protocol-File should be written which captures all the problems like broken cable, loose cable, mobile error and any other equipment error, unusual behaviour behaviour or lunch break. break. The protocol shall as well capture the DT-engineers name and vehicle number. Pre and Post DT shall be done using the same vehicle and measurement equipment. The DT-En DT-Engine gineer er has to check check the stabili stability ty and functio functional nality ity of the equipm equipment ent during the DT. In case of major problems during the DTs the DT Engineers should contact Huawei technical assistance personnel. It is important to check if the right network is measured. Information: Tests will be done by using National Roaming disabled SIM-Cards. The Log-Files should not be too big. A proper proper splitting of files length is necessary necessary.. Recommend every log file less than 20M.
2012-07-21
HUAWEI Confidential
Page17, Total25
For internal use only
7
DT data procession
1.9 Coverage KPI 1.9.1 Rx-lev log file PSDL_070620_Toapayoh_1.gen
1.9.2 Rx-lev Distribution
Figure 1.1 Rx-lev Coverage
1.9.3 Rx-lev Coverage Rx-lev (>=-95dBm) Ratio = 99.24% 2012-07-21
HUAWEI Confidential
Page18, Total25
For internal use only
RSCP (>=-85dBm) Ratio = 96.17%
1.9.4 Rx-Qual log file PSDL_070620_Toapayoh_1.gen
1.9.5 Rx-Qual Distribution
Figure 1.2 Rx-Qual Coverage
1.9.6 Rx-Qual Coverage Rx-Qual (>=4dB) Ratio = 96.54%
1.10 Accessibility KPI 1.10.1 Call Setup Success Rate of MS Origination log file PSDL_070620_Toapayoh_1.gen
2012-07-21
HUAWEI Confidential
Page19, Total25
For internal use only
1.10.2 Origination Call Setup Success Rate Table 2.1 Origination Call Setup Success Rate
Call Statistics
Result
Rate
Total Number of Calls
26
100.0%
Total Drops
2
7.7%
Total Complet ed Calls
23
88.5%
Total Call Setup Success
26
100.0%
Total Call Setup Failure
0
0.0%
Total Number of Outgoing Call
26
100.0%
Total Drops of Outgoing Call
2
7.7%
Outgoing Call Setup Success
26
100.0%
0
0.0%
Outgoing Call Setup Failure
Origination Call Setup Success Rate = Total Call Setup Success / Total Number of Calls = 26 / 26 * 100% = 100.0%
1.10.3 Call Setup Success Rate log file PSUL_070620_Toapayoh_1.gen
1.10.4 Call Setup Success Rate Table 2.2 Call Setup Success Rate
Call Statistics
Result
Rate
Total Number of Calls
23
100.0%
Total Drops
0
0.0%
Total Complet ed Calls
22
95.7%
2012-07-21
HUAWEI Confidential
Page20, Total25
For internal use only
Total Call Setup Success
23
100.0%
Total Call Setup Failure
0
0.0%
Total Number of Outgoing Call
23
100.0%
Total Drops of Outgoing Call
0
0.0%
Outgoing Call Setup Success
23
100.0%
0
0.0%
Outgoing Call Setup Failure
Call Setup Success Rate of VP = Total Call Setup Success / Total Number of Calls = 23 / 23 * 100% = 100.00%
1.11 Retainability KPI 1.11.1 Call-Drop Rate log file PSDL_070620_Toapayoh_1.gen
1.11.2 Call-Drop Rate Table 2.3 Call Drop Rate
Call Statistics
Result
Rate
Total Number of Calls
26
100.0%
Total Drops
2
7.7%
Total Complet ed Calls
23
88.5%
Total Call Setup Success
26
100.0%
Total Call Setup Failure
0
0.0%
Total Number of Outgoing Call
26
100.0%
Total Drops of Outgoing Call
2
7.7%
2012-07-21
HUAWEI Confidential
Page21, Total25
For internal use only
Outgoing Call Setup Success Outgoing Call Setup Failure
26
100.0%
0
0.0%
‐
One drop call was not only caused by the normal release from DL_DCCH but also traffic time was 120sec should be excluded from the KPI
‐
The other drop call was caused by the Abis interface problems between BSC 501 and BSC 403.
Origination Call Drop Rate = Total Drops/ Total Number of Calls = 0 / 26 * 100% = 0.00 %
2012-07-21
HUAWEI Confidential
Page22, Total25
For internal use only
Figure 1.3 Drop Call Location-1
2012-07-21
HUAWEI Confidential
Page23, Total25
For internal use only
Figure 1.4 Drop Call Reason-2
1.12 Handover 1.12.1 Handover Statistics log file PSDL_070620_Toapayoh_1.gen
1.12.2 Handover Statistics Table 4.1 Handover Statistics Handover Statistics
Total
Total Handover Performed
490
Total Handover Success
489
Total Handover Failure Handover Success Rate
1
99.80%
8
Summary
Below is the Drive Test KPI summary: Table 1.1 Drive Test KPI summary KPI
2012-07-21
Original network Test Results
Contractual KPI criteria
HUAWEI Confidential
Acceptance Results
Page24, Total25
For internal use only
Rx-lev
97.62%@-100dBm
>=96%@-95dBm
Rx-Qual
97.52%@-12dB
>=96%@-12dB
Call Setup Success Rate
100.00%
Call Drop Rate HO Success Rate
4.17%
1%
100.00%
KPI: Means Drive Test KPI items agreed by both Huawei and operator. Original network Test Results: Means the original network KPI test values. Contractual KPI criteria: Means Drive Test KPI criteria agreed by both Huawei and operator. Acceptance Results: It will fill with the KPI test values that test from the swapped network. Please note when Acceptance Results of one KPI item achieves any one of the Original network Test Results or Contractual KPI criteria, that KPI item is passed.
2012-07-21
HUAWEI Confidential
Page25, Total25
