Request for Proposal
Network Benchmarking Services
CONFIDENTIAL
Page 1 of 12
2 AIRTEL’s Network Benchmarking RFP Scope Scope of Service To define ‘End to End ‘requirement for conducting Competitive Network Benchmarking and Report generation for 36 Towns of Airtel. Detailed Methodology: Airtel aims to get the 36 Towns benchmarked once in a Quarter. The lists of 36 towns are:
CONFIDENTIAL
SI No.
List of 36 Towns
1
Durgapur
2
Bhuwneshwar
3
Patna
4
Ranchi
5
Guwahati
6
Aizwal
7
Nagpur
8
Nasik
9
Surat
10
Baroda
11
Indore
12
Bhopal
13
Jodhpur
14
Udaipur
Page 2 of 12
15
Kota
16
Mysore
17
Hubli
18
Manglore
19
Trichy
20
Madurai
21
Pondy
22
Trivendrum
23
Vijayawada
24
Vizag
25
Warangal
26
Agra
27
Dehradun
28
Meerut
29
Kanpur
30
Allahabad
31
Amritsar
32
Jalandhar
33
Patiala
34
Shimla
35
Srinagar
36
Jammu
The Benchmarking partner will have to cover both Voice and Data Services for 2G and 2.5G. Also, if Airtel launches any new technology (like: 3G, 3.5G etc) in these town then Benchmarking Partner should be able to capture the
CONFIDENTIAL
Page 3 of 12
data as per the Airtel requirements. The drive test tool should be able to capture the above requirements.
2.1
Benchmarking Spread
The competitive Network Benchmarking should cover minimum 04 GSM operators and 02 CDMA operators. As per the requirement of the Airtel the operator (GSM as well as CDMA) can be increased. The list of operators to be benchmarked in each city will be given by Bharti. 2.2
TOOL Requirement
The Competitive Network Benchmarking Drive Test tool should be state of the art tool with minimum 12 Mobile handsets measurement capability. The 12 handsets are needed to do the Voice and the Data measurements simultaneously in any given city. Benchmarking Drive test tool should be able to test the EVDO services of at-least 2 CDMA operators along with other tests. Benchmarking Drive test tool should be able to test the latest technology available in the market for all the operators. Also the tool should be updated time to time with the new technologies introduced by the operators and Airtel will be able to get the comparable data. Apart from the 12 Handsets, the Receive level information of all the operators to be taken with the help of scanner integrated as part of the Benchmarking tool. The tools should be able to give the Voice Quality in MOS values using PESQ or SQUAD LQ. If the algorithm is changed by ITU from PESQ and new ITU standards introduced in the market than The Partner should upgrade the tool accordingly. The tool should be able to perform the entire data test including emails, WAP test, FTP, HTTP, PDP, GPRS attach/detach, SMS, RTT and MMS tests. The tool should be able to perform all the Voice and Data tests with Blackberry handsets. The Benchmarking Partner needs to arrange their own FTP servers for all data tests. The Landlines required for measuring individual operator performance if need be can be provided by Airtel or Partner has the flexibility to do this from his own. The partner while submitting the proposal should clearly mention the scope of Airtel and the partners for each of the activity.
CONFIDENTIAL
Page 4 of 12
2.3
SIM CARDS
The Partner is responsible to procure all the operators SIM/EVDO cards. The Partner has the flexibility of giving quote for either arranging the AIRTEL Network SIM cards of its own or telling Airtel to procure and give. The final price quote can be given with and without the AIRTEL SIM Card options. Airtel will decide that which SIM cards with which scheme (Pre paid or Post paid) needs to be used for the drive. The Partner needs to submit all the options to Airtel with all possible schemes available in the market for all the operators.
2.4
Drive Test Routes
The Drive Test routes will be given by the Airtel team at each city. The minimum drive test area to be covered in any town out of 36 Towns should be 600 Kms. The kilometers to be driven in each of the city in accordance to the range given above will be defined by Airtel. The Drive Test routes will cover Dense Urban, Urban, Rural, VIP areas / routes, important roads / express way etc. The Drive Test routes will be divided in to different clusters / region in order for the partner to report the KPIs cluster / region wise and also as a whole city. The same set of tests (used in benchmarking drive) needs to be done from Airport (International & Domestic) to City center and vice versa also with different city’s roaming SIMs of Airtel. Airtel will guide to the Benchmarking Partner that which Circles Roaming SIMs need to be used during this testing. 2.5
In-building Tests
Since there are important commercial complexes, Hotels, Shopping mall etc, depending upon the cities, minimum 25 to 75 buildings to be visited and benchmarked. In each building individual floor to be walk tested and results to be reported floor wise and building wise. The list of building will be given by the Airtel Team. The partner to arrange for all permissions to do the walk test and Airtel will fully support for taking permissions. The pricing of this In-building test to be quoted separately. 2.6
Measurement Period
CONFIDENTIAL
Page 5 of 12
The Drive test measurement period should be from 9 am to 7pm. If the Network Busy Hour of the city extends till 9 pm and then the Drive test team will be allowed to drive till 9 pm. On a city to city basis, the Drive time to be agreed with the Bharti Team. If any Operator’s network test behaviour is abnormal then it is the Benchmarking Partner responsibility to check the reason for the same. If that is because of temporary network issues (Like: outage of sites, etc) than drive will not be valid and The Partner needs to do drive again of that route. 2.7
Measurement Test
Airtel requires a comprehensive list of measurements to be provided. The entire town will be divided in to different clusters as mentioned in the table in section 2.7.2. For each cluster and at town level the KPI Test results to be furnished. The detailed list of test and the KPIs are mentioned in the following sections. 2.7.1
Voice test:
The voice test should be based on Long calls and short calls. The long calls to be used for obtaining Voice Quality (MOS) and the short calls to determine Voice Accessibility. In a normal drive test mode, 2 mts to 3 mts call sequence to be used to determine the Voice KPIs along the drive test route. If the call drops or failed then with a minimum pause immediately the call to be initiated. At strategic locations where the call volumes are expected to be huge, short calls to be made to determine how well the accessibility of the network where the customer demand is maximum.
The list of Voice test to be reported cluster wise is given below: 1. 2. 3. 4. 5.
Call Setup time Call Setup Success Rate Call Completion Rate (Drop Call Rate) Voice Quality in MOS (Uplink and Downlink separately) RX Level Sample distribution in 04 levels: RX level samples >-70dbm (GSM 1800 band) and >-75dbm (GSM 900 band) RX level samples <= -70dbm >-85dbm (GSM 1800 band) and <= 75dbm >-85dbm RX level samples <=-85dbm >-92dbm <=-92dbm Any time as per the requirement new Voice test can be added by AIRTEL
CONFIDENTIAL
Page 6 of 12
2.7.2
Data Test
The following data test to be done continually in sequential manner (One after another): 1. Attach 2. PDP Activation 3. FTP Upload (100 kbps file) 4. FTP Download (500 Kbps file) 5. HTTP Browsing 6. EMAIL Send and Receive Apart from the above tests during the drive, at strategic locations across the city, additional test to be carried out: 1. WAP Test (Operator Home Page download) 2. WAP Test (Downloading picture or ringtone etc from the operator home page) 3. EMAIL send and receive 4. FTP Download (1 Mb file) 5. SMS 6. MMS 7. RTT (for all AIRTEL APN Servers) Minimum 5 Strategic locations need to be captured in each town and the location will be provided by AIRTEL. Any time as per the requirement new Data test can be added by AIRTEL
Based on the drive and static test, the final report will be generated which will have all the following KPIs: 1. Attach - Attach Time - Success Rate 2. PDP Activation - Activation Time - Success Rate 3. FTP UP link Test - Average Throughput speed - Success Rate 4. FTP Downlink Test - Average Throughput speed - Success Rate 5. HTTP Browsing Test
CONFIDENTIAL
Page 7 of 12
- Average Throughput speed - Success Rate 6. Email Send and Receive (with and without blackberry handsets) - Average time to send - Average time to receive - Success rate for sending and receiving 7. WAP Browsing (Operator Home Page) - Average time taken to download the page - Success Rate 8. SMS - Average Time taken to send - Average time taken to receive - Success Rate 9. MMS - Average Time taken to send - Average time taken to receive - Success Rate 10. RTT (For all AIRTEL APN Servers) - Minimum RTT - Average RTT - Maximum RTT - Ping Success Rate
2.8
Benchmarks
For each of the mentioned Voice and Data Test, Domestic and International benchmarks to be provided. The international benchmarks to be based upon the Drive Test conducted in various international networks. Minimum sample size should be 50 operators world wide. Similar to the International benchmarks, every test to be benchmarked with Domestic Benchmarks. The Domestic benchmarks can be based on the current drive test with all the competitors value plotted. In addition to the individual test Benchmarks, the partner should provide an overall indicator for Voice and Data separately and benchmarked with International and Domestic operators. 2.9
Final Benchmarking Report
The final Benchmarking report will have broadly the following sections: 1. Executive Summary - The overall snap shot of the town in Voice and Data Test
CONFIDENTIAL
Page 8 of 12
- Detailed description of the overall assessment of the Network Quality viz viz with the competitors - Clearly mention whether the Network Quality degradation was observed across the city or only in specific pockets. 2. Business Strengths and Weaknesses - Airtel stand over the competition - Airtel Strengths and weakness over the competition - Reasons for weaknesses - Improvement recommendations - Spectrum efficiency 3. Benchmarking methodology - Detailed explanation on the Benchmarking methodology - Type of equipment used - Type of test conducted, volumes of calls made and tested etc 4. Routing details - Route definitions - Plots indicating clearly the route drive tested - Total kilometers driven cluster wise and at town level 5. KPI Benchmarks definitions and representation - International Benchmarks (Best, Median, Worst) - Domestic Benchmarks ((Best, Median, Worst)= - Individual KPI Definitions and detailed measurement methodology 6. Voice and Data KPIs results town level and cluster level The city divided in to many clusters. The KPI / Test to be presented at City and individual cluster / Zone level. - Detailed breakup of each KPI at town and cluster level with all domestic (All competitive networks) and international benchmarks. - One consolidated cluster level report putting together all clusters for each of Voice and Data KPI. In the case of Data Test, the strategic locations test results to be published in terms of individual location level and overall along with other Drive test. 7. In-building Test Results
CONFIDENTIAL
Page 9 of 12
Voice and Data test results per building to be submitted. The report should have floor wise Network Quality KPIs (Preferably all the floors of the building shown separately in one single table with all Voice and Data KPIs) and then consolidated at the Building level.
8.
RCA for poor performance and further improvement Recommendations Every KPI to be analyzed at Cluster and Town level and the reasons for the poor performance of the KPIs, to be mentioned very clearly. e.g. a) Individual drop call reasons with serving cell id to be mentioned clearly in the backup sheet b) If the call accessibility failed then mention clearly for each of the call the reasons for not able to access the network c) If the FTP Download failed then mention clearly the reasons for the failed download. d) Layer 3 message analysis is also required for all the KPI degradations. e) If there is any difference observed in the drive results of the kit (used at the time of testing) and the other kits results than vendor is responsible to do the detailed audit and submit the audit report to Airtel with proper justification. f) Detailed cell level analysis/comparison (for all the operators) is also required to identify that how many cells are covering the derived route and where new site requirement or optimization in existing sites are required to improve the network quality. g) As per the guidelines from Airtel, Benchmarking tracker needs to be circulated by the benchmarking Partner on daily basis to the concern/Authorized persons of Airtel. Airtel will provide the list of email Ids of the authorized persons.
9. Charts / Graphs and Tables (Cluster and Town Level) Every Test results to be mentioned / displayed against the local competitors and the international benchmarks in terms of charts, graphs and tables. Rx lev and MOS plots with details of calls failed and dropped to be clearly mentioned The 04 levels of RX lev samples mentioned in the Voice Test in Sec.2.7.1 to be clearly mentioned in all the cluster and town level plots. Spectrum efficiency calculations cluster wise and Town wise. Cluster level and Town level Cell density plots.
CONFIDENTIAL
Page 10 of 12
Separate MCQI (Messaging connection quality index) to be including in the benchmarking report with the weightage of SMS (80%) & MMS (20%). The weightage can be changed by AIRTEL as per the requirement. Apart from the detailed charts and table, the GPRS and EDGE Data Usage to be clearly given in terms of % of samples during the Drive test. The GPRS and EDGE codec usage to be given in terms of graphs / charts for individual cluster level and Town level along with detailed observations and recommendations.
10.
Consolidated Report: One Consolidated report of all the 36 Towns to be provided for every circle comparing all the Towns. This should give a snapshot of Airtel Towns standing against each other (In all the KPIs / Test) against its own and the competition. After finalization of the report’s structure, it can be changed by AIRTEL as per the new requirements. All the towns should be derived in breakup of clusters and after finishing of every cluster the benchmarking partner should be responsible to submit the detail report of the derived cluster with in 2 days.
11. KPI Trend per cluster and Town The Benchmarking is planned every once in a Quarter for all the 36 Towns. From 2nd report onwards, KPI trending should be given in the form of Graphs / Plots to see how the KPI has performed from the last submitted report. This should continue for all the subsequent reports. From 3rd report onwards, 1st Benchmarking report will become the baseline and other will be used for trending.
2.10
Log file of Benchmarking
The log file generated during Drive Test to be submitted to Bharti along with details of all the Static locations measurements and In-building measurements.
2.11
Pricing
The Pricing to be given for doing the Benchmarking 04 times in a year as well as twice a year in all the 36 Towns. The Pricing of the Drive Test and the Indoor Measurements to be quoted in 02 separate sections namely: 1.
Drive Test + Indoor measurements cost
CONFIDENTIAL
Page 11 of 12
2.
2.12
Drive Test cost and Indoor Measurement cost separately
Penalty for Non Compliance and not meeting the Timelines The contract is for completing all the 36 Towns every Quarter and report submission to be done within 05 days of completion of each of the city. For Non Compliance on the Timelines, the partner will pay the following penalties: a. Benchmarking start time delayed by 03 to 05 days i. 5% of the Total contract value b. Benchmarking start time delayed by >05 <09 days i. 10% of the Total contract value c. Benchmarking start time delayed by >09 days - 15% of the Total contract value d. Benchmarking start time delayed by >15 days - 20% of the Total contract value The same is applicable for delayed reports also. If there is any problem (like: any data or voice related services are deactivated in any operator) occurred during the drive test, than also all the penalty clause will be applicable. After completion of drive, if it is found that the report submitted by the benchmarking partner is not giving the right picture than as a penalty the benchmarking re-drive needs to be done by the benchmarking Partner. 2.13 RFP Response A sample benchmarking report to be submitted with all the mentioned requirements while giving a response to the proposal. Detailed activity list with clear responsibilities and timelines to be mentioned. Detailed methodology with project charter to be presented on how the Benchmarking will be done covering all the 36 Towns every Quarter. Capabilities and references on carrying out this kind of comprehensive benchmarking.
CONFIDENTIAL
Page 12 of 12
