MOP1002
NOKIA WBTS SOFTWARE UPGRADE FROM NETACT No index entries found.
Doc No: MOP1002
Date Issued: 01/12/2015
Version: 2.1
Version Status: ASSIGNED
Category: WBTS
Sub-Category: NOKIA
Copy No:
Comments:
Dylan Crewdson
Page 1 of 29
MOP1002
AMENDMENT HISTORY Version
Date
1A 1B 1.0 1.1
12th October 2011 12th October 2011 13th October 2011 23rd September 2013
Originator / Modified By Dylan Crewdson Dylan Crewdson Dylan Crewdson Dylan Crewdson
2.0
3rd November 2015
Dylan Crewdson
2.1
1st December 2015
Dylan Crewdson
Dylan Crewdson
Reason(s) for Issue / Re-Issue First Draft. Note added to p.11 [section 2.1.3 c)] First published. Various screenshots updated. Activation of Software (section 2.3 l.), failure thresholds for escalation updated and Toronto RNC 4 added. NSN WBTS Software Naming Convention Explained (section 3.1), updated to new WN7.0 standard. End to end MOP overhaul to align with Netact 15.5. All references to ‘NSN’ replaced with ‘Nokia’ as per corporate re-branding. Appendix B (Application Launcher Info) & C (Rollback Procedure) added. Step added to deselect all MOs prior to selecting MOs in section 2.2 & 2.3. This is to prevent accidental MO inclusion from previous tasks.
Page 2 of 29
MOP1002
TABLE OF CONTENTS 1.
INTRODUCTION ..................................................................................................................................... 4 1.1
PURPOSE ....................................................................................................................................... 4
1.2
SCOPE ............................................................................................................................................ 4
1.3
RESPONSIBILITIES .......................................................................................................................... 4
1.4
DOCUMENT VALIDATION INTERVAL ............................................................................................. 4
1.5
DEFINITION OF TERMS .................................................................................................................. 4
2.
UPGRADE OF WBTS SW ........................................................................................................................ 5 2.1
PRE-DOWNLOAD AUDIT................................................................................................................ 5
2.1.1
LOGIN TO ‘SOFTWARE MANAGER’ APPLICATION ON NETACT ............................................. 5
2.1.2
CHECK ACTIVE & PASSIVE SW VERSIONS FOR TARGET WBTSs ............................................. 7
2.1.3
UPLOAD OF WBTS SW VERSION DATA................................................................................ 10
2.2
DOWNLOAD OF SOFTWARE ........................................................................................................ 14
2.3
ACTIVATION OF SOFTWARE ........................................................................................................ 20
APPENDIX .................................................................................................................................................... 27 A.
NOKIA WBTS SOFTWARE NAMING CONVENTION EXPLAINED ....................................................... 27
B.
APPLICATION LAUNCHER INFO ....................................................................................................... 28
C.
ROLLBACK PROCEDURE ................................................................................................................... 29
Dylan Crewdson
Page 3 of 29
MOP1002
1. INTRODUCTION 1.1 PURPOSE This document details the procedure to follow to upgrade the software on Nokia WBTSs from Netact using the Software Manager tool.
1.2 SCOPE Switch teams will periodically be required to implement software upgrades on Nokia WBTSs. Typically this will cover either a cluster of sites during trial phase or the complete market during roll out.
1.3 RESPONSIBILITIES Tier 2 - RAN:
Determine when a software upgrade is required and which software level to load. Ensure the software is uploaded to Netact in advance of the WO being issued to the switch teams. Consult with the switch teams to agree resource availability and schedule a date. Generate the WO with all appropriate documentation and assign to the local switch team. Provide support as needed to ensure successful implementation.
Switch Team:
Implement software upgrade to agreed schedule. Maintain a vigilant watch of the affected market throughout software download and activation to ensure there are no problems as a result of the activity. Inform the NOC before and after each stage of the activity (download & activation will occur at different times).
1.4 DOCUMENT VALIDATION INTERVAL This document shall be considered valid for 12 months after each amendment issue. Where a document has not been amended in the preceding 12 months, it shall be subject to an annual review by the Document Owner.
1.5 DEFINITION OF TERMS NOC – Network Operations Centre CM – Change Management SW – Software MO – Managed Object
Dylan Crewdson
Page 4 of 29
MOP1002
2. UPGRADE OF WBTS SW 2.1 PRE-DOWNLOAD AUDIT 2.1.1 LOGIN TO ‘SOFTWARE MANAGER’ APPLICATION ON NETACT a.) Login to Netact via your web browser using the following address (you may also login via Citrix): https://tor2na8waslb.r01.netact.engint.windmobile.ca/authentication/Login *In case of a certificate error, you should select to proceed to the website anyway. Enter your credentials:
Select ‘O.K’ at the welcome page:
Dylan Crewdson
Page 5 of 29
MOP1002 b.) Open the ‘Configuration’ group of tools.
c.) Open the ‘Software Manager’ tool.
The Software Manager tool will now open.
Dylan Crewdson
Page 6 of 29
MOP1002
2.1.2 CHECK ACTIVE & PASSIVE SW VERSIONS FOR TARGET WBTSs Before starting the download process, it is good practice to check both the sw version currently running in the WBTS (active) as well as the version stored in the inactive bank (passive) . This quick audit will confirm if any of the WBTSs are already running on the desired software or already have it downloaded to the inactive (passive) bank. This will save you the time of downloading it again and / or activating it. a.) From the ‘Software Manager’ tool, you will need to select the ‘MO type’ from the drop down box in the top left hand corner as shown below, then select ‘WBTS’.
This will then display all ‘Parent’ nodes for the WBTSs (these are RNCs which the WBTSs connect to).
Dylan Crewdson
Page 7 of 29
MOP1002 b.) Select the ‘Parent ‘ node (RNC) of the WBTS you wish to audit. After a brief moment, all the WBTSs connected to the selected RNC will populate on the main display.
From left to right, this will display the following information for each WBTS: MO DN – Distinguish Name of managed object. It uniquely identifies the object in network. MO name – The name of managed object used in GUI for user presentation. Active SW ID – Active software on selected managed object. Passive SW ID – Passive software on selected managed object. Release - Managed object release. MO parent – Distinguish Name of parent object. Last software task – The type of the last software task executed on particular managed object. Latest upload – The date of the latest upload executed on particular managed object. Latest backup – The date of the latest backup executed on particular managed object. Once all the WBTSs are shown for the RNC you are working on, you may adjust the width of the columns in the same way as you would on a spreadsheet, this will make it easier to view the information you require. You can also click on each column to sort automatically in sequence by that parameter. For instance, if you wish to sort all the WBTSs into ascending order by MO name, simply click once on ‘MO name’, clicking a second time will change the sequence to descending. Each column is also equipped with a filter to help focus only on the information you need.
Dylan Crewdson
Page 8 of 29
MOP1002 From this table you can easily identify both the active and passive sw versions available in each WBTS. Where no information is available, there can be one of two explanations. 1) The WBTS has no O&M connection to the RNC, this can be easily checked by going to the object browser for the RNC, selecting the WBTS and checking the state of the ‘DCN link status’.
If it is ‘Disabled’, it may be because the WBTS is not yet integrated into the RAN. WBTSs which are not yet integrated on the RNC will not come under the scope of a sw upgrade and can be ignored. 2) The WBTS has been integrated into the RAN since the last upgrade or upload was completed and therefore has yet to be uploaded by the Software Manager (uploads will only occur after a sw activation by the user or an upload by the user, there is no automated upload). If a WBTS is found to contain no information in Active or Passive sw fields of the Software Manager, yet is integrated and on air as detailed in point 2 above, then it is suggested to do an upload, in which case proceed to section 2.1.3. Otherwise, if you have identified the WBTSs with missing information to be either a non integrated or non existent WBTS, you can progress to the download phase detailed in section 2.2.
Dylan Crewdson
Page 9 of 29
MOP1002
2.1.3 UPLOAD OF WBTS SW VERSION DATA An upload of one or more WBTS to collect the sw version data can be done at any time and is not service affecting. The following steps should be followed to accomplish this: a.) Place a check mark in the box beside the WBTS(s) you wish to upload, then select ‘New Task’, a task window will open.
b.) Select ‘Upload’ from the ‘Software Operations’ tree and enter a task name, eg. , then select ‘Next’.
Dylan Crewdson
Page 10 of 29
MOP1002
c.) Select ‘Immediately’, then select ‘Next’.
d.) At the summary page, double check your selected objects, task name, task type and frequency, if all is correct, select ‘Finish’ to start the task (window will close).
Dylan Crewdson
Page 11 of 29
MOP1002 e.) After starting the task, you can then click on the ‘Network Operations’ tab as shown below, where you can monitor the progress of the task.
Each object in the task will also show the status individually in the ‘Last software task’ column:
Dylan Crewdson
Page 12 of 29
MOP1002 f.) Once the upload is complete (can take approx 1-5 mins), you can return to the ‘Network Status’ tab, locate the WBTS(s) and verify the existing Active sw.
End of section.
Dylan Crewdson
Page 13 of 29
MOP1002
2.2 DOWNLOAD OF SOFTWARE Download of the new WBTS software is done in a very similar manner as the upload mentioned in section 2.1.3 (previous section), however the time it will take will of course be longer as it has to transfer the multiple new software files to the WBTS rather than just read the Active version as it does in an upload. Whilst a download can be done at anytime (as it is not service impacting), it must be taken into consideration that the O&M connection will have decreased available bandwidth on the Iub during busy traffic periods. For this reason and for the purpose of risk management, it is suggested that downloads be done during low traffic periods. Whilst this can be accomplished by scheduling a download by automation during the MW, it is preferred to do this during the low traffic periods of regular business hours (9am-11am) where the radio network can be monitored closely in case of a failure. Software should be downloaded to WBTSs in batches to mitigate risk of any complications that may occur. For instance, where a cluster of 15 sites is to be upgraded, it is suggested that we separate these into batches of 5 sites at a time. Where a full market roll out is required, we must increase the sizes of these batches, as a rule of thumb we should divide a market by 5 to get our batch size, however where this gives us batch sizes in excess of 50 (recommended maximum), we will instead do batches of 50 at a time regardless of how many batches we have. For example: Market A: 100 WBTSs / 5 = 20 WBTSs at a time (batch size). Market B: 350 WBTS / 5 = 70WBTSs, as this exceeds our maximum size 50), we will instead download to 50 WBTSs at a time, which will result in 7 batches. Furthermore, it is recommended that the the download batch file size start small and then increase with size to the maximum (say 5, then 10, then 20 etc up to 50 WBTSs). This will mitigate the risk of user error as well as build confidence in the process. a.) Login to the OMS, select the RNC you are working on from the Topology tree on the left side, then select ‘WCELs Overview Page’, this will show all cells on the RNC. Once this view has loaded, select the ‘Blocked WCELs’ to show only those cells which are blocked.
Dylan Crewdson
Page 14 of 29
MOP1002
b.) Make a note of all cells which are blocked, either by using the export option in the OMS, or by a series of screen shots. This will be a reference source for you in the event that any cells go OOS during software download.
c.) Open the ‘Application Launcher’ (installed on your PC), login to the OMS, then select the required RNC and open ‘Fault Management’. Copy and paste all the shown alarms into another sheet in your spreadsheet, this will be used as your reference point for all alarms. You may instead choose to collect the alarm info from Netact if you wish.
Dylan Crewdson
Page 15 of 29
MOP1002
d.) If you are not already logged into the ‘Software Manager’ tool, then log in as per the instructions covered in section 2.1.1. e.) If you are not already in the group for your selected Parent Node (RNC), then select the correct node as detailed in section 2.1.2 a) – b). f.) At the top of the far left column (where check marks are applied to select MOs), hover your mouse over the box, a menu will appear, select ‘Deselect all MOs’. This will remove any MOs previously selected so that no nodes are unintentially included.
g.) Place a check mark in the box beside the WBTS(s) you wish to download software to, then select ‘New Task’, a task window will open.
Dylan Crewdson
Page 16 of 29
MOP1002 g.) Select ‘Download’ from the ‘Software Operations’ tree, enter a task name, eg. , select the sw package (this will be specified in the CTASK), then select ‘Next’.
h.) Select ‘Immediately’, then select ‘Next’.
Dylan Crewdson
Page 17 of 29
MOP1002 i.) At the summary page, double check your selected objects, task name, task type and frequency, if all is correct, select ‘Finish’ to start the task (window will close).
j.) After starting the task, you can then click on the ‘Network Operations’ tab as shown below, where you can monitor the progress of the task. During download, maintain a vigilant watch on the state of the radio network from the OMS.
Dylan Crewdson
Page 18 of 29
MOP1002 Each object in the task will also show the status individually in the ‘Last software task’ column:
k.) Once the download is complete (can take approx 10-15 mins), the Software Manager will automatically run an upload on the BTS (to check which sw versions the WBTS now has). You can then return to the ‘Network Status’ tab, locate the WBTS(s) and verify the downloaded software, shown in the Passive SW ID column.
l.) Repeat steps 2.2 f.) to l.) for each additional batch until all WBTSs have the new software downloaded.
Dylan Crewdson
Page 19 of 29
MOP1002
2.3 ACTIVATION OF SOFTWARE Activation of WBTS software is very similar in process to that of downloading, however the activation of new software is service affecting and therefore must only ever be done during the maintenance window. Software should be activated in batches to mitigate risk of any complications that may occur. For instance, where a cluster of 15 sites is to be upgraded, it is suggested that we separate these into batches of 5 sites at a time. Where a full market roll out is required, we must increase the sizes of these batches, as a rule of thumb we should divide a market by 5 to get our batch size, however where this gives us batch sizes in excess of 50 (recommended maximum), we will instead do batches of 50 at a time regardless of how many batches we have. For example: Market A: 100 WBTSs / 5 = 20 WBTSs at a time (batch size). Market B: 450 WBTS / 5 = 90WBTSs, as this exceeds our maximum size (50), we will instead activate 50 WBTSs at a time, which will result in 9 batches. Furthermore, it is recommended that the the activation batch file size start small and then increase with size to the maximum (say 5, then 10, then 20 etc up to 50 WBTSs). This will mitigate the risk of user error as well as build confidence in the process. a.) Login to the OMS, select the RNC you are working on from the Topology tree on the left side, then select ‘WCELs Overview Page’, this will show all cells on the RNC. Once this view has loaded, select the ‘Blocked WCELs’ to show only those cells which are blocked.
b.) Make a note of all cells which are blocked, either by using the export option in the OMS, or by a series of screen shots. This will be a reference source for you in the event that any cells go OOS during software download.
Dylan Crewdson
Page 20 of 29
MOP1002
c.) Open the ‘Application Launcher’ (installed on your PC*), login to the OMS, then select the required RNC and open ‘Fault Management’. Copy and paste all the shown alarms into another sheet in your spreadsheet, this will be used as your reference point for all alarms. You may instead choose to collect the alarm info from Netact if you wish.
d.) If you are not already logged into the ‘Software Manager’ tool, then log in as per the instructions covered in section 2.1.1.
Dylan Crewdson
Page 21 of 29
MOP1002 e.) If you are not already in the group for your selected Parent Node (RNC), then select the correct node as detailed in section 2.1.2 a) – b). f.) At the top of the far left column (where check marks are applied to select MOs), hover your mouse over the box, a menu will appear, select ‘Deselect all MOs’. This will remove any MOs previously selected so that no nodes are unintentially included.
g.) Place a check mark in the box beside the WBTS(s) you wish to activate software on, then select ‘New Task’, a task window will open.
Dylan Crewdson
Page 22 of 29
MOP1002 h.) Select ‘Activation’ from the ‘Software Operations’ tree, enter a task name, eg. , select the sw package (this will be specified in the CTASK), then select ‘Next’.
i.) Select ‘Immediately’, then select ‘Next’.
Dylan Crewdson
Page 23 of 29
MOP1002 j.) At the summary page, double check your selected objects, task name, task type and frequency, if all is correct, select ‘Finish’ to start the task (window will close).
k.) You will then be presented with a confirmation warning, select ‘Continue’. l.) After starting the task, you can then click on the ‘Network Operations’ tab as shown below, where you can monitor the progress of the task. During activation, maintain a vigilant watch on the state of the radio network from the OMS.
Dylan Crewdson
Page 24 of 29
MOP1002 Each object in the task will also show the status individually in the ‘Last software task’ column:
The activation task does not take long to complete from the Software Manager perspective ( a couple of mins or so). However the WBTS will reset once the activation command is sent, which will cause the WBTS to go OOS for approximately 5-10 minutes. *Note: The Software Manager will automatically schedule an upload for 15 minutes after the activation task is completed, this is done to verify that the WBTSs ar running on the new software after reset (a 15 minute delay is used to avoid any conflict while the WBTS is resetting). The new software will not show in the ‘Active SW ID’ column until the upload has finished. m.) Once both the activation task and upload task is completed, you can go back to the ‘Network Status’ tab and check the active sw for each Node B. The new sw should now be shown in the ‘Active SW ID’ column and the ‘Last software task’ coumn should show ‘ ’, the upload time will show in the ‘Latest upload’ column.
Dylan Crewdson
Page 25 of 29
MOP1002
Ensure all WBTSs recover fully and come back on air (WO) by checking the cell states. Any cells which do not recover should be highlighted to the NOC who will then raise a ticket and dispatch to the field for investigation. *Where the number of simultaneous faulty cells exceeds the following thresholds during an upgrade (not batch), Tier II should be contacted immediately before proceeding further:
Toronto RNC 1: 4 or more cells faulty, stop upgrade and contact Tier II RAN immediately. Toronto RNC 2: 4 or more cells faulty, stop upgrade and contact Tier II RAN immediately. Toronto RNC 4: 3 or more cells faulty, stop upgrade and contact Tier II RAN immediately. Toronto RNC 5: 5 or more cells faulty, stop upgrade and contact Tier II RAN immediately. Ottawa RNC: 3 or more cells faulty, stop upgrade and contact Tier II RAN immediately. Vancouver RNC: 2 or more cells faulty, stop upgrade and contact Tier II RAN immediately.
n.) Repeat steps 2.3 f.) to m.) for each additional batch until all required WBTSs have the new software activated. o.) Compare the active alarms on the RNC after the upgrade with the active alarms prior to the upgrade (collected in 2.3 b.)). Ensure any new alarms are communicated to the NOC so that tickets can be opened and dispatched. p.) Initiate drive test (if applicable) by contacting the designated individual for this task. (Unless otherwise stated in the CTASK, all software activation will be followed by a drive test. This will be carried out by either the NQ team or Field Ops, details of which will be communicated to the implementor ahead of time.) END
Dylan Crewdson
Page 26 of 29
MOP1002
APPENDIX A. NOKIA WBTS SOFTWARE NAMING CONVENTION EXPLAINED Each WBTS software package from Nokia will be delivered with two names, the first is the commercial name which will be the one referred to in the emails, work orders etc. The second will be the exact name of the sw file as it exists on Netact, understanding the dual naming of these files will help avoid any confusion. The following example is given: WN9.1 2.2.1 – This is the commercial name, where WN9.1 is the main sw platform and 2.2.1 is the subsequent sw package release. Main sw update packages will be denoted by the first numeral, ie 2.x.x, while interim sw updates will use the second numeral, ie 2.2.x. The last numeral will denote urgent or specific updates to the interim packages (2.2.1). For the example given above (WN9.1 2.2.1), the actual name of the sw package will be WN9.1_2000_0988_62. This is how the package will appear in Netact on the Software Manager as well as on BTS Manager (shown below). However, outside of Netact and the WBTS, it will still be referred to by it’s commercial name (WN9.1 2.2.1 for the example given).
Dylan Crewdson
Page 27 of 29
MOP1002
B. APPLICATION LAUNCHER INFO Application Launcher is an external application which we use to connect to to the OMS for the purpose of viewing alarms (amongst other things), this is done using the ‘Fault Management’ tool inside the Application Launcher. Usually the Application Launcher may be found through your start menu under a folder named ‘NSN’ -> ‘Application Launcher Client 2’ -> ‘Application Launcher Client 2’.
In the event you find that it is not installed on your PC, you may download it directly from the OMS as shown below.
Dylan Crewdson
Page 28 of 29
MOP1002
C. ROLLBACK PROCEDURE In the event that drive testing of the new software produces poor results, a rollback to the previous software version may be executed only after consultation with Tier 2. Once approved, you may implement section 2.3 (Activation) again, this time selecting instead the previous software version in step g).
Dylan Crewdson
Page 29 of 29
