Major Incident Draft (MID) Template
Methodology
There is a close relationship between problem management and the major incident process. An incident is any event that is not part of the standard operation of a service and that causes an interruption or a reduction in the quality of that service. Incidents are recorded in a standardized system which is used for documenting and tracking outages and disruptions. A Major Incident is an unplanned or temporary interruption of service with severe negative consequences . Examples are outages involving core infrastructure equipment/services that affect a significant customer base, such as isolation of a company site, which is considered a Major Incident. Any equipment or service outage that does not meet the criteria necessary to qualify as a Major Incident is either a Moderare, Minor or Normal Incident. Major incident reports are escalated to the problem manager for quality assurance. Incident Pyramid
The scale of incidents follows an Incident Pyramid where the most incidents are normal, escalating up to a singular Major Incident.
The major incident process consists of following components: Thinking problem management! * Default
MID Template– Page: 1
•
Impact assessment: Classification, outage analysis and risk management. management.
•
Measurement Measurement using the Incident User Metric (IUM).
•
Grading the resources involved in a major incident.
Major incidents can be seen in the context of a quadrant diagram. In this quad major incidents are either failures or misses and result from undesired outcomes. The difference between a miss and a failure is that the former is associated with desired activity while the latter is not. Success is measured as having both desired activity and outcomes. In the context of problem management management a lucky break, desired outcome associated with undesired activity, is not optimal and should be investigated.
Iceberg
Incidents are a portion of activity in problem management management that forms the tip of an iceberg. The major incident process deals with the visible portion of the iceberg, while in the greater field of problem management a large number of non-visible issues are lurking.
Notification (issued within 6 working hours of trigger)
Preliminary (issued within 6 working hours of workaround)
Thinking problem management! * Default
Final (issued within 6
working hours of normal business operations MID Template– Page: 2
resuming)
Details
Description Service desk / Risk logging Trigger (who requested the r eport/notification) Service affected Data networks Messaging Payments Voice Hosting Intranet Documents Ecommerce Backups Storage
AD Security Operations Service desk Monitoring Printing Third party Extranet
Identification (please clearly describe the incident and its symptoms – immediate and visual causes)
Business impact (please describe clearly the undesired outcome)
Conditions (please describe the environment – business or IT – conditions that caused or were present during the incident)
Resolution Initial (describe the workaround)
Final (describe the steps taken to resume normal operations)
Proximate cause analysis Change Capacity Carrier Environmental Hardware Process
Thinking problem management! * Default
Component Failure Configuration Service Provider Bug Vendor
MID Template– Page: 3
Identification
Asset Owner (who does the asset belong to) Physical component or system (CI name in CMDB)
<$x>
Value (in $)
Resources Headcount affected (application or service) Please supply the names of the following contacts involved in the incident , 1st line escalation , 2nd line escalation rd , 3 line escalation) Region including head count (Mark with a X) , Location , Country
Business Unit
(Mark with a X) Please try and supply an accurate count of the number of users involved. , Business unit including head count
Execution
Timelines (date and times) the expanded incident lifecycle
Time of restoration (normal operations resume – the service is back in production)
Time of workaround (Service is back in
Time when incident started (actual – something has happened to a CI or a risk event has occurred)
Time when incident was detected (incident is detected either by monitoring tools, IT personnel or, worse case, the user/customer)
Time of diagnosis (underlying cause – we know what happened?)
Time of repair (process to fix failure started or corrective action initiated)
Time of recovery (component recovered – the CI is back in production – business ready to be resumed)
production with workaround)
Time of escalation (to problem management team)
Time period service was unavailable (SLA measure) Time period service was degraded (SLA measure)
Measurement
Function Please select the most appropriate
Cause Please select the most appropriate
Type (mark with a X) To calculate the IUM please select a single type which best describes the incident Thinking problem management! * Default
MID Template– Page: 4
Outage
Usage
Multiplier
Scrutiny by Profit 3 management Effect on Staff 10 productivity* Impact on Share price 5 company’s image Direct financial Assets Liability or Nominal 1 vulnerability Limited to Budget 2 internal IT process Incident User Metric Cost of Downtime Analysis
Yes No* Yes
No
*
Yes No* Yes
No*
Yes
No*
Yes
No*
Metric is based on Outage type multiplier multiplier * outage time in minutes * percentage of users effected. Calculation is also weighted based on degradation at 60% and non-business hours at 50%. Default outage type is “Effect on productivity.”
Classification
Scope (Mark with a X) Dashboard designation = S (4) More than 50% of customers affected (3) More than 25% of customers affected (2) Less than 25% of customers affected* (1) Less than 1% of users affected (0) Single IT customer affected
Credibility (Mark with a X) Dashboard designation = CR (4) Areas outside the company will be affected negatively (3) Company affected negatively (2) Multiple business units affected negatively (1) Single business units affected negatively (1) No credib credibility ility issue*
Operations (Mark with a X) Dashboard designation = OP (4) Interferes with core business functions (3) Interferes with business activities* (2) Significant interference with completion of work (1) Some interference with normal completion of work (0) No work interference
Urgency (Mark with a X) Dashboard designation = U (4) Underway and could not be stopped (3) Caused by unscheduled change or maintenance (2) Incident caused by a change (1) Incident caused by scheduled maintenance (0) Completion time not important i mportant*
Prioritization (Mark with a X) Dashboard designation = P Reviewing the scope , credibility, operations and urgency please classify the priority of the incident (4) Critical - An immediate i mmediate and sustained effort using all available resources until resolved. On-call procedures Thinking problem management! * Default
MID Template– Page: 5
activated, vendor support invoked. (3) High - Technicians respond immediately, assess the situation, and may interrupt other staff working low or medium priority jobs for assistance. (2) Medium - Respond using standard procedures and operating within normal supervisory management management structures. (1) Low - Respond using standard operating procedures as time allows. * (0) No prioritization ROC calculation S CR
OP
U
P
TOTAL / 20
%
Outage analysis
Service period outage classification
(Mark with a X) Dashboard designation = P
(4) Critical - App, server, link l ink (network or voice) unavailable for greater than 4 hours or degraded for greater than 1 day – negative business delivery for more than 1 month (3) Major - App, server, link (network or voice) unavailable for greater than 1 hour or degraded for greater than 4 hours - negative business delivery for more than 1 week (2) Moderate - App, server, link (network or voice) unavailable for greater than 30 minutes or degraded for greater than 1 hour - negative business delivery for more than 1 day (1) Minor - App, server, link (network or voice) unavailable greater than 5 minutes or degraded for greater than 30 minutes - negative business delivery for more than 1 hour (0) Low* - App, server, link (network or voice) unavailable for less than 5 minutes or degraded for less than 30 minutes - negative business delivery for less than 1 hour
Service consequence outage classification
(Mark with a X) Dashboard
designation = C
(4) Critical - Financial loss, which puts a business unit in a critical position - greater than $10m or substantial loss of credibility or litigation or prosecution or fatality or disability. (3) Major - Financial loss which severely impacts the profitability of a business business unit - greater than $1m or serious loss of credibility or sanction or impairment (2) Moderate - Financial loss which impacts the profitability of the business unit, greater than $100k or embarrassment or reported to regulator or hospitalization. (1) Minor -Financial loss with a visible impact on profitability but no real effect, greater than $10k or some embarrassment or rule or process breaches or medical treatment (0) Low* - Financial loss with no real effect, less than R50k or irritating or no legal or regulatory issue or no Thinking problem management! * Default
MID Template– Page: 6
medical treatment. ROC analysis P
C
Total / 8
%
Risk management
Risk impact
(Mark with a X) Dashboard designation = I
Evaluate the data and information that is directly effected by the incident taking into account the involvement of the people, process, products and partners. “At Risk” issues Confidentiality (Information is Secure accessible only to those authorized) Confidential Restricted* Public Integrity (Safeguarding the accuracy and Very high completeness of information) High Moderate* Low Availability (Authorised users have Mandatory access to information when required.) Very high High Moderate* Low (4)Critical (3)Major (2)Moderate (1)Low (0)None Rating Taking into
account the above please rate the Risk impact
Risk vulnerability
(Rate as either low, moderate, high or major) Dashboard designation = V
Rate the vulnerability in the following categories of the information or data that is affected by the incident Loss Error Failure* (4)Critical (3)Major (2)Moderate (1)Low (0)None Rating Taking into account the above please rate the Risk vulnerability
Countermeasures Dashboard designation = CM What measures are in place to mitigate any risks identified with the information or data affected by the incident
Rating
Taking into account the above please rate the Risk Countermeasures
(4)Critical
(3)Major
(2)Moderate
(1)Low
(0)Non e
ROC analysis I
V
CM
TOTAL / 12
%
Thinking problem management! * Default
MID Template– Page: 7
Closure
Escalations (Mark with a X) Further Root Cause analysis required Escalated to Operational Risk Escalated to CTO/CIO (logged as a problem at Service Desk) Escalated to Infrastructure Manager No Escalation
Thinking problem management! * Default
Yes Yes Yes
No* No* No*
Yes Yes*
No* No
MID Template– Page: 8
