Splunk® Enterprise Security User Manual 4.6.0 Cloud only Generated: 3/10/2017 3:17 pm
Copyright (c) 2017 Splunk Inc. All Rights Reserved
Table of Contents Introduction..........................................................................................................1 About Splunk Enterprise Security...............................................................1 Incident Review....................................................................................................2 Incident Review...........................................................................................2 Notable events..........................................................................................11 Security Posture dashboard......................................................................19 Correlation Searches and Alerts......................................................................21 Correlation search overview......................................................................21 Creating correlation searches...................................................................21 Configuring correlation searches..............................................................22 Export content as an app from Splunk Enterprise Security.......................30 Set up adaptive response actions in Splunk Enterprise Security..............31 Included adaptive response actions with Splunk Enterprise Security.......33 Investigations.....................................................................................................38 Create and track investigations in Splunk Enterprise Security..................38 Manage security investigations in Splunk Enterprise Security..................47 Assets and Identities.........................................................................................51 Asset and Identity dashboards..................................................................51 Asset and Identity Investigator dashboards..............................................54 User Activity Monitoring............................................................................59 Add asset and identity data to Splunk Enterprise Security.......................62 Configure asset and identity correlation in Splunk Enterprise Security.....68 How Splunk Enterprise Security processes and merges asset and identity data..............................................................................................69 Asset and identity lookup header and field reference...............................74 Modify asset and identity lookups in Splunk Enterprise Security..............81 Example methods of adding asset and identity data to Splunk Enterprise Security...................................................................................83 Access and Endpoint Domain..........................................................................86 Access dashboards...................................................................................86 Endpoint dashboards................................................................................92
i
Table of Contents Risk Analysis....................................................................................................103 Risk Analysis...........................................................................................103 Configure risk scoring.............................................................................106 Threat Intelligence...........................................................................................110 Threat Intelligence dashboards...............................................................110 Configure threat intelligence sources......................................................113 Web Intelligence dashboards..................................................................131 Network Domain...............................................................................................138 Network dashboards...............................................................................138 Web Center and Network Changes dashboards.....................................146 Port and Protocol Tracker dashboard.....................................................150 Protocol Intelligence dashboards............................................................152 Dashboards......................................................................................................160 Dashboard overview...............................................................................160 Advanced Filter.......................................................................................163 Key indicators..........................................................................................165 Creating new content in Splunk Enterprise Security...............................169 Create a glass table................................................................................173 Managing glass tables............................................................................175 Audit dashboards....................................................................................177 Predictive Analytics dashboard...............................................................187 Included Add-ons.............................................................................................190 Analyze Splunk UBA threats and anomalies in Splunk ES.....................190 Send correlation search results to Splunk UBA to be processed as anomalies...............................................................................................192 Configurations and Troubleshooting.............................................................195 Configuration Settings.............................................................................195 Content Management..............................................................................204 Configure lists and lookups.....................................................................205 Dashboard Troubleshooting....................................................................217 Dashboard requirements matrix..............................................................218
ii
Table of Contents Advanced Guidance........................................................................................234 Extreme Search......................................................................................234 An Extreme search example...................................................................240
iii
Introduction About Splunk Enterprise Security Splunk Enterprise Security provides the security practitioner with visibility into security-relevant threats found in today's enterprise infrastructure. Enterprise Security is built on the Splunk operational intelligence platform and uses the search and correlation capabilities, allowing customers to capture, monitor, and report on data from security devices, systems, and applications. As issues are identified, security administrators can quickly investigate and resolve the security threats across the access, endpoint, and network protection domains.
Access Splunk Enterprise Security 1. Open a web browser and navigate to Splunk Web. 2. Log in with your username and password. 3. From the Apps list, click Enterprise Security.
1
Incident Review Incident Review The Incident Review dashboard displays notable events and their current status. As an analyst, you can use the dashboard to gain insight into the severity of events occurring in your system or network. You can use the dashboard to triage new notable events, assign events to analysts for review, and examine notable event details for investigative leads.
How Splunk Enterprise Security identifies notable events Splunk Enterprise Security detects patterns in your data and automatically reviews events for security-relevant incidents using correlation searches. When a correlation search detects a suspicious pattern, the correlation search creates an alert called a notable event. The Incident Review dashboard surfaces all notable events, and categorizes them by potential severity so you can quickly triage, assign, and track issues. For more information on notable events, see Notable Events.
Incident review workflow You can use this example workflow to triage and work notable events on the Incident Review dashboard. 1. An administrative analyst monitors the Incident Review dashboard, sorting and performing high-level triage on newly-created notable events. 2. When a notable event warrants investigation, the administrative analyst assigns the event to a reviewing analyst to start investigating the incident. 3. The reviewing analyst updates the status of the event from New to In Progress, and begins investigating the cause of the notable event. 4. The reviewing analyst researches and collects information on the event using the fields and field actions in the notable event. The analyst records the details of their research in the Comments field of the notable event. 5. After the reviewing analyst addresses the cause of the notable event and any remediation tasks have been escalated or solved, the analyst sets the notable event status to Resolved. 6. The analyst assigns the notable event to a final analyst for verification. 7. The final analyst reviews and validates the changes made to resolve the 2
issue, and sets the status to Closed.
Triage notable events on the Incident Review dashboard Use this dashboard as part of your incident triage workflow. You can monitor notable events and the actions that analysts take to resolve the issues that triggered a notable event. You must wait for your search to complete before you can manage notable events or view event details. Accelerate triage with tags and filters Speed up your notable event triage with search filters, tagging, and sorting. For example, focus on groups of notable events or an individual notable event with the search filters and time range selector. Notable events contain Urgency, Status, and Owner fields to help you categorize, track, and assign events. Simplify searching and add identifiers to notable events using tags. Click Edit Tags in the field actions menu for a notable event field such as Title, Status, or Owner to add new tags or modify existing ones. After you create a tag, you can use it to filter the dashboard. Assign notable events You can assign one event at a time or several at once. 1. Select a notable event. 2. Click Edit selected. 3. Select an Owner to assign the event or events to. Or, click Assign to me to assign the event or events to yourself. 4. Save your changes. Owners are unassigned by default, and you can assign notable events to any user with an administrator, ess_admin, or ess_analyst role. For more on user roles, see Configure users and roles in the Installation and Upgrade Manual. Update the status of a notable event New notable events have the New status. As analysts triage and move a notable event through the incident review workflow, the owner can update the status of the notable event to reflect the actions they take to address the event.
3
1. Select one or more events, then click Edit all selected. To take action on all displayed events, click Edit all ## matching events. 2. In the Edit Events window, update the fields to reflect your actions. 3. (Optional) Add a Comment to describe the actions you took. 4. Save changes. Note: If your changes are not immediately visible, check the dashboard filters. For example, if the filter is set to "New" after you changed an event to "In Progress", your updated event will not display. You can require analysts to enter comments when updating a notable event. See Customize Incident Review for more. You can choose from the following notable event statuses. Status
Description
Unassigned The event has not been assigned to an owner. New
Default status. The event has not been reviewed.
In Progress
An owner is investigating the event.
Pending
An action must occur before the event can be closed.
Resolved
The owner has addressed the cause of the event and is waiting for verification.
Closed The resolution of the event has been verified. You can customize the notable event status names and workflow progression to match your process. For more information, see Managing and monitoring notable event statuses. Prioritize notable events by urgency Use the urgency level of a notable event to prioritize incident review. Every notable event is assigned an urgency. Urgency levels can be unknown, low, medium, informational, high, or critical. Urgency levels are calculated using the severity of the correlation search event and the priority of the asset or identity involved in the event. See How the urgency of an event is assigned. By default, security analysts can change the urgency of a notable event. See Customize Incident Review to learn how to change that default.
4
Notify an analyst of untriaged notable events You can use a correlation search to notify an analyst if a notable event has not been triaged. 1. Select Configure > Content Management. 2. Locate the Untriaged Notable Events correlation search using the filters. 3. Modify the search, changing the notable event owner or status fields as desired. 4. Set the desired alert action. 5. Save the changes. 6. Enable the Untriaged Notable Events correlation search.
Review details of notable events After you finish triaging notable events, begin your investigation. Use the available fields on a notable event to assess the urgency, contributing events, and risk scores associated with the notable event. Open the event details to learn more about a notable event. • Review the History to see the recent investigation activity on the notable event. Click View all recent activity for this Notable Event to see analyst comments, status changes, and other activities for the event. • See which correlation search generated the notable event. Click the name of the correlation search to make changes to or review the correlation search to understand why the notable event was created. • View the Contributing Events that caused the notable event to be created. • Review the risk scores listed for assets and identities involved in a notable event. Click a risk score to open the Risk Analysis dashboard filtered on that asset or identity. • If one original event created a notable event, you can see the full details. • Review the Adaptive Responses to see which adaptive response actions have been performed for this notable event, whether the actions were successfully performed, and drill down for more details. Click the name of the response action to see potential results generated by this action's invocation. Click View Adaptive Response Invocations to see the raw audit events for the response actions associated with this correlation search. It takes up to five minutes for updates to appear on this table. • Review the Next Steps to see if any next steps for notable event triage are defined.
5
Take action on a notable event From Incident Review, you can suppress or share a notable event, add an event or multiple events to an investigation, analyze the risk that an asset or identity poses to your environment, or investigate a field in more detail on another dashboard. Run an adaptive response action Based on the details in a notable event, you may want to run a response action to gather more information, take an action in another system, send information to another system, modify a risk score, or something else. 1. From a notable event, select the arrow to expand the Actions column. 2. Click Run Adaptive Response Actions. 3. Click Add New Response Action and select an adaptive response action from the list. You can use the category filter or search to reduce the number of actions that you can select. 4. Fill out the form fields for the response action. Use the field name to specify a field, rather than the name that shows on Incident Review. For example, type "src" instead of "Source" to specify the source field for an action. 5. Click Run. You can check the status of the response action in the notable event details. You cannot run adaptive response actions from the Search dashboard. View the original field names of fields displayed on Incident Review on the Incident Review - Event Attributes panel of the Incident Review Settings dashboard. Note: Adblock extensions in your browser can cause response actions to fail. Add the host name of your Splunk Enterprise Security host to the site whitelist for the adblock extension. See Included adaptive response actions with Splunk Enterprise Security for more about the different adaptive response actions included with Splunk Enterprise Security. Share or bookmark a notable event Share a link to a notable event with another analyst, or bookmark it for later. From the event actions, click Share Notable Event. You cannot share a notable event from the Search dashboard.
6
Analyze risk of an asset or identity You can analyze the risk that an asset or identity poses to your environment in the Incident Review dashboard. 1. Open the event details. 2. Review the risk score next to asset or identity fields such as src or host. 3. Click the risk score to open the Risk Analysis dashboard filtered on the asset or identity. Not all assets and identities display a risk score. Risk scores that display for an asset or identity in Incident Review may not match the risk score on the Risk Analysis dashboard for that risk object. See How risk scores display in Incident Review for more. Add a notable event to an investigation Investigate notable events that could be a part of a security incident by adding them to an investigation. Add a notable event to an existing investigation 1. Add one or more notable events to an investigation. 1. Add a single notable event by selecting Add Event to Investigation from the Event Actions. 2. Add multiple notable events by selecting the check boxes next to the notable events and click Add Selected to Investigation. 2. Select an investigation to add the notable events to. If you selected an investigation in the investigation bar, that investigation is selected by default. 3. Click Save. 4. After the event or events are successfully added to the investigation, click Close. Add a notable event to a new investigation 1. Select one or several notable events and click Add Selected to Investigation. 2. Click Create Investigation to start a new investigation. 3. Type a title for the investigation. 4. (Optional) Type a description. 5. Click Save to save the investigation.
7
6. Click Save to add the notable event or notable events to the investigation. Clicking Cancel does not add the selected notable events, but the new investigation is still created. 7. After the event or events are successfully added to the investigation, click Close. See Create and track investigations in Splunk Enterprise Security for more. Investigate a field in more detail Take action on a specific field, such as host, src, src_ip, dest, or dest_ip. Different actions are available to take depending on the field you select. • Tag fields by selecting Edit tags. • Investigate an asset by selecting Asset Investigator to open the Asset Investigator dashboard filtered on the asset. • Search for access-related events for a specific destination IP address by selecting Access Search (as destination). • Investigate a domain by selecting Domain Dossier. • Find other notable events with matching malware signatures by selecting Notable Event Search. Suppress a notable event Hide notable events from the Incident Review dashboard by suppressing them. Creating a notable event suppression does not change the counts of notable events on the posture or auditing dashboards. See Create and manage notable event suppressions for more details. 1. Select a notable event on the Incident Review dashboard. 2. From the Actions menu, select Suppress Notable Events. 3. Type a Suppression Name. For example, Excessive_Failed_Logins.
4. (Optional) Provide a reason for the suppression using the Description field.
8
5. (Optional) Set a date range. After the time limit ends, the suppression filter expires and stops hiding events. 6. Review the Selected Fields to validate the fields that you want to suppress notable events from. For example, the src field 7. (Optional) Click change to modify the notable event fields used for the suppression.
8. Save changes. This example notable event suppression hides all notable events created after June 10, 2016 that contain a src=_jdbc_ field from Incident Review. You cannot suppress notable events from the Search dashboard.
Audit incident review activity You can audit and review incident review activity on the Incident Review Audit dashboard.
Customize Incident Review Customize the display of the Incident Review dashboard, and also modify analyst capabilities and permissions. Modify analyst capabilities and permissions You can change the default capabilities and permissions assigned to analysts to better fit your workflow. Configure whether analysts can override the calculated urgency of a notable event and choose whether to require an analyst to add a comment when updating a notable event on the Incident Review Settings page.
9
1. Select Configure > Incident Management > Incident Review Settings to view the Incident Review settings. 2. Allow or prevent analysts from overriding the calculated urgency of a notable event with the Allow Overriding of Urgency checkbox. Analysts are allowed to override urgency by default. 3. Require analysts to add a comment when updating a notable event by checking the Required checkbox under Comments. 4. If you require analysts to add a comment, enter the minimum character length for required comments. The default character length is 20 characters. Configure the desired capacity of your security analysts on the General Settings page. 1. Select Configure > General > General Settings to view the General Settings. 2. Enter a preferred number of incidents that should be assigned to an analyst with the Incident Review Analyst Capacity setting. The default is 12. Note: This value is used for tracking purposes, and does not prevent more than the default number of notable events from being assigned to an analyst. Change Incident Review columns You can change the columns displayed on the Incident Review dashboard. 1. Review the existing columns in Incident Review - Table Attributes. 2. Use the action column to edit, remove, or change the order of the available columns. 3. Add custom columns by selecting Insert below or selecting More..., then Insert above. Change notable event fields Make changes to the fields displayed on the Incident Review dashboard for notable events on the Incident Review Settings dashboard. 1. From the Splunk Enterprise Security menu bar, select Configure > Incident Management > Incident Review Settings. 2. Review the Incident Review - Event Attributes. 3. Click Edit to change the field or the field label for a specific field.
10
4. Click Remove to remove a field from displaying on the Incident Review dashboard.
How risk scores display in Incident Review Risk scores do not display in Incident Review for every asset or identity. Only assets or identities (risk objects) that have a risk score and a risk object type of "system" or "user" display in Incident Review. Risk scores only show for the following fields: orig_host, dvc, src, dest, src_user, and user. The risk score for an asset or identity might not match the score on the Risk Analysis dashboard. The risk score is a cumulative score for an asset or identity, rather than a score specific to an exact username. • For example, if a person has a username of "buttercup" that has a risk score of 40, and an email address of "
[email protected]" with a risk score of 60, and the identity lookup identifies that "buttercup" and "
[email protected]" belong to the same person, a risk score of 100 displays on Incident Review for both "buttercup" and "
[email protected]" accounts. • As another example, if an IP of 10.11.36.1 has a risk score of 80 and an IP of 10.11.36.19 has a risk score of 30, and the asset lookup identifies that a range of IPs "10.11.36.1 - 10.11.36.19" belong to the same asset, a risk score of 110 displays on Incident Review for both "10.11.36.1" and "10.11.36.19" IP addresses. Risk scores are calculated for Incident Review using the Threat - Risk Correlation - Lookup Gen lookup generation search. The search runs every 30 minutes and updates the risk_correlation_lookup lookup file. To see more frequent updates to the risk scores in Incident Review, update the cron_schedule of the saved search.
Notable events Splunk Enterprise Security detects patterns in your data and automatically reviews events for security-relevant incidents using correlation searches. When a correlation search detects a suspicious pattern, the correlation search creates an alert called a notable event. A notable event represents one or more anomalous incidents detected by a correlation search across data sources. For example, a notable event can 11
represent: • The repeated occurrence of an abnormal spike in network usage over a period of time • A single occurrence of unauthorized access to a system • A host communicating with a server on a known threat list
Triage and manage notable events The Incident Review dashboard displays all notable events, and categorizes them by potential severity so you can quickly triage, assign, and track issues as part of an incident response workflow. See Incident Review in this manual.
Manually create a notable event You can manually create a notable event from an indexed event, or create one from scratch. Note: By default, only administrators can manually create notable events. To grant other users this capability, see Configure user and roles in the Installation and Upgrade Manual. Create a notable event from an existing event You can create a notable event from any indexed event using the Event Actions menu. Do not create a notable event from notable events on the Incident Review dashboard. 1. From an event, view the event details and click Event Actions. 2. Select Create notable event. 3. Type a Title for the event. 4. (Optional) Select a security Domain. 5. (Optional) Select an Urgency level. 6. (Optional) Select an Owner. 7. (Optional) Select a Status. 8. Type a Description for the event that describes why you created the notable event or what needs to be investigated. The description field does not support formatting of any kind. 9. Save the new notable event. The Incident Review dashboard displays with your new notable event. Note: A notable event created in this way includes tracking fields such as Owner and Status, but does not include the unique fields or links created when a 12
notable event is generated by a correlation search alert action. Create a notable event from scratch Create a notable event based on observations, a finding from a security system outside Splunk, or something else. 1. Select Configure > Incident Management > New Notable Event. 2. Type a Title for the event. 3. (Optional) Select a security Domain. 4. (Optional) Select an Urgency level. 5. (Optional) Select an Owner. 6. (Optional) Select a Status. 7. Type a Description for the event that describes why you created the notable event or what needs to be investigated. The description field does not support formatting of any kind. 8. Save the new notable event. The Incident Review dashboard displays with your new notable event.
Change notable event settings Navigate to Configure > Incident Management in Enterprise Security to make various configuration changes to notable events. • Review the default Notable Event Statuses and add, remove or change a status as desired. • Review existing or create new Notable Event Suppressions. • Customize Incident Review to allow an urgency override or enforce comment usage.
Managing and monitoring notable event statuses An analyst assigns a status to a notable event to communicate the state of the notable event in the investigation workflow. The status aligns with the stages of an investigation, and can be used to review and report on the progress of a notable event investigation on the Incident Review Audit dashboard. To see the available statuses for notable events, select Configure > Incident Management > Notable Event Statuses Label
Description
13
Can be edited
Unassigned
A notable event has not been assigned.
No
New (default) A notable event has not been reviewed.
No
In Progress
An investigation or response to the notable event is in progress.
Yes
Pending
A notable event closure is pending some action.
Yes
Resolved
A notable event has been resolved and awaits verification.
Yes
Closed A notable event has been resolved and verified. Yes Every notable event is assigned a status of New by default when it is created by a correlation search. You can customize notable event statuses to match an existing workflow at your organization. Edit notable event statuses Change the available statuses for notable events on the Edit Notable Event Status page. 1. On the Splunk Enterprise Security toolbar, select Configure > Incident Management > Notable Event Statuses. 2. Select a notable event status to open the Edit Notable Event Status page. 3. Change the Label or Description as desired. Note: You cannot edit the Unassigned and New statuses because they are defaults used when creating notable events. Manage notable event status history Notable events are associated with users, statuses, and comments. Changes made to status names affect only the name of a status, not the status ID assigned to the notable event in the notable index. If you change the name of a default notable event status, the name will change for both past and future notable events. For example, if you rename pending to waiting for customer, all notable events with a status of pending will then have a status of waiting for customer. The status ID assigned to the notable events will remain the same.
14
Notable event status transitions The status names represent the steps required in investigating a notable event. Status transitions define the path of a notable event investigation. An analyst assigned a notable event will change the status of the notable event as the investigation progresses. To change the current status on a notable event: • The analyst must be a member of a role that has permission to change a status. Notable event status transitions are available to the ess_analyst and ess_admin roles by default. • The follow-on status must allow a transition from the current status. Every status can transition to any other status by default. For example, a notable event in a New status can transition directly to any other status including Closed. Restricting status transitions You can define a transition workflow and limit which status can transition to another status, creating a predefined path for the notable event investigation workflow. By default, no transition path is defined or required and every status can transition to every other status. Prerequisites • In order to edit status transitions, you must have the ess_analyst role or your role must be assigned the Edit Notable Event Statuses capability. For more information about user roles and capabilities, see Configure user and roles in the Installation and Upgrade Manual. • Define the status workflow for notable event investigations. Determine which statuses to require, and whether analysts should follow a specific sequence of statuses before completing the investigation workflow. Determine whether any roles can bypass the full workflow. Restrict notable event status transitions 1. On the Splunk Enterprise Security toolbar, select Configure > Incident Management > Notable Event Statuses. 2. Select a notable event status to open the Edit Notable Event Status page. 3. In Status Transitions, modify the To Status fields. 1. To define which roles are allowed to transition a notable event to the selected status, choose the Authorization field and add or remove roles. 15
2. To remove a transition an event to the selected status, choose Unselect All. 4. Save the changes. 5. Test the changes to the status workflow. If any transitions required adding or removing roles, test with credentials assigned to each role. Add a new status Add a new status to the notable event investigation workflow. If you restrict status transitions, determine where this status fits in the workflow. 1. Define the status workflow for notable events. 1. Determine where the new status is needed in the workflow. 2. Determine whether any roles (e.g. ess_admin) will be allowed to bypass the new status in the workflow. 2. On the Splunk Enterprise Security toolbar, open Configure > Incident Management and select Notable Event Statuses. 3. Select New. 4. Add a label. This is the Status field value used on the Incident Review dashboard and for notable event status reporting. Example: Waiting on ITOps 5. Add a description. The description is only referenced in the Notable Event Status page. Example: Waiting on another department. 6. (Optional) Select Default status. Choose only if you are replacing the New status for notable events 7. (Optional) Select End status. Choose when adding an additional Closed status for notable events. 8. Define the Status Transitions by modifying the To Status fields. 1. Review the status workflow and determine which statuses a notable event can transition to. 2. Choose the Authorization field and add the roles allowed to transition a notable event to the selected status. 9. Save the changes. Example: In our workflow, the "Waiting on ITOps" status occurs after "New" and "In Progress", but before "Pending." It is not a required status and can be skipped over to choose "Pending." Edit the Status Transitions in "Waiting on ITOps" for "Pending," "Resolved," and "Closed" and add the roles ess_admin and ess_analyst added under Authorization. 10. Edit the statuses that will precede the new status in the workflow, and add the roles allowed to perform the transition. Example: In our workflow, a notable event can be given a status of "Waiting on ITOps" from a status of "New" and "In Progress." Edit the Status Transitions in both "New" and "In Progress" adding the ess_admin 16
and ess_analyst roles under Authorization for "Waiting on ITOps". 11. Test to ensure the status can be assigned and that any status transitions involving it work.
Create and manage notable event suppressions You can hide notable events from the Incident Review dashboard by creating a notable event suppression. A suppression is a search filter that hides additional notable events from view, and is used to stop excessive or unwanted numbers of notable events from appearing on the Incident Review dashboard. Notable events that meet the search conditions are still created and added to the notable index. Suppressed notable events continue to contribute to notable event counts on the Security Posture and auditing dashboards. To prevent notable events that meet certain conditions from being created, see Throttling. You can create a suppression filter in two ways. • Create a suppression from Incident Review. See Suppress a notable event. • Create a suppression from the Configure menu. See Create a suppression from Notable Event Suppressions. Create a suppression from Notable Event Suppressions 1. Select Configure > Incident Management > Notable Event Suppressions. 2. Click Create New Suppression. 3. Type a Name and Description for the suppression filter. 4. Type a Search to use to find notable events to be suppressed. 5. Set the Expiration Time. This defines a time limit for the suppression filter. If the time limit is met, the suppression filter is disabled.
17
Edit notable event suppressions 1. Select Configure > Incident Management > Notable Event Suppressions. 2. Select a notable event suppression to open the Edit Notable Event Suppression page. 3. Edit the Description and Search fields used for the suppression filter. Disable notable event suppressions 1. Select Configure > Incident Management > Notable Event Suppressions. 2. Select Disable in the Status column for the notable event suppression. Remove a notable event suppression 1. From the Splunk platform toolbar, select Settings > Event types. 2. Search for the the suppression event: notable_suppression-
. 3. Select delete in the Actions column for the notable event suppression.
Audit notable event activity You can audit analyst incident review activity on the Incident Review Audit dashboard. Audit notable event suppressions with the Suppression Audit dashboard.
How urgency is assigned to notable events Notable events are assigned an urgency level that is a combination of the severity of the correlation search and the priority assignment of the relevant asset or identity. You can use the Urgency field to prioritize the investigation of notable events.
You can change which severity and priority values result in which calculated urgency values for notable events in Enterprise Security. You cannot modify the names of the notable event urgency values. 18
1. On the Enterprise Security menu bar, select Configure > Data Enrichment > Lists/Lookups. 2. Choose the Urgency Levels lookup. An editable, color coded table representing the urgency lookup file displays. 3. In any row where the priority or severity is listed as unknown review the assigned Urgency. 4. (Optional) Edit the table and change the Urgency from unknown to one of the accepted values. 5. Save the changes Note: When calculating the severity level, a notable event displays a default of "low" urgency when an asset or identity is categorized as "unknown." The "unknown" classification typically represents an object that has no match in the asset and identities system. Note: A notable event can be assigned an unknown urgency level if the severity value assigned by the correlation search or in a triggering event is not recognized by Enterprise Security. This indicates an error in the severity value provided by the correlation search syntax. Verify that the correlation search severity is unknown, informational, low, medium, high, or critical.
Notable event index See Notable event index in the Splunk developer portal for information about the notable index and how to search it effectively with macros.
Security Posture dashboard The Security Posture dashboard is designed to provide high-level insight into the notable events across all domains of your deployment, suitable for display in a Security Operations Center (SOC). This dashboard shows all events from the past 24 hours, along with the trends over the past 24 hours, and provides real-time event information and updates.
Dashboard panels Panel
Description
19
Key Indicators
Notable Events by Urgency
Displays the count of notable events by security domain over the past 24 hours. For more information, see Key indicators in this manual. Displays the notable events by Urgency for the last 24 hours. Notable Events by Urgency uses an urgency calculation based on the priority assigned to the asset and the severity assigned to the correlation search. The drilldown opens the Incident Review dashboard showing all notable events with the selected urgency in the last 24 hours.
Displays a timeline of notable events by security domain. The drilldown opens Notable the Incident Review dashboard Events Over showing all notable events in the Time selected security domain and time frame. Displays the top notable events by rule name, including a total count and a Top Notable sparkline to represent activity spikes over time. The drilldown opens the Events Incident Review dashboard scoped to the selected notable event rule. Displays the top 10 notable event by src, including a total count, a count per Top Notable correlation and domain, and a sparkline Event to represent activity spikes over time. Sources The drilldown opens the Incident Review dashboard scoped to the selected notable event source.
20
Correlation Searches and Alerts Correlation search overview A correlation search scans multiple data sources for defined patterns. When the search finds a pattern, it performs an adaptive response action. Correlation searches can search many types of data sources, including events from any security domain (access, identity, endpoint, network), asset lists, identity lists, threat intelligence, and other data in Splunk platform. The searches then aggregate the results of an initial search with functions in SPL, and take action in response to events that match the search conditions with an adaptive response action. • To create a correlation search, see Create a correlation search in Splunk Enterprise Security Tutorials. • To set up or modify correlation searches in your environment, see Configuring correlation searches.
Examples of correlation searches • Identify an access attempt from an expired account by correlating a list of identities and an attempt to authenticate into a host or device. • Identify a high number of hosts with a specific malware infection, or a single host with a high number of malware infections by correlating an asset list with events from an endpoint protection system. • Identify a pattern of high numbers of authentication failures on a single host, followed by a successful authentication by correlating a list of identities and attempts to authenticate into a host or device. Then, apply a threshold in the search to count the number of authentication attempts.
Creating correlation searches You can create your own correlation searches to create notable events, modify risk scores, and perform other adaptive response actions automatically based on a correlation in events. You can use the Content Management page to create a correlation search. There are two ways to create correlation searches in Splunk Enterprise Security.
21
• Create a correlation search manually if you are an expert with SPL. You can review the included correlation searches for examples of the search methodology and available options. Test your correlation search ideas on the Search page before implementing them. • For more assistance with the syntax of correlation searches, use the guided search creation wizard to create a correlation search. The guided search creation wizard allows you to create a correlation search that uses data models or lookups as the data source. The wizard takes your choices about the data source, time range, filtering, aggregate functions, split-by fields, and other conditions and builds the syntax of the search for you. See Create a correlation search in Splunk Enterprise Security Tutorials for a step-by-step tutorial of creating a correlation search.
Configuring correlation searches Configure correlation searches to update the settings associated with how they run. See Correlation search overview to learn more about a correlation search.
Enable correlation searches Enable correlation searches to start running adaptive response actions and receiving notable events. Splunk Enterprise Security installs with all correlation searches disabled so that you can choose the searches that are most relevant to your security use cases. 1. From the Splunk ES menu bar, select Configure > Content Management. 2. Filter the Content Management page by a Type of Correlation Search to view only correlation searches. 3. Review the names and descriptions of the correlation searches to determine which ones to enable to support your security use cases. For example, if compromised accounts are a concern, consider enabling the Concurrent Login Attempts Detected and Brute Force Access Behavior Detected correlation searches. 4. In the Actions column, click Enable to enable the searches that you want to enable. After you enable correlation searches, dashboards will start to display notable events, risk scores, and other data.
22
Change correlation search scheduling Change the default search type of a correlation search from real-time to scheduled. Splunk Enterprise Security uses indexed real-time searches by default. 1. From the Content Management page, locate the correlation search you want to change. 2. In the Actions column, click Change to scheduled.
Editing correlation searches You can make changes to correlation searches to fit your environment. For example, modify the thresholds used in the search, change the response actions that result from a successful correlation, or change how often the search runs. Modifying a correlation search does not affect existing notable events. Click the name of a correlation search on the Content Management page to edit it. If you modify the start time and end time for the correlation search, use relative time modifiers. See Specify time modifiers in your search in the Splunk Enterprise Search Manual. Edit the correlation search in guided mode You can edit some correlation searches in guided mode. Not all correlation searches support guided search editing. If a search appears grayed-out and has the option to Edit search manually or Edit search in guided mode, the search was built in guided mode and can be edited in guided mode. If a search can be edited in the search box and only has the option to Edit search in guided mode, editing the search in guided mode overwrites the existing search. 1. Click Edit search in guided mode to open the guided search creation wizard. 2. Review the search elements in the correlation search, making changes if you want. 3. Save the search.
Throttle the number of response actions generated by a correlation search
23
Set up throttling to limit the number of response actions generated by a correlation search. When a correlation search matches an event, it triggers a response action. By default, every result returned by the correlation search generates a response action. Typically, you may only want one alert of a certain type. You can use throttling to prevent a correlation search from creating more than one alert. Some response actions allow you to specify a maximum number of results in addition to throttling. See Set up adaptive response actions in Splunk Enterprise Security. 1. Select Configure > Content Management. 2. Click the title of the correlation search you want to edit. 3. Type a Window duration. During this window, any additional event that matches any of the Fields to group by will not create a new alert. After the window ends, the next matching event will create a new alert and apply the throttle conditions again. 4. Type the Fields to group by to specify which fields to use when matching similar events. If a field listed here matches a generated alert, the correlation search will not create a new alert. You can define multiple fields, and available fields depend on the search fields that the correlation search returns. 5. Save the correlation search. Throttling applies to any type of correlation search response action and occurs before notable event suppression. See Create and manage notable event suppressions for more on notable event suppression.
Obtain a list of correlation searches To obtain a list of correlation searches enabled in Splunk Enterprise Security, use a REST search to extract the information that you want in a table. For example, create a table with the app, security domain, name, and description of all correlation searches in your environment. | rest splunk_server=local count=0 /services/saved/searches | where match('action.correlationsearch.enabled', "1|[Tt]|[Tt][Rr][Uu][Ee]") | rename eai:acl.app as app, title as csearch_name, action.correlationsearch.label as csearch_label, action.notable.param.security_domain as security_domain | table csearch_name, csearch_label, app, security_domain, description
As another example, create a table with only the enabled correlation searches and the adaptive response actions associated with those searches in your 24
environment. To see the adaptive response actions for all correlation searches, remove | where disabled=0. | rest splunk_server=local count=0 /services/saved/searches | where match('action.correlationsearch.enabled', "1|[Tt]|[Tt][Rr][Uu][Ee]") | where disabled=0 | eval actions=split(actions, ",") | table title,actions
Correlation searches migration to savedsearches.conf Starting in Splunk Enterprise Security version 4.6.0, correlationsearches.conf is no longer used to define correlation searches. Instead, savedsearches.conf uniquely identifies correlation searches using the action.correlationsearch.enabled=1 parameter. The correlationsearches.conf file is deprecated. For stability, the Threat - Correlation Searches - Lookup Gen saved search continues to use the contents of both correlationsearches.conf and savedsearches.conf to populate the correlationsearches KV Store collection used by Incident Review. Changes Splunk Enterprise Security makes at upgrade When you upgrade to Splunk Enterprise Security 4.6.0, Splunk Enterprise Security migrates all correlation searches in your environment, including custom correlation searches. The confcheck_es_correlationmigration.py script migrates all entries in correlationsearches.conf into updated entries in savedsearches.conf. The migration can take up to five minutes to complete after the upgrade. During the upgrade, Splunk Enterprise Security continues to create notable events without interruption. Changes you have to make after upgrade After upgrading to Splunk Enterprise Security 4.6.0, you have to make additional changes. • Check for searches that did not migrate successfully and migrate the correlationsearches.conf entries manually using the parameter definitions below. • Update searches that call the correlationsearches REST endpoint. ♦ For example, a search that displays a list of correlation searches in 25
your environment would change from | rest splunk_server=local /services/alerts/correlationsearches | rename eai:acl.app as app, title as csearch_name | table app security_domain csearch_name description
to | rest splunk_server=local count=0 /services/saved/searches | where match('action.correlationsearch.enabled', "1|[Tt]|[Tt][Rr][Uu][Ee]") | rename eai:acl.app as app, title as csearch_name, action.correlationsearch.label as csearch_label, action.notable.param.security_domain as security_domain | table csearch_name, csearch_label, app, security_domain, description
♦ See Obtain a list of correlation searches for more examples of updated searches. Both the correlationsearches.conf and savedsearches.conf files are used to populate the KV Store collection used by Incident Review. Custom search macros that reference that KV Store collection continue to work as before, but consider updating them anyway. correlationsearches.conf
parameter translation to savedsearches.conf
All correlationsearches.conf parameters now exist in savedsearches.conf and the correlationsearches.conf file has been deprecated. Do not update it directly except to manually migrate correlation search definitions. Identification parameters for correlation searches
New parameters identify whether a saved search is a correlation search and the name of the correlation search. correlationsearches.conf
parameter in pre-4.6.0 versions
N/A
parameter starting in 4.6.0
savedsearches.conf
action.correlationsearch=0
action.correlationsearch.enabled=1
26
Notes This is an internal parameter and can be ignored.
A stanza for the search exists
rule_name
description
This parameter identifies a saved search as a correlation search.
action.correlationsearch.label
This parameter provides the name of the correlation search.
description
This parameter provides the description of the correlation search.
Notable event parameters for correlation searches
The action.notable parameter identifies a notable event associated with a correlation search. The parameters that describe additional details associated with the notable event now exist in the savedsearches.conf file. correlationsearches.conf
parameter starting in 4.6.0
savedsearches.conf
parameter in pre-4.6.0 versions security_domain
action.notable.param.security_domain
severity
action.notable.param.severity
rule_title
action.notable.param.rule_title
rule_description
action.notable.param.rule_description
nes_fields
action.notable.param.nes_fields
drilldown_name
action.notable.param.drilldown_name
drilldown_search
action.notable.param.drilldown_search
default_status
action.notable.param.default_status
27
default_owner action.notable.param.default_owner Related search parameters for correlation searches
Searches related to a correlation search, such as the context-generating searches associated with a correlation search that uses extreme search, are now part of a JSON blob action.correlationsearch.related_searches parameter. correlationsearches.conf
parameter in pre-4.6.0 versions
parameter starting in 4.6.0
savedsearches.conf
related_search_name = action.correlationsearch.related_searches Endpoint - Emails By Source - = [\ Context Gen "Endpoint - Emails By Source Context Gen",\ related_search_name.0 = "Endpoint - Emails By Destination Endpoint - Emails By Count - Context Gen"\ Destination Count - Context ] Gen Example correlation search stanzas from this version and previous versions The savedsearches.conf stanza for a correlation search looks as follows starting in 4.6.0.
[Access - Concurrent App Accesses - Rule] action.correlationsearch = 0 action.correlationsearch.enabled = 1 action.correlationsearch.label = Concurrent Login Attempts Detected action.email.sendresults = 0 action.notable = 0 action.notable.param.security_domain = access action.notable.param.severity = medium action.notable.param.rule_title = Concurrent Access Event Detected For $user$ action.notable.param.rule_description = Concurrent access attempts to $app1$ by $user$ from two different sources( $src1$, $src2$ ) have been detected. action.notable.param.nes_fields = user action.notable.param.drilldown_name = View access attemps by $user$ action.notable.param.drilldown_search = | datamodel Authentication Authentication search | search Authentication.user="$user$" action.risk = 1 action.risk.param._risk_object = user action.risk.param._risk_object_type = user action.risk.param._risk_score = 20 alert.suppress = 1
28
alert.suppress.fields = user alert.suppress.period = 86300s alert.track = false cron_schedule = 10 * * * * description = Alerts on concurrent access attempts to an app from different hosts. These are good indicators of shared passwords and potential misuse. disabled = True dispatch.earliest_time = -70m@m dispatch.latest_time = -5m@m enableSched = 1 is_visible = false request.ui_dispatch_app = SplunkEnterpriseSecuritySuite search = | tstats `summariesonly` count from datamodel=Authentication.Authentication by _time,Authentication.app,Authentication.src,Authentication.user span=1s | `drop_dm_object_name("Authentication")` | eventstats dc(src) as src_count by app,user | search src_count>1 | sort 0 + _time | streamstats current=t window=2 earliest(_time) as previous_time,earliest(src) as previous_src by app,user | where (src!=previous_src) | eval time_diff=abs(_time-previous_time) | where time_diff<300 In previous versions of Splunk Enterprise Security, the savedsearches.conf and correlationsearches.conf definitions for the same correlation search would look as follows. savedsearches.conf
[Access - Concurrent App Accesses - Rule] action.email.sendresults = 0 action.risk = 1 action.risk.param._risk_object = user action.risk.param._risk_object_type = user action.risk.param._risk_score = 20 alert.suppress = 1 alert.suppress.fields = user alert.suppress.period = 86300s alert.track = false cron_schedule = 10 * * * * disabled = True dispatch.earliest_time = -70m@m dispatch.latest_time = -5m@m enableSched = 1 is_visible = false request.ui_dispatch_app = SplunkEnterpriseSecuritySuite search = | tstats `summariesonly` count from datamodel=Authentication.Authentication by _time,Authentication.app,Authentication.src,Authentication.user span=1s | `drop_dm_object_name("Authentication")` | eventstats dc(src) as src_count by app,user | search src_count>1 | sort 0 + _time | streamstats current=t window=2 earliest(_time) as previous_time,earliest(src) as previous_src by app,user | where
29
(src!=previous_src) | eval time_diff=abs(_time-previous_time) | where time_diff<300 correlationsearches.conf
[Access - Concurrent App Accesses - Rule] security_domain = access severity = medium rule_name = Concurrent Login Attempts Detected description = Alerts on concurrent access attempts to an app from different hosts. These are good indicators of shared passwords and potential misuse. rule_title = Concurrent Access Event Detected For $user$ rule_description = Concurrent access attempts to $app1$ by $user$ from two different sources( $src1$, $src2$ ) have been detected. nes_fields = user drilldown_name = View access attemps by $user$ drilldown_search = | datamodel Authentication Authentication search | search Authentication.user="$user$" default_owner = default_status =
Export content as an app from Splunk Enterprise Security Export content from Splunk Enterprise Security as an app from the Content Management page. Use the export option to share custom content with other ES instances, such as migrating customized searches from a development or testing environment into production. You can export any type of content on the Content Management page, such as correlation searches, glass tables, and views. By default, only admin users can export content. To add the export capability to another role, see Adding capabilities to a role in the Installation and Upgrade Manual. 1. From the ES menu bar, select Configure > Content Management. 2. Select the check boxes of the content you want to export. 3. Click Edit Selection and select Export. 4. Type an App name. This will be the name of the app in the file system. For example, SOC_custom. 5. Select an App name prefix. If you want to import the content back into Splunk Enterprise Security without modifying the default app import conventions, select DA-ESS-. Otherwise, select No Prefix. 6. Type a Label. This is the name of the app. For example, Custom SOC app. 30
7. Type a Version and Build number for your app. 8. Click Export. 9. Click Download app now to download the app package to the search head at the location $SPLUNK_HOME/etc/apps/SA-Utils/local/data/appmaker/*. 10. Click Close to return to Content Management.
Limitations to exported content Exported content may not work on older versions of Enterprise Security. For example, the following items are included or not included in exported content. Included in exported content • Content exported from the Content Management page includes only the savedsearches.conf, correlationsearches.conf, and governance.conf settings for the selected objects. • Alert actions and response actions, including risk assignments, script names, and email addresses. Not included in exported content • Macros, script files, lookups, or any binary files referenced by the search object. • Extreme Search objects, such as the context generating search, the contexts, or the concepts referenced by the search object.
Set up adaptive response actions in Splunk Enterprise Security Adaptive response actions allow you to gather information or take other action in response to the results of a correlation search or the details of a notable event. Splunk Enterprise Security includes several adaptive response actions. See Included adaptive response actions with Splunk Enterprise Security. You can add adaptive response actions and alert actions to correlation searches, or run adaptive response actions from notable events on the Incident Review dashboard. Collect information before you start your investigation to save time at triage by adding adaptive response actions to correlation searches. Take action at triage time by running adaptive response actions from the Incident Review dashboard. 31
Add new adaptive response actions To add new adaptive response actions, you can install add-ons with adaptive response actions or create your own adaptive response actions. See Create an adaptive response action on the Splunk developer portal for information on creating adaptive response actions. See Install and deploy add-ons in the Install and Upgrade Manual.
Audit adaptive response actions Audit all adaptive response actions on the Adaptive Response Action Center.
Configure permissions for adaptive response actions Restrict certain adaptive response actions to certain roles by adjusting the permissions for adaptive response actions in the alert actions manager. You can find information about the alert actions manager in the Splunk platform documentation. • For Splunk Enterprise, see Using the alert actions manager in the Splunk Enterprise Alerting Manual. • For Splunk Cloud, see Using the alert actions manager in the Splunk Cloud Alerting Manual. In order to run adaptive response actions from the Incident Review dashboard that have credentials stored in the credential manager, you must have the appropriate capability. • For Splunk platform version 6.5.0 and later, list_storage_passwords. • For earlier Splunk platform versions, admin_all_objects.
Add an adaptive response action to a correlation search 1. On the Splunk Enterprise Security menu bar, click Configure > Content Management. 2. Click an existing correlation search, or click Create New > Correlation Search. 3. Click Add New Response Action and select the response action you want to add. 4. Complete the fields for the action. If you want, add another response action. 5. Click Save to save all changes to the correlation search. 32
Run an adaptive response action from Incident Review See Run an adaptive response action on the Incident Review dashboard.
Troubleshoot why an adaptive response action is not available to select If an adaptive response action is not available to select on the correlation search editor or Incident Review, several things could be the cause. • Your role may not have permissions to view and use the adaptive response action. See Using the alert actions manager in the Alerting Manual. • Check the alert actions manager to determine if the adaptive response actions exist in Splunk platform. See Using the alert actions manager in the Alerting Manual. • If the adaptive response actions from an add-on do not appear in Splunk Enterprise Security, but do appear in the alert actions manager, make sure that the add-on is being imported by Splunk Enterprise Security. See Install and deploy add-ons in the Install and Upgrade Manual. • If you can select the adaptive response action on the correlation search editor, but not on Incident Review, the adaptive response action might be an ordinary alert action, or the response action does not support ad-hoc invocation. See Determine whether your action supports ad hoc invocation on the Splunk developer portal.
Included adaptive response actions with Splunk Enterprise Security Splunk Enterprise Security includes several adaptive response actions. • Create a notable event. • Modify a risk score by creating a risk modifier. • Send an email. • Start a stream capture with Splunk Stream. • Ping a host. • Run Nbtstat to, for example, troubleshoot a NetBios name resolution problem. • Run Nslookup to look up the domain name of an IP address, or the IP address of a domain name. 33
Create a notable event Create a notable event when the conditions of a correlation search are met. 1. On the Splunk Enterprise Security menu bar, click Configure > Content Management. 2. Click an existing correlation search, or click Create New > Correlation Search. 3. Click Add New Response Action and select Notable to add a notable event. 4. Type a Title of the notable event on the Incident Review dashboard. Supports variable substitution from the fields in the matching event. 5. Type a Description of the notable event. Supports variable substitution from the fields in the matching event but does not support formatting of any kind such as links, line breaks, bold, or other formatting. 6. Select the Security Domain of the notable event from the drop-down list. 7. Select the Severity of the notable event from the drop-down list. The severity is used to calculate the Urgency of a notable event. 8. (Optional) Change the default owner of the notable event from the system default, unassigned. 9. (Optional) Change the default status of the notable event from the system default, New. 10. Type a drill-down name for the Contributing Events link in the notable event. 11. Type a drill-down search for the Contributing Events link in the notable event. 12. In the Drill-down earliest offset field, type the amount of time before the time of the triggering event to look for related events for the Contributing Events link in the notable event. For example 2h to look for contributing events 2 hours before the triggering event. 13. In the Drill-down latest offset field, type the amount of time after the time of the triggering event to look for related events for the Contributing Events link in the notable event. For example, 1h to look for contributing events 1 hour after the triggering event. 14. Type Next Steps for an analyst to take after triaging a notable event. Type text or click Insert Adaptive Response Action to reference a response action in the text of the next steps. You can only type plain text and links to response actions in the next steps field. Use next steps if you want to recommend response actions that should be taken in a specific order. For example, ping a host to determine if it is active on the network. If the host is active, increase the risk score by 100, otherwise, increase the risk 34
score by 50. 15. Select Recommended Actions to complement the next steps. From the list of all adaptive response actions, click the name of an action that you recommend as a triage or investigation step for this notable event to add it to the list of recommended actions that analysts can take for this notable event. You can add as many recommended actions as you like. Use recommended actions to recommend response actions that do not need to be taken in a specific order. For example, increase the risk score on a host and perform an nslookup on a domain name.
Modify a risk score with a risk modifier Modify a risk score as a result of a correlation search or in response to notable event details with the Risk Analysis adaptive response action. The risk adaptive response action creates a risk modifier event. You can view the risk modifier events on the Risk Analysis dashboard in Enterprise Security. 1. Click Add New Response Action and select Risk Analysis. 2. Type the score to assign to the risk object. 3. Type a field in the search to apply the risk score to for the Risk Object Field. For example, type "src" to specify the source field. 4. Select the Risk Object Type to apply the risk score to.
Send an email Send an email as a result of a correlation search match. Prerequisite Make sure that the mail server is configured in the Splunk platform before setting up this response action. • For Splunk Enterprise, see Configure email notification settings in the Splunk Enterprise Alerting Manual. • For Splunk Cloud, see Configure email notification settings in the Splunk Cloud Alerting Manual. Steps 1. Click Add New Response Action and select Send email. 35
2. In the To field, type a comma-separated list of email addresses to send the email to. 3. (Optional) Change the priority of the email. Defaults to Lowest. 4. Type a subject for the email. The email subject defaults to "Splunk Alert: $name$", where $name$ is the correlation search Search Name. 5. Type a message to include as the body of the email. Defaults to "The scheduled report '$name$' has run." 6. Select the check boxes of the information you want the email message to include. 7. Select whether to send a plain-text or HTML and plain-text email message.
Run a script Run a script stored in $SPLUNK_HOME/bin/scripts. 1. Click Add New Response Action and select Run a script. 2. Type the filename of the script. More information about scripted alerts can be found in the Splunk platform documentation. • For Splunk Enterprise, see Configure scripted alerts in the Splunk Enterprise Alerting Manual. • For Splunk Cloud, see Configure scripted alerts in the Splunk Cloud Alerting Manual.
Start a Stream capture Start a Stream capture to capture packets on the IP addresses of the selected protocols over the time period that you select. You can view the results of the capture session on the Protocol Intelligence dashboards. A Stream capture will not work unless you integrate Splunk Stream with Splunk Enterprise Security. See Splunk Stream integration. 1. Click Add New Response Action and select Stream Capture to start a packet capture in response to a correlation search match. 2. Type a Description to describe the stream created in response to the correlation search match. 3. Type a Category to define the type of stream capture. You can view streams by category in Splunk Stream. 36
4. Type the comma-separated event fields to search for IP addresses for the Stream capture. The first non-null field is used for the capture. 5. Type the comma-separated list of protocols to capture. 6. Select a Capture duration to define the length of the packet capture. 7. Type a Stream capture limit to limit the number of stream captures started by the correlation search.
Ping a host Determine whether a host is still active on the network by pinging the host. 1. Click Add New Response Action and select Ping. 2. Type the event field that contains the host that you want to ping in the Host Field. 3. Type the number of maximum results that the ping returns. Defaults to 1.
Run nbtstat Learn more about a host and the services that the host runs by running nbtstat. 1. Click Add New Response Action and select Nbtstat. 2. Type the event field that contains the host that you want to run the nbtstat for in the Host Field. 3. Type the number of maximum results that the nbtstat returns. Defaults to 1.
Run nslookup Look up the domain name of an IP address, or the IP address of a domain name, by running nslookup. 1. Click Add New Response Action and select Nslookup. 2. Type the event field that contains the host that you want to run the nslookup for in the Host Field. 3. Type the number of maximum results that the nslookup returns. Defaults to 1.
37
Investigations Create and track investigations in Splunk Enterprise Security Track your investigations into security incidents on an investigation timeline. This tool lets you visualize and document the progression of an incident and the steps you take during your investigation. Add notable events, other events, and information from your action history.
Start an investigation You can start an investigation in several ways in Splunk Enterprise Security. • Start an investigation from Incident Review while triaging notable events. See Add a notable event to an investigation. • Start an investigation with an event workflow action. See Add a Splunk event to an investigation. • Start an investigation from the Investigations dashboard. • Start an investigation when viewing a dashboard using the investigation bar. By default, users with the ess_admin and ess_analyst roles can start an investigation. See Access to investigations. Start an investigation Start an investigation from the Investigations dashboard. 1. Click Create New Investigation. 2. Type a title. 3. (Optional) Type a description. 4. Click Save. Start an investigation from the investigation bar When viewing dashboards in Splunk Enterprise Security, you can see an investigation bar at the bottom of the page. You can use the investigation bar to track your investigation progress from any page in Splunk Enterprise Security.
38
1. Click the icon to create an investigation. 2. Type a title. 3. (Optional) Type a description. 4. Click Save. The investigation is loaded in the investigation bar.
Track an investigation Track the progress of your investigation into a potential security incident, or plot the timeline of an attack, exfiltration, or other time-based security incident. While you conduct your investigation using Splunk Enterprise Security, you can add notable events or Splunk events that add insight to the investigation. Add searches, suppression filters, and dashboard views to the investigation from your action history. See Your action history. Record important investigation steps that you take, such as phone, email, or chat conversations as notes on the timeline. You can use notes to add relevant information like links to online press coverage, tweets, or upload screenshots and files. You can track an investigation using the investigation bar, maintaining context for your investigation as you move across different dashboards in Enterprise Security. • Load an existing investigation timeline into the bar by clicking the icon and selecting an investigation. You can only add investigations to the investigation bar if you are a collaborator with write permissions on the investigation. • Click the icon to view a timeline view of your investigation and review the progression of events in the investigation. Add a notable event to an investigation You can add a notable event to an investigation from the Incident Review dashboard. See Add a notable event to an investigation. Add a Splunk event to an investigation 1. Expand the event details to see the Event Actions menu and other information. 2. Click Event Actions and select Add to Investigation. 3. A tab opens. Select from existing investigations, or create one. 39
4. Click Save. Add an entry from your action history to an investigation The action history stores a history of the actions that you have performed in Splunk Enterprise Security, such as searches that you have run, dashboards you have viewed, and per-panel filtering actions that you have performed. Add an entry from your action history to an investigation from a dashboard with the investigation bar. You can filter action history items by type or time to find the action history items. 1. From the investigation bar, click the icon. 2. Find the actions that you want to add to the investigation.
1. The most recent actions that you've taken display in the action history dialog box. You can only add actions from your own action history. 2. Search, sort by time, or filter by action type (search run, dashboard viewed, panel filtered, notable status change, or notable events suppressed) to locate the action you want to add. 3. For searches, click the plus sign to view the full search string and verify that you are adding the correct search. 3. Select the check box next to the action or actions that you want to add to the investigation timeline. 4. Click Add to Investigation. The actions are added to the investigation that you are viewing or that is selected in the investigation bar. See Your action history.
40
Add a note Add a note to an investigation to record investigation details or add attachments. You can add a note from dashboards in Splunk Enterprise Security. 1. From the investigation bar, click the icon. 2. Type a title. For example, "Phone conversation with police." 3. (Optional) Select a time. The default is the current date and time. For example, select the time of the phone call. 4. (Optional) Type a description. For example, a note to record a phone conversation might include the description: Called the police. Spoke with Detective Reggie Martin. Discussed an employee stealing identities from other employees.
5. (Optional): Attach a file to the note. 1. From the note, click the icon or drag the file onto the note. 2. Select a file to add from your computer. The maximum file size is 4 MB. You can add multiple files to a note. The first file you add to the note previews on the investigation timeline. 6. Click Add to Investigation to add the note to the open investigation or click Save as Draft. Note: When you save a note as a draft, it stays associated with the investigation that was selected when you created the note but does not appear on the investigation. Retrieve draft notes by clicking the icon.
41
Make changes to investigation entries Make changes to the entries on an investigation from the list view or the timeline view. Delete a single investigation entry from the timeline view 1. Find the entry on the timeline view. 2. Click Action > Delete Entry. 3. Click Delete to confirm deleting the entry. Delete investigation entries from the list view 1. Click List to view the investigation as a list of entries. 2. Select the check box next to the investigation entries that you want to delete. 3. Click Action and select Delete. 4. Click Delete to confirm deleting the entry. Change a note 1. Find the note in the investigation and open the note for editing. 1. From the timeline view of the investigation click Action > Edit Note 2. From the list view of the investigation click Edit in the Actions column. 2. Make changes. For example, add a new attachment and add a sentence to the description describing the new attachment. 3. Remove a file attachment by clicking the X next to the file name. 4. Click Save. Close an investigation You can indicate that na investigation is closed in several ways. • Change the title to include the word "Closed" so that you can filter on closed investigations on My Investigations. • Add a note at the end of the investigation to identify the investigation as closed. Change the title and description of an investigation You can also change the title and description of an investigation. For example, change the name of the investigation as your investigation progresses to more 42
accurately describe the security incident you are investigating. 1. From the investigation bar, click the click Edit. 2. Change the title or description. 3. Click Save.
icon. From the investigation view,
Run a quick search from the investigation bar Run a search without opening the search dashboard by clicking Quick Search on the investigation bar. • Add the search to the investigation in the investigation bar by clicking Add to Investigation. • Use the Event Actions to add specific events in the search results to an investigation. • To save the search results at investigation time, click Export to export the search results as a CSV file. Add the search results as an attachment to a note on the investigation. • Click Open in Search to view the search results on the Search dashboard. • Enlarge or shrink your view of the search results by clicking and dragging the corner of the window. Double click to expand the search view to cover most of your screen, or double click again to shrink it.
Collaborate with another analyst You can collaborate with other analysts on an investigation. Add a collaborator to an investigation 1. Open the investigation that you want to add a collaborator to. 2. Click the icon. 3. Type the name of the person you want to add and select their name from
the list to add them to the investigation. 4. Their initials appear in a circle to confirm that they were added. 43
You can add any Splunk user in your deployment as a collaborator. By default, a collaborator has write permissions on the investigation. View the collaborators assigned to an investigation You can view the collaborators assigned to an investigation from an individual investigation or from the Investigations dashboard. • Hover over the collaborator icons to see the names of the collaborators on your investigation. • If a collaborator does not have write permissions for an investigation, the icon is gray and (read-only) is appended to their name. • Click the icon of a collaborator to see information about them. See their name and the permissions that the user has for the investigation.
Make changes to the collaborators on an investigation If you are a collaborator on an investigation with write permissions, you can change the permissions of other collaborators on the investigation. 1. Click the icon of a collaborator. 2. Change the Write permissions. By default, all collaborators have Yes for Write permissions. All investigations must have at least one collaborator with write permissions. You can remove a collaborator if they are not the only collaborator on the investigation with write permissions. 1. Click the icon of a collaborator. 2. Click Remove.
Review an investigation Revisit past investigations, or view a current investigation by clicking the title from the investigation bar or from the Investigations dashboard. Users with the capability to manage all investigations can view all investigations. Only 44
collaborators on an investigation with write permissions can edit an investigation. See Access to investigations. Review an investigation for training or research purposes. Click an entry on an investigation to see all details associated with it. • For notes with file attachments, click the file name to download the file attachment. • For notable events, click View on Incident Review to open the Incident Review dashboard filtered on that specific notable event. • For action history entries, you can repeat the previously-performed action. For a search action history entry, click the search string to open it in search. For a dashboard action history entry, click the dashboard name to view the dashboard.
Gain insight into an attack or investigation by viewing the entire investigation timeline or view only part of it by expanding or contracting the timeline.
Click the timeline to move it and scan the entries. View a chronological list of all timeline entries by clicking the list icon, or refine your view of the timeline using filters. You can filter by type or use the Filter box to filter by title.
Share or print an investigation To share an investigation with someone that does not use Splunk Enterprise Security, such as for auditing purposes, you can print any investigation or save any investigation as a PDF.
45
1. From the investigation, click the icon. Splunk Enterprise Security generates a formatted version of the investigation timeline with entries in chronological order. 2. Print the investigation or save it as a PDF using the print dialog box options.
Example investigation workflow 1. You are notified of a security incident that needs investigation through a notable event, an alert action, or by an email, ticket from the help desk, or a phone call. 2. Create an investigation in Splunk Enterprise Security. 3. If you must work with someone else on the investigation, add them as a collaborator. 4. Investigate the incident. While you investigate, add helpful or insightful steps to the investigation. 1. Run searches, adding useful searches to the investigation from your action history with the investigation bar or relevant events using event actions. This makes it easy to replicate your work for future, similar investigations, and to make a comprehensive record of your investigation process. 2. Filter dashboards to focus on specific elements, like narrowing down a swim lane search to focus on a specific asset or identity on the asset or identity investigator dashboards. Add insightful filtering actions from your action history to the investigation using the investigation bar. 3. Triage and investigate potentially related notable events. Add relevant notable events to the investigation. 4. Add notes to record other investigation steps, such as notes from a phone call, email or chat conversations, links to press coverage or social media posts. Upload files like screenshots or forensic investigation files. 5. Complete the investigation and add a note to record a summary of your findings.
Your action history While you investigate an attack or other security incident, actions that you take in Splunk Enterprise Security are recorded in your action history. You can only view your own entries in your action history. After you add an item to an investigation, all collaborators on the investigation can view that entry. Your action history tracks the following types of actions using saved searches: 46
• Dashboards you visit • Searches you run • Per-panel filtering actions you take • Changes you make to a notable event • Changes you make to the suppression filters of a notable event See Data sources for investigations. Splunk Enterprise Security tracks these actions to help you add context to an investigation, audit an investigation, and give a complete history of actions taken during an investigation that resulted in relevant findings. For example, if you run a search that gives helpful information for an investigation, you can add that search to the investigation. You can then find that search string in the investigation, run the search again, or revisit a search to save it as a report when the investigation is over.
Manage security investigations in Splunk Enterprise Security You can manage, start, and track investigations on the Investigations dashboard. View or filter the investigations assigned to you, or create one. You can view all investigations that you collaborate on using this dashboard. Users with admin permissions can also view all investigations that exist in Splunk Enterprise Security.
By default, analysts that use this page only see investigations assigned to them unless they also have the capability to manage all investigations.
Manage your investigations Manage ongoing investigations from the Investigations dashboard. You can see the titles, descriptions, creation times, and collaborators on the investigations assigned to you or on all investigations in Splunk Enterprise Security.
47
Filter investigations Quickly find an investigation or refine the list of investigations by filtering. Type in the Filter box to search the title and description fields of investigations. Delete investigations You can delete individual or several investigations on the Investigations dashboard. After you delete a timeline, you cannot restore it. Assess the audit or research value of a timeline before deleting it. 1. Select the check box next to the investigation or investigations you want to delete. 2. Click Edit Selection and click Delete. 3. Click Delete to confirm deleting the timeline. Edit an investigation Edit the title or description of an investigation by opening the investigation. Only collaborators with write permissions on an investigation can make changes to an investigation. 1. Find the investigation you want to edit on the Investigations dashboard. 2. Click the name of the investigation to open it. See Create and track investigations in Splunk Enterprise Security.
Data sources for investigations Splunk Enterprise Security stores investigation information in several KVStore collections. The investigations on the Investigations dashboard, items added to the investigation, and attachments added to the investigation each have their own collection. See Investigations in the Dashboard requirements matrix. Investigation details from investigations created in pre-4.6.0 versions of Splunk Enterprise Security are stored in two KV Store collections: investigative_canvas, investigative_canvas_entries. Those collections are preserved in version 4.6.0 but the contents are added to the new investigation KV Store collections.
48
Action history data sources Action history items do not immediately appear in your action history after you perform an action. You can only view action history items and add them to an investigation after the saved searches that create action history items run. By default, the searches run every two minutes. Five saved searches create action history items. • Dashboard Views - Action History • Search Tracking - Action History • Per-Panel Filtering - Action History • Notable Suppression - Action History • Notable Status - Action History View the searches by navigating to Configure > Content Management and using the filters on the page. If you change these saved searches, action history items might stop appearing in your action history. To exclude a search from your action history, use the Action History Search Tracking Whitelist lookup. See Configure lists and lookups.
Access to investigations Users with the ess_admin role can create, view, and manage investigations by default. Users with the ess_analyst role can create and edit investigations. Make changes to capabilities with the Permissions dashboard. • To allow other users to create or edit an investigation, add the Use Investigations capability to their role. Users can only make changes on investigations on which they are a collaborator. • To allow other users to manage, view, and delete all investigations, add the Manage all investigations capability to their role. See Configure users and and roles in the Installation and Upgrade Manual. You can manage who can make changes to an investigation by setting write permissions for collaborators on a specific investigation. By default, all collaborators have write permissions for the investigations to which they are added, but other collaborators on the timeline can change those permissions to read-only. See Make changes to the collaborators on an investigation. After a user creates an investigation, any user with the Manage all investigations capability can view the investigation, but only the collaborators on the investigation can edit the investigation. You cannot view the investigation KV 49
Store collections as lookups.
50
Assets and Identities Asset and Identity dashboards The Identity domain dashboards provide information about the assets and identities defined in Splunk Enterprise Security. See Add asset and identity data to Splunk Enterprise Security for instructions on defining assets and identities.
Asset Center dashboard Use the Asset Center dashboard to review and search for objects in the asset data added to Enterprise Security. The asset data represents a list of hosts, IP addresses, and subnets within the organization, along with information about each asset. The asset list correlates asset properties to indexed events, providing context such as asset location and the priority level of an asset. Dashboard filters Use the available dashboard filters to refine the results displayed on the dashboard panels. Filter by
Description
Asset
A known or unknown asset
Priority
Filter by the Priority field in the Asset table.
Business Unit A group or department classification for the asset. Category
Filter by the Category field in the Asset table.
Owner
Filter by the Owner field in the Asset table.
Time Range Select the time range to represent. Dashboard Panels Panel
Description
Assets by Priority
Displays the number of assets by priority level. The drilldown opens a search with the selected priority level.
Assets by Business Unit
Displays the relative amount of assets by business unit. The drilldown opens a search with the selected business unit.
51
Assets by Category
Displays the relative amount of assets by category. The drilldown opens a search with the selected category.
Asset Information
Shows all assets that match the current dashboard filters. The drilldown opens the Asset Investigator dashboard if the "ip", "nt_host", "mac", or "dns" fields are selected. Any other field will open a search with the selected field.
Data sources The reports in the Asset Center dashboard reference fields in the Asset and Identities data model. Relevant data sources include lists of assets and identities collected and loaded as lookups, scripted inputs, or search-extracted data.
Identity Center dashboard Use the Identity Center dashboard to review and search for objects in the identity data added to Enterprise Security. Identity data represents a list of account names, legal names, nicknames, and alternate names, along with other associated information about each identity. The identity data is used to correlate user information to indexed events, providing additional context. Filtering Identities in Identity Center The filter for the Identity Center dashboard uses a key=value pair search field. To filter identities, enter a key=value pair instead of a name or text string. Some sample key=value pairs are email=*acmetech.com or nick=a_nickname. Use the available dashboard filters to refine the results displayed on the dashboard panels. Filter by
Description
Username
A known or unknown user
Priority
Filter by the Priority field in the Identities table
Business Unit
A group or department classification for the identity.
Category
Filter by the Category field in the Identities table.
Watchlisted Identities Only
Filter by the identities tagged as "watchlist" in the Identities table.
Time Range
Select the time range to represent. 52
Dashboard Panels Panel
Description
Identities by Priority
Displays the count of Identities by priority level. The drilldown opens a search with the selected priority level.
Identities by Business Unit
Displays the relative number of Identities by business unit. The drilldown opens a search with the selected business unit.
Identities by Category
Displays the relative number of Identities by category. The drilldown opens a search with the selected category.
Identity Information
Shows all assets that match the current dashboard filters. The drilldown opens the Identity Investigator dashboard if you select the identity field. Any other field opens a search with the selected field.
Data sources The reports in the Identity Center dashboard reference fields in the Asset and Identities data model. Relevant data sources include lists of assets and identities collected and loaded as lookups, scripted inputs, or search extracted data.
Session Center dashboard The Session Center dashboard provides an overview of network sessions. Network sessions are used to correlate network activity to a user using session data provided by DHCP or VPN servers. Use the Session Center to review the session logs and identify the user or machine associated with an IP address used during a session. Dashboard Panels Panel
Description
Sessions Over Time
Displays the total count of network sessions over time. The drilldown opens a search with the selected session and time range.
Session Details
Displays the top 1000 network sessions that have been most recently opened, based on the session start time. The drilldown opens a search with the selected session details.
53
Troubleshooting Identity dashboards The dashboards reference data from various data models. Without the applicable data, the panels will remain empty. See Dashboard Troubleshooting in this manual.
Asset and Identity Investigator dashboards The Asset and Identity Investigator dashboards visually aggregate security-related events over time using category-defined swim lanes. Each swim lane represents an event category, such as authentication, malware, or notable events. The swim lane uses a heat map to display periods of high and low activity. The color saturation on the swim lane corresponds to the event density for a given time. For example, high activity periods display a darker color. An analyst can visually link activity across the event categories and form a complete view of a host or user's interactions in the environment.
Asset Investigator The Asset Investigator dashboard displays information about known or unknown assets across a pre-defined set of event categories, such as malware and notable events. Use the Asset Investigator dashboard You can use the Asset Investigator dashboard to triage an asset's interactions
with the environment. The dashboard contains multiple event categories, with each one represented by its own swim lane. Each event category contains relevant events that correspond to a data model. For example, the Malware Attacks swim lane displays events 54
from an anti-virus management or other malware data source, limited to the asset searched. Multiple swim lanes are displayed at once to make it easier for you to track the actions of an asset across event categories. Additionally, you can use this dashboard for ad-hoc searching. 1. Browse to Security Intelligence > User Intelligence > Asset Investigator. 2. Type the host name or IP address in the search bar with an optional wildcard. 3. Set a time range and click Search. A workflow for asset investigation To initiate the asset investigation workflow, perform a workflow action from any dashboard that displays events with network source or destination addresses. For more information, see Workflow Actions in this manual. 1. Look at the asset description at the top of the dashboard to confirm that you are viewing the asset you would like to investigate. All events displayed in the swim lanes are limited to the selected asset. 2. Use the time range picker to narrow down the general time range you are interested in. Use the time sliders to isolate periods of interesting events or peak event counts. 3. Add or change the swim lanes using the edit menu. For example, to display data collected on an asset from packet analysis tools, change the selected collection from Default to Protocol Intelligence, which represents packet capture data. See Edit the swim lanes. 4. Review individual and grouped events. After selecting an event, you can use the Event Panel to examine common fields represented in the individual or grouped events. 5. If there is an event or pattern that you want to share or investigate further, you can do this using the Event Panel. 1. Click Go to Search to view a drilldown of the selected events. 2. Click Share for a shortened link to the current view. 3. Click Create Notable Event to open a dialog box to create an ad-hoc notable event. See Notable events in this manual. Data sources The event categories in the Asset Investigator dashboard display events from a number of data models containing an asset or host field. In any given time selection, a selected asset may not have data to display in one or more event 55
categories. When a data model search returns no matching events, the swim lane displays "Search returned no results." See Dashboard Troubleshooting in this manual.
Identity Investigator The Identity Investigator dashboard displays information about known or unknown user identities across a pre-defined set of event categories, such as change analysis or malware. Use the Identity Investigator dashboard You can use the Identity Investigator dashboard to triage a user identity's interactions with the environment.
The dashboard contains multiple event categories, with each one represented by its own swim lane. Each event category contains relevant events that correspond to a data model. For example, the Malware Attacks swim lane displays events from an anti-virus management or other malware data source, limited to the user identity or credential searched. Multiple swim lanes are displayed at once to make it easier for you to track the actions of a user across event categories. Additionally, you can use this dashboard for ad-hoc searching. 1. Browse to Security Intelligence > User Intelligence > Identity Investigator. 2. Type a user credential in the search bar. Optionally, include a wildcard. 3. Set a time range and click Search.
56
A workflow for identity investigation The identity investigation workflow is initiated through a workflow action from any dashboard that displays events with network source or destination address. For more information, see Workflow Actions in this manual. 1. Look at the identity description at the top of the dashboard to confirm that you are viewing the identity you would like to investigate. All events displayed in the swim lanes are limited to the selected identity. 2. Use the time range picker to narrow down the general time range you are interested in. Use the time sliders to isolate periods of interesting events or peak event counts. 3. Add or change swim lanes by using the edit menu. For example, to display identity information collected for user activity monitoring, change the selected collection from Default to User Activity. See Edit the swim lanes. 4. Review individual and grouped events. After selecting an event, you can use the Event Panel to examine common fields represented in the individual or grouped events. 5. If there is an event or pattern that you would like to share or investigate further, you can do this using the Event Panel. 1. Click Go to Search to view a drilldown of the selected events. 2. Click Share for a shortened link to the current view. 3. Click Create Notable Event to open a dialog box to create an ad-hoc notable event. See Notable events in this manual. Data sources The event categories in the Identity Investigator dashboard display events from a number of data models containing an identity or a user field. In any given time selection, an identity may not display data in one or more event categories. When a data model search returns no matching events, the swim lane displays "Search returned no results." See Dashboard Troubleshooting in this manual.
Edit the swim lanes You can add or remove swim lanes from the Entity Investigator dashboards by opening the Edit Lanes customization menu. The Entity Investigator dashboards support the addition of custom swim lanes bundled with add-ons or created using ES Content Management. 1. Choose Edit at the top of the dashboard. 2. Select the radio button for a Custom collection. 3. Select a checkbox to add a swim lane to the dashboard. 57
4. Deselect a checkbox to remove a swim lane from the dashboard. 5. Click the color next to a swim lane to change it. 6. Click the X to close the edit menu. The order of swim lanes can be changed on the dashboard and does not require the Edit Lanes menu. 1. Select a swim lane category. 2. Drag and drop the swim lane where you would like it. The Asset Investigator has additional, optional swim lanes in the collection Protocol intelligence to display data collected about an asset using packet analysis tools. The Identity Investigator has additional, optional swim lanes in the collection User Activity to display data collected about an identity for user activity monitoring. Swimlane Name
Asset or Identity dashboard
Description
All Authentication
Both
Matches events in the Authentication data model.
All Changes
Both
Matches events in the Change Analysis data model.
Threat List Activity
Both
Matches events in the Threat Lists data model.
IDS Attacks
Both
Matches events in the Intrusion Detection data model.
Malware Attacks
Both
Matches events in the Malware data model.
Notable Events
Both
Matches events in the Notable index.
Risk Modifiers
Both
Matches events in the Risk Analysis data model.
DNS Errors
Asset only
Matches events in the Network Resolution DNS data model.
Cloud Emails
Asset only
Matches events in the Email data model.
SSL Expired Certs
Asset only
Matches events in the Certificates data model.
58
HTTP Errors
Asset only
Matches events in the Web data model.
Non-corporate Emails
Identity only
Matches events in the Email data model.
Non-corporate Web Uploads
Identity only
Matches events in the Web data model.
Remote Access
Identity only
Matches events in the Authentication data model.
Ticket Activity
Identity only
Matches events in the Ticket Management data model.
Watchlisted Sites
Identity only
Matches events in the Web data model.
Troubleshooting Asset and Identity Investigator dashboards The Asset and Identity Investigator dashboards display events from the data model named in each swim lane. When a data model search returns no matching events, the swim lane displays "Search returned no results." See "Dashboard Troubleshooting" in this manual for more.
User Activity Monitoring User Activity The User Activity dashboard displays panels representing common risk-generating user activities such as suspicious website activity. For more information about risk scoring, see Risk scoring in this manual. Dashboard filters You can use the available dashboard filters to refine the results displayed on the dashboard panels. The filters do not apply to key security indicators. Filter by
Description
User
A known or unknown identity
Business Unit
A group or department classification for the identity.
Watchlisted Users Designates a monitored identity. Time Range
Select the time range to represent. 59
Dashboard Panels Panel
Description
Key Indicators
Displays the metrics relevant to the dashboard sources over the past 48 hours. Key indicators represent summary information and appear at the top of the dashboard. See Key indicators in this manual.
Users By Risk Scores
Displays the top 100 highest risk users. As an insider threat can represent subtle and indirect changes in behavior, this panels assists an analyst in focusing on the riskiest users in the organization. The drilldown opens the Identity Investigator dashboard and searches on the selected user.
Non-corporate Web Uploads
Displays high volume upload and download activity by user. An irregular pattern of upload or download activity can be an indicator of data exfiltration. The drilldown opens the Identity Investigator dashboard and searches on the selected user.
Non-corporate Email Activity
Displays the top 100 users performing high volume email activity to non-corporate domains. A pattern of large or high volume email activity can be an indicator of data exfiltration. The drilldown opens the Identity Investigator dashboard and searches on the selected user.
Watchlisted Site Activity
Displays web access by user. Accessing specific categories of web sites while using workplace resources and assets can be an indicator of insider threat activity. The drilldown opens the Identity Investigator dashboard and searches on the selected user.
Remote Access
Displays remote access authentication by user. A user performing risky web or email activity while using remote access services can be an indicator of data exfiltration, or exploited credentials. The drilldown opens the Identity Investigator dashboard and searches on the selected user.
Ticket Activity
Displays ticketing activity by user. A user performing risky web or email activity while filing tickets to provide additional services or internal access can be an indicator of data exfiltration, or exploited credentials. The drilldown opens the Identity Investigator dashboard and searches on the selected user.
60
Data sources The reports in the User Activity dashboard reference data fields in multiple sources. Relevant data sources include proxy servers, gateways and firewalls, or other sources that reference a distinct user. In order for the dashboards to populate, new lookup content and fields in the identities list must be added. For a list of additional data sources, see Dashboard Troubleshooting in this manual.
Access Anomalies The Access Anomalies dashboard displays concurrent authentication attempts from different IP addresses and improbable travel anomalies using internal user credentials and location-relevant data. Dashboard filters Use the available dashboard filters to refine the results displayed on the dashboard panels. Filter by
Description
Action
A successful or failed authentication attempt.
App
The application field in the authentication data model.
User
A known or unknown identity.
Business Unit A group or department classification for the identity. Time Range Select the time range to represent. Dashboard Panels Panel
Geographically Improbable Accesses
Concurrent Application Accesses
Description Displays users that initiated multiple authentication attempts separated by an improbable time and distance. Authenticating from two geographically distant locations in a time frame lower than typical transportation methods provide can be an indicator of exploited credentials. The drilldown opens the Access Search dashboard and searches on the selected user. Displays users that initiated multiple authentication attempts from unique IP addresses within a short time span. This pattern of authentication can be an indicator of shared or stolen credentials. The drilldown redirects the page to the 61
Access Search dashboard and searches on the selected user. Data sources The reports in the Access Anomalies dashboard reference data fields in the Authentication data model. Relevant data sources include proxy servers, gateways and firewalls, or other sources that reference a distinct user. See Dashboard Troubleshooting in this manual.
Troubleshooting This dashboard references data from various data models. Without the applicable data, the dashboards will remain empty. See Dashboard Troubleshooting in this manual.
Add asset and identity data to Splunk Enterprise Security Splunk Enterprise Security uses an asset and identity system to correlate asset and identity information with events to enrich and provide context to your data. This system takes information from external data sources to populate lookups, which are then correlated with events at search time. See Configure asset and identity correlation in Splunk Enterprise Security. Add asset and identity data to Splunk Enterprise Security to take advantage of asset and identity correlation. 1. Collect and extract asset and identity data. 2. Format the asset or identity list as a lookup. 3. Configure a new asset or identity list. 4. Define identity formats on the identity configuration page. 5. Splunk Enterprise Security merges the asset and identity lists. 6. Verify that your asset or identity data was added to Splunk Enterprise Security. 7. Configure asset and identity correlation in Splunk Enterprise Security.
Collect and extract asset and identity data Collect and extract your asset and identity data in order to add it to Splunk Enterprise Security. In a Splunk Cloud deployment, work with Splunk 62
Professional Services to design and implement an asset and identity collection solution. Determine where the asset and identity data in your environment is stored, and collect and update your asset and identity data automatically to reduce the overhead and maintenance that manual updating requires and improve data integrity. • Use Splunk DB Connect or another Splunk platform add-on to connect to an external database or repository. • Use scripted inputs to import and format the lists. • Use events indexed in the Splunk platform with a search to collect, sort, and export the data to a list. See Example methods of adding asset and identity data to Splunk Enterprise Security. Suggested collection methods for assets and identities. Technology
Asset or Identity data
Collection methods
Active Directory
Both
SA-ldapsearch and a custom search.
LDAP
Both
SA-ldapsearch and a custom search.
CMDB
Asset
DB Connect and a custom search.
ServiceNow
Both
Splunk Add-on for ServiceNow
Asset Discovery
Asset
Asset Discovery App
Bit9
Asset
Splunk Add-on for Bit9 and a custom search.
Cisco ISE
Both
Splunk Add-on for Cisco ISE and a custom search.
Microsoft SCOM
Asset
Splunk Add-on for Microsoft SCOM and a custom search.
Okta
Identity
Splunk Add-on for Okta and a custom search.
Sophos
Asset
Splunk Add-on for Sophos and a custom search.
Symantec Endpoint Protection
Asset
Splunk Add-on for Symantec Endpoint Protection and a custom search. 63
Splunk platform
Asset
Add asset data from indexed events in Splunk platform.
Format the asset or identity list as a lookup Create a plain text, CSV-formatted file with Unix line endings. Use the correct headers for the CSV file. See Asset and identity lookup header and field reference. For an example asset list, review the demo_assets.csv file in SA-IdentityManagement/package/lookups. If you use a custom search to generate a lookup, make sure that the lookup produced by the search results contains fields that match the headers.
Configure a new asset or identity list Configure a new asset or identity list as a lookup in Splunk Enterprise Security. This process creates the lookup in Splunk Enterprise Security and defines the lookup for the merge process. Prerequisites The lookup file must be a plain text, CSV-format file with Unix line endings and include a .csv filename extension. Add the new lookup table file. 1. From the Splunk menu bar, select Settings > Lookups > Lookup table files. 2. Click New. 3. Select a Destination App of SA-IdentityManagement. 4. Select the lookup file to upload. 5. Type the Destination filename that the lookup table file should have on the search head. The name should include the filename extension. For example, network_assets_from_CMDB.csv 6. Click Save to save the lookup table file and return to the list of lookup table files. Set permissions on the lookup table file to share it with Splunk Enterprise Security. 1. From Lookup table files, locate the new lookup table file and select Permissions. 2. Set Object should appear in to All apps. 3. Set Read access for Everyone. 64
4. Set Write access for admin or other roles. 5. Click Save. Add a new lookup definition. 1. From the Splunk menu bar, select Settings > Lookups > Lookup definitions. 2. Click New. 3. Select a Destination App of SA-IdentityManagement. 4. Type a name for the lookup source. This name must match the name defined later in the input stanza definition on the Identity Management dashboard. For example, network_assets_from_CMDB. 5. Select a Type of File based. 6. Select the lookup table file created. For example, select network_assets_from_CMDB.csv. 7. Click Save. Set permissions on the lookup definition to share it with Splunk Enterprise Security. 1. From Lookup definitions, locate the new lookup definition and select Permissions. 2. Set Object should appear in to All apps. 3. Set Read access for Everyone. 4. Set Write access for admin or other roles. 5. Click Save. Add an input stanza for the lookup source. 1. Return to Splunk Enterprise Security. 2. From the Splunk ES menu bar, select Configure > Data Enrichment > Identity Management. 3. Click New. 4. Type the name of the lookup. For example, network_assets_from_CMDB. 5. Type a Category to describe the new asset or identity list. For example, CMDB_network_assets. 6. Type a Description of the contents of the list. For example, network assets from the CMDB. 7. Type asset or identity to define the type of list. For example, asset.
65
8. Type a Source that refers to the lookup definition name. For example, lookup://network_assets_from_CMDB.
Define identity formats on the identity configuration page Define the identity formats that identify users in your environment on the Identity Lookup Configuration page. Changes made on the Identity Lookup Configuration page modify the identityLookup.conf file. 1. From the Splunk ES menu bar, select Configure > Data Enrichment > Identity Lookup Configuration. 2. (Optional) Deselect the check box for Email if email addresses do not identify users in your environment. 3. (Optional) Deselect the check box for Email short if the username of an email address does not identify users in your environment. 4. (Optional) Select the check box for Convention if you want to define custom conventions to use to identify users. Click Add a new convention to add a custom convention. For example, identify users by the first 3 letters of their first name and last name with the convention first(3)last(3). 5. (Optional) Select the check box for Case Sensitive to require case sensitive identity matching. Case sensitive identity matching produces fewer matches. 6. Click Save.
Splunk Enterprise Security merges the asset and identity lists Splunk Enterprise Security merges the asset and identity lists every five minutes with a saved search. See How Splunk Enterprise Security processes and merges asset and identity data.
Verify that your asset or identity data was added to Splunk Enterprise Security Verify that your asset or identity data was added to Splunk Enterprise Security by searching and viewing dashboards. Review asset lookup data Verify that a specific asset record exists in the asset lookup.
66
1. Choose an asset record with data the ip, mac, nt_host, or dns fields from an asset list. 2. Search for it in Splunk Web. | makeresults | eval src="1.2.3.4" | `get_asset(src)`
View the available assets on the Asset Center dashboard. See Asset Center dashboard in this manual. View all available assets with the assets macro. | `assets`
View all available assets using the data model. |`datamodel("Identity_Management", "All_Assets")` |`drop_dm_object_name("All_Assets")`
Review identity lookup data Verify that a specific identity record exists in the identity lookup. 1. Choose an identity record with data in the identity field. 2. Search for it in Splunk Web. | makeresults | eval user="VanHelsing" | `get_identity4events(user)`
View the available identities on the Identity Center dashboard. See Identity Center dashboard. View all available identities with the identities macro. | `identities`
View all available identities in the data model. |`datamodel("Identity_Management", "All_Identities")` |`drop_dm_object_name("All_Identities")`
67
Configure asset and identity correlation in Splunk Enterprise Security After you Add asset and identity data to Splunk Enterprise Security, configure asset and identity correlation in Splunk Enterprise Security.
About asset and identity correlation in Splunk Enterprise Security To effectively detect security intrusions, an organization must be able to correlate events in log data with specific assets and identities that may be responsible for, or affected by the intrusion. Splunk Enterprise Security compares indexed events with asset and identity data in the asset and identity lists to provide data enrichment and context. The comparison process uses automatic lookups. You can find information about automatic lookups in the Splunk platform documentation. • For Splunk Enterprise, see Make your lookup automatic in the Splunk Enterprise Knowledge Manager Manual. • For Splunk Cloud, see Make your lookup automatic in the Splunk Cloud Knowledge Manager Manual. Asset and identity correlation enriches events with asset and identity data at search time. • Asset correlation compares events that contain data in any of the src, dest, or dvc fields against the merged asset lists for matching IP address, MAC address, DNS name, or Windows NetBIOS names. Asset correlation no longer occurs automatically against the host or orig_host fields. • Identity correlation compares events that contain data in any of the user or src_user fields against the merged identity lists for a matching user or session. • Enterprise Security adds the matching output fields to the event. For example, correlation on the asset src field results in additional fields such as src_is_expected and src_should_timesync. Asset and identity correlation allows you to determine whether multiple events can relate to the same asset or identity. You can also perform actions on the identity and asset fields added to events to open additional searches or dashboards scoped to the specific asset or identity. For example, open the Asset Investigator dashboard on a src field.
68
Configure asset and identity correlation Choose whether to enable or disable asset and identity correlation. You can restrict correlation to occur only for select sourcetypes. 1. From the Splunk ES menu bar, select Configure > Data Enrichment > Identity Correlation. 2. Select Enable correlation or Disable correlation or Enable selectively by sourcetype. 3. If you choose Enable selectively by sourcetype, type a sourcetype and select the check box for asset and/or identity. Note: Disabling asset and identity correlation completely prevents events from being enriched with asset and identity data from the asset and identity lookups. This may prevent correlation searches, dashboards, and other functionality from working as expected. Consult with Splunk Professional Services or Splunk Support before disabling asset and identity correlation.
How Splunk Enterprise Security processes and merges asset and identity data Splunk Enterprise Security takes the asset and identity data that you add as lookups and generates combined lookup files. Splunk Enterprise Security uses the generated lookup files to correlate asset and identity data with events using automatic lookups. 1. You collect asset and identity data from data sources using an add-on and a custom search or manually with a CSV file. See Collect and extract asset and identity data. 2. You format the data as a lookup, using a search or manually with a CSV file. See Format the asset or identity list as a lookup. 3. You configure the list as a lookup table, definition, and input. See Configure a new asset or identity list. 1. Splunk Enterprise Security identity manager modular input detects changed content in the identity_manager:// and changes to stanzas in the input. 4. You configure any settings in the identity lookup configuration setup. See Define identity formats on the identity configuration page. 1. Splunk Enterprise Security identity manager modular input updates settings in the transforms.conf stanza identity_lookup_expanded
69
5. Splunk Enterprise Security identity manager modular input updates the macros used to identify the input sources based on the currently enabled stanzas in inputs.conf. For example, update the `generate_identities` macro dynamically based on the conventions specified on the Identity Lookup Configuration page. 6. Splunk Enterprise Security identity manager modular input dispatches lookup generating saved searches if it identifies changes that require the asset and identity lists to be merged. 7. Splunk Enterprise Security merges all configured and enabled asset and identity lists using lookup generating saved searches. 1. The primary saved searches concatenate the lookup tables referenced by the identity manager input, generate new fields, and output the concatenated asset and identity lists into target lookup table files. 2. Secondary saved searches generate lookup tables for asset categories, identity categories, and asset PCI domains (in Splunk App for PCI Compliance). 8. You verify that the data looks as expected. See Verify that your asset or identity data was added to Splunk Enterprise Security. The merging of identity and asset lookups does not validate or de-duplicate input. Errors from the identity manager modular input are logged in identity_manager.log. This log does not show data errors.
Merging assets and identities creates new lookup files After the merging process completes, four lookups store your asset and identity data. Function
String-based asset correlation
Table name
Saved search
Lookup name
Identity Asset LOOKUP-zu-asset_lookup_by_str-de String LOOKUP-zu-asset_lookup_by_str-dv Matches LOOKUP-zu-asset_lookup_by_str-sr - Lookup Gen
assets_by_str.csv
CIDR assets_by_cidr.csv subnet-based asset correlation
Identity - LOOKUP-zv-asset_lookup_by_cidr-d Asset LOOKUP-zv-asset_lookup_by_cidr-d CIDR LOOKUP-zv-asset_lookup_by_cidr-s Matches - Lookup 70
Gen String-based identity correlation
Default field correlation
identities_expanded.csv
Identity Identity LOOKUP-zy-identity_lookup_expand Matches LOOKUP-zy-identity_lookup_expand - Lookup Gen
identity_lookup_default_fields.csv asset_lookup_default_fields.csv
LOOKUP-zz-asset_identity_lookup_ LOOKUP-zz-asset_identity_lookup_ LOOKUP-zz-asset_identity_lookup_ LOOKUP-zz-asset_identity_lookup_ LOOKUP-zz-asset_identity_lookup_
Asset fields after processing Asset fields of the asset lookup after the saved searches perform the merge process. Field
Action taken by ETL
bunit
unchanged
city
unchanged
country
unchanged
dns
Accepts pipe-delimited values and converts them to a multi-value field.
lat
unchanged
long
unchanged
mac
Accepts pipe-delimited values and converts them to a multi-value field.
nt_host
Accepts pipe-delimited values and converts them to a multi-value field.
owner
unchanged
priority
unchanged
asset_id
Generated from the values of dns, ip, mac, and nt_host fields.
asset_tag
Generated from the values of category, pci_domain, is_expected, should_timesync, should_update, requires_av, and bunit fields.
category 71
Appends "pci" if the value contains "cardholder". Accepts pipe-delimited values and converts them to a multi-value field. ip
validates and splits the field into CIDR subnets as necessary. Accepts pipe-delimited values and converts them to a multi-value field.
pci_domain
Appends "trust" or "untrust" based on certain field values. Accepts pipe-delimited values and converts them to a multi-value field.
is_expected
Normalized to a boolean.
should_timesync Normalized to a boolean. should_update
Normalized to a boolean.
requires_av
Normalized to a boolean.
key
Generated by the ip, mac, nt_host, and dns fields after the original fields are transformed.
Identity fields after processing Identity fields of the identity lookup after the saved searches perform the merge process. Field
Action taken by ETL
bunit
unchanged
email
unchanged
endDate
unchanged
first
unchanged
last
unchanged
managedBy
unchanged
nick
unchanged
phone
unchanged
phone2
unchanged
prefix
unchanged
priority
unchanged
startDate
unchanged
suffix
unchanged 72
work_city
unchanged
work_country unchanged work_lat
unchanged
work_long
unchanged
watchlist
Normalized to a boolean.
category
Appends "pci" if the value contains "cardholder". Accepts pipe-delimited values and converts them to a multi-value field.
identity
Generated based on values in the input row and conventions specified in the Identity Lookup Configuration. Accepts pipe-delimited values and converts them to a multi-value field.
identity_id
Generated from the values of identity, first, last, and email.
identity_tag
Generated from the values of bunit, category, and watchlist.
Test the asset and identity merge process Test the asset and identity merge process to confirm that the data produced by the merge process is expected and accurate. Run the saved searches that perform the merge process without outputting the data to the merged lookups to determine what the merge will do with your data without actually performing the merge. Test the merge process without performing a merge and outputting the data to a lookup. 1. From the Splunk ES menu bar, select Configure > Content Management. 2. Locate the first of the three primary saved searches Identity - Asset CIDR Matches - Lookup Gen. 3. Click the search name to open it. 4. Copy the search from the Search field. 5. Open the Search page. 6. Paste the search and remove the `output_*` macro. For example, change | `asset_sources` | `make_assets_cidr` | `output_assets("SA-IdentityManagement", "assets_by_cidr.csv")` to | `asset_sources` | `make_assets_cidr`. 7. Run the search. 8. Repeat steps 2-7 for the other two searches, Identity - Asset String Matches - Lookup Gen and Identity - Identity Matches - Lookup Gen.
73
Force a merge Run the primary saved searches directly to force a merge immediately without waiting the five minutes for the scheduled search to run. 1. Open the Search page. 2. Run the primary saved searches. | savedsearch "Identity - Asset String Matches - Lookup Gen"
| savedsearch "Identity - Asset CIDR Matches - Lookup Gen"
| savedsearch "Identity - Identity Matches - Lookup Gen"
Customize the asset and identity merge process You can modify the saved searches that perform the asset and identity merge process to perform additional field transformations or data sanitization. Add any operations that you want to change in the merge process to the search before the `output_*` macro. Caution: Certain modifications to the saved searches are unsupported and could break the merge process or asset and identity correlation. Do not perform the following actions. • Add or delete fields from the output. • Change the output location to a different lookup table or a KV store collection. • Replace the `output_*` macros with the outputlookup command.
Asset and identity lookup header and field reference To add asset and identity data to Splunk Enterprise Security, format the lookup files with the expected headers and fields. See Add asset and identity data to Splunk Enterprise Security for the full list of steps.
74
Asset lookup header
ip,mac,nt_host,dns,owner,priority,lat,long,city,country,bunit,category,pci_domain,is_exp
Asset lookup fields Populate the following fields in an asset lookup. Note: To add multi-homed hosts or devices to the asset list, add each IP address to the ip field for the host, pipe-delimited. Multi-homed support is limited, and having multiple hosts with the same IP address on different network segments can cause conflicts in the merge process. Field
ip
mac
Data type
Description
Examp
A pipe-delimited list of single IP address or IP ranges. An asset is required to have an entry pipe-delimited in the ip, mac, 2.0.0.0/8|1.2.3.4|192.168.15.9-192.169.15.27|5 numbers nt_host, or dns fields. Do not use pipe-delimiting for more than one of these fields per asset.
pipe-delimited A 00:25:bc:42:f4:60|00:50:ef:84:f1:21|00:50:ef:84 strings pipe-delimited list of MAC address. An asset is required to have an entry in the ip, mac, nt_host, or dns fields. Do not use 75
pipe-delimiting for more than one of these fields per asset.
nt_host
A pipe-delimited list of Windows machine names. An asset is required to have an entry pipe-delimited in the ip, mac, ACME-0005|SSPROCKETS-0102|COSWCOG strings nt_host, or dns fields. Do not use pipe-delimiting for more than one of these fields per asset.
dns
A pipe-delimited list of DNS names. An asset is required to have an entry pipe-delimited in the ip, mac, acme-0005.corp1.acmetech.org|SSPROCKET nt_host, or strings dns fields. Do not use pipe-delimiting for more than one of these fields per asset.
owner
string
The user or [email protected], DevOps, Bill department associated with 76
the device
priority
string
The priority assigned to the device for calculating the Urgency field for notable events. An "unknown" priority reduces unknown, informational, low, medium, high or c the assigned Urgency by default. For more information, see Notable Event Urgency assignment.
lat
string
The latitude of the asset
41.040855
long
string
The longitude of the asset
28.986183
string
The city in which the asset is located
Chicago
string
The country in which the asset is located
USA
bunit
string
The business unit of the asset
EMEA, NorCal
category
pipe-delimited A server|web_farm|cloud strings pipe-delimited list of logical classifications for assets. Used for asset and identity
city
country
77
correlation and categorization. See Categories.
pci_domain
is_expected
A pipe-delimited pipe-delimited strings list of PCI domains.
boolean
should_timesync boolean
PCI_Web|PCI_point_of_sale
Indicates whether events from this asset should always be expected. If set to true, the Expected Host Not Reporting "true", or blank to indicate "false" correlation search performs an adaptive response action when this asset stops reporting events. Indicates "true", or blank to indicate "false" whether this asset must be monitored for time-sync events. It set to true, the Should Timesync Host Not Syncing correlation search performs an adaptive response action if this 78
asset does not report any time-sync events from the past 24 hours.
should_update
requires_av
boolean
Indicates whether this asset must be "true", or blank to indicate "false" monitored for system update events.
boolean
Indicates whether this asset must "true", or blank to indicate "false" have anti-virus software installed.
Identity lookup header
identity,prefix,nick,first,last,suffix,email,phone,phone2,managedBy,priority,bunit,categ
Identity lookup fields Field Identity
Data type
Description
Example
pipe-delimited Required. A a.vanhelsing|abraham.vanhelsing|a.vanhelsing@a strings pipe-delimited list of username strings representing the identity. After the merge process completes, this field includes generated values based on the identity lookup configuration 79
settings. prefix
string
Prefix of the identity.
Ms., Mr.
nick
string
Nickname of an identity.
Van Helsing
first
string
First name of an identity.
Abraham
last
string
Last name of an identity.
Van Helsing
suffix
string
Suffix of the identity.
M.D., Ph.D
email
string
Email address of an identity.
[email protected]
string
A telephone number of an identity.
123-456-7890
string
A secondary telephone number of an identity.
012-345-6789
string
A username representing [email protected] the manager of an identity.
string
The assigned priority of an identity.
unknown, informational, low, medium, high or critic
bunit
string
A group or department classification for identities.
Field Reps, EMEA, APAC
category
pipe-delimited A Privileged|Officer|CISO strings pipe-delimited list of logical classifications for identities. Used for asset and identity
phone
phone2
managedBy
priority
80
correlation and categorization. See Categories.
boolean
Marks the identity for activity monitoring.
string
The start or hire date of an Formats: %m/%d/%Y %H:%M, %m/%d/%y %H:% identity.
string
The end or termination date of an identity.
string
The primary work site City for an identity.
work_country string
The primary work site Country for an identity.
work_lat
string
The latitude of primary work site City in DD with compass direction.
string
The longitude of primary work site City in DD 122.41W with compass direction.
watchlist
startDate
endDate
work_city
work_long
Accepted values: "true" or empty. See User Activit in this manual.
Formats: %m/%d/%Y %H:%M, %m/%d/%y %H:%
37.78N
Modify asset and identity lookups in Splunk Enterprise Security Make changes to the asset and identity lookups in Splunk Enterprise Security to add new assets or identities, or change existing values in the lookup tables. You can also disable or enable existing lookups. 81
Edit asset and identity lookups Edit an asset or identity lookup in the Identity Management dashboard. 1. In Enterprise Security, select Configure > Data Enrichment > Identity Management. 2. Find the name of the asset or identity list you want to edit, and select Source. The list opens in an interactive editor. 3. Use the scroll bars to view the columns and rows in the table. Double click a cell to add, change, or remove content. 4. Click Save when you are finished. Changes made to an asset or identity list will be reflected in search results after the next scheduled merge. See How Splunk Enterprise Security processes and merges asset and identity data.
Disable or enable asset and identity lookups Disable or enable an asset or identity lookup table file. Disable a list to prevent the contents of that list from being included in the merge process. Enable a disabled list to allow the list to be merged at the next scheduled merge of the asset or identity data. Disabling a list does not delete the data from Splunk Enterprise Security. 1. In Enterprise Security, select Configure > Data Enrichment > Identity Management. 2. Locate the asset or identity lookup you want to disable. 3. Click Disable or Enable. Disable the demo asset and identity lookups Disable the demo asset and identity lookups to prevent the demo data from being added to the primary asset and identity lookups used by Splunk Enterprise Security for asset and identity correlation. Splunk Enterprise Security enables the demo asset and identity lookups after installation or upgrade. After you disable the demo data lookups, saved searches update the primary asset and identity lookups and removes the data from the disabled lookups from the primary lookups. 1. In Enterprise Security, select Configure > Data Enrichment > Identity Management. 2. Locate the demo_assets and demo_identities lookups. 3. Click Disable for each. 82
Example methods of adding asset and identity data to Splunk Enterprise Security These example methods cover some common ways to add asset and identity data to Splunk Enterprise Security. You can work with Splunk Professional Services to find the best solution for your environment.
Add asset and identity data from Active Directory Add asset and identity data from Active Directory. Set up the Splunk Support for Active Directory app Collect asset and identity data with the Splunk Support for Active Directory app. Install and configure the app. See Install the Splunk Supporting Add-on for Active Directory. Collect asset and identity data from Active Directory Collect asset and identity data from Active Directory by searching the data in SA-ldapsearch. 1. Follow the steps to configure a new asset or identity list. See Add asset and identity data to Splunk Enterprise Security. 2. Disable the lookup file you created until you finish setting up the saved search to prevent the asset or identity data from merging with incomplete or inaccurate data. See Disable or enable asset or identity lookups. 3. Create a saved search in SA-IdentityManagement to populate the lookup table file with the ldapsearch command. The exact syntax of this search varies depending on your AD configuration. 4. Test the merge process. See Test the asset and identity merge process. Example search for collecting identity data from Active Directory
This example search assigns static values for "suffix", "endDate", "category", "watchlist", and "priority". Use it as a guide to construct and test a working search, then replace the static values with information from your AD environment. Rename the lookup my_identity_lookup to something appropriate for your environment. |ldapsearch domain= search="(&(objectclass=user)(!(objectClass=computer)))" |makemv userAccountControl
83
|search userAccountControl="NORMAL_ACCOUNT" |eval suffix="" |eval priority="medium" |eval category="normal" |eval watchlist="false" |eval endDate="" |table sAMAccountName,personalTitle,displayName,givenName,sn,suffix,mail,telephoneNumber,mobile |rename sAMAccountName as identity, personalTitle as prefix, displayName as nick, givenName as first, sn as last, mail as email, telephoneNumber as phone, mobile as phone2, manager as managedBy, department as bunit, whenCreated as startDate |outputlookup my_identity_lookup Example search for collecting asset data from Active Directory
This example search assigns static values for several fields. Use it as a guide to construct and test a working search, then replace the static values with information from your AD environment. Rename the lookup my_asset_lookup to something appropriate for your environment.
|ldapsearch domain= search="(&(objectClass=computer))" |eval city="" |eval country="" |eval priority="medium" |eval category="normal" |eval dns=dNSHostName |eval owner=managedBy |rex field=sAMAccountName mode=sed "s/\$//g" |eval nt_host=sAMAccountName |makemv delim="," dn |rex field=dn "(OU|CN)\=(?.+)" |table ip,mac,nt_host,dns,owner,priority,lat,long,city,country,bunit,category,pci_domain,is_exp | outputlookup create_empty=false createinapp=true my_asset_lookup
Add asset data from indexed events in Splunk platform Identify hosts that appear in indexed events that are not currently associated with existing asset data and add those hosts to your asset lookup. Use this example search to compare hosts communicating with the Splunk platform to the set of existing asset information and review the table of unmatched hosts. You can then export the table as an asset list. | `host_eventcount` | search host_is_expected=false NOT host_asset_id=* | fields - firstTime,recentTime,lastTime,_time, host_owner_*,host_asset_tag,host_asset_id | sort -totalCount,dayDiff
84
| table host,ip,mac,nt_host,dns,owner,priority,lat,long,city,country,bunit,category,pci_domain,i
Manually add new asset or identity data Manually add new asset or identity data to Splunk Enterprise Security by editing the static_assets and static_identities lists. For example, add internal subnets, IP addresses that should be whitelisted, and other static asset and identity data. 1. From the Splunk ES menu bar, Select Configure > Data Enrichment > Lists and Lookups. 2. To add asset data, click the "static_assets" list to edit it. To add identity data, click the "static_identities" list to edit it. 3. Use the scroll bars to view the columns and rows in the table. Double click in a cell to add, change, or remove content. 4. Click Save.
85
Access and Endpoint Domain Access dashboards The Access Protection domain monitors authentication attempts to network devices, endpoints, and applications within the organization. Access Protection is useful for detecting malicious authentication attempts, as well as identifying systems users have accessed in either an authorized or unauthorized manner.
Access Center dashboard Access Center provides a summary of all authentication events. This summary is useful for identifying security incidents involving authentication attempts such as brute-force attacks or use of clear text passwords, or for identifying authentications to certain systems outside of work hours. Dashboard filters Use the available dashboard filters to refine the results displayed on the dashboard panels. The filters do not apply to key security indicators. Filter by
Description
Action
Action
Filter based on authentication success or failure.
Drop-down: select to filter by
App
Filter based on authentication application.
Drop-down: select to filter by
Business A group or department classification for the Unit identity.
Text field. Empty by default. Wildcard strings with an asterisk (*)
Category
Filter based on the categories to which the host Drop-down: select belongs. See Dashboard Filters in this manual to filter by
Special Access
Restricts the view to events related to privileged access. See Configure privileged accounts in this topic.
Drop-down: select to filter by
Time Range
Select the time range to view.
Drop-down: select to filter by
86
Dashboard Panels Panel
Description
Access Over Time By Action
Displays the count of authentication events over time by action.
Access Over Time By App
Displays the count of authentication events over time by app. For example, "win:local" refers to the local authentication performed on a Windows system and "win:remote" refers to remote API access.
Displays a table of highest access counts by source. This table Top Access By is useful for detecting brute force attacks, since aggressive authentication attempts display a disproportionate number of Source auth requests. Top Access By Displays a table of the sources generating the highest Unique Users authentication events by count.
Access Tracker dashboard The Access Tracker dashboard gives an overview of account statuses. Use it to track newly active or inactive accounts, as well as those that have been inactive for a period of time but recently became active. Discover accounts that are not properly de-provisioned or inactivated when a person leaves the organization. As inactive accounts or improperly active accounts are vulnerable to attackers, it is a good idea to check this dashboard on a regular basis. You can also use this dashboard during an investigation to identify suspicious accounts and closely examine user access activity. Dashboard filters Use the available dashboard filters to refine the results displayed on the dashboard panels. The filters do not apply to key security indicators. Filter by
Description
Business A group or department classification for the Unit identity. Category
Action Text field. Empty by default. Wildcard strings with an asterisk (*)
Filter based on the categories to which the host Drop-down: select
87
belongs. For more information, see Dashboard to filter by Filters in this manual Dashboard Panels Panel
Description
First Time Access - Last 7 days
Displays new account access by user and destination.
Inactive Account Usage - Last 90 days
Displays accounts that were inactive for a period of time, but that have shown recent activity.
Completely Inactive Accounts Last 90 days
Displays accounts that have shown no activity. Use this panel to identify accounts that should be suspended or removed. If the organization has a policy that requires password change after a specified interval, then accounts that have shown no activity for more than that interval are known to be inactive. This panel also indicates the effectiveness of the enterprise's policy for closing or de-provisioning accounts. If a large number of accounts display here, the process may need to be reviewed.
Account Usage For Expired Identities Last 7 days
Displays activity for accounts that are suspended within the specified time frame. Use this panel to verify that accounts that should be inactive are not in use.
Access Search dashboard Use the Access Search dashboard to find specific authentication events. The dashboard is used in ad-hoc searching of authentication data, but is also the primary destination for drilldown searches used in the Access Anomalies dashboard panels. The Access Search page displays no results unless it is opened in response to a drilldown action, or you set a filter and/or time range and click Submit. Dashboard filters Use the available dashboard filters to refine the results displayed on the dashboard panels. The filters do not apply to key security indicators.
88
Filter by
Description
Action
Action
Filter based on authentication success or failure.
Drop-down: select to filter by
App
Filter based on authentication application.
Drop-down: select to filter by
Source
A string that the source field src must match.
Text field. Empty by default. Wildcard strings with an asterisk (*)
A string that the destination field dest Destination must match.
Text field. Empty by default. Wildcard strings with an asterisk (*)
User
A string that the user field user must match.
Text field. Empty by default. Wildcard strings with an asterisk (*)
Time Range
Select the time range to view.
Drop-down: select to filter by
Account Management dashboard The Account Management dashboard shows changes to user accounts, such as account lockouts, newly created accounts, disabled accounts, and password resets. Use this dashboard to verify that accounts are being correctly administered and account administration privileges are being properly restricted. A sudden increase in the number of accounts created, modified, or deleted can indicate malicious behavior or a rogue system. A high number of account lockouts could indicate an attack. Dashboard filters Use the available dashboard filters to refine the results displayed on the dashboard panels. The filters do not apply to key security indicators. Filter by
Description
Business A group or department classification for the Unit identity.
89
Action Text field. Empty by default. Wildcard strings with an asterisk (*)
Category
Filter based on the categories to which the host Drop-down: select belongs. For more information, see "Dashboard to filter by Filters" in this manual
Restricts the view to events related to Special privileged access. See Configure privileged Accounts accounts in this topic.
Drop-down: select to filter by
Time Select the time range to view. Range Dashboard Panels Panel
Drop-down: select to filter by
Description
Account Management Over Time
Displays all account management events over time.
Account Lockouts
Displays all account lockouts, including the number of authentication attempts per account.
Account Management by Source User
Tracks the total account management activity by source user, and shows the source users with the most account management events. The source user is the user that performed the account management event, rather than the user that was affected by the event. For example, if user "Friday.Adams" creates an account "Martha.Washington", then "Friday.Adams" is the source user. This panel helps identify accounts that should not be managing other accounts and shows spikes in account management events, such as the deletion of a large number of accounts.
Top Account Management Events
Shows the most frequent management events in the specified time period.
Default Account Activity dashboard The Default Account Activity dashboard shows activity on "default accounts", or accounts enabled by default on various systems such as network infrastructure devices, databases, and applications. Default accounts have well-known passwords and are often not disabled properly when a system is deployed. Many security policies require that default accounts be disabled. In some cases, 90
you may need to monitor or investigate authorized use of a default account. It is important to confirm that the passwords on default accounts are changed before use. Abnormal or deviant user behavior from a default account can indicate a security threat or policy violation. Use this dashboard to ensure that security policies regarding default accounts are properly followed. Dashboard filters Use the available dashboard filters to refine the results displayed on the dashboard panels. The filters do not apply to key security indicators. Filter by
Description
Action
Business A group or department classification for the Unit identity.
Category
Text field. Empty by default. Wildcard strings with an asterisk (*)
Filter based on the categories to which the host Drop-down: select belongs. For more information, see Dashboard to filter by Filters in this manual
Time Select the time range to view. Range Dashboard panels Panel
Drop-down: select to filter by
Description
Default Account Usage Over Time by App
Shows default account activity on all systems and applications during the selected time frame, split by application. For example, sshd or ftpd. Application accounts are shown by the number of successful login attempts and when the last attempt was made. Use this chart to identify spikes in default account login activity by application, which may indicate a security incident, as well as to determine whether default account use is common (for example, a daily event) or rare for a certain application.
Default Accounts in Use
Shows all default user accounts with a high number of login attempts on different hosts, including the last attempt made. Abnormal default user account activity that could indicate a security threat. Also helps ensure that default account behavior matches the security policy.
Default Local Accounts
Lists all default accounts that are active on enterprise systems, including accounts "at rest". Any available default 91
accounts are listed, regardless of whether the account is actually in use. Only accounts detected on a local system, for example by examining the users list on a host, are included in this list.
Configure privileged accounts An account that is known to have administrator or super-user access is considered privileged, such as root or administrator accounts. This list of accounts can be configured with the Administrative Identities lookup. 1. Navigate to Configure > Data Enrichment > Lists and Lookups and select the Administrative Identities lookup. 2. The list categorizes privileged default accounts as default|privileged. Select the field and begin typing to make changes.
Troubleshooting Access dashboards This dashboard references data from various data models. Without the applicable data, the dashboards will remain empty. See Dashboard Troubleshooting in this manual.
Endpoint dashboards The Endpoint Protection domain provides insight into malware events including viruses, worms, spyware, attack tools, adware, and PUPs (Potentially Unwanted Programs), as well as your endpoint protection deployment.
Malware Center dashboard Malware Center is useful to identify possible malware outbreaks in your environment. It displays the status of malware events in your environment, and how that status changes over time based on data gathered by Splunk. Search malware events directly using Malware Search, or click chart elements or table rows to display raw events. See dashboard drilldown for more information on this feature. Configure new data inputs through the Settings menu. You can use the filters to refine which events are shown.
92
Filter by Action
Description
Action Drop-down: select to filter by
All, allowed, blocked, or deferred.
Business A group or department Unit classification for the identity.
Text field. Empty by default. Wildcard strings with an asterisk (*)
Filter based on the Drop-down: categories to which the Category malware belongs. For more select to information, see "Dashboard filter by Filters" in this manual Drop-down: select to filter by The following table describes the panels for this dashboard. Time Range
Select the time range to represent.
Panel
Key Indicators
Description Displays the metrics relevant to the dashboard sources over the past 48 hours. Key indicators represent summary information and appear at the top of the dashboard. See "Key indicators" in this manual.
Malware Activity Shows all malware detected over the specified time period, Over Time By split by action (allowed, blocked, deferred). Use this chart to Action detect whether too many malware infections are allowed. Shows all malware detected over the specified time period, Malware Activity split by signature. Example signatures are Mal/Packer, Over Time By LeakTest, EICAR-AV-Test, TROJ_JAVA.BY. Use this chart to Signature detect which infections are dominant in your environment. Top Infections
Shows a bar chart of the top infections in your environment, split by signature. This panel helps identify outbreaks related to a specific type of malware.
New Malware Last 30 Days
Shows new malware detected on the network over the last 30 days. For each malware signature identified, the date and time
93
it was first detected and the total number of infections are shown. First-time infections are the most likely to cause outbreaks.
Malware Search dashboard The Malware Search dashboard assists in searching malware-related events based on the criteria defined by the search filters. The dashboard is used in ad-hoc searching of malware data, but is also the primary destination for drilldown searches used in the Malware Center dashboard panels. The Malware Search dashboard displays no results unless it is opened in response to a drilldown action, or you update a filter, select a time range, and click Submit. Filter by
Description
Action
Action
Filter by the action taken on the Drop-down: select to malware (allowed, filter by blocked, or deferred).
Signature
Filter on malware with matching signatures.
File
Text field. Empty by default. Wildcard Filter on file name. strings with an asterisk (*)
Text field. Empty by default. Wildcard strings with an asterisk (*)
Filter on endpoint systems.
Text field. Empty by default. Wildcard strings with an asterisk (*)
User
Filter based on username.
Text field. Empty by default. Wildcard strings with an asterisk (*)
Time Range
Select the time range to view.
Drop-down: select to filter by
Destination
94
Malware Operations dashboard The Malware Operations dashboard tracks the status of endpoint protection products deployed in your environment. Use this dashboard to see the overall health of systems and identify systems that need updates or modifications made to their endpoint protection software. This dashboard can also be used to see how the endpoint protection infrastructure is being administered. You can click chart elements or table rows to display raw events. See dashboard drilldown for more information on this feature. Configure new data inputs through the Settings menu. Use the filters to refine which events are shown. Filter by
Description
Action
Business A group or department Unit classification for the identity.
Text field. Empty by default. Wildcard strings with an asterisk (*)
Filter based on the Drop-down: categories to which the Category malware belongs. For more select to information, see "Dashboard filter by Filters" in this manual Drop-down: select to filter by The following table describes the panels for this dashboard. Time Range
Select the time range to represent.
Panel
Description
Displays the metrics relevant to the dashboard sources over the past 48 hours. Key indicators represent summary Key Indicators information and appear at the top of the dashboard. See "Key indicators" in this manual. Clients by Product Version
Shows a bar chart of the number of clients with a certain version of the endpoint protection product installed. 95
Clients by Signature Version
Shows a bar chart of the number of clients with a certain signature version.
Repeat Infections
Shows repeated malware infections. Sort by signature, destination, action, or number of days.
Oldest Infections
Shows the oldest malware infections in your environment. Sort by date that the infection was detected (first or last time), the signature, destination host (affected system), or days the infection has been active.
System Center dashboard The System Center dashboard shows information related to endpoints beyond the information reported by deployed anti-virus or host-based IDS systems. It reports endpoint statistics and information gathered by Splunk. System configuration and performance metrics for hosts, such as memory usage, CPU usage, or disk usage, can be displayed on this dashboard. Click chart elements or table rows to display raw events. See dashboard drilldown for more information on this feature. Configure new data inputs through the Settings menu. Use the filters to refine which events are shown. Filter by
Destination
Business Unit
Category
Description
Action
Host name of the affected endpoint system.
Text field. Empty by default. Wildcard strings with an asterisk (*)
A group or department classification for the identity.
Text field. Empty by default. Wildcard strings with an asterisk (*)
Filter based on the categories to which the
Drop-down: select to 96
malware belongs. For more information, see "Dashboard Filters" in this manual
filter by
Drop-down: select to filter by The following table describes the panels for this dashboard. Time Range
Select the time range to represent.
Panel
Description
Operating Systems
Shows the operating systems deployed on the network. Use this chart to detect operating systems that should not be present in your environment.
Top-Average CPU Load by System
Shows the systems on the network with the top average CPU load.
Services by System Count
Shows services ordered by the number of systems on which they are present.
Ports By System Shows the transport method (e.g., tcp) and destination Count ports, ordered by the number of systems. Note: If incorrect or missing data is showing up in the System Center dashboard, be sure that the technology add-ons that supply the data for this dashboard are installed on the full forwarders in the deployment. Technology add-ons containing knowledge needed for parsing of data need to be installed on the full forwarders.
Time Center dashboard The Time Center dashboard helps ensure data integrity by identifying hosts that are not correctly synchronizing their clocks. Splunk will create an alert when it discovers a system with time out of sync. When you receive an alert, you can drill down to the raw data and investigate further by clicking any of the chart elements or table rows on the dashboard. See dashboard drilldown for more information on this feature. Use the filters to refine which events are shown. Filter by
Description
Action
97
Show only systems that should timesync
Drop-down: Select true to filter by select to systems categorized as should_timesync=true in the filter by Asset table or false to filter by systems categorized as should_timesync=false in the Asset table. See "Configuring a new asset or identity list" in this manual for more about asset configuration.
Business A group or department Unit classification for the identity.
Text field. Empty by default. Wildcard strings with an asterisk (*)
Filter based on the Drop-down: categories to which the Category malware belongs. For more select to information, see "Dashboard filter by Filters" in this manual Drop-down: select to filter by The following table describes the panels for this dashboard. Time Range
Select the time range to represent.
Panel
Description
Time Synchronization Failures
A list of systems where time synchronization has failed.
Systems Not Time Synching
Shows a list of systems that have not synchronized their clocks in the specified time frame.
Indexing Time Delay Shows hosts with significant discrepancies between the timestamp the host places on the event and the time that the event appears in Splunk. For example, if the timestamp on an event is later than the time that Splunk indexes the event, the host is timestamping events as future events. A large difference 98
(on the order of hours) indicates improper time zone recognition. Time Service Start Mode Anomalies
Shows hosts that have a time service start mode, such as Manual that others do not.
Endpoint Changes dashboard The Endpoint Changes dashboard uses the Splunk change monitoring system, which detects file-system and registry changes, to illustrate changes and highlight trends in the endpoints in your environment. For example, Endpoint Changes can help discover and identify a sudden increase in changes that may be indicative of a security incident. You can click chart elements or table rows on this dashboard to display raw events. See dashboard drilldown for more information on this feature. Use the filters to refine which events are shown. Filter by
Description
Action
Business A group or department Unit classification for the identity.
Text field. Empty by default. Wildcard strings with an asterisk (*)
Filter based on the categories to which the Drop-down: Category malware belongs. For more select to information, see "Dashboard filter by Filters" in this manual Drop-down: select to filter by The following table describes the panels for this dashboard. Time Range
Select the time range to represent.
Panel Endpoint Changes by Action
Description Summarizes changes over time. A substantial increase in changes may indicate the presence of an incident that is causing changes on the endpoints (such as a
99
virus or worm). Endpoint Changes by Type
Summarizes the type of changes observed on the endpoints, such as file or registry changes.
Changes by System
Summarizes changes by system
Recent Endpoint Changes
Shows the most recent endpoint changes observed.
Update Center dashboard The Update Center dashboard provides additional insight into systems by showing systems that are not updated. It is a good idea to look at this dashboard on a monthly basis to ensure systems are updating properly. You can click any of the chart elements or table rows on the dashboard to see raw events. See dashboard drilldown for more information on this feature. Use the filters to refine which events are shown. Filter by
Description
Action
Select true to filter by systems categorized as should_update=true in the Asset table or false to filter Show only by systems categorized as Drop-down: systems should_update=false in select to that should the Asset table. See filter by update "Configuring a new asset or identity list" in this manual for more about asset configuration.
Destination Host name of the system.
Business Unit
A group or department classification for the identity.
Text field. Empty by default. Wildcard strings with an asterisk (*) Text field. Empty by default.
100
Wildcard strings with an asterisk (*)
Category
Filter based on the categories to which the malware belongs. For more information, see "Dashboard Filters" in this manual
Drop-down: select to filter by
Drop-down: select to filter by The following table describes the panels for this dashboard. Time Range
Select the time range to represent.
Panel
Description
Key Indicators
Displays the metrics relevant to the dashboard sources over the past 48 hours. Key indicators represent summary information and appear at the top of the dashboard. See "Key indicators" in this manual.
Top Systems Needing Updates
A bar chart of the top systems that need updates installed.
Top Updates Needed
A bar chart of the top updates needed across the environment, sorted by signature, such as the KB number.
Systems Not Updating - Greater Than 30 Days
Systems that have not been updated, sorted by the number of days for which they have not been updated.
Update Service Start Mode Anomalies
Shows all systems where the update startup task or service is disabled. Administrators sometimes disable automatic updates to expedite a restart and can forget to re-enable the process.
Update Search dashboard The Update Search dashboard shows patches and updates by package and/or device. This dashboard helps identify which devices have a specific patch installed. This is useful when, for example, there is a problem caused by a patch and you need to determine exactly which systems have that patch installed.
101
The Update Search dashboard displays no results unless it is opened in response to a drilldown action, or you update a filter, select a time range, and click Submit. Filter by
Description
Action
Select true to filter by systems categorized as should_update=true
in the Asset table or false to filter by Show only systems categorized Drop-down: select systems as to filter by that should should_update=false update in the Asset table. See "Configuring a new asset or identity list" in this manual for more about asset configuration. Update Status
Filter by the status of Drop-down: select the update on a to filter by machine.
Signature
Filter by the signature, for example the KB number, of a particular update.
Text field. Empty by default. Wildcard strings with an asterisk (*)
Destination
Filter on affected endpoint systems.
Text field. Empty by default. Wildcard strings with an asterisk (*)
Time Range
Select the time range Drop-down: select to view. to filter by
102
Risk Analysis Risk Analysis The Risk Analysis dashboard displays recent changes to risk scores and objects that have the highest risk scores. As an analyst, you can use this dashboard to assess relative changes in risk scores and examine the events that contribute to an object's risk score.
Risk scoring A risk score is a single metric that shows the relative risk of a device or user object in the network environment over time. An object represents a system, a user, or an unspecified other. Enterprise Security uses correlation searches to correlate machine data with asset and identity data, which comprises the devices and user objects in a network environment. Correlation searches search for a conditional match to a question. When a match is found, an alert is generated as a notable event, a risk modifier, or both. • A notable event becomes a task. It is an event that must be assigned, reviewed, and closed. • A risk modifier becomes a number. It is an event that will add to the risk score of a device or user object. See Configure Risk Scoring in this manual. Risk scoring example The host RLOG-10 is a jump server that is generating several notable events. The correlation searches Excessive Failed Logins, and Default Account Activity Detected are creating one notable event a day for that system. As RLOG-10 is a jump server, several network credentials are being used against this host, and software or other utilities may have been installed. As a jump server, this behavior is less interesting than if the same behavior is observed on the production DNS server. Rather than ignoring or suppressing notable events generated by jump servers, you can create jump-server-specific rules to monitor those servers differently.
103
You can do this by creating a correlation search that assigns a risk modifier when the correlation matches hosts that serve as jump servers. 1. Isolate jump servers from the existing correlation searches using a whitelist. See Whitelist events for more information. 2. Create and schedule a new correlation search based on Excessive Failed Logins, but isolate the search to the jump server hosts and assign a risk modifier alert type only. 3. Verify the risk modifiers are applied to the jump server hosts by raising their risk score incrementally. With the new correlation search, no notable events will be created for those hosts based on failed logins. As the relative risk score goes up, RLOG-10 can be compared to all network servers and to other jump servers. If the relative risk score for RLOG-10 exceeds its peers, that host would be investigated by an analyst. If the risk scores of all jump servers are higher relative to other network hosts, an internal security policy may need to be reviewed or implemented differently. See the Risk Analysis With Enterprise Security 3.1 blog post for additional examples.
Use the Risk Analysis dashboard You can use the Risk Analysis dashboard to review changes to an object's risk score, determine the source of a risk increase, and decide if additional action is needed. Dashboard filters Use any of the available filters on the Risk Analysis dashboard to search and filter the results. A filter is applied to all panels in the dashboard, but not the key security indicators. Filter by Source
Description Filter by the correlation search that has risk modifiers
Risk Select a risk object type and type a string to filter by risk object. Risk Object object type defaults to All. The Risk Object filter works by performing a reverse lookup against the asset and identity tables to find all fields that have been associated with the specified Risk Object. All associated objects found by the reverse lookup then display on the dashboard. For example, if you select a risk object type of system and type a Risk Object of 10.10.1.100, the reverse lookup against the assets table could return a MAC address. The Risk Analysis dashboard will update to display any risk score applied to the 10.10.1.100 address and a MAC address. If no match to 104
another object was found in the asset table, only the IP address matches from the Risk Analysis data model will be displayed. Dashboard panels The Risk Analysis dashboard offers additional views to help analyze risk scoring changes and what caused the changes. Use the filters to refine the view to a specific object or group of objects. Use the drilldown to explore the data as events. Panel
Description
Key Indicators
Displays the metrics relevant to the dashboard sources over the past 48 hours. Key indicators represent summary information and appear at the top of the dashboard. See "Key indicators" in this manual.
Risk Modifiers Over Time
Displays the changes made to risk modifiers over time. Use the dashboard filters to scope the view to a specific object or group of objects. The drilldown opens a search on all events in the Risk data model scoped to the selected time frame.
Risk Score By Object
Displays the objects with the highest risk score. The drilldown opens a search with the selected risk object and scoped to the selected time frame.
Most Active Sources
Displays the correlation searches that contribute the highest amount of risk to any object. The drilldown opens a search with the selected source.
Recent Risk Modifiers
Displays a table of the most recent changes in a risk score, the source of the change, and the object.
Create an Ad-Hoc Risk Entry Creating an ad-hoc risk entry allows you to make a manual, one-time adjustment to an object's risk score. You can use it to add a positive or negative number to the risk score of an object. Several fields should be completed when adding an ad-hoc risk entry. Ad-hoc Risk Score field Score
Description The number added to a Risk object. Can be a positive or negative integer.
Description 105
A reason or note for manually adjusting an object's risk score. The Description field is mandatory for an ad-hoc risk score. Risk object
Text field. Wildcard with an asterisk (*)
Risk object type
Drop-down: select to filter by.
Configure risk scoring A risk score is a single metric that shows the relative risk of a device or user in the network environment over time. Splunk Enterprise Security classifies a device as a system, a user as a user, and unrecognized devices or users as other. Enterprise Security uses risk analysis to take note of and calculate the risk of small events and suspicious behavior over time to your environment. The Risk Analysis dashboard displays these risk scores and other risk-related information.
Assigning risk Create a risk analysis response action, or risk modifier, to assign risk to an object. You can assign risk to objects in several ways. • Assign risk automatically as part of a correlation search. See Modify a risk score with a risk modifier. • Assign risk on an ad-hoc basis from Incident Review. See Modify a risk score with a risk modifier. • Assign risk through a search. See Risk Analysis Framework. Enterprise Security indexes all risk as events in the risk index.
Score ranges for risk Risk scoring offers a way to capture and aggregate the activities of an asset or identity into a single metric using risk modifiers. The correlation searches included in Enterprise Security assign a risk score between 20 and 100 depending on the relative severity of the activity found in the correlation search. The searches scope the default scores to a practical range. This range does not represent an industry standard. Enterprise Security does not define an upper limit for the total risk score of an identity or asset, but operating systems can impose a limit. For example, 32-bit operating systems limit a risk 106
score to two million. Risk score levels use the same naming convention as event severity. You can assess relative risk scores by comparing hosts with similar roles and asset priority. • 20 - Info • 40 - Low • 60 - Medium • 80 - High • 100 - Critical Edit a correlation search to modify the risk score that the risk analysis response action assigns to an object. See Included adaptive response actions with Splunk Enterprise Security.
Managing risk objects Enterprise Security associates risk modifiers with risk objects. Risk object field The risk object field is a reference to a search field returned by a correlation search. Correlation searches use fields such as src and dest to report on matching results. The risk object field represents a system, host, device, user, role, credential, or any object that the correlation search is designed to report on. Review any correlation search that assigns a risk score for examples of fields that receive a risk score. Risk object types Splunk Enterprise Security defines three risk object types. Object type
Description
System
Network device or technology. Can represent a device in the asset lookup.
User
Network user, credential, or role. Can represent an identity in the identity lookup.
Other
Any undefined object that is represented as a field in a data source. 107
If a risk object matches an object in the asset or identity table, Enterprise Security maps the object as the associated type. For example, an object that matches an asset in the asset lookup is mapped to a risk object type of system. However, devices and users do not need to be represented in the corresponding asset and identity tables to be identified as system or user risk objects. ES categorizes undefined or experimental object types with a risk object type of Other. Create a new risk object 1. From the Enterprise Security menu, select Configure > Data Enrichment > Lists and Lookups and select the Risk Object Types list. 2. Highlight the last risk_object_type cell in the table and right-click to see the table editor. 3. Insert a new row into the table. 4. Double-click in the new row to edit it, then add the new object type name. 5. Save the changes. Edit an existing risk object 1. From the Enterprise Security menu, select Configure > Data Enrichment > Lists and Lookups 2. Select the Risk Object Types list. 3. Highlight the risk object type and change the name. 4. Save the changes.
Example of assigning a risk score through search A correlation or other search can directly modify a risk score without using an alert. In this way, it can alter the risk score of a system or user based on the results of a search, rather than only when search results match a particular set of conditions. For example, the Threat Activity Detected correlation search uses search-assigned risk in addition to an alert-type risk modifier. When the search finds an asset or identity communicating with a host that matches a configured threat list, the search modifies the risk score accordingly. In this case, the risk modifier reflects the number of times the system or user communicated with the threat list, multiplied by the weight of the threat list. As a formula, risk score of a system or user + (threat list weight x event count) = additional risk.
108
As a more specific example, if a search detects host DPTHOT1 communicating with a host on a spyware threat list during a particular time period, the base risk score is set to 40. Then, because DPTHOT1 communicated with the host on the threat list twice, and the spyware threat list has a weight of one, the search modifies the risk score to a total risk score of 42. See Risk Analysis Framework for more about assigning risk scores with search.
109
Threat Intelligence Threat Intelligence dashboards Threat Activity The Threat Activity dashboard provides information on threat activity by matching threat intelligence source content to events in Splunk Enterprise. Dashboard filters Use the available dashboard filters to refine the results displayed on the dashboard panels. The filters do not apply to key security indicators. Filter by
Description
Threat Group
A named group or entity representing a known threat, such as a malware domain.
Threat Category
A category of threat, such as advanced persistent threat, financial threat, or backdoor.
Search
Used for searching on a value related to fields: Destination, Sourcetype, Source, Threat Collection, Threat Collection Key, Threat Key, Threat Match Field, and Threat Match Value.
Time Select the time range to represent. Range Dashboard panels Panel
Description
Displays the metrics relevant to the dashboard sources over the past 48 hours. Key indicators represent summary information, Key Indicators and appear at the top of the dashboard. See Key indicators in this manual. Displays the count of events by all threat collections over the selected time. The drilldown opens a search with the selected Threat Activity threat collection and scoped to the selected time frame. To Over Time review the threat collections, see Supported threat intelligence groups in this manual.
110
Most Active Threat Collections
Displays the top threat collections by event matches over the selected time, with a sparkline representing peak event matches. The drilldown opens a search with the selected threat collection.
Most Active Threat Sources
Displays the top threat sources over the selected time by event count matches. The drilldown opens a search with the selected threat source.
Threat Activity Details
Displays a breakout of the most recent threat matches. Use the event selection box Threat Activity Details with the Advanced Filter option to: • Whitelist by threat_match_value to remove matches. • Highlight specific threat_match_value matches and place them at the top of the table.
Data sources The reports in the Threat Activity dashboard use fields in the Threat_Intelligence data model. Relevant data sources include threat source event matches in the threat_activity index along with the associated threat artifacts. See Dashboard Troubleshooting in this manual.
Threat Artifacts The Threat Artifacts dashboard provides a single location to explore and review threat content sourced from all configured threat download sources. It provides additional context by showing all threat artifacts related to a user-specified threat source or artifact. The dashboard offers multiple selection filters and tabs to isolate the threat content. Begin by changing the Threat Artifact to select from available threat artifact types. Filter by
Description
Threat A collection of objects grouped by the threat collection, such as Artifact network, file, and service. Other available filters will change depending on your selection. Filter by Drop-down 111
Threat Artifact selection
Filter by Text: (*) wildcard defaulted Malware Alias, Intel Source ID, and Intel Source Path
Threat Category, Threat Group
Network
IP, Domain
HTTP. Select from: Referrer: User Agent, Cookie, Header, Data, or URL and add a string to search.
File
File Name, File Extension, File Path, and File Hash
Registry
Hive, Path, Key Name, Value Name, Value Type, and Value Text
Service
Name, Descriptive Name:, Description:, and Type
User
User, Full Name, Group Name, and Description
Process
Process, Process Arguments, Handle Names, and Handle Type
Certificate
Serial Number, Subject, Issuer, Validity Not After, and Validity Not Before
Threat ID
Email Address, Subject, and Body Use the tabs to review threat source context: Tab
Panels
Threat Overview
Endpoint Artifacts, Network Artifacts, Email Artifacts, Certificate Artifacts
Network
HTTP Intelligence, IP Intelligence, Domain Intelligence
Endpoint
File Intelligence, Registry Intelligence, Process Intelligence, Service Intelligence, User Intelligence
Certificate
Certificate Intelligence
Email Data sources
Email Intelligence
The Threat Artifacts dashboard references fields in the threat collection KVStore. Relevant data sources include threat sources such as STIX and OpenIOC documents.
112
Troubleshooting This dashboard references data from the Threat Intelligence KVStore collections. Without the applicable data, the dashboard panels will remain empty. To determine why data is not displaying in the dashboard, follow these troubleshooting steps. 1. Confirm that the inputs are properly configured in the Threat Intelligence Downloads and Threat Intelligence Manager pages. Those inputs are responsible for ingesting data from the threat sources and placing it into the KVStore collections. 2. Use the Threat Intelligence Audit dashboard panel Threat Intelligence Audit Events to review log entries created by the modular inputs. See Dashboard Troubleshooting in this manual for more.
Configure threat intelligence sources Correlate indicators of suspicious activity, known threats, or potential threats with your events by adding threat intelligence to Splunk Enterprise Security. Use threat intelligence to enhance your security monitoring capabilities and add context to your investigations. Splunk Enterprise Security includes a selection of threat intelligence sources, and supports multiple types of threat intelligence so that you can add your own threat intelligence. See Threat intelligence sources included with Splunk Enterprise Security.
Supported types of threat intelligence Splunk Enterprise Security supports several types of threat intelligence. The supported types of threat intelligence correspond to the KV Store collections in which the threat intelligence is stored. Threat collection in KV Store
Supported IOC data types
Local lookup file
certificate_intel
X509 Certificates
Local Certificate Intel
email_intel
Email
Local Email Intel
file_intel
File names or hashes
Local File Intel
113
http_intel
URLs
Local HTTP Intel
ip_intel
IP addresses or domains
Local IP Intel and Local Domain Intel
process_intel
Processes
Local Process Intel
registry_intel
Registry entries
Local Registry Intel
service_intel
Services
Local Service Intel
user_intel Users Local User Intel The collections.conf file in the DA-ESS-ThreatIntelligence subdirectory lists these KV Store collections.
Adding threat intelligence to Splunk Enterprise Security Splunk administrators can add threat intelligence to Splunk Enterprise Security by downloading a feed from the Internet, uploading a structured file, or directly from events in Splunk Enterprise Security. • Download a threat intelligence feed from the Internet • Upload a STIX or OpenIOC structured threat intelligence file • Upload a custom CSV file of threat intelligence • Add threat intelligence from events • Add and maintain threat intelligence locally in Splunk Enterprise Security • Add threat intelligence with a custom lookup file • Add OpenIOC or STIX files using the file system • Add threat intelligence using the REST API. See Threat Intelligence API reference in Splunk Enterprise Security REST API Reference.
Download a threat intelligence feed from the Internet Splunk Enterprise Security can periodically download a threat intelligence feed available from the Internet, parse it, and add it to the relevant KV Store collections. Add a URL-based threat source Add a non-TAXII source of threat intelligence that is available from a URL on the Internet. 1. On the Enterprise Security menu bar, select Configure > Data Enrichment > Threat Intelligence Downloads. 2. Click New to add a new threat intelligence source. 114
3. Type a Name for the threat download. The name can only contain alphanumeric characters, hyphens, and underscores. The name cannot contain spaces. 4. Type a Type for the threat download. The type identifies the type of threat indicator that the feed contains. 5. Type a Description. Describe the indicators in the threat feed. 6. Type an integer to use as the Weight for the threat indicators. Enterprise Security uses the weight of a threat feed to calculate the risk score of an asset or identity associated with an indicator on the threat feed. A higher weight indicates an increased relevance or an increased risk to your environment. 7. (Optional) Change the default download Interval for the threat feed. Defaults to 43200 seconds, or every 12 hours. 8. (Optional) Type POST arguments for the threat feed. 9. (Optional) Type a Maximum age to define the retention period for this threat source, defined in relative time. Enable the corresponding saved searches for this setting to take effect. See Configure threat source retention. For example, -7d. If the time that the feed was last updated is greater than the maximum age defined with this setting, the threat intelligence modular input removes the data from the threat collection. 10. Fill out the Parsing Options fields to make sure that your threat list parses successfully. You must fill out either a delimiting regular expression or an extracting regular expression. You cannot leave both fields blank. Field Description
Example
A delimiter used to split lines in a threat source. Delimiting Delimiters must be a single character. For regular , or : or \t more complex delimiters, use an extracting expression regular expression.
Extracting A regular expression used to extract fields from ^(\S+)\t+(\S+)\t+\S+\t+\S+\ regular individual lines of a threat source document. expression Use to extract values in the threat source. Fields
Required if your document is line-delimited. Comma-separated list of fields to be extracted from the threat list. Can also be used to rename or combine fields. Description is a required field. Additional acceptable fields are the fields in the corresponding KV Store collection for the threat intelligence, visible in the local lookup files or the DA-ESS-ThreatIntelligence/collections.conf
115
:$,.$ ip:$1,description:domain_bl
file. Defaults to description:$1,ip:$2. Ignoring A regular expression used to ignore lines in a regular threat source. Defaults to ignoring blank lines expression and comments.
^\s*$)
Skip The number of header lines to skip when 0 header processing the threat source. lines 11. (Optional) Change the Download Options fields to make sure that your threat list downloads successfully. Field Description Example Retry interval
Number of seconds to wait between download retry attempts. Review the recommended poll interval of the threat source provider before changing the retry interval.
Remote site user
If the threat feed requires authentication, type the user name to use in remote authentication, if required. The user name you add in this field must admin match the name of a credential in Credential Management.
Retries
The maximum number of retry attempts.
60
3
Number of seconds to wait before marking a 30 download attempt as failed. 12. (Optional) If you are using a proxy server, fill out the Proxy Options for the threat feed. See Configure a proxy for retrieving threat intelligence. 13. Save your changes. Timeout
See Verify that you added threat intelligence successfully. Add a ransomware threat feed to Splunk Enterprise Security
This example describes how to add a list of blocked domains that could host ransomware to Splunk Enterprise Security to better prepare your organization for a ransomware attack. The feed used in this example is from abuse.ch 1. On the Enterprise Security menu bar, select Configure > Data Enrichment > Threat Intelligence Downloads. 2. Click New to add a new threat intelligence source. 3. Type a Name of ransomware_tracker to describe the threat download source.
116
4. Type a Type of domain to identify the type of threat intelligence contained in the threat source. 5. Type a Description of Blocked domains that could host ransomware. 6. Type a URL of https://ransomwaretracker.abuse.ch/downloads/RW_DOMBL.txt. 7. (Optional) Change the default Weight of 1 to 2 because ransomware is a severe threat and you want an extra risk score multiplier for assets or identities associated with blocked ransomware domains. 8. Leave the default Interval of 43200 seconds, or every 12 hours. 9. Leave the POST arguments field blank because this type of feed does not accept POST arguments. 10. Decide whether to define a Maximum age for the threat intelligence. According to the ransomware tracker website, items on the blocklist stay on the blocklist for 30 days. To drop items off the blocklist in Enterprise Security sooner than that, set a maximum age of less than 30 days. Type a maximum age of -7d. 11. Type a default Delimiting regular expression of : so that you can enrich the threat indicators by adding fields. 12. Leave the Extracting regular expression field blank because the domain names do not need to be extracted because they are line-delimited. 13. Type Fields of domain:$1,description:ransomware_domain_blocklist to define the fields in this blocklist. 14. (Optional) Leave the default Ignoring regular expressions field. 15. Change the Skip header lines field to 0 because the ignoring regular expression ignores the comments at the top of the feed. 16. Leave the Retry interval at the default of 60 seconds. 17. (Optional) Leave the Remote site user field blank because this feed does not require any form of authentication. 18. Leave the Retries field at the default of 3. 19. Leave the Timeout field at the default of 30 seconds. 20. Ignore the Proxy Options section unless you are using a proxy server to add threat intelligence to Splunk Enterprise Security. 21. Click Save. 22. From the Splunk platform menu bar, select Apps > Enterprise Security to return to Splunk Enterprise Security. 23. From the Enterprise Security menu bar, select Audit > Threat Intelligence Audit. 24. Fiind the ransomware_tracker stanza in the Threat Intelligence Downloads panel and verify that the status is threat list downloaded. 25. From the Enterprise Security menu bar, select Security Intelligence > Threat Intelligence > Threat Artifacts. 26. Type an Intel Source ID of ransomware_tracker to search for domains added to Splunk Enterprise Security from the new threat feed. 117
27. Click Submit to search. 28. Click the Network tab and review the Domain Intelligence panel to verify that threat intelligence from the ransomware_tracker threat source appears. Add a TAXII feed Add threat intelligence provided as a TAXII feed to Splunk Enterprise Security. 1. On the Enterprise Security menu bar, select Configure > Data Enrichment > Threat Intelligence Downloads. 2. Click New to add a new TAXII feed. 3. Type a Name for the threat intelligence feed. 4. Type a Type of taxii. 5. Type a Description for the threat intelligence feed. 6. Type a URL to use to download the TAXII feed. 7. (Optional) Change the default Weight for the threat intelligence feed. Increase the weight if the threats on the threat feed are high-confidence and malicious threats that should increase the risk score for assets and identities that interact with the indicators from the threat source. 8. (Optional) Adjust the interval at which to download the threat intelligence. Defaults to 43200 seconds, or twice a day. 9. Type TAXII-specific space-delimited POST arguments for the threat intelligence feed. POST Description Example argument collection
Name of the data collection from a TAXII feed.
earliest
The earliest threat data to pull from the earliest="-1y" TAXII feed.
collection="A_TAXII_Feed_Name"
An optional method taxii_username to provide a TAXII taxii_username="user" feed username. taxii_password
An optional method taxii_password="password" to provide a TAXII feed password. If you provide a username without providing a 118
password, the threat intelligence modular input attempts to find the password in Credential Management.
cert_file
Add the certificate file name if the TAXII feed uses certificate cert_file="cert.crt" authentication. The file name must match exactly and is case sensitive.
Add the key file name for the certificate if the TAXII feed uses key_file="cert.key" certificate key_file authentication. The file name must match exactly and is case sensitive. 10. TAXII feeds do not use the Maximum age setting. 11. TAXII feeds do not use the Parsing Options settings. 12. (Optional) Change the Download Options. 13. (Optional) Change the Proxy Options. See Configure a proxy for retrieving threat intelligence. 14. Save the changes. You cannot use an authenticated proxy with a TAXII feed because the libtaxii library used by Enterprise Security does not support authenticated proxies. If possible, use an unauthenticated proxy instead. Add a TAXII feed with certificate authentication You need file system access to add the certificates needed for certificate authentication. In a Splunk Cloud deployment, work with Splunk Support to add or change files on cloud-based nodes. Add the certificate and keys to the same app directory in which you define the TAXII feed. For example, DA-ESS-ThreatIntelligence. 119
1. Add the certificate to the $SPLUNK_HOME/etc/apps//auth directory. 2. Add the private key for the certificate to the same /auth directory. 3. Follow the steps for adding a TAXII feed to Splunk Enterprise Security, using the cert_file and key_file POST arguments to specify the file names of the certificate and private key file. Configure a proxy for retrieving threat intelligence If you use a proxy server to send threat intelligence to Splunk Enterprise Security, configure the proxy options for the threat source. The user must correspond to the name of a Splunk secure stored credential in Credential Management. If you remove an existing proxy user and password in the Threat Intelligence Download Setting editor, the download process will no longer reference the stored credentials. Removing the referenced to credential does not delete the stored credentials from Credential Management. 1. On the Enterprise Security menu bar, select Configure > Data Enrichment > Threat Intelligence Downloads. 2. Select the threat download source or add a new threat download source. See Add a URL-based threat source or Add a TAXII feed. 3. Configure the proxy options. 1. Type a proxy server address. The Proxy Server cannot be a URL. For example, 10.10.10.10 or server.example.com. 2. Type a proxy server port to use to access the proxy server address. 3. Type a proxy user credential for the proxy server. Only basic and digest authentication methods are supported. 4. Save your changes.
Upload a STIX or OpenIOC structured threat intelligence file Add threat intelligence in the form of a structured file to Splunk Enterprise Security. OpenIOC, STIX, and CSV file types are supported by Splunk Enterprise Security. 1. On the Enterprise Security menu bar, select Configure > Data Enrichment > Threat Intelligence Uploads. 2. Type a file name for the file you want to upload. The file name you type becomes the name of the file saved to $SPLUNK_HOME/etc/apps/DA-ESS-ThreatIntelligence/local/data/threat_intel.
The file name cannot include spaces or special characters. 3. Upload an OpenIOC or STIX-formatted file. 120
4. Type a Weight for the threat intelligence file. The weight of a threat intelligence file increases the risk score of objects associated with threat intelligence on this list. 5. (Optional) Type a Threat Category. If you leave this field blank and a category is specified in the OpenIOC or STIX file, Splunk Enterprise Security uses the threat category specified in the file. 6. (Optional) Type a Threat Group. If you leave this field blank and a group is specified in the OpenIOC or STIX file, Splunk Enterprise Security uses the threat group specified in the file. 7. (Optional) Select the Overwrite check box. If you have previously uploaded a file with the same file name, select this check box to overwrite the previous version of the file. 8. Click Save. 9. Verify that your threat intelligence was successfully added. See Verify that you added threat intelligence successfully.
Upload a custom CSV file of threat intelligence You can add a custom file of threat intelligence to Splunk Enterprise Security. If you add threat indicators in a CSV file, they must all be the same type. For example, the file can only include one type of intelligence. If you want to mix types of indicators in one file, create an OpenIOC or STIX file instead using an editor available on the web and follow the instructions to Upload a STIX or OpenIOC structured threat intelligence file. Identify whether your custom file contains certificate, domain, email, file, HTTP, IP, process, registry, service, or user threat intelligence and make sure that the custom CSV file is properly formatted. 1. Select Configure > Data Enrichment > Lists and Lookups. 2. Find the lookup file that matches the local threat intel you are providing. For example, Local File Intel. 3. Open the relevant lookup to view the required headers. 4. Create a new .csv file with a header row containing the required fields. 5. Add the threat data to the .csv file. Add the custom file to Splunk Enterprise Security. 1. On the Enterprise Security menu bar, select Configure > Data Enrichment > Threat Intelligence Uploads. 2. Type a file name for the file you want to upload. The file name you type becomes the name of the file saved to $SPLUNK_HOME/etc/apps/DA-ESS-ThreatIntelligence/local/data/threat_intel.
121
The file name cannot include spaces or special characters. 3. Upload the CSV-formatted file. 4. Type a Weight for the threat list. The weight of a threat file increases the risk score of objects associated with threat intelligence on this list. 5. (Optional) Type a Threat Category. 6. (Optional) Type a Threat Group. 7. (Optional) Select the Overwrite check box. If you have previously uploaded a file with the same file name, select this check box to overwrite the previous version of the file. 8. Click Save. 9. Verify that your threat intelligence was successfully added. See Verify that you added threat intelligence successfully.
Add threat intelligence from events You can add threat intelligence from events to the local threat intelligence lookups. 1. Write a search that produces threat indicators. 2. Add | outputlookup local__intel append=t to the end of the search. For example, write a search that produces a list of IP addresses that are testing a web server for vulnerabilities and add them to the local_ip_intel lookup to be processed by the modular input and added to the ip_intel KV Store collection.
Add and maintain threat intelligence locally in Splunk Enterprise Security Each threat collection has a local lookup file that you can use to manually add threat intelligence. 1. On the Enterprise Security menu bar, select Configure > Data Enrichment > Lists and Lookups. 2. Find the local lookup that matches the type of threat indicator you want to add. For example, Local Certificate intel to add information about malicious or spoofed certificates. 3. Click the lookup name to edit the lookup. 4. Add indicators to the lookup. Right-click and select Insert Row Below to add new rows as needed. 5. (Optional) Type a numeric Weight to change the risk score for objects associated with indicators on this threat intelligence source. 122
6. Click Save.
Add threat intelligence with a custom lookup file Add threat intelligence to Splunk Enterprise Security as a custom lookup file. A lookup-based threat source can add data to any of the supported threat intelligence groups, such as file or IP intelligence. Prerequisite Identify whether the custom threat source is certificate, domain, email, file, HTTP, IP, process, registry, service, or user intelligence. Steps Based on the type of intelligence you add to Splunk Enterprise Security, you must identify the headers for the csv file. 1. Select Configure > Data Enrichment > Lists and Lookups. 2. Find the lookup file that matches the local threat intel you are providing. For example, Local File Intel. 3. Open the relevant lookup to view the required headers. 4. Create a .csv file with a header row with the required fields. 5. Add the threat data to the .csv file. After you create the lookup file, you must add it to Splunk Enterprise Security. 1. On the Splunk platform menu bar, select Settings > Lookups 2. Next to Lookup table files, click Add New. 3. Select a Destination App of SA-ThreatIntelligence. 4. Upload the .csv file you created. 5. Type a Destination filename for the file. For example, threatindicatorszerodayattack.csv. 6. Save. After adding the threat intel lookup to Enterprise Security, set appropriate permissions so Enterprise Security can use the file. 1. Open Lookup table files. 2. Find the lookup file that you added and select Permissions. 3. Select All apps for the Object should appear in field. 4. Select Read access for Everyone. 5. Select Write access for admin. 123
6. Save. Define the lookup so that Splunk ES can import it and understand what type of intelligence you are adding. 1. On the Splunk platform menu bar, select Settings > Lookups. 2. Next to Lookup definitions, click Add New. 3. Select a Destination App of SA-ThreatIntelligence. 4. Enter a name for the threat source. The name you enter here is used to define the threatlist in the input stanza. For example, zero_day_attack_threat_indicators_list. 5. Select a Type: of File based. 6. Select the Lookup File: that you added in step one. For example, threatindicatorszerodayattack.csv. 7. Save. Set permissions on the lookup definition so that the lookup functions properly. 1. Open Lookup definitions 2. Find the definition you added in step four and select Permissions. 3. Set Object should appear in to All apps. 4. Set Read access for Everyone. 5. Set Write access for admin. 6. Save. Add a threat source input stanza that corresponds to the lookup file so that ES knows where to find the new threat intelligence. 1. Select Configure > Data Enrichment > Threat Intelligence Downloads. 2. Choose a threat source input that matches your new content. For example, local_file_intel. 3. Click Clone in the Actions column. 4. Type a Name. The name cannot include spaces. For example, zero_day_attack_threat_indicators. 5. Type a Type. For example, zero_day_IOCs 6. Type a Description. For example, File-based threat indicators from zero day malware. 7. Type a URL that references the lookup definition you created in step three. lookup://zero_day_attack_threat_indicators_list. 8. (Optional) Change the default Weight for the threat data. 9. (Optional) Change the default Retry interval for the lookup.
124
Add OpenIOC or STIX files using the file system You can also add threat intelligence to Splunk Enterprise Security by adding a properly-formatted file to a file system folder. 1. Add a STIX-formatted file with a .xml file extension or an OpenIOC file with a .ioc file extension to the $SPLUNK_HOME/etc/apps/DA-ESS-ThreatIntelligence/local/data/threat_intel
folder on your Splunk Enterprise Security search head or make it available to that file directory on a mounted local network share. 2. By default, the da_ess_threat_local modular input processes those files and places the threat intelligence found in the relevant KV Store collections. 3. By default, after processing the intelligence in the files, the modular input deletes the files because the sinkhole setting is enabled by default. Change the da_ess_threat_local inputs settings 1. On the Enterprise Security menu bar, select Configure > Data Enrichment > Threat Intelligence Management. 2. Click the da_ess_threat_local modular input. 3. Review or change the settings as required. Do not change the default da_ess_threat_default or sa_threat_local inputs. Configure a custom folder and input monitor for threat sources You can also add threat intelligence to Splunk Enterprise Security by adding a properly-formatted file to a custom file directory. The file directory must match the pattern $SPLUNK_HOME/etc/apps//local/threat_intel, and you must create an input monitor to monitor that file directory for threat intelligence. Create an input monitor for threat sources to add threat intelligence to a different folder than the one monitored by the da_ess_threat_local modular input. 1. From the Enterprise Security menu bar, select Configure > Data Enrichment > Threat Intelligence Management. 2. Click New 3. Type a descriptive name for the modular input. The name cannot include spaces. 4. Type a path to the file repository. The file repository must be $SPLUNK_HOME/etc/apps//local/threat_intel
5. (Optional) Type a maximum file size in bytes. 125
6. (Optional) Select the Sinkhole check box. If selected, the modular input deletes each file in the directory after processing the file. 7. (Optional) Select the Remove Unusable check box. If selected, the modular input deletes a file after processing it if it has no actionable threat intelligence. 8. (Optional) Type a number to use as the default weight for all threat intelligence documents consumed from this directory.
Verify that you added threat intelligence successfully After you add new threat intelligence sources or configure included threat intelligence sources, verify that the threat intelligence is being parsed successfully and that threat indicators are being added to the threat intelligence KV Store collections. The modular input responsible for parsing threat intelligence runs every 60 seconds. Verify that the threat feed is being downloaded 1. From the Enterprise Security menu bar, select Audit > Threat Intelligence Audit 2. Find the threat source. 3. Confirm that the download_status column states threat list downloaded. 4. Review the Threat Intelligence Audit Events to see if there are errors associated with the threat lookup name. Verify that threat indicators exist in the threat collections 1. Select Security Intelligence > Threat Intelligence > Threat Artifacts. 2. Search for the threat source name in the Intel Source ID field. 3. Confirm that threat indicators exist for the threat source. Troubleshoot parsing errors Review the following log files on the Threat Intelligence Audit dashboard to troubleshoot parsing errors that can occur when parsing threat intelligence sources in order to add them to Enterprise Security. • Review the Threat Intelligence Audit Events panel for issues related to downloading threat content in the threatlist.log file or the threatintel:download sourcetype. • Review the Threat Intelligence Audit Events panel for issues related to parsing or processing in the threat_intelligence_manager.log file or the 126
threatintel:manager
sourcetype.
For errors that result from uploading a file, review the threat_intel_file_upload_rest_handler.log file.
For additional parsing errors, make sure that the modular inputs are running as expected. • python_modular_input.log for errors associated with modular input failures. How Splunk Enterprise Security processes threat intelligence data See Threat Intelligence Framework on the Splunk > dev portal.
Threat intelligence sources included with Splunk Enterprise Security Splunk Enterprise Security includes some threat intelligence sources to help you correlate indicators of suspicious activity and known or potential threats with your events. Configure threat intelligence sources included with Splunk Enterprise Security Each threat source website provides suggestions for polling intervals and other configuration requirements separate from Splunk Enterprise Security. When configuring the included threat intelligence sources, use the links to the threat source websites to review the threat source provider's documentation. Some threat intelligence sources are enabled by default. 1. From the Enterprise Security menu bar, select Configure > Data Enrichment > Threat Intelligence Downloads. 2. Review the Description field for all defined threat intelligence sources to learn more about the types of indicators that can be correlated with your events. 3. Enable or disable the threat intelligence sources that fit your security use cases. 4. Configure the enabled threat intelligence sources that fit your security use cases.
127
After you enable threat intelligence sources, Verify that you added threat intelligence successfully. Threat sources included with Enterprise Security Splunk Enterprise Security includes several threat intelligence feeds that retrieve information across the Internet. If your deployment is not connected to the Internet, disable these threat sources or source them in an alternate way. Splunk Enterprise Security expects all threat intelligence feeds to send properly-formatted data and valuable threat intelligence information. Feed providers are responsible for malformed data or false positives that could be identified in your environment as a result. To set up firewall rules for these threat feeds, you might want to use a proxy server to collect the threat intelligence before forwarding it to Splunk Enterprise Security and allow the IP address for the proxy server to access Splunk Enterprise Security. The IP addresses for these threat sources can change. If you determine that your Splunk Enterprise Security installation is retrieving data from unexpected IP addresses, do a WHOIS or nslookup to determine if the IP address matches that of one of the threat sources configured in your environment. Threat source
Threat list provider
Website for the threat source
Emerging Threats compromised IPs blocklist
Emerging Threats
http://rules.emergingthreats.net/blockrules
Emerging Threats firewall IP rules
Emerging Threats
http://rules.emergingthreats.net/fwrules
Malware domain host list
Hail a TAXII.com
http://hailataxii.com
iblocklist Logmein
I-Blocklist
https://www.iblocklist.com/lists
iblocklist Piratebay
I-Blocklist
https://www.iblocklist.com/lists
iblocklist Proxy
I-Blocklist
https://www.iblocklist.com/lists
iblocklist Rapidshare I-Blocklist
https://www.iblocklist.com/lists
iblocklist Spyware
I-Blocklist
https://www.iblocklist.com/lists
iblocklist Tor
I-Blocklist
https://www.iblocklist.com/lists
128
iblocklist Web attacker
I-Blocklist
https://www.iblocklist.com/lists
Malware Domain Blocklist
Malware Domains
http://mirror1.malwaredomains.com
abuse.ch Palevo C&C IP Blocklist
abuse.ch
https://palevotracker.abuse.ch
Phishtank Database
Phishtank
http://www.phishtank.com/
SANS blocklist
SANS
http://isc.sans.edu
abuse.ch ZeuS blocklist (bad IPs only)
abuse.ch
https://zeustracker.abuse.ch
abuse.ch ZeuS blocklist (standard)
abuse.ch
https://zeustracker.abuse.ch
Some lists included in Splunk Enterprise Security are not added to the threat intelligence collections and are instead used to enrich data in Enterprise Security. Data list
Data provider
Website for data provider
Alexa Top 1 Million Sites
Alexa Internet http://www.alexa.com/topsites
Mozilla Public Suffix List
Mozilla
https://publicsuffix.org
ICANN Top-level Domains IANA List
http://www.iana.org/domains/root/db
Change existing threat intelligence After you add threat intelligence to Splunk Enterprise Security, you can make changes to the settings to make sure the threat intelligence you correlate with events is useful. Enable or disable a threat intelligence source Enable or disable a threat intelligence source to prevent your events from matching data in the collections of threat intelligence. 1. From the Enterprise Security menu bar, select Configure > Data Enrichment > Threat Intelligence Downloads. 2. Find the threat intelligence source. 3. Under Status, click Enable or Disable. 129
Disable individual threat artifacts To prevent individual threat artifacts on a threat list from creating notable events if they match events in your environment, disable individual threat artifacts. If you have command line access to the Enterprise Security search head, you can disable individual threat artifacts using the REST API. See Threat Intelligence API reference in Splunk Enterprise Security REST API Reference. Edit a threat source Change information about an existing threat source, such as the retention period or the download interval for a threat source. 1. From the Enterprise Security menu bar, select Configure > Data Enrichment > Threat Intelligence Downloads. 2. Click the name of the threat source you want to edit. 3. Make changes to the fields as needed. 4. Save your changes. By default, only administrators can edit threat sources. To allow non-admin users to edit threat sources, see Adding capabilities to a role in the Installation and Upgrade Manual. Configure threat source retention Remove threat intelligence from the KV Store collections in Splunk Enterprise Security based on the date that the intelligence was added to Enterprise Security. 1. If the threat intelligence source is not a TAXII feed, define the maximum age of the threat intelligence. This field is not used for TAXII feeds. 1. From the Enterprise Security menu bar, select Configure > Data Enrichment > Threat Intelligence Downloads. 2. Select a threat source. 3. Change the Maximum age setting using a relative time specifier. For example, -7d or -30d. 2. Enable the retention search for the collection. 1. From the Splunk platform menu bar, select Settings and click Searches, reports, and alerts. 2. Search for "retention" using the search filter. 3. Enable the retention search for the collection that hosts the threat source. All retention searches are disabled by default.
130
Web Intelligence dashboards Use the Web Intelligence dashboards to identify potential and persistent threats in your environment.
HTTP Category Analysis dashboard The HTTP Category Analysis dashboard looks at categories of traffic data. Any traffic data, such as firewall, router, switch, or network flows, can be summarized and viewed in this dashboard. • Compare statistical data to identify traffic outliers, or traffic different from what is typically found in your environment. • Look for category counts that fall outside of the norm (small or large) that may indicate a possible threat. • Find low volume traffic activity and drill down from the summarized data to investigate events. • Use sparklines to identify suspicious patterns of activity by category. Unknown traffic categories Use the "Show only unknown categories" filter on the HTTP Category Analysis dashboard to filter and view unknown categories of web traffic. Before you can filter unknown traffic, define which categories are unknown. 1. Select Settings > Tags. 2. Click List by tag name. 3. Select an App context of DA-ESS-NetworkProtection or a related network add-on, such as TA-websense. 4. Click New. 5. Type a Tag name of unknown. 6. Type a Field value pair to define as unknown traffic. For example, category=undetected. 7. Click Save. Dashboard filters Filters can help refine the HTTP category list. Filter by
Description
Time Range Select the time range to represent. 131
Advanced Click to see the list of category events that can be filtered for this Filter dashboard. See Advanced Filter in this manual for information. Dashboard panels Click chart elements or table rows to display raw events. See dashboard drilldown for more information on this feature. The following table describes the panels for this dashboard. Panel
Description
Displays the metrics relevant to the dashboard sources over the past 48 hours. Key indicators represent summary information Key Indicators and appear at the top of the dashboard. See Key indicators in this manual. Category Distribution
Displays category counts as a scatter plot, with count as the x-axis and src_count as the y-axis. The chart updates when you change filters or the time range. Hover over an item to see details.
Category Details
Displays details of the HTTP categories, including a sparkline that represents the activity for that HTTP category over the last 24 hours.
HTTP User Agent Analysis dashboard Use the HTTP User Agent Analysis dashboard to investigate user agent strings in your proxy data and determine if there is a possible threat to your environment. • A bad user agent string, where the browser name is misspelled (like Mozzila) or the version number is completely wrong (v666), can indicate an attacker or threat. • Long user agent strings are often an indicator of malicious access. • User agent strings that fall outside of the normal size (small or large) may indicate a possible threat that should be looked at and evaluated. The Advanced Filter can be used to whitelist or blacklist specific user agents. Use the statistical information to visually identify outliers. In the summarized data, you can evaluate user agents for command and control (C&C) activity, and find unexpected HTTP communication activity. Dashboard filters The dashboard includes a number of filters that can help refine the user agent list. 132
Filter by
Description
Standard Deviation Index
The percentage (%) shows the amount of data that will be filtered out if that number of standard deviations is selected. Choose a higher number of deviations to see fewer user agent strings, or choose a lower number of deviations to see a greater number of user agent strings.
Time Range
Select the time range to represent.
Advanced Click to see the list of category events that can be filtered for this Filter dashboard. See Advanced Filter in this manual for information. Dashboard panels Click chart elements or table rows to display raw events. See dashboard drilldown for more information on this feature. The following table describes the panels for this dashboard. Panel
Description
Displays the metrics relevant to the dashboard sources over the past 48 hours. Key indicators represent summary information Key Indicators and appear at the top of the dashboard. See Key indicators in this manual. User Agent Distribution
Displays user agent strings as a scatter plot, with length as the x-axis and count as the y-axis. The chart updates when you change the filters or the time range. Hover over an item to see details about the raw data.
User Agent Details
Displays details of the user agents in your environment, including the string value of the user agent and a sparkline that represents the activity for that user agent string over the last 24 hours.
New Domain Analysis dashboard The New Domain Analysis dashboard shows any new domains that appear in your environment. These domains can be newly registered, or simply newly seen by ES. Panels display New Domain Activity events, New Domain Activity by Age, New Domain Activity by Top Level Domain (TLD), and Registration Details for these domains. • View hosts talking to recently registered domains. • Discover outlier activity directed to newly registered domains in the New Domain Activity by Age panel. 133
• Identify unexpected top level domain activity in the New Domain Activity by TLD panel. • Investigate high counts of new domains to find out if your network has an active Trojan, botnet, or other malicious entity. Dashboard filters The dashboard includes a number of filters to refine the list of domains displayed. Filter by
Description
Domain
Enter the domain (Access, Endpoint, Network).
New Domain Type
Select Newly Registered or Newly Seen to filter the types of domains to be viewed.
Maximum Age (days)
The time range for the newly seen or newly registered domains. The default is 30 days.
Time Range
Select the time range to represent.
Click to see the list of category events that can be filtered for Advanced Filter this dashboard. See Advanced Filter in this manual for information. Dashboard panels Click chart elements or table rows to display raw events. See dashboard drilldown for more information on this feature. The following table describes the panels for this dashboard. Panel
Description
New Domain Activity
Table view of information about new domain activity
New Domain Activity by Age
Scatter plot that displays Age as the x-axis and Count as the y-axis. Hover over a square for the exact age and number of new domains.
A bar chart with Count as the x-axis and TLD as New Domain Activity by TLD the y-axis. Hover over a bar for the current (Top Level Domain) number of events for a top level domain.
Registration Details
A table view of information about new domain registrations. Click a domain in the table to open a search on that domain and view the raw events.
134
Configure the external API for WHOIS data To see data in the New Domain Analysis dashboard, you must configure a connection to an external domain lookup data source. The dashboard will only report whether or not a domain is newly seen until this modular input is configured and enabled. The domain lookup uses the external domain source domaintools.com, which provides a paid API for WHOIS data. 1. Sign up for a domaintools.com account. 2. Collect the API host name and your API access credentials from the site. Note that the API access credentials are different from your account email address. Use the API information to set up a modular input in Splunk Enterprise Security. 1. From the ES menu bar, Select Configure > Data Enrichment > WHOIS Management. 2. Click Enable next to whois_domaintools. 3. Click the name of the modular input to add the API hostname and username used to access the domaintools API. 4. Save the API credentials on the Credential Management dashboard. Note: Until you enable the modular input, domains processed by the input will not be queued. This prevents the checkpoint directory from filling up with files. After enabling the modular input, enable the outputcheckpoint_whois macro to create checkpoint data. 1. Select Configure > General > General Settings. 2. Select Enable for the Domain Analysis setting to enable WHOIS tracking. The modular input stores information in the whois_tracker.csv lookup file. After a file exists in the $SPLUNK_HOME/var/lib/splunk/modinputs/whois directory, the whois index will begin to populate with data. After they are processed, checkpoint files will be deleted.
135
URL Length Analysis dashboard The URL Length Analysis dashboard looks at any proxy or HTTP data that includes URL string information. Any traffic data containing URL string or path information, such as firewall, router, switch, or network flows, can be summarized and viewed in this dashboard. • Compare each URL statistically to identify outliers. • Investigate long URLs that have no referrer. • Look for abnormal length URLs that contain embedded SQL commands for SQL injections, cross-site scripting (XSS), embedded command and control (C&C) instructions, or other malicious content. • Use the details table to see how many assets are communicating with the URL. Use the key indicators to compare each new URL and to identify outlier URL strings, ones that are different from what is typically found in your environment. URLs that fall outside of the normal size (small or large) may indicate a possible threat. Unusually long URL paths from unfamiliar sources and/or to unfamiliar destinations are often indicators of malicious access and should be examined. Dashboard filters Use the filters to refine the URL length events represented on the dashboard. Filter by
Description
Standard Deviation Index
The percentage (%) shows the amount of data that will be filtered out if that number of standard deviations is selected. Choose a higher number of deviations to see fewer user agent strings, or choose a lower number of deviations to see a greater number of user agent strings.
Time Range
Select the time range to represent.
Advanced Click to see the list of category events that can be filtered for this Filter dashboard. See Advanced Filter in this manual for information. Dashboard panels Click chart elements or table rows to display raw events. See dashboard drilldown for more information on this feature. The following table describes the panels for this dashboard. Panel
Description 136
Key Indicators
Displays the metrics relevant to the dashboard sources over the past 48 hours. Key indicators represent summary information and appear at the top of the dashboard. See Key indicators in this manual.
The chart displays a count of URL length anomalies across time. It displays URL lengths greater than the number of URL Length Anomalies Over standard deviations selected in the filter (2 by default) displayed in a line graph with time as the x-axis and count as Time the y-axis. URL Length Details
Table that displays the URL strings and details such as the full URI string. If there is more that one event from a source IP address, the count column shows how many events are seen. Z indicates the standard deviations for the URL length.
137
Network Domain Network dashboards The Network Protection domain provides insight into the network and network-based devices, including routers, switches, firewalls, and IDS devices. This domain aggregates all the traffic on the network, including overall volume, specific patterns of traffic, what devices or users are generating traffic, and per-port traffic. It also shows results from the vulnerability scanners on the network.
Traffic Center dashboard The Traffic Center dashboard profiles overall network traffic, helps detect trends in type and changes in volume of traffic, and helps to isolate the cause (for example, a particular device or source) of those changes. This helps determine when a traffic increase is a security issue and when it is due to an unrelated problem with a server or other device on the network. You can use the filters to limit which items are shown. Configure new data inputs through the Settings menu, or search for particular network intrusion events directly through Incident Review. Filter by Action
Description
Action Drop-down: select to filter by
Filter based on firewall rule actions.
Business A group or department Unit classification for the identity.
Text field. Empty by default. Wildcard strings with an asterisk (*)
Filter based on the categories to which the host Drop-down: Category belongs. For more select to information, see "Dashboard filter by Filters" in this manual 138
Time Range
Drop-down: select to filter by
Select the time range to represent.
Dashboard Panels Panel
Description
Displays the metrics relevant to the dashboard sources over the past 48 hours. Key indicators represent summary Key Indicators information and appear at the top of the dashboard. See "Key indicators" in this manual. Displays network traffic by action. The Traffic drilldown redirects the page to the "Traffic Over Time Search" dashboard and searches on the by Action selected action and time range. Displays the number of events per day for Traffic a specified protocol. The drilldown Over Time redirects the page to the "Traffic Search" By dashboard and searches on the selected Protocol protocol and time range.
Top Sources
Displays the top sources of total traffic volume over the given time frame with a sparkline representing peak event matches. The drilldown opens the "Traffic Search" dashboard and searches on the selected source IP and time range.
Displays network activity from port scanners or vulnerability scanners and Scanning helps identify unauthorized instances of Activity these scanners. The drilldown redirects (Many the page to the "Traffic Search" dashboard Systems) and searches on the selected source IP and time range. Traffic Search dashboard The Traffic Search dashboard assists in searching network protocol data, refined by the search filters. The dashboard is used in ad-hoc searching of network data, but is also the primary destination for drilldown searches used in the Traffic Center dashboard panels. 139
The Traffic Search dashboard displays no results unless it is opened in response to a drilldown action, or you update a filter, select a time range, and click Submit. Filter by
Description
Action
Action
Filter based on firewall rule actions.
Drop-down: select to filter by
Source
Filter based on source IP or name.
Text field. Empty by default. Wildcard strings with an asterisk (*)
Filter based on Destination destination IP or name. Transport Protocol
Text field. Empty by default. Wildcard strings with an asterisk (*)
Filter based on Drop-down: select to transport protocol. filter by
Filter based on Destination destination host port port.
Text field. Empty by default. Wildcard strings with an asterisk (*)
Time Range
Drop-down: select to filter by
Select the time range to view.
Intrusion Center dashboard The Intrusion Center provides an overview of all network intrusion events from Intrusion Detection Systems (IDS) and Intrusion Prevention Systems (IPS) device data. This dashboard assists in reporting on IDS activity to display trends in severity and in volume of IDS events. Filter by
Description
Action
Filter based on events IDS Type matching a specified type of IDS.
Drop-down: select to filter by
Filter based on events IDS matching vendor-defined Category categories.
Drop-down: select to filter by
140
Severity
Drop-down: select to filter by
Filter based on event severity.
Business A group or department Unit classification for the identity.
Text field. Empty by default. Wildcard strings with an asterisk (*)
Filter based on the categories to which the host Drop-down: select to Category belongs. For more information, see "Dashboard filter by Filters" in this manual Time Range
Drop-down: select to filter by
Select the time range to view.
Dashboard Panels Panel
Description
Displays the metrics relevant to the dashboard sources over the past 48 hours. Key indicators represent summary Key Indicators information and appear at the top of the dashboard. See "Key indicators" in this manual. Displays the top attacks over time by Attacks severity. The drilldown opens the Over Time "Intrusion Search" dashboard and By searches on the selected severity and Severity time range. Top Attacks
Displays the top attacks by count and signature. The drilldown opens the "Intrusion Search" dashboard and searches on the selected signature.
Scanning Activity (Many
Displays source IP's showing a pattern of attacks. The drilldown opens the "Intrusion Search" dashboard and searches on the 141
Attacks)
selected source IP and time range.
Displays attacks that have been identified for the first time. New attack vectors indicate that a change has occurred on the New network, potentially due to the presence of Attacks a new threat, such as a new malware Last 30 infection. The drilldown opens the Days "Intrusion Search" dashboard and searches on the selected signature and time range. Intrusion Search dashboard The Intrusion Search dashboard assists in searching IDS-related events such as attacks or reconnaissance-related activity, based on the criteria defined by the search filters. The dashboard is used in ad-hoc searching of network data, but is also the primary destination for drilldown searches used in the Intrusion Center dashboard panels. The Intrusion Search dashboard displays no results unless it is opened in response to a drilldown action, or you update a filter, select a time range, and click Submit. Filter by
Description
Action
IDS Category
Filter based on events matching vendor-defined categories.
Drop-down: select to filter by
Severity
Filter based on event severity.
Drop-down: select to filter by
Signature
Filter based on IDS signature name.
Text field. Empty by default. Wildcard strings with an asterisk (*)
Source
Filter based on source IP or name.
Text field. Empty by default. Wildcard strings with an asterisk (*)
Destination Filter based on destination IP or name.
Text field. Empty by default. Wildcard strings with an 142
asterisk (*) Time Range
Select the time range to view.
Drop-down: select to filter by
Vulnerability Center dashboard The Vulnerability Center provides an overview of vulnerability events from device data. Filter by Severity
Description
Action
Filter based on event Drop-down: select to severity. filter by
A group or Business department classification for the Unit identity.
Text field. Empty by default. Wildcard strings with an asterisk (*)
Filter based on the categories to which the host belongs. Category For more information, see "Dashboard Filters" in this manual
Drop-down: select to filter by
Time Select the time Range range to represent. Dashboard Panels Panel
Drop-down: select to filter by
Description
Displays the metrics relevant to the dashboard sources over the past 60 days. Key indicators represent Key Indicators summary information and appear at the top of the dashboard. See "Key indicators" in this manual. Top Displays the most common issues Vulnerabilities reported by the vulnerability scanners. The reported issues are aggregated by host so that the chart represents the number of unique occurrences of the issue as opposed to the number of 143
times the issue was detected (since scanning a single host multiple times will likely reveal the same vulnerabilities each time). The drilldown opens the "Vulnerability Search" dashboard and searches on the selected signature and time range.
Most Vulnerable Hosts
Displays the hosts with the highest number of reported issues. The drilldown opens the "Vulnerability Search" dashboard and searches on the selected severity, host, and time range.
Displays issues by the severity assigned by the vulnerability scanner. Helps identify trends that are not Vulnerabilities visible when looking at vulnerabilities individually. The drilldown opens the by Severity "Vulnerability Search" dashboard and searches on the selected severity and time range. Displays the most recent new vulnerabilities detected as well as the date each one was first observed. Helps identify new issues appearing on the network that need to be New Vulnerabilities investigated as potential new attack vectors. The drilldown opens the "Vulnerability Search" dashboard and searches on the selected signature and time range.
Vulnerability Operations dashboard The Vulnerability Operations dashboard tracks the status and activity of the vulnerability detection products deployed in your environment. Use this dashboard to see the overall health of your scanning systems, identify long-term issues, and see systems that are no longer being scanned for vulnerabilities. Filter by
Description
Action
144
Business A group or Unit department classification for the identity.
Text field. Empty by default. Wildcard strings with an asterisk (*)
Filter based on the categories to which the host belongs. Category For more information, see "Dashboard Filters" in this manual
Drop-down: select to filter by
Time Select the time Range range to represent. Dashboard Panels Panel
Scan Activity Over Time
Drop-down: select to filter by
Description Displays vulnerability scan activity by systems over time. Hover over item for details. The drilldown opens the "Vulnerability Search" dashboard and searches on the selected time range.
Displays detected vulnerabilities by age, with signature, destination, and event time. Click an item to view in the Vulnerabilities Vulnerability Profiler for more detail. by Age The drilldown opens the "Vulnerability Search" dashboard and searches on the selected signature or destination host, and time range. Displays vulnerability scans with a severity of "high". Includes signature. Delinquent The drilldown opens the "Vulnerability Scanning Search" dashboard and searches on the selected destination host and time range. Vulnerability Search dashboard The Vulnerability Search dashboard displays a list of all vulnerability-related events based on the criteria defined by the search filters. The dashboard is used in ad-hoc searching of vulnerability data, but is also the primary destination for 145
drilldown searches used in the Vulnerability Center dashboard panels. The Vulnerability Search dashboard displays no results unless it is opened in response to a drilldown action, or you update a filter, select a time range, and click Submit. Filter by
Description
Action
Vuln. category
Filter based on events matching vendor-defined categories.
Drop-down: select to filter by
Severity
Filter based on event severity.
Drop-down: select to filter by
Signature
Filter based on vendor signature name.
Text field. Empty by default. Wildcard strings with an asterisk (*)
Reference (bugtraq, cert, cve, etc.)
Text field. Empty by Filter based on default. Wildcard common reference strings with an standards. asterisk (*)
Filter based on Destination destination IP or name. Time Range
Select the time range to represent.
Text field. Empty by default. Wildcard strings with an asterisk (*) Drop-down: select to filter by
Troubleshooting Network Dashboards This dashboard references data from various data models. Without the applicable data, the dashboards will remain empty. See "Dashboard Troubleshooting" in this manual.
Web Center and Network Changes dashboards
146
Web Center You can use the Web Center dashboard to profile web traffic events in your deployment. This dashboard reports on web traffic gathered by Splunk from proxy servers. It is useful for troubleshooting potential issues such as excessive bandwidth usage, or proxies that are no longer serving content for proxy clients. You can also use the Web Center to profile the type of content that clients are requesting, and how much bandwidth is being used by each client. You can configure new data inputs through Splunk Settings, or search for particular traffic events directly through Incident Review. Use the filters at the top of the screen to limit which items are shown. Filters do not apply to Key Indicators. Filter by
Description
Action
Business A group or department Unit classification for the identity.
Text field. Empty by default. Wildcard strings with an asterisk (*)
Filter based on the categories to which the host Drop-down: select to Category belongs. For more information, see "Dashboard filter by Filters" in this manual Time Range
Drop-down: select to filter by
Select the time range to represent.
Dashboard Panels Panel
Key Indicators
Description Displays the metrics relevant to the dashboard sources over the past 48 hours. Key indicators represent summary information and appear at the top of the dashboard. See "Key indicators" in this manual.
147
Events Over Shows the total number of proxy events Time by over time, aggregated by Method, or Method the HTTP method requested by the client (POST, GET, CONNECT, etc.). Events Over Shows the total number of proxy Time by events, aggregated by Status, or the Status HTTP status of the response. Sources associated with the highest volume of network traffic. This is useful for identifying sources that are using an excessive amount of network traffic (for Top Sources example, file-sharing hosts), or frequently-requested destinations generating large amounts of network traffic (for example, YouTube or Pandora). Destinations associated with the highest volume of network traffic. This is useful for identifying sources that are using an excessive amount of network Top traffic (for example, file-sharing hosts), Destinations or frequently-requested destinations generating large amounts of network traffic (for example, YouTube or Pandora). Web Search The Web Search dashboard assists in searching for web events that are of interest based on the criteria defined by the search filters. The dashboard is used in ad-hoc searching of web data, but is also the primary destination for drilldown searches used in the "'Web Search dashboard panels. The Web Search dashboard displays no results unless it is opened in response to a drilldown action, or you update a filter, select a time range, and click Submit. Filter by
HTTP Method
Description
Action
Text field. Empty by default. Filter based on HTTP Wildcard strings Method. with an asterisk (*) 148
HTTP Status
Source
Text field. Empty by default. Filter based on HTTP Wildcard strings Status code. with an asterisk (*)
Filter based on source IP or name.
Filter based on Destination destination IP or name.
Text field. Empty by default. Wildcard strings with an asterisk (*) Text field. Empty by default. Wildcard strings with an asterisk (*) Text field. Empty by default. Wildcard strings with an asterisk (*)
URL
Filter based on URL details.
Time Range
Select the time range Drop-down: select to view. to filter by
Network Changes Use the Network Changes dashboard to track configuration changes to firewalls and other network devices in your environment. This dashboard helps to troubleshoot device problems; frequently, when firewalls or other devices go down, this is due to a recent configuration change. Filter by
Description
Action
Business A group or department Unit classification for the identity.
Text field. Empty by default. Wildcard strings with an asterisk (*)
Category Filter based on the categories to which the host
Drop-down: select to
149
belongs. For more filter by information, see "Dashboard Filters" in this manual Time Range
Drop-down: select to filter by
Select the time range to represent.
Dashboard Panels Panel
Description
Shows all changes to the devices by the type of change, or whether a device was Network added, deleted, modified, or changed. The Changes drilldown opens the "New Search" by Action dashboard and searches on the selected action and time range. Shows all devices that have been changed as well as the number of the changes, Network sorted by the devices with the highest Changes number of changes. The drilldown opens by Device the "New Search" dashboard and searches on the selected device and time range. Recent Shows a table of the most recent changes Network to network devices in the last day. Changes
Troubleshooting This dashboard references data from various data models. Without the applicable data, the dashboards will remain empty. See "Dashboard Troubleshooting" in this manual.
Port and Protocol Tracker dashboard The Port and Protocol Tracker tracks port and protocol activity, based on the rules set up in Configure > Data Enrichment > Lists and Lookups in Enterprise Security. The lookup table specifies the network ports that the enterprise allows. From this dashboard, you can view new activity by port to identify devices that are not in compliance with corporate policy, as well as detect prohibited traffic.
150
Filter by
Description
Action
Business A group or department Unit classification for the identity.
Text field. Empty by default. Wildcard strings with an asterisk (*)
Filter based on the categories to which the host Drop-down: select to Category belongs. For more information, see "Dashboard filter by Filters" in this manual Dashboard Panels Panel
Description
Displays the volume network transport and port activity over time, to evaluate if port activity is trending upwards or downwards. Sudden increases in Port/Protocol unapproved port activity may indicate a change on the networked devices, Profiler such as an infection. The drilldown opens the "New Search" dashboard and searches on the selected transport destination port and time range. Displays a table of transport and port New Port traffic communication over time. The Activity - Last drilldown opens the "Traffic Search" 7 Days dashboard and searches on the selected transport and time range. Displays the volume of prohibited network port activity over time, and Prohibited Or helps determine if unapproved port Insecure activity is trending upwards or Traffic Over downwards. The drilldown opens the Time - Last "New Search" dashboard and searches 24 Hours on the selected transport destination port and time range.
151
Prohibited Traffic Details - Last 24 Hours
Displays a table of the number of prohibited network traffic events. The drilldown opens the "New Search" dashboard and searches on the selected source IP, destination IP, transport, port, and time range.
Troubleshooting This dashboard references data from various data models. Without the applicable data, the dashboards will remain empty. See "Dashboard Troubleshooting" in this manual.
Protocol Intelligence dashboards Protocol Intelligence is a collection of dashboards and searches that report on the information collected from common network protocols. As an analyst, you can use these dashboards to gain insight into HTTP, DNS, TCP/UDP, TLS/SSL, and common email protocols across your system or network. The Protocol Intelligence dashboards use packet capture data. Packet capture data contains security-relevant information not typically collected in log files. Integrating network protocol data provides a rich source of additional context when detecting, monitoring, and responding to security related threats. Obtain packet capture data from apps such as Splunk Stream and the Splunk Add-on for Bro IDS. The dashboards will be empty without applicable data. • For information about integrating Splunk Stream with Splunk Enterprise Security, see Splunk Stream integration in the Enterprise Security Installation and Upgrade Manual. • For information about the protocols supported in Splunk Stream, see Supported Protocols in the Splunk Stream User Manual.
Protocol Center The Protocol Center dashboard provides an overview of security-relevant network protocol data. The dashboard searches display results based on the time period selected using the dashboard time picker.
152
Dashboard Panels Panel
Description
Key Indicators
Displays metrics relevant to the dashboard sources over the past 48 hours. Key indicators represent summary information and appear at the top of the dashboard. See Key indicators in this manual.
Connections By Protocol
Displays the sum of all protocol connections, sorted by protocol over time. The connection distribution by protocol shows the most common protocols used in an environment, such as email protocols and HTTP/SSL. An exploited protocol may display a disproportionate number of connections for its service type.
Usage By Protocol
Displays the sum of all protocol traffic in bytes, sorted by protocol over time. The bandwidth used per protocol will show consistency relative to the total network traffic. An exploited protocol may display a traffic increase disproportionate to its use.
Top Connection Sources
Displays the top 10 hosts by total protocol traffic sent and received over time. A host displaying a large amount of connection activity may be heavily loaded, experiencing issues, or represent suspicious activity. The drilldown redirects the page to the Traffic Search dashboard and searches on the selected source IP.
Displays the sum of protocol traffic, sorted by ports under 1024 over time. The bandwidth used per port will show Usage For Well consistency relative to the total network traffic. An exploited Known Ports port may display an increase in bandwidth disproportionate to its use. The drilldown redirects the page to the Traffic Search dashboard and searches on the selected port. Long Lived Connections
Displays TCP connections sustained longer than 3 minutes. A long duration connection between hosts may represent unusual or suspicious activity. The drilldown opens the Traffic Search dashboard and searches on the selected event.
Data sources The reports in the Protocol Center dashboard use fields in the Network Traffic data model. Relevant data sources include all devices or users generating TCP and UDP protocol traffic on the network captured from vulnerability scanners and packet analysis tools such as Splunk Stream and the Bro network security 153
monitor.
Traffic Size Analysis Use the Traffic Size Analysis dashboard to compare traffic data with statistical data to find outliers, traffic that differs from what is normal in your environment. Any traffic data, such as firewall, router, switch, or network flows, can be summarized and viewed on this dashboard. • Investigate traffic data byte lengths to find connections with large byte counts per request, or that are making a high number of connection attempts with small byte count sizes. • Use the graph to spot suspicious patterns of data being sent. • Drill down into the summarized data to look for anomalous source/destination traffic. Dashboard filters Use the filters to refine the traffic size events list on the dashboard. Filter by
Description
Standard Deviation Index
The percentage (%) shows the amount of data that will be filtered out if that number of standard deviations is selected. Choose a higher number of deviations to see fewer user agent strings, or choose a lower number of deviations to see a greater number of user agent strings.
Time Range
Select the time range to represent.
Advanced Click to see the list of category events that can be filtered for this Filter dashboard. See "Advanced Filter" in this manual for information. Dashboard panels Click chart elements or table rows to display raw events. See dashboard drilldown for more information on this feature. The following table describes the panels for this dashboard. Panel
Description
Displays the metrics relevant to the dashboard sources over the past 48 hours. Key indicators represent summary information Key Indicators and appear at the top of the dashboard. See Key indicators in this manual. 154
Traffic Size Anomalies Over Time
The chart displays a count of anomalous traffic size in your environment over time. It displays traffic volume greater than the number of standard deviations selected in the filter (2 by default) displayed in a line graph with time as the x-axis and count as the y-axis.
Traffic Size Details
Table that displays each of the traffic events and related details such as the size of the traffic event in bytes. If there is more that one event from a source IP address, the count column shows how many events are seen. In the bytes column, the minimum, maximum, and average number of bytes for the traffic event are shown. Z indicates the standard deviations for the traffic event.
DNS Activity The DNS Activity dashboard displays an overview of data relevant to the DNS infrastructure being monitored. The dashboard searches display results based on the time period selected using the dashboard time picker. Dashboard Panels Panel
Description
Key Indicators
Displays metrics relevant to the dashboard sources over the past 48 hours. Key indicators represent summary information and appear at the top of the dashboard. See Key indicators in this manual.
Top Reply Codes By Unique Sources
Displays the top DNS Reply codes observed across hosts. A host initiating a large number of DNS queries to unknown or unavailable domains will report a large number of DNS lookup failures with some successes. That pattern of DNS queries may represent an exfiltration attempt or suspicious activity. The drilldown opens the DNS Search dashboard and searches on the selected Reply Code.
Top DNS Query Sources
Displays the top DNS query sources on the network. A host sending a large amount of DNS queries may be improperly configured, experiencing technical issues, or represent suspicious activity. The drilldown opens the DNS Search dashboard and searches on the selected source IP address.
Top DNS Queries
Displays the top 10 DNS QUERY requests over time. The drilldown opens the DNS Search dashboard and searches on the queried host address.
155
Queries Per Domain
Displays the most common queries grouped by domain. An unfamiliar domain receiving a large number of queries from hosts on the network may represent an exfiltration attempt or suspicious activity. The drilldown opens the DNS Search dashboard and searches on the queried domain address.
Recent DNS Queries
Displays the 50 most recent DNS Response queries with added detail. The drilldown opens the DNS Search dashboard and searches on the selected queried address.
Data sources The reports in the DNS dashboard use fields in the Network Resolution data model. Relevant data sources include all devices or users generating DNS protocol traffic on the network captured from vulnerability scanners and packet analysis tools such as Splunk Stream and the Bro network security monitor. DNS Search The DNS Search dashboard assists in searching DNS protocol data, refined by the search filters. The dashboard is used in ad-hoc searching of DNS data, but is also the primary destination for drilldown searches in the DNS dashboard panels. The DNS Search page displays no results unless it is opened in response to a drilldown action, or you set a filter and/or time range and click Submit. Filter by
Description
Source
Source IP address
Destination
Destination IP address
Query
DNS Query
Message Type DNS Message type: Query, Response, or All. Reply Code
DNS Reply type: All, All Errors, and a list of common Reply Codes
SSL Activity The SSL Activity dashboard displays an overview of the traffic and connections that use SSL. As an analyst, you can use these dashboards to view and review SSL encrypted traffic by usage, without decrypting the payload. The dashboard searches display results based on the time period selected using the dashboard time picker.
156
Dashboard Panels Panel Key Indicators
Description Displays metrics relevant to the dashboard sources over the past 48 hours. Key indicators represent summary information and appear at the top of the dashboard. See "Key indicators" in this manual.
Displays outbound SSL connections by common name (CN) of the SSL certificate used. An unfamiliar domain receiving a large SSL Activity number of SSL connections from hosts on the network may By Common represent unusual or suspicious activity. The drilldown redirects Name the page to the SSL Search dashboard, and searches on the selected common name.
SSL Cloud Sessions
Displays the count of active sessions by CN that represents a known cloud service. The CN is compared to a list of cloud service domains pre-configured in the Cloud Domains lookup file. For more information about editing lookups in ES, see Lists and Lookup editor in this manual. The drilldown opens the SSL Search dashboard and searches on the selected source IP and common name.
Recent SSL Sessions
Displays the 50 most recent SSL sessions in a table with additional information about SSL key. The fields ssl_end_time, ssl_validity_window, and ssl_is_valid use color-coded text for fast identification of expired, short lived, or invalid certificates. The drilldown redirects the page to the SSL Search dashboard and displays the full details of the selected event.
Data sources The reports in the SSL Activity dashboard use fields in the Certificates data model. Relevant data sources include all devices or users generating SSL protocol traffic on the network captured from vulnerability scanners and packet analysis tools such as Splunk Stream and the Bro network security monitor. SSL Search The SSL Search dashboard assists in searching SSL protocol data, refined by the search filters. The dashboard is used in ad-hoc searching of SSL protocol data, but is also the primary destination for drilldown searches in the SSL Activity dashboard panels.
157
The SSL Search page displays no results unless it is opened in response to a drilldown action, or you set a filter and/or time range and click Submit. Filter by
Description
Source
Source IP address
Destination
Destination IP address
Subject/Issuer Common Common name retrieved from the x.509 certificate Name Subject or Issuer fields. Certificate Serial Number
The x.509 certificate Serial Number field.
Certificate Hash
The x.509 certificate Signature field.
Email Activity The Email Activity dashboard displays an overview of data relevant to the email infrastructure being monitored. The dashboard searches displays result based on the time period selected using the dashboard time picker. Dashboard Panels Panel
Description
Key Indicators
Displays metrics relevant to the dashboard sources over the past 48 hours. Key indicators represent summary information and appear at the top of the dashboard. See Key indicators in this manual.
Top Email Sources
Displays the hosts generating the most email protocol traffic. A host sending excessive amounts of email on the network may represent unusual or suspicious activity. Periodicity displayed across hosts viewed on the sparklines may be an indicator of a scripted action. The drilldown opens the Email Search dashboard and searches on the selected source IP.
Large Emails
Displays the hosts sending emails larger than 2MB. A host that repeatedly sends large emails may represent suspicious activity or data exfiltration. The drilldown opens the Email Search dashboard and searches on the selected source IP.
Rarely Seen Displays Sender email addresses that infrequently send email. Senders An address that represents a service account or non-user sending email may indicate suspicious activity or a phishing attempt. The drilldown opens the Email Search dashboard and 158
searches on the selected Sender. Displays Receiver email addresses that infrequently receive email. An address that represents a service account or non-user Rarely Seen receiving email may indicate suspicious activity or a phishing Receivers attempt. The drilldown opens the Email Search dashboard and searches on the selected Recipient. Data sources The reports in the Email dashboard use fields in the Email data model. Relevant data sources include all the devices or users generating email protocol traffic on the network captured from vulnerability scanners and packet analysis tools such as Splunk Stream and the Bro network security monitor. Email Search The Email Search dashboard assists in searching email protocol data, refined by the search filters. The dashboard is used in ad-hoc searching of email protocol data, but is also the primary destination for drilldown searches used in the Email Activity dashboard panels. The Email Search page displays no results unless it is opened in response to a drilldown action, or you set a filter and/or time range and click Submit. Filter by
Description
Email Protocol The email communication protocol. Source
Source IP address
Sender
The sender's email address.
Destination
Destination IP address
Recipient
The recipient's email address.
Troubleshooting Protocol Intelligence dashboards The Protocol Intelligence dashboards use packet capture data from apps such as Splunk Stream and the Splunk Add-on for Bro IDS. Without applicable data, the dashboards remain empty. For an overview of Splunk Stream Integration with ES, see Splunk Stream integration in the Enterprise Security Installation and Upgrade Manual. See Dashboard Troubleshooting in this manual.
159
Dashboards Dashboard overview Splunk Enterprise Security includes more than 100 dashboards to identify and investigate security incidents, reveal insights in your events, accelerate incident investigations, monitor the status of various security domains, and audit your incident investigations and your ES deployment. The specific dashboards that will be most useful to you depend on how you plan to use Splunk Enterprise Security.
Identify and investigate security incidents You can identify and investigate security incidents with a suite of dashboards and workflows. Splunk Enterprise Security uses correlation searches to identify notable events in your environment that represent security incidents. • Security Posture provides a high-level overview of the notable events in your environment over the last 24 hours. Identify the security domains with the most incidents, and the most recent activity. See Security Posture dashboard. • Incident Review shows the details of all notable events identified in your environment. Triage, assign, and review the details of notable events from this dashboard. Incident Review. • My Investigations shows all investigations in your environment. Open and work investigations to track your progress and activity while investigating multiple related security incidents. My Investigations.
Accelerate your investigations with security intelligence A set of security intelligence dashboards allow you to investigate incidents with specific types of intelligence. • Risk analysis allows you to assess the risk scores of systems and users across your network and identify particularly risky devices and users posing a threat to your environment. Risk Analysis. • Protocol intelligence dashboards use packet capture data from stream capture apps to provide network insights that are relevant to your security investigations. Identify suspicious traffic, DNS activity, email activity, and 160
review the connections and protocols in use in your network traffic. Protocol Intelligence dashboards. • Threat intelligence dashboards use the threat intelligence sources included in Splunk Enterprise Security and custom sources that you configure to provide context to your security incidents and identify known malicious actors in your environment. Threat Intelligence dashboards. • User intelligence dashboards allow you to investigate and monitor the activity of users and assets in your environment. Asset and Identity Investigator dashboards and User Activity Monitoring. • Web intelligence dashboards help you analyze web traffic in your network and identify notable HTTP categories, user agents, new domains, and long URLs. Web Intelligence dashboards.
Monitor security domain activity Domain dashboards provided with Splunk Enterprise Security allow you to monitor the events and status of important security domains. You can review the data summarized on the main dashboards, and use the search dashboards for specific domains to investigate the raw events. • Access domain dashboards display authentication and access-related data, such as login attempts, access control events, and default account activity. Access dashboards. • Endpoint domain dashboards display endpoint data relating to malware infections, patch history, system configurations, and time synchronization information. Endpoint dashboards. • Network domain dashboards display network traffic data provided by devices such as firewalls, routers, network intrusion detection systems, network vulnerability scanners, proxy servers, and hosts. Network dashboards and Web Center and Network Changes dashboards and Port & Protocol Tracker dashboard. • Identity domain dashboards display data from your asset and identity lists, as well as the types of sessions in use.Asset and Identity dashboards.
Visualize security metrics Create a glass table to visualize security metrics in your environment. Monitor threat activity in your environment, assess the state of your Splunk Enterprise Security deployment, or map out the pathway that an attacker took through your network to monitor future intrusion attempts by an attacker in the future. See Create a glass table.
161
Audit activity in Splunk Enterprise Security The audit dashboards provide insight into background processes and tasks performed by Splunk Enterprise Security. Some audit dashboards allow you to review actions taken by users in Splunk Enterprise Security, while others provide insight into your deployment and the status of your data models and content use. Audit dashboards.
Customize dashboards to fit your use cases You can make changes to dashboards and the searches behind dashboard panels to make them more relevant to your organization, environment, or security use cases. View the search behind a dashboard panel with the panel editor to see where the data is coming from. Edit the title of a panel, the search behind a panel, and even the visualization. • For Splunk Enterprise, see Edit dashboards with the Dashboard Editor in Splunk Enterprise Dashboards and Visualizations. • For Splunk Cloud, see Edit dashboards with the Dashboard Editor in Splunk Cloud Dashboards and Visualizations.
Switch between dashboards and events Dig deeper into data on dashboards by drilling down to raw events, and use workflow actions to move from raw events to investigating specific fields on dashboards, or performing other actions outside of the Splunk platform. You can drill down to raw events from charts and tables in dashboards. You can find information about the drilldown behavior in the Splunk platform documentation. • For Splunk Enterprise, see Drilldown behavior in Splunk Enterprise Dashboards and Visualizations. • For Splunk Cloud, see Drilldown behavior in Splunk Cloud Dashboards and Visualizations. You can take action on raw events with workflow actions. You can also create custom workflow actions. You can find information about workflow actions in the Splunk platform documentation. • For Splunk Enterprise, see Control workflow action appearance in field and event menus in the Splunk Enterprise Knowledge Manager Manual. 162
• For Splunk Cloud, see Control workflow action appearance in field and event menus in the Splunk Cloud Knowledge Manager Manual.
Advanced Filter Some dashboards in Splunk Enterprise Security include the Advanced Filter option, which can filter items out of dashboard views ("per-panel filtering") making it easier to find those events that require investigation. • If you determine that an event is a threat, use the Advanced Filter editor to add the item to your blacklist of known threats. • If you determine that an event is not a threat, you can add it to your whitelist to remove it from the dashboard view. Note: The Advanced Filter icon won't appear unless the user has permission. To configure this permission, see Configure users and roles in the Installation and Configuration manual.
Whitelist events After you determine that an event is not a threat, you can whitelist the event to hide it from the dashboard view. The summary statistics will continue to calculate whitelisted items, but they will not be displayed in the dashboard. To whitelist an event Use the Advanced Filter to whitelist, or filter, events on a dashboard. For example, to whitelist traffic events on the Traffic Size Analysis dashboard: 1. Use the checkboxes to select the items to filter. 2. Click Advanced Filter... in the top right corner to display options for events that can be filtered in this dashboard. 3. Select the radio button to filter events on this dashboard. For example, on the Traffic Size Analysis dashboard, you can either filter events so that they no longer appear or highlight them so that they are flagged as important. 4. Click Save when you are done.
163
Note: Filtered events are not removed from the calculations for this dashboard, only removed from view. In this example, after an item is added to the whitelist, it is considered good (not a threat) and will no longer show up on the Traffic Size Analysis dashboard. To remove an item from the whitelist 1. Click Advanced Filter, then View/edit lookup file to see the list of entries currently being filtered. 2. Right-click a cell in the table to view the context menu. 3. Select Remove row to remove the row containing the whitelisted item. 4. Click Save.
Blacklist events An event can also be blacklisted. Blacklisting an item means that you have identified an event that is known to be malicious, or thought to communicate with a command and control server that is known to be malicious. Anytime the event or string shows up in the data, you will want to investigate the system, the user associated with the system, and the web activity to understand the nature and possible proliferation of the threat. Blacklisting an event or string is similar to whitelisting. Events can only be blacklisted after they have been filtered from the dashboard. To blacklist an event To blacklist a traffic event on, for example, the Traffic Size Analysis dashboard, do the following: 1. From the Advanced Filter page, click View/edit lookup files to see the list of entries currently being filtered. 2. Locate the entry you want to add to the blacklist. Under the filter column, double-click the word whitelist to edit the cell. Delete "whitelist" and type "blacklist". 3. Click Save.
Edit the per-panel filter list To see a current list of per-panel filters by dashboard, navigate to Configure > Data Enrichment > Lists and Lookups. Lists with a description indicating that they are a dashboard filter will show the current per-panel filters for that 164
dashboard. Events added to the whitelist for a dashboard will be listed here. For example, the Threat Activity Filter list displays the filters for the Threat Activity dashboard. Edit the per-panel filter list. 1. Open the filter list for the relevant dashboard. The name of the filter, for example ppf_threat_activity, is shown in the upper left-hand corner. 2. To edit a field, select a cell and begin typing. 3. To insert or remove a row or column in the filter, right-click the field for edit options. Removing a row adds that item back to the dashboard panel view and removes it from the whitelist. 4. To "blacklist" an item, use the editor to add a new row to the table and use "blacklist" in the "filter"column. 5. Click Save when you are finished. Audit per-panel filters Changes made to the per-panel filters are logged in the per-panel filtering audit logs. The lookup editor and the per-panel filter module modify per-panel filters. Use the Per-Panel Filter Audit dashboard to audit per-panel filters.
Key indicators Splunk Enterprise Security includes predefined key indicators that identify key security metrics for the security domains covered by Splunk ES. You can view the key indicators on dashboards in Splunk Enterprise Security, or add them to custom glass tables as security metrics. Key indicators provide a visual reference for several security metrics. Key indicator searches populate the security metrics of key indicators.The key indicator searches run against the data models defined in Enterprise Security, or the data models defined in the Common Information Model app. Some key indicator searches run against the count of notable events.
Key indicators on dashboards On dashboards, each key indicator includes a value indicator, a trend amount, a trend indicator, and a threshold value used to indicate the importance or priority of the indicator. The key indicator searches default to running over a relative 165
time span of 48 hours.
Field
Description
Description
Brief description of the security-related metric.
Value indicator
Current count of events. If a threshold is set, the numbers will change color as they cross thresholds. Click the value indicator to drill down into the key indicator search and view the raw events. If the value indicator is wrong, such as a percentage value greater than 100%, there could be missing or wrong data in the data model dataset used by the key indicator search to calculate a value.
Trend amount
Displays the change in event count over the time period defined in the key indicator search.
Trend indicator
Displays a directional arrow to indicate the direction of the trend. The arrow changes color and direction over time.
Edit key indicators on dashboards Enterprise Security includes preconfigured key indicators. Each dashboard key indicator row includes an editor that allows simple, visual changes to be made directly to the key indicators without leaving the dashboard. You can make changes to the search generating the key indicator on the Content Management dashboard. See Edit a key indicator search. 1. Click the Edit pencil icon to the top left of the indicator bar. The editing tools display above the indicators.
2. Drag and drop the indicators to rearrange them. There can be 5 indicators per row, and multiple indicator rows. 3. Click the checkmark icon to save.
166
Remove key indicators from a dashboard Remove a key indicator from a dashboard. 1. Click the Edit pencil icon to the top left of the indicator bar. The editing tools display above the indicators.
2. Click the X to the top right of the indicator. 3. Click the checkmark icon to save. Removing the indicator from a dashboard does not remove the key indicator from Enterprise Security. Add key indicators to a dashboard Add key indicators to a dashboard. 1. Click the Edit pencil icon to the top left of the indicator bar. The editing tools display above the indicators.
2. Click the plus icon to open the Add indicators panel. 3. Click the checkmark icon to save. Set a threshold for a key indicator on a dashboard You can set a threshold for a key indicator on a dashboard to change the color of the key indicator. A threshold defines an acceptable value for the event count of an indicator. An event count above the threshold causes the key indicator to display as red, while an event count below the threshold causes the key indicator to display as green. If the threshold is undefined, the event count remains black. 1. Click the Edit pencil icon to the top left of the indicator bar. The editing tools display above the indicators.
167
2. Type a Threshold for the key indicator. 3. Click the checkmark icon to save.
Configuring key indicator searches Configure key indicator searches on Content Management in Splunk Enterprise Security. Use the filters to select a type of key indicator to view only key indicator searches. Schedule a key indicator search Key indicators included with Splunk Enterprise Security use data model acceleration. Enable acceleration and schedule the search to run as a scheduled report. Scheduled report results are cached, allowing the indicator to display results on the dashboard more quickly. 1. Select Configure > Content Management. 2. Locate the key indicator search that you want to accelerate. 3. Click Accelerate in the Actions column. 4. In the Edit Acceleration window, select the Accelerate check box. 5. Select a Refresh Frequency for how often Enterprise Security should update the cached results. 6. Click Save. After a key indicator is accelerated, the Next Scheduled Time populates on the Content Management page and the lightning bolt for that indicator changes from grey to yellow.
Edit a key indicator search Make changes to a key indicator search. 1. From the ES menu bar, select Configure > Content Management 2. Select a key indicator search. 3. (Optional) Change the search name. 4. (Optional) Change the destination app where the search is stored. 5. (Optional) Change the title of the key indicator. The title appears above the key indicator on a dashboard, or next to the security metric on a glass 168
table. 6. (Optional) Change the sub-title of the key indicator that is used to describe the type of the key indicator function on dashboards. 7. (Optional) Change the search string that populates the key indicator. 8. (Optional) Add a drilldown URL such as a custom search or dashboard link to override the default drilldown behavior. By default, the key indicator drilldown opens the search results that produced the key indicator value. For key indicators on glass tables, you can set a custom drilldown when you add the key indicator to the glass table. 9. (Optional) Select the Schedule check box to enable acceleration for a key indicator and allow it to load faster on a dashboard. 10. (Optional) Change the Cron Schedule frequency using standard cron notation. 11. (Optional) Change the Threshold behavior to determine the color assigned to the value indicator. By default, no threshold produces a black value indicator, a threshold number higher than the count of a value indicator produces a green value indicator, and a threshold number lower than the count of a value indicator produces a red value indicator. 12. (Optional) Add a Value suffix to describe the value indicator. For example, specify units. On dashboards, the value suffix appears between the value indicator and the trend indicator. 13. (Optional) Select the Invert check box to change the default colors of the trend indicator threshold. If this check box is selected, a threshold number higher than the count of a value indicator produces a red value indicator, and a threshold number lower than the count of a value indicator produces a green value indicator. 14. Click Save.
Create custom key indicators See Creating content in Splunk Enterprise Security.
Creating new content in Splunk Enterprise Security Create new content on Content Management.
Create a correlation search See Create a correlation search in Splunk Enterprise Security Tutorials.
169
Create a key indicator search Create a key indicator search to create a key indicator that you can add to a dashboard or glass table as a security metric. 1. From the Enterprise Security menu bar, select Configure > Content Management. 2. Click Create New Content and select Key Indicator Search. 3. Type a key indicator name. In order for the key indicator to show up in the list of security metrics on glass table, type a category or security domain at the beginning of the key indicator name followed by a hyphen. For example, APT - Example Key Indicator or Access - Sample Key Indicator. 4. Type a search, and other details. The key indicators that come with Enterprise Security use data models to accelerate the return of results. 5. (Optional) Select Schedule to use data model acceleration for your custom key indicator. 6. Type the name of the field that corresponds to the value of the key indicator in the Value field. 7. Type the name of the field that corresponds to the change in the key indicator in the Delta field. 8. (Optional) Type a Threshold for the key indicator. The threshold controls whether the key indicator changes color. You can also set the threshold in dashboards and on glass tables. 9. Type a Value Suffix to indicate units or another word to follow the key indicator. 10. Select the Invert check box to invert the colors of the key indicator. Select this check box to indicate that a high value is good and a low value is bad. 11. Click Save.
Create a new saved search or scheduled report Create a saved search, also called a scheduled report, in Splunk Enterprise Security. 1. From the Enterprise Security menu bar, select Configure > Content Management. 2. Click Create New Content and select Saved Search. 3. Create a saved search, also called a scheduled report, following the instructions in the Splunk platform documentation. ♦ For Splunk Enterprise, see Create a new report in the Splunk Enterprise Reporting Manual. 170
♦ For Splunk Cloud, see Create a new report in the Splunk Cloud Reporting Manual. 4. Modify the permissions of the report to share it with Enterprise Security so that you can view and manage the search in Enterprise Security, following the instructions in the Splunk platform documentation. ♦ For Splunk Enterprise, see Set report permissions in the Splunk Enterprise Reporting Manual. ♦ For Splunk Cloud, see Set report permissions in the Splunk Cloud Reporting Manual.
Create a search-driven lookup See Create a search-driven lookup.
Create a swim lane search Create a swim lane search to create a swim lane that you can add to the Asset Investigator or Identity Investigator dashboard. Swim lanes on the investigator dashboards help you profile activity by a specific asset or identity over time. 1. From the Enterprise Security menu bar, select Configure > Content Management. 2. Click Create New Content and select Swim Lane Search. 3. Type a Search Name. 4. Select a Destination App. 5. Type a Title for the swim lane that appears on the dashboard. 6. Type a Search that populates the swim lane. 7. Type a Drilldown Search that runs when a user clicks a swim lane item. By default, the swim lane item drilldown shows the raw events. 8. Select a color. 9. Select an Entity Type of Asset or Identity. 10. Type Constraint Fields. Type a field to specify constraints on the search. Your search must contain where $constraints$ to use these constraint fields in the search. Only specific constraints are valid for each type of swim lane search. For example, an Asset Investigator swim lane search using the Malware data model and the Malware_Attacks data model dataset could specify the Malware_Attacks.user field as a constraint. 11. Click Save. For example, create a swim lane to identify all authentication events involving a specific asset. 171
1. Type a Search Name of Authentication by Asset - Example 2. Select a Destination App of DA-ESS-AccessProtection. 3. Type a Title for the swim lane that appears on the dashboard. All Authentication. 4. Type a Search that populates the swim lane. | tstats `summariesonly` values(Authentication.action) as action,values(Authentication.app) as app,values(Authentication.src) as src,values(Authentication.dest) as dest,values(Authentication.user) as user,count from datamodel=Authentication.Authentication where $constraints$ by _time span=$span$
5. Type a Drilldown Search. | `datamodel("Authentication","Authentication")` | search $constraints$
6. Select the color Purple. 7. Select an entity type of Asset because you want to investigate all authentication events by asset and be able to add this swim lane to the Asset Investigator dashboard. With this specified, all constraints specified as constraint fields perform a reverse lookup against the other fields that identify an asset. 8. Type constraint fields of Authentication.src and Authentication.dest to identify authentications originating from or targeting a specific asset. Assuming an asset lookup entry with an IP address of 1.2.3.4, dns of server.example.com, and nt_host of server1, the search for this swim lane searches for all authentication events where the source or destination of the authentication event is 1.2.3.4, server.example.com, or server1. ... Authentication.src=1.2.3.4 OR Authentication.src=server.example.com OR Authentication.src=server1 OR Authentication.dest=1.2.3.4 OR Authentication.dest=server.example.com OR Authentication.dest=server1
Create a new view or dashboard Create a new view or dashboard using Simple XML from Content Management. Prerequisite Creating new views and dashboards from Content Management requires familiarity with Simple XML. For an overview of building and editing dashboards, including working with Simple XML, see the Splunk platform documentation.
172
• For Splunk Enterprise, see Dashboard overview in Splunk Enterprise Dashboards and Visualizations. • For Splunk Enterprise, see Dashboard overview in Splunk Enterprise Dashboards and Visualizations. Task 1. From the Enterprise Security menu bar, select Configure > Content Management. 2. Click Create New Content and select View. 3. Create a new dashboard with Simple XML. 4. Modify the permissions to share the new view with Enterprise Security so that you can view and manage it in Enterprise Security. 1. From the Splunk bar, select Settings > User interface > Views. 2. Locate the View name that you created. 3. Click Permissions and modify the permissions to share the view with Enterprise Security. 4. Click Save. You can also create a new dashboard with the interactive dashboard editor. Select Search > Dashboards to open the Dashboards page. You can find information about the Dashboard Editor in the Splunk platform documentation. • For Splunk Enterprise, see Open the Dashboard Editor in Splunk Enterprise Dashboards and Visualizations. • For Splunk Cloud, see Open the Dashboard Editor in Splunk Cloud Dashboards and Visualizations. Use the Navigation editor to change which dashboards are visible on the menu in your deployment. For more information, see Navigation in this manual.
Create a glass table Create a glass table to visualize and monitor the security status of your environment. You can add security metrics like key indicators or ad-hoc searches that update in real-time against a background that you design. 1. In the Splunk Enterprise Security main menu, click Glass Tables. 2. Click Create New Glass Table. 3. Type a Title, Description, and set Permissions for your new glass table. 4. Click Create Glass Table to create the glass table. 173
Build a glass table visualization Create a glass table using the flexible canvas and editing tools on the glass table editor. 1. From the list of Glass Tables, click the name of the glass table. 2. Use the editing tools to upload images, draw shapes, add icons, add text, and make connections to reflect the relationship between the metrics. 3. In the panel of security metrics, click any metric to view the key indicator search widgets available to add. 4. Click and drag one or more of the key indicator search widgets onto the drawing canvas. A widget appears on the canvas, displaying the associated search values, which continuously update in real-time. 5. Add additional widgets to build out the dynamic elements of your visualization. 6. Click Save. Key indicator search values update at regular intervals according to the search schedule that you define when you create the search.
Configure widgets After you add a widget to your glass table, configure it to optimize performance, add a custom drilldown, and customize the widget appearance for a particular glass table design. Key indicator searches populate the widgets included in the glass table. Make changes to the key indicator searches on the Content Management dashboard. 1. In the Glass Table editor, click a widget. 2. For Custom Drilldown, click On. 3. Select a drilldown destination or type a URL. 4. For Viz Type, select an appropriate option to display your search results. Visualization types include single-value, gauge, sparkline, and single value delta. 5. Click Update to update the widget configuration. 6. Click Save.
Create and configure search widgets You can also create a custom widget to display search results. Add a new search to any glass table, define a custom search string, and customize the appearance 174
of the search widget using a variety of visualization types. Write your custom search outside of glass table to confirm that it produces expected results. Your custom search must include the timechart command, or stats by _time to use thresholding. 1. In the glass table editor, click and drag Ad hoc Search onto the canvas. 2. In the Configurations panel, for Search Type, type your custom search string. 3. Use the time picker to select the end time for your search. Defaults to Now. 4. In the Earliest Time menu, select the earliest time for the search. This determines the start time for your search, relative to the End Date and Time that you set in the time picker, and determines the time range over which your search applies. Security metrics by default display results from the previous 48 hours. For example, if the time range picker is set to Now, the security metric searches the previous 48 hours and displays results. If you change the time range picker to 6 hours ago, the security metric displays results from -54 hours to -6 hours. 5. For Threshold Field, type the field that you want to use as the threshold for your search. For example, count. 6. For Thresholds, click On to enable the thresholds for the search widget. 7. Click Edit to edit the threshold. 8. In the threshold window, add thresholds for the search widget. This determines the color of the widget, which indicates the current status of the metric. 9. Select a Viz Type for your search widget. 10. Click Update to update the widget to the new visualization and display your search results over the specified time range. 11. Click Save.
Managing glass tables Glass tables allow you to visualize security metrics in your environment in a flexible way. Manage the glass tables included with Splunk Enterprise Security and the glass tables that you create yourself on the Glass Tables lister page. • From the Splunk Enterprise Security menu bar, click Glass Tables.
175
Modify a glass table After you create a glass table, you can continue to make changes to it. 1. From the list of glass tables, click Edit next to the glass table that you want to modify. 2. Choose whether you want to edit the glass table itself, edit the title or description, or edit permissions.
Clone a glass table to make a template You can clone a glass table to make a template, or to preserve a glass table included with Splunk Enterprise Security as an original and make experimental changes on another version. 1. From the list of glass tables, click Edit next to the glass table that you want to modify. 2. Click Clone. 3. Type a new title. 4. (Optional) Type a new description. 5. (Optional) Change the permissions of the cloned glass table. 6. Click Clone Page.
Access to glass tables All users can view glass tables, but you must have the ess_analyst, ess_admin, or admin role or have the capability to edit glass tables to create glass tables. See Configure users and roles.
Searches available to glass tables Ad hoc search widgets that you create on individual glass tables cannot be shared automatically with other glass tables. Key indicator searches populate the list of security metrics available to add as predefined widgets, and those can be edited on the Content Management page. See Creating new content in Splunk Enterprise Security.
Performance and storage of glass tables Glass table content is stored in the KV store. The glass table definitions are stored in the SplunkEnterpriseSecuritySuite_glasstables collection. Files added to glass tables, such as images, are stored in the 176
collection. Custom widgets, images, and other items that you add to a glass table are all stored in this collection. SplunkEnterpriseSecuritySuite_files
The performance of individual glass tables depends on the number of search widgets on a glass table. When you open a glass table for viewing, each search runs at the same time. Searches on glass tables with 200 or more search widgets could take 10-15 seconds to show data on the glass table.
Audit dashboards Use the audit dashboards to validate the security and integrity of the data in Enterprise Security. Ensure that forwarders are functioning, that data has not been tampered with and is secured in transmission, and that analysts are reviewing the notable events detected by correlation searches.
Incident Review Audit The Incident Review Audit dashboard provides an overview of incident review activity. The panels display how many incidents are being reviewed and by which user, along with a list of the most recently reviewed events. The metrics on this dashboard allow security managers to review the activities of analysts. Panel Review Activity by Reviewer
Description Displays the numbers of events reviewed by each user. This panel is useful for determining which user is performing the incident reviews and if the total number of incidents reviewed is changing over time. The drilldown opens a search with all activity by the selected reviewer.
Displays the top users that have performed incident reviews. The panel includes details for each user, including the date they first performed an incident review, the date they last performed Top Reviewers a review, and the total number of incidents reviewed. The drilldown opens a search with all activity by the selected reviewer. Notable Events By Status - Last 48 Hours
Displays the status, count, and urgency for all notable events in the last 48 hours. This panel is useful for determining if the incident review users are keeping up with incidents, or whether a backlog of unreviewed incidents is forming. The drilldown opens the Incident Review dashboard and searches on the selected urgency and status over the lat 48 hours. 177
Notable Events By Owner - Last 48 Hours
Displays the owner, count, and urgency for all notable events in the last 48 hours. This panel is useful for determining how many events are assigned to a user and the urgency of the events. The drilldown opens the Incident Review dashboard and searches on the selected urgency over the lat 48 hours.
Mean Time to Triage - Last 14 days
Displays the average time it took for a notable event to be triaged after it was created over the last 14 days, split by the name of the notable event. This panel is useful for determining how quickly analysts are triaging notable events, or whether certain types of events take longer to triage than others. The drilldown opens the Incident Review dashboard and searches on the matching notable event names over the last 14 days.
Displays the average time it took for a notable event to be closed after it was created over the last 60 days, split by the Mean Time to name of the notable event. This panel is useful for determining Closure - Last how long it takes to close certain types of notable event investigations. The drilldown opens the Incident Review 60 days dashboard and searches on the matching notable event names that have a status of closed from the last 60 days. Displays the 10 most recent changes on the incident review Recent dashboard, such as triage actions. The drilldown opens a Review search with the selected rule ID. Activity To audit data from Incident Review from Enterprise Security prior to version 3.2, you must perform an ad hoc search like this example.
index=_audit sourcetype=incident_review | rex field=_raw "^(?[^,]*),(?[^,]*),(?[^,]*),(?[^,]*),(?[^,]*
Data sources The reports in the Incident Review Audit dashboard reference fields in the notable index and the incident review objects in a KVStore collection. See Notable event index in this manual for more on the notable index.
Suppression Audit The Suppression Audit dashboard provides an overview of notable event suppression activity. This dashboard shows how many events are being suppressed, and by whom, so that notable event suppression can be audited and reported on.
178
The metrics on this dashboard allow security managers to review the activities of analysts, which is useful for tuning correlation searches. You can identify correlation search rules that are generating more events than your analysts are capable of looking at, and tune them accordingly. Panel
Description
Suppressed Events Over Displays notable events suppressed in Time - Last 24 the last 24 hours. Hours Suppression History Over Displays the history of suppressed Time - Last 30 notable events. Days Suppression Management Activity
Displays suppression management activity for the time period.
Expired Displays expired suppressions. Suppressions Data sources The reports in the Suppression Audit dashboard reference events in the Notable index.
Per-Panel Filter Audit The Per-Panel Filter Audit dashboard provides information about the filters currently in use in your deployment. The following table describes the panels for this dashboard. Panel
Description
Per-Panel Displays the count of updates to By Reviewer per-panel filters by user Top Users
Shows users, sparkline for trends, number of views, and first and last time viewed.
Recent Filter
Activity by time, user, action, and filename 179
Activity
Adaptive Response Action Center The Adaptive Response Action Center dashboard provides an overview of the response actions initiated by adaptive response actions, including notable event creation and risk scoring. Panel
Description
Action Invocations Over Time By Name
Displays a time chart of the adaptive response actions triggered by name.
Top Actions By Name
Displays the top adaptive response actions by name.
Top Actions By Search
Displays the top adaptive response actions by search.
Recent Response Actions
Displays the most recent adaptive response actions.
Data sources The reports in the Adaptive Response Action Center dashboard reference fields in the Audit data model. For a list of data model objects and constraints, see Splunk Audit Logs in the Common Information Model Add-on manual.
Threat Intelligence Audit The Threat Intelligence Audit dashboard tracks and displays the current status of all threat intelligence sources. As an analyst, you can review this dashboard to determine if threat sources are current, and troubleshoot issues connecting to threat intelligence sources. Panel Threat Intelligence Downloads
Description Displays the status of all threat sources defined on the Threat Intelligence Downloads configuration page. Use the filters to sort by status or download location.
Threat Displays log events related to threat Intelligence intelligence downloads configured on Audit Events the Threat Intelligence Downloads and Threat Intelligence Manager 180
configuration pages. Use the filters to sort and filter the events displayed. A system message is automatically created if a threat source download fails. Data sources The reports in the Threat Intelligence Audit dashboard reference events in the _internal index and state information from the /services/data/inputs/threatlist REST endpoint.
ES Configuration Health Use the ES Configuration Health dashboard to compare the latest installed version of Enterprise Security to prior releases and identify configuration anomalies. The dashboard does not report changes to add-ons (TA.) Select the previous version of Enterprise Security installed in your environment using the Previous ES Version filter. Mode
Unshipped
Removed Stanzas
Description The Unshipped setting compares the latest installed version of Enterprise Security with the content in the ES installation package. Any item that was not provided as part of the Enterprise Security installation, such as files or scripts used for customization, is labeled as an Unshipped item. Review Unshipped items to evaluate their use, determine if they are still needed, and reconcile if necessary. The Unshipped setting ignores the Previous ES Version filter. The Removed Stanzas setting compares the latest installed version of Enterprise Security with the version that you select in the filter. Removed Stanzas are configuration stanzas that changed between versions, such as a deprecated threat list or input. Review Removed Stanzas to evaluate their use, determine if they are still needed, and 181
reconcile if necessary.
Local Overrides
The Local Overrides setting compares the installed version of Enterprise Security with the version that you select in the filter. A setting that conflicts with or overrides the installed version of Enterprise Security is labeled as a Local Override. Review any Local Override settings to evaluate their use, determine if they are still needed, and reconcile if necessary.
Content Profile The Content Profile dashboard compares the knowledge objects provided in Enterprise Security to the data models the objects require, and expresses the results as a deployment completeness percentage. Panel
Description
Displays the percentage of knowledge objects provided in Enterprise Security that reference Deployment populated data models. To achieve Completeness 100%, each data model referenced by a knowledge object must have data.
Unused Data Models
Displays the number of unpopulated data models. If you select Unused Data Models, the Data Model Info view displays only the unpopulated data models.
Unused Knowledge Objects
Displays the number of unavailable knowledge objects due to an unpopulated data model.
Data Model Info
Displays each of the data models by name. A selection drop down displays a detailed breakdown of the object type and name. For example, selecting the Application_State data model displays over 20 unique searches, and panels by app. 182
Data sources The reports in the Content Profile dashboard examine the knowledge objects in Enterprise Security and determine which data models they use. If the referenced data model contains events, the knowledge objects are considered available and complete. The analysis of a knowledge object is limited to the data model name, and does not extend to the data model objects referenced in a knowledge object. Therefore, this dashboard can display 100% completeness and a search or view can still display no data. To verify there is accelerated data in a data model, use the Data Model Audit dashboard. To verify the data model objects referenced by a dashboard panel, see Dashboard Requirements Matrix in this manual.
Data Model Audit The Data Model Audit dashboard displays information about the state of data model accelerations in your environment. Field Name Panel
Description
Displays the accelerated data models Top Accelerations sorted in descending order by MB on disk By Size Top Displays the accelerated data models Accelerations sorted in descending order by the time By Run spent on running acceleration tasks. Duration Displays a table of the accelerated Accelerations data models with additional Details information. Data sources The reports in the Data Model Audit dashboard reference fields in the Splunk Audit data model. For a list of data model objects and constraints, see Splunk Audit Logs in the Common Information Model Add-on Manual.
183
Forwarder Audit The Forwarder Audit dashboard reports on hosts forwarding data to Splunk Enterprise. Use the search filters and time range selector to focus on groups of forwarders or an individual forwarder. Filter by
Description
Action
Show only expected hosts
An expected host is a host defined in ES by the expected host field is_expected in the Asset table.
Drop-down, select to filter by
Host
Filter by the host field in the Asset table.
Text field. Wildcard with an asterisk (*)
Business Unit
Filter by the business unit bunit field in Text field. Wildcard the Asset table. with an asterisk (*)
Category
Filter by the category field in the Asset table.
Panel
Drop-down, select to filter by
Description
Event Count Displays the number of events reported Over Time over the time period selected in the filter. The events are split by host. By Host Hosts By Displays a list of hosts, ordered by the Last Report last time they reported an event. Time Splunkd Process Utilization
Displays the resource utilization of the forwarder's Splunk daemon splunkd.
Splunk Service Start Mode
Displays the host names that are forwarding events, but are not configured to have splunkd start on boot.
Data sources Relevant data sources for the Forwarder Audit dashboard include data from all forwarders in your Splunk environment and the Application_State data model. See the Common Information Model Add-on Manual for more information. The Common Information Model fields bunit and category are derived by automatic identity lookup, and do not need to be mapped directly. 184
Indexing Audit The Indexing Audit dashboard is designed to help administrators estimate the volume of event data being indexed by Splunk Enterprise. The dashboard displays use EPD (Events Per Day) as a metric to track the event volume per index, and the rate of change in the total event counts per index over time. The EPD applies only to event counts, and is unrelated to the Volume Per Day metric used for licensing. Panel
Description
Key Indicators
The key indicators on this dashboard are scoped to "All Time," not the "Last 24 hours".
Events Per Day Over Time
Displays a column chart representing the event counts per day.
Events Per Day
Displays a table representing event counts per day and the average eps.
Events Per Displays a table of event counts per Index (Last index for the last day. Day) Data sources The reports in the Indexing Audit dashboard reference data generated by the Audit - Events Per Day - Lookup Gen saved search and are stored within a KVStore collection.
Search Audit The Search Audit dashboard provides information about the searches being executed in Splunk Enterprise. This dashboard is useful for identifying long running searches, and tracking search activity by user. Panel
Searches Over Time by Type
Description Shows the number of searches executed over time by type, such as ad-hoc, scheduled, or real-time. Helps determine whether Splunk's performance is being affected by excessive numbers of searches. 185
Searches Over Time by User
Shows the number of searches executed by each user. Helps determine when a particular user is executing an excessive number of searches. The splunk-system-user is the name of the account used to execute scheduled searches in Splunk Enterprise.
Lists the most expensive searches in Top terms of duration. Helps to identify Searches by specific searches that may be adversely Run Time affecting Splunk performance. Data sources The reports in the Search Audit dashboard reference scheduled search auditing events from the audit index.
View Audit The View Audit dashboard reports on the most active views in Enterprise Security. View Audit enables tracking of views that are being accessed on a daily basis and helps to identify any errors triggered when users review dashboard panels. Panel
Description
Displays the Enterprise Security views that have the greatest access counts View Activity over time. The drilldown opens a search Over Time view of all page activity for the time selected. Lists the views set up in the Expected View list. You want to review these views on a daily basis for your deployment. Select a dashboard to see Expected details in the Expected View Scorecard View Activity panel below. Use Configure > Lists and Lookups > Expected Views to set up the Expected View list.
186
Web Service Errors
Displays errors that occurred while loading the web interface. Helps identify custom views that contain errors or an underlying issue that need to be escalated to Splunk.
Data sources The reports in the View Audit dashboard reference fields in the Splunk Audit data model. For a list of data model objects and constraints, see Splunk Audit Logs in the Common Information Model Add-on Manual.
Data Protection The Data Protection dashboard reports on the the status of the data integrity controls. Panel
Description
Data Integrity Control By Index
Displays a view of all indexes with data protection enabled, sorted by search peer. For more information on configuring and validating data integrity, see Manage data integrity in Securing Splunk Enterprise. If you use Splunk Cloud, file a support case to request enablement of data integrity control.
Sensitive Data
Displays the count of events with sensitive data. This panel requires enabling the Personally Identifiable Information Detected correlation search.
Predictive Analytics dashboard Use the Predictive Analytics dashboard to search for different varieties of anomalous events in your data. Predictive Analytics uses the predictive analysis functionality in Splunk to provide statistical information about the results, and identify outliers in your data. The predict command can take some time to generate results. To analyze data with predictive analytics, choose a data model, then an object, a function, an attribute, and a time range, and click Search. 187
Dashboard filters Use the available dashboard filters to refine the results displayed on the dashboard panels. The Predictive Analytics dashboard filters are implemented in a series from left to right. For example, the Object filter is populated based on the Data Model selection. Filter by
Description
Data Model
Specifies the data model for the search. Available data models are shown in the drop-down list.
Object
Specifies the object within the data model for the search. You must select a Data Model to apply an Object.
Function
Specifies the function within the object for the search. Functions specify the type of analysis to perform on the search results. For example, choose "avg" to analyze the average of search results. Choose "dc" to create a distinct count of the results.
Attribute
Specifies the constraint attributes within the object for the search. Attributes are constraints on the search results. For example, choose "src" to view results from sources. You must select an Object to apply an Attribute.
Time Range
Select the time range to represent.
Advanced Access to the options for the predict command. You can find information about the predict command options in the Splunk platform documentation. • For Splunk Enterprise, see predict options in the Splunk Enterprise Search Reference. • For Splunk Cloud, see predict options in the Splunk Cloud Search Reference. Dashboard Panels Panel
Prediction Over Time
Description The Prediction Over Time panel shows a predictive analysis of the results over time, based on the time range you chose. The shaded area shows results that fall within two standard deviations of the mean value of the total search results. 188
Outliers
The Outliers panel shows those results that fall outside of two standard deviations of the search results.
Data sources The Predictive Analytics dashboard references data in any user selected data model. If the data model accelerations are unavailable or incomplete for the chosen time range, the dashboard reverts to searching unaccelerated, raw data.
Create a correlation search From this dashboard, create a correlation search based on the search parameters for your current predictive analytics search. This correlation search will create an alert when the correlation search returns an event. 1. Click Save as Correlation Search... to open the Create Correlation Search dialog. 2. Select the Security domain and Severity for the notable event created by this search. 3. Add a search name and description. 4. Click Save.
To view and edit correlation searches, go to Configure > Content Management. See Configuring correlation searches in this manual.
Troubleshooting This dashboard references data from various data models. Without the applicable data, the panels will remain empty. See Dashboard Troubleshooting in this manual.
189
Included Add-ons Analyze Splunk UBA threats and anomalies in Splunk ES Use the threats and anomalies identified by Splunk User Behavior Analytics alongside the notable events in Splunk Enterprise Security to gain further insight into your environment's security posture. This is one point of integration with Splunk UBA. To analyze Splunk UBA threats and anomalies in Enterprise Security, set up your environment. See Send UBA Threats and Anomalies to Splunk ES in the Splunk UBA User Manual. To see both threats and anomalies in ES, you must have Splunk UBA version 2.1.2 or later. After you set up this integration, you can investigate UBA threats and anomalies in Enterprise Security.
View threats on Security Posture and Incident Review Splunk UBA threats appear as notable events in Enterprise Security. View a count of UBA threats as a UBA notables key security indicator on the Security Posture dashboard. • You can see the specific notable events created by Splunk UBA threats on the Incident Review dashboard. • Expand the notable event details to see more about the Splunk UBA threat. The description, threat category, and correlation search reference UBA. • Use the event workflow actions to View Contributing Anomalies and open the Threat Details in Splunk UBA.
View anomalies on the UBA Anomalies dashboard Use the UBA Anomalies dashboard to understand anomalous activity in your environment. To view the dashboard, select Security Intelligence > User Intelligence > UBA Anomalies. • See how the count of various metrics have changed over the past 48 hours in your environment with the key indicators. Review the count of 190
UBA notables, UBA anomaly actors, UBA anomaly signatures, UBA anomalies per threat, and the total count of UBA anomalies. • Investigate spikes in anomalous activity and compare the number of actors with the number of anomalies over time on the Anomalies Over Time panel. • Identify the most common types of anomalous activity on the Most Active Signatures panel. • Determine which users, devices, apps, and other actors are responsible for the most anomalous activity on the Most Active Actors panel. • See the latest anomalous activity on the Recent UBA Anomalies panel.
View an anomaly in Splunk UBA by clicking on a value on the dashboard to drill down to the search. Use the event actions on a specific anomaly event to View Contributing Anomalies and open Splunk UBA to view the Anomaly Details view. See Anomaly Details in the Splunk UBA User Manual. Note: This dashboard displays in the Splunk Enterprise Security menu bar after you integrate Splunk UBA and Splunk ES. See Send UBA Threats and Anomalies to Splunk ES. You can manually add the dashboard to the navigation before you complete the integration. See Navigation.
View threat and anomaly swim lanes on the Entity Investigator dashboards You can use swim lanes on the Asset and Identity Investigator dashboards to correlate counts of UBA threats and anomalies with other notable events in ES. To see anomaly and threat information associated with each asset or identity that you search, add the UEBA Threats and UBA Anomalies swim lanes to the Asset Investigator and Identity Investigator dashboards. See Edit the swim lanes. View an anomaly in Splunk UBA by clicking the swim lane and opening a drill down to the search. Use the event actions to View Contributing Anomalies and open Splunk UBA to view the Anomaly Details or Threat Details. See Review current threats for more.
Anomalies and threats modify risk scores Splunk ES uses the risk score of anomalies and threats in Splunk UBA to modify risk for the assets and identities associated with the threats and anomalies. The risk modifier is 10 times the risk score of the anomaly or threat in Splunk UBA. 191
For example: • Splunk UBA sends Splunk ES an anomaly that applies to the host 10.11.12.123. The anomaly has a risk score of 8. • Splunk ES modifies the risk for the host 10.11.12.123 in response to the anomaly. A risk modifier of 10 * UBA risk score results in a risk modifier of 80. You can see the source of increased risk when analyzing risk scores on the Risk Analysis dashboard.
Investigate anomalous activity and threats in Enterprise Security View the raw threats and anomalies sent from Splunk UBA to Enterprise Security using these searches. • View all UBA anomalies and threats sent to Enterprise Security: | datamodel UEBA All_UEBA_Events search
• View all UBA threats sent to Enterprise Security: | datamodel UEBA UEBA_Threats search
• View all UBA anomalies sent to Enterprise Security: | datamodel UEBA UEBA_Anomalies search
Send correlation search results to Splunk UBA to be processed as anomalies If your environment includes both Splunk User Behavior Analytics (UBA) and Splunk Enterprise Security, you can send the results of correlation searches from Splunk ES to Splunk UBA to be processed as anomalies. Anomalies that result from correlation search results can then be used in Splunk UBA to generate threats. You must have version 3.0 of Splunk UBA in order for the correlation search results to be processed successfully. You can also set up Splunk UBA to send anomalies and threats to Splunk ES. See Analyze Splunk UBA threats and anomalies in Splunk ES for more. 192
Set up Splunk ES to send correlation search results to Splunk UBA Before you can send correlation search results from Splunk Enterprise Security to Splunk UBA, set up the Splunk UBA management server as an output location. You must have the ess_admin role or the edit_forwarders capability to set up this connection. 1. From the Splunk ES menu bar, select Configure > UBA Setup. 2. In the Management server field, type the host name and port of the Splunk UBA management server. 3. In the Type field, select whether to use the TCP or UDP protocol to send the notable events to Splunk UBA. 4. Click Save. You must restart the Splunk platform after setting up this connection. If you are on a search head cluster, use the deployer to deploy the change from the Splunk_TA-ueba outputs.conf file to the cluster members.
Set up Splunk UBA to receive correlation search results from Splunk ES Set up a new data source in Splunk UBA to receive correlation search results from Splunk Enterprise Security. 1. In Splunk UBA, select Config > Data Sources and click New Data Source. 2. Select a data source of Netcat. 3. Specify a name for the data source, such as ESnotables. The data source name must be alphanumeric, with no spaces or special characters. 4. Select a format of SplunkES Correlation Search. 5. Click Next. 6. Deselect the check box for Test Mode. 7. Click OK to save the new data source.
Send correlation search results to Splunk UBA After you set up Enterprise Security and Splunk UBA, you can start sending correlation search results to Splunk UBA. You can send correlation search results automatically, or you can send correlation search results in an ad-hoc manner by sending notable events from the Incident Review dashboard.
193
Automatically send correlation search results to Splunk UBA Edit an existing correlation search or create a new correlation search to add a response action of Send to UBA to automatically send correlation search results to Splunk UBA. 1. From the Splunk ES menu bar, select Configure > Content Management. 2. Click the name of a correlation search or click Create New to create a new correlation search. 3. Click Add New Response Action and select Send to UBA. 4. Type a Severity to set the score in Splunk UBA for an anomaly that might be created from the correlation search result. For example, type 7 to represent a high severity. 5. Save the correlation search. Send correlation search results ad-hoc from Incident Review Send notable events created by correlation search results to Splunk UBA in an ad-hoc manner from the Incident Review dashboard. 1. On the Incident Review dashboard, locate the notable event that you want to send to Splunk UBA. 2. From the Actions column, select Run Adaptive Response Actions. 3. Click Add New Response Action and select Send to UBA. 4. (Optional) Type a Severity to set the score in Splunk UBA for the anomaly that might be created from the notable event. The notable event severity, if available, takes precedence over the provided value. 5. Click Run to run the response action and send the notable event details to Splunk UBA.
Types of results to send to Splunk UBA Only some correlation search results create anomalies in Splunk UBA. Splunk UBA parses the correlation search results as external alarms, and correlation searches with a source, destination, or user in the results are most likely to produce anomalies in Splunk UBA. Not all correlation search results sent from Splunk ES will appear as anomalies in Splunk UBA. Splunk UBA only triggers anomalies for the correlation search results with relevant data, and ignores other correlation search results.
194
Configurations and Troubleshooting Configuration Settings As a Splunk Enterprise administrator, you can make configuration changes to your Splunk Enterprise Security installation.
General Settings Change threshold values, macro definitions, search filters, and other commonly changed values on the General Settings page. 1. On the Enterprise Security menu bar, select Configure > General > General Settings. Setting
Description
Asset Sources
A search macro that enumerates the lookup tables that contain asset information used for asset correlation.
Auto Pause
Type the time in seconds before a drilldown search will pause.
Default Watchlist Define the watchlisted events for the 'Watchlisted Events' Search correlation search Domain Analysis Enable or disable WHOIS tracking for Web domains. Domain From URL Extraction Regex
A regular expression used to extract domain (url_domain) from a URL.
Enable Identity Generation Autoupdate
If true, permit the Identity Manager to auto-update asset_sources, identity_sources, and generate_identities macros. True by default.
HTTP Category Analysis Sparkline Earliest
Set the start time for sparklines displayed on the HTTP User Category Analysis dashboard.
HTTP Category Analysis Sparkline Span
Set the time span for sparklines displayed on the HTTP User Category Analysis dashboard.
195
HTTP User Agent Analysis Sparkline Earliest
Set the start time for sparklines displayed on the HTTP User Agent Analysis dashboard.
HTTP User Agent Analysis Sparkline Span
Set the time span for sparklines displayed on the HTTP User Agent Analysis dashboard.
IRT Disk Sync Delay
Set the number of seconds for Enterprise Security to wait for a disk flush to finish. Relevant to Indexed real time searches.
Identity Generation
Defines the transformations used to normalize identity information. See How Splunk Enterprise Security processes and merges asset and identity data
Identity Generation Timeout
Number of seconds the Identity Manager waits before warning of slow search completion in identity_manager.log.
Identity Sources
Enumerates the source lookup tables that contain identity information.
Incident Review Estimated maximum capacity of notable events assigned to Analyst Capacity an analyst. Relative measure of analyst workload. Indexed Realtime
Enable or disable indexed real-time mode for searches.
Large Email Threshold
An email that exceeds this size in bytes is considered large.
Licensing Event Define the list of indexes to exclude from the "Events Per Count Filter Day" summarization. New Domain Analysis Sparkline Span
Set the time span for sparklines displayed in the New Domain Analysis dashboard.
Notable Modalert Pipeline
SPL for the notable adaptive response action.
Predicate pushdown
Dynamically replaces |datamodel drilldown searches with a new search that approximates the same constraints to speed up the return of events in search results. If a drilldown search references an evaluated field (example: src="unknown",) the replacement drilldown search will always return No results found. The setting is global and changes the Incident Review notable 196
event drilldown for contributing events and all simple XML dashboards that use a specific search syntax for drilldown searches. Disabled by default. Change this setting only if the search head performance of drilldown searches is unacceptable. If you use Splunk platform version 6.5.x or later, search optimization of this type happens automatically. You do not need to select this setting to get the optimization benefits. Search Disk Quota (admin)
Set the maximum amount of disk space in MB that an admin user can use to store search job results.
Search Jobs Quota (admin)
Set the maximum number of concurrent searches allowed for admin users.
Search Jobs Quota (power)
Set the maximum number of concurrent searches for power users.
Short Lived Account Length
An account creation and deletion record that exceeds this threshold is anomalous.
TSTATS Allow Old Summaries
Enable or disable searching of data model accelerations containing fields that do not match the current data model configuration.
TSTATS Local
Determine whether or not the TSTATS macro will be distributed.
TSTATS Determine whether or not the TSTATS or summariesonly Summaries Only macro will only search accelerated events. Use Other
Enable or disable the term OTHER on charts that exceed default series limits.
Website A list of watchlisted websites used by the "Watchlisted Watchlist Search Events" correlation search.
Credential Management The Credential Management page displays stored credentials for objects, such as threat lists or lookups, that run as scripted or modular inputs. An input configuration that references a credential will attempt to find the credential values here. Add a new credential for an input 1. On the Enterprise Security menu bar, select Configure > General and open Credential Management. 2. Click New Credential to add a new user credential. 197
3. Use the edit panel to add the username and password for the new credential.
4. (Optional) Use the Realm field to differentiate between multiple credentials that have the same username. 5. Select the Application for the credential. 6. Click Save. Edit an existing input credential 1. On the Enterprise Security menu bar, select Configure > General and open Credential Management. 2. In the Action column of a credential, select Edit. 3. Use the editor to change the username, password, or application for the credential. You cannot change the realm after it has been applied to a credential. You must create a new credential to change the realm.
4. Click Save. Delete an existing input credential 1. On the Enterprise Security menu bar, select Configure > General and open Credential Management. 2. In the Action column of a credential, select Delete.
Permissions Use the Permissions page to view and assign Enterprise Security capabilities to non-admin roles. 198
1. On the Enterprise Security menu bar, select Configure > General > Permissions. 2. Select the checkbox for the role and permissions for that role. 3. Click Save. For more information about ES capabilities, see Adding capabilities to a role in the Installation and Upgrade Manual.
Customize the menu bar in Splunk Enterprise Security Customize the menu bar in Splunk Enterprise Security with the navigation editor. Add new dashboards, reports, views, links to filtered dashboards, or links to the web to your menu bar. You must have Enterprise Security administrator privileges to make changes to the menu bar navigation. Upgrading Enterprise Security overwrites customizations you make to the menu bar. You can add views to the menu bar as part of a collection that groups several views together or as an individual item on the menu bar. For example, Incident Review is an individual dashboard in the menu bar, and Audit is a collection of the audit dashboards. Set a default view for Splunk Enterprise Security To see a specific view or link when you or another user opens Splunk Enterprise Security, set a default view. 1. On the Enterprise Security menu bar, select Configure > General > Navigation. 2. Locate the view or link that you want to be the default view. 3. Click the checkmark icon that appears when you mouse over the view to Set this as the default view.
4. Click Save to save your changes 5. Click OK to refresh the page and view your changes. Edit the existing menu bar navigation 1. On the Enterprise Security menu bar, select Configure > General > Navigation. 199
2. Click and drag views or collections of views to change the location of the views or collections of views in the menu. 3. Click the X next to a view or collection to remove it from the menu. 4. Click the icon to edit the name of a collection. 5. Click the icon to add a divider and visually separate items in a collection. 6. Click Save to save your changes 7. Click OK to refresh the page and view your changes. Add a single view to the menu bar You can add a new view to the menu bar without adding it to a collection. 1. On the Enterprise Security menu bar, select Configure > General > Navigation. 2. Click Add a New View. 3. Leave View Options set to the default of View. 4. Click Select a View from Unused Views. 5. Select a dashboard or view from the list. 6. Click Save. The dashboard appears on the navigation editor. 7. If you are finished adding items to the menu, click Save to save your changes 8. Click OK to refresh the page and view your changes. Add a collection to the menu bar Use a collection to organize several views or links together in the menu bar. 1. On the Enterprise Security menu bar, select Configure > General > Navigation. 2. Click Add a New Collection. 3. Type a Name. For example, Audit. 4. Click Save. The collection appears on the navigation editor. You must add a view or link to the collection before it appears in the menu navigation. Add a view to an existing collection Add views to an existing collection. 1. On the Enterprise Security menu bar, select Configure > General > Navigation. 2. Locate the collection that you want to add views to. 200
3. Click the icon. 4. Leave View Options set to the default of View. 5. Click Select a View from Unused Views. 6. Select a view from the list. 7. Click Save. The view appears on the navigation editor. 8. If you are finished adding items to the menu, click Save to save your changes 9. Click OK to refresh the page and view your changes. Add a link to the menu bar You can add a link to the menu bar of Splunk Enterprise Security. For example, add a link to a specifically-filtered view of Incident Review or to an external ticketing system. Create a link in the menu to an external system or webpage
1. On the Enterprise Security menu bar, select Configure > General > Navigation. 2. Click Add a New View to add it to the menu, or locate an existing collection and click the icon to add the link to an existing collection of views. 3. Select Link from View Options. 4. Type a Name to appear on the Splunk Enterprise Security menu. For example, Splunk Answers. 5. Type a link. For example, https://answers.splunk.com/ 6. Click Save. 7. If you are finished adding items to the menu, click Save to save your changes 8. Click OK to refresh the page and view your changes. Add a link to a filtered view of Incident Review
A common link to add to the menu bar is a filtered view of Incident Review. 1. Filter Incident Review with your desired filters. When you filter the dashboard, the URL updates with query string parameters matching your filters. 2. In the web browser address bar, copy the part of the URL that starts with /app/SplunkEnterpriseSecuritySuite/ and paste it in a plain text file for reference. For example, if you filtered the dashboard to show only critical notable events, the part of the URL that you copy looks like 201
/app/SplunkEnterpriseSecuritySuite/incident_review?form.selected_urgency=critical.
3. On the Enterprise Security menu bar, select Configure > General > Navigation. 4. Click Add a New View to add it to the menu, or locate an existing collection and click the Add View icon to add the link to an existing collection of views. 5. Select Link from View Options. 6. Type a Name to appear on the Splunk Enterprise Security menu. For example, IR - Critical. 7. In the Link field, paste the URL section. For example, /app/SplunkEnterpriseSecuritySuite/incident_review?form.selected_urgency=critical
8. Click Save. 9. If you are finished adding items to the menu, click Save to save your changes. 10. Click OK to refresh the page and view your changes. If you add a link with multiple parameters you must modify the query string parameters by encoding the & separating the parameters as &. For example, type the link for a filtered view of Incident Review that shows new and unassigned notable events as
/app/SplunkEnterpriseSecuritySuite/incident_review?form.status_form=1&form.owner_for
You can also construct a URL manually using the parameters in the following table. Use an asterisk to show all results for a specific parameter. Parameter
Description
Possible values
form.selected_urgency
Display notable events critical, high, with the medium, low, urgency informational specified by this parameter.
form.status_form
Display notable events with the status specified by this parameter. An integer corresponds to each status value. 202
form.selected_urgency=critica
0 for unassigned, 1 for new, 2 for in progress, 3 for form.status_form=0 pending, 4 for resolved, 5 for closed
form.owner_form
Display notable events owned by the usernames user specified by this parameter.
form.rule_name
Display notable events created by the correlation search specified by this parameter. HTML-encode spaces in the correlation search name and use the name that appears in the notable event rather than the name that appears on Content Management.
form.tag
Displays notable events malware, any with the tag custom tag specified by value this paramter.
form.srch
form.owner_form=admin
Endpoint - Host With Multiple form.rule_name=Endpoint%20-%2 Infections Rule
form.tag=malware
Displays dest=127.0.0.1 form.srch=dest%3D127.0.0.1 notable events that match the SPL specified in this parameter. HTML-encode special characters such as = for 203
key-value pairs.
form.security_domain_form
earliest=
and latest=
Displays notable events in the security domain specified by this parameter.
access, endpoint, form.security_domain_form=end network, threat, identity, audit
Displays notable events in the time range specified by these parameters. -24h@h, now Specify a relative time range. HTML-encode special characters such as @.
Displays notable events that do not critical, high, form.new_urgency_count_form have the medium, low, urgency informational specified by this parameter.
earliest=-24h%40h&latest=now
form.new_urgency_count_form=i
Content Management You can use the Content Management page to display, configure, and edit the correlation searches, key indicators, saved searches, and swim lane searches unique to Splunk Enterprise Security. • See Configuring correlation searches for more on configuring and modifying correlation searches. • See Creating new content in Splunk Enterprise Security for more on creating or modifying correlation searches, key indicators, saved searches, and swim lane searches. 204
• See Export content as an app from Splunk Enterprise Security for more on exporting content from the Content Management page.
Configure lists and lookups To configure or edit the lists or lookup files used with the Splunk Enterprise Security, select Configure > Data Enrichment > Lists and Lookups. Use Lists and Lookups to view and edit the default lists and lookups in Enterprise Security. Click the name of a list to view or edit it. Click Export to export a copy of the file in CSV format.
Internal lookups Splunk Enterprise Security maintains internal lookups to provide information for dashboards or to create notable events. See Available internal lookups for more details on the lookups included with Splunk Enterprise Security. These lookups are created in three ways. • Populated by a static lookup table. • Populated internally by search commands, called a search-driven lookup. • Populated with information from the Internet. The internal lookups populated with information from the Internet are used by some correlation searches to identify hosts that are recognized as malicious or suspicious according to various online sources, such as the SANS Institute. If Splunk Enterprise Security is not connected to the Internet, the lookup files are not updated and the correlation searches that rely on the lookups might not function correctly. Most of the internal lookups populated by the Internet are threat intelligence sources. See Threat intelligence sources included with Splunk Enterprise Security in this manual.
Edit lists and lookups From the ES menu bar, select Configure > Data Enrichment > Lists and Lookups to view the list of current lookup files. Click a file name to open that lookup file in the lookup editor.
205
The name of the CSV file is shown in the upper left-hand corner of the panel, assets.csv in this example. The lookup fields are shown at the top of the table, the values for the fields are displayed in the rows below that. Positive numbers are in green, negative numbers are shown in red. The priority values in this file are color-coded. Each CSV file looks slightly different depending on the fields it contains. Lookups do not accept regular expressions. Only users with appropriate permissions can edit lookups. See Permissions in this manual to edit permissions for a user role. Edit lookup content 1. From the ES menu bar, select Configure > Data Enrichment > Lists and Lookups to view the list of current lookup files. 2. Click a file name to open that lookup file in the lookup editor. 3. Change a value in a cell by selecting the cell and typing the new value. 4. Right-click the table to open a context menu that you can use to add columns or rows to the file. 5. Click Save to save your changes or Cancel to return to the list of lookups without saving. Note: You cannot save a lookup file that contains empty header fields. To review the last time a lookup file was edited and by whom, use a search. For example index=_internal uri_path="/splunk-es/en-US/app/SplunkEnterpriseSecuritySuite/ess_lookups_edit"
206
Add new lookup files An admin might add new CSV files to support new functions and data enrichment in the application. CSV files used as lookups must be created with Unix-style line endings (\n). Note: CSV files used as lookups must be created with Unix-style line endings ("\n"). Splunk will not correctly read lookup files saved using Macintosh ("\r") or Windows line endings ("\r\n"). 1. From the Splunk platform system menu, select Settings > Lookups. 2. Next to Lookup table files, click Add New. 3. Verify that SplunkEnterpriseSecuritySuite is selected as the Destination app. 4. Upload a lookup file. 5. Type a Destination filename to be displayed in the lookup list. 6. Click Save. By default, lookups are saved as Private. To share the information with other users, searches, and upgrade events, change the file permissions. 1. Click Permissions next to the newly imported CSV file. 2. Select the appropriate level and type of permissions for this file. Set access to This app only to limit access to Splunk Enterprise Security or select All apps to allow all apps in this Splunk instance to access the lookup. 3. Click Save.
Verify lookup files Confirm that you added a lookup file successfully by using the inputlookup search command to display the list. 207
inputlookup append=T application_protocol_lookup
Search-driven lookups A search-driven lookup lets you create a lookup based on the results of a search that runs at regular scheduled intervals. Create a search-driven lookup if you want to know when something new happens in your environment, or need to consistently update a lookup based on changing information from a data model or another lookup. The search can run only against data stored in data models or in an existing lookup. Lookups created as search-driven lookups are excluded from bundle replication and are not sent to the indexers. The search-driven lookup collects and stores information from data models or other lookups. The data stored in the lookup represents a historical summary of selected fields gathered from events. You can view changes on a dashboard or use a correlation search to compare data from the search-driven lookup with new events, and alert if there is a match. For example, to find out when a new user logs in to a web server. 1. Search for user data in the Authentication data model and filter by the web server host name with the where command. 2. Verify the search results match the known hosts and users in your environment. 3. Create a guided search-driven lookup to collect and store information on a recurring schedule about users logging in to the web servers. 4. Create a correlation search that alerts you when a user logs in to one of the web servers that has not in the past, based on the historical information in the search-driven lookup. Create a search-driven lookup Create a search-driven lookup. 1. From the Splunk Enterprise Security menu bar, select Configure > Content Management. 2. Click Create New Content and select Search-Driven Lookup. 3. (Optional) Select an App. The default app is SplunkEnterpriseSecuritySuite. You can create the lookup in a specific app, such as SA-NetworkProtection, or a custom app. You cannot change the app after you save the search-driven lookup. 4. (Optional) Type a description for the search. 5. Type a label for the lookup. This is the name of the search-driven lookup that appears on Content Management. 208
6. Type a name for the lookup. After you save the lookup, the name cannot be changed. 7. Type a cron schedule to define how often you want the search to run. 8. Select real-time or continuous scheduling for the search. Real-time scheduling prioritizes search performance, while continuous scheduling prioritizes data integrity. 9. Type a Search Name to define the name of the saved search. After you save the lookup, the name cannot be changed. 10. Select a mode of Guided to create a search without having to write the search syntax yourself, or select Manual to write your own search. See the example for help building a search with the guided search editor. 11. If you create a search in manual mode, type a search. 12. Click Save to save the search. In this example search-driven lookup included with Splunk Enterprise Security, you want to track attacks identified by your intrusion detection system (IDS). You can then be notified of new attacks with a correlation search, or determine whether an attack is new to your environment or not. The Intrusion Center dashboard uses this search-driven lookup for the New Attacks - Last 30 Days panel. See Intrusion Center dashboard. 1. From the Splunk Enterprise Security menu bar, select Configure > Content Management. 2. Click Create New Content and select Search-Driven Lookup. 3. (Optional) Select an App of SA-NetworkProtection. You cannot change the app after you save the search-driven lookup. 4. Type a description of "Maintains a list of attacks identified by an IDS and the first and last time that the attacks were seen." 5. Type a label of IDS Attack Tracker Example for the lookup. This is the name of the search-driven lookup that appears on Content Management. 6. Type a unique and descriptive name for the lookup of ids_attack_tracker_example. After you save the lookup, the name cannot be changed. 7. Type a cron schedule to define how often you want the search to run. If your IDS collects data often, type a cron schedule of 25 * * * * to run the search at 25 minutes every hour every day. 8. Select a Continuous Schedule because the lookup must track all data points. 9. Type a Search Name of Network - IDS Attack Tracker - Example Lookup Gen. 10. Select guided mode to use the guided search editor to create the search. 11. Click Open guided search editor to start creating the search.
209
12. Select a data source of Data Model because the IDS Attack data is stored in a data model. 13. Select a data model of Intrusion_Detection and a data model dataset of IDS_Attacks. 14. Select Yes for the summaries only field to run the search against only the data in the accelerated data model. 15. Select a time range that uses Relative time that begins with an earliest time of 70 minutes ago, starting at the beginning of the minute, and ends now. Click Apply to save the time range. 16. Click Next. 17. (Optional) Type a where clause to filter the data from the data model to only the data from a specific IDS vendor and click Next. 18. Add aggregate values to track specific statistics about the data and store that information in the lookup. At least one aggregate is required. 1. To track the first time that an IDS attack was seen in your environment, add a new aggregate with a function of min and a field of _time and save it as firstTime. 2. Track the last time an attack was seen by adding another aggregate with a max function and a field of _time and saving it as lastTime. This creates two columns in the lookup, firstTime and lastTime. 19. Add split-by clauses to track more data points in the lookup. All split-by clauses appear as columns in the lookup. 1. Add a split-by clause of IDS_Attacks.ids_type and rename it as ids_type to monitor the IDS type in the lookup. 2. Add a split-by clause to rename IDS_Attacks.signature as signature. 3. Add a split-by clause to rename IDS_Attacks.vendor_product as vendor_product. 20. Click Next. 21. Select a retention period that defines the age of the data to be stored in the lookup. For example, you want to keep 5 years of IDS attack evidence stored in this lookup. Select a time field of lastTime to base the retention on the last time an attack was identified by the IDS. Type an earliest time of -5y and indicate the format of the time value that you entered: %s. You can find guidance on the time format in the Splunk platform documentation. ♦ For Splunk Enterprise, see Date and time format variables in the Splunk Enterprise Search Reference manual. ♦ For Splunk Cloud, see Date and time format variables in the Splunk Cloud Search Reference manual.
210
22. Click Next. 23. Review the search created by the wizard and click Done to finish using the guided search editor. 24. Click Save to save the search. Modify a search-driven lookup 1. From the Splunk Enterprise Security menu bar, select Configure > Content Management. 2. Select a Type of Search-Driven Lookup. 3. Click the lookup that you want to edit. 4. Make changes and click Save. Existing search-driven lookups ES system searches that end with Lookup Gen can be edited as guided search-driven lookups. As of Splunk Enterprise Security version 4.2.0, the following search-driven lookups were migrated to guided search-driven lookups and have a default retention period of 5 years. • Access - Access App Tracker - Lookup Gen • Endpoint - Listening Ports Tracker - Lookup Gen • Endpoint - Malware Tracker - Lookup Gen • Endpoint - System Version Tracker - Lookup Gen • Network - IDS Attack Tracker - Lookup Gen • Network - IDS Category Tracker - Lookup Gen • Network - Port And Protocol Tracker - Lookup Gen Any locally-defined search-driven lookups can also be converted to guided mode, making editing easier, or kept in manual mode, requiring that you edit the search directly. Open a locally-defined search-driven lookup in Content Management to convert it to guided mode. Enable or disable the search populating a search-driven lookup You can enable or disable the search of a search-driven lookup to prevent the search from updating the lookup. If you disable the search that populates a search-driven lookup, the search stops updating the lookup and the data in the lookup will stop being updated. Correlation searches or dashboards that rely on the data inside the lookup will be out-of-date. 1. Select Configure > Content Management.
211
2. Filter on a type of search-driven lookup and open the search-driven lookup that you want to enable or disable. 3. Find the Search name of the search-driven lookup. 4. From the Splunk platform menu bar, select Settings > Searches, reports, alerts. 5. Find the search and enable or disable it.
Available internal lookups The following lookups are available by default in Splunk Enterprise Security. Select Configure > Data Enrichment > Lists and Lookups to view the internal lookups. Application Protocols The Application Protocols list is a list of port/protocol combinations and their approval status in the organization. This list is used by the Port & Protocol Tracker dashboard. See Port & Protocol Tracker dashboard. The following fields are available in this file. Field
Description
dest_port The destination port number (must be 0-65535) transport
The protocol of the network traffic (icmp, tcp, udp).
app
application name
status
The approval status of the port (approved, pending, unapproved). By default, the port is considered approved.
Assets The Assets lookup contains information about the assets in your environment. This list of assets is matched to incoming events. See Add asset and identity data to Splunk Enterprise Security. Categories The category list can contain any set of categories you choose for organizing an asset or an identity. A category is logical classification or grouping used for assets and identities. Common choices for assets include compliance and security standards such as PCI, or functional categories such as server and web_farm. Common choices for identities include titles and roles. For more examples, see Asset and identity lookup header and field reference. 212
Note: To enrich events with category information in asset and identity correlation, you must maintain the category field in the asset and identity lists rather than the Categories list. See Asset and identity lookup header and field reference. There are two ways to maintain the Categories list. Run a saved search to maintain a list of categories
Splunk Enterprise Security includes a saved search that takes categories defined in the asset and identity lists and adds them to the Asset/Identity Categories list. The search is not scheduled by default. 1. From the Splunk platform menu bar, select Settings > Searches, reports, alerts. 2. Enable the Identity - Make Categories - Lookup Gen saved search. Manually maintain a list of categories
Maintain the Categories list manually by adding categories to the lookup directly. By default, you must maintain the list manually. 1. Select Configure > Data Enrichment > Lists and Lookups. 2. Click the Asset/Identity Categories list. 3. Add new categories to the list. 4. Save your changes. Expected Views The Expected Views list specifies Splunk Enterprise Security views that are monitored on a regular basis. The View Audit dashboard uses this lookup. See View Audit for more about the dashboard. The following table shows the fields in this file. Field
Description
app
The application that contains the view (SplunkEnterpriseSecuritySuite)
is_expected
Either "true" or "false". If not specified, Splunk Enterprise Security assumes by default that activity is not expected.
view The name of the view. Available in the URL. To find the name of a view:
213
1. Navigate to the view in Enterprise Security 2. Look at the last segment of the URL to find the view name For example, the view in the URL below is named incident_review:
Identities The Identities lookup contains a list of identities that are matched to incoming events. See Add asset and identity data to Splunk Enterprise Security in this manual. Interesting Ports Interesting Ports contains a list of TCP and UDP ports determined to be required, prohibited, or insecure in your deployment. Administrators can set a policy defining the allowed and disallowed ports and modify the lookup to match that policy. To get alerts when those ports are seen in your environment, enable the correlation search that triggers an alert for those ports, such as Prohibited Port Activity Detected. If you open the lookup file interesting_ports.csv in the lookup editor, the header of the file describes the fields in the file and also described in this table. Field
Description
Example
app
The application or service name
Win32Time
dest
The destination host for the network service. Accepts a wildcard.
DARTH*, 10.10.1.100, my_host, etc. Using just a wildcard * will match all hosts.
dest_pci_domain
An optional PCI Domain. Accepts a wildcard.
trust, untrust, etc.
dest_port
The destination port number. Accepts a wildcard.
443, 3389, 5900, etc.
transport
The transport protocol. Accepts a wildcard.
tcp or udp
is_required
Is the service required to be running? Alert if not present.
true or false
is_prohibited
true or false 214
Is the service/traffic/port prohibited from running? Alert if present. is_secure
Is the service traffic encrypted?
A brief description of the service and use-case Interesting Processes note
true or false Unencrypted telnet services are insecure.
Interesting Processes contains a list of processes. This list is used to determine whether a process is required, prohibited, and/or secure. Use the List and Lookup editor to modify or add to this list. The Interesting Processes lookup is named interesting_processes.csv. The following table shows the fields in this file. Column
Description
app
application name
dest
destination of process
dest_pci_domain PCI domain, if available is_required
true or false
is_prohibited
true or false
is_secure
true or false
note Any additional information about this process Interesting Services Interesting Services contains a list of services in your deployment. This list is used to determine whether a service is required, prohibited, and/or secure. Use the List and Lookup editor to modify or add to this list. The Interesting Services is named interesting_services.csv. The following table shows the fields in this file. Column
Description
app
application name
dest
destination of process
dest_pci_domain PCI domain, if available 215
is_required
true or false
is_prohibited
true or false
is_secure
true or false
note Any additional information about this process Primary Functions Primary Functions contains a list of primary processes and services, and their function in your deployment. Use this list to designate which services are primary and the port and transport to use. The Primary Functions lookup file is named primary_functions.csv. The following table shows the fields in this file. Column
Description
process
name of process
service
name of service
dest_pci_domain PCI domain, if available transport
tcp or udp
port
port number
is_primary
true or false
function
function of this process (for example, Proxy, Authentication, Database, Domain Name Service (DNS), Web, Mail)
Prohibited Traffic Prohibited Traffic lists processes that will generate an alert if they are detected. This list is used by the System Center dashboard and is useful for detecting software that is prohibited by the security policy, such as IRC or data destruction tools, or for software that is known to be malicious, such as malware that was recently implicated in an outbreak. The Prohibited Traffic file is named prohibited_traffic.csv. The following table shows the fields in this file. Field app
Description The name of the process (such as echo, chargen, etc.)
is_prohibited Either "true" or "false" 216
note A text description of why the process is rejected Urgency Levels Urgency Levels contains the combinations of priority and severity that dictate the urgency of notable events. See Notable Event Urgency assignment in this manual.
Dashboard Troubleshooting Each dashboard in Enterprise Security references data from various data models. Without the relevant data, the dashboards will remain empty. If you expect data to appear, or if the data appearing is older than you expect, follow these troubleshooting steps. 1. Perform a search against the data model. Click Open in Search in the lower left corner of a dashboard view to perform a direct search against the data model. The New Search dashboard also exposes the search commands and objects used to populate a particular view. 2. If the search yields no results, determine if any data required for a dashboard is available in the data model. 1. See the Dashboard requirements matrix in this manual to determine the data model datasets used by a dashboard. 2. Use the data model and data model dataset to search for events in the data model. Action Search Expected Result | datamodel data_model_name root_object_name search | table _time, sourcetype, Returns a list of Verify the data root_object_name.* sourcetypes and is normalized to For example, the data model the Common objects and fields Information | datamodel populated by that Model Network_Traffic sourcetype. All_Traffic search | dedup sourcetype | table _time, sourcetype, All_Traffic.*
3. If no data is available, confirm the data model is being accelerated. 1. In Enterprise Security, browse to Audit > Data Model Audit.
217
2. Review the Acceleration Details panel for information about the data model acceleration status, such as when the latest data model acceleration occurred, or whether it is 100% complete. See Configure data models for Splunk Enterprise Security in the Installation and Upgrade Manual. 4. If the data model acceleration status is as expected, validate that additional required data sources are available. For example, the User Activity dashboard uses additional data sources. Dashboard Data type Data source Name
User Activity
Access Anomalies
Lookups
The Cloud Domains, Corporate Email Domains, and Corporate Web Domains lookup files.
Identities
The Identity fields: bunit, email, watchlist, work_city, work_country, work_lat, and work_long. For more details, see Identity correlation in this manual.
Correlation Searches
* High Volume Email Activity with Non-corporate Domains * Watchlisted Event Observed * Web Uploads to Non-corporate Sites by Users
Correlation Searches
* Impossible Travel Events Detected For Users
Dashboard requirements matrix The Enterprise Security dashboards rely on events that conform to the Common Information Model (CIM), and are populated from data model accelerations unless otherwise noted.
Dashboard panel to data model A-E Dashboard Name Access
Panel Title
Data Model
Data Model Dataset
Geographically Authentication Authentication.app, .src, .user_bunit
218
Anomalies
Improbable Accesses Concurrent Application Accesses
Access Center
Authentication.app, .src, .user
Access Over Time By Action
Authentication.action
Access Over Time By App
Authentication.app
Top Access By Authentication Authentication.src Source Top Access By Unique User
Authentication.user,.src Authentication.action, .app, src, .dest, .user, src_user
Access Search First Time Access - Last 7 days
Access Tracker
Inactive Account Usage None. Calls access_tracker lookup - Last 90 days Completely Inactive Accounts - Last 90 days Account Usage For Expired Authentication Authentication.dest Identities - Last 7 days Account Management Over Time
All_Changes.Account_Management, .action
Account Lockouts
All_Changes.Account_Management, .result
Account Change Management Account Analysis Management By Source User 219
All_Changes.Account_Management, .src_user
Top Account Management Events
All_Changes.Account_Management, .action
Assets By Priority Assets By Asset Center Business Unit
Assets And Identities
All_Assets.priority, .bunit, .category, .owner
Assets By Category Asset Information Asset Investigator
Asset Investigator
Dashboard Name
Panel Title
Data Protection
Data Integrity Control By Index
Based on swim lane selection
Data Model
Data Model Dataset
Incident Management
Sensitive None. Calls a REST search on indexes checking for Data data integrity controls.
Default Account Activity
Default Account Usage Over Time By App
Authentication.Default_Authentication, .action, .app Authentication
Default Accounts In Use
Authentication.user_category, .dest, .user
Default Local None. Calls useraccounts_tracker lookup Accounts Top Reply Codes By Unique Sources
DNS.message_type, DNS.reply_code
220
DNS Activity
Network Resolution DNS
Top DNS Query Sources
DNS.message_type, DNS.src
Top DNS Queries
DNS.message_type, DNS.query
Queries Per Domain
DNS.message_type, DNS.query
Recent DNS Queries
DNS.message_type DNS.message_type, DNS.reply_code, DNS.dest, DNS.src ,DNS.query_type, DNS.query, DNS.answer
DNS Search Dashboard Name
Email Activity
Panel Title
Data Model
Top Email Sources
All_Email.src
Large Emails
All_Email.size, src, .src_user, .dest
Rarely Seen Senders
Email
Rarely Seen Receivers
All_Email.protocol, .src, .src_user, .recipient All_Email.protocol, .src, .recipient All_Email.protocol, .recipient, .src, .src_user, .dest
Email Search
Endpoint Changes
Data Model Dataset
Endpoint Changes By Action
All_Changes.Endpoint_Changes, .action
Change Endpoint Changes By Type Analysis
All_Changes.Endpoint_Changes, .object_category
Endpoint Changes By System
All_Changes.Endpoint_Changes, .object_category, .dest
221
F-M Dashboard Panel Title Name
Data Model
Data Model Dataset
Event Count Over Time By Host None. Calls host_eventcount macro and search. Forwarder Audit
Hosts By Last Report Time Splunkd Process Utilization Splunk Service Start Mode
HTTP Category Analysis HTTP User Agent Analysis Dashboard Name
Category Distribution
Application State
All_Application_State.Processes.cpu_load_percent, .mem_used, .process, All_Application_State.dest All_Application_State.Services.start_mode, .status, .service Web.src, .category
Web
Category Details
Web.src, .dest, .category,
User Agent Distribution
Web.http_user_agent_length, .http_user_agent Web
User Agent Details
Web.http_user_agent_length, .src, .dest, .http_user_agent
Panel Title
Data Model
Data Model Dataset
Identities By Priority Identities By Business Identity Center Unit
Assets and Identities
All_Identities.priority, .bunit, .category
Identities By Category Identity Information Identity Investigator
Identity Investigator
Based on swim lane selection
Incident Review Audit
Review Activity By Reviewer
None. Calls a search over the es_notable_events KVStore 222
Top Reviewers
collection.
Notable Events By Status - Last 48 hours Notable Events By Owner - Last 24 hours Recent Review Activity Events Per Day Over Time
None. Calls a search over the licensing_epd KVStore collection.
Indexing Audit Events Per Day Events Per Index (Last Day)
Intrusion Center
Attacks Over Time By Severity
IDS_Attacks.severity
Top Attacks
IDS_Attacks.dest, .src, .signature
Scanning Activity (Many Attacks)
Intrusion Detection
New Attacks
IDS_Attacks.ids_type IDS_Attacks.severity, .category, .signature, .src, .dest
Intrusion Search
Investigations
Investigations
None. Calls a search over the investigation KVStore collection.
None. Calls a search over the Investigation timelines investigation_event KVStore collection. Investigation attachments
None. Calls a search over the investigation_attachment KVStore collection.
Action history
None. Calls a search over the action_history KVStore collection.
Dashboard Panel Title Name
Data Model
Malware Activity Over Time
Data Model Dataset Malware_Attacks.action
Malware Malware Center
IDS_Attacks.signature
223
By Action Malware Activity Over Time By Signature
Malware_Attacks.signature
Top Infections
Malware_Attacks.signature, .dest
New Malware Last 30 Days
None. Calls malware_tracker lookup.
Clients By Product Version Malware Operations
Clients By Signature Version
None. Calls malware_operations_tracker lookup.
Oldest Infections Repeat Infections
Malware_Attacks.action, .signature, .dest Malware
Malware Search
Modular Action Center
Action Invocations Over Time By Name
Splunk Top Actions Audit Logs By Name Top Actions By Search
Malware_Attacks.action, .file_name, .user, .signature, .dest Modular_Actions.Modular_Action_Invocations, .action_name Modular_Actions.Modular_Action_Invocations, .action_mode, .user, .duration, .search_name, .rid, .sid Modular_Actions.Modular_Action_Invocations, .action_name, .action_mode, .user, .search_name, .rid, .sid
224
N-S Dashboard Name Network Changes
Panel Title
Data Model
Network Changes By Action Change Network Changes Analysis By Device
Data Model Dataset All_Changes.Network_Changes, .action All_Changes.Network_Changes, .dvc
New Domain Activity New Domain Analysis
New Domain Activity By Age
Web
Web.dest
New Domain Activity By TLD Registration Details Dashboard Name
Port & Protocol Tracker
Protocol Center
None
Panel Title
Data Model
Data Model Dataset
Port/Protocol Profiler
All_Traffic.transport, .dest_port
Prohibited Or Insecure Network Traffic Over Time Traffic Last 24 Hours
All_Traffic.src_category, .dest_category, .src, .dest, .transport, .dest_port
Prohibited Traffic Details - Last 24 Hours
All_Traffic.src_category, .dest_category, .src, .dest, .transport, .dest_port
New Port Activity Last 7 Days
None. Calls the application protocols lookup.
Connections By Protocol
Network Traffic
All_Traffic.app
Usage By Protocol
All_Traffic.app, .bytes
Top Connection Sources
All_Traffic.src
Usage For Well Known Ports
All_Traffic.bytes, .dest_port
225
Long Lived Connections
All_Traffic.src, .src_port, .duration, .dest, .dest_port, .transport
Risk Modifiers Over Time Risk Analysis
Dashboard Name
All_Risk.risk_score Risk Analysis
Risk Score By Object
All_Risk.risk_score
Most Active Sources
All_Risk.risk_score, .risk_object
Recent Risk Modifiers
All_Risk.*
Panel Title
Data Model
Data Model Dataset
Notable Events By Urgency Security Posture
Notable Events Over Time
None. Calls a search over the es_notable_events KVStore collection.
Top Notable Events Top Notable Event Sources Session Center
Sessions Over Time Session Details SSL Activity By Common Name
SSL Activity
Network Sessions
All_Sessions.Session_* All_Sessions.*
Certificates All_Certificates.SSL.ssl_subject_common_name
SSL Cloud Sessions
All_Certificates.SSL.ssl_subject_common_name, .src,
Recent SSL Sessions SSL Search
All_Certificates.src, .dest, .ssl_subject_common_name, .ssl_subject_email, .ssl_issuer_common_name, 226
.ssl_issuer_organization, .ssl_start_time, .ssl_end_time, .ssl_validity_window, .ssl_is_valid Suppressed Events Over Time - Last 24 Hours
Calls a macro to search on notable events.
Suppression Suppression History Over Time - Last 30 None Audit Days Suppression Management Activity
Calls a search by eventtype.
Expired Suppressions
Calls a search by eventtype.
Operating Systems System Center
Calls a macro and a search on Summary Gen information.
None. Calls system_version_tracker lookup.
Top-Average All_Performance.CPU.cpu_load_percent, CPU Load By Performance All_Performance.dest System Services By System Count Application State Ports By System Count
All_Application_State.Services All_Application_State.Ports
T-Z Dashboard Name
Panel Title
Data Model
Data Model Dataset
Threat Activity Over Time Threat Activity
Most Active Threat Collections
Intrusion Detection, Network Traffic, and Web. For more details, see Threat Activity Data Sources.
Most Active Threat Sources Threat Activity Details 227
Threat Overview
Threat Artifacts
Endpoint Artifacts Network Artifacts
None. Calls the threat intelligence KVStore collections. For more details, see Configure threat intelligence sources.
Email Artifacts Certificate Artifacts
Threat Intelligence Audit
Threat Intelligence Downloads
None. Calls a search by REST endpoint.
Threat Intelligence Audit Events
None. Calls a search by eventtype.
Time All_Performance.OS.Timesync, Synchronization All_Performance.dest, Failures Performance .dest_should_timesync, Systems Not OS.Timesync.action Time Center Time Synching
Traffic Center
Indexing Time Delay
None. Calls the results of a Summary Gen search.
Time Service Start Mode Anomalies
Application State
All_Application_State.Services.start_mode, .Services.status, .dest_should_timesync, .tag, .dest
Traffic Over Time By Action
All_Traffic.action
Traffic Over Time By Protocol
All_Traffic.transport
Scanning Activity (Many Systems)
Network Traffic All_Traffic.dest, .src
Top Sources
All_Traffic.src All_Traffic.action, .src_port, .src, .dest, .transport, .dest_port
Traffic Search
All_Traffic.transport, .src 228
Traffic Size Analysis
Traffic Size Network Anomalies Over Traffic Time Traffic Size Details
Dashboard Name
Panel Title
All_Traffic.bytes, .dest, .src Data Model
Top Systems Needing Updates Top Updates Needed Update Center
Updates.status, .dest, .signature_id, .vendor_product Updates
Systems Not Updating Greater Than 30 Days Update Service Start Mode Anomalies
Data Model Dataset
Updates.status, .dest, .signature_id, .vendor_product Updates.dest_should_update, .dest, .signature_id, .vendor_product, .status
Application State
All_Application_State.Services.start_mode, .Services.status, .Services.service, .tag
Update Search
Updates
Updates.dest_should_update, .status, .dest, .signature_id, .vendor_product
URL Length Anomalies URL Length Over Time Analysis URL Length Details
Web
Web.http_method, .url
Web.url_length, .src, .dest, .url
Users By Risk Risk Analysis Scores
All_Risk.risk_object
Non-corporate Web Web Uploads
Web.bytes, .user, .http_method, .url
Non-corporate User Activity Email Activity Email
All_Email.size, .recipient, .src_user,
Watchlisted Site Activity
Web
Remote Access
Authentication Authentication.src, .user
Web.src, .url
229
Ticket Activity Dashboard Name
View Audit
Ticket Management
Panel Title View Activity Over Time Expected View Activity
Data Model
Splunk Audit Logs
Most Vulnerable Hosts
Vulnerability Operations
Calls vuln_signature_reference lookup Vulnerabilities.dest Vulnerabilities.severity, Vulnerabilities .signature, .dest
Delinquent Scanning
Vulnerabilities.dest Vulnerabilities.category, .signature, .dest, .severity, .cve,
Vulnerability Search
Web Center
View_Activity.app, .view, .user
Vulnerabilities.signature, .severity, .dest
Scan Activity Over Time Vulnerabilities By Age
View_Activity.app, .view
Vulnerabilities Vulnerabilities.signature, .severity, .dest
Vulnerabilities By Severity New Vulnerabilities
Data Model Dataset
Vulnerabilities.signature, .dest
Top Vulnerabilities Vulnerability Center
All_Ticket_Management.description, .priority, . severity, .src_user
Events Over Time By Method
Web.http_method
Events Over Time By Status Web
Web.status
Top Sources
Web.dest, .src
Top Destinations
Web.dest, .src Web.http_method, .status, .src, .dest, .url
Web Search
Dashboards to Add-on These dashboards are included in Splunk Enterprise Security. Use the Navigation editor to add or rearrange dashboards on the menu bar.
230
To view the entire list of dashboards in Enterprise Security, select Search > Dashboards. To review the list of dashboards in Enterprise Security by add-on, see the Content Profile dashboard. See Content Profile. Dashboard name
Security Domain
Part of Add-on
Access Anomalies
Access
DA-ESS-AccessProtection
Access Center
Access
DA-ESS-AccessProtection
Access Search
Access
DA-ESS-AccessProtection
Access Tracker
Access
DA-ESS-AccessProtection
Account Management
Access
DA-ESS-AccessProtection
Asset Center
Asset
SA-IdentityManagement
Asset Investigator
Asset
SA-IdentityManagement
Content Profile
Audit
SplunkEnterpriseSecuritySuite
Data Model Audit
Audit
Splunk_SA_CIM
Default Account Activity
Access
DA-ESS-AccessProtection
DNS Activity
Network
DA-ESS-NetworkProtection
DNS Search
Network
DA-ESS-NetworkProtection
Email Activity
Network
DA-ESS-NetworkProtection
Email Search
Network
DA-ESS-NetworkProtection
Endpoint Changes
Endpoint DA-ESS-EndpointProtection
Forwarder Audit
Audit
SA-AuditAndDataProtection
HTTP Category Analysis
Network
DA-ESS-NetworkProtection
HTTP User Agent Analysis
Network
DA-ESS-NetworkProtection
Identity Center
Identity
SA-IdentityManagement
Identity_investigator Identity
SA-IdentityManagement
Incident Review
Threat
SA-ThreatIntelligence
Incident Review Audit
Threat
SA-ThreatIntelligence
231
Indexing Audit
Audit
SA-AuditAndDataProtection
Intrusion Center
Network
DA-ESS-NetworkProtection
Intrusion Search
Network
DA-ESS-NetworkProtection
Malware Center
Endpoint DA-ESS-EndpointProtection
Malware Operations Endpoint DA-ESS-EndpointProtection Malware Search
Endpoint DA-ESS-EndpointProtection
Network Changes
Network
DA-ESS-NetworkProtection
New Domain Analysis
Network
DA-ESS-NetworkProtection
Per-Panel Filter Audit
Audit
SA-Utils
Port & Protocol Tracker
Network
DA-ESS-NetworkProtection
Predictive Analytics
Splunk_SA_CIM
Protocol Center
Network
DA-ESS-NetworkProtection
REST Audit
Audit
SA-Utils
Risk Analysis
Threat
SA-ThreatIntelligence
Search Audit
Audit
SA-AuditAndDataProtection
Security Posture
SplunkEnterpriseSecuritySuite
Session Center
Identity
SA-IdentityManagement
SSL Activity
Network
DA-ESS-NetworkProtection
SSL Search
Network
DA-ESS-NetworkProtection
Suppression Audit
Threat
SA-ThreatIntelligence
System Center
Endpoint DA-ESS-EndpointProtection
Threat Activity
Threat
DA-ESS-ThreatIntelligence
Threat Artifacts
Threat
DA-ESS-ThreatIntelligence
Threat Intelligence Audit
Audit
DA-ESS-ThreatIntelligence
Time Center
Endpoint DA-ESS-EndpointProtection
Traffic Center
Network
DA-ESS-NetworkProtection
Traffic Search
Network
DA-ESS-NetworkProtection
232
Traffic Size Analysis Network
DA-ESS-NetworkProtection
Update Center
Endpoint DA-ESS-EndpointProtection
Update Search
Endpoint DA-ESS-EndpointProtection
URL Length Analysis
Network
DA-ESS-NetworkProtection
User Activity
Identity
DA-ESS-IdentityManagement
View Audit
Audit
SplunkEnterpriseSecuritySuite
Vulnerability Center
Network
DA-ESS-NetworkProtection
Vulnerability Operations
Network
DA-ESS-NetworkProtection
Vulnerability Search Network
DA-ESS-NetworkProtection
Web Center
Network
DA-ESS-NetworkProtection
Web Search
Network
DA-ESS-NetworkProtection
233
Advanced Guidance Extreme Search Extreme search enhances the Splunk platform search language. As implemented in Splunk Enterprise Security, you can use the Extreme search commands to: • Build dynamic thresholds based on event data. • Provide context awareness by replacing event counts with natural language. For example, in the Enterprise Security Malware Center dashboard, the Key Security Indicator Total Infections displays the total number of systems with malware infections over the last 48 hours.
Splunk ES determines the displayed rate of change by comparing the current count of infections against the count of infected systems from the day before. There is no automatic determination of a normal daily range for infected systems in your environment. The threshold is entirely user-configured. Infections have increased by three, but the value has no context to indicate whether it is a notable increase. The same indicator using Extreme search displays the relevant information, but includes a depth of information that was not available with the default Total Infections indicator.
Using Extreme Search, Splunk ES calculates the infection count and rate of new infections using a dynamically-updating model. The key security indicator uses contextual and easy-to-understand language. In this case, you know that the total malware infection count is not higher than it would be any other day, and the rate of change in infections is not alarming.
234
The use of context and concept in Extreme search The core ideas of context and concept are critical to the understanding of Extreme search. These ideas are responsible for the data model used for dynamic thresholds by an Extreme search command. 1. Context: A context defines a relationship to a field or data in numerical terms. The data to be modeled must be represented by numerical values as the result of a search. Example contexts include total network throughput over the last 24 hours or network latency over the last 24 hours. 2. Concept: A term that applies to data, representing a qualitative rather than quantitative description. Example concepts include the terms "extreme," "high," "medium," "low," and "minimal". By combining context and concept, Extreme search adds meaning and value to the data. • The total network throughput over the last 24 hours was Extreme, high, medium, low, or minimal. • The network latency over the last 24 hours was extreme, high, medium, low, or minimal. The concept terms describe network activity in both examples, but have different meanings based on the context they are applied to. If your environment reports that total network throughput is minimal, it is a warning. If the environment reports that network latency is minimal, the network is operating normally. Data models and Extreme search After you choose a context and concept to represent your data, Splunk ES creates a data model. Using the Extreme search commands, the data model maps the context and event statistics by concept. Extreme search commands refer to this combined model as a context. Saved searches update contexts, such as the dynamic threshold context. The saved search searches event data for statistics to update the context. For a list of the saved searches that update contexts, see Containers, contexts, and saved searches in this topic.
235
Configuring Extreme search for Enterprise Security The use of Extreme search commands in Enterprise Security requires no additional configuration. The default installation of ES provides all contexts used by the Extreme search commands and enables the saved searches that maintain them. • For a list of the contexts and saved searches implemented in Enterprise Security, see Containers, contexts, and saved searches in this topic. • For a list of the key security indicators that use Extreme search, see Extreme Search Key Security Indicators in this topic. • For a list of the correlation searches that use Extreme search, see Extreme Search Correlation searches in this topic. All correlation searches are disabled by default. Extreme Search Correlation searches All correlation searches in Enterprise Security are disabled by default. See Enable the correlation searches in this manual. Guided Search Creation is not available for correlation searches that use Extreme search commands. These correlation searches use Extreme search. Search Name
Context
Brute Force Access Behavior Detected
failures_by_src_count_1h
Brute Force Access Behavior Detected Over One Day
failures_by_src_count_1d
Abnormally High Number of Endpoint Changes By User
change_count_by_user_by_change_type_1d
Host Sending Excessive Email
recipients_by_src_1h
Substantial Increase in Events
count_by_signature_1h
Substantial Increase in Port Activity
count_by_dest_port_1d
Unusual Volume of Network Activity
count_30m
Abnormally High Number of HTTP Method Events By Src
count_by_http_method_by_src_1d
236
Extreme Search Key Security Indicators You can easily identify the key indicators that use Extreme search by their use of semantic language instead of numerical values. The key security indicators on each dashboard are enabled by default. Search Name
Contexts
Access - Total Access Attempts
authentication: count_1d, percentile
Malware - Total Infection Count
malware: count_1d, percentile
Risk - Median Risk Score
median_object_risk_by_object_type_1d, percentile
Risk - Median Risk Score By System
median_object_risk_by_object_type_1d, percentile
Risk - Median Risk Score By User
median_object_risk_by_object_type_1d, percentile
Risk - Median Risk Score By Other
median_object_risk_by_object_type_1d, percentile
Risk - Aggregated Risk
total_risk_by_object_type_1d, percentile
Risk - Aggregated System Risk
total_risk_by_object_type_1d, percentile
Risk - Aggregated User Risk
total_risk_by_object_type_1d, percentile
Risk - Aggregated Other Risk total_risk_by_object_type_1d, percentile Containers, contexts, and saved searches Enterprise Security stores contexts in objects called containers. A container is both an object in the file system and a logical configuration used to classify contexts. In Enterprise Security, the containers are files with the .context extension. A container can contain multiple contexts. You can view the saved searches that generate contexts on the Content Management page in Enterprise Security. Note: Enterprise Security enables the dynamic context saved searches by default. Container name
Context name
237
App location
se
authentication
failures_by_src_count_1h
Ac Au Fa So Co
failures_by_src_count_1d
Ac Au Fa So Da Ge
SA-AccessProtection
Ac Au Vo Da Ge
count_1d
change_analysis change_count_by_user_by_change_type_1d SA-EndpointProtection
destinations_by_src_1h email
SA-EndpointProtection
Ch Ch By Ch Pe Co
En Em De Co Co
recipients_by_src_1h
En Em So Co
malware
count_1d
SA-NetworkProtection
En Ma Co Co
ids_attacks
count_by_signature_1h
SA-NetworkProtection
238
Ne Ev By
Pe Co
count_by_dest_port_1d
Ne Ac De Po Ge
src_count_30m
Ne Tr Co 30 Ge
count_30m
Ne Tr Pe Co
count_by_http_method_by_src_1d
W Ev By HT Pe Co
network_traffic
web
SA-NetworkProtection
SA-NetworkProtection
Ri Ob Pe Co
median_object_risk_by_object_type_1d risk
default
SA-ThreatIntelligence total_risk_by_object_type_1d
Ri Ri Ob Pe Co
percentile
ES Pe Co
SA-Utils
height default
No
trendchange
Splunk_SA_ExtremeSearch No
compatibility
No
239
Examples For an extended walkthrough on a correlation search that implements Extreme search commands, see "An Extreme search example" in this manual.
An Extreme search example You can convert existing correlation searches to use Extreme search commands. In this example, Splunk Enterprise Security includes the converted search. You do not need to make any configuration changes or modifications to use searches converted to use Extreme search.
The Brute Force Access Behavior Detected search The correlation search "Brute Force Access Behavior Detected" searches for an excessive number of failed login attempts, followed by a successful attempt. The base search finds relevant events, counts the events by type "failure" and looks for a trailing "success" event for every host authentication over the last hour. If the identified events meet a threshold, the search triggers an alert action to create a notable event or other alert types. "Brute Force Access Behavior Detected" correlation search without Extreme search commands: | `datamodel("Authentication","Authentication")` | stats values(Authentication.tag) as tag,count(eval('Authentication.action'=="failure")) as failure,count(eval('Authentication.action'=="success")) as success by Authentication.src | `drop_dm_object_name("Authentication")` | search failure>6 success>0 | `settags("access")`
Without Extreme search commands, the search defines a static threshold for the "success" events with the string | search failure>6. The Enterprise Security administrator has to select a threshold value, or accept the default value. If the administrator sets the threshold too low, the search creates a storm of notable events. If they set the threshold too high, the search could miss notable events, creating a potential blind spot to a security threat. A search that implements Extreme search removes the static value and uses, in this example, the authentication data ingested by Splunk Enterprise to determine a notable level of authentication failures in your environment.
240
1. Examine the data To use Extreme search, you must build a data model for the commands to rely on. To build the data model, you must understand what the data represents and what question you are trying to answer. In this example, the "Brute Force Access Behavior Detected" correlation search, you know that the count of authentication failures will not go below zero, and may range much higher. A scale of magnitude represents the authentication values being searched.
2. Choose a context You can choose one of three types of contexts, each requiring three data points. • Median or mean average: requires a median or a mean value, a standard deviation, and a total count of events. • Domain: requires a minimum, a maximum, and a total count of events. In this example, the count of authentication events does not include a negative value and is progressive, so a domain is the best fit for the authentication data.
3. Choose a concept A concept represents a qualitative description of the data. Splunk Enterprise Security includes predefined concepts for interpreting change, direction, and magnitude as a qualitative value. Concepts are differentiated by the terms used. • Change uses the terms: "minimally, slightly, moderately, greatly, extremely." • Direction uses the terms: "decreasing, unchanged, increasing" • Magnitude uses the terms: "minimal, low, medium, high, extreme" In this case, the magnitude concept best represents the behavior of authentication failures.
4. Create the context As described in Extreme Search in this manual, a context has both a name and a container, with the container residing in an app. The "Brute Force Access Behavior Detected" search runs against authentication events, so the context container is called "authentication." The "authentication" container is located in 241
the "SA-AccessProtection" app along with the authentication searches and other objects. ES includes a pre-initialized authentication context. This context will not represent your environment unless a saved search updates it with events. Splunk Enterprise Security contains this context so that updates will carry a greater weight than the values used during the creation of the context. The domain for this authentication context is defined with a min=0, max=10, and count=0. For the "Brute Force Access Behavior Detected" search, the context name is chosen to facilitate quick identification: failures_by_src_count_1h. Create the initial content: | xsCreateUDContext app="SA-AccessProtection" name=failures_by_src_count_1h container=authentication scope=app terms=`xs_default_magnitude_concepts` min=0 max=10 count=0 type=domain
Display the context, once created: | xsdisplaycontext failures_by_src_count_1h in authentication
Before implementing Extreme search, the static threshold for authentication failures was six. Using the context failures_by_src_count_1h, a count of six is modeled at the end of the term "medium". The model will change after the updated "Brute Force Access Behavior Detected" search searches the authentication data and the saved search that updates the failures_by_src_count_1h runs. List the terms used in a context: | xslistconcepts failures_by_src_count_1h in authentication
5. Apply the context in the search You can use the search command xsWhere to evaluate a data value against a context. This correlation search uses xsWhere to compare the count of authentication failures against the context failures_by_src_count_1h to determine if the count represents a value above "medium."
242
In this example, a concept of medium represents the range of values that change after the context is updated with data. A saved search updates the context. If the count of events identified by the saved search is greater than medium, the correlation search using extreme search will trigger an alert action and create a notable event. "Brute Force Access Behavior Detected" with Extreme search capabilities | `datamodel("Authentication","Authentication")` | stats values(Authentication.tag) as tag,count(eval('Authentication.action'=="failure")) as failure,count(eval('Authentication.action'=="success")) as success by Authentication.src | `drop_dm_object_name("Authentication")` | search success>0 | xswhere failure from failures_by_src_count_1h in authentication is above medium | `settags("access")`
6. Update the context A search threshold can be dynamic because it uses a saved search to update a context. The saved searches included with ES that generate context information for Extreme search end with "Context Gen" to provide easy identification. The domain context used by the "Brute Force Access Behavior" correlation search requires values for minimum, maximum, and count. Those values are drawn from the authentication data model. The "Access - Authentication Failures By Source - Context Gen" saved search that generates the failures_by_src_count_1h context for the "Brute Force Access Behavior" correlation search. For the failures_by_src_count_1h context, the results of the context generating search change the maximum value to a multiple of the median to prevent outliers from skewing the underlying context and potentially introducing oversights. "Access - Authentication Failures By Source - Context Gen" saved search | tstats `summariesonly` count as failures from datamodel=Authentication where Authentication.action="failure" by Authentication.src,_time span=1h | stats median(failures) as median, min(failures) as min, count as count | eval max = median*2 | xsUpdateDDContext app="SA-AccessProtection" name=failures_by_src_count_1h container=authentication scope=app This search updates the failures_by_src_count_1h context with xsUpdateDDContext. In this case, the data from the search is added to the
context, creating a historical trend that informs the context.
243
Both the correlation search and the saved search "Access - Authentication Failures By Source - Context Gen" are scheduled to run hourly by default.
7. Use hedges to modify the results Hedges are semantic terms that modify the range represented by a concept. Use a hedge to limit, shrink, or modify the shape of the curve that a concept term uses to model the data. The hedges "above" and "below" are useful for alerting searches as they redefine the range of values that will match. The "Brute Force Access Behavior Detected" correlation search using extreme search applies a hedge so an alert action triggers only when the count of failures is "above medium." Examples of a concept with various hedges applied: Hedge example
Image
| xsDisplayConcept medium from failures_by_src_count_1h in authentication | xsDisplayConcept very medium from failures_by_src_count_1h in authentication | xsDisplayConcept above medium from failures_by_src_count_1h in authentication | xsDisplayConcept below medium from failures_by_src_count_1h in authentication | xsDisplayConcept around medium from failures_by_src_count_1h in authentication The synonyms.csv lookup file in the Splunk_SA_ExtremeSearch app contains the Extreme search hedges.
Summary The "Brute Force Access Behavior Detected" correlation search using Extreme search is included with Splunk Enterprise Securirty. The context generation search runs and updates the context on a recurring interval. The correlation search references the context, and the concept within the context sets the threshold. The concept is hedged to "above medium" so that the correlation search will only create a notable event when the count of failed authentications followed by a successful authentication is "high" or "extreme."
244
In plain language, Extreme search transformed the "Brute Force Access Behavior Detected" correlation search from "find all authentication attempts where X count of failed authentications are followed by a successful authentication" to "find all authentication attempts where a high or extreme number of failed authentications are followed by a successful authentication."
Extreme search commands Search command
Description
xsWhere
Used to match a concept within a specified context, and determine compatibility.
xsFindBestConcept
Used when evaluating a search count and comparing the count to a context. The closest match returns the term used by the concept. The key security indicators use this command.
Used to update a data-defined context. A scheduled report that calls "xsUpdateDDContext" builds a context xsUpdateDDContext that represents a historical view. |xsUpdateDDContext in app= name= container= scope=app
xsListContexts
Used to list all contexts in a container | xsListContexts
xsListConcepts
Used to list all concepts in a context | xsListConcepts
xsDisplayContext
Used to display the range of values in a context, including the terms used in the concept: | xsDisplayContext
in from in
IN
Used to display the range of values used for a concept: | xsDisplayConcept
xsDisplayConcept from in | xsDisplayConcept from in
245
