Splunk Training Manual
Lab #1 Start Splunk Enterprise and Launch Splunk Web 1. Launch Splunk Web
Please enter the following url: http://www.uxcreate.com/guacamole
2. Login Screen Please enter your username and password to be obtained by the instructor
1
2
Lab 2 Navigating Splunk Web AboutSplunkHome
Splunk Home is your interactive portal to the data and apps accessible from this Splunk instance. The main parts of Home include the Splunk Enterprise navigation bar, the Apps menu, the Explore Splunk Enterprise panel, and a custom default dashboard (not shown here).
Apps
The Apps panel lists the apps that are installed on your Splunk instance that you have permission to view. Select the app from the list to open it. For an out-of-the-box Splunk Enterprise installation, you see one App in the workspace: Search & Reporting. When you have more than one app, you can drag and drop the apps within the workspace to rearrange them.
You can do two actions on this panel: •
•
Click the gear icon to view and manage the apps that are installed in your Splunk instance. Click the plus icon to browse for more apps to install.
Explore Splunk Enterprise
The options in the Explore Splunk Enterprise panel help you to get started using Splunk Enterprise. Click on the icons to open the Add Data view, browse for new apps,
3
open the Splunk Enterprise Documentation, or open Splunk Answers. About the Splunk bar
Use the Splunk bar to navigate your Splunk instance. It appears on every page in Splunk Enterprise. You can use it to switch between apps, manage and edit your Splunk configuration, view system-level messages, and monitor the progress of search jobs.
The following screenshot shows the Splunk bar in Splunk Home.
The Splunk bar in another view, such as theSearch & Reporti ng app's Search view, also includes an App menu next to the Splunk logo.
4
Return to Splunk Home
Click the Splunk logo on the navigation bar to return to Splunk Home from any other view in Splunk Web. Settings menu
The Settings menu lists the configuration pages for Knowledge objects, Distributed environment settings, System and licensing, Data, and Authentication settings. If you do not see some of these options, you do not have the permissions to view or edit them.
User menu
The User menu here is called "Administrator" because that is the default user name for a new installation. You can change this display name by selectingEdit account and changing the Full name . You can also edit the time zone settings, select a default app for this account, and change the account's password. The User menu is also where you Logout of this Splunk installation.
5
Messages menu
All system-level error messages are listed here. When there is a new message to review, a notification displays as a count next to theMessages menu. Click the X to remove the message.
Activity menu
The Activity menu lists shortcuts to the Jobs, Triggered alerts, and System Activity views.
•
•
•
Click Jobs to open the search jobs manager window, where you can view and manage currently running searches. Click Triggered Alerts to view scheduled alerts that are triggered. This tutorial does not discuss saving and scheduling alerts. See "About alerts" in the Alertin g Manual. Click System Activity to see Dashboards about user activity and status of the system.
Help
Click Help to see links to Video Tutorials, Splunk Answers, the Splunk Support
6
Portal, and online Documentation.
Find
Use Find to search for objects within your Splunk Enterprise instance. Find performs non-case sensitive matches on the ID, labels, and descriptions in saved objects. For example, if you type in "error", it returns the saved objects that contain the term "error".
These saved objects includeReports , Dashboards , Alerts, and Data mo dels . The results appear in the list separated by the categories where they exist.
You can also run a search forerror in the Search & Reporti ng app by clicking Open error in s earch .
7
8
Lab 4 Searching the tutorial data
Start searching
In this section, you start searching that tutorial data. This topic discusses searches that retrieve events from the index. Before you can start this section, you need to first download and add the tutorial data. What to search
1.
Click Search in the App navigation bar.
2.
In the Search landing page, Look at the What to search panel.
3.
Click Data Summ ary .
Review the tutorial data, which represents a fictitious online game store, called Buttercup Games. The data summary tells you where the data comes from and what type of data it is. There are five hosts, eight sources, and three source
9
types. The three source types are Apache web access logs (access_combined_wcookie), Linux secure formatted logs (secure), and the vendor sales log (vendor_sales). Most of this tutorial covers searching the Apache web access logs and correlating it with the vendor sales logs. Search assistant
You have forhave an online storeonthat how manydata errors occurred thesells site.a variety of games. Try to find out 1. Open Splunk Search, and type buttercupgames into the search bar.
As you type, the Search Assistant opens. There are two parts to search assistant: the matching search history and search help. Search assistant suggests completions for your search based on terms it matches in your event data. These search completions are listed under Matching terms or Matching searches . It does not list terms or phrases that do not exist in your event data. Search assistant also displays the number of matches for the search term. This number gives you an idea of how many search results Splunk will return. Here, buttercupgames appears in 36,819 events.
10
Here, search assistant also provides Steps to help you learn How to Search . Step 1 explains searches to retrieve events with examples for searching with terms, quoted phrases, boolean operators, wildcards, and field values. Step 2 introduces how to use search commands. Search assistant has more uses after you start learning the search language. When you type in search commands, search assistant displays the command syntax and usage. If you do not want search assistant to open automatically, click Auto Open to toggle it off. You can click the down arrow below the search bar to open it back up again. Retrieve events from the index
1. Type in keywords to find errors or failures and use Boolean operators: AND,
OR, NOT. but t er cupgames ( er r or OR f ai l * OR sever e)
Boolean operators need to be capitalized. The AND directive is implied between terms, so you do not need to write it. You can use parentheses to group terms. When evaluating expressions, terms inside parentheses. OR boolean clauses are evaluatedprecedence before ANDisorgiven NOTto clauses. The asterisk wildcard is used to match terms that start with "fail". These terms can include: failure, failed, and so on.
11
This search retrieves 427 matching events. The search command
sear ch Each time to you type keywords andaphrases, you implicitly use the command retrieve events from Splunk index. The search command lets you use keywords, quoted phrases, field values, boolean expressions, and comparison expressions to specify which events you want to retrieve.
You can also explicitly invoke the search command later in the pipeline to filter search results. Read "Use the search command" in the Search manual.
12
Use fields to search
You can not take full advantage of the more advanced search features in Splunk Enterprise without understanding what fields are and how to use them. About fields
When you look at the Data Summary in the search view, you see tabs for the Hosts, Sources, and Source Types that described the type of data you added to your Splunk index.
These are also default fields (host , sour ce , sour cet ype) that Splunk Enterprise extracts from the data during indexing. They help to specify exactly which events you want to retrieve from the index.
What are fields? Fields exist in machine data in many forms. Often, a field is a value (with a fixed, delimited position on the line) or a name and value pair, where there is a single value to each field name. A field can be multivalued, that is, it can appear more than once in an event and has a different value for each appearance. Some examples of fields are cl i enti p for IP addresses accessing your Web server, _t i me for the timestamp of an event, and host for domain name of a server. One of the more common examples of multivalue fields is email address
13
fields. While the From field will contain only a single email address, the To and Cc fields have one or more email addresses associated with them. In Splunk Enterprise, fields are searchable name and value pairings that distinguish one event from another because not all events will have the same fields and field values. Fields let you write more tailored searches to retrieve the specific events that you want.
Extracted fields Splunk extracts fields from event data at index-time and at search-time.
Default and other indexed fields are extracted for each event that is processed when that data is indexed. Default fields include host , sour ce , and sour cet ype.. Splunk Enterprise extracts different sets of fields, when you run a search.. You can also use the field extractor to create custom fields dynamically on your local Splunk instance. The field extractor lets you define any pattern for recognizing one or more fields in your events. Find and sele ct fi elds
1.
Go to the Search dashboard and type the following into the search bar:
sour cet ype=" access_ * " Search for fields use the syntax: f i el dname=" f i el dval ue" . Field names are case sensitive, but field values are not. You can use wildcards in field values. Quotes are required when the field values include spaces. This search indicates that you want to retrieve only events from your web access logs and nothing else.
14
This search uses the wildcard access_* to match any Apache web access sour cet ype, which can be access_common, access_combined, or access_combined_wcookie.
2.
In the Events tab, scroll through the list of events.
If you are familiar with the access_combined format of Apache logs, you recognize some of the information in each event, such as: • • • •
IP addresses for the users accessing the website. URIs and URLs for the pages requested and referring pages. HTTP status codes for each page request. GET or POST page request methods.
15
These are events for the Buttercup Games online store, so you might recognize other information and keywords, such as Arcade, Simulation, productId, categoryId, purchase, addtocart, and so on. To the left of the events list is the Fields sidebar. As Splunk Enterprise retrieves the events that match your search, the Fields sidebar updates with Selected fields and Inte resting fields . These are the fields that Splunk Enterprise extracted from your data.
16
Selected Fields are the fields that appear in your search results. The default
fields host, source, and sourcetype are selected. These fields appear in all the events. You can hide and show the fields sidebar by clicking Hide Fie lds and Show Fields , respectively.
3.
Click All Fi elds .
The Select Fields dialog box opens, where you can select the fields to show in the events list.
17
You see more default fields, which includes fields based on each event's t i mest amp (everything beginning with dat e_* ), punctuation (punct ), and location (i ndex).
Other field names apply to the web access logs. For example, there are cl i ent i p, met hod, and status . These are not default fields. They are extracted at search time. Other extracted fields are related to the Buttercup Games online store. For example, there are act i on, cat egor yI d, and pr oduct I d. 4.
Select act i on, cat egor yI d, and pr oduct I d and close the Select Fields window.
The three fields appear under Selected Fields in the sidebar. The selected fields appear under the events in your search results if they exist in that particular event. Every event might not have the same fields.
18
The fields sidebar displays the number of values that exist for each field. These are the values that Splunk Enterprise indentifies from the results of your search.
5.
Under Selected Fields , click the act i on field.
This opens the field summary for the action field.
In this set of search results, Splunk Enterprise found five values for acti on, and that the act i on field appears in 49.9% of your search results. Close this window and look at the other two fields you selected, cat egor yI d (what types of products the shop sells) and pr oduct I d (specific catalog number for products). 6.
7.
Scroll through the events list.
If you click on the arrow next to an event, it opens up the list of all fields in that event.
19
Use this panel to view all the fields in a particular event and select or deselect individual fields for an individual event. Run more targeted searches
The following are search examples using fields. Example1: Search for successful purchases from the Buttercup Games store.
20
sour cet ype=access _* s t at us=200 act i on=pur chase
This search uses the HTTP status field, status , to specify successful requests and the act i on field to search only for purchase events. You can search for failed purchases in a similar manner using st at us! =200, which looks for all events where the HTTP status code is not equal to 200. sour cet ype=access _* st at us! =200 act i on=pur chase
Example 2: Search for general errors.
( er r or OR f ai l * OR sever e) OR ( st at us=404 OR st at us=500 OR st at us=503) This doesn't specify a source type. The search retrieves events in both the secure and web access logs. Example 3: Search for how many simulation games were bought yesterday.
Select the Preset time range, Yesterday, from the time range picker and run: sour cet ype=access _*
st at us=200
act i on=pur chase
cat egor yI d=si mul at i on The count of events returned are the number of simulation games purchased. To find the number purchases for each type of product sold at the shop, run this search for each unique categoryId. For the number of purchases made each day of the previous week, run the search again for each time range.
21
Use the search language
The searches you have run to this point have retrieved events from your Splunk index. You were limited to asking questions that could only be answered by the number of events returned. For example, in the last topic, you ran this search to see how many simulation games were purchased: sour cet ype=access_ * st at us=200 act i on=pur chase cat egor yI d=si mul at i on
To find this number for the days of the previous week, you have to run it against the data for each day of that week. To see which products are more popular than the other, you have to run the search for each of the eight cat egor yI d values and compare the results.
Learn with search assistant
This section explains in more detail one of the ways you can use the search assistant to learn about the Splunk search processing language and construct searches. 1.
Return to the search dashboard and restrict your search to Yesterday:
sour cet ype=access _* s t at us=200 act i on=pur chase As you type in the search bar, search assistant opens with syntax and usage information for the search command (on the right side). If search assistant doesn't open, click the down arrow under the left side of the search bar.
22
You've seen before that search assistant displays type ahead for keywords that you type into the search bar. It also explains briefly how to search. 2.
Type a pipe character, " | ", into the search bar.
The pipe indicates to Splunk that you're about to use a command, and that you want to use the results of the search to the left of the pipe as the input to this command. You can pass the results of one command into another command in a series, or pipeline, of search commands.
23
You want Splunk to give you the most popular items bought at the online store. 3.
Under common next commands , click to p .
Splunk Enterprise appends the t op command to your search string.
According to search assistant's description and usage examples, the to p command "displays the most common values of a field."
24
4.
Type the cat egoryI d field into the search bar to complete your search.
sour cet ype=access_ * st at us=200 act i on=pur chase | t op cat egor yI d 5.
Run the search.
The count of events under the search bar indicates the number of events retrieved that match the search for sour cet ype=access _* s t at us=200 act i on=pur chase. The results of the top command appear in the Statistics tab.
25
View reports in the Statistics tab
The results of a search are reports. The t op command is a transforming command and returns a tabulated report for the most common values of cat egor yI d. You can view the results of transforming searches in the Statistics tab.
In this search for successful (stdifferent purchases (actreport at us=200) i on=lists pur chase), Splunk Enterprise found seven category IDs. This the category ID values in order from highest to lowest. The top command also returns two new fields: count is the number of times each value of the field occurs, and per cent is how large that count is compared to the total count.
ViewandformatreportsintheVisualizationtab You can also view the results of transforming searches in the Visualizations tab where you can format the chart type. For example, a search using the t op command can be illustrated with a pie chart. 1.
Click the Visualization tab.
26
By default, the Visualizations tab opens with a Column Chart.
2.
Click Column to open the visualization type selector.
Column , Bar , and Pie charts are recommended for this data set.
3.
Select Pie .
Now, your report looks like this:
27
You can turn on drill down to delve deeper into the details of the information presented to you in the tables and charts that result from your search.
4.
Mouse over each slice of the pie to see the count and percentage values for each categoryId.
28
5.
Click on a slice, such as "Strategy".
Splunk Enterprise appends cat egoryI d=st r at egy to your srcinal search for successful purchases and runs this new search.
29
30
31
Lab 5 Subsearchs
Use a subsearch
This topic walks you through examples of correlating events with subsearches. A subsearch is a search with a search pipeline as an argument. Subsearches are contained in square brackets and evaluated first. The result of the subsearch is then used as an argument to the primary, or outer, search.
Example 1: Without a subsearch
Let's try to find the single most frequent shopper on the Buttercup Games online store and what this customer has purchased. To do this, search for the customer who accessed the online shop the most. 1.
Use the t op command:
sour cet ype=access_* st at us=200 act i on=pur chase | t op l i mi t =1 cl i ent i p
Limit the t op command to return only one result for the cl i enti p.
32
This search returns one cl i ent i p value, which we'll use to identify our VIP customer. 2.
Use the stats command to count this VIP customer's purchases:
sour cet ype=access_ * st at us=200 act i on=pur chase cl i ent i p=87. 194. 216. 51 | st at s co unt , dc(pro ductI d) by c l i ent i p
This search used the count() function which only returns the total count of purchases for the customer. The dc() function is used to count how many different products he buys. The drawback to this approach is that you have to run two searches each time you want to build this table. The top purchaser is not likely to be the same person at any given time range. Example2:Withasubsearch
1.
Type or copy/paste the following into the search bar.
sour cet ype=access_ * s t at us=200 act i on=pur chase [ sear ch sour cet ype=access_* st at us=200 act i on=pur chase | t op l i mi t =1 cl i ent i p | t abl e cl i ent i p] | st at s co unt , dc(p r oductI d) , val ues( pr oductI d) by cl i enti p
Here, the subsearch is the segment that is enclosed in square brackets, [ ]. This search, sear ch sour cet ype=access_ * st at us=200 act i on=pur chase | t op
33
l i mi t =1 cl i ent i p | t abl e cl i ent i p is the same as Example 1 Step 1, except for the last piped command, | t abl e cl i ent i p
Because the t op command returns count and per cent fields as well, the t abl e command is used to keep only the cl i enti p value.
34
These results should match the previous result, if you run it on the same time range. But, if you change the time range, you might see different results because the top purchasing customer will be different.
2.
Rename the columns to make the information more understandable.
sour cet ype=access_ * st at us=200 act i on=pur chase [ sear ch sour cet ype=access_ * st at us=200 act i on=pur chase | t op l i mi t =1 cl i ent i p | t abl e cl i ent i p] | sta t s count AS "To t al Pur chased", dc( pr oduct I d) AS "T ot al Pr oduct s" , val ues( pr oduct I d) AS "Pr oduct s I D" by cl i ent i p | r ename cl i ent i p AS "VI P Cust omer "
35
Lab 3 Add the sample data into Splunk Enterprise 1.
Log into Splunk.
If you are not in Splunk Home, click the Splunk logo on the Splunk bar to go to Splunk Home. Get file tutorialdata.zip Under Explore Splunk Enterprise, click Add data.
The Add Data view opens. The Add Data displays three options for adding data, lists of common data types, and add-ons you can use to extend Splunk Enterprise's capabilities to add data.
Under "How do you want to add data?", click Upload.
36
2.
Under Select Source, click Select File to browse for the tutorial data or Drop the data file into the outlined box.
Because the tutorial data file is an archived data file, the next step in the Add Data workflow changes from Set Sourcetype to Input Settings. 3.
Click Next to continue to Input Settings.
Under Input Settings, you can override the default settings for Host, Source type, and Index.
37
4.
Modify the host settings to assign host names using a portion of the path name:
4.1.
Select Segment in path from the menu.
4.2.
Type in 1 for the segment number.
5.
Click Review to review your input settings.
38
6. 7.
8.
Click Submit .( PLEASE DO NOT SUBMIT BECAUSE INSTRUCTOR ALREADY SUBMITTED FOR ENTIRE CLASS)
To confirm that the data added successfully, click Start Searching. This opens the Search view and runs a search for the tutorial data source.
39
40
Lab 6 Field Lookup Use field lookups This topic takes you through using field lookups to add new fields to your events. Field lookups let you reference fields in an external CSV file that match fields in your event data. Using this match, you can enrich your event data by adding more meaningful information and searchable fields to each event. Uncompress the following file: Prices.csv.zip •
Find the Lookups manager
1.
In the Splunk bar, on the upper right, click Settings .
2.
Under Knowledge , click Lookups.
41
This opens the Lookups editor where you can create new lookups or edit existing ones.
Uploa d the looku p table file
In the Lookups manager under "Actions" for Lookup t able files , click Add new . 1.
42
This takes you to the Add new' lookup table files view where you upload CSV files to use in your definitions for field lookups.
To save your lookup table file in the Search app, leave the Destination app as search. 2.
3.
Under Upload a lookup f ile , browse for the CSV file (prices.csv) to upload.
4.
Under Destination filename , name the file prices.csv.
This is the name you use to refer to the file in a lookup definition. 5.
Click Save.
This uploads your lookup file to the Search app and returns to the lookup table files list.
43
Note: If Splunk does not recognize or cannot upload the file, check that it was uncompressed before you attempt to upload it again.
Share the lo okup table file globally
If the lookup file is not shared, you can not select it when you define the lookup. 1.
Go to the Lookup table files list.
44
2.
Under Sharing for the prices.csv lookup table's Path , click Permissions .
This opens the Permission dialog box for the prices.csv lookup file.
3.
Under Object sh ould appear in , select All ap ps .
4.
Click Save.
Now, the lookup table should be shared with Global permissions. Addthefieldlookupdefinition
1.
Return to the Lookups manager.
2.
Under Act io ns for Lookup definitions , click Add New.
This takes you to the Add n ew lookups definitions view where you define your field lookup.
45
3.
Leave the Destination app as search .
4.
Name your lookup prices_lookup .
5.
Under Type , select File-based .
46
File-based lookups add fields from a static table, usually a CSV file. 6.
Under Lookup file , select prices.csv (the name of your lookup table).
7.
Leave Configure time-ba sed lookup and Ad vanced op ti ons unselected.
8.
Click Save.
This defines prices_lookup as a file-based lookup.
Sharethelookupdefinitionwithallapps
1.
Return to the Lookup definitions
list.
2.
Under Sharing for prices_lookup , click Permissions .
The Permission dialog box for the prices.lookup opens.
3.
Under Object sh ould appear in , select All ap ps .
47
4.
Click Save.
Now, prices_lookup should be shared with Global permissions. Make the lookup automatic
In the Lookups manager, under Actions for Auto matic lo ok up s , click Add New. 1.
48
This takes you to the Add New automatic lookups view where you configure the lookup to run automatically.
2.
Leave the Destination app as search .
3.
Name your automatic lookup price_lookup .
4.
Under Lookup table , select prices_lookup .
49
5.
Under Appl y to and named , select sourcetype and type in
access_combined_wcookie.
6.
Under Lookup input fields
type in pr oduct I d in both text areas under
Lookup input fields .
Splunk Enterprise matches the field in the lookup table (which is the one specified on the left) with the field on the right (which is the field in your events).
50
In this case the field names match. Under Lookup output fields , type in the name of the fields that you want to add to your event data based on the input field matching and rename the fields. 7.
7.1 In the first text area, type pr oduct _name, which contains the descriptive name for each pr oduct I d.
In the second text area, after the equal sign, type pr oduct Name. This renames the field to pr oduct Name. 7.2.
7.3.
Click Add another fi eld to add more fields after the first one.
Add the field pri ce, which contains the price for each pr oduct I d. Do not rename this field. 7.4.
8.
Leave Overwri te field values unchecked.
9.
Click Save.
This returns you to the list of automatic lookups and you should see your configured lookup.
51
Show the ne w fields in your s earch r esults
1.
Return to Search .
2.
Run the search for web access activity.
sour cet ype=access_ * 3.
Scroll through the list of Intere stin g Fields in the fields sidebar, and find the
pr i ce field.
4.
Click price to open its field summary dialog box.
52
5.
Next to Selected , click Yes .
6.
Close the dialog box.
The price field appears under Selected Fields in the fields sidebar.
6. Repeat Steps 3 to 5 for the productName field.
Searchwiththenewlookupfields
1. Copy and paste or type in the previous subsearch example to see what the VIP customer bought. This time, replace the pr oduct I d field with pr oduct Name. sour cet ype=access_ * s t at us=200 act i on=pur chase [ sear ch sour cet ype=access_* st at us=200 act i on=pur chase | t op l i mi t =1 cl i ent i p |
53
t abl e cl i ent i p] | st at s co unt AS "To t al Pur chased", dc( pr oduct I d) AS " Tot al Prod uct s" , val ues( pr oduct Name) AS "Pr oduct Names" by cl i ent i p | r ename cl i ent i p AS "VI P Cust omer "
The result is the same as in the previous subsearch example, except that the VIP customer's purchases are more meaningful with the added descriptive product names.
54
Lab 7 Saving and sharing Reports About saving and sharing reports
This lab takes you through saving searches and more search examples. Saveasareport
1.
Select the time range Yesterday and run the following search
2.
sour cet ype=access_ * s t at us=200 act i on=pur chase [ sear ch sour cet ype=access_* st at us=200 act i on=pur chase | t op l i mi t =1 cl i ent i p | t abl e cl i ent i p] | st at s co unt AS "To t al Pur chased", dc( pr oduct I d) AS " Tot al Prod uct s" , val ues( pr oduct Name) AS " Prod uct Names" by cl i ent i p | r ename cl i ent i p AS "VI P Cust omer "
3.
Click Save as above the search bar and selectReport .
This opens the Save as Report dialog.
4.
Enter a Title VIP Customer .
55
5.
(Optional) Enter a DescriptionButtercup Game s mos t frequent shopper .
6.
Because the report is a table, for Visualization, clickNone .
7.
To include a Time Range Picker, clickYes .
8.
Click Save .
The Your report has been create d dialog box opens.
56
There are other options in this window. • • •
9.
Continue Editing lets you refine the search and report format. Ad d t o d ashb oard lets you add the report to a new or existing dashboard. View lets you view the report.
Click View.
View and edit saved reports
You can view and edit the saved report from its report view. 1.
In the report view for "VIP Customer", clickEdit .
57
You can open the report in the search view and edit the saved search's description, permissions, schedule, and acceleration. You can also clone, embed, and delete the report from this menu. 2.
Click More Info .
You can view and edit different properties of the report, including its schedule, acceleration, permissions, and embedding. 3.
Look at the time range picker, located to the top left.
You saved this report with a time range picker. The time range picker lets you change the time period to run this search. For example, you can use this time range picker to run this search for the VIP CustomerWeek to date , Last 60 minutes , Last 24 hours just by selecting the Preset time range or defining a custom time range.
58
Find and share saved reports
You can access your saved reports using the app navigation bar. 1. Click Reports to open the Reports listing page.
When you save a new report, itsPermission s are set to Private . This means that only you can view and edit the report. You can allow other apps to view, or
59
edit, or view and edit the reports by changing its Permissions.
Under Acti on s for the VIP Custom er report, click Edit and select Edit Permissions . 1.
This opens the Edit Pe rmiss ions dialog box.
In the Edit Pe rmiss ions dialog box, set Display For to App and check the box under Read for Everyone . 2.
60
This action gives everyone who has access to this app the permission to view it. 3.
Click Save .
Back at the Reports listing page, you see that the Sharing for VIP Customer now reads App .
61
About report acceleration
If your search has a large number of events and is slow to finish, you might be able to accelerate the resulting report so it finishes faster when you run it again. This option is available when the report produced by your search qualifies for acceleration. The "VIP Customer" report does not qualify for acceleration, because it is based on a transforming search.
62
Lab 8 More Searches and Reports
Example1:Comparecountsofuseractions
In this example, calculate the number of views, purchases, and adds to cart for each type of product. This report requires the pr oduct Name field from the fields lookup example . If you did not add the lookup, refer to that example and follow the procedure. 1.
Run this search:
sour cet ype=access_ * s t at us=200 | char t count AS vi ews count ( eval ( act i on="addt ocar t ") ) AS addt ocar t count ( eval ( act i on=" pur chase" ) ) AS pur chases by pr oduct Name | r ename pr oduct Name AS " Pr oduct Name", vi ews AS " Vi ews" , addt ocar t AS " Adds t o Cart " , pur chases AS "Pu r chases"
63
This search uses the chart command to count the number of events that are act i on=pur chase and act i on=addt ocar t .
2.
Use the Visualization view options to format the results as a column chart.
Example 2: Overlay Actions and Conversion Rates on one chart 1.
Run this search:
sour cet ype=access_ * st at us=200 | st at s count AS vi ews count ( eval ( act i on="addt ocar t ") ) AS addt ocar t count ( eval ( act i on=" pur chase" ) ) AS pur chases by pr oduct Name | eval vi ewsToPur chase=( pur chases/ vi ews) *100 | eval car t ToPur chase=( pur chases/ addt ocar t ) *100 | t abl e pr oduct Name vi ews addt ocar t pur chases vi ewsToPur chase car t ToPur chase | r ename pr oduct Name AS "Pr oduct Name" vi ews AS " Vi ews" , addt ocar t as " Adds To Car t " , pur chases AS " Pur chases"
Instead of the chart command, this search uses the stats command to count the user actions. Then, it uses the eval command to define two new fields which
64
calculate conversion rates for "Product Views to Purchases" and "Adds to cart to Purchases".
Steps 2 to 6 reformat the visualization to overlay the Conversion series onto the Actions series.
2.
Click Visualization .
This is the same chart as in Example 1, with two additional series, "viewsToPurchase" and "cartToPurchase".
65
3.
Click Format and X-Axis .
4.
4.1
Rotate the label -45 degrees and do not truncate the label.
4.2
Click Appl y .
66
5.
Click Format and Y-Axis .
5.1
For Title , choose Custom and type in "Actions".
5.2
Set the Max Value to 2500 and the Interval to 500.
5.3
Click Appl y .
6.
Click Format and Chart Overlay .
67
6.1
Type in or select the fields, "viewsToPurchase" and "cartToPurchase".
6.2
For View as Axis , click On .
6.3
For Title , choose Custom and type in Conversion Rates .
6.4
For Scale , choose Linear .
6.5
Set the Max Value to 100 and the Interval to 20.
6.6
Click Appl y .
68
7.
Click Save As and select Report .
In the Save Repor t As dialog box, enter a Title, "Comparison of Actions and Conversion Rates by Product". 7.1
7.2
(Optional) Enter a Description, "The number of times a product is viewed, added to cart, and purchased and the rates of purchases from these actions." 8.
Click Save.
69
Example 3: Products purchased over time
For this report, chart the number of purchases that were completed for each item. This report requires the pr oduct Name field from the fields lookup example. If you didn't add the lookup, refer to that example and follow the procedure. 1.
Search for:
sour cet ype=access_ * | t i mechar t count ( eval ( act i on=" pur chase" ) ) by pr oduct Name usenul l =" f " useot her =" f "
Use the count() function to count the number of events that have the field act i on=pur chase . Use the usenul l and useot her arguments to make sure the chart counts events that have a value for pr oduct Name.
This produces the following statistics table.
70
Click the Visualization tab and Format the X-Axis , Y-Axis , and Legend to produce the following line chart. 2.
3.
Click Save As and select Report .
71
In the Save Repor t As dialog box, enter a Title, "Product Purchases over Time". 3.1
3.2
4.
(Optional) Enter a Description, "The number of purchases for each product."
Click Save and View the report.
72
Example4:Purchasingtrends
This example uses sparklines to trend the count of purchases made over time. For stats and chart searches, you can add sparklines to their results tables. Sparklines are inline charts that appear within the search results table and are designed to display time-based trends associated with the primary key of each row. See "Add sparklines to your search results" in the
Search Manual.
This example requires the pr oduct Name field from the fields lookup example . If you did not add the lookup, refer to that example and follow the procedure.
1. Run the following search: sour cet ype=access_ * st at us=200 act i on=pur chase| char t spar kl i ne( count ) AS " Pur chases Tr end" count AS Tot al by cat egor yI d | r ename cat egor yI d AS " Cat egor y"
This search uses the char t command to count the number of purchases, act i on=" pur chase" , made for each product, pr oduct Name. The difference is that the count of purchases is now an argument of the spar kl i ne( ) function.
3.
Click Save As and select Report .
73
4.
In the Save Repor t As dialog box, enter a Title, "Purchasing trends".
5.
(Optional) Enter a Description, "Count of purchases with trending."
6.
Click Save and View the report.
74
Lab 9 Dashboards About dashboards Dashboards are views that are made up of panels that can contain modules such as search boxes, fields, charts, tables, and lists. Dashboard panels are usually hooked up to saved searches. After you create a visualization or report, you can add it to a new or existing dashboard using the Save as repor t dialog box. You can also use the Dashboard Editor to create dashboards and edit existing dashboards. Using the Dashboard editor is useful when you have a set of saved reports that you want to quickly add to a dashboard. Change dashboard permissions
You can specify access to a dashboard from the Dashboard Editor. However, your user role (and capabilities defined for that role) might limit the type of access you can define. If your Splunk user role is admin (with the default set of capabilities), then you can create dashboards that are private, visible in a specific app, or visible in all apps. You can also provide access to other Splunk user roles, such as user, admin, and other roles with specific capabilities. Change dashboard panel visualizations
After you create a panel with the Dashboard Editor, use the Visualization Editor to change the visualization type in the panel, and to determine how that visualization displays and behaves. The Visualization Editor lets you choose from visualization types that have their data structure requirements matched by the search that has been specified for the panel.
75
Edit the XML configuration of a dashboard
Although you are not required to use XML to build dashboards, you can edit a dashboard's panels by editing the XML configuration for the dashboard. This provides editing access to features not available from the Dashboard Editor. For example, edit the XML configuration to change the name of dashboard or specify a custom number of rows in a table.
Create dashboards and dashboard panels This topic walks you through saving a search as a dashboard panel and adding an input element to the dashboard. Save a search as a dashboard panel
1.
Run the following search:
sour cet ype=access_ * st at us=200 act i on=pur chase | t op cat egor yI d
76
2.
Click the Visualization tab and select the Pie chart type.
3.
In the Search view, click Save as and select Dashb oard Panel .
The Save as Dashboard Panel dialog box opens. 4.
Define a new dashboard and dashboard panel.
77
For Dashboard, click New. Enter the Dashboard Title , "Buttercup Games Purchases", The Dashboard ID updates with "Buttercup_games_purchases". 4.1. 4.2.
(Optional) Add a Dashboard Descrip tion , "Reports on Buttercup Games purchases data". 4.3.
4.4.
Type in the Panel Title , "Top Purchases by Category"
4.5.
Leave the Panel Powered B y as Inline search .
5.
Click Save.
78
6.
Click View Da shb oard .
This creates a dashboard with one report panel. To add more report panels, you can run new searches and save them to this dashboard, or you can add saved reports. View and edit dashboard panels
1.
Click Dashboards in the app navigation bar.
This takes you to the Dashboards listing page.
79
You can Create a new dashbo ard and edit existing dashboards. You see the Buttercup Games Purchases
dashboard that you created.
Under the i column, click the arrow next to Buttercup Games Purchases to see more information about the dashboard: What app context it is in, whether or not it is scheduled, and its permissions. 2.
You can use the quick links that are inline with the information to edit the dashboard's Schedule and Permissions. Add an input to the dashboard
1.
In the Dashboards list, click Buttercup Games Purchases to return to that dashboard.
80
2.
Click Edit and select Edit Panels .
The Edit: Butt ercup Games Purchase s view opens.
In this view, you have edit buttons: Add In pu t, Add Pan el, and Edit Source .
3.
Click Add In pu t and select Time .
81
This adds a time range picker input to the dashboard editor.
4.
Click the Edit Input icon for the time range picker. It looks like a pencil.
This opens a set of input controls. The Time input type should be preselected.
5.
Change the Token value to Buttercup_Games_Time_Range
and click
Apply.
This optional step redefines the name of the input token for the time range picker. Because the default names of input tokens are not very descriptive (field1, field2,
82
field3, and so on), you may want to do this when you give your dashboard multiple inputs. It makes it easier to understand which input you are working with. You can also optionally change the default time range for the picker by changing the value of Default . Right now it defaults to Al l t im e. In the next two steps you connect your dashboard panel to this time range picker.
6.
In the new dashboard panel, click the Inline Sea rch icon and select Edit Search String . The Edit Sea rch dialog opens.
Click Time Range Scope and select Shared Time Picker (Buttercup_Games_Time_Range) . 7.
8.
Click Save.
83
The panel is now hooked up to the shared time range picker input. The inline search that powers the panel now uses the time range selected for the shared time range picker. As you add panels to this dashboard, repeat steps 6 through 8 to hook the new panels up to the shared time range picker input. You can have dashboards that offer a mix of panels that work with the shared time range picker and panels that show data for fixed time ranges. 9.
Click Done to save your changes to the dashboard.
Add more panels to the dashboard In the previous section, you ran searches and saved them as reports. In this topic, you add the saved reports to an existing dashboard. Add saved reports to the dashboard
1.
Return to the Buttercup Games Purchases
84
dashboard.
2.
Click Edit and select Edit Panels .
85
3.
In the Buttercup Games Purchases
dashboard editor, click Add Panel .
4.
The Add Panel sidebar menu slides open.
5.
To add a new panel from a report, click New fro m Report .
This opens the list of saved reports.
86
6.
Select Purchasing Trends .
This opens a preview of the saved Report.
7.
Click Add t o Das hb oard .
87
The new panel is placed in the dashboard editor. You can click anywhere to close the Add Pan el sidebar menu or choose another report to add to the dashboard. Note: If you want the new panels to work with the shared time range picker input, repeat steps 6 through 8 from the "Add an input to the dashboard" procedure to connect them to that input.
Select the report Compa rison of Action s and Conversion Rates by Product and add it to the dashboard. 8.
88
9.
Close the Add Panel sidebar and rearrange the panels on the dashboard.
While in the dashboard editor, you can drag and drop a panel to rearrange it on the dashboard.
10.
Click Done .
Your finished dashboard should look like this:
89
Moredashboardactions
After you complete the dashboard, you can Expor t to PDF and Print the dashboards using the buttons to the upper right. You can also share the dashboard with other users by changing its permissions.
90
Lab 10 Create a new data model
This topic shows you how to create new data models based on the tutorial data. Data models are created within Pivot and you need to have admin or power role to create a data model. Enable roles to create a data model
By default only users with the admin or power role can create data models. For other users, the ability to create a data model is tied to whether their roles have "write" access to an app. Since this is a first time install, you have admin privileges by default and should be able to continue. If you are not able to create or edit a data model, you may need to check your permissions. For more information, read "About data model permissions" in the Knowledge Manager Manual.
Navigate to the Data Models management pag 1.
In the Splunk bar, clickSettings .
91
2.
Under Knowledge , click Data Models .
This takes you to theData Models management page. The Data Models management page is a listing page of data models. If you have existing data models in this Splunk Enterprise instance, this page lists them. Use this page to manage the permissions, acceleration, cloning, and removal of existing data models. You can also use this page to upload a data model or create new data models, using the Upload Data Model and New Data Model buttons on the top right. Create a new data model
1.
In the Data Models management page, clickNew Data Model .
This opens the New Data Model dialog box.
92
2.
Enter the Title , "Buttercup Games"
The Title field accepts any character, as well as spaces. The value you enter here is what appears on the data model listing pages. 3.
(Optional) Enter the ID, "Tutorial"
If you don't change the ID, it automatically reads "Buttercup_Games". The ID must be a unique identifier for the data model. It cannot contain spaces or any characters that aren't alphanumeric, underscores, or hyphens (a-z, A-Z, 0-9, _, or -). Spaces between characters are also not allowed. Once you define the data model ID, you can't change it. 4.
Next to App, select "Search & Reporting" from the menu.
(Optional) Enter the Description , "Enables data analysis and reporting for tutorial data." 5.
6.
Click Create.
This opens the Buttercup GamesEdit Objects page.
Use this page to create objects for the new data model, define their constraints and attributes, arrange the objects in logical hierarchies, and manage them.
93
Lab 11 Define a root object for the data model
In the last lab, you created the data model "Buttercup Games". This lab walks you through adding a root object for Buttercup Games purchases. Edit data model objects
1. From the Data Models list, click Buttercup Games .
This opens the Buttercup Games object editor view.
Use the Edit Objects page to design a new data model or redesign an existing data model. On the Edit Objects page, you can create objects for your data model, define their constraints and attributes, arrange them in logical object hierarchies, and maintain them.
Add a ro ot object
Data models are typically composed of object hierarchies built on root event objects. Each root event object represents a set of data that is defined by a
94
constraint, which is a simple search that filters out events that are not relevant to the object. Let's create an object to track purchase requests on the Buttercup Games website.
1.
To define the data model's first event base object, clickAdd Object .
Your first root object can be either aRoot event or Root search .
2.
Select Root event .
This takes you to the Add Event Object editor.
95
3.
Enter the Object Name:Purchase Requests
The Object Name field can accept any character, as well as spaces. It's what you'll see on the Choose an Object page and other places where data model objects are listed. 4.
Enter the Object ID:Purchase_Requests
This should automatically populate when you type in the Object Name. You can edit it if you want to change it. The Object ID must be a unique identifier for the object. It cannot contain spaces or any characters that aren't alphanumeric, underscores, or hyphens (a-z, A-Z, 0-9, _, or -). Spaces between characters are also not allowed. Once you save the Object ID value, you can't edit it. 5.
Enter the following search Constraints:sour cet ype=access _*
act i on=pur chase
This defines the web access page requests that are purchase events. After you provide Constraints for the event base object you can clickPreview to test whether the constraints you've supplied return the kinds of events you want.
96
6.
Click Save.
97
The list of attributes for the root object include: host, source, sourcetype, and _time. If you want to add child objects to client and server errors, you need to edit the attributes list to include additional attributes.
98
Lab 12 Designing a Pivot report About Pivot
The Splunk Enterprise Pivot tool lets you quickly design reports with tables and data visualizations that present different aspects of a selected Data Model. Pivot lets you generate these reports with a UI interface instead of having to use the search processing language. Pivot views
Pivot is part of theSearch & Reporti ng app.
1.
On the Search & Reportin g app's navigation bar, clickPivot .
Entering Pivot takes you to theSelect a Data Model page, where you should see a list of the data models if any have been created. For example, this list
99
includes the Buttercup Games data model that you created earlier in this tutorial. It also includes two sample data models that track Splunk Enterprise i nt er nal and audi t logs. If you view Pivot in smaller browser windows, the Search & Reporting app's navigation bar is hidden. To use the navigation bar, click the menu icon on the upper right. The navigation bar slides down.
100
2.
Use the arrows under thei column to view information forButtercup Games .
Clicking Edit objects takes you to the object editor for the Buttercup Games data model.
3.
Click Buttercup Games .
This takesbeen you to theSelect This view Games lists all the a Data that have created for this dataObject model.view. The Buttercup dataobjects model consists of the Purchase Requests parent object and the Successful Purchases and Failed Purchases child objects.
4.
Use the arrows under thei column to view the information for the objects.
101
5.
Click Purchase Requests .
102
Selecting an object from theSelect a Data Object view takes you to theNew Pivot editor for that data model.
Components of Pivot
The following illustrates the Pivot editor components.
Visualiza tion types: The left-hand vertical bar contains icons that represent different visualization types. Selecting a different icon controls which Pivot builder and report interfaces display. Visualization types are: Statistics Table (default), Column Chart, Bar Chart, Scatter Chart, Bubble Chart, Area Chart, Line Chart, Pie Chart, Single Value Display, Radial Gauge, Marker Gauge, and Filler Gauge. Document Actions:
The upper horizontal bar displays document-related
actions. These actions include: •
( eport ) or as a Save as...: Save the current report as a new one R
103
•
•
dashboard panel (Dashbo ard Panel). Clear: Reset the interface to its initial state, which will dismiss the saved report (if applicable), change the visualization type to Statistics Table, and populate the report with a single Column Value for the count of the object and a time filter for all time (if _time is an applicable field). Data model o bject: This is the right-most button. It takes its label from the data model object that was selected. For example, in the screenshot it is "Purchase Requests". Use this menu to navigate back to the list of data models (Select another Data Model ), navigate back to the list of data model objects (Select another Object ), or edit the selected data model object (Edit Object ). Additionally, you can rebuild acceleration and inspect the acceleration job.
104
Job Actions: The Pause and Stop buttons control the progress of the Pivot job. Other actions include:Share, Export , Print , and Open in Search . Clicking Open in Sea rch opens the Search view and runs the current search string.
105
Create and save a Pivot
This topic shows you how to use pivot to create and save a simple report. This is a very simple example. More complicated examples are shown in later topics of this tutorial.
Create a new Pivot
When you set out to design a report, you first need to select a data model that represents the broad category of event data that you want to work with. For this tutorial, that data model is the "Buttercup Games".
106
From the app navigation bar, select Pivot to enter theSelect a Data Model page. 1.
2.
In the data models list, clickButtercup Games .
This takes you to theSelect an Object page.
The Buttercup Games data model has a root object to track Purchase Requests from the game website. The Purchases object breaks down into Successful and Failed purchases. 3.
Select "Purchase Requests".
This opens a New Pivot editor for the Purchase Requests object.
107
By default, the Pivot Editor interface displays elements to define a pivot table. There are four basic pivot element categories: Filters, Split Rows, Split Columns, and Column Values. When you first open the Pivot Editor for a specific object, only two elements will be defined: • •
A time range Filter element (set to All time). A Column Values element (set to "Count of ".
This gives you the single value, which is the total count of events returned by the object over all time. In this case, this count is the "Count of Purchase Requests". 4.
Select the Single Value Display element from the visualization bar.
108
4.a
Next to Under Label , type Purchase Requests .
• •
•
By default, the time range filter element is set to All time. Single value visualizations (single value, the three gauge types) use the first column value element to get their single value. Here, the field is "Count of Purchase Requests". Single value visualizations do not use Split Row or Split Column elements.
Save the Pivo t as a repor t
After you define a pivot, you can save it as either a report or a dashboard panel. In this example, you save the single value display as a report. Dashboards and dashboard panels are discussed in a later chapter. 1.
Click Save As... and select Report .
109
The Save as Report dialog box opens.
2.
Enter a Title "Total Purchase Requests" and Description (optional).
110
3.
Select Yes to include the time range picker. (This should be the default.)
4.
Click Save.
After the report saves, a window displays that "Your report has been created". You can continue editing the current Pivot, add the pivot to a dashboard, change additional settings for the saved report, or view the report. 5.
Click View to view the report.
111
View saved reports
A report that is created from Pivot will always be saved under the current app and owner namespace. 1.
Click Reports in the app navigation bar to view the list of all saved reports.
Use the arrow in thei column to view information aboutTotal Purchase Requests report. 2.
3.
Click the report name to view the report.
112
Lab 13 Pivots and Visualizations Create a pivot table
In the previous topic you used pivot to find the total number of purchase requests and saved the single value display as a report. In this topic, you will use the pivot visualization editor to create a pivot table of the Buttercup Games Successful Purchases object. The Successful Purchases object has attributes for the products purchased from the Buttercup Games website. This includes the automatically extracted attributes (categoryId and productId) as well as the lookup attributes (price and product_name). The Buttercup Games online store offers hundreds of products, of a variety of categories, and you want to know more about the items that were purchased over the past week. You can create a pivot report that breaks down the total number of purchase events by product name, and through that quickly see which of your products were the top sellers for that period. Define a new Pivot
From the app navigation bar, select Pivot to enter the "Select a Data Model" page. 1.
2.
Choose the Buttercup Games data model and select theSuccessful
Purchases child object.
113
The New Pivot editor for Successful Purch ases opens.
Add pivot elements
You can add multiple elements from each pivot element category to define your pivot table. It's easy to add, define, and remove pivot elements in the process of determining what information your table should provide. •
To add a pivo t element: Click the + icon. This opens up the element dialog, where you choose an attribute and then define how the element
114
•
•
•
uses that attribute. To inspect o r edit an element: Click the "pencil" icon on the element. This opens the element dialog. To reorder and transfer pivot elements: Drag and drop an element within its pivot element category to reorder it. Drag and drop elements between element categories to transfer them. To remove pivot eleme nts fr om the Pivot Editor: Open its element dialog and click the Remove button, or drag the element up or down until it turns red and drop it.
Under Filters , the time filter is always present when you build a pivot; you cannot remove it. It defines the time range for which the pivot returns results. It operates exactly like the time range menu that is in use throughout Splunk Web. Change the time range filter
Currently your Pivot table shows a single value, the total count of Successful Purchases over Al l t im e. Change the time filter to view the Successful Purchases over a different time range:
1.
Under Filter, click the pencil next toAll t im e to open the time range picker.
115
2.
Under Presets and Relative , click "Last 7 days".
(If this shows no events, you can select "All time" and continue.) Add a Split Row element
Add Pivot elements to see the Count of Successful Purchases for each product by name: Under Split Rows , click + and select pr oduct Name, the lookup field that contains the name of each product, based on thepr oduct I d. 1.
116
This opens a dialog box that lets you format the field.
2.
Rename the field, Produc t Name and Click Add To Tabl e.
117
Add a Column Value element
Add a Column Value to see total earned for each product that was successfully purchased: 1.
Under Column Values , click + and select pri ce.
2.
In the dialog box, format the field:
118
2.a 2.b
Enter the label Total Revenue . Select the Value Sum .
This creates a field calledTotal Revenue , which is the summation of the price for each successful purchase of the product. (You can add thepr i ce values as another Split Row, if you want to see the cost of each individual product in this table.) 3.
Click Add To Tabl e.
119
Save the Pivot table
Save the Pivot table as a report namedPurchase s b y Product .
1.
Click Save as and select Report .
2.
In the Save as Repor t dialog box:
2.a
Enter the Title "Purchases by Product".
2.b
(Optional) Add the Description "Table of Product Purchases".
2.c
Include a Time Range Picker .
3.
Click Save .
4.
In the Your Report Has B een Cre ated dialog box, click View .
120
Create a pivot chart
In the previous topic you used Pivot visualization editor to build a table. In this topic, you will use the same object to create chart visualizations.
Define a new Pivot
From the app navigation bar, select Pivot to enter the "Select a Data Model" page. 1.
Choose the Buttercup Games data model and select theSuccessful Purchases child object. 2.
121
The New Pivot editor for Successful Purch ases opens.
Visualization types are listed in the black sidebar that runs down the left-hand side of the Pivot editor. By default, the statistics table visualization is selected when you enter Pivot. It can be helpful to begin building your pivot as a table and then switch over to the visualization of your choice. When you switch between pivot visualization types, Pivot will find the elements it needs to create the visualization, discard the elements it does not need, and notify you when needed elements need to be defined. This happens when you switch between tables and charts as well as between chart types. Add Pivot elements
In the last topic, we looked at purchases by product ID and name. Now, let's report on the count of successful purchases by category.
122
Add a Split Row for the cat egoryI d field.
1.
Under Split Rows, click + and select cat egoryI d from the list.
2.
Enter the label Category and click Ad d t o t abl e.
This returns the following Pivot table.
123
Change the visualization type
1.
Click the Column Chart icon from the visualization bar.
The New Pivot editor for the Column chart displays.
124
•
•
Column charts use provide their X-axis Column charts use provide their Y-axis Purchases .
the first values. the first values.
split row element in pivot table definitions to In this case, thatSplit Row is Category . column value element in pivot table definitions to Here, that Column Value is Count of Successful
This data can also be visualized as a pie chart. 2.
Click the Pie Chart icon from the visualization bar:
125
The New Pivo t editor for the Column chart displays. Pie charts use the values from the firstSplit Row element (Category) to determine the number and colors of their slices. Pie charts use the first Column Value element (Count of Successful Purchases) to determine the relative sizes of their slices. •
•
126
Mouseover a slice of the pie chart to view the metrics: Category, Count of Successful Purchases, and percentage of the total Count of Successful Purchases.
127
Lab 14 Pivots and Dashboards
Creating a dashboard
About dashboards
Splunk Enterprise makes it easy to interactively build and editdashboards without writing a single line of XML code. •
•
Ad d a p iv ot you h ave ju st c reated to a new or exis ti ng das hb oar d: You can jump right into dashboard creation after creating a pivot visualization you like with the Create Dashboard Panel feature. It guides you through the process of creating a dashboard panel based on the search and adding it to a new or preexisting dashboard. When you finish, you are still in the Pivot view. Use the Da shboard Editor to create da shboards and popu late the m with dashboard panels: You can also use the Dashboard Editor to edit existing dashboards. This method of dashboard creation is useful if you have a set of pivot reports that you want to quickly base a set of
dashboard panels upon. Change dashboard permissions
You can specify access to a dashboard from the Dashboard Editor. However, your user role (and capabilities defined for that role) may limit the type of access you can define. If your user role is admin (with the default set of capabilities), then you can create dashboards that are private, visible in a specific app, or visible in all apps. You can also provide access to other user roles, such asuser, admin, and other roles with specific capabilities.
For additional information on setting up permissions for dashboards and other knowledge objects refer to "Manage knowledge object permissions" in the Admin Manual.
128
Change dashboard panel visualizations
After you create a panel with the Dashboard Editor, use the Visualization Editor to change the visualization type displayed in the panel, and to determine how that visualization displays and behaves. The Visualization Editor only allows you to choose from visualization types that have their data structure requirements.
Save a Pivot as a dashboard panel
You just created a pie chart in the last lab, now let's save it to a dashboard panel. 1.
Click Save as and select Dashboard panel .
This opens the Save as Dashbo ard Panel dialogue.
129
2.
Define a new dashboard to save the panel to: • •
•
3.
Define the dashboard panel: • •
4.
For Dashboard , click New . Enter the Dashboard Title: Buttercup Games. The Dashb oard ID will update with Buttercup_games. (Optional) Add a Dashboard Descript ion: Reports on Buttercup Games online shop data.
Enter the Panel Title: Successful Purchases by Category Leave the Panel Pow ered BY as Inline search.
Click Save.
The dashboard was successfully created. 5.
To continue, click View Da shb oard .
130
View and edit dashboard panels
After you save a dashboard, you can access it by clicking Dashboards in the app navigation bar. 1.
Click Dashboards in the app navigation bar.
This takes you to the Dashboards listing page.
You can Create a new dashbo ard and edit existing dashboards. You see the Buttercup Games dashboard you just created.
Under the i column, click the arrow next toButtercup Games to see more information about the dashboard: What app context it is in, whether or not it is scheduled, and its permissions. 2.
131
There are also quick links to edit the dashboard's Schedule and Permissions inline with the information. To view the dashboard, click the dashboard'sTitle or select the Edit option under Acti on s . Note: If you click to view a dashboard and you cannot view it (or it displays blank), check that you have read access to the data model. To do this, go to the Manage Data Models view and edit the Permissions for theButtercup Games data model to share in the App.
Add an input to the dashboard
1.
In the Dashboards list, click Buttercup Games to return to that dashboard.
2.
Click Edit and select Edit Panels .
The Edit: Butt ercup Games view opens.
132
In this view, you have edit buttons:Add In pu t , Add Pan el , and Edit Source .
3.
Click Add In pu t and select Time .
This adds a shared time range picker input to the dashboard editor.
133
4.
Click the Edit Input icon for the time range picker. It looks like a pencil.
This opens a set of input controls. TheTime input type should be preselected.
5.
Change the Token value to Buttercup_Games_Time_Range
and click
Apply.
This optional step redefines the name of the input token for the time range picker. Because the default names of input tokens are not very descriptive (field1, field2,
134
field3, and so on), you may want to do this when you give your dashboard multiple inputs. It makes it easier to understand which input you are working with. You can also optionally change the default time range for the picker by changing the value of Default . Right now it defaults toAl l t im e. In the next two steps you connect your dashboard panel to this time range picker. In the new dashboard panel, click theInline Pivot icon and select Edit Search String . 6.
The Edit Sea rch dialog opens.
Click Time Range Scope and select Shared Time Picker (Buttercup_Games_Time_Range) . 7.
8.
Click Save .
135
The panel is now hooked up to the shared time range picker input. The inline search that powers the panel now uses the time range selected for the shared time range picker. As you add panels to this dashboard, repeat steps 6 through 8 to hook the new panels up to the shared time range picker input. You can have dashboards that offer a mix of panels that work with the shared time range picker and panels that show data for fixed time ranges. 9.
Click Done to save your changes to the dashboard.
Add another panel using one of the saved reports you created earlier. 1.
In the Buttercup Games dashboard, click Edit and select Edit Panels .
136
2.
In the Edit: Buttercup Games
view, click Ad d Pan el .
The Add Panel sidebar menu slides open.
3.
To add a new panel from a report, clickNew fro m Report .
137
4.
Click Total Purchase Requests .
This slides open a preview panel with information about the saved report.
4.a
Click Add t o Das hb oard .
The new panel is placed in the dashboard editor. You can click anywhere to close the Add Pan el sidebar menu or choose another report to add to the dashboard.
138
Before you close the Add Pan el sidebar menu, add a second report. 5.
Click Purchase s by Produc t .
5.a
6.
Click Add t o Das hb oard .
Close the sidebar menu.
While in the dashboard editor view, drag and drop the panels to rearrange them on the dashboard. 7.
Click Done .
Your dashboard should look like this:
139
140
Lab 15 Alerts Create alerts
Create scheduled alerts
A scheduled alert evaluates the results of a historical search that runs over a specified time range on a regular schedule. The alert fires when it encounters the trigger condition. For example, you can create a scheduled alert to monitor online sales. The search runs daily at midnight and triggers when the sum of the sales of a specific item is below 500 for the previous day. When the alert triggers, it sends an email to the appropriate administrators monitoring sales. 1.
From the Search Page, create the following search. SelectLast 24 H our s for the time range: i ndex=_i nt er nal ( l og_l evel =ERROR OR l og_l evel =WARN* OR l og_l evel =FATAL OR l og_l evel =CRI TI CAL) | st at s count as l og_event s
2.
Select Save As > Alert The Save As Al ert dialog box opens.
3.
Specify Settings :
4.
Title : Server Errors Last 24 hours Al ert Type : Scheduled ♦ Time Range : Run Every Day ♦ Schedule At : 0:00 ♦ Trigger Condition : Number of Results ♦ Trigger if number of results : is Greater than 5 Specify Trigger C ondition s : ♦ ♦
♦
5.
Trigger alert when : Number of Results is Greater than 5
♦ Trigger it : Once Specify Trigger Actions :
141
♦ Ad d Acti on s : List in Triggered Alerts See Set up alert actionsfor information on other actions.
6.
Click Save .
Use cron notation for scheduled alerts
When scheduling an alert, you can use cron notation for customized schedules. When specifying a cron schedule, only five cron parameters are available, not six. The sixth parameter for year, common in other forms of cron notation, is not available. The following cron parameters: * * * * *
correspond to: mi nut e hour day mont h day- of - week
Following are some cron examples:
*/ 5 * * * *
Ever y 5 mi nut es.
*/ 30 * * * *
Ever y 30 mi nut es.
0 */ 12 * * *
Ever y 12 hour s, on t he hour .
*/ 20
* * * 1- 5
0 9 1- 7 * 1
Ever y 20 mi nut es, Monday t hr ough Fr i day. Fi r st Monday of each mont h, at 9am.
When you select Run on Cron Schedule for the time range of a scheduled alert, enter the earliest and latest parameters for a search. What you enter overrides the time range you set when you first ran the search. To avoid overlaps or gaps, the execution schedule should match the search time range. For example, to run a search every 20 minutes the search's time range should also be 20 minutes (-20m).
142
Manage the priority of concurrently scheduledsearches
Depending on your Splunk Enterprise deployment, you might be able to run only one scheduled search at a time. In this case, when you schedule multiple searches to run at approximately the same time, the search scheduler ensures that all scheduled searches run consecutively for the period of time over which they gather data. However, you might have cases where you need certain searches to run ahead of others. This is to ensure that the searches obtain current data or to ensure that there are no gaps in data collection. You can configure the priority of scheduled searches in thesavedsear ches. conf configuration file.
143
Best practices for scheduled alerts
This section discusses some best practices for scheduled alerts. Coordinate an alert's schedule with the search time range
Coordinating the alert's schedule with the search time range prevents situations where event data is evaluated twice by the search. This can happen if the search time range exceeds the search schedule, resulting in overlapping event data sets. In cases where the search time range is shorter than the time range for the scheduled alert, an event might never be evaluated. Schedule alerts with at least 60 seconds of delay
This practice is important in distributed search deployments where event data might not reach the indexer precisely at the moment when it is generated. A delay ensures that you are counting all events, not just the events that were quickest to get indexed. Best practices example
This example shows how to configure an alert that builds 30 minutes of delay into the alert schedule. Both the search time range and the alert schedule span one hour, so there are no event data overlaps or gaps.
144
The alert runs every hour at the half hour. It collects an hour's worth of event data, beginning an hour and a half before the search runs. When the scheduled search kicks off at a designated time, such as 3:30 pm, it collects the event data that was indexed from 2:00 pm to 3:00 pm. 1. 2.
From the Search Page, create a search and selectSave As > Alert . In the Save As Al ert dialog, specify the following to schedule the alert: ♦
Title : Alert Example (30 Minute Delay)
Al ert Type : Scheduled Time Range : Run on Cron Schedule ♦ Earliest : -90m ♦ Latest : -30m ♦ ♦
Earliest and Latest values set the time that the search covers to a period that begins 90 minutes before the search launch time, ending 30 minutes before the search launch time. ♦ Cron Expression : 30 * * * * The alert runs every hour on the half hour
3.
Continue defining actions for the alert.
Set up triggering conditions for a scheduled alert
Trigger conditions apply to two types of conditional alerts: • •
Basic conditional alert Advanced conditional alert
145
Set the triggering conditions when you set values for theTrigger condition in the Save As Alert dialog box, as described in the following subtopics.
field
Basic conditional alert
A basic condi tional ale rt triggers when the number of results of a scheduled search meet, exceed, or are less than a specified numerical value. When you create the alert, you can specify the following conditions:
• • •
Number of of hosts results Number Number of sources
The alert triggers when the number of hosts in the results rises by a count of more than 12. 1. 2.
From the Search Page, create a search and selectSave As > Alert . In the Save As Al ert dialog box, specify the following fields to schedule the alert: ♦ ♦
Title : Alert Example (Basic Conditional) Al ert Type : Scheduled
You can also select Real Time for a basic conditional search. ♦
3.
♦
Time Range and Schedule Select any time range and schedule. Trigger Condition : Number: of Hosts
♦
You can also select Number of Results or Number of Sources Trigger if number of results : Select a comparison operator and trigger value.
Continue defining actions for the alert.
146
Basic conditional alert for rolling-window alerts
The behavior for basic conditional alerts differs slightly for a rolling-window alert. The alert triggers when the set condition occurs within the rolling time window of the search. For example, a rolling-window alert that triggers when a time window of 60 seconds has five or more results. If the real-time search returns one result and then four more results five minutes later, the alert does not trigger. The alert does trigger If the search returns five results within a single 60-second span. Advanced conditional alert
An adva nced co nditional alert uses a secondary, custom conditional search to evaluate the results of a scheduled or real-time search. The alert triggers when the custom search returns any number of results. If the alerting conditions are not met, then the custom conditional search returns zero results. A secondary conditional search can help reduce the incidence of false positive alerts. In the following example, the alert triggers when there are 10 or more log level events that are not INFO. When the alert triggers, it sends an email with the results of the search. The search results detail the count for each log level. 1.
From the Search Page, create the following search. SpecifyLast 7 days for the time period. i ndex=_i nt er nal ( l og_l evel =ERROR OR l og_ l evel =FATAL OR l og_l evel =CRI TI CAL) | st ats co unt by l og_l evel
2. 3.
Select Save As > Alert . In the Save As Al ert dialog box, specify the following fields to schedule the alert: ♦ ♦
Title : Alert Example (Advanced Conditional) Al ert Type : Scheduled
You can also select Real Time for an advanced conditional search. Time Range and Schedule : Select any time range and schedule. ♦ Trigger Condition : Custom ♦
147
♦
4.
Custom condition : search count > 10
Define an action that sends an email that includes the results of the search. When you configure a Send Email action that includes search results, the email contains the results of the srcinal base search. It does not include the results of the custom search.
It might appear that you can get the same results if you specify instead, the following search for the base search of a basic conditional search: l og_l evel =ERROR OR l og_l evel =FATAL OR l og_ l evel =CRI TI CAL) | st at s count by l og_l evel | sear ch count > 10
However, a basic conditional alert based on this search provides different results. The search results contain only log level values that are greater than 10. The results from the advanced conditional search details the count for all log levels, but triggers only when the log levels are greater than 10. Advanced conditional alert for rolling-window alerts
The behavior for advanced conditional alerts differs slightly for a rolling-window alert, which runs in real-time. For a rolling-window alert, the alert triggers when the set condition occurs within the rolling time window of the search.
148
For the previous example, you can design a rolling-window alert with the same base search and get similar results with the custom condition search. Set the rolling window to a 10 minutes time span. When the real-time search returns 10 log level entries within the 10 minute time span, the alert triggers. Create per-result alerts
The per-result alert is the most basic type of alert. It runs in real-time over an "all-time" time span. The alert triggers whenever the search returns a result. You can create a search to retrieve events from an index. You can also use transforming commands to return results based on processing the retrieved events. A per-result alert triggers in both cases, when the search returns an event or when a transforming command returns results. Create a per-result alert
The following procedure shows how to create a per-result alert. 1.
From the Search Page, enter the following search: i ndex=_i nt er nal ( l og_l evel =ERROR OR l og_l evel =WARN* OR l og_l evel =FATAL OR l og_l evel =CRI TI CAL) | st at s count as l og_event s
Select Save As > Alert In the Save As Al ert dialog box, enter a Title for the alert. 4. For Al ert Ty pe , select Real Tim e . A per-result alert is always a real-time alert type. 5. For trigger condition, selectPer-Result . 6. Select the actions you want to enable. For this example, selectList i n Triggered Alert . 2. 3.
7.
See Set up alert actionsfor information on other actions. Click Save .
149
Create rolling-window alerts
Use a rolling-window alert to monitor and evaluate events in real time within a rolling window. The alert triggers only when it meets the trigger condition within a specified time period. The rolling-window alert type is in some ways a hybrid of a per-result alert and a scheduled alert. A rolling-window alert and a per result alert both run in real-time. But unlike the per result alert, a rolling-window alert does not trigger each time the searchtrigger returns a result. within A rolling-window alert fires only when meetsthe alert specified conditions the specified time window. This itmakes similar to a scheduled alert.
1.
From the Search Page, create the following search. SelectLast 24 H our s for the time range: i ndex=_i nt er nal ( l og_l evel =ERROR OR l og_l evel =WARN* OR l og_l evel =FATAL OR l og_l evel =CRI TI CAL) | st at s count as l og_event s
2. 3.
Select Save As > Alert In the Save As Al ert dialog box, specify the following: Title : Alert Example (Rolling-Window) Al ert Type : Real Time ♦ Trigger alert when : Number of Results is Greater than 5 ♦ in : 30 minutes ♦ ♦
150
4.
Continue defining actions for the alert.
Set the width of the rolling window
When you create a rolling-window alert, you specify a time span for a real-time search window. Real-time search windows can be any number of minutes, hours, or days. The alert monitors events as they pass through the window in real-time.
For example, you can create an alert that triggers when a login for a user fails four times in a 10 minute period. When the alert runs, various login failure events pass through this window. The alert triggers only when four login failures for the same user occur within the span of the 10 minute window. This example might appear to fail in the following scenario. A user experiences three login failures in quick succession. After 11 minutes pass, the user has another login failure. The alert does not trigger because the first three failures and the fourth failure are in different time windows. Set up triggering conditions for a rolling-window alert
Trigger conditions apply to two types of rolling-window alerts: • •
Basic conditional alert Advanced conditional alert
You set the triggering conditions when you set values for theTrigger condition field in the Save As Al ert dialog, as described in the following subtopics.
151
Basic conditional alert
A basic condi tional ale rt triggers when the number of results from a search, within a specified time window, meet, exceed, or are less than a specified numerical value. When you create the alert, you can specify the following conditions: Number of results Number of hosts • Number of sources • •
You create a basic conditional alert for a rolling-window similarly to how you create one for a scheduled alert. Advanced conditional alert
An adva nced co nditional alert uses a secondary, custom conditional search to evaluate the results of a scheduled or real-time search. For a rolling-window alert, the alert triggers when the custom search returns any number of results within the specified time window. If the alerting conditions are not met, then the custom conditional search should return zero results. A secondary conditional search can help reduce the incidence of false positive alerts.
Scheduled alert example
A scheduled alert searches for events on a regular basis. It triggers an alert action if results meet the conditions that you specify. Al ert ex ampl e su mm ary
Use case Send an email notification if there are more than five errors in a twenty-four hour period.
152
Alert type Scheduled Search Look for error events in the last twenty-four hours. Schedule Run the search every day at the same time. In this case, the search runs at 10:00 A.M. Trigger conditions Trigger the alert action if the search has more than five results. Alert action Send an email notification with search result details. Set up the alert 1.
From the Search Page, create the following search.i ndex=_i nt er nal " er r or " NOT debug sour ce=*s pl unkd. l og* ear l i est =- 24h l at est =now
2.
Select Save As > Alert .
153
Specify the following values for the fields in theSave As Alert dialog box. ♦ Title : Errors in the last 24 hours ♦ Al ert t yp e: Scheduled ♦ Time Range : Run every day ♦ Schedule: At 10:00 ♦ Trigger condition : Number of Results ♦ Trigger when number of results : is greater than 5. 4. Select the Send Email alert action. 5. Set the following email settings, using tokens in theSubject and Message fields. 3.
To : email recipient Priority : Normal ♦ Subject : Too many errors alert: $name$ ♦ Message : There were $job.resultCount$ errors reported on ♦ ♦
♦
6.
$trigger_date$. Include : Link to Alert and Link to Results
Accept defaults for all other options. Click Save .
Real-time alert example
A real-time alert searches results real time. You the cantrigger configure realtime alerts to trigger everycontinuously time there is for a result orin if results match conditions within a particular time window. Al ert ex amp le s um mar y
Use case Monitor for errors as they occur on a Splunk platform instance. Send an email notification if more than five errors occur within one minute. Alert type Real-time
Search Look continuously for errors on the instance.
154
Trigger conditions Trigger the alert if there are more than five search results in one minute. Alert action Send an email notification. Set up the alert
1.
From the Search Page, create the following search.i ndex=_i nt er nal " er r or " NOT debug sour ce=*s pl unkd. l og*
Select Save As > Alert . Specify the following values for the alert fields. ♦ Title : Errors reported (Real-time) ♦ Al ert t yp e: Real-time ♦ Trigger condition : Number of Results ♦ Trigger if number of results : is greater than 5 in 1 minute. 4. Select the Send email alert action. 5. Specify the following email settings, using tokens in theSubject and Message fields. 2. 3.
To : email recipient Priority : Normal ♦ Subject : Real-time Alert: $name$ ♦ Message : There were $job.resultCount$ errors. ♦ ♦
♦
Include : Link to Alert, Link to Results, Trigger Condition, and
Trigger Time. Accept defaults for all other options. 6. Click Save . Throttle the real-time alert
Throttle an alert to reduce its triggering frequency and limit alert action behavior. For example, you can throttle an alert that generates more email notifications than you need. Throttle the example real-time alert. The following settings change the alert triggering behavior so that email notifications only occur once every ten minutes. From the Alerts page in the Search and Reporting app, select the alert. The alert details page opens. 2. Next to the alert Trigger conditions , select Edit . 1.
155
3. 4.
Select the Throttle option. Specify a 10 minute period. Click Save .
Custom trigger condition example
When you create an alert you can use one of the available result or field count trigger condition options. You can also specify a custom trigger condition. The custom condition works as a secondary search on the initial results set. Al ert ex amp le s um mar y
Use case Use the Triggered Alerts
list to record WARNINGerror instances.
Alert type Real-time Search Look for all errors in real-time. Triggering condition Check the alert search results for errors of typeWARNING. Trigger the alert action if results include anyWARNINGerrors. Alert action List the alert in the Triggere d Alerts page.
Set up the alert 1.
From the Search and Reporting home page, create the following search. i ndex=_i nt er nal sour ce=" *s pl unkd. l og" ( l og_l evel =ERROR OR l og_l evel =WARN* OR l og_l evel =FATAL OR l og_l evel =CRI TI CAL)
2. 3.
Select Save As > Alert . Specify the following alert field values. ♦ Title : Warning Errors ♦ Al ert t yp e: Real-time ♦ Trigger condition : Custom
156
♦ Custom C onditio n : search log_level=WARN* in 1 minute Select the List in Triggere d Alerts alert action. 5. Click Save .
4.
157
Lab 16 Macros Define and use search macros Search m acros are chunks of a search that you can reuse in multiple places, including
saved and ad hoc searches. Search macros can be any part of a search, such as an eval statement or search term, and do not need to be a complete command. You can also arguments. specify whether or not the macro field takes any
Create search macros in Splunk Web
In Settings > Advanc ed Search > Se arch macr os , click "New" to create a new search macro.
Define the search macro and its arguments
Your search macro can be any chunk of your search string or search command pipeline that you want to re-use as part of another search. Destination app is the name of the app you want to restrict your search macro to; by
default, your search macros are restricted to the Search app. Name is the name of your search macro, such as mymacr o.
If your search macro takes an argument, you need to indicate this by appending the number of arguments to the name; for example, if mymacr o required two arguments, it should be named mymacr o( 2) . You can create multiple search macros that have the same name but require different numbers of arguments: f oo, f oo(1 ) , f oo( 2) , et c . Definition is the string that your search macro expands to when referenced in another
search. If the search macro requires the user to input arguments, they are tokenized and indicated wrapping dollarspecified signs around for is example, argumentsbyvalues are then whenthe thearguments; search macro invoked.$ar g1$. The
158
If Eval G enerated D efiniti on? is checked, then the 'Definition' is expected to be an eval expression that returns a string that represents the expansion of this macro. If a ma cro definition in cludes a lea ding p ipe characte r (" |"), you may not u se it as the first term in searches fro m the UI.
Example: "| metadata type=sources". The UI does not do the macro expansion and cannot correctly identify the initial pipe to differentiate it from a regular search term. The UI constructs the search as if the macro name were a search term, after invalid. expansion would cause the metadata command to be incorrectly formed andwhich therefore Ar gu men ts are a comma-delimited string of argument names. Argument names may
only contain the characters: alphanumeric 'a-Z, A-Z, 0-9'; underscore '_'; and dash '-'. This list should not contain any repeated elements. If a ma cro argument includes quotes, you need to escape the quotes when you call the macro in your search. For example, if you wanted to pass a quoted
string as your macro's argument, you would use: `my- macro( "He sa i d \ "hel l o! \ "" ) ` . Validate your argument values
You can verify that the argument values used to invoke the search macro are acceptable. How to invoke search macros are discussed in the following section, "Apply macros to saved and ad hoc searches". Validation Expression is a string that is an 'eval' expression that evaluates to a
boolean or a string. If the validation expression is a boolean expression, validation succeeds when it returns true. If it returns false or is null, validation fails, and the Validati on Error Message is returned. If the validation expression is not a boolean expression, it is expected to return a string or NULL. If it returns null, validation is considered a success. Otherwise, the string returned is rendered as the error string. Apply macros to saved and ad hoc searches
To include a search macro in your saved or ad hoc searches, use the left quote (also known as a grave accent) character; on most English-language keyboards, this character is located on the same key as the tilde (~). You can also reference a search macro within other search macros using this same syntax.
159
Example - Combine search macros and transactions
Transactions and macro searches are a powerful combination that you can use to simplify your transaction searches and reports. This example demonstrates how you can use search macros to build reports based on a defined transaction. Here, a search macro, named "makesessions", defines a transaction session from events that share the same clientip value that occurred within 30 minutes of each other: t r ansact i on cl i ent i p maxpause=30m
This search takes web traffic events and breaks them into sessions, using the "makesessions" search macro: sour cet ype=access_ * | `makesess i ons`
This search returns a report of the number of pageviews per session for each day: sour cet ype=access_ * | `makesess i ons` | t i mechar t span=1d sum( event count ) as pagevi ews count as sessi ons
If you wanted to build the same report, but with varying span lengths, just save it as a search macro with an argument for the span length. Let's call this search macro, "pageviews_per_second(1)": sour cet ype=access_ * | `makesessi ons` | t i mechar t $spanar g$ sum( event count ) as pagevi ews count as sessi ons
Now, you can specify a span length when you run this search from the Search app or add it to a saved search: `pagevi ews_ per _second( span=1h) `
160
Lab 17 Lookups Example of HTTP status lookup This examples walks through defining a static lookup that adds two informational fields, st at us_de scr i pt i on and st at us_t ype, into your Web access events. This lets you search for the events you want when you might not know the specific error code. For example, instead of searching for all the server error codes, you can use st at us="Ser ver Er r or " . Upload the lookup table to Splunk Ente rprise 1. Download the ht t p_st at us. c sv file: http_status.csv Here's a sampling of the file: st at us, st at us_de scr i pt i on, st at us_t ype 100, Cont i nue, I nf ormat i onal 101, Swi t chi ng Pr ot ocol s, I nf or mat i onal 200, OK, Successf ul 201, Cr eat ed, Success f ul 202, Accept ed, Succes sf ul 203, Non- Aut hor i t at i ve I nf or mat i on, Success f ul ...
2. Go back to the Search app, then select Settings > Look ups . 3. In the Lookups page, select Add n ew for Looku p table file s . 4. In the Add n ew page, Select search for the destination app. Browse for the CSV file that you downloaded earlier. Name the lookup table http_status. Click Save. • • • •
Now, let's go back to the Settings > Look ups view. To do this, click on the Lookups link in the page's breadcrumb. You can always use this to navigate back to a previous view. Define the look up 1. From Settings > Lookups, select Add n ew for Lookup defini tions . In the Add new page: 2. Select search for the Destination app . 3. Name your lookup definition http_status. 4. Select File-based under Type . 5. Click Save. After Splunk Enterprise saves your lookup definition, it takes you to the following page: Notice there are some actions you can take on your lookup definition. Permissions lets you change the accessibility of the lookup table. You can Disable , Clone , and Move the lookup definition to a different app. Or, you can
161
Delete the definition. Once you define the lookup, you can use the l ookup command to invoke it in a search or you can configure the lookup to run automatically. Set the lookup to run automatically 1. Return to the Settings > Lookups view and select Add new for Auto mat ic lookups . In the Add n ew page: 2. Select search for the Destination app . 3. Name the lookup http_status . 4. Select http_status from the Lookup t able drop down. 5. Apply the lookup to the sour cet ype named access _combi ned. 6. Lookup input fields are the fields in our events that you want to match with the lookup table. Here, both are named status (the CSV column name goes on the left and the field that you want to match goes on the right): 7. Lookup output fields are the fields from the lookup table that you want to add to your events: status_description and status_type. The CSV column name goes on the left and the field that you want to match goes on the right. 8. Click Save.
162
Lab 18 Workflows
Example - Google search from field values Here's an example of the setup for a GET link workflow action that sets off a Google search on values of the t opi c field in search results:
163
In this example, we set the Label value to Googl e $t opi c$ because we have a field called t opi c in our events and we want the value of t opi c to be included in the label for this workflow action. For example, if the value for t opi c in an event is Cr eat ef i el dact i onsi nSpl unkWeb the field action displays as Google CreatefieldactionsinSplunkWeb in the t opi c field menu. The Googl e $t opi c$ action applies to all events.
164
The Googl e $t opi c$ action URI uses the GET method to submit the t opi c value to Google for a search.
Example - Provide an external IP lookup You have configured your Splunk Enterprise app to extract domain names in web services logs and specify them as a field named domai n. You want to be able to search an external WHOIS database for more information about the domains that appear. Here's how you would set up the GET workflow action that helps you with this. In the Workflow actions details page, set Action type to link and set Link method to get. You then use the Label and URI fields to identify the field involved. Set a Label value of WHOIS: $domain$. Set a URI value of http://whois.net/whois/$domain$. After that, you can determine: • • •
whether the link shows up in the field menu, the event menu, or both. whether the link opens the WHOIS search in the same window or a new one. restrictions for the events that display the workflow action link. You can target the workflow action to events that have specific fields, that belong to specific event types, or some combination of the two.
Set up a POST workflow action You set up POST workflow actions in a manner similar to that of GET link actions. However, POST requests are typically defined by a form element in HTML along with some inputs that are converted into POST arguments. This means that you have to identify POST arguments to send to the identified URI. 1. Navigate to Settings > Fields > Workflow Actions . 2. Click New to open up a new workflow action form. 3. Define a Label for the action.
The Label field enables you to define the text that is displayed in either the field or event workflow menu. Labels can be static or include the value of relevant fields. 4. Determine whether the workflow action applies to specific fields or event types in your data.
Use Apply only to the following fields to identify one or more fields. When you identify fields, the workflow action only appears events that have those fields, either in their event menu or field menus. If you leave it blank or enter an asterisk the action appears in menus for all fields.
165
Use Apply only to the following event types to identify one or more event types. If you identify an event type, the workflow action only appears in the event menus for events that belong to the event type. 5. For Show action in determine whether you want the action to appear in the Event menu, the Fields menus, or Both. 6. Set Action type to Link. 7. Under URI provide the URI for a web resource that responds to POST requests. 8. Under Open link in, determine whether the workflow action displays in the current window or if it opens the link in a new window. 9. Set Link method to Post. 10. Under Post arguments define arguments that should be sent to web resource at the identified URI.
These arguments are key and value combinations. On both the key and value sides of the argument, you can use field names enclosed in dollar signs to identify the field value from your events that should be sent over to the resource. You can define multiple key/value arguments in one POST workflow action. Enter the key in the first field, and the value in the second field. Click Add another field to create an additional POST argument. 11. Click Save to save your workflow action definition.
Splunk Enterprise automatically HTTP-form encodes variables that it passes in POST link actions via URIs. This means you can include values that have spaces between words or punctuation characters.
Example - Allow an http error to create an entry in an issue tracking application You have configured your Splunk Enterprise app to extract HTTP status codes from a web service log as a field called http_status . Along with the ht t p_st at us field the events typically contain either a normal single-line description request, or a multiline python stacktrace srcinating from the python process that produced an error. You want to design a workflow action that only appears for error events where http_status is in the 500 range. You want the workflow action to send the associated python stacktrace and the HTTP status code to an external issue management system to generate a new bug report. However, the issue management system only accepts POST requests to a specific endpoint. Here's how you might set up the POST workflow action that fits your requirements:
166
Note that the first POST argument sends ser ver er r or $ht t p_st at us$ to a t i t l e field in the external issue tracking system. If you select this workflow action for an event with an
167
ht t p_st aus of 500, then it opens an issue with the title ser ver er r or 500 in the issue tracking
system. The second POST argument uses the _r aw field to include the multiline python stacktrace in the descr i pt i on field of the new issue. Finally, note that the workflow action has been set up so that it only applies to events belonging to the er r or s_ i n_500_r ange event type. This is an event type that is only applied to events carrying http_error values in the typical HTTP error range of 500 or greater. Events with HTTP error codes below 500 do not display the submit error report workflow action in their event or field menus.
168
Lab 19 Tagging Tagging
Tags are used to label specific values of a ffield. For example, many names of servers may not be immediately recognized, and using a tag format can help them be more easily recognizable or distinguishable from each other. To tag the value of a ffield, use the following steps: 1. Go to Settings | Tags. A window will open, as shown in the following screenshot:
Adding Tags
2. Under List by tag name, click Add new. 3. Here we want to tag an item as ITEM14 whenever the value of itemId=EST-14, as shown in the following screenshot:
169
Naming Tags and Specifying Ffield Value Pairs
4. You will now see your tag listed as shown in the following screenshot:
List by Tag Name
170
5. Go back to the event list and click the > sign next to an event. You will see details of the event open up in a way similar to that presented in the following screenshot. You can see here thatitemid=EST-14has been tagged asITEM14. Now everywhere thatEST-14 occurs, it will be tagged asITEM14.
Note that itemid=EST-14 has been tagged as ITEM14
Tags enable you to search more easily and to convey meaning about the field values. When you search tag=ITEM14, all the cases where itemid=EST-14 show up. By using tags in this manner, you can facilitate your analysis.
Settingeventtypes
Another way of preparing data to be reported is to set event types, which let you put events into categories. When setting event types, you can use wildcards, field values, and Boolean expressions. This capability makes event types more versatile and powerful than tags, for which you can only use field values. As with tags, you can choose the categories you like. When setting event types, be aware of the following: 1. You can't do a sub-search to create an Event type. 2. You can't use pipes in a search that create an Event type.
171
As an example of how to create an Event type, take the following steps using the but t er cupgames ile: °
Enter this into the search bar: sourcetype="access_*" status=200 action=purchase
°
This creates a search for events where the sourcetype is an accessed web page, the access was successful (status=200), and it ended in a purchase:
Search that will be saved as an Event Type
172
3. Click Save As | Event Type in the upper-right corner of the screen and create a name for the event type. In this case, we have used the name success. 4. In this screenshot, when we enter but t er cupgames | st at s count by event t ype, we get a count of each event type. In this case, we have only one event type, so we get only one count in our table, but we could easily put other event types in:
Shows Count by Eventtype
173
5. If you want to remove an event type, go to Settings | Event types, and you will get a screen similar to what is shown in the following screenshot. Just ind the event type you want to remove and click on Delete:
Event Types (Notice that you can Delete the one you just made.)
174
