FortiAnalyzer 6.2 Cookbook

FortiAnalyzer - Cookbook
Version 6.2
FORTINET DOCUMENT LIBRARY
https://docs.fortinet.com
FORTINET VIDEO GUIDE

https://video.fortinet.com
FORTINET BLOG
https://blog.fortinet.com
CUSTOMER SERVICE & SUPPORT

https://support.fortinet.com
FORTINET TRAINING & CERTIFICATION PROGRAM

https://www.fortinet.com/support-and-training/training.html
NSE INSTITUTE
https://training.fortinet.com
FORTIGUARD CENTER
https://fortiguard.com/
END USER LICENSE AGREEMENT

https://www.fortinet.com/doc/legal/EULA.pdf
FEEDBACK
Email: [email protected]
September 11, 2020

FortiAnalyzer 6.2 Cookbook
05-620-556993-20120911
TABLE OF CONTENTS
Change Log 5
FortiAnalyzer System Setup 6
FortiClient user avatar 6
Enabling logging from FortiClient to FortiAnalyzer 6
Setting up a FortiAnalyzer HA cluster 9
Manage logs and data sources 12
Fetching logs from one FortiAnalyzer to another 12
Creating an admin on the remote FortiAnalyzer 12
Creating a fetch profile on the local FortiAnalyzer 13
Requesting a log fetch 14
Reviewing and approving fetch requests 15
Performing post request actions 16
What is the difference between Log Forward and Log Aggregation modes? 17
Historical reports 19
Configuring a report to run an LDAP user filter 19
Configuring an LDAP server 19
Filtering a report with an LDAP server 21
Running the report to verify the results 22
Creating charts in Log View with Chart Builder 23
Building charts with Chart Builder 23
Applying a chart to a report 24
Real-time dashboards 26
Configuring the FortiSandbox Detection dashboard in FortiAnalyzer 26
Creating a firewall policy on FortiSandbox 26
Creating a log server for FortiAnalyzer 27
Adding a FortiSandbox to FortiAnalyzer and viewing scanned files 28
Fabric connectors 30
Integrating FortiAnalyzer with ServiceNow 30
Locating your ServiceNow API URL 30
Creating a fabric connector for ServiceNow 31
Sending notifications to ServiceNow 32
Creating a Google cloud connector 33
Creating a Google storage bucket 33
Locating the remote path in Google cloud 38
Importing a CA certificate 39
Creating a Google Cloud connector 40
Testing a Google cloud connector 42
FortiView and NOC-SOC 44
How IOC works 44
Understanding suspicious list detection 44
Viewing IOC licenses and TIDB package downloads 45
Configuring FortiGate to FortiAnalyzer REST API authentication 46
Throttling IOC alerts 46
FortiAnalyzer 6.2 Cookbook 3

Fortinet Technologies Inc.
Debugging IOC notifications 47
Troubleshooting 48
Troubleshooting report performance issues 49
Check the report diagnostic log 49
Check hardware and software status 52
get system status 52
diagnose fortilogd lograte 53
Check data policy and log storage policy 53
Check report and chart settings 53
Check and adjust report auto-cache daemon 54
get system performance 54
Check and adjust report hcache 55
diagnose test application sqlrptcached 2 55
execute sql-report hcache-check <ADOM> <schedule-id> 56
Report performance troubleshooting commands 56
Troubleshooting a dataset query 57
Troubleshooting a custom dataset 58
SQL functions for formatting and converting data types 58
Macros for formatting date and time in a dataset 59
Troubleshooting an empty chart 59
Common issues 60
CLI commands for troubleshooting 61

Change Log
Date Change Description
2019-11-21 Initial release.
2019-11-21 Added the following sections:

l FortiAnalyzer System Setup on page 6
l Manage logs and data sources on page 12
l Historical reports on page 19
l FortiView and NOC-SOC on page 44
l Troubleshooting on page 48
2019-11-26 Added Integrating FortiAnalyzer with ServiceNow on page 30.
2019-11-27 Added Configuring the FortiSandbox Detection dashboard in FortiAnalyzer on page 26.
2019-12-03 Added FortiClient user avatar on page 6 and Creating a Google cloud connector on page 33.
2020-09-11 Updated title for Configuring the FortiSandbox Detection dashboard in FortiAnalyzer on page
26 configuration.

FortiAnalyzer System Setup
This section contains information about FortiAnalyzer system setup.

l Where is the user avatar from Log View, FortiView, and Event
l Setting up a FortiAnalyzer HA cluster on page 9
FortiClient user avatar
FortiClient user avatars are included in logs sent to FortiAnalyzer. Where you can view FortiClient logs and avatars in
FortiAnalyzer depends on the version of FortiClient and whether FortiClient Telemetry connects to FortiGate or to
FortiClient Endpoint Management Server (EMS):
l When FortiClient Telemetry connects to FortiGate, FortiClient sends logs (including avatars) to FortiGate, and the
logs display in FortiAnalyzer under the FortiGate device as a sub-type of security.
The avatar is synchronized from FortiGate to FortiAnalyzer by using the FortiOS REST API.
l When FortiClient Telemetry connects to FortiClient EMS, FortiClient sends logs (including avatars) directly to
FortiAnalyzer, and logs display in a FortiClient ADOM.
The following table summarizes where FortiClient versions can establish Telemetry connections:
FortiClient Version Where FortiClient Telemetry Connects Location of Logs in FortiAnalyzer
FortiClient 6.0.x and FortiGate Under the FortiGate device as a sub-type of

earlier security
FortiClient EMS In a FortiClient ADOM
FortiClient 6.2.x and FortiClient Telemetry can connect only to In a FortiClient ADOM
later FortiClient EMS
Enabling logging from FortiClient to FortiAnalyzer
FortiClient 6.0.x and 6.2.x can send logs directly to FortiAnalyzer when FortiClient Telemetry connects to FortiClient
EMS, and logging to FortiAnalyzer is enabled in FortiClient EMS.
This section provided a high-level overview of how to configure FortiClient to send logs directly to FortiAnalyzer. With
this configuration, FortiClient logs are displayed in the FortiClient ADOM in FortiAnalyzer.
To enable logging from FortiClient to FortiAnalyzer:
1. Install FortiClient on endpoints.

2. Install FortiClient EMS on a Windows server.

3. In FortiClient, connect Telemetry to FortiClient EMS.
4. In FortiClient EMS, edit a profile to enable Upload Logs to FortiAnalyzer/FortiManager and the associated
settings, and ensure the profile is assigned to the endpoints.
FortiClient automatically receives the profile from FortiClient EMS, and the profile updates FortiClient settings on
the endpoint. FortiClient now sends logs to FortiAnalyzer.

5. In FortiClient, go to Settings to view the settings in the Logging section.
You can also view the settings in the FortiClient configuration file:
6. On the FortiClient endpoint, you can force FortiClient to resend avatar metadata to FortiAnalyzer by ending the
FortiTray.exe service.

Setting up a FortiAnalyzer HA cluster
You can configure two or more FortiAnalyzer units in a High Availability (HA) cluster to provide real-time redundancy in
case a primary unit fails. High Availability clusters also alleviate the load on the primary unit by using backup units for
processes such as running reports.
The following is an overview of how to configure FortiAnalyzer units in an HA cluster:
1. Go to System Settings > HA.
2. Set the Operation Mode of the primary unit to High Availability.
3. Configure the settings for the primary (Master) unit.
4. Configure the settings for the secondary (Slave) units.
All the units must:

l Be of the same FortiAnalyzer series
l Be visible on the network
l Run in the same operation mode: Analyzer or Collector
To configure the primary unit in an HA cluster:
1. Go to System Settings > HA.

2. Set the Operation Mode to High Availability.
3. Set the Preferred Role to Master.
4. Configure the Cluster Virtual IP settings:
Interface Select the interface to be used as the clustered Virtual IP.
IP Address Type the IP address to be used by the HA cluster to provide redundancy.
5. In the Peer IP and Peer SN box, type the Peer IP and Peer SN for each secondary (Slave) unit. The maximum is
three units.
6. Type the Group Name, Group ID, and Password. These settings must be the same for all the units in the cluster.
7. Click Apply.
To configure secondary units in an HA cluster:
1. Set the Preferred Role to Slave.

2. Configure the Cluster Virtual IP settings with the HA cluster's Interface and IP Address.
Interface Select the interface being used by the cluster as the Virtual IP.
IP Address Type the IP address being used by the cluster to provide redundancy.
3. In the Peer IP and Peer SN box, type the Peer IP and Peer SN for the primary (Master) unit and each secondary
(Slave) unit.

4. Type the Group Name, Group ID, and Password. These settings must be the same for all the units in the cluster.
5. Click Apply.
Cluster Settings
Cluster Status
Operation Mode Select High Availability to configure the FortiAnalyzer unit for HA.
Select Standalone to stop operating in HA mode.
Preferred Role Select the preferred role when this unit first joins the HA cluster.
If the preferred role is Master, then this unit becomes the primary unit if it is configured first
in a new HA cluster. If there is an existing primary unit, then this unit becomes a backup
(slave) unit.
The default is Slave so that the unit can synchronize with the primary unit. A slave or
backup unit cannot become a master or primary unit until it is synchronized with the current
primary unit.
Cluster Virtual IP
Interface Select the interface the FortiAnalyzer HA unit uses to provide redundancy.
IP Address Type the IP address for which the FortiAnalyzer HA unit is to provide redundancy.
Cluster Settings
Peer IP Type the IP address of another FortiAnalyzer unit in the cluster.
Peer SN Type the serial number of the FortiAnalyzer unit corresponding to the entered IP address.

Group Name Type a group name that uniquely identifies the FortiAnalyzer HA cluster. All units in a
cluster must have the same Group Name, Group ID and Password.
Group ID Type a group ID from 1 to 255 that uniquely identifies the FortiAnalyzer HA cluster. The
primary unit and all backup units must have the same Group ID.
Password Type a password for the HA cluster. All members of the HA cluster must have the same
password.
Heart Beat Interval The time the primary unit waits between sending heartbeat packets, in seconds. The
heartbeat interval is also the amount of time that backup units waits before expecting to
receive a heartbeat packet from the primary unit.
Priority The priority or seniority of the backup unit in the cluster.
Log Data Sync This option is on by default. It provides real-time log synchronization among cluster
members.

Manage logs and data sources
This section contains information about managing logs and data sources:
l Fetching logs from one FortiAnalyzer to another on page 12
l What is the difference between Log Forward and Log Aggregation modes? on page 17
Fetching logs from one FortiAnalyzer to another
Log fetching allows administrators to retrieve archived logs from one FortiAnalyzer device to another. The fetching
FortiAnalyzer can query the server FortiAnalyzer and retrieve the log data for a specified device and time period, based
on specified filters. The retrieved data are then indexed, and can be used for data analysis and reports.
The following is an overview of the log fetch workflow.
1. Preparing the log fetch:
l Identify what is to be fetched, including the log device, time period, log type, and the size of the log.
l Prepare the local FortiAnalyzer by allocating enough storage to the ADOM that will receive the logs.
2. Creating an admin on the remote FortiAnalyzer on page 12.
3. Creating a fetch profile on the local FortiAnalyzer on page 13.
4. Requesting a log fetch on page 14.
5. Reviewing and approving fetch requests on page 15.
6. Performing post request actions on page 16.
Creating an admin on the remote FortiAnalyzer
To request logs from a remote device, you will need to create or assign an admin account to be used as the fetch login
credentials. The admin account can be either a Standard_User or Super_User.
To create a new admin:
1. Go to System Settings > Administrators > Create New.

2. Configure the settings in the New Administrator pane, then click OK.
Creating a fetch profile on the local FortiAnalyzer
Create a Log Fetch profile on the FortiAnalyzer that will receive the fetched logs. You will use this profile to request logs.
To create a new log fetch profile:
1. Go to System Settings > Fetcher Management > Profiles.

2. Click Create New and configure the profile settings.
Name Enter a name for the profile.
Server IP Enter the IP address of the remote FortiAnalyzer.
User Enter the username of the admin you created on the remote device.
Password Enter the password of the admin you created on the remote device.
Click OK.

Requesting a log fetch
You can use the fetch profile you created to fetch logs from the remote device. You have the option of selecting an
ADOM on the local device to receive the logs, or a new one.
To request a log fetch:
1. Go to System Settings > Fetcher Management, the click the Profiles tab.
2. Right-click the fetch profile you created and click Request Fetch. Configure the settings in the Fetch Logs dialog,
then click Request Fetch.
Name Displays the name of the fetch server you have specified.
Server IP Displays the IP address of the server you have specified.
User Displays the username of the server administrator you have provided.
Secure Connection Select to use SSL connection to transfer fetched logs from the server.
Server ADOM Select the ADOM on the server the logs will be fetched from. Only one ADOM
can be fetched from at a time.
Local ADOM Select the ADOM on the client where the logs will be received.
Either select an existing ADOM from the dropdown list, or create a new ADOM
by entering a name for it into the field.
Devices Add the devices and/or VDOMs that the logs will be fetched from. Up to 256
devices can be added.
Click Select Device, select devices from the list, then click OK.
Enable Filters Select to enable filters on the logs that will be fetched.
Select All or Any of the Following Conditions in the Log messages that
match field to control how the filters are applied to the logs.
Add filters to the table by selecting the Log Field, Match Criteria, and Value
for each filter.
Time Period Specify what date and time range of log messages to fetch.
Index Fetch Logs If selected, the fetched logs will be indexed in the SQL database of the client
once they are received. Select this option unless you want to manually index
the fetched logs.

3. (Optional) Synchronize the devices and ADOMs.

a. Go to System Settings > Fetcher Management and click the Profiles tab.
b. Select the log fetch profile then click Sync Devices in the toolbar.
The devices and ADOMs must be synchronized with the server if this is the first time the
fetching device is fetching logs from the remote device, or if any changes have been made
to the devices or ADOMs since the last fetch.
If a new ADOM is created, the new ADOM will mirror the disk space and data policy of the
corresponding server ADOM. If there is not enough space on the device, the client will
create an ADOM with the maximum allowed disk space and give a warning message. You
can then adjust disk space allocation as required.
Reviewing and approving fetch requests
You will need to approve the fetch request on the remote FortiAnalyzer to send the logs to the local device.
To review and approve a fetch request:
1. On the remote FortiAnalyzer, go to System Settings > Fetcher Management, and click the Sessions tab.
2. Find the request in the Received Request section. The status of the request will be Waiting for approval.

3. Click Review, then Accept the request to transfer the logs.
You can Pause and Resume the transfer as required.
Performing post request actions
You may need to rebuild the ADOM after the transfer is complete depending on the Log Fetch settings:
To perform post fetch actions:
Is Index Fetched Logs enabled Yes The ADOM is rebuilt automatically and the log fetch workflow is
in the Log Fetch settings? complete.
No You will need to rebuild ADOM manually from the CLI.

What is the difference between Log Forward and Log Aggregation

modes?
Log Forwarding: Logs are forwarded to a remote server in real-time or near real-time as they are received as specified
by a device filter, log filter, and log format.
Log Aggregation: As FortiAnalyzer receives logs from devices, it stores them, and then forwards the collected logs to
a remote FortiAnalyzer at a specified time every day.
Log Forwarding and Log Aggregation appear as different modes in the system log-forwarding configuration:
FAZVM64 # config system log-forward
(log-forward)# edit 1
(1)# set mode
aggregration Aggregate logs and archives to Analyzer.
disable Do not forward or aggregate logs.
f orwarding Realtime or near realtime forwarding logs to servers.
The following table lists the differences between the two modes:
Log Forwarding Log Aggregation
Configuration Portal GUI or CLI CLI
Remote Server Type FortiAnalyzer FortiAnalyzer

Syslog/CEF
Device Filter Support Yes Yes
Log Filter Support Yes No
Log Archive Support Yes Yes
Server Port customization Yes (Except for FortiAnalyzer) No
Log Field Exclusion Yes No
Log Delay Real-time (max 5 minutes delay) Max 1 day

Log Forwarding Log Aggregation
Meta-data synchronization Yes No
Secure channel support Yes (SSL as reliable connection) Yes (rsync + SSH)
Network bandwidth Normal (as log traffic received) Peak hour as aggregation starts to
finish
Impact on remote Normal (as log volume received) Potentially large table
FortiAnalyzer (If there is a mix of incoming real-time
and real-time logs.)

Historical reports
Historical reports
This section contains information about historical reports:

l Configuring a report to run an LDAP user filter on page 19
l Creating charts in Log View with Chart Builder on page 23
Configuring a report to run an LDAP user filter
You can filter a report to only the show members of a group in an LDAP server.
To configure the report:
1. Add the LDAP server to System Settings. SeeConfiguring an LDAP server on page 19
2. Apply the LDAP server to the report filter. See Filtering a report with an LDAP server on page 21.
3. Run the report to verify the output. See Running the report to verify the results on page 22.
The topics that follow will demonstrate how to filter the Admin and System Events Report to show data for the group
members in Distinguished Name: cn=group1,ou=groups,dc=fortinet,dc=com in the report output.
Configuring an LDAP server
You can use the GUI or CLI console to configure an LDAP server in System Settings.
Requirements
l The LDAP server is ready and accessible

l The group members are configured properly

Historical reports
To configure an LDAP server with the GUI:
1. Go to System Settings > Remote Authentication Server.

2. Click Create New > LDAP Server.
3. Configure the LDAP server settings, and then click OK.
Name Enter a name to identify the LDAP server.
Server Name/IP Enter the IP address or fully qualified domain name of the LDAP server.
Port Enter the port for LDAP traffic. The default port is 389.
Common Name Identifier The common name identifier for the LDAP server. Most LDAP servers use cn.
However, some servers use other common name identifiers such as UID.
Distinguished Name The distinguished name is used to look up entries on the LDAP server.
The distinguished name reflects the hierarchy of LDAP database object
classes above the common name identifier. Clicking the query distinguished
name icon will query the LDAP server for the name and open the LDAP
Distinguished Name Query window to display the results.
Bind Type Select the type of binding for LDAP authentication: Simple, Anonymous, or
Regular.
User DN When the Bind Type is set to Regular, enter the user DN.
Password When the Bind Type is set to Regular, enter the password.
Secure Connection Select to use a secure LDAP server connection for authentication.
Protocol When Secure Connection is enabled, select either LDAPS or STARTTLS.
Certificate When Secure Connection is enabled, select the certificate from the
dropdown list.
Administrative Domain Choose the ADOMs that this server will be linked to for reporting: All ADOMs
(default), or Specify for specific ADOMs.
Advanced Options
adom-attr Specify an attribute for the ADOM.

Historical reports
attributes Specify the attributes such as member, uniquemember, or memberuid.
connect-timeout Specify the connection timeout in millisecond.
filter Specify the filter in the format (objectclass=*)
group Specify the name of the LDAP group.
memberof-attr Specify the value for this attribute. This value must match the attribute of the
group in LDAP Server. All users part of the LDAP group with the attribute
matching the memberof-attr will inherit the administrative permissions
specified for this group.
profile-attr Specify the attribute for this profile.
secondary-server Specify a secondary server.
tertiary-server Specify a tertiary server.
To configure an LDAP server with the CLI console:
Click the CLI Console icon on the right side of the banner on any page.
The following script demonstrates how to create an LDAP server with the CLI console:
config system admin ldap
(ldap)# edit ldap1
(ldap1)# get
name : ldap1
server : 10.2.129.132
secondary-server : (null)
tertiary-server : (null)
cnid : cn
dn : ou=groups,dc=fortinet,dc=com
port : 389
type : regular
username : cn=admin,dc=fortinet,dc=com
password : *
memberof-attr : (null)
profile-attr : (null)
adom-attr : (null)
group : (null)
filter : (objectclass=*)
attributes : uniquemember
secure : disable
connect-timeout : 500
adom:
== [ all_adoms ]
adom-name: all_adoms
(ldap1)# end
Filtering a report with an LDAP server
After you add an LDAP server to System Settings, you can use it in a report filter.

Historical reports
This example uses the Admin and System Events Report to demonstrate how to apply an LDAP server to a report
filter.
1. Go to Reports and select the Admin and System Events Report.
2. Click the Settings tab, then expand the Filters section.
3. Use the following settings to configure the filter:
Log Field Select Group (group).
Match Criteria Select Equal to.
Value Select group1.
4. Click LDAP Query and set LDAP server to the LDAP server you created, then click Apply.
Running the report to verify the results
To run the report:
Click View Report > Run report.

The report displays the users in the group: cn=group1,ou=groups,dc=fortinet,dc=com in the Login
Summary chart and the group name in the Report Filters.

Historical reports
Creating charts in Log View with Chart Builder
You can use Chart Builder to create custom charts based on the view and filters in Log View . After the chart is created, it
is saved to the Chart Library where you can apply it to a report.
To create a chart and apply it to a report:
1. Create the chart in Chart Builder. See Building charts with Chart Builder on page 23.
2. Apply a custom chart to a report. See Applying a chart to a report on page 24.
Building charts with Chart Builder
You can create a new chart based on the log view or a filter. To view the chart you created, go to Reports > Report
Definitions > Chart Library.
To create a chart with Chart Builder:
1. Go to Log View.
2. Select a log view and apply filters as required.

3. In the toolbar, click Tools > Chart Builder.

Historical reports
4. In the Chart Builder dialog, configure the chart settings, and click Save.
Name Type a name for the chart.
Columns Select which columns of data to include in the chart based on the log
messages that are displayed on the Log View page.
Group By Select how to group data in the chart.
Order By Select how to order data in the chart.
Sort Select a sort order for data in the chart.
Show Limit Show Limit
Device Displays the device(s) selected on the Log View page.
Time Frame Displays the time frame selected on the Log View page.
Query Displays the query being built.
Preview Displays a preview of the chart.
Applying a chart to a report
You can add a chart in the chart library to a report from the report's Layout tab.
To insert a chart in a report:
1. Go to Reports > All Reports.

2. Click Create New to open the Create Report dialog. Complete the fields in the dialog, then click OK.
Name Type a name for the new report.

The following characters are not supported in report names: \ / " ' < > & , | # ?
% $ +
Create from l Select Blank to create a report without using a template.

l Select From Template, then select a template from the dropdown list.

Historical reports
The template populates the Layout tab of the report.
Save to Folder Select the folder that the new report will be saved to from the dropdown list.
3. Click Layout > Insert Chart to open the Insert Chart dialog.
4. In the Chart area, select the chart you created in Log View. Configure the chart settings, then click OK.
5. Click Apply in the Layout tab.

6. Run the report.

Real-time dashboards
This section contains information about real-time dashboards.

l Configuring the FortiSandbox Detection dashboard in FortiAnalyzer on page 26
Configuring the FortiSandbox Detection dashboard in FortiAnalyzer
You can use FortiAnalyzer to monitor FortiSandbox detections in the FortiSandbox Detection dashboard in FortiView .
Some configurations are required on FortiSandbox to add the device to FortiAnalyzer. After you add the device, go to
SOC > FortiView > Threats > FortiSandbox Detection to view the scanned files.
To detect FortiSandbox in FortiAnalyzer:
1. Create a firewall policy. See Creating a firewall policy on FortiSandbox on page 26.
2. Create a log server. See Creating a log server for FortiAnalyzer on page 27
3. Add FortiSandbox to FortiAnalyzer and viewing the FortiSandbox Detection dashboard. See Adding a
FortiSandbox to FortiAnalyzer and viewing scanned files on page 28.
Creating a firewall policy on FortiSandbox
You can use the CLI console in FortiGate to configure a firewall policy, then specify the IP address of the FortiAnalyzer
you want to monitor the FortiSandbox.
To configure FortiGate System settings:
1. In the FortiGate device, click the CLI Console icon on the right side of the banner on any page.
2. Specify the FortiSandbox in the global configuration:
config antivirus profile
edit "test"
set ftgd-analytics everything config http
set options scan avmonitor
end config ftp
set options scan avmonitor
end config imap
set options scan
end config pop3
set options scan
end config smtp
set options scan
end config nntp
set options scan
end
next
end

3. Create an antivirus profile to allow FortiGate to submit all files scanned by AntiVirus to FortiSandbox. The following
is a sample antivirus profile.
config firewall policy
edit 13
set name "to-server1"
set uuid 5107b480-3d19-51e8-f1c1-571602a6375b
set srcintf "lan"
set dstintf "wan1"
set srcaddr "net-local"
set dstaddr "server1"
set action accept
set schedule "always"
set service "ALL"
set utm-status enable
set logtraffic all
set fsso disable
set av-profile "test"
set ssl-ssh-profile "certificate-inspection"
set nat enable
next
end
4. Use the antivirus profile in the firewall policy. The following is a sample firewall policy:
config firewall policy
edit 13
set name "to-server1"
set uuid 5107b480-3d19-51e8-f1c1-571602a6375b
set srcintf "lan"
set dstintf "wan1"
set srcaddr "net-local"
set dstaddr "server1"
set action accept
set schedule "always"
set service "ALL"
set utm-status enable
set logtraffic all
set fsso disable
set av-profile "test"
set ssl-ssh-profile "certificate-inspection"
set nat enable
next
end
5. Specify the IP address of the FortiAnalyzer unit for FortiGate to send logs.
configure log fortianalyzer setting
set status enable
set server <ip address of FortiAnalyzer> set upload-option realtime
end
Creating a log server for FortiAnalyzer
Use FortiSandbox to create a log server to specify the FortiAnalyzer that will monitor the scanned files.

To create a log server:
1. Open FortiSandbox and go to Log & Report > Log Servers.

2. Click Create New in the toolbar and configure the following settings:
Name Enter a name for the new server entry.
Type Select FortiAnalyzer from the dropdown list.
Log Server Address Enter the log server IP address for the FortiAnalyzer device.
Port Enter the port number. The default port is 514.
Status Select Enable to send logs to the server.
Log Level Set the logging levels to be forwarded to the log server. The following options
are available:
l Enable Alert Logs. By default, only logs of non-Clean rated jobs are sent.
Users can choose to send Clean Job Alert Logs by selecting Include job
with Clean Rating.
l Enable Critical Logs
l Enable Error Logs
l Enable Warning Logs
l Enable Information Logs
l Enable Debug Logs
3. Click OK.
Adding a FortiSandbox to FortiAnalyzer and viewing scanned files
You can use the IP address of the FortiSandbox you configured to add it to FortiAnalyzer with Device Manager.
To add the FortiSandbox:
If using ADOMs, ensure that you are in the correct ADOM.

1. In FortiAnalyzer, go to Device Manager.
2. Click Add Device to enter the FortiSandbox information in the dialog box.
IP Address Type the IP address for the FortiSandbox device.

SN Type the serial number for the FortiSandbox device.
Device Name Type a name for the FortiSandbox device.
Device Model Select the model of the FortiSandbox device.
Firmware Version Select the firmware version of the FortiSandbox device.
Description Type a description of the FortiSandbox device (optional).
3. Click Next.
The device is added to the ADOM and, if successful, is ready to begin sending logs to the FortiAnalyzer unit.
4. Click Finish.
5. In the Device Manager, select the FortiSandbox you added, and click Edit in the toolbar.
6. Enter the Admin User and Password to allow FortiAnalyzer to access the FortiSandbox, then click OK.
To view FortiSandbox scanned files in the FortiSandbox Detection dashboard:
1. Go to SOC > FortiView > Threats > FortiSandbox Detection to view the files scanned by FortiSandbox.
2. Click a file to view the Drilldown Panel.
3. Click the FortiSandbox Scan link to view the Sandbox Execution Details panel.

Fabric connectors
Fabric connectors
This section contains information about fabric connectors:

l Integrating FortiAnalyzer with ServiceNow on page 30
l Creating a Google cloud connector on page 33
Integrating FortiAnalyzer with ServiceNow
Admins can use ServiceNow to manage incidents and events with the FortiAnalyzer App. To notify ServiceNow when an
incident is raised in FortiAnalyzer, create a fabric connector, then enable notifications for the fabric connector you
created.
Before you begin, ensure you have completed the following tasks in ServiceNow:
l Install the ServiceNow FortiAnalyzer App.
l Go to FortiAnalyzer App > FortiAnalyzer System Properties, and create a connection for the ServiceNow API.
To integrate FortiAnalyzer with ServiceNow:
1. Record the ServiceNow API URL. See Locating your ServiceNow API URL on page 30
2. Create a fabric connector for ServiceNow. See Creating a fabric connector for ServiceNow on page 31
3. Enable notifications to notify ServiceNow when an incident is raised. See Sending notifications to ServiceNow on
page 32.
Locating your ServiceNow API URL
You will need to know the ServiceNow API URL and login credentials to create a fabric connector in FortiAnalyzer.

Fabric connectors
To locate your ServiceNow API URL:
1. Open ServiceNow and go to FortiAnalyzer App > FortiAnalyzer System Properties.

2. In the Connect to ServiceNow API section, copy the URL in the ServiceNow API URL field.
Creating a fabric connector for ServiceNow
You will need to create a fabric connector to notify ServiceNow when an incident is raised in FortiAnalyzer. To configure
the fabric connecter, you need to know the ServiceNow API URL and login credentials.
To create a fabric connector for ServiceNow:
1. Open FortiAnalyzer and go to Fabric View.
2. Click Create New. The Create New Fabric Connector dialog opens.

Fabric connectors
3. Select the ServiceNow connector type, then click Next.
4. Configure the fabric connector.
Setting Description
Name Type a name for the fabric connector. The name cannot be changed once the fabric
connector is created.
Description (Optional) Type a description for the fabric connector. You can change the description
after the fabric connector is created.
Protocol Select HTTPS.
Method Select POST.
Title Type a title for the fabric connector. You can change the title after the fabric connector is
created.
URL Type the ServiceNow API URL located in FortiAnalyzer App > FortiAnalyzer System
Properties.
User Name Type the Username located in FortiAnalyzer App > FortiAnalyzer System Properties.
Password Type the Password located in FortiAnalyzer App > FortiAnalyzer System Properties.
Status Toggle ON to enable the fabric connector.
5. Click OK.
Sending notifications to ServiceNow
You will need to enable notifications in FortiAnalyzer to trigger an incident in ServiceNow.

Fabric connectors
To enable notifications in FortiAnalyzer:
1. Go to Incidents & Events > Incidents.

2. Click Settings in the toolbar.
3. From the Fabric Connector 1 dropdown, select the fabric connector you created for ServiceNow.
4. Select the notification settings, and click OK.
Creating a Google cloud connector
When logs hit a certain size, they rollover and begin deleting the earliest entries to make room for additional logs. To
prevent losing any log entries, FortiAnalyzer can periodically back up older logs to an external object storage location in
Google Cloud. This off-site log archive will help ensure compliance and data redundancy in case there is a local storage
or outage in FortiAnalyzer.
To create a Google cloud connector:
1. Create a storage bucket on Google cloud.Creating a Google storage bucket on page 33.
2. Import the requited CA certificates on FortiAnalyzer. See Importing a CA certificate on page 39
3. Create a cloud connector on FortiAnalyzer. See Creating a Google Cloud connector on page 40.
4. Roll the logs to the target bucket. See Creating a Google Cloud connector on page 40
5. Test the connector. See Testing a Google cloud connector on page 42
Creating a Google storage bucket
Google storage buckets must be globally unique. For simplicity, this example uses the project name. However, you can
use any name you like.
For more information about creating Google storage buckets, see the product help.

Fabric connectors
To create a Google storage bucket:
1. Open the Cloud Storage browser in the Google Cloud Console and click Create Bucket.
2. Name your bucket.
3. Select a region for the bucket. You will need this location when you create a cloud connector in FortiAnalyzer.
4. Set the object storage type to Standard.

Fabric connectors
5. Set the access control to Fine grained.
6. Set the encryption to Google-managed key.
7. Click Create.
To view the bucket details:
Go to Storage > Browser.

l Use the Objects tab to test the cloud connector. See Testing a Google cloud connector on page 42.
l Use the Permissions tab to see who can access this bucket. The Google account JSON key will be tied to these
permissions. See Creating a Google service account key on page 36

Fabric connectors
Locating a Google project number
The project number is located in the Project info widget.

1. Open the project in the Google Cloud Platform.
2. Open the Home page.
3. Locate the Project Info widget and copy the Project Number.
Creating a Google service account key
A private key is required to create a fabric connector for Google Cloud. After you create the key, save it to your computer
and paste the entire contents of the JSON file in the Service Account Credentials field when you create the cloud
connector.
You can download an existing service account key from the bucket details page. See Creating a Google storage bucket
on page 33.
To create a private key in Google Cloud:
1. Open your project in Google Cloud Platform.

2. In the Navigation pane, go to IAM & admin > Service Accounts. The Service accounts page opens.
3. Click Create Service Account. The Create service account page opens.
4. Type the Service account name, Service account ID, and Service account description, then click Create.

Fabric connectors
5. Select the account permissions from the Role dropdown, then click Continue.
6. In the Grant users access to this service account section, click Create Key.
7. Click Create and save the key to your computer.

Fabric connectors
8. Paste the entire contents of the JSON file in the Service Account Credentials field when you create the cloud
connecter.
Locating the remote path in Google cloud
Use the Google bucket name for the Remote Path in the Device Logs Settings. The bucket name is also the name of
the fabric connector.

Fabric connectors
To locate the bucket name in Google cloud:
1. In the navigation pane, go to Storage > Browser.
2. Copy the name of the bucket as it appears in the Name column and paste it into the Remote Path field.
Importing a CA certificate
Google requires you provide CyberTrust and GobalSign certificates when creating a cloud object.
To import a CA certificate:
1. Go to System Settings > Certificates > CA Certificates.

2. Click Import in the toolbar, or right-click and select Import. The Import dialog box opens.
3. Click Browse... and locate the certificate file on the management computer, or drag and drop the file onto the
dialog box.
4. Click OK to import the certificate.
To view a CA certificate's details:
1. Go to System Settings > Certificates > CA Certificates.

2. Select the certificates you need to see details about.
3. Click View Certificate Detail in the toolbar, or right-click and select View Certificate Detail. The View CA
Certificate page opens.

Fabric connectors
4. Click OK to return to the CA certificates list.
Creating a Google Cloud connector
Before you begin creating a Google cloud connector, ensure you have:
l Imported the required CA certificates. See Importing a CA certificate on page 39.
l Downloaded the private key from Google Cloud. See Creating a Google service account key on page 36.
To create Google cloud connector:
1. Go to Fabric View, and click Create New in the toolbar. The Create New Fabric Connector dialog opens.
2. In the Storage section, click Google then click Next.

Fabric connectors
3. Configure the fabric connector settings, then click OK.
Property Description
Name Type a name for the fabric connector.
Comments (Optional) Add comments about the connector.
Title Type a title for the fabric connector.
Cloud Project Number Type the project number from the Google Cloud Platform dashboard.
See Locating a Google project number on page 36
Service Account Credentials Paste the entire Google account JSON key into the field. Click the eye icon to
Show or Hide the key.
See Creating a Google service account key on page 36.
Cloud Location Type the bucket region. See Creating a Google storage bucket on page 33
The fabric connecter appears in the Fabric Connectors pane.
To roll the logs to Google cloud:
1. Go to System Settings > Device Log Settings.

2. In the Registered Device Logs section, click Upload logs to cloud storage > Create New.
3. Configure the following cloud storage settings and click OK.

Fabric connectors
Property Description
Cloud Storage Connector Type the name you gave to the fabric connector.
Remote Path Type the globally unique name you gave to your bucket. For simplicity use the
project name. See Locating the remote path in Google cloud on page 38.
Upload option Choose between Rolling or Schedule.
4.
Testing a Google cloud connector
You can use the CLI console to test the cloud connector before the logs have rolled over or a scheduled backup is
performed.
To test a cloud connector:
1. Open the CLI console on any page and type: diag test application uploadd 62 <connector
name> <bucket name>.
If the connector is working, the output will show success.
2. Go to the storage bucket on Google Cloud and look for the test files you uploaded.

Fabric connectors
To test a cloud connector with a shell prompt:
1. With the default settings, access to shell will give the following message:
FAZ1000D # execute shell
Shell disabled.
2. Use the following commands to enable shell on the FortiAnalyzer:
FAZ1000D # config system admin setting
(setting)# set shell-access enable
Enter new password: *****
Confirm new password: *****
FAZ1000D # end
The shell is now enabled.
FAZ1000D # execute shell
Enter password:
sh-4.3#
sh-4.3#
3. Open the CLI console on any page and type: rclone --config=/drive0/private/rclone.cfg ls
<connector-name>:<bucketname>
If the connector is working you will not see any errors.

FortiView and NOC-SOC
The following section provides information about FortiView and NOC-SOC.

l How IOC works on page 44
How IOC works
IOC (Indicators of Compromise) detects compromised client hosts (endpoints) by comparing the IP, domain, and URL
visited against the TIDB package, downloaded daily from FortiGuard. Compromised hosts are listed in FortiView in a
table or map style, and drilling down on a compromised endpoint displays the details of detected threats.
l The TIDB package contains a blacklist which is made up of IPs, domains and URLs, and a suspicious URL list (also
called Crowdsource URLs). Only suspicious URLs have a score rating in the TIDB package. Once a URL is
included in the blacklist, the suspicious score rating is no longer performed.
l Once a new TIDB package has been downloaded by FortiAnalyzer, the previous package becomes obsolete.
l The blacklist statistics by endpoint are updated in near realtime (ASAP), and suspicious rating statistics by
endpoint are updated on a half-hour schedule.
l The IOC inspection is performed on a daily cycle because the updated FortiGuard TIDB package is received daily.
At the end of the day, the IOC endpoint summary is fixed and will not receive additional changes, and a new
summary will be created for the next day.
l Currently, only FortiGate Web Filter, DNS, and traffic logs are inspected.
l The IOC module requires a license. Without a license, only demo TIDB packages are loaded into the FortiAnalyzer
image, and no updated package from FortiGuard is used in the IOC function.
l When a threat is detected, FortiAnalyzer sends a notification to the FortiGate via REST API. The FortiGate can be
configured to take automatic action against detected threats.
l IOC threat detection can be performed in both realtime and rescan mode. Realtime detection monitors new
incoming logs, whereas rescan mode checks historical logs against the new blacklist once an updated TIDB
package is available. Rescan mode does not check historical logs against the suspicious list. Realtime detection is
always enabled, and IOC rescan can be enabled or disabled.
Understanding suspicious list detection
The suspicious list is crowdsourced each day by FortiGuard AI from millions of global endpoint devices. The list is
comprised of IPs, URLs, and domains that have a low reputation, usually because they are questionable websites.
The TIDB package includes threat ranking scores which FortiAnalyzer normalizes using its internal logic. When an
endpoint visits a site that matches one included in the suspicious list, the score is deposited into the “reputation
account” for that endpoint. The total normalized score is then used to determine a verdict for the endpoint. The higher
the score, the higher the confidence.
When a new TIDB package becomes available, the process to determine a verdict begins again. FortiAnalyzer
processes logs for all monitored endpoints against the new TIDB and will determine a verdict for each endpoint based
on their new normalized score.

Endpoints that visit suspicious sites on an infrequent basis are at a low risk for compromise and are not included in the
Compromised Host watch list. The FortiAnalyzer IOC engine continues to monitor these endpoints until it has enough
confidence to produce a verdict, at which point they are given the verdict Low Suspicious and are added to the watch
list.
Endpoints that regularly visit suspicious sites are at a higher risk for infection or may already be infected with zero-day
malware. These endpoints are assigned a verdict and are added to the Compromised Host watch list.
Suspicious verdicts include:
l High suspicious (high confidence)
l Medium suspicious (medium confidence)
l Low suspicious (low confidence)
In the example below, an endpoint visits multiple sites included in the suspicious list, and as a result, has its verdict
changed from Low suspicious to Medium suspicious. The data included in this example is purely hypothetical for the
purpose of illustration.
Activity time Suspicious site Ranking of Suspicious score FortiAnalyzer

stamp visited by endpoint suspicious site of endpoint IOC verdict
Time stamp 1 suspicious-url-1 60 60 Low suspicious
Time stamp 2 suspicious-ip-2 100 160 Low suspicious
Time stamp 3 suspicious-domain-3 40 200 Medium suspicious
The specific algorithm used for the decision to change the verdict of an endpoint is internal to FortiAnalyzer.
Viewing IOC licenses and TIDB package downloads
To check the license downloaded from FortiGuard in the CLI:
diagnose fmupdate dbcontract fds

FL-1KE3R16000271 [SERIAL_NO]
AccountID:
Industry:
Company:
Contract: 1
PBDS-1-99-20250104
Contract Raw Data:
Contract=PBDS-1-99-20250104:0:1:1:0
In the output, PBDS is the IOC license.
To check the IOC package in the CLI:
diagnose fmupdate fds-getobject
FAZ object version information

ObjectId Description Version Size Created Date Time
------------------------------------------------------------------------------------------

---------
...
00001000TIDB00100 ThreatIntel DB 00000.01052 34 MB 19/04/14 20:10
ext_desc:ThreatIntel DB
00001000TIDB00100 ThreatIntel DB 00000.01053 37 MB 19/04/16 04:13
<latest> ext_desc:ThreatIntel DB
...
FortiAnalyzer periodically syncs its own IOC TIDB files to the version of IOC package downloaded by fmupdate. This is
performed on a one hour schedule.
To check the license and TIDB version used by FortiAnalyzer in the CLI:
diagnose test application sqllogd 204 stats
License of post breach detection installed.

License expiration : 2025-Jan-04
TIDB version : 00000.01017-1902242107
TIDB load time : 2019-02-24 14:11:2
Configuring FortiGate to FortiAnalyzer REST API authentication
FortiGate to FortiAnalyzer REST API authentication allows the FortiAnalyzer to send IOC alerts and trigger configured
automation rules, if configured.
To configure REST API authentication:
1. Go to the Device Manager in the FortiAnalyzer.

2. Edit the FortiGate device to set the FortiGate super admin username and password.
This is the only way to configure REST API authentication prior to 6.2.
Alternatively, when configuring logging to FortiAnalyzer on FortiGate, you can go to Security Fabric > Settings and
enable Allow access to FortiGate REST API and Trust FortiAnalyzer by serial number.
Throttling IOC alerts
To avoid flooding FortiGate with event alerts, you can configure a throttle which allows only one alert to be sent within a
set period of time for the same endpoint.
The default time period is one day (1440 minutes).
To set an IOC alert throttle in the CLI:
config system log ioc

(ioc)# set
notification Disable/Enable Ioc notification.
notification-throttle Minute value for throttling the rate of IoC notifications.
(ioc)# get

notification : enable
notification-throttle: 1440
Debugging IOC notifications
Check for the FortiGate system event: IOC detected by FortiAnalyzer.

If the system event is not present, check FortiAnalyzer's OFTP debug or FortiGate's httpsd debug for the same
message.

Troubleshooting
Troubleshooting
This section contains information about troubleshooting reports and dataset queries.
l Troubleshooting report performance issues on page 49
l Troubleshooting a dataset query on page 57
l Troubleshooting an empty chart on page 59

Troubleshooting report performance issues
The following topics provide guidance when troubleshooting report performance issue:
l Check the report diagnostic log on page 49
l Check hardware and software status on page 52
l Check data policy and log storage policy on page 53
l Check report and chart settings on page 53
l Check and adjust report auto-cache daemon on page 54
l Check and adjust report hcache on page 55
l Report performance troubleshooting commands on page 56
Check the report diagnostic log
For reports that take a long time to run, check the report diagnostic log to troubleshoot performance issues.
To retrieve a report diagnostic log, go to Reports > Generated Report, right-click the report and select Retrieve
Diagnostic to download the log to your computer. Use a text editor to open the log and check the log for possible causes
of performance issues.
Following are parts of a sample report diagnostic log and what to look for when troubleshooting report performance.
NAME SCHEDULED AUTO-CACHE REPORT GROUP REPORT TITLE
==================================================================================
1 V V - Security Analysis
per-device option: disable

hostname-resolve: disable
Report Status
Max pending rpts: 100000
Current pendings: 0
Max running rpts: 10
Current runnings: 2
Section What to look for
NAME / SCHEDULED / Check the SCHEDULED, AUTO-CACHE, and REPORT GROUP columns.
AUTO-CACHE / REPORT l Schedule the reports that run regularly. To configure report schedules, see
GROUP / REPORT TITLE Scheduling reports in the FortiAnalyzer Administration Guide.
l Enable auto-cache for reports that run regularly, especially schedule reports. See
How auto-cache works and Enabling auto-cache in the FortiAnalyzer
Administration Guide.
l Group reports that run regularly. To group reports, see Grouping reports in the
FortiAnalyzer Administration Guide.


hostname-resolve Ensure hostname-resolve is set to disable. Resolving hostnames usually takes
a long time. If the DNS server is slow or does not support reverse DNS, report
generation might hang.
Total Quota Summary:

Total Quota Allocated Available Allocate%
27201.3GB 1024.0GB 26177.3GB 3.8 %
System Storage Summary:

Total Used Available Use%
27501.3GB 1117.6GB 26383.6GB 4.1 %
------------------------------------------
System Performance
Fri Aug 25 12:00:02 2017
------------------------------------------
CPU
Used: 34.4%
Used(Excluded NICE): 34.4%
Memory
Total: 34939888 KB
Used 23899636 KB 68.4%
Hard Disk
Total: 28837161872 KB
Used: 11171927688 KB 38.7%
IoStat:
Log Rate
logs/sec: 20326.8, logs/30sec: 20395.6, logs/60sec: 20274.2
Message Rate
msgs/sec: 3057.4, msgs/30sec: 3068.1, msgs/60sec: 3039.1
Total Quota Summary l Ensure there is enough disk quota and disk space for logging and reporting.
and System Storage Insufficient disk quota might affect report accuracy.
Summary Disk quota must be big enough so that quota enforcement does not affect logs
used for reporting. If quota enforcement trims the logs or tables used for the
reporting period, there might be empty charts or incorrect data.
System Performance l Check that there is enough system resources including CPU, memory, and disk
space.
l Check that the log rate and message rate is not so high that it slow report
generation.
l If the log rate is higher than the sustained rates for your FortiAnalyzermodel, the
hardware is overloaded and needs an upgrade. The sustained rates for
FortiAnalyzermodels are listed in the Data Sheet on the FortiAnalyzer web page.

------------------------------------------
Run Report
Fri Aug 25 12:00:03 2017
------------------------------------------
[12:00:03] Request hcaches for 9 log tables
chart Traffic-Bandwidth-Summary-Day-Of-Month done, 1 subqrys
1/1 took 17.88s, 0 hcaches ready, 2 hcaches requested
overall time used 18.13s
chart Session-Summary-Day-Of-Month done, 1 subqrys
chart Traffic-History-By-Active-User done, 1 subqrys
chart Top-Attack-Victim done, 1 subqrys
chart Top-Attack-Source done, 1 subqrys
chart Top-Attacks-Detected done, 1 subqrys
…
…
…
chart System-Summary-By-Severity done, 1 subqrys
chart System-Critical-Severity-Events done, 1 subqrys
chart System-High-Severity-Events done, 1 subqrys
Run Report l Check the number of log tables.

l Check the number of hcaches requested vs ready.
If many hcaches are not ready, then those charts will take a long time.
If the number of log tables is high but the number of hcaches ready is low, retrieve
the diagnostic log after five minutes. A change in the number of hcaches ready
means the report is still running.
Since the diagnostic log is updated every five minutes, you can check this log to
view reporting progress.
l Check which charts take a long time to generate and reconfigure those charts to
improve performance.
------------------------------------------
Report Summary
Fri Aug 25 12:00:56 2017
------------------------------------------
Number of charts: 58
Number of tables: 9

Number of hcaches requested: 109
HCACHE building time: 53.32s

Rendering time: 13.33s
Total time: 1m7.67s
Report Summary l Check the number of hcaches requested, hcache building time, and rendering
time.
The number of hcaches requested = number of charts per report *
number of master tables * number of reports.
Check hardware and software status
get system status
This command shows the system status such as platform type (hardware or VM), firmware version, system time, disk
usage, and file system format.
Use this information to check if the hardware is overloaded. This information also helps you and customer support to
quickly identify any issues and narrow down the investigation.
Following is a sample result of running this command.
Platform Type : FAZ3500E
Platform Full Name : FortiAnalyzer-3500E
Version : v5.4.3-build1187 170517 (GA)
Serial Number : FL99999999999999
BIOS version : 00010001
System Part-Number : P15168-01
Hostname : SAMPLEFZ350
Max Number of Admin Domains : 4000
Admin Domain Configuration : Disabled
FIPS Mode : Disabled
Branch Point : 738
Release Version Information : GA
Current Time : Tue May 23 10:22:53 PST 2017
Daylight Time Saving : Yes
Time Zone : (GMT-8:00) Pacific Time (US & Canada).
x86-64 Applications : Yes
Disk Usage : Free 17020.10GB, Total 40314.71GB
File System : Ext4
Line Notes
Current Time This is the SQL insert start time.
File System Ensure the file system is Ext4. Other file systems will likely cause performance
issues.

What to look for:
l Check the hardware Platform Type. Consider upgrading older hardware, especially older hardware running
newer software such as 5.2 or later.
l Version shows the software version. Ensure you are running the latest software version with the newest report
engine.
l Ensure File System is Ext4. Other file systems will likely cause performance issues.
diagnose fortilogd lograte
This command shows the log receive rate.

logs/sec: 121091.0, logs/30sec: 119613.9, logs/60sec: 116695
What to look for
l If the log rate is higher than the sustained rates for your FortiAnalyzer model, the hardware is overloaded and
needs an upgrade. The sustained rates for FortiAnalyzer models are listed in the Data Sheet on the FortiAnalyzer
web page.
Check data policy and log storage policy
Check that the data policy and log storage policy are configured properly for each ADOM in each FortiAnalyzer unit. The
data policy specifies how long to keep logs. The log storage policy affects logs and the SQL database. For details, see
the FortiAnalyzer Administration Guide.
Check report and chart settings
Resolving hostnames usually takes a long time. If the DNS server is slow or does not support reverse DNS, report
generation might hang. Check that Resolve Hostname is disabled:
l In Reports Settings tab > Advanced Settings, check that Resolve Hostname is not selected.
l In the Chart Library, check that Resolve Hostname is set to Disabled.
If you do not need to show all results, specify a lower maximum number of entries:
l In the Chart Library, check that the chart's Show Top (0 for all results) is not set too high.
Setting this field to 0 for all results causes FortiAnalyzer to list all logs for the chart.

Check and adjust report auto-cache daemon
get system performance
This command shows system performance statistics such as CPU, memory, and I/O usage.
CPU:
Used: 49.51%
Used(Excluded NICE): 49.51%
%used %user %nice %sys %idle %iowait %irq %softirq
CPU0 27.89 20.60 0.00 5.40 96.42 0.80 0.00 1.79
CPU1 21.62 12.61 0.00 8.20 98.38 0.40 0.00 0.40
Memory:
Total: 6,134,200 KB
Used: 3,770,260 KB 61.5%
Hard Disk:
Total: 82,434,736 KB
Used: 65,283,648 KB 79.2%
IOStat: tps r_tps w_tps r_kB/s w_kB/s queue wait_ms svc_ms %util sampling_sec
4.7 0.2 4.4 27.5 144.2 0.2 52.5 8.4 3.9 599578.78
Flash Disk:
Total: 499,656 KB
Used: 314,416 KB 62.9%
IOStat: tps r_tps w_tps r_kB/s w_kB/s queue wait_ms svc_ms %util sampling_sec
0.0 0.0 0.0 0.0 0.0 0.0 13.6 4.6 0.0 599578.78
Following is a sample result of high %iowait. To see the iowait usage and limit, first enable debug messages for
SQL commands (diagnose debug enable) and set the debug level (diagnose debug application
sqlrptcached 8).
FAZVM64 # [530] iowait usage (27.5%) is over limit (23%).
[530] iowait usage (25.9%) is over limit (23%).
[530] iowait usage (28.3%) is over limit (23%).
What to look for
l Check the Used and IOStat lines to see if I/O is busy.

l If both CPU %used and %iowait are high, check if the report cache daemon is running:
diagnose debug enable
diagnose debug application sqlrptcached 8
l If iowait is over the limit, cache building (by sqlrptcached) will be paused until iowait drops below the limit.
In this case, do one or both of the following:
l Change the report schedule to run at a less busy time. To see scheduled reports, run execute
sql-report list-schedule <ADOM>. To configure report schedules, see Scheduling reports in the
l Enable aggressive-schedule so the report auto-cache daemon does not stop even under heavy system
load:
config system report auto-cache
set aggressive-schedule enable
end

Check and adjust report hcache
diagnose test application sqlrptcached 2
This command shows if hcache creation is able to catch up.

Number of log table read: all=6453(fortiview=0, rpt=6453) pending=1
Number of log table done: all=6453(fortiview=0, rpt=6453) rpt=6453
Current hcache table entries: 155750
Number of hcache requests sent: 70999
Number of log table vacuums: 39401, pending=2
FortiView hcache load: rounds=817, tbl=653600
ncmdb:
cache hit: sch=0, config=27, chart=140, macro=0, dataset=140 config=27
calls : sch=130, config=11, chart=23, macro=0, dataset=23
The following table provides notes about some output lines in the example.
Line Notes
Number of log table pending=0 means hcache creation is able to catch up. If pending is above 0,
read see What to look for below.
Number of log table The number of master tables used to calculate the Number of hcache
done requests sent.
Current hcache table Total hcache on the system.
entries
Number of hcache The number of charts per report * the number of master tables * the number of
requests sent reports.
Number of log table The postgres built-in status. A pending number above 0 indicates insufficient
vacuums postgres resources.
FortiView hcache load rounds is the number of FortiView caches proactively loaded into memory.
ncmdb Report configuration database.
cache hit config is the number of enabled auto cache.
What to look for
l In Number of log table read, if the pending number is continuously above 0 or is increasing, that
indicates there are too many pending log tables to read and the system lacks resources to create cache. In this
case, consider disabling auto-cache on some reports. See Enabling auto-cache and Reports Settings tab in the
l Run execute sql-report list-schedule <ADOM> and check if there are too many scheduled reports
and if auto-cache is enabled. See Scheduling reports and Enabling auto-cache in the FortiAnalyzer
Administration Guide.
l Run execute top to check which applications are using the most system resources.

execute sql-report hcache-check <ADOM> <schedule-id>
This command shows a specific report’s hcache status.

If necessary, check the hcache status of a specific report that you think might be a problem.
For example, if the ADOM is root and schedule-id is 10004, then run execute sql-report hcache-check
root 10004.
To get the schedule-id, run execute sql-report list-schedule root and see the NAME column.
Following is a sample result of running the execute sql-report hcache-check <ADOM> <schedule-id>
command.
layout_num:1
start [0] get layout-id:10004.
start report_process, layout-id:10004, layout title:Admin and System Events Report.
device list:All_FortiGates.
reports num:1.
device list[0].FWF60C3G13006291[root].
device list[1].FG3K2C3Z11800039[root].
......
> checking (10004_t10004-Admin and System Events Report) ...

checking chart Admin-Login-Summary...
8/8 (100%) done 0.131 secs used.
checking chart Admin-Login-Summary-By-Date...
8/8 (100%) done 0.128 secs used.
...
What to look for
l If a few reports are causing a bottleneck, check those reports’ Check the report diagnostic log on page 49 and
consider reconfiguring those reports. See also Check and adjust report auto-cache daemon on page 54.
Report performance troubleshooting commands
CLI Description
diagnose debug application Set the debug level of the SQL report cache daemon.
sqlrptcached 8
diagnose debug crashlog Print information of all crashed daemons.
read If daemons crash frequently, contact customer support for assistance.
diagnose debug disable Disable debug message.
diagnose debug enable Enable debug messages to run SQL diagnostic commands.

CLI Description
diagnose fortilogd lograte Show the log receive rate.
diagnose fortilogd msgrate Show message receive rate. One message might contain multiple logs.
diagnose log device Show disk quota for all logging devices.
diagnose report status Show the maximum number of pending and running reports, and the
current number of pending and running reports.
diagnose test application Show if hcache creation is able to catch up.
sqlrptcached 2
diagnose sql show Show the hcache size.
hcache-size
diagnose sql status List the number of log tables, hcaches, and the time to generate each chart
run-sql-rpt in the report.
diagnose sql status Show SQL query connections and hcache status.
sqlreportd
execute sql-report hcache- Show a specific report’s hcache status.
check <ADOM> <schedule-id>
execuate sql-report Show a summary table of all configured reports with their configuration
list-schedule <ADOM> status.
execuate top List the processes running on the FortiAnalyzer system.
get system performance Show system performance statistics such as CPU, memory, and I/O usage.
get system status Show the system status such as platform type (hardware or VM), firmware
version, system time, disk usage, and file system format.
Use this information to check if the hardware is overloaded. This
information also helps you and customer support to quickly identify any
issues and narrow down the investigation.
l Ensure Version is the latest software version.
l Check the hardware Platform Type. Consider upgrading older
hardware, especially older hardware running newer software such as
5.2 or later.
l Ensure File System is Ext4. Other file systems will likely cause
performance issues.
show system report Show non-default settings in the report auto-cache.
auto-cache Ensure auto-cache is enabled by running these commands:
config system report auto-cache
set status enable
end
Troubleshooting a dataset query
The following topics provide guidance when troubleshooting a dataset query:

l Troubleshooting a custom dataset on page 58

l SQL functions for formatting and converting data types on page 58
l Macros for formatting date and time in a dataset on page 59
Troubleshooting a custom dataset
This topic provides a list and an example of common issues in a custom dataset that cannot be identified by the dataset
test console.
Common issues:
l $filter is not applied.

l No ### for inner query.
l distinct is used in inner query.
l No column alias for column with function.
l no hcache merge for count distinct.
l No group by or order by.
l Log tables are not joined. For example, join traffic log with IPS log.
l Dataset test console is out of memory.
The image below indicates where common issues may appear in the dataset:
SQL functions for formatting and converting data types
The following SQL functions can be used to format or convert different data types:

SQL function Description

from_itime / from_dtime Converts timestamp to formatted date/time.
ipstr Converts srcip/dstip field from inet to string.
app_group_name Groups similar application names.
root_domain Groups similar hostnames.
vpn_trim Groups similar VPN tunnels.

nullifna Converts N/A to null.
logid_to_int Trims logid.
Macros for formatting date and time in a dataset
The following macros can be used to fine tune date and time formatting in a dataset:
Macros Description Example
$flex_timescale Time scale changes according to the report time period:

l Time period > 28 days l Display day: 2018-02-25
l Time period > 12 hours and <= 28 days l Display hour: 2018-02-25
l Time period > 4 hours and <= 12 hours 14:00
l Time period > 1 hour l Display 30 min granularity:
l <= hour 2018-02-25 14:30
l Display 5 min granularity:
2018-02-25 14:40
l display 1 min granularity:
2018-02-25 14:42
$hour_of_day Displays hour in 24 hr format. 18:00
$HOUR_OF_DAY Displays date (YYYY-MM-DD) and hour in 24 hr format. 2018-01-13 18:00
$DAY_OF_MONTH Displays month in format YYYY-MM-DD (2017-01-10). 2018-01-01
$day_of_month Displays day of the month in two digits format 01-12. 01
$day_of_week Displays number and name of the day of the week Mon
(WDAY 2-Mon).
Troubleshooting an empty chart
To troubleshoot an empty chart in a report, go to Log View to verify logs are incoming.
l If you see logs check for SQL errors.
l If you don't see any logs the daemon may have stopped working.

Common issues
The following table provides a list of common issues that may produce an empty chart in a report:
Issue Description
Wrong report filter applied Go to Log View and search for:

l Field “status” changed to “action” (since 5.0.6)
l Data type of srcip and dstip changed from string to inet.
Log field changed after This can be identified by a dataset test console or SQL debug.
upgrade
Hcache corrupt Clear hcache before running the report (dia sql remove hcache).
Log traffic l High log rate (diagnose fortilogd lograte)

l Device or ADOM quota reached (diagnose log device)
“logver” issue Some datasets are using field “logver” to identify FOS log version.
Go to Log View and search for logver=*
If there are no records, you may need to upgrade.
"out of memory” File system error. This occurs mostly in 5.2.

CLI commands for troubleshooting
The following table provides a list of CLI commands to troubleshoot an empty chart in a report:
Command Description
Check report diagnose report status {running | pending}

running/pending status
Debug sql query diagnose debug enable

diagnose debug application sqlplugind 4 -----errors only
diagnose debug application sqlplugind 8
List current SQL process diagnose sql process list
Configure global report config system report auto-cache

automatic cache setting
List report schedule/auto- execute sql-report list-schedule <ADOM-name>

cache status by ADOM
Diagnose report hcache diagnose test application sqlrptcached 2

working status
Check individual report execute sql-report hcache-check <ADOM-name> <schedule-

hcache status id>
Check report status during diagnose debug enable

report running diagnose sql status sqlreportd

Copyright© 2020 Fortinet, Inc. All rights reserved. Fortinet®, FortiGate®, FortiCare® and FortiGuard®, and certain other marks are registered trademarks of Fortinet, Inc., in
the U.S. and other jurisdictions, and other Fortinet names herein may also be registered and/or common law trademarks of Fortinet. All other product or company names may be
trademarks of their respective owners. Performance and other metrics contained herein were attained in internal lab tests under ideal conditions, and actual performance and
other results may vary. Network variables, different network environments and other conditions may affect performance results. Nothing herein represents any binding
commitment by Fortinet, and Fortinet disclaims all warranties, whether express or implied, except to the extent Fortinet enters a binding written contract, signed by Fortinet’s
General Counsel, with a purchaser that expressly warrants that the identified product will perform according to certain expressly-identified performance metrics and, in such
event, only the specific performance metrics expressly identified in such binding written contract shall be binding on Fortinet. For absolute clarity, any such warranty will be
limited to performance in the same ideal conditions as in Fortinet’s internal lab tests. In no event does Fortinet make any commitment related to future deliverables, features or
development, and circumstances may change such that any forward-looking statements herein are not accurate. Fortinet disclaims in full any covenants, representations, and
guarantees pursuant hereto, whether express or implied. Fortinet reserves the right to change, modify, transfer, or otherwise revise this publication without notice, and the most
current version of the publication shall be applicable.

FortiAnalyzer 6.2 Cookbook

Uploaded by

Copyright:

Available Formats

FortiAnalyzer 6.2 Cookbook

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

FortiAnalyzer 6.2 Cookbook

Uploaded by

Copyright:

Available Formats

FortiAnalyzer - Cookbook

FORTINET VIDEO GUIDE

CUSTOMER SERVICE & SUPPORT

FORTINET TRAINING & CERTIFICATION PROGRAM

END USER LICENSE AGREEMENT

September 11, 2020

FortiAnalyzer 6.2 Cookbook 3

FortiAnalyzer 6.2 Cookbook 4

Date Change Description

2019-11-21 Initial release.

2019-11-21 Added the following sections:

2019-11-26 Added Integrating FortiAnalyzer with ServiceNow on page 30.

FortiAnalyzer 6.2 Cookbook 5

FortiAnalyzer System Setup

This section contains information about FortiAnalyzer system setup.

FortiClient user avatar

FortiClient Version Where FortiClient Telemetry Connects Location of Logs in FortiAnalyzer

FortiClient 6.0.x and FortiGate Under the FortiGate device as a sub-type of

FortiClient EMS In a FortiClient ADOM

Enabling logging from FortiClient to FortiAnalyzer

To enable logging from FortiClient to FortiAnalyzer:

1. Install FortiClient on endpoints.

FortiAnalyzer 6.2 Cookbook 6

3. In FortiClient, connect Telemetry to FortiClient EMS.

FortiAnalyzer 6.2 Cookbook 7

5. In FortiClient, go to Settings to view the settings in the Logging section.

FortiAnalyzer 6.2 Cookbook 8

Setting up a FortiAnalyzer HA cluster

All the units must:

To configure the primary unit in an HA cluster:

1. Go to System Settings > HA.

Interface Select the interface to be used as the clustered Virtual IP.

IP Address Type the IP address to be used by the HA cluster to provide redundancy.

To configure secondary units in an HA cluster:

1. Set the Preferred Role to Slave.

FortiAnalyzer 6.2 Cookbook 9

FortiAnalyzer 6.2 Cookbook 10

Priority The priority or seniority of the backup unit in the cluster.

FortiAnalyzer 6.2 Cookbook 11

Manage logs and data sources

Fetching logs from one FortiAnalyzer to another

Creating an admin on the remote FortiAnalyzer

To create a new admin:

1. Go to System Settings > Administrators > Create New.

FortiAnalyzer 6.2 Cookbook 12

Creating a fetch profile on the local FortiAnalyzer

To create a new log fetch profile:

1. Go to System Settings > Fetcher Management > Profiles.

Name Enter a name for the profile.

Server IP Enter the IP address of the remote FortiAnalyzer.

FortiAnalyzer 6.2 Cookbook 13

Requesting a log fetch

To request a log fetch:

Server IP Displays the IP address of the server you have specified.

FortiAnalyzer 6.2 Cookbook 14

3. (Optional) Synchronize the devices and ADOMs.

Reviewing and approving fetch requests

To review and approve a fetch request:

FortiAnalyzer 6.2 Cookbook 15

3. Click Review, then Accept the request to transfer the logs.