Enterprise SIEM, Centralized Log Management, Security, Compliance, Server Monitoring and Uptime Monitoring Software
Table of Contents

Application and Text Log Consolidation Template

Application and text log consolidation is the process of saving text-based log file entries to a Data Provider, also known as a Log Database. Server Manager supports text log files, CSV files and W3C files (e.g. IIS logs).

Text logs can be consolidated using several API technologies.

APIDescription
Windows SharesMonitor text-based logs on Windows servers.
SFTP/SSHMonitor text-based files on any Unix flavor.
FTP/SMonitor text-based logs on any remote endpoint such as a hosted webserver.

How to Configure Text Log Consolidation:

  • From the Menu Bar, select File | New. The Create New Object View displays.
  • From the Create New Object view, expand Template | Log Management then select Log Consolidation. The New Log Consolidation Template Properties view displays.
  • From the New Log Consolidation Template Properties view, use the Sub type drop-down to select Text Log. The Template Properties view now contains 7 tabs.

Read Options

The Read Options tab enables you to configure how to read the log file.

  • Use the Log drop-down to select the target log or search criteria to configure. Select (All) to configure all files and search criteria at the same time.
  • Use the Logical filename text box when monitoring files that contain either dates or instance numbers within the log file name. When set, each unique file is saved to the same database table, otherwise each dated file or instance file will be added to its own database table.
  • Use the Enable entry pattern recognition check box to configure a pattern that defines each unique log entry.
    OptionDescription
    Entries start withSpecify a unique character such as <.
    Entries start with date or timeSpecify a date or time mask such as yyyy-MM-dd.
    Entries end withSpecify a unique character such as >.
    Entries end with (No CR, LF or CRLF)Specify a null terminating character such as |.
  • Use the Read method drop-down to select where to begin reading the file.
    OptionDescription
    Beginning of FileEach time a file is newly discovered, the entire file is read. The next time the file is read, it is read from the last known position.
    End of FileEach time a file is newly discovered, the file is opened then the read position moved to the end and closed. The next time the file is read, it is read from the last known position.
    Read AllEach a time a file is read, the entire file is read.
  • use the Open as Unicode option if the file is written in Unicode format otherwise the file is opened as UTF-8.
  • To test the entry pattern, use the Test host, device or endpoint drop-down to select the target host then click the Test button. The software applies your rules to the file then reads the first 10 entries. Update your entry pattern rules until you get the results you desire.

Rules

The Rules tab enables you to configure the consolidation filter and Log Entry Retention Policy to enforce.

  • Use the Log drop-down to select the target log or search criteria to configure. Select (All) to configure all files and search criteria at the same time.
  • Optionally assign a Consolidation filter to dump entries you do not want saved to the Log Database. When assigned, only entries that pass the assigned consolidation filter are saved to the Log Database.
  • Use the Log Entry Retention Policy drop-down to select the retention policy. The retention policy is another template that defines the number of days to retain in the Primary and Archive Log Databases, for example, archive entries older than 30 days and retain entries for 150 days for a total of 180 days. Assign multiple retention policies to remove entries that match filter criteria defined in each retention policy. For more information see: Log Entry Retention Policy Template

Monitor

The Monitor tab enables you to configure various monitors to apply when consolidating the log entries.

  • Use the Log drop-down to select the target log or search criteria to configure. Select (All) to configure all files and search criteria at the same time.
  • Use the File size exceeds controls to monitor the file size (e.g. The file exceeds 10 GBs).
  • Use the File size changes by controls to monitor changes in file size between scans (e.g. The file increases by 1 MB between scans indicating an ongoing attack).
  • Use the File has not been written to for controls to receive notifications when a file has been idle for a period of time (e.g. The file has not been written to indicating a webserver is down).
  • Use the Rules controls to configure log entry monitor rules. For more information see: Log Monitor

Related Topics

Data Provider

Log Consolidation Templates

Log Entry Retention Policy Template

Data Providers

Log Monitor