Log Management Overview
Log management encompasses the processes of log archiving and analysis that enable an organization to collect and consolidate audit logs from operating systems, application and network devices. Logs of security events typically are collected, consolidated and stored, normalized and preserved (archived) as a record of system activity. In many cases, logs are analyzed to identify incident trends, for audit and review purposes, or used in a forensic investigation of a specific security incident.
Proper log management is now critical to organizations affected by laws and regulations that require the collection and archiving of event logs for the purpose of annual—or more frequent—audit and control reviews. These reviews are designed to uncover violations of organizational policy or separation of duties, as well as breaches in the use of private or confidential customer or patient information.
This paper begins with a discussion of the benefits and challenges of implementing log management. Some popular regulations are then discussed that require organizations to implement the ability to collect logs, as well as perform review and analysis upon those log data. Finally NetIQ Security Manager will be introduced, and it will be shown how the product can help organizations comply with regulatory log management requirements.
Benefits of Log Management
With the aid of automated tools—such as those included in NetIQ Security Manager —organizations are able to implement log management to efficiently collect and consolidate events over time, analyze those events to minimize vulnerabilities and prevent future breaches, to expedite forensic investigations into security incidents, and to improve an effective response to security incidents. Taking advantage of log management capabilities available today, organizations can:
- Comply with regulations, standards and policies by archiving and protecting log files and security event data.
- Normalize log and security events to enable effective comparisons of events on similar systems or between dissimilar systems.
- Manage and make sense of large volumes of log files by employing security trend analysis and summary log reports, and gaining a better understanding of event trends over days, weeks, months or even years.
- Preserve log files and security event data for forensic investigations by removing them from hosts and sensor devices into a system that completely manages security data.
- Speed up incident response by enabling queries of log data and security events across numerous hosts, applications, sensors and other technologies at a single time.
- Enhance query, reporting, filtering and summarization capabilities beyond native log viewers such as the Windows Event Viewer or Check Point’s Log Viewer.
Log Management Challenges
While log files contain the record of what has happened within a particular environment over time, the challenge for security professionals comes in being able to collect, and then to actually manage the sheer volume of information available. Nearly every business technology today is capable of creating a log file—a record of transactions or events. These include: 1) operating systems such as Windows, Unix and i5/OS which create and maintain log files of system events such as logons, resource accesses and services starting and stopping; 2) Web servers such as Internet Information Server and Apache which maintain a record of URLs and files accessed and other events; 3) Database management systems such as Oracle and SQL Server which maintain a record of schema changes and other important events.
Moreover, network devices and security systems often maintain records of significant (and even not-so-significant) events. Firewalls record information on sessions and other traffic information such as protocols used, number of packets routed, rules matched, as well as, accepts, rejects and drops. Finally, intrusion detection systems maintain a record of attacks, intrusions and, sometimes, policy violations.
The result is vast amounts of data on security events that must be collected, consolidated and stored, normalized, and preserved (archived) as a record of system activity. The log files must then be managed so that they can be analyzed to: identify incident trends; satisfy compliance audits and reviews; or verify activity that occurred during a specific security incident through forensic investigation. This sheer volume of data, along with the human politics that can hinder gaining access to the logs, can both present significant challenges to organizations trying to meet log management requirements.
