member sign-in
Forgot password? Create new account Close

Database Activity Monitoring / Database Firewall

Definition

Database activity monitoring (DAM) / Database Firewall (DBF) monitors database activity to identify fraudulent, illegal or other undesirable behavior, by using  embedded knowledge about database structures and access to analytics and reporting and enforce policies and control. The DAM/DBF solutions operates independently of the database management system (DBMS) audit functionality of the database itself. The DAM/DBF can be regarded to either as an alternative to the DBMS functionality (due to heavy overload on the database servers), either as complementary control to it.

DAM solutions contain also database vulnerability assessment and user account audit, coupled with firewall file access monitoring and web application monitoring.

User Benefits

The user benefits can be cuantified as:

1. Monitoring

  • Privileged users monitoring – DBAs, root, system admins – which have access to access and alter data either via the application either by logging in at the system OS or local console. Their access has to be monitored in order to prevent privileged users from accessing data, making modifications to schema or table structure, or creating or modifying user accounts or permissions
  • User activity monitoring in order to track the users and the applications that connect to the database. Beside fraud access, an important aspect is also to monitor and eventually prevent also malicious or unintended activity of the legitimate users.
  • If possible, also the user accounts have to be constantly monitored in order to detect the dormant user accounts, and take appropriate action.

2. Risk and Compliance – the risk and security teams are seeking to implement tight controls around the data stores in order ensure data confidentiality and integrity while limiting access to privileged users and subsequently identifying fraudulent activities. The preventive security solutions and controls such as encryption and access management, are not effective for authorized / legitimate user access.  Thus, the DAM solution can be successfully deployed in order to fulfill the security controls required by:

  • Data Governance
  • Risk Management
  • Audit
  • Regulatory Compliance.

The benefits provided by the DAM solution, via a tight integration with DLP and SIEM solutions respectively, allow the extension of the network controls and the security framework also to the databases and data stores.  

3. Policy Enforcement
Database Firewall includes a complete set of predefined, customizable security and audit policies. Security alerts can be sent to SIEM, ticketing systems, and other third-party solutions to streamline business processes.

Business Impact

The  solutions can be deployed offpath either inline. In order to comprehensively monitor and detect fraudulent activity, a solution must monitor all the “gateways” to the data store. Thus, the employment of software agents in order to monitor also the local activity (server console or other applications connecting to the database) is to be taken into consideration.

If the solutions are deployed offpath, then there is no impact on the monitored network segments. 

If the solutions are to be deployed inline in protect mode (database firewall), then several considerations are to be taken into account:

  • Usually the solution is deployed in transparent mode, with no IP addressing on the traffic interfaces. Thus, the DAM should have fail-open functionalities in order to allow the traffic to pass through in case of platform malfunction
  • Minimum latency – the latency induced in the network has to be minimum
  • The enforcement of the security policies is to be done in stages – first deployed non-intrusively, and upon successful testing only the security policies are to be enforced in place.
  • Proper performance dimensioning in order to withstand peak traffic, in terms of both legitimate and malicious traffic; if the device cannot operate properly under heavy load this will have a direct impact on the business process
  • Access policies have to be reviewed each time a modification has to be operated to the application, at both the application and at user access policies; failing to do this can result in blocking legitimate traffic and/or blocking legitimate users’ access. 
  • Data audit – when turned on, it consumes very severely the resources – thus, a powerful enough hardware appliance has to be deployed.

Products supporting this technology

Application Security Imperva McAfee

A Key Technology For Security And Compliance

Over the past five years major changes are made, in both the threats and the regulatory compliance landscape. Both the bad guys and the regulators are now focused on our data, not just our networks. Breach disclosures and the regulations meant to protect them are growing every year, with no end in sight. But managing this risk is more complicated than simply dropping in firewall or installing antivirus software. Applications and databases run in complex environments with numerous dependencies and business requirements. Companies want to protect information, and also need to do it in a way that doesn’t materially interfere with the business. To balance of these needs make new technologies to arise, one of the most significant of which is Database Activity Monitoring/Database Firewall (DAM/DBF). DAM/DBF tools provide powerful, immediate, non-intrusive benefits for security and compliance, and a long-term platform for comprehensive protection of databases and applications.

Database Activity Monitors capture and record, at a minimum, all Structured Query Language (SQL) activity in real time or near real time, including database administrator activity, across multiple database platforms; and can generate alerts on policy violations. While a number of tools can monitor various level of database activity, Database Activity Monitors are distinguished by five features:

  • The ability to independently monitor and audit all database activity, including administrator activity and SELECT transactions. Tools can record all SQL transactions: DML, DDL, DCL, (and sometimes TCL) activity
  • The ability to store this activity securely outside the database.
  • The ability to aggregate and correlate activity from multiple heterogeneous Database Management Systems (DBMSs). Tools can work with multiple DBMSs (e.g., Oracle, Microsoft, IBM) and normalize transactions from different DBMSs despite differences between SQL flavors.
  • The ability to enforce separation of duties on database administrators. Auditing must include monitoring of DBA activity, and solutions should prevent DBA manipulation or tampering with logs or recorded activity.
  • The ability to generate alerts on policy violations. Tools don't just record activity, they provide real-time monitoring and rule-based alerting. For example, can create a rule that generates an alert every time a DBA performs a select query on a credit card column which returns more than 5 results.

If, the policies enforcement is added at the previous characteristics, the product is as Database Firewall.

Other tools provide some level of database monitoring, including Security Information and Event Management (SIEM), log management, and database management, but DAM/DBF products are distinguished by their ability to capture and parse all SQL in real time or near real time and monitor DBA activity.
Depending on the underlying platform, a key benefit of most DAM/DBF tools is the ability to perform this auditing without relying on local database logging, which often entails a substantial performance cost. All the major tools also offer other features beyond simple monitoring and alerting, ranging from vulnerability assessment to change management.

DAM/DBF tools are extremely flexible and often deployed for what may appear to be totally unrelated reasons. Deployments are typically prompted by one of three drivers:

  • Auditing for compliance. One of the biggest boosts to the DAM/DBF market has been increasing auditor requirements to record database activity for SOX (Sarbanes-Oxley) compliance. Some enterprises are required to record all database activity for SOX, and DAM/DBF tools can do this with less overhead than alternatives
  • As a control for compliance. We are seeing greater use of DAM/DBF tools to address specific compliance requirements, even though database auditing itself isn't the specified control. The most common example is using DAM/DBF as an alternative to encrypting credit card numbers for PCI compliance.
  • As a security control. DAM/DBF tools offer significant security benefits and can be deployed in a blocking mode. They are particularly helpful in detecting and preventing data breaches for web facing databases and applications, or to protect sensitive internal databases through detection of unusual activity.
  • DAM/DBF tools are also beginning to expand into other areas of database and application security, as we'll see a bit later. Today, SOX compliance is the single biggest market driver, followed by PCI. Despite impressive capabilities, internally- driven security initiatives motivate a distant third of DAM/DBF deployments as most database security projects seem to be driven by compliance needs.

Since Database Activity Monitoring/Database Firewall are so versatile, here are a few examples of how it can be used:

  • To enforce separation of duties on database administrators for SOX compliance by monitoring all their activity and generating SOX-specific reports for audits.
  • If an application typically queries a database for credit card numbers, a DAM/DBF tool can generate an alert if the application requests more card numbers than a defined threshold (often a threshold of "1"). This can indicate that the application has been compromised via SQL injection or some other attack. Using DBF the attack can be stopped.
  • To ensure that a service account only accesses a database from a defined source IP, and only runs a narrow group of authorized queries. This can alert on compromise of a service account either from the system that normally uses it, or if the account credentials show up in a connection from an unexpected system.
  • For PCI compliance some organizations encrypt the database files or media where they're stored, and also use DAM/DBF to audit, alert and stop the access to the credit card field. The encryption protects against physical theft, while the DAM/DBF protects against insider abuse and certain forms of external attack.
  • As a change and configuration management tool. Some DAM/DBF tools offer closed-loop integration with external change management tools to track approved database changes implemented in SQL. Other tools can then track administrator activity and provide change management reports for manual reconciliation

One of the key strengths of DAM/DBF is its ability to monitor multiple databases running on multiple database management systems (DBMSs) across multiple platforms (Windows vs. Unix vs. ...). The DAM/DBF tool aggregates collected information from multiple collectors to a central, secure server. In some cases the central server/management console also collects information while in other cases it serves merely as a repository for collectors to send data.
This creates three potential options for deployment, depending on organizational requirements:

  • Single Server/Appliance: A single server or appliance serves as both the sensor/collection point and management console.
  • Two-tier Architecture: This option consists of a central management server and remote collection points/sensors. The central server does no direct monitoring and just aggregates information from remote systems, manages policies, and generates alerts. The remote collectors may use any of the collection techniques, and feed data back to the central server.
  • Hierarchical Architecture: Collection points/sensors aggregate to business- level or geographically distributed management servers, which in turn report to an enterprise wide management server. Hierarchical deployments are best suited for large enterprises which may have different business unit or geographic needs. They can also be configured to only pass certain kinds of data between the tiers to manage large volumes of information or maintain unit/geographic privacy and policy needs.

Whatever deployment architecture is chose, the central server aggregates all collected data (except deliberately excluded data), performs policy based alerting, and manages reporting and workflow.
These options focus on typical DAM/DBF deployments for database monitoring and alerting; as we delve into the technology we'll see additional deployment options for more advanced features like blocking.
At the core of all DAM/DBF solutions are the collectors that monitor database traffic and either store it locally or send it to the central management server, depending on filtering rules and configuration. These collectors are, at a minimum, capable of monitoring SQL traffic. This is one of the defining characteristics of DAM/DBF and what differentiates it from log management, Security Information and Event Management, and other tools that also offer some level of database monitoring. There are three major categories of collection techniques.

  • Network Monitoring: This technique monitors network traffic for SQL, parses the SQL, and stores it in the collector’s internal database. Most tools monitor bidirectionally, but some early tools only monitored inbound SQL requests. The advantages of network monitoring are that it has zero overhead on the monitored databases, can monitor independent of platform, requires no modification to the databases, and can monitor multiple heterogenous database management systems at once. The disadvantages are that it has no knowledge of the internal state of the database and will miss any database activity that doesn't cross the network as SQL, such as logging in locally and remote console connections. For this last reason, network monitoring is only recommended when used in conjunction with another monitoring technique that can capture local activity. Network monitoring can still be used if connections to the databases are encrypted via SSL or IPSec by placing a VPN appliance in front of the  databases, and positioning the DAM/DBF collector between the VPN appliance and the database, where the traffic is unencrypted.
  • Remote Monitoring: With this technique, the collector is given administrative access to the target database and native database auditing is turned on. The collector externally monitors the DBMS and collects activity recorded by native auditing or other internal database features that output activity data. The overhead on the monitored system is thus the overhead of the native logging/auditing. In some cases this is completely acceptable — in particular, Microsoft SQL Server is designed to provide remote monitoring of the database with little or no overhead. In other cases, particularly Oracle before version 10g, the overhead is material and may not be acceptable for performance reasons. Advantages include the ability (depending on DBMS platform) to monitor all database activity including local activity, performance equal to the performance of native logging/monitoring, and monitoring of all database activity, including internal activity, regardless of client connection method. The major disadvantage is   performance issues depending on the database platform, especially older versions of Oracle. This also requires opening of an administrative account on the database and possibly configuration changes.
  • Local agent: This technique entails the installation of a software agent on the database server to collect activity. Individual agents vary widely in performance and techniques used, even within a product line, due to requirements for DBMS and host platform support. Some early agents relied on locally sniffing a network loopback interface, which misses some types of client connections. Current agents hook into the platform kernel to audit activity without modification to the DBMS and with minimal performance impact, or leverage shared memory monitoring. Leading agents typically impact performance by no more than 3-5%, which seems to be the arbitrary limit database administrators are willing to accept. Advantages include collection of all activity without turning on native auditing, ability to monitor internal database activity such as stored procedures, and potentially low overhead. Disadvantages include limited platform support (a new agent must be coded for every platform) and the requirement to install an agent on every monitored server.

Which collection technique is best? The answer is “all of them”. Different collection techniques have different advantages and disadvantages depending on circumstances. It often makes sense to use multiple techniques to meet the different needs of different databases. In some cases, network monitoring involves less database and management overhead, but it misses many types of database connections and activity. Audit log monitoring has lower overhead, but can be problematic on some platforms. Not all organizations are willing to install agents on critical database servers and current offerings aren't available on every platform. Thus a combination of techniques can allow for effective monitoring with acceptable performance impact.
The one characteristic Database Activity Monitoring/Database Firewall solutions share with log management and even Security Information and Event Management, tools is their ability to collect disparate activity logs from a variety of database management systems. Where they tend to exceed the capabilities of related technologies is their ability to not only aggregate, but to normalize and correlate events. By understanding the Structured Query Language (SQL) of each database platform, they can interpret queries and parse their meaning. A more advanced feature is to then correlate activity across different transactions and platforms, rather than just looking at single events.
Since they collect a massive amount of data, DAM/DBF tools must support automatic archiving. Archiving should support separate backups of system activity, configuration, policies, alerts, and case management.

One of the distinguishing characteristics of Database Activity Monitoring/Database Firewall tools is that they don't just collect and log activity, they analyze it in real or near-real time for policy violations and enforce controls. While still technically a detective control (we'll talk about preventative deployments later), the ability to alert and respond in practically real time offers security capabilities far beyond simple log analysis. Successful loss-bearing database attacks are rarely the result of a single malicious query — they involve a sequence of events leading to the eventual DAM/DBF age. Ideally, policies will be established to detect the activity early enough to prevent the final loss-bearing act. Even when an alert is triggered after the fact, it facilitates immediate incident response, and investigation can begin immediately, instead of after days or weeks of analysis. Policies fall into two basic categories:

  • Rule-based: Specific rules are set up and monitored for violations. They can include specific queries, result counts, administrative functions (new user creation, rights changes), signature-based SQL injection detection, UPDATE or other transactions by users of a certain level on certain tables/fields, or any other activity that can be specifically described. Advanced rules can correlate across different parts of a database or even different databases, accounting for data sensitivity based on DBMS labels or through registration in the DAM/DBF tool.
  • Heuristic: The DAM/DBF solution monitors database activity and builds a profile of "normal" activity (we sometimes call this behavioral profiling). Deviations then generate policy alerts. Heuristics are complicated and require tuning to work effectively. They are a good way to build a base policy set, especially with complex systems where manually creating deterministic rules by hand isn't realistic. Policies are then tuned over time to reduce false positives. For well-defined systems where activity is consistent, such as an application talking to a database using a limited set of queries, they are very useful.

The more mature a solution, the more likely it is to come with sets of pre-packaged policies. For example, some tools come with pre-defined policies for standard deployments of databases behind major applications, like Oracle Financials and SAP. Yes, it is necessary to tune the policies, but they are far better than starting from nothing. Built-in policies for PCI, SOX, and other generic compliance requirements may need even more tuning, but will help to start the process and save many hours of policy building.
Policies should include user/group, source/destination, and other important contextual options. They should also support advanced definitions, such as complex multi-level nesting and combinations. Ideally, the DAM/DBF solution will include policy creation tools that limit the need to write everything out in SQL or some other definition language. Sometimes, it is not possible to avoid to do some things by hand, but basic policies should be as point-and-click easy as possible.
For common kinds of policies, including detecting privileged user activity and count thresholds on sensitive data, policy wizards are extremely useful.

An emerging feature in some tools is support for content-based policies. As with Data Loss Prevention, these tools are able to analyze queries and results for specific content.
Identifying all known locations of sensitive data within multiple heterogenous database management systems is a complex process, even with the support of content discovery which we'll talk about later. Credit card and Social Security Numbers can easily be placed where they shouldn't be, either deliberately or by accident. Content based policies, typically using regular expressions, analyze database activity for unapproved use of sensitive data. For example, a policy could look for credit card numbers in any result set except those previously approved.
It's very early days, but we expect to see more and more content and context awareness in DAM/DBF tools over time. The most critical data we're usually trying to protect (at least these days) falls into structured formats we can define and detect outside its proper bounds (using data labeling and other registration techniques). Long-term, we'll be able to do really interesting things as we improve our ability to monitor and understand business context with the content, moving us ever closer to the elusive goal of stopping misuse of legitimate access.

DAM/DBF tools should support both active alerting and an incident handling queue, similar to DLP. These alerts take a few different forms, from email integration, to self-contained events, to communications with outside security tools such as SIEM, using anything from SNMP to syslog to proprietary integration.
Policies should support granular alerting based on conditions, such as thresholds. For example, detection of a single errant query might trigger a low level incident within the included incident handling system, while an incident involving an administrator or high count of credit cards is emailed to a security admin and dropped into the SIEM tool as a high alert.
This is not to say that a company should rely on a SIEM or other external tool to manage database incidents; those tools will never contain the full context and investigative abilities of a dedicated DAM/DBF workflow. External alerts play a valuable role in escalating incidents and correlating against external events, but the primary handling will tend to be managed within the DAM/DBF tool itself. Databases are complex beasts, and full understanding of what's going on internally requires a dedicated tool.

Policy-based alerts tend to fall into two or three interrelated categories which often overlap:

  • User activity: Incidents where a user takes an action that violates policy. It could be a user running a query on sensitive data, or updating an existing financial transaction outside of an application, or an application running a query never seen before.
  • Attack activity/signatures: Some DLP solutions include built-in detection for certain attack activity. This may be linked to vulnerability analysis, signature-based, or heuristic.
  • System and administrative activity: Incidents involving administrative or internal system activity, e.g., new account creation, privilege escalation, DML/DDL changes, system updates, stored procedures, and other configuration changes. These alerts are typically being focused on SQL (and non-SQL) outside of simple SELECT, INSERT, UPDATE, and DELETE queries.

Once an incident is created and any external alerts sent out, it should appear in an incident handling queue for management. This is similar to what we see in Data Loss Prevention and many other security tools, but optimized for database activity.

The queue should be visually well designed to make critical information easier to find, and allow customization for different work styles and interests. Unlike DLP, it's less important that the queue appeal to non-technical handlers since it's far less likely that anyone without database or security knowledge will work directly within the system. For DAM/DBF, we tend to rely more on reports for the auditors, risk managers, and other non-security types.
Incidents should be easy to sort and include color coding for sensitivity and criticality. When you click on an incident, it should let you drill down into more detail to assist the investigative process. Handlers should be able to assign, share, and route incidents to different users within the system. I'm a big fan of drop-down fields to change incident status right on the incident row. The system should also support role-based administration, allowing you to assign specific handlers/administrators based on the policy violated, database affected, and other factors.
The basic workflow must allow for quick sorting, analysis, and investigation of incidents. Once an incident is detected, the handler can close it, add supporting investigative material, change the priority, assign it to someone else, or escalate it. To support investigations you should be able to correlate the current incident with other activity in that database by that user, violations of that policy across different systems, and other factors, in order to determine what's going on.
Since incident handlers may come from either a database or a security background, look for a tool that appeals to both audiences and supplies each with the information they need to understand incidents and investigate appropriately.
DAM/DBF products to date have focused on database-only incidents, but some systems are now expanding into platform activity on the database host and application activity.

As with nearly any security tool, you'll want flexible reporting options, but pay particular attention to compliance and auditing reports to support compliance needs. Aside from all the security advantages we've been talking about, many organizations initially deploy DAM/DBF to meet database audit and compliance requirements. Built-in report templates can save valuable time, and some vendors have worked with auditors from the major firms to help design reports for specific regulations, like SOX.
Reports should fall into at least three broad categories: compliance and non-technical reports, security reports (incidents), and general technical reports.
An advanced feature of some Database Activity Monitoring/Database Firewall solutions allows them to track and correlate individual query activity back to the application user. This typically involves integration or monitoring at the application level. You now know which database transactions were performed by which application users, which is extremely valuable for both audit and security reasons.

Today, most users just deploy Database Activity Monitoring/Database Firewall to audit and alert on user activity, but many of the tools are also perfectly capable of enforcing preventative policies. Enforcement happens at either the network layer or on the database server itself, depending on product architecture.
Enforcement policies tend to fall into two categories. The first, similar to many of the monitoring policies we've described, focus on user behaviors like viewing and changing sensitive records. Rather than just alerting when a DBA pulls every account number out of the system, you can block the query. The second is focused on database exploits; similar to an intrusion prevention solution, the system blocks queries matching signatures for known attacks like SQL injection.
The nature and level of blocking vary based on the architecture of the DAM/DBF tool. Integrated agent solutions may offer features like transaction rollback, while network tools might block the traffic from hitting the DBMS in the first place. Digging into specific architectures and benefits is beyond the scope of this report.

Databases rarely exist in a vacuum; more often than not they are extensions of applications, yet we tend to look at them as isolated components. Application Activity Monitoring adds the ability to watch application activity, not just the database queries that result from it. This information can be correlated between the application and the database to gain a clear picture of just how data is being used at both levels, and identify anomalies which may indicate a security or compliance failure.
Since application design and platforms vary even more than databases, off-the-shelf products cannot cover every custom application in  the enterprise environment. We see vendors focusing on major custom application platforms, such as SAP and Oracle, and monitoring web-based application activity.
As Database Activity Monitoring/Database Firewall evolves into Application and Database Monitoring and Protection (ADMP), the combination of application integration, content awareness, and enforcement options will be critical.
With or without application monitoring, some DAM/DBF solutions come with pre-configured policies for common applications (e.g., PeopleSoft). Although they must be tuned to account for application customization, they can jump start the policy building process and save the company from manually building all the compliance and security policies.
Although no tool will make the company compliant out of the box, pre-configured compliance policies for common platforms facilitate the process, especially when nobody know where to start. Most vendors hire or partner with auditors to help them build their compliance policy and reporting packages for common regulations, such as SOX and PCI-DSS, which frequently impact databases.
Although many organizations have rigorous change management policies for the database platform and underlying system, far fewer enforce change management at the query level (which is invisible to traditional change management tools). An advanced feature of certain DAM/DBF solutions offers integration with external change management tools for closed-looped tracking of query-level changes. The requested change is approved in the change management system and a ticket number issued. The DBA enters that ticket number as part of their session, and all database changes (even to individual field updates) are recorded and correlated back to the original change ticket.

Database Activity Monitoring/Database Firewall is an extremely valuable tool for compliance and security; it is critical to the emerging practice of information-centric security. Database Activity Monitoring/Database Firewall gives insight into our most sensitive systems in a non-intrusive way, and can evolve into a proactive security defense. It’s one of the few tools that can immediately improve security and reduce compliance overhead without interfering with business processes.
Although deploying a tool that crosses the organizational and technical expertise boundaries between database, security, and application management can be daunting, by understanding the companies’ needs and following a structured selection process, can help to ensure a successful deployment. Many clients start with a narrowly scoped project which quickly expands throughout the enterprise.
From a business standpoint, the most important factor in a successful deployment is pulling together the key stakeholders across database, security, and application administration to determine initial requirements from a technical and process standpoint. In working with hundreds of clients implementing database security, the most common obstacle I’ve seen is a failure to manage expectations and responsibilities across these different groups.
After level setting and determining project goals, the most important technical necessity is to understand the platform support and performance requirements.  Aside from growing in terms of systems covered, programs also expand into new use cases as administrators become more comfortable with the tools and their capabilities. A company can start with auditing only, then add alerting for a few basic security policies, and end with full application integration and security blocking.Database Activity Monitoring/Database Firewall is a powerful platform for compliance and information-centric security. It is relatively easy to deploy, non-intrusive, scalable, and flexible.

  • manufacturer
  • Supported SecureSphere Products
  • Type