Imperva Cyber Community

In this Ask Me Anything live Webinar, and answer all questions, customers and partners had,  around Imperva's Database Activity Monitoring (DAM). 

Read the list of Q&A on this thread.
Database Activity Monitoring (DAM) Ask Me Anything Session

Take a look at the Community Resource bundle for DAM products. A list of FAQ's by topic and an easy way to find it. 
Database Activity Monitoring Q&A
DAM Deployment Resource Bundle

Transcript below: 

Chris Detzel
I appreciate you guys coming today. Let me post this in the chat so that if you haven't asked your questions, you can ask it. Just a sec chat. This thing sometimes starts coming up. So please go ask your questions here and then let me share my screen. Alex, we'll get to that question here later. So first question to our experts. Let me minimize this so I can see everything. So any tips for creating signatures so I can figure all the time do not audit signature and always struggle. I might not audit import feature events. Who wants to take that question?

Brian Anderson:
Sure. I can help with that. So around signatures is a general topic. We typically say don't go create arbitrary signatures because there's a lot of out of the box configurations that we already have at your disposal. If we do want to write something custom like that, it would be fire because you've already implemented all of the other components that we have. And by that, I mean we're looking for select statements against sensitive data. We're looking for privileged operations that are already present in command groups and privilege operation groups. So if you're already monitoring those things for those specific users and you're interested in anything that's outside of that, it would be possible to create a policy to say, if my privilege operation doesn't equal one of these, then I can audit that. And that kind of captures anything else that may come as far as future commands that deviate from what's already known. That would be one suggestion that I can make off of that. If there's something specific that you're looking forward, there's two parts to a signature certificate, there's a match and then there's a pattern.

Brian Anderson:
The first part of it is kind of from a reject implementation perspective. Do I see this pattern there first, then I can apply my more complex signature to that from a performance perspective. We don't want to just create a signature that could potentially be expensive and resource intensive to look at all traffic for. So when you're creating signatures like that, are there certain things about this particular command that you know are going to be present that you can do a quick check for before apply the more extensive, broad expensive signature to that and then only apply that to the parts of the database or the specific users that you may be looking for that activity to originate from so that we're not applying a signature to all traffic that comes into the database from that perspective.

Brian Anderson:
Does that make sense? And just as a general statement, I think that we're going to have written responses to all of these questions posted on the community. So if we don't have enough of a chance to talk about this in detail, we will be able to elaborate that more. And also for folks that weren't able to attend, we will have that caption on the community so you can go back and refer to it after the fact as well.

Chris Detzel:
Absolutely. Just kind of as a followup, if you're on the call and you ask this question, feel free to unmute yourself. And if you have an additional question about it, I think that would be okay too. Great. I'm going to go to the next question. And the next question is one problem that I've run into is that DAM is deployed without really understanding what is necessary to effectively use the product and how to handle the information that gets generated. The appliances are easy to configure and deploy, but we are faced with trying to get involvement from other groups to make effective use of DAM, any suggestions.

Brian Anderson:
Absolutely. So typically there is a number of databases that are in scope for you to monitor with a database activity monitoring deployment. Those databases are in scope typically because of regulatory compliance requirements, right? So either that or there's a security initiative that we have to protect those from a potential incident that happens. So if it's regulatory compliance and the application or business owners have that burden of going through that audit, really helping them with producing audit reports and offloading the work that the DBAs would have to do around generating those reports, making that easy for them is a good way to show value to the business and also help to adopt the implementation on a per business unit or per application basis. Another component here that's really helpful is the use of classification.

Brian Anderson:
So we don't want to go watch every channel on TV. We don't want to capture every audit event for every database. What we want to do is refine that down to only what is in scope. We don't want to write more to disc than we need to. And that really comes into classification. So I want to know what about my databases are sensitive. We can help either ingest that from something external if you already know what those things are, or we can scan the database ourselves, or we can do a combination of both. But once you have that list of tables that you know are in scope where your risk is or what is sensitive really helps to drive your policy definition. And then you can go back to the business and say, I know what your sensitive assets are. I also know who's doing what with those and then helping to produce reports for that. Ultimately offloading the remedial repetitive work of generating reports, helping to streamline that and centralize that. And that's how one of the ways we can show value to the business.

Chris Detzel:
Love it. Anybody else have anything on that?

Jim Burtoft:
Yeah. Chris, this is Jim. So this is a question we get all the time with DAM because we've got a lot of functionality in DAM that not everybody is using. And so I want to really encourage everybody to dig into, look at what you're using now but check out those other tabs, because there's a lot of things, whether it's the sensitive data scanning that Brian mentioned, which is useful for you from a policy perspective, but it can also be useful for the compliance team, for their CCPA compliance, the vulnerability scans, where we can look to see whether certain patches or certain configurations are applied on the database can be great for the standards team that is responsible for making sure that databases are deployed with a certain configuration. And you have that ability to kind of go to these other departments and say, "Hey, look, we already have a tool that does this. You can use it."

Brian Anderson:
Yeah. One other thing to add to that, by the way, is from a process perspective and I think Jim and I were talking about this earlier is that we can help to integrate with your change control process. So typically if you have to modify tables on a database, there's a change control that's open, there's a window that that's supposed to be executed under. The DBA is supposed to do that according to that change control ID, then there's this exercise of when you go through an internal or external audit, show me the queries that were run for this particular ticket. And typically I've been through these audits with customers PricewaterhouseCoopers or whoever comes in and say, "Okay." They pick out a random ID from the last six months to show me this one. And then you have to go through this process, no matter what the database technology is to find here's the queries that were run during that window, prove that it was in the window, prove that what was run, what is the process to actually do that?

Brian Anderson:
We can help ingest those change control IDs and we can help enrich your data, set anything that is run against that database, validate that A, that it's a valid ticket and B, we can help enrich all the queries with that. So it becomes a very, very simple, easy report. And lastly, if you try to perform coverage operation against the database and don't have a valid ticket, we could generate an alert saying, "Hey, we saw somebody try to do something that wasn't associated with a ticket." Or we can even go back into that change control system like service now, remedy or whatever it is and update those change control records with the actual queries programmatically. So if you have a database, DBA login runs a bunch of queries that will ultimately end up back in the change control records so you don't have to go do this back and forth. That's a really, really valuable operational integration. It's very, very straightforward to set up, but the business use a huge optimization and process and it helps save a lot of time for all people involved.

Paul Aiuto:
Yeah. I just want to piggy back on that one more so one level higher also operationalizing the information network I think is very important as well. So sending it over to your SOC, your NOC, right? How can we refine that data so they can see what they need to see for forensic investigation? So not only are they... You can leverage the traditional security through database monitoring, but also leveraging a solution that sits on top data which is kind of a mix, right? Where we're sort of refining the millions of events that are coming in to more narrative contextual information that really allows the SOC to know what is going on in the environment and essentially how to react to that accordingly.

Chris Detzel:
Good. So Brian, you mentioned we can help you do these things. What does that mean? Like the customer would have to create a case or how do we help them go do those things that you mentioned?

Brian Anderson:
To set up those configurations if you go to github.com/Imperva, there's lots of projects that are on there that are open source packages that we can help to integrate with stuff, whether it be integrating with Ansible or shape up Puppet to deploy all of your agents in automated fashion, or write integrations with service now, or have some kind of automated tasks to ingest data or push data somewhere, we have lots of different solutions that are posted up there. I've authored several of them. The team here has contributed to them as well, but that would be kind of where we'd point as far as documentation of resources for how to implement that. And then of course we always have services that can help you from an implementation perspective to put those things in place as well.

Chris Detzel:
Okay, good. And kind of when you reply to this later Brian or whoever wants to reply to it, put some of that information in that particular.

Chris Detzel:
Next question, what is the best way to keep an eye on the gateways to make sure they are not missing audit data or becoming overloaded via CLI?

Brian Anderson:
This is a great question and customers would typically ask if they can install third party software like performance monitoring agents on our day which is for that very purpose. So we have that as SNMP control but really you're looking for more information. So we have developed going back to GitHub, there is a project called the MXToolbox that I'll post a link to in the response too soon, but that has scripts for both the MX and the gateway to take everything from disc consumption to CPU utilization, to memory utilization, et cetera and offload that to either a New Relic, InfluxDB or Grafana, or it can even output to a generic SIM with a uniquely index, J sound object that has all this information in it. And we typically have that run as a crone to output every minute, so that you can see on an ongoing basis, what all those metrics are. And so that's kind of the best way, I would say, to get visibility from a performance perspective into your environment.

Brian Anderson:
And it uses top and SAR and all the counters that we have in our proxies and everything that we know about from that appliance and keeping in mind that this isn't limited to just our database product, it's also something that applies to our web application firewall product and for appliances that live in Amazon, for example, they spin up and they'd tear down. These can dynamically build into the startup process to where even though there are stainless machines that could only be there for a small period of time, we still have the ability to get the performance metrics out of that. So that's what I would point to. And again, I'll post a link to that on the community after this is done with specific reference to that, but I would definitely advise or suggest temperament that performance monitoring components have been very helpful for our customers from a visibility perspective.

Chris Detzel:
Perfect. Anybody want to add anything else? We'll take that as a no. Next question, Brian, you're doing a great job. Team, you did good job here. Appreciate it. So CJ asks, so in the future, all DAM hardware calculation methods will be changed from traffic to IPU. So as I look at this graph, love graphs, love pictures. I think you've seen this already. So you probably know what that means. So however, the interpretation of IPU and in portable hardware appliances is Imperva Performance Units. IPU is a priority. I can't say that word well. A metric that represents the maximum recommended load on a gateway and-

Brian Anderson:
I was taking a look at this and basically there's questions about how do we size an environment? And there is a notion of a transaction per second versus Imperva performance unit, which is kind of just a general term that we would use to scope something. And really the answer here is how do we size something for the database activity monitoring deployment? The answer is it's really difficult and there's no silver bullet that can apply universally to every single environment in the same way. And by that, I mean, no two databases are utilized in the same way. The schema is the underlying data and the audit requirements for that are different. And so if I initially monitor a database and I only look at a few privilege operations or if I look at that exact same database and now I'm monitoring for selects against all your sensitive data, I'm looking for a variety of other excessive record access or modifications or et cetera, the more audit requirements that I have then deciding what is kind of a moving target there.

Brian Anderson:
So basically the best answer that I can give is let's monitor the appliances and look at what the utilization of that is. If you have an increase or spike in traffic on those databases, at some point it would make sense to increase the resources that it would take to monitor them. So that's kind of a moving target both on the customer side as well as on the database activity monitoring side depending on what we are monitoring, because as we apply more policies, more resources are used there. Maybe if we have expensive policies that we apply, those things will kind of be kind of a moving target. As a general swag answer is if you know nothing about the environment, historically what we've used is 100 transactions per second per core or 100 IPUs per second, per core as a general swag answer.

Brian Anderson:
But again, that's not something that's definitive or hard set for every environment that you run into. Because, like I said, it's a moving target depending on the utilization and depending on the amount of traffic and the spikes in the environment that may need to optimize for different holiday windows versus off hours versus low traffic months and so forth. So the best thing that I would suggest is use that kind of general methodology of 100 IPUs per core as a general swag if you know nothing and then from there, let's implement performance monitoring and keep track of how our gateways are being utilized and what the capacity of those are and as you need to increase, that would be one recommendation of justification to do.

Brian Anderson:
So this is one of the reasons that we moved to more of what we refer to as flex pricing, because we stopped selling the perpetual licenses for X number of gateways and move more toward let's monitor databases and however many gateways you need, let's deploy as many as you need because of this moving target in this challenge around sizing. So if you buy licensing for X number of databases, now it's open for you to adjust and optimize your infrastructure to support that as you need. Hopefully, that answers the question there.

Chris Detzel:
I think so. Anybody else have some thoughts here? No. Okay.

Speaker 6:

Hey, Brian.

Chris Detzel:

Go ahead, sir. Yep.

Speaker 6:

I'm from Singapore and I've a doubt, is it the correct way to calculate the load based on the CPU cores?

Brian Anderson:
All right. That's one way that if we know nothing about the environment and it's a brand new environment, and we're trying to size for an environment, that's one method that we've used in the past. Just saying, how many cores does the database have? And that tells us what the potential capacity, the maximum capacity of how many queries that could potentially process. That's a swag that we have used in the past to come up with inappropriate sizing, if it's four quarters versus a hundred and something cores, that's a different size of database and different number of transactions or database queries that we could potentially run into on that database.

Brian Anderson:
So yes, as a general statement, that's kind of one methodology we've used, but again, I go back to if it's only 10% utilized or if it's 80% utilized, the number of gateways and the infrastructure needed to support that will change based on that utilization or based on that traffic consumption there. So it really is kind of a moving target depending on your environment. But that certainly will give you, the CPUs will give you a ceiling of what you know the capacity of the database can be.

Chris Detzel:
Again, these questions are being asked and we'll get through them as quickly, but not super fast. We want to make sure they're detailed answers. If we don't get to your question today, which I think we might, we will make sure to post them on the community here and then send that out to everybody who was on the webinar. Next question, what is the best way to check the health or current load free space, backup options, alarm of the DAM, MX, GW, you can read that, appliances servers?

Brian Anderson:
So two things. Let me go to back to the performance monitoring packages that are referenced on Github is kind of one really good practice that will out put all this information and dump it into a time series database or something that is designed to performance, monitor visualization stuff. So either Jason or InfluxDB, Grafana or New Relic are things that we have seen requirements for in the past. But secondarily, you can also create system events that if there is a load that's been exceeded or if there's a disconnection from the MX to the gateway or the gateway to the agent, you can have events that are generated in your environment that you could then go create tickets for, get alerts for, send emails for. So that's something that's helpful just from a visibility of notification.

Brian Anderson:
But then performance monitoring is definitely something that would be helpful to give visibility to the rest of those components. And that applies to the MX and the gateway. The MX script should work exactly the same for SOM. For DRA you don't really have the same challenge because it's not really looking at live traffic coming in. It doesn't have the responsibility of potentially remediating that traffic. It's more of an offline analytics server of you certainly can monitor metrics on that as well, but this applies more to MX, gateway and SOM generally speaking.

Chris Detzel:
Next question. Can you talk about things like reporting, saving reports to shared locations, examples of a few reports that show value to management? So example, if you come up with a report that says how many blocks and non-blocked events occurred and on-prem, WAF and DBA and DAM environment. Make it look pretty. I don't know if you can talk more about that and hey, there could be a webinar one day around this, it's a good question from Jason.

Brian Anderson:
Absolutely. Great question. So around reporting there are... Well, first of all, if you do generate a report, you can create an action set and the action set can be responsible for copying that report to somewhere and that could be a shared drive location. It's very common to have a one shared drive or one now as a stand or whatever it is you drop these reports on for historical reference, and you can schedule these reports Monday morning, you get your previous week's reports or whatever it is as often as you'd like. And so that's one underlying mechanism that it does make sense to output those to a particular location. And the action set is what we would typically use to do that. Second part around this is what kind of reports are useful? In working with customers in the past what I've seen is there's a handful of reports that are really useful.

Brian Anderson:
So reporting on like privilege operations, looking at what your users are doing in the environment and seeing what that looks like, are you modifying something? What's happening? And that could be then given to each one of the business units or each one of the application owners to help them get visibility into what's happening in their environment. So that's one aspect to it. If you have excessive failed logins, attempts to log into the database, if there's SQL exceptions, those are the things that they're giving visibility to maybe your top queries, most expensive queries, given that back to the business unit as well. So those are things that have been helpful from a reporting perspective.

Brian Anderson:
As a general statement like our web application firewall product, Jim and I were talking about this as well, it blocks a lot of stuff because half the traffic on the internet is automated and a good chunk of that is malicious. So you're always going to be blocking a lot of things on the web application firewall side, so noise. The database activity monitoring world doesn't operate in the same way. Like you don't always see a bunch of malicious traffic every day. But you do want to find nefarious things. You do want to find stuff that deviates from the norm and you also want to prevent or detect things that should not be happening. So dropping a table on a production box might be an example of something that you'd want to block, but reporting on things that should not be happening is really how you show value in database activity monitoring, looking at sensitive assets and looking at things that should or shouldn't be happening around privileges or access.

Brian Anderson:
So if you have our database or risk analytics or data risk analytics product, that can really help pinpoint this user did something and this is an actionable explanation of what that user did that was a deviation from normal traffic. That's really helpful to deliver to management saying we invested a billion events and I have seven things and that refines down instead of dumping everything to the SOC or having to run reports constantly, this just gives you that actionable bite-size visibility into what you should be paying attention to. But apart from that, looking at specific user activity and then looking at excessive record access to sensitive data, operations that are performed around that sensitive data are general useful reports that we've seen for customer environments there.

Chris Detzel:
Thanks, Brian. Does anyone else have some tips, thoughts, tricks there? No. That's a great answer. So next question. So thank you for the opportunity and really cool with somebody like this besides me. So that's positive. That means that somebody else wants them. So my first question is how long is enough to run your policies and monitoring mode? So maybe we can answer that question and then we'll go to the next.

Brian Anderson:
Sure. So around the topic of monitoring mode versus blocking mode there is a little bit more concern in putting things into a blocking mode on the database side versus on the web application firewall side. The web application firewall side, it's intended to mitigate a bunch of traffic. And again, like we talked about, you get a bunch of noise there and you're expecting to block that based on what you know about. On the database activity monitoring side, there's fear of blocking something that's a production transaction. So we want to be selective about what we want to block. A lot of customers would be very selective about to do that successfully, things that we know that we can block or what we would start with. Like for example, should you ever be dropping a table on a production database? That's something I can confidently say, let's put a policy in place and block or prevent that from happening. Certain operations like I should never escalate user privileges or create new users or delete users on a production box unless it's following a change control process.

Brian Anderson:
So those sorts of things you can absolutely put into a block mode confidently. As far as other monitoring the block mode, it would really depend on the policy definition, really what you're looking to accomplish. But again, I go back to what is it that we know for a fact that we can block and mitigate against? What is our biggest risk associated with that? That would be modification to schema like dropping tables. Secondarily, there may be things that we can easily implement like if it's a PCI type application and we have a credit card table, should there ever be more than one record containing a credit card, leaving that table, at any time. Applications don't typically do a select star against the credit card table. So we can look at the number of record leaving certain tables and have policies around that as well to prevent excessive record access.

Brian Anderson:
Those are things as a general statement that you could put in like X number of records over a certain period of time, that sort of a thing. The last part about this is we really get into... There's a question here about when DBAs are all logged in with the database with assist DBA role. So there are two types of users with databases, there's programmatic users that are invoking access from code, maybe applications of scripts, then there are interactive users, DBAs that log in from their toad or SQL developer or whatever client they have and do different types of things all the time. The programmatic users, those behaviors and those activities are repetitive and they fall into a pattern. It shouldn't really deviate that much because it's code, it's executing and it does the same thing every single time depending on the input parameters, right?

Brian Anderson:
And then the DBAs, you wouldn't really be able to put them into that box of saying, "Hey, I'm expecting the same things from you." Because they do different activities all the time, every day. But what we can do is look for deviations from what is known and expected from our service accounts. So if an application user, a service account logs in from an associate or the application servers IP address from the correct client and the access tables that are known, great. This is where the profiling feature can actually come in really useful, I can develop a profile for my service accounts and say, this is what a positive security model looks like. This is what you should be doing.

Brian Anderson:
If you deviate from this, I want to block that or prevent any deviation from that. So that's another example of a way that you can put something into blocking confidently because you already know what those behaviors and activities should look like. But those are some general examples of how we've seen blocking be useful in production. And then of course you can set policies to a lots of other things that don't necessarily need to be blocking.

Jim Burtoft:
I think since we have a bunch of DAM users online, I think it's important to point out that while you can do blocking in DAM, it is not something that most of our customers do for everything. It's not that common to do it across the board because like Brian said, you really need to think about and oftentimes you want to get that buy in from upper management to say, what is it that we want to make sure we block in every circumstance, right? And like Brian said, there are some things like a drop table on a production database where you can say, yes, we definitely want to do this. A lot of times you can say, okay, nobody should be accessing the database from anywhere other than this list of IP addresses. So block anything other than that, but very rarely do you want to have a block on just kind of a general security policy. You want to be very specific and make sure that everybody knows what you're blocking and why you're blocking it so that it doesn't come back on you.

Brian Anderson:
Yeah. This also opens a different question of like how it's configured. So if you have an agent only deployment, which is very common to have clustering of database of gateways and then agents reporting back into that, the agent runs in a particular mode, may be a stiffing mode or an inline mode. And really what that means is if I'm going to block something and the way that I can inspect the traffic is let's say that I want to look at the responses like, hey, I want to know the number of records that are leaving. So the agent will basically look at the request and the response and evaluate that before it makes the determination, if it needs to block it or not. You can change the mode of what the agent runs in based on an agent monitoring rule. So if I say by default, I'm running in a stiffing mode and this is getting a little bit into it here, but I think this is the intention of this call here. I can say for these specific examples, I want to be in inline mode and I want to block these particular examples.

Brian Anderson:
So you can tell the agent when in these scenarios, if the operation matches drop table or whatever the case is on these particular databases. In that particular case, I want to be in line and then I want to block for those examples. That's something that I've seen as the best practice using agent monitoring rules to be able to make that happen. And also there's a notion of using agent monitoring rules. We haven't discussed this, but maybe excluding traffic, it's not relevant around the whole performance monitoring and optimization discussion. Maybe your ETL jobs that run don't need to be audited every day and you don't want to put 10,000 transactions in your audit just because they run on the database when they're coming from a known place and already authorized. So there's a way we can exclude traffic based on agent monitoring rules as well if it's already known and authorized like that.

Jim Burtoft:
And just one more call back Chris talking about the CIS, DBA role, that's a great example of what Brian was talking about with the tie in with change management, where you can say, "Hey, anything coming from this particular role must have a change management ticket because you should never be logging in with any kind of assist DBA role, unless you've got some kind of a change management setup." And then let's audit that, let's feed it back in and make sure that somebody sees this is what was approved in the change management, these were the commands that were run. Is that what you were expecting?

Brian Anderson:

Right. And not to get a little bit too off topic here about roles and databases, but we do have a user rights management component where we can scan your database and tell you who the user is, how they got that access, what the role is. I was working with one customer a few years ago that had a mandate to get rid of all default roles in the database and use their own customer implemented roles. And what they found in that process as we were doing these scans is that users for whatever reason were creating custom roles that had nested default roles that were included in them. So they found this because this role contain that role contain this role. And so it's really interesting to see what your underlying schema definition is and what your role definition is and how people were able to get these permissions. And so that's one thing that's very helpful to get visibility into, but then the notion of, if somebody does have a particular role being able to follow it and enforce that we're following that change management process and make that an automated process is very helpful.

Chris Detzel:
So Jana has another question. Can you walk through a quick example of the benefits of configuring stored procedure groups and the link between having to define both the stored procedure group and the table group in order to define the actions and the audit data. Let me finish just because I want to make sure we get it all. We would like to leverage this feature but it seems like a lot of overhead. And if we have to define store procedure groups and then turn around and build table groups on that. So not really following how the table group fits into the whole process.

Brian Anderson:
So stored procedures just for the group context here is a stored procedure is like a function that you can call the does some other stuff under the hood, but you won't see that in audit because all you see an audit is just the function that was called, but it does a bunch of other things. So we don't really know generally speaking what that function did, that function could be dropped 25 tables and you just don't know because you don't have visibility to inside of it. So what we can do is the stored procedure groups will retrieve that stored procedure and tell you what tables this particular function interacts with. And so it helps you give... It connects the store procedure function to the tables operations that are performed under the hood and there are two things that are required to configure that. One is we need to define our stored procedures. So stored procedure name, the table names and the operations that are defined so we can pull those from the database and help you programmatically define what those are.

Brian Anderson:
But then in order to associate that to our elements in the site tree, we have a site server group service and application based on the different tier in the environment. There are three, four, five, and six and seven. So the application level of the site tree is where you would need to add your service user groups. But you also have to add a table group there to tell this application what tables you're going to be monitoring. And those two things are complimentary. They need to be both assigned to the application and the site tree. You do have to have both of them there. The value proposition here is that you now have visibility to what your store procedures are actually doing at the database level but we're working to implement an API for our stored procedure groups and we already have one for table groups.

Brian Anderson:
So at least you can programmatically define table groups and associate those to the side tree. So that's one thing that can help offload or optimize that process. But you do have to have both of those. And again, the value proposition here is mapping that general purpose function that you would see in audit down to the operations and the tables and objects that are being accessed when that stored procedure is executed.

Chris Detzel:
That makes sense. Just wants an easier way to do it. So thanks for that. Another question. So Sabe asked what can we suggest in terms of security policy to a client who doesn't use the baseline feature in DAM, so it can add value to that deployment. So it's from a partner.

Brian Anderson:
So one thing that we would say is a general purpose is if we know nothing about the databases, let's open up the discussion around classification, because you can certainly look to implement security policies for privileged operations. That's always something that's viable but let's also understand where the sensitive data lives, because if I can create a policy that says, hey, did X amount of records leave my database? That's something that I would want to get visibility into. If they are production systems getting visibility into those privilege operations like dropping tables, escalating privileges for users, schema modifications, those things are valuable. Also security policies, audit policies that you can generate for SQL exceptions and security policies for the number of failed login attempts are very useful right out the gate. So those are things you can give visibility directly to. If we know nothing about the database, these are general things that you'd always want to see from a security perspective.

Chris Detzel:
Great. Anybody else add anything to that?

Jim Burtoft:
Yeah, I'd add to that. Take a look at some of the command groups that we have in the system, because a lot of them, questionable maintenance, OS interaction, questionable ops, if you're going into a customer that says, "I don't know what I should be looking for in DAM," then those are kind of some nice things to throw out. Obviously, excessive rows returned as well and access of failed logins.

Chris Detzel:
That's a good one. Okay. Next question from Rakesh. So here, it's basically three questions in one. So one, how to review which files in view and identify the issues from tech info log. So agent the GW and MX. So we'll tackle that question first.

Brian Anderson:
So there's the get tech info log that you can pull from your MX view gateway that basically brings the entire configuration of the MX and gateway together, including interface configurations, everything that's on the box that tries to put together. That's for the general statement as we give that to support so they can understand what your configuration is, what's going on in the environment. Typically, what customers would be interested in in looking for the tech info logs, they typically create a case because they're experiencing some kind of issue they want to investigate something. And so is it performance related? Do we see a massive spike in traffic? Do we have something that we want to get more information about? Instead of going to the GTI logs, I would point more to the performance monitoring packages that we have.

Brian Anderson:
Essentially that gives you visibility to what your CPU consumption number of worker threads that are allocated, all the number of transactions that are being executed on a gateway. We also break that down into server group level. So if you have one very high consuming, high transaction server group versus other ones or not, you can pinpoint that from a traffic consumption perspective, which is an interesting and great way to do internal chargebacks based on utilization within the organization. So I would point us more toward the performance monitoring as opposed to trying to break down GTI files which is really more intended for support.

Brian Anderson:
But one thing I can share with you is if there is a particular area that you're looking for because maybe you've experienced something and there's an error that generated and you want to know in more real time, if that error comes up, because something will happen, if that error occurs, our performance monitoring package has an array or a list of files, and then a list of patterns that you can look for. So array log messages or the whatever array log file that you're looking to see if that is present, every time this performance monitoring package will run, it could then add if it sees this error to the outbound event that you're sending to your SIM.

Brian Anderson:
So you can create triggers to say, "Oh, I saw one, here it is." That's been really helpful in working through escalations with support in the case you're experiencing something with one of the appliances. But I would definitely direct more towards the performance monitoring package as opposed to dissecting the GTI logs, but the GTI logs really you're looking for and it's organized the same way it is on the operating system whereas your network statistics, where's your CPU consumption? What are all the configurations that you have set up? They would typically look at what those configs are the same way that you would look up locally on a box that's the recitation to live.

Chris Detzel:
Thanks for that. And by the way, everyone, hopefully this is helpful. Again, this is the first time and especially doing live on Ask Me Anything. So feel free to post in the chat, not on this one, but it just in the zooming chat, is this helpful? Is this good? And that kind of thing. So next question, when we download alerts in CSV format, so the action save as a CSV, the error messages reported and those DAM alerts are not being captured. Whereas when we save alerts as a PDF, so action save as PDF, we are able to retrieve the error messages captured in the DAM alerts.

Brian Anderson:
Well, I have to go see and try to reproduce that because it's certainly the expectation would be that we would be able to output that SQL exception string is what it's referred to as there as actual error message. So if it's coming in the PDF and not on the CSV, that's interesting and I'll help investigate that and try to reproduce that. But just as a general response I know that we can capture that with audit. So if we have a security policy that you're not able to see the SQL exception for, if we have an auto policy that's capturing those events as well, you could certainly get it that way and that would output in CSV format. So that'd be one quick way to be able to give visibility to that, but I can certainly help investigate why the security CSV export wouldn't have that and get back to you on that.

Chris Detzel:
Great. Yeah. And we'll just post that in the comments around the reply. And the next question that Rakesh has is basic investigation steps for common issues in DAM, so Agent, MX, Gateways and SOM et cetera, so that customers can do more investigation on their end while waiting for in purpose support team to resolve the issue.

Brian Anderson:
Again, I would go back to point to our performance monitoring package, because typically at times where you experienced an issue, you could then look over the course of time for the last X number of days, weeks and months to see what the pattern was of utilization. If you have a massive spike and there's the number of worker threads that are involved in that and some of those things are indicators that you're experiencing an issue. That would be something that as a general statement, you could point to. There's also some log files that we can direct you to see if you see any kind of known errors. And we'll post this in the chat here on the community to give reference to which policy we're referring to. But as far as that goes, looking for errors that are there and then also looking at the general performance of the box, getting all those indicators is something that I think that would be a helpful step while you're waiting to hear back from support.

Chris Detzel:
Great. And I'm going to test. So I'm going to... Somebody wants to say something, sorry.

Rakesh:

Hi. Yes, this is Rakesh. I had another question, can I add it here?

Chris Detzel:

Yes, absolutely.

Brian Anderson:

Sure.

Rakesh:

Okay. How do I identify if an MX or SOM has reached the maximum number of policies that can be created? Is there a way to find out?

Brian Anderson:
A maximum number of policies that can be created?

Rakesh:

Yeah.

Brian Anderson:
So I'm not aware of a maximum number but I said maybe I have to go do some investigation on that. I haven't run into that threshold and I know that there are maximums generally speaking with a number of server groups and services and number of agents, of course, for just the general capacity but I don't know that there's a maximum number of policies but I can do some investigation on that. Do you have an extremely high number of policies for a particular reason or is there something that you've run into that that's the reason that you're asking that?

Rakesh:

Yeah, actually as in when we start working on it for each application, we tend to create more and more policies. So we are not sure whether we are actually reaching the capacity of MX or SOM in creating the policies or laws. So we just wanted to understand from you experts how many or what is the capacity of SOM of the MX for the number of policies.

Brian Anderson:
That's a great question. And I don't know of a maximum number but again, in technology, nothing is infinite. You can't have a billion, obviously. So let me do a little investigation on that and try to get back to you on that but I don't know of a known number that comes off the top. I haven't run into that in the field at least but I can certainly try to help investigate that with our product group and give you a general number of the best practices of what a maximum would be for that. One question that I can ask is, there may be some opportunity for consolidation of policies. I'm not sure what the use cases are that are implementing these policies but I would certainly love to work with you on trying to see what it is we're looking to accomplish and why we're creating these policies and maybe help to kind of assess that and see what we can do to maybe optimize some of that.

Rakesh:

Yeah. That could be helpful Brian.

Brian Anderson:
Yeah, I'll do it.

Jim Burtoft:
And Rakesh I think it also goes back to what Brian was talking about before with the performance, because you can create all the policies that you want, but if the gateways that you're monitoring the mind can't handle that, then that's where you're going to run into problems.

Brian Anderson:
I said or if you have a ton of policies in low traffic, then those things kind of balance each other out if you will, but I would certainly love to work with you offline here and maybe set up another session or we can kind of go through that together if you'd like.

Rakesh:

Yeah. Sure.

Chris Detzel:
Yeah. Feel free to reach out to me and I'll make sure that that happens if you don't have his info. So the next question. We have 12 minutes, so we'll try to get to what's in the chat and it looks like this is also on the community. Good. So Alex has a question, considerations to make a gateway cluster and VMware how to do it. There's a document that explains implementation more clearly. So is there something there or if you have more explicit document or diagrams, certainly appreciate it.

Brian Anderson:
Certainly. So, I mean, I would point to the documentation. Essentially, there's two ways to create a cluster and you can either have a single interface or multiple interfaces where the cluster network is separated out to a separate segment and separate niche. And then the traffic that goes to the MX and the traffic that comes from the agents to the gateway, the listeners are on the second niche. So you can kind of delineate to separate the cluster traffic from the rest of the traffic that the gateway would be connecting with. And that's just kind of from a network topology design, you would have four groups that you would define in VMware that you would have the cluster network be connected to one. And then the traffic coming to and from your MX and also coming from the agents, the listeners that would be on a separate niche.

Brian Anderson:
So those are the two modes are ways to design a cluster. So after you deploy that, basically there's a model of gateway 2500, 4500, 6500, that you would deploy your VM. And that has the associated resources for that gateway. So the number of CPUs use, number of memory, whatever that will then spin up and all the gateways that are in our cluster have to be the same model. And the reason for that is, one of them is a standby and then you have N number of active gateways in that cluster. So N plus one, one of them goes down to stand by needs to take over for that one. It needs to be the same model with the same resources that can then take over for that. But that's the general purpose.

Brian Anderson:
We try to keep all of our gateways that are in a cluster on the same land, very close in proximity. So if you do have to fail over to one of them, it's not now failing over to another network, we're failing over to something that introduces some latency or other issues. So we try to keep them all together, like they're one cluster or one unit, if you will. But that's a general topology and general design practice that I've seen, but happy to set up another session to talk about that in more detail, if you like.

Chris Detzel:
It's very interesting. So the next question is not on the community, but it's here in Zoom. It says, hi, Chris, we got ASO misconfiguration agent alerts so often we want to know in what situation we need to set ASO to true in the agent advanced configuration. Our case is we didn't see ASO configuration in the sqlnet.ora file, however, we heard ASO connections enabled by default. So if you check your database and don't see ASO configured there, it means that ASO is enabled. We want to get some clarification on this and how exactly we need to look at, determine, set the ASO agent advanced configurations?

Brian Anderson:
If you're seeing that although those are things that are by default... Those encryption services, ASO is being enabled by default with newer versions. So it's something that if you see those alerts, it's an indication you do need to enable that with the agent would be my response to that. So that's no longer something that's configured to the TNS. If you don't see it there, then you do see the alert, it would be an indication you do need to put that in place in the advanced config or just consult or assess with whoever administers the databases to verify what those configurations are, because that's should be something that is a known quantity or known configuration within your database group. So if it's not something that's available to you and you don't have access or visibility to what those configurations are but you're responsible for monitoring it and you see those alerts, that's an indication that we need to put that advanced config in there.

Paul Aiuto:
Yeah. I want to only just piggyback on that over Brian as well. The newer agents are, I believe it's 13 and above actually who support that natively. So the ASO and the Microsoft Diffie-Hellman, the agents are by default in there. We do have a document out on the docs and a private account about ASO support. Also we can post it in the community as a followup as well.

Brian Anderson:
Great. Thank you Paul.

Chris Detzel:
So we have eight minutes, so we'll try to get through as many as we can, and luckily we will make sure these all get done and then you can go to the post. So another question, is it a correct way to calculate server load based on the server cores?

Brian Anderson:
All right. So we've talked a little bit about performance and sizing. That has been a historical way to generally size an environment based on the number of cores, 100 IPUs or transactions per core is a general statement to kind of size an environment, but as we've discussed here, it really breaks down to the topic that you see after the fact and implementing performance monitoring. Because even though you size based off of that, if I have 128 cores on a database and I'm only 5% utilized, I don't need the gateways, or if I'm 80% utilized, I need more gateways. So it really depends on the environment utilization, the traffic consumption perspective there as far as what sizing should look like. But again, if you know nothing about it, cores are a good way to do a rough measurement of what the capacity could be.

Chris Detzel:
Good. By the way, Alex or Alejandro, if you could put a pic on your profile, it'd be great. Anyways, hi. Lucky is new to Imperva, is there an add-on that can allow me to see the before and after fields on database tables using Imperva.

Brian Anderson:
So any modification that's made looking at the previous state we don't really have a show me what was there before I modify this because we see the query as it happens but what we can do is output all of the records that we see and that can be in sequence so we could output that to a SOM or somewhere else that would say, "Hey, here's everything we saw from a modification perspective." So if you knew today that the new value is X, tomorrow you see another new value, if we output those in sequence, that would be a way to go do historical reference on what that value was in every value that changed over the course of time. So we don't really have a state machine to keep track of those things before and after, but we can certainly simulate that by outputting those events and giving you a way to do that in sequence with some kind of other storage like a SOM or elastic or something along those lines.

Chris Detzel:
Cool. Thank you. Now Miharaj posted something, we're going to come back to this question because we've answered one question from him. And so James has a question which I have not... He's had a hard time looks like getting on the site. So what is the timeline for the no longer needing to register the public key in order to deploy an agent. We heard that there will be a change from kernel space to user space deployment, is there a timeline for this?

Brian Anderson:
So we are in the process of releasing user space agents and we are doing that in sequence by operating system. And so I think that there's a few of them that are out there. I need to check with the product group to see which ones are available and in what order they're releasing them. But the idea is to come out of the kernel and get into the user space, making our agent code more reusable or more applicable. You don't have to have as many agents because you're not tying to the specific kernel anymore. So I can get back to you after I work with our product group on updates on all of the agent coverage that we have and where we are in that process. But that is certainly something is the direction that we're going with our agent design. And we are already in that process. So some of them do work in the user space today and then others are still in the works. So I can get back to you on the community with confirmation on those.

Chris Detzel:
Yeah, James, and I'll post this on your behalf. So if you're part of the community, hopefully I could see what your last name is and I can just post on your behalf or else I'll just post it under my name. So another question around user space agent with the change from kernel space to user space deployment, is there going to be an increase in requirements for agents? So RAM, CPU storage, usage and server, and will we be able to push the new user space agent and part of an agent via the GUI push? So this is a long question, but I'm going to continue to say it with the new agent being in the user space, we will need to register the public key if the server... Will we really need to register the public key if the server has secure boot enabled. With the new agent in the operating system was updated with the users pace agent installed, we would not need to upgrade the agent for compatibility, can you please confirm that? And if you need me to repeat any of this, let me know.

Brian Anderson:
Right. So I think that the first part of the question was if I recall correctly, it was around... Could you reread the first two parts of that because that is a good amount of words in there.

Chris Detzel:
With the change of kernel space to user space deployment, is there going to be an increased requirement of the storage for the agent, so RAM, things like that.

Brian Anderson:
The expectation would be no, that you would not have to allocate more resources on the server to be able to run the agent. It shouldn't be a performance impact that we're experiencing unless... I mean, the expectation is that it would operate the same way.

Chris Detzel:
Okay. Will we be able to push the new user space agent and part of an agent via the GUI push?

Brian Anderson:
Yes, that should work the same way. We have an installation manager and then we have an agent. And the installation manager is responsible for brokering the version and upgrade process for the agent package itself. So those are two different installers. One of them is you can just push your installation manager out everywhere and then from the GUI itself, then push your agent manager, agent versions from the GUI. So that should be a seamless process with our software update feature set.

Chris Detzel:
Cool. So I'm going to finish his question and then I've got to finish up, but with the new agent being in the user space will we need to register the public key if the server was secure boot enabled?

Brian Anderson:
I need to validate that and get back to you unless somebody else on the call knows that specifically. I need to evaluate with our product group and get back to you on that comment there.

Chris Detzel:
I assume that's a no. So we'll get back to you. With the new agent, if the operating system was updated with the user space agent installed, would we need to upgrade the agent for compatibility? So can you please confirm.

Brian Anderson:
Right. So historically if you were to update the operating system, it may change the kernel. So you would have to then upgrade the agent to support that. And that's the entire value proposition of the reason that we're going to the user space agents is that you wouldn't have that same level of dependency. So you'd be able to run your updates in the operating system and not have to change your agent near as frequently or at all for that matter, because you don't have the same level of dependency on a specific version of kernel.

Chris Detzel:
Great. So is this the case for patching event as well?

Brian Anderson:
That's correct. So if you apply patches or you modify the OS the user space agent should be able to sustain and it won't have that same level of dependency based on the underlying kernel.

Chris Detzel:
And last question and sorry, we couldn't get out to all of them. This hour just went crazy. Will the Imperva MX version 13 be compatible with the user space agent?

Brian Anderson:
Again, I need to validate what versions we have for the use of space agent, because I know that we're in a lifecycle of releasing those and developing those. So my thought would be 13 is... We're now at 14.2 and 13.6 and 13.5 have been out there for some time. So I would think that we would need to upgrade to a new version, but again, I need to validate that with our product group. And let me get back to you on a specific supportability for user space agents as a whole for all of our versions and where we are across all the operating systems as well. And I'll post that to the community here just for your visibility.