Chris Detzel: (00:14)Welcome everybody, my name is Chris Detzel, and this is our first Community Online Webinar. I'm very excited about today's topic with Brian Anderson, Director of Technology, and our focus today will be GitHub Tools- Imperva API Composer: Open Source On-Prem and Cloud WAF Request Call Automation. That's a lot to say, but I could have done a better job on the topic. But I think that you're in for a treat today. I had the opportunity to talk to Brian about three or four weeks ago around this topic. He walked me through the tools and it's just jam-packed full of really great stuff and so I asked him to do an online webinar that's live.
Chris Detzel: (01:00)And so let me go here, there are some rules, so keep your audio on mute if you have questions my ask is that you ask it in the chat and or go to community.imperva.com. The reason I show this is I want you to... can I take a look? I added a post here, right? And you go in here and just reply and ask your question, and it might take a couple of days to get to the answer but I will have Brian and maybe some others that know some of the answers to the questions go in and answer those directly.
Chris Detzel: (01:43)Again, you can put those in chat. I will capture those in chat and we'll try to answer some of these questions within the Q and A. Again, the webinar will be recorded and it's being recorded now. We will share this to the community once we kind of fix it up a little bit. Now, I am looking at doing more of these. The goal is to do it every other week, maybe even more. I'm trying to find experts within Imperva, but I'm also looking for the audience for volunteers to also help with some of these webinars. Some of these could be like how-tos and things like that around Imperva products, tips, tricks, those kinds of things. Email me directly if you're interested in contributing to the community.
Chris Detzel: (02:34)The last thing I would say is a promise to promote this other event that Imperva is having. We have a Imperva Brainshare going on, that's April 21st and 23rd for people just like you and you can go in and I will share this all in the chat once Brian starts. Really great agenda, we are certainly looking forward to it and hope that if possible you can attend. With that said, there's so much more for me to say but I am not going to say anything else, but I'm going to let Brian take it away and he's going to share his screen. There really is no presentation except for what Brian's going to show. And Brian, I would just kick it off by saying, have it kicked off the same kind of, the goal here is the intention and what we're going to try to accomplish. That's fair.
Brian Anderson: (03:28)Thank you very much for the introduction and thank you for having me again. The intention for today is we're going to walk through some of the tools that are on
https://github.com/imperva. Some of those things would be focused around automation (
Imperva API Composer: Open Source On-Premise and Cloud WAF Automation) use cases and some of the open source packages that we've been able to release in the last two years. We'll do some live demos of how some of these things work and the intent here is that we can spark some conversation, hopefully continue that in the form of kind of back and forth contributions to the community and if there are new feature requests that come in or if there's some interesting use cases or if there is the opportunity to implement this in customer environments. That is the goal is for us to get more familiar with what capabilities are available there.
Brian Anderson: (04:09)If you go to
https://github.com/imperva, as of like May a year and a half ago or so, we were able to finally work with legal to come out with a license that allows us to release things in an open-source fashion. Since that time there was a ton of projects that are put on GitHub and a lot of the things has to do with automation and customization around an implementation of our security offerings. The first one we're going to talk about is one that's called the API Composer, and I didn't name it, it was something that we worked with marketing and others to come up with some name for it. But basically this is as a tool that helps you come up to speed with our APIs.
Brian Anderson: (04:49)Now, APIs are application programming interfaces. It's basically a user is to a gooey like a machine is to a machine. An API can enable machines to talk to other machines to perform different tasks and different operations. Part of the reason the APIs are leveraged is simply an automation and primarily these are used in the form of integrations. If you go to a website and you see a newsfeed that is typically an API call made to some other platform, and then pulling that data back in and rendering it in some way.
Brian Anderson: (05:21)In the realms of security, it would be really focused on tying security configurations into things like the application provisioning process. If I have a CI/CD process as a part of my software development life cycle, I want to programmatically an automated deploy all my websites and update configurations and so forth in that manner. We want to have security be a part of that build process and so the way that that can happen is for us to leverage the APIs that are exposed within our security offerings to have security accompany that application.
Brian Anderson: (05:55)What we're looking at here is the web API Composer tool. An installation of this is very, very straightforward, it leverages the technology I referred to as Docker. Is a containerized package basically that allows you to spin this up with a single command. A docker-compose up -d will have this run in the background and then you could simply have this running locally without having to have any other packages running. Well, the install is very, very simple.
Brian Anderson: (06:23)Once you get up and running there's a couple of tabs here. We have an On-Premise offering referred to as SecureSphere or
Gateway WAF now as we call it. And then there's a Cloud Web Application Firewall offering as well. So this tool is very similar to, if you're familiar with something called Postman, it allows you to interactively test API calls. One of the reasons that we put this tool together is that it's not always convenient or easy to test some of these calls because of the number of parameters and how dynamic that is. But you can simply enter in your credentials here and put in an endpoint.
Brian Anderson: (07:02)If you were to say, "I want to add my own, if I can edit Cron I can put in my endpoint or I can select one that I've already I've already configured. It'll log me in and what this does it pulls down the schema dynamically from our Mx and tells you what calls are available in the form of a dropdown. Now, if you go to
https://docs.imperva.com/, there's also some excellent documentation that can walk you through the capabilities of this, but the idea here is that we want to be able to do this interactively. If we are familiar with a management server, some of the things that we would want to configure within our management server or MX with our on premise solution would be to configure something called the site tree.
Brian Anderson: (07:42)Now, this is the... the site tree of course, my sessions logged down, let me log back in just a moment. But the site tree basically reflects the infrastructure that we're looking to monitor. We have in our terms a server group, which is kind of like Layer 3, a service Layer 4 or 5 and Layer 6 and 7 is the application. If we're looking to secure these applications, we need to have entries here in the site tree for anything that we're looking to protect, right?
Brian Anderson: (08:07)One of those tasks that you may want to do is tie into your CMDB and any new web properties or any new assets that are entered there we might want to create that automatically in the manager to say, "Hey, I'm going to protect this site as well." By scrolling down here I can look at creating a new server group, for example. I can simply say, "I'm going to add a new server group and this is my new application." Now, one of the interesting things about the way that our API is structured is that we put these dynamic values into the URL itself.
Brian Anderson: (08:39)I can say, what this tool does it pulls those values down on the fly. Maybe I want to put this into a site called Imperva One and I'm going to add in my new server group right here and save. This allows you kind of put these values directly into your requests kind of on the fly without having to copy and paste things directly from the MX. Now, there are four methods that are exposed to your post as a create, get as a read, put as an update and delete as delete and so you can use those respectively. But if I were to do this now, I've just created a brand new server group in my Mx and it gives me a 200 response, that's a successful response there.
Brian Anderson: (09:20)But that's kind of the ideas that we'll be able to look at all of these different calls that are available to us and understand and learn and interactively test against this to figure out how those things work and how I can leverage them. The hardest part about implementing integrations like this is really bringing the developer up to speed with how the scheme of this API works, and the more tools that we could put out to make that as easy as possible, the easier the implementation is going to be and the easier it's going to be for those folks to get the those integrations up and running.
Brian Anderson: (09:51)There's a lot more that we could dive into as far as what calls can do what, but basically you can manage policies, you can manage dynamic policy definition with datasets and you can, again, integrating your site tree with your internal assets for example from a CMDB is an important use case there as well. But that's kind of a high-level of the on-premise Gateway WAF portion of this. In the very same UI, the next tab over, we have our Cloud WAF capabilities as well. And so in the very same similar usability perspective we have all these things presented to you in the form of a dropdown.
Brian Anderson: (10:30)I can say I want to look at all the sites that have configured in Incapsula that were in our Cloud WAF environment. I'm going to go down and say, "Give me a list of all my sites that I have available." If I do this it gives me an example of what that response can look like. Within the cloud environment, maybe something similar whenever you onboard a new web property you want to have DDoS protection and CDN capabilities and security rules that are defined there as well. And so this is something that we could tie directly into, again, your CI/CD process or whatever your software development life cycle is, natively integrating with a lot of the tools that manage that.
Brian Anderson: (11:10)Typically, there are configuration management systems like Chef or Puppet or Salt or Ansible or things along those lines that will help administer and manage that life cycle of deployment and configuration. And this is something that we can introduce a couple of calls in that process to automatically give the sectional security coverage right alongside the deployment process of your applications there as well. Those are at a high-level the ability to just come in and test interactively what calls are available and how to configure those.
Brian Anderson: (11:44)These tools will automatically render all the parameters that are required for each one of these calls kind of on the fly. And it just like in the SecureSphere or the Gateway WAF portion of it, will pull down lists of what sites are available and allow you to dynamically select them within a list because I won't always know off the top of my head what my site ID is, but I will know the DNS name of that, so we presented to you in a easy to select type of format that enables you to test that kind of on the fly.
Brian Anderson: (12:12)We've covered the on-premise Gateway WAF portion and also the cloud portion of that. The next step of this is potentially moving policies from your on-premise environment over to your cloud environment, right? We also have this section here, migration of WAF security policies over to Incapsula. And now, there is not 100% parity between some of the things that you can do from your on-premise Gateway WAF environment to your cloud environment. A couple of the predicates that are in policies don't exactly directly translate because just the technology is slightly different. But if I do convert these it'll tell me which one doesn't work or the ones that do work.
Brian Anderson: (12:54)So I do have the ability to say, "Select them in bulk, convert them, and there's not that many in here, but this just gives you the ability to convert in all the custom policies that I have and I can just move them over. If I want to convert and save, this will save those locally here within the browser. This will tell me that I already have those but it's going to overwrite them and I'll show you where those actually get written to here. That's where after I converted those policies over to Incapsula format, under settings here we have all of these policies that are stored locally.
Brian Anderson: (13:26)Now, what's interesting or important about this is that in addition to being able to export and convert Gateway WAF policies, I can then migrate those and provision them in bulk within the Incapsula environment. Under settings here I can say I want to create a group of these policies and maybe push those over to one or multiple sites and Incapsula. The migration tools tab gives me the ability to select what I want to actually migrate. If I want to migrate an individual policy or a group of policies to an individual site or a group of a group of sites.
Brian Anderson: (14:02)Let's say that I have an enterprise set of policies, there's 10 or 15 or 20 of them from one site and I want to migrate those to any number of other sites, I can now simply go to this portion, Policy Groups, and say, "I want to take all of my enrichment policies that I have." These are things that could potentially add insights from Cloud WAF and then it will give you long lat values, what the client type is, city, state, zip, etc. This is enrichment we can pass down to the origin servers. Maybe I want to have that across all of my sites with Incapsula, giving your origin servers and the applications a little bit more insight as far as what's seen at the edge.
Brian Anderson: (14:41)I can create a group of these called, Enrichment Headers, and under migration tools I can say, "Let's push my policy group of enrichment headers over to an individual site or a group of sites." And in this case I'll just select one of these sites here. And if I run this, the idea is that it actually does make these calls and it tells you in the form of a log here what the exact syntax is that it's using to do this. This gives you two things. One is the ability to actually push this out in bulk if you want to, the second part of is it tells the developer exactly what's happening under the hood to be able to do this in their own environment, in their own configuration management system if they wanted to. This can directly translate to something like the Ansible, Chef or Puppet or Salt and so forth. But it basically gives you the ability to manage things in bulk across lots of different sites.
Brian Anderson: (15:29)And so that's kind of a high-level. We've covered the API for Gateway WAF and how you can leverage this. We've covered the Incapsula API as well and then some tools that you have to work between both of those. The last part that I wanted to cover here is under Incapsula sites. This gives you the ability to kind of look at an account you have, and it kind of outputs all the configurations for what sites it's seeing here. And I could page through these a hundred at a time, but what's interesting about this is I could look at an existing site that has a bunch of these policies there and I can also save these down into the tool here as well.
Brian Anderson: (16:03)Even though I have one call, this is going to override it but this is actually saved into the local storage of the browser. It's just a poor man's database basically, but it makes it simple so you don't have to actually have a database running deep. This is the ability to kind of look at all your configurations there, I can save them down and then I can reuse them in the form of these migration tools that are there. In the interest of time I wanted to cover a couple of other projects that we have here as well. And again, happy to follow-up with any questions at the end of this with anything that that does come up, if there's some more interest around that.
Brian Anderson: (16:37)The second tool that I wanted to cover here is another API tool that we have, but it's CLI, the CLI for Incapsula or for our Cloud WAF environment. It really enables you to do a lot of similar things only this just gives you... instead of having to write this code yourself, it gives you the ability to run very simple commands. I could do in Incap, it works very similar to the Amazon's command interface, like AWS space, Ec2 space, whatever. This can do Incap space and here's our options we have available to us.
Brian Anderson: (17:09)Let's say I want to do an Incap site and then I help on that and it tells me what my operations or options are, right? I could say I'm going to list all the sites that I have available to me. This tool will enable you to manage your policies, create sites, manage your configurations, also check the status of things, and it does that in a programmatic fashion. So where if you do use configuration management systems, maybe it's easier for you to construct a very simple command like this as opposed to having to manage the web requests yourself in whatever language that you're implementing that in, right?
Brian Anderson: (17:43)I could say, "Give me," I could do a status on this site here, and it gives me all the configurations associated with this site. And so again, this makes it really easy to integrate into a process for deployment provisioning and then syncing configurations, it's very, very straightforward in that way. But another thing that I wanted to share is that we also have the ability to export site configurations and also restore a part of the development process here. Actually deleted a site configuration, but I already had them exported in the form of chason.
Brian Anderson: (18:18)What this does is you can do a restore for a previous configuration of a site or I can export this as a Cron every day, for example, and I could see exactly what may have changed over the course of time. What I'm going to do here is just recursively export all of the site configurations that I have for this account. And again, this is something that could run in the background, but it's going to give me the ability to see what happens over the course of time if I were to commit these into a version control system, like GitHub...
Brian Anderson: (18:53)I'm hearing this directory that has all of these sites on files and these are the different site configurations that I just exported. If I were to do a git status, on this I see that there's a lot of new files that were introduced, but there's also some other ones that were in here previously, right? So if I do a git diff, for example, against the site configuration, I see that what's in red has been modified, what's in green is new. It used to be alert only, but now it's block... and programmatically over the course of time, what has been modified within my site configuration, but it also gives me the ability to restore anything that may have been deleted or something that happened in that manner.
Brian Anderson: (19:36)It's just an interesting capability. Again, it's available today, it's free. There's no reason to not do it, it's a great utility and good practice to have in place. But that is the command line interface for Incapsula. Again, a lot of great packages on GitHub, those are two of them. Another one that I wanted to cover here is we have recently completed... are you guys still there?
Chris Detzel: (20:02)Yeah, I can hear you, Brian. Yep, I can hear you.
Brian Anderson: (20:12)Interesting. Just a moment. Let me see if I can jump back on here. I apologize about that. One of the thing I wanted to share with you guys is that we recently announced the completion of a Terraform module with HashiCorp. And so this is the ability to leverage Terraform to manage configurations. I'm sorry about my connection here, just a second let me see if I could jump back on.
Brian Anderson: (21:13)Fantastic. All right. One thing I was discussing here is the new Terraform package. Terraform is basically a cloud-agnostic automated provisioning tool that allows you to automate the management of pod environments. This could be for Amazon, this could be for Google, it could be for ESX or whatever, whatever kind of a hypervisor that you're using, but then it also incorporates other packages, other vendors to be a part of that ecosystem. If you wanted to deploy a site on an instance in Amazon and create a load balancer but then also configure Cloud WAF or also configure other security products to be a part of that build, this is simply a pod-agnostic method of being able to do that.
Brian Anderson: (22:00)We have recently announced this custom provider type here and giving you some configurations that you can then deploy this in the form of, it's called Infrastructure as Code. Basically, when I create a HCL file or HashiCorp config language file, it basically just has some configs that look similar to this. You create resources and then you can manage the state of these resources with Terraform. They're very similar to what you saw with the commandment interface, just being able to run a command will then make something happen on the Cloud WAF environment. Terraform gives you that ability to do that, only you can stand up entire regions with a single command from one cloud environment in Amazon over to Google for example, and all of the associated configurations can accompany that.
Brian Anderson: (22:47)Very, very straightforward way of, if you haven't worked with Terraform it's pretty awesome. I have a little quick demo here that I can do for you the way that Terraform works. In here we have a main .tf and that is a Terraform file. If I jump over to what is in that file, basically it's a bunch of resources. In Terraform terms it allows you to create different security rules and if you're looking at the policies that we just migrated with the composer tool, a lot of these are very similar, so I can change things from a block state to an alert state or I can create a bunch of custom rules. This, if I don't have a header, add the header and so forth.
Brian Anderson: (23:28)These are a lot of the exact same policies that we saw from the commandment interface as well as the API composer tool are available here in the form of config language, HashiCorp Config Language, HCL. In here the way that Terraform works is you do a TFE in it, and any resources that you have defined or any modules that you looking to use it will go ahead and download those for you. And in this case I'm using Kubernetes, I've Incapsula and the AWS modules for this particular project. And then if I do a plan, my plan will tell me what Terraform is actually going to go build.
Brian Anderson: (24:07)In a very simple definition of a configuration file, again, Infrastructure as Code here, I can have one thing that I define locally and then as soon as I want to create that I can run a terraform apply, and so the plan will simply tell me what it's going to do. And currently it's already created these but if I were to do an apply, this would then go and push all these configurations both to Amazon, route to these three configurations, secret manager within Amazon, but then also Incapsula rules directly as a part of that same provisioning and creation process. I can do a state list.
Brian Anderson: (24:42)The way that Terraform works also is not to get too deep in the weeds here, but it writes everything locally to a state file, so it basically has a way to remember what it did before. If something changed in your cloud environment or something changed locally in your configurations, let's say you want to increase a cluster from two nodes to 20 nodes, you change one thing in a local config file and hit apply and it would go automatically and create and spin up all those additional instances to increase that capacity. But at the same time, I can now manage all of my... thousands and thousands of security policies on a per site basis programmatically like this.
Brian Anderson: (25:23)Now, they don't necessarily need, in my opinion, all those rules but from a compliance perspective to do like for like they want to migrate from another environment with the exact same policy configuration, well certainly they can do that programmatically in this way. That's another pretty exciting announcement that we have. If you haven't used Terraform it's a great tool, I highly suggest looking into it. But we have a native package that whenever you download Terraform you get all Incapsula providers had brought alongside that.
Brian Anderson: (25:52)That's kind of a high-level overview of what Terraform is, how it works and how Incapsula and our Cloud WAF environment natively integrates directly into that. The last package I wanted to talk about here on GitHub is something called the MxToolbox. And again, there's a lot more in here that we can get into, but I'll just give you kind of a high-level of some of the things that are available here. The MxToolbox is just kind of a hodgepodge of a bunch of things that we've done within the MX that are customizations that have been really useful in customer environments.
Brian Anderson: (26:20)For example, exporting reverse proxy rules in CSV it's not a native report that's easy to export in that way, but this gives you something that can output that in a very, very easy to run type of a scenario. What I wanted to focus on is the gateway performance monitoring script that we have here. There's something you could drop onto your gateways if you have on-premise WAF appliances that would give you visibility to the utilization and performance on those boxes as well. This will pull out disc utilization memory, the throughput, this being processed both at the appliance level as well as on a per application level. And what's interesting about that is that this gives you the ability to understand what applications are consuming how much bandwidth and throughput within these environments.
Brian Anderson: (27:08)And let me just go ahead and pull up a screenshot of what this dashboard can look like. Based on the data that we're outputting here... and of the visibility and into the performance of your environment. Again, going back into, not to get too nerdy here, but this is an example of the kind of level of granularity of performance information that you can pull out of your gateway.
Brian Anderson: (27:30)And if you do this, and again set it on a Cron to run every minute, you now have over the course of time you can output this and get a good visibility into the performance monitoring within your environment in that way. These are just really useful tools and packages that are available. There's lots of other things that we're working on trying to more packages within GitHub, but that's kind of a high-level overview of what I had planned to cover today. So with the time left are there any thoughts or questions so we can help field?
Chris Detzel: (28:01)No questions yet in the chat, but just to confirm this recording has been recorded and I'm going to keep it open a little bit for chat. Now, something I did just now and I should have done this in the beginning, but just to make it more interactive from a community standpoint I was trying to get people to talk a little bit about themselves. There's a question, Brian, what about the web API coverage by all tools?
Brian Anderson: (28:30)The web API coverage, are we asking for... can you elaborate a bit on that question? Are we asking for everything to be exposed via the API or what's meant by that question?
Chris Detzel: (28:41)Yeah, let's, if you want to open up your mic or just type it here that'd be great.
Speaker 4: (28:50)Yes, do you hear me?
Chris Detzel: (28:51)Yes.
Brian Anderson: (28:55)I can hear you.
Speaker 4: (28:55)I wanted to know what feature do you call by each... I mean for APIs that are direct from a provider?
Brian Anderson: (29:05)Oh, for the Terraform provider. The Terraform provider, we are working to fully cover everything that our API offers. The composer tool, we're in the process right now of adding in all of the latest V2 APIs that are introduced as well as our API security offering, and we're in the same process working on implementing those on the Terraform side. There are two or three things that we need to implement, but by and large the vast majority of those capabilities are already implemented within Terraform. But it's, we're actively working on as soon as the APIs were introduced, also implement those within Terraform and then we can input that natively into repos that are managed by HashiCorp, so you can get those natively as soon as we release them.
Speaker 4: (29:51)And what kind of support we can expect from the tool as it's freely released?
Brian Anderson: (29:58)I'm sorry, what was the question?
Speaker 4: (30:01)Is there any support on the tools because it's... you freely release tools you'll get an account?
Brian Anderson: (30:08)Right. It is an open source project and we have even received pool requests from people in the field that have said, "Hey, I wanted this new feature, here's a code example of how I propose to implement that." It's an open source project, it's community driven, but it's certainly our intent and what we've been doing is implementing those new APIs that are introduced as quickly as we can and we'd like to keep and maintain the Terraform package capabilities to be consistent with what our API capabilities are. So if this new functionality is introduced, we're actively working on implementing those things, but it is a community open source project, there's not necessarily a formal support for the Terraform project.
Chris Detzel: (30:50)Thanks, Brian. A couple things here is one of our partners, so we have both customers and partners and employees on the line. One of our partners said, Speaker, "I might have a project to export bulk sites to a new MX and these resources seem gold," so that's really great feedback. Thank you, Speaker. and we do have a question from [Speaker 2 00:31:10]. "Performance monitoring under MX tool is only for WAF Gateway or for data security as well?"
Brian Anderson: (31:16)Oh that is also for data security. It will take the counters, if you're familiar with gateways there's a proxy in these counters that basically tells you what is being processed, how many events and so forth. It'll work for any gateway that we have and reach out to me separately and I can certainly work with you on getting that in place, but it's pretty straightforward to put in the bar user data folder, which is the folder that will survive upgrades, but it works for both a WAF and database gateways.
Chris Detzel: (31:44)Great. There's no other questions. I'm going to, if you want to go ahead and let me have the reigns real quick, just put a reminder. And if there are other questions just let me know before we head off here. But a couple of things here. Remember a question, somebody already asked a question on the community. We always try to answer these questions as much as possible, so please ask your question on the community. If you don't get it in today, I really appreciate that and then we will have other upcoming events here in the near future.
Chris Detzel: (32:21)I would like to do some more of networking type events to where we have people like yourself share a little bit about yourself, what you're doing and things like that. Lastly, you remember about the Imperva Brainshare, so if you can attend that, I put that in the chat. I can also put that in the notes as well. I think we might have another question. Brian, let me look. Oh no, no another questions. Anything else, Brian? This is really great, I really appreciate it.
Brian Anderson: (33:00)Yeah, I just wanted to say thank you everybody for joining and certainly looking forward to starting up more conversation in the community and certainly interested in hearing more about interesting use cases that may come up around automation and wanting to support the field in that way to give you guys the tools to make it easy to use and implement our products.
Chris Detzel: (33:17)Thanks Brian. And lastly I'll say this is, Brian has several community blogs coming out specifically about this session, so look for that in the next week or two for the first one and then we'll kind of set those out for the upcoming weeks. By the way, we will record it, I will post that into that post that you see already at the community and then some of those will be embedded into the blogs. Everyone, thank you. We had people here from Japan and all over the world, so this is really, really awesome. I just really think that this is hopefully very valuable to you, and if so we will keep doing these things. Somebody said, "I'd highly recommend sharing the same info internally with your teams." We will, Danny, thank you.
Related Blogs:
#CloudWAF(formerlyIncapsula)#On-PremisesWAF(formerlySecuresphere)#Webinar#video#Github