Your Advanced Bot Mitigation Dashboard is full of tools you can use to stop bots in their tracks.
Cloud WAF users with Advanced Bot MItigation enabled have a powerful tool at their disposal.
After logging into My Portal, you are one click away from leveraging some of the most sophisticated bot protection defenses on the market. Simply launch Advanced Bot Protection and you will be taken directly to your account’s user dashboard.
This dashboard is your entry point into managing your bot protection framework. This is where you are going to establish bot policies and review the effectiveness of the mitigation policies you’ve put in place.
How Cloud WAF Advanced Bot Mitigation Works
Imperva’s bot mitigation tools use policy guidelines to identify suspicious users. If a particular user’s behavior meets the guidelines of a malicious bot, the mitigation policy goes into effect.For browsers, the Imperva Bot mitigation solution requires users to support two technologies to access web content:
- JavaScript. All mainstream web browsers support JavaScript. JavaScript injection works by assigning a unique signature to each user and tracking their interaction with your web page. Most bots do not run JavaScript, and those that do will have to give up user information that will lead to them getting caught.
- Cookies. Asking a user’s web browser to send back a validated security cookie can help differentiate legitimate users from automated bots. Many bots do not save cookies, while sophisticated bots will end up having to send fraudulent information at odds with their actual behavior.
Check out the Cloud WAF Advanced Bot Protection webinar for an in depth look.
When taken together, these two technologies allow the bot mitigation system to create user-specific profiles describing each user’s device specifications, user behaviors, and other data. The system can then pinpoint security anomalies across large populations of users over long periods of time.
Users attempting to browse Imperva-protected websites without supporting JavaScript or cookies will simply receive a message saying they must enable these technologies to view the web page.
It is important to mention Imperva’s bot protection can also be used to protect API endpoints that are used by both web browsers and mobile applications developed on iOS and Android. In order to enable protection on web endpoints that mobile applications use, the Imperva bot mobile SDK must be integrated into your mobile application. The mobile SDK works in a similar way as the JavaScript challenge where the SDK will ask the Imperva bot mitigation integration point for a challenge information, the mobile device will complete the challenge with postback information to the integration, and then the integration will return a token that can be used during the course of the users’ session.
How Policies Generate Data for Reports
Before going into the reports available on the Bot Mitigation dashboard, we should cover how your specific policies turn user behaviors into report data. Your bot mitigation policy may allow you to control how to take an action on automation. With CloudWAF ABP some of the actions available are to monitor the traffic, captcha the automation, or return a block page.
No matter the action that is taken (aka directive that is used), the system will identify the characteristics that led to the action via the flags and tags applied to the request. Every bot mitigation policy you set is based on a specific set of Match Criteria. For example, if you block a user for exhibiting undesirable bot-like behavior, then that event will be flagged accordingly with the specific flags or tags. If you block a user for accessing your website from a disreputable IP address, the request will be flagged differently in the logs.
Every blocking action generates data that will then be shown on your Advanced Bot Mitigation Dashboards and the reports included in the dashboards. These data will give you detailed insight into the effectiveness of your bot mitigation policies.
Dashboard Reports
The first report you will see upon opening your Advanced Bot Mitigation dashboard is the Traffic Overview. This report graphs the number of blocked users over time, displaying the volume of bot mitigation actions over 15-minute intervals.
Just underneath the main traffic report you will find Custom Tags over Time. The latter of these is important because it will show you how many security incidents have occurred matching the specific custom tags you set in your bot mitigation policies.
Since your custom tags will generally reflect more complex bot behaviors informed by real use case scenarios, this graph can be helpful for gauging the effectiveness of recent changes made to your bot mitigation policies.
Further down you will find Machine Learning Threats over Time. This offers a fascinating glimpse into how specific models lead to items being tagged over time, and you can take action based on the patterns shown.
Imperva is constantly working on new ways to leverage state-of-the-art machine learning technology in its bot mitigation strategies. The greater the amount of data our system has as its disposal, the more in-depth its analysis can be. Comparing and contrasting the results of the various machine learning models at play can help you optimize your approach to bot identification and mitigation.
Known Violator Data Center Flags
Imperva’s bot mitigation solution can also identify bad behavior by assessing the reputation of regional data centers responsible for incoming web traffic. We maintain an up-to-the-minute database comparing traffic from known data centers, and automatically generate reports on locations that consistently generate bot traffic.
While having traffic come through an AWS data center or a Microsoft Azure server is not enough to classify it categorically as a bot, it can be suspicious enough to warrant a closer look. This will help identify the difference between legitimate users using virtual desktop infrastructure and illegitimate bots trying to scrape web content or conduct DDoS attacks.
Should You Block Data Center Traffic?
The specifics of your website’s security profile and user population will determine whether blocking data center traffic is worth doing. Organizations whose users are likely to use virtual desktop infrastructure may end up cutting important traffic off and angering users.
On the other hand, there are many industries where it’s extremely uncommon for users not to access web assets from their own devices. An e-commerce retailer, for instance, may find that there is almost no legitimate user traffic originating from data centers, and that blocking this traffic positively impacts their struggle against Grinch bots and other policy-violating automated users.
Distinguishing between the two can be challenging, but Imperva’s Advanced Bot Mitigation software also lets you see the specific URLs that data center traffic is targeting. This can help you draw conclusions about the legitimacy of incoming data center traffic.
Are these users jumping straight to a specific product page without navigating from your home page first? If they do navigate, do they appear to be using the exact same navigation method to get to a specific page? These are both tell-tale signs of bot behavior. Bot operators are running a business just like the website are targeting are. Bot operators will look for the most efficient way to achieve their goals, which may include steps that normal users’ behavior would include.
The Bad User-Agent Report
Another valuable report in your Advanced Bot Mitigation toolset is the bad user-agent report. This is where you can see exactly what kind of automation software is being used to access your web assets. You can see the number of Python requests, cURL activity, and other automation that is not taking even the most basic steps to avoid detection in one place, ranked by severity and volume.
This report also lets you identify neutral bot behavior that you may not necessarily wish to block. You can see the types of request paths that the system has caught bots using here, and you may be surprised to see automated activity on pages that simply don’t represent a great value to illicit bot operators.
If you see a request path leading to “/healthcheck.html”, for instance, you might be dealing with an automated load balancing application that is of tremendous use internally, and highly unlikely to be the ultimate attack point for a malicious botnet. This is an example of the kind of automated user agent you may want to whitelist. I would recommend that you follow up internally with your Operations or Product Development teams with questions about the legitimacy of these types of requests before taking an action other than “monitor”.
Learn More with Imperva Community
The Imperva Community is a great place to learn more about how to use Imperva cybersecurity technologies like On-Prem WAF, Cloud WAF, Advanced Bot Protection and more to establish efficient, secure processes for enterprise networks. Rely on the expertise of Imperva partners, customers and technical experts.
Related Content: