Users and security professionals aren’t always on the same page when it comes to defining a “bad bot”.
“How do I buy a good bot so I can buy the sneakers I want?”
This question, which came up during a recent seminar on Cloud WAF Advanced Bot Protection, points to something incredibly important in the world of bot mitigation. It’s not always easy to tell what distinguishes a “good bot” from a “bad bot”, or what those terms really imply when it comes to using automation tools on the Internet.
There are entire capital enterprises built on the use of automation tools to collect, categorize, and analyze data online. Bots are incredibly useful tools for these use cases, but they are not always greeted with open arms by website owners.
The Case for Sneaker Bots
Sneaker bots make a great example of the popular misunderstanding of what bots are, what they do, and what constitutes bad behavior.
For the uninitiated, sneaker bots are automated e-commerce applications designed to purchase new, extremely-limited-edition sneakers the moment they are released. There is a considerable aftermarket value for limited edition sneakers – some experts predict the national sneaker resale industry will grow to $6 billion by 2025.
Collectors will pay tens of thousands of dollars for sneakers that originally cost a few hundred. This incentivizes enterprising sneaker resellers to invest in sophisticated automation software that allows them to purchase new products the very second they come out.
In essence, sneaker bots do (legally) what concert ticket bots do (illegally). Unlike ticket-scalping bots, however, they enjoy a high degree of cultural acceptance. Sneaker bot developers see themselves as providing a valuable service to the collectible sneaker community.
This is why someone might ask if there is a “good” bot that can achieve something that almost certainly counts as “bad bot” behavior.
Website Owners Are Responsible for Defining Good Behavior
Sneaker bots, like their more generalized Grinch Bot cousins, are not illegal. However, that does not mean that website owners have to accept them. Footwear manufacturers take increasingly advanced steps to prevent bot users from leveraging an unfair advantage when releasing products, and sneaker bot users continue to find innovative ways to get around those obstacles.
Every ecommerce website’s Terms and Conditions page lays out what they define as bad behavior. Most reputable retailers do not see scalpers and resellers as customers who represent long-term value.
Users who use automated software to violate website owners’ expressly stipulated terms and conditions are acting in bad faith. The software they use to do so meets the textbook definition of a bad bot.
Good Bots Don’t Deceive Website Owners or Bypass Security
The Googlebot is typically presented as the number-one example of a good bot. If a website owner doesn’t want the Googlebot to crawl their website, all they have to do is tell it not to.
Editing robots.txt is enough to accomplish this. Google is not going to try to circumvent the policy outlined in that document.
On the other hand, when retailers put up obstacles that prevent bot users from instantly buying their entire stock of a new item the moment it launches, those users do not simply give up and go home. They find ways to bypass whatever system the retailer has put in place.
For example, when streetwear manufacturer Supreme started using CAPTCHA technology to stop bots, developers responded by allowing users to log into their gmail accounts. The particular CAPTCHA solution Supreme was using would not present CAPTCHA challenges to users with legitimate-looking activity on their gmail account.
The fundamental premise behind this behavior is deception. Bad bots fool website owners into thinking that legitimate human users, equipped with nothing more than a keyboard and a mouse, are accessing their websites and getting past whatever bot mitigation solutions are in place.
Good bots make their intentions clear and follow the rules laid out for them. They do not hide the fact that there is automation at play. These users might be using popular health checking or monitoring tools with transparent, well-established use cases, or they may be leveraging virtual desktop infrastructure in a way that is neutral to the website’s fundamental business premise.
Use Whitelisting to Keep Good Bots on Your Good Side
Imperva’s Cloud WAF Advanced Bot Protection stops all types of automation. It offers comprehensive protection from web scraping, sneaker copping, and Grinch bots, and it can very well stop legitimate bots from doing their jobs as well.
Whitelisting is key to making sure that good and neutral bots are able to work without running into bot mitigation obstacles. Every website owner may have a different set of commercially appropriate use cases for automation (which should be outlined on their Terms and Conditions page), and it is up to their security professional to identify what kinds of user signatures correspond to that behavior.
Doing this requires gaining deep, granular insight into what bots are doing on web properties. Security professionals need to identify which resource paths are valuable for bot operators, and which ones are not. This is especially true for enterprises with internal operations teams that use bots. Being able to collect and scale data from your own web asset using a bot can drive immense value for your operations team – but those tools will have to be whitelisted and monitored.
Advanced Bot Mitigation Ensures a Level Playing Field for Users
The primary case against automated ecommerce user software like sneaker bots and Grinch bots is based on fairness. The ability to use automation to access newly released, limited edition products and then resell them at a huge markup is fundamentally unfair for the broad majority of legitimate customers. Bot mitigation is just one of the ways that manufacturers and retailers can ensure their customers enjoy equal opportunities to purchase products with a limited inventory.
Learn More with Imperva Community
The Imperva Community is a great place to learn more about how to use Imperva cybersecurity technologies like On-Prem WAF, Cloud WAF and more to establish efficient, secure processes for enterprise networks. Rely on the expertise of Imperva partners, customers and technical experts.
Related Content: