Bot Basic Feature Management
Overview
Many public or commercialized programs, including search engine crawlers, have fixed or default User-Agent header features and have specific purposes. Bot Basic Feature management policies include most public bot type features, and you can directly manage bot tools that meet these features, which can help you:
1) Allow search engine crawlers to access and avoid being blocked wrongly;
2) Identify specific-purpose commercialized tools and limit their access.
EdgeOne will regularly update the features of automated tools to ensure that your management strategy continues to cover control scenarios.
Usage Scenarios
By default, the Bot Basic Feature management strategy is in a disabled state. When you have the following scenario demands, you can enable and adjust the bot basic management protection strategy as needed:
Control requests from IDC (data center)
Most of the access to To C applications comes from mobile networks, broadband providers, or educational networks, and normal requests do not come from data centers (IDC). Therefore, requests from cloud providers or data centers are mostly from proxies or crawlers. You can choose to control requests from data centers (IDC) and intercept or perform JavaScript challenges to mitigate the risk of malicious access.
Control valid bot requests with search engine features
Search engine crawlers are currently one of the few valid bot types. In order for sites to distinguish valid crawlers from search engines, most search engine providers provide the IP segment and UA features used by their crawler engines. EdgeOne's search engine feature rules include search engine public IP features, User-Agent header features, rDNS resolution features, and other matching methods. You can configure bot requests with search engine features to be released to avoid being intercepted by bot management policies.
Control requests from commercial or open-source tools
Commercial software or open-source tools often carry specific User-Agent features. EdgeOne classifies these automated tools based on their usage and regularly updates the corresponding User-Agent library. If you do not allow bot requests from these commercial or open-source tools, you can intercept them.
Adjust Basic Feature Management Protection Strategy
For example, in an e-commerce site
shop.example.com, to prevent users from placing orders and rushing to purchase through tools, you should disable the automated shopping cart bot. You can take the following steps:1. Log in to the EdgeOne console. In the left sidebar, click Site List. Within the site list, click the Site you wish to configure.
2. Click Security > Web Security.By default, it is a Site-level security policy. Click Domain-level security policy Tab, in the Domain-level security policy, click Target Domain to enter the Target Domain security policy Configuration Interface, for example:
shop.example.com.3. Navigate to Bot Management card, select Basic feature management in the UA feature rules, click the top right corner Rules.
4. On the rules page, you can modify the action for a specified rule ID individually. If you need to perform batch configuration, you can click Batch configure, batch check the rule IDs to be configured, select the action, and then apply. In this example scenario, you can control the automated shopping cart bot by modifying the action of this rule to Block.
5. Click OK to complete the modification.