In addition to analyzing the received client requests, identifying features in the headers and client IP, EdgeOne also provides an active detection bot identification method. Active detection can perform Cookie verification and session tracking on the client, as well as client behavior verification for interaction, and further identify whether the current visitor is a tool based on the client's interaction feedback. Active detection has the following advantages:
It has a strong identification effect on tools that can simulate browser behavior (such as: Headless Chrome, etc.).
Compared with other front-end verification methods (such as: CAPTCHA human-machine verification), the integration of active detection is less intrusive to the business, and users can hardly perceive it, which can bring you better bot identification results and integration experience.
If your current site service provides login/registration/payment services and has high business value (for example: you can obtain the value within the account after obtaining the account, and you can obtain scarce goods or services through payment, etc.), it is recommended that you enable active detection for key business interfaces.
Note:
1. Due to the characteristics of the active detection mechanism, before enabling it, please confirm that your business is a Web browser client, or restrict the active detection rules to resources that only allow Web browser access through matching conditions to avoid compatibility issues affecting mobile app access.
2. This function is still in beta. If you need to enable it, please contact us.
Supported capabilities
Active detection supports the following two capability configurations:
Cookie verification and session tracking: Through the HTTP session state (Cookie mechanism), a dynamic session token is issued to each visitor, and the visitor's request must carry a valid session token. In this way, requests from different visitors can be tracked and their behavior characteristics can be identified. In addition to verifying the legality of the Cookie in the request, Cookie verification will also identify tampered session information and high-frequency collection of Cookie information behavior, reducing the security risks caused by session hijacking.
Client behavior verification: Advanced automation tools (such as: Headless Chrome) can already simulate browser behavior. Client behavior verification will inject JavaScript code into the HTML response page, collect the client's JavaScript runtime environment, device environment, and client interaction behavior, and thus identify the tool environment and normal request visitors.
Client Interaction Process
Scenario 1: Intercept ordinary Web tool crawlers and access to the media site
Scenario Example
The media site media.example.com only allows H5 clients and browsers to obtain site content, and all legal clients support Cookies. Therefore, clients that do not support Cookies need to be intercepted, including crawlers that have hijacked other visitors' sessions. For clients that maliciously tamper with Cookies, use the silent mode to counteract, maintain the connection but no longer respond to requests.
Directions
1. Log in to the EdgeOne console and click Site List in the left sidebar. In the site list, click the target Site.
2. Click Security > Web Security . By default, it is a site-level security policy. Click the Domain-level security policy tab and then click the target domain name such as media.example.com , to enter the configuration page for the security policy of the target domain name.
3. Locate the Bot Management tab and click Add rule under Active detection to enter the configuration page.
4. Configure the judgment conditions and actions. In this example scenario, you can configure the matching fields as Request path (Path) matching the regular expression /* and Request method (Method) equals GET . Configure the operation to Validate cookie and the verification method to Update and validate .
For requests carrying no cookies or an expired cookie, configure the trigger threshold to 300 times within 10s and the action to Block .
For requests carrying an invalid cookie, configure the action to Drop w/o response .
For action details of the bot management module, see Action. Other related configurations are described as follows:
Configuration item
Description
Verification method
Update Cookie and verify: For requests that do not carry valid session information or have expired session information, EdgeOne will create a session in the response with the Set-Cookie header and continuously update the session information. It is recommended to use this verification method for paths accessed by GET.
Only verify: EdgeOne only verifies whether the session information carried in the request is legal. When the session information in the request expires or the request does not carry valid session information, it will not create a new session by updating the Cookie. It is recommended to use the only verification method for APIs accessed by POST (such as: registration, login, add to cart, etc.).
Validation result
For requests that have failed the Cookie verification, the processing can be done according to the check result as follows:
No Cookie or expired Cookie: The session information carried in the Cookie header has a time limit and is only valid for a certain period of time. If the request does not carry valid session information or the session information has expired, the session information needs to be updated to pass the Cookie verification. When the client frequently uses requests without session information to access, there may be a risk of harvesting Cookies and hijacking sessions. You can choose to dispose of requests from the request sources (client IP) that do not carry valid session information when the session information is not carried at a specified rate.
Trigger threshold: You can configure the upper limit of the number of sessions that can be created without carrying a Cookie or an expired Cookie within a certain period of time, and limit the initiation rate of new sessions. When the trigger threshold is exceeded, it will be processed according to the configured action.
Invalid cookie: The session information issued by EdgeOne has encryption verification capabilities, and tampering with session information often means malicious requests. You can choose to dispose of requests with tampered session information.
Cookie-based session check
Requests that pass the Cookie verification are divided into high-risk, medium-risk, and low-risk categories according to the specified rate features. You can configure different actions for each risk level to more effectively identify and mitigate malicious behaviors:
High risk: In a single session (corresponding to the same EO-Bot-SessionId value in the Cookie header), more than 1000 requests in each 5-minute statistics window. When client behavior verification is enabled, also verify that the same client verification token (corresponding to the same EO-Bot-Token value in the Cookie header) is used more than 200 times in 1 minute.
Medium risk: In a single session (corresponding to the same EO-Bot-SessionId value in the Cookie header), more than 500 requests in each 5-minute statistics window. When client behavior verification is enabled, also verify that the same client verification token (corresponding to the same EO-Bot-Token value in the Cookie header) is used more than 100 times in 1 minute.
Low-risk: In a single session (corresponding to the same EO-Bot-SessionId value in the Cookie header), more than 100 requests in each 5-minute statistics window. When client behavior verification is enabled, also verify that the same client verification token (corresponding to the same EO-Bot-Token value in the Cookie header) is used more than 20 times in 1 minute.
5. Click Save and publish . The rule will be deployed and take effect.
Scenario 2: Strengthening the e-commerce site's password reset page and API using client behavior verification to combat against Account Take Over (ATO) attacks from bulk password reset attempts
Scenario Example
The password reset API /api/password_reset of the e-commerce site shop.example.com has a large number of failed reset requests from a large amount of IPs with low frequency and no obvious User-Agent or header aggregation. Therefore, the active detection function is used to strengthen the bot protection rules for the password reset API /api/password_reset and the password reset page /account/forgot_password.html, using silent mode to combat against automated bulk password reset tools.
Directions
1. Log in to the EdgeOne console and click Site List in the left sidebar. In the site list, click the target Site.
2. Click Security > Web Security . By default, it is a site-level security policy. Click the Domain-level security policy tab and then click the target domain name such as shop.example.com , to enter the configuration page for the security policy of the target domain name.
3. Locate the Bot Management tab and click Add rule under Active detection to enter the configuration page.
4. Configure the judgment conditions and actions. In this example scenario, you can configure the matching field as Request path (Path) equals /account/forgot_password.html . Configure the operation to Validate client behavior , the Proof-of-work strength to High , and the delay to Delay for 100 ms :
For clients not enabled JS , configure the action to Add long latency when the requests are greater than 10 times within 10s .
For clients timed out , configure the action to Drop w/o response.
For Bot clients , configure the action to Drop w/o response .
Note:
Client behavior verification will be performed by injecting JavaScript only when the response Content-Type is text/html . Other requests will be handled based on the current verification result.
For action details of the bot management module, see Action. Other related configurations are described as follows:
Configuration item
Description
Proof-of-work strength
Client behavior verification supports adjusting the strength of proof of work verification. By adjusting the strength, the balance between the client's computational load and the identification effect on bots can be achieved.
Execution method
The JavaScript code used for detection will run after the whole page is loaded, and it also supports delaying the execution of the JavaScript detection code for a certain time. This helps to avoid affecting the normal page rendering, ensuring that the browser loads the page first before performing the verification, thus avoiding affecting the user's browsing experience.
Validation result
Client does not enabled JS (not completed detection): For clients that do not support JavaScript or requests initiated before the verification is completed, they are classified into this category. Since JavaScript verification usually takes some time, you can allow a certain rate of requests to pass before the client completes the verification, and dispose of clients that have not passed the verification and initiate high-frequency requests.
Client timed out: The client supports JavaScript and has started the verification, but it cannot be completed within 60 seconds. 60 seconds is enough for normal browser clients to complete the client behavior verification, while IoT proxies with less computing power have a higher probability of verification timeout. This option can be used to distinguish and dispose of requests from distributed bot networks with low computing power.
Bot client: The client has successfully completed the JavaScript verification, and the detection module finds that the client's running environment is abnormal, and it is not a normal human accessing through a browser.
5. Click Save and publish. The rule will be deployed and take effect.
