Real-Time Log Push Filter Conditions
Real-time Log Push supports configuring the filter conditions to help you filter out specific types of logs and reduce the volume of downstream log processing. The following are the supported log fields and comparison operators.
Note
Currently, only Real-time Logs - Site Acceleration Logs support configuring the log push filter conditions.
The Real-time Log Push Filter Conditions feature is in beta testing. If needed, please contact us.
Supported Log Fields
Field Name | Data Type | Description |
SecurityAction | String | Final handling action after a request matches the security rules. Valid values include: -: unknown/not matched Monitor: observation JSChallenge: JavaScript challenge Deny: block Allow: pass BlockIP: IP banning Redirect: redirect ReturnCustomPage: returning custom pages ManagedChallenge: managed challenge Silence: Silence LongDelay: response after a long delay ShortDelay: response after a short delay |
SecurityModule | String | Name of the security module finally handling the request, corresponding to SecurityAction. Valid values include:-: unknown/not matched CustomRule: Web Protection - Custom Rules RateLimitingCustomRule: Web Protection - Rate Limiting Rules ManagedRule: Web Protection - Managed Rules L7DDoS: Web Protection - CC Attack Protection BotManagement: Bot Management - Bot Basic Management BotClientReputation: Bot Management - Client Reputation BotBehaviorAnalysis: Bot Management - Bot Intelligent Analysis BotCustomRule: Bot Management - Custom Bot Rules BotActiveDetection: Bot Management - Proactive Feature Recognition |
EdgeResponseStatusCode | Integer | Response status code returned to the client by the node. |
OriginResponseStatusCode | Integer | Response status code of the origin server. If there is no origin-pull, it is recorded as -1. |
Supported Comparison Operators
Comparison Operator Name | Supporting the Data Type or Not | |
| String | Integer |
Equals (matching any value in the list) | ✓ | ✓ |
Greater than | ✕ | ✓ |
Less than | ✕ | ✓ |
Greater than or equal to | ✕ | ✓ |
Less than or equal to | ✕ | ✓ |
Example: Filtering out Logs with HTTP Status Codes of 4xx/5xx
Sample Scenario
In a large e-commerce platform's IT Ops team, you are responsible for monitoring and analyzing real-time logs of the website. Due to the high volume of site visits and the enormous amount of log data, you wish to reduce unnecessary log data push by setting up filtering rules, thus avoiding unnecessary burden on the analysis platform. For instance, you can perform configuration to push only the access logs with HTTP status codes of 4xx/5xx, which usually indicate some kind of error. In this way, you can focus on logs that may point to user experience issues or system failures requiring immediate attention. You can follow the directions below for configuration.
Directions
1. Log in to the EdgeOne console and click Site List in the left sidebar. Then click on the site to be configured in the site list, to enter the site details page.
2. On the site details page, click Log Service > Real-time Logs.
3. On the real-time logs page, click Create Push Task.
4. On the log source selection page, enter a task name, select a log type, service area, and domain name/L4 proxy instance requiring log push, and click Next.
5. On the push content definition page, configure the log push range.
5.1 Select Filtered logs.
5.2 Enter the filtering conditions, as shown in the figure below:
6. After configuring the destination, click Push, confirm the related cost tips in the pop-up window, and click Confirm Creation to save the configuration.