Observability
  • Log Service
    • Overview
    • Real-time Logs
      • Real-time Logs Overview
      • Push to Tencent Cloud CLS
      • Push to AWS S3-Compatible COS
      • Push to HTTP Server
    • Offline Logs
    • Related References
      • Field description
        • L7 Access Logs
        • L4 Proxy Logs
      • Real-Time Log Push Filter Conditions
      • Custom Log Push Fields
  • Data Analysis
    • Overview
    • Traffic Analysis
    • Cache Analysis
    • Security Analysis
      • Site Security Overview
      • Web Security Analysis
    • L4 Proxy
    • DNS Resolution
    • Related References
      • How to use filter condition
      • How to Modify Query Time Range
      • How to Export Statistical Data and Reports

Real-Time Log Push Filter Conditions

Real-time Log Push supports configuring the filter conditions to help you filter out specific types of logs and reduce the volume of downstream log processing. The following are the supported log fields and comparison operators.
Note
Currently, only Real-time Logs - Site Acceleration Logs support configuring the log push filter conditions.
The Real-time Log Push Filter Conditions feature is in beta testing. If needed, please contact us.

Supported Log Fields

Field Name
Data Type
Description
SecurityAction
String
Final handling action after a request matches the security rules. Valid values include:
-: unknown/not matched
Monitor: observation
JSChallenge: JavaScript challenge
Deny: block
Allow: pass
BlockIP: IP banning
Redirect: redirect
ReturnCustomPage: returning custom pages
ManagedChallenge: managed challenge
Silence: Silence
LongDelay: response after a long delay
ShortDelay: response after a short delay
SecurityModule
String
Name of the security module finally handling the request, corresponding to SecurityAction. Valid values include:
-: unknown/not matched
CustomRule: Web Protection - Custom Rules
RateLimitingCustomRule: Web Protection - Rate Limiting Rules
ManagedRule: Web Protection - Managed Rules
L7DDoS: Web Protection - CC Attack Protection
BotManagement: Bot Management - Bot Basic Management
BotClientReputation: Bot Management - Client Reputation
BotBehaviorAnalysis: Bot Management - Bot Intelligent Analysis
BotCustomRule: Bot Management - Custom Bot Rules
BotActiveDetection: Bot Management - Proactive Feature Recognition
EdgeResponseStatusCode
Integer
Response status code returned to the client by the node.
OriginResponseStatusCode
Integer
Response status code of the origin server. If there is no origin-pull, it is recorded as -1.

Supported Comparison Operators

Comparison Operator Name
Supporting the Data Type or Not
String
Integer
Equals (matching any value in the list)
Greater than
Less than
Greater than or equal to
Less than or equal to

Example: Filtering out Logs with HTTP Status Codes of 4xx/5xx

Sample Scenario

In a large e-commerce platform's IT Ops team, you are responsible for monitoring and analyzing real-time logs of the website. Due to the high volume of site visits and the enormous amount of log data, you wish to reduce unnecessary log data push by setting up filtering rules, thus avoiding unnecessary burden on the analysis platform. For instance, you can perform configuration to push only the access logs with HTTP status codes of 4xx/5xx, which usually indicate some kind of error. In this way, you can focus on logs that may point to user experience issues or system failures requiring immediate attention. You can follow the directions below for configuration.

Directions

1. Log in to the EdgeOne console and click Site List in the left sidebar. Then click on the site to be configured in the site list, to enter the site details page.
2. On the site details page, click Log Service > Real-time Logs.
3. On the real-time logs page, click Create Push Task.
4. On the log source selection page, enter a task name, select a log type, service area, and domain name/L4 proxy instance requiring log push, and click Next.
5. On the push content definition page, configure the log push range.
5.1 Select Filtered logs.
5.2 Enter the filtering conditions, as shown in the figure below:

6. After configuring the destination, click Push, confirm the related cost tips in the pop-up window, and click Confirm Creation to save the configuration.