Observability
  • Log Service
    • Overview
    • Real-time Logs
      • Real-time Logs Overview
      • Push to EdgeOne Log Analysis
      • Push to Tencent Cloud CLS
      • Push to AWS S3-Compatible COS
      • Push to HTTP Server
    • Log Analysis (Beta)
      • Log Analysis(Beta) Overview
      • How to use Log Analysis filter condition
      • How to Modify Log Analysis Query Time Range
    • Offline Logs
    • Related References
      • Field description
        • L7 Access Logs
        • L4 Proxy Logs
        • Edge Function Running Logs
        • Managed Rule Logs
      • Real-Time Log Push Filter Conditions
      • Custom Log Push Fields
      • Customizing Log Output Formats
  • Data Analysis
    • Overview
    • Analytics
    • Web Security Analysis
    • Data Report
    • Related References
      • Sampling Statistics
      • How to use filter condition
      • How to Modify Query Time Range
      • How to Export Statistical Data and Reports
      • How to Use API to Implement Grouping Aggregation in a Single Call
  • AlarmService
    • Custom Statistical Metrics
Docs Overview/
Observability/
Log Service/
Log Analysis (Beta)/
How to use Log Analysis filter condition/

How to use Log Analysis filter condition

Log analysis filters are used to filter log data precisely. They can be configured via the console's visual interface or by using search expressions.

Filter Types

Log analysis supports three types of filters:
1. Log Source (Required): View log data for the selected log type (Layer 7 access logs or managed rule logs) and the selected region.
2. Time Range (Required): View data within the selected time range. For details, see How to Modify Log Analysis Query Time Range.
3. Other Filter Conditions: Filter logs based on supported filter items. Supported filter items vary by log type and are described in the following sections.

Relationship Between Multiple Filters

Multiple filters are combined using AND logic, while multiple values within the same filter are combined using OR logic.
For example, if the following filters are added at the same time: ClientRegion is in Singapore;Thailand and EdgeResponseStatusCodeis in 404, the query returns requests from clients in Singapore or Thailand and an edge response status code of 404.

Search Query Syntax

Log analysis supports filtering logs by using search expressions. The expression syntax is based on Condition syntax.

Expression Structure

A log analysis search expression consists of three basic components: Filter Item, Operator, and Value. Its basic format is shown in the following figure:

Filter Item: A built-in log field in the format of ${…}, such as ${RequestID} and ${RequestHost}. Used to extract specific information from requests. Use PascalCase.
Operator: For example, in, greater than (>), etc. Used to determine the relationship between a target field and a value.
Value: can be a string, number, list or other variable, and should be wrapped in square brackets [ ] according to the operator, such as ['POST'] or [100, 200].

Operator

Matching Operators

Term
Operator
Example
Equal to
in
${RequestHost} in ['example.com', 'demo.com']
Greater than
>
${EdgeResponseTime} > 500
For the matching operators supported by each field, see Supported Filter Items.

Logical Operators

Supported Filters

Layer 7 Access Logs

When the log type is Layer 7 Access Logs, the following filters are supported. For detailed descriptions and values of each field, see Layer 7 Access Logs.

General Information

Filter Name
Data Type
Operator
LogTime
Equal to
RequestID
String
Equal to, Not equal to
ContentID
String
Equal to, Not equal to
EdgeEndTime
Equal to, Greater than
EdgeFunctionSubrequest
Integer
Equal to, Not equal to
ParentRequestID
String
Equal to, Not equal to

Request Information

Filter Name
Data Type
Operator
RequestHost
String
Equal to
RequestMethod
String
Equal to, Not equal to
RequestTime
Equal to
RequestUrl
String
Equal to, Not equal to
RequestUrlQueryString
String
Equal to, Not equal to
RequestReferer
String
Equal to, Not equal to
RequestUA
String
Equal to, Not equal to
RequestProtocol
String
Equal to, Not equal to
RequestScheme
String
Equal to, Not equal to
RequestSSLProtocol
String
Equal to, Not equal to
RequestStatus
String
Equal to, Not equal to
RequestRange
String
Equal to
RequestBytes
Integer
Equal to, Greater than
RequestBodyBytes
Integer
Equal to, Greater than
RemotePort
Integer
Equal to, Not equal to

Client Information

Filter Name
Data Type
Operator
ClientIP
String
Equal to, Not equal to
ClientRegion
String
Equal to, Not equal to
ClientState
String
Equal to, Not equal to
ClientISP
String
Equal to, Not equal to
ClientDeviceType
String
Equal to, Not equal to
ClientPort
Integer
Equal to, Not equal to
ClientConnectionID
String
Equal to, Not equal to

Response Information

Filter Name
Data Type
Operator
EdgeCacheStatus
String
Equal to, Not equal to
EdgeResponseStatusCode
Integer
Equal to, Not equal to
EdgeResponseBytes
Integer
Equal to, Greater than
EdgeResponseBodyBytes
Integer
Equal to, Greater than
EdgeInternalTime
Integer
Equal to, Greater than
EdgeResponseTime
Integer
Equal to, Greater than

Edge Server Information

Filter Name
Data Type
Operator
EdgeServerID
String
Equal to, Not equal to
EdgeServerIP
String
Equal to, Not equal to
EdgeServerRegion
String
Equal to, Not equal to
EdgeServerRegionTopDivision
String
Equal to, Not equal to
EdgeException
String
Equal to, Not equal to

Origin Server Information

Filter Name
Data Type
Operator
OriginDNSResponseDuration
Double
Equal to, Greater than
OriginIP
String
Equal to, Not equal to
OriginRequestHeaderSendDuration
Double
Equal to, Greater than
OriginResponseHeaderDuration
Double
Equal to, Greater than
OriginResponseStatusCode
Integer
Equal to, Not equal to
OriginSSLProtocol
String
Equal to, Not equal to
OriginTCPHandshakeDuration
Double
Equal to, Greater than
OriginTLSHandshakeDuration
Double
Equal to, Greater than

Security Information

Filter Name
Data Type
Operator
SecurityAction
String
Equal to, Not equal to
SecurityRuleID
String
Equal to, Not equal to
SecurityModule
String
Equal to, Not equal to
BotCharacteristic
String
Equal to, Not equal to
BotClassAttacker
String
Equal to, Not equal to
BotClassMaliciousBot
String
Equal to, Not equal to
BotClassProxy
String
Equal to, Not equal to
BotClassScanner
String
Equal to, Not equal to
BotClassAccountTakeOver
String
Equal to, Not equal to
BotTag
String
Equal to, Not equal to
JA3Hash
String
Equal to, Not equal to

Managed Rule Logs

When the log type is Managed Rule Logs, the following filter conditions are supported. For detailed descriptions and values of each field, see Managed Rule Logs.

Request Information

Filter Name
Data Type
Operator
RequestHost
String
Equal to, Not equal to
RequestID
String
Equal to, Not equal to
RequestTime
Integer
Equal to
RequestMethod
String
Equal to, Not equal to
RequestUA
String
Equal to, Not equal to
RequestURI
String
Equal to, Not equal to
RequestBody
String
Equal to, Not equal to

Client Information

Filter Name
Data Type
Operator
ClientIP
String
Equal to, Not equal to
ClientCountry
String
Equal to, Not equal to

Security Information

Filter Name
Data Type
Operator
SecurityRuleID
String
Equal to, Not equal to
SecurityModule
String
Equal to, Not equal to
SecurityAction
String
Equal to, Not equal to
SecurityMatchingField
String
Equal to, Not equal to
SecurityMatchingPosition
String
Equal to, Not equal to

Search Expression Examples

Single-Filter Query

1. Filter requests for specified domains:
${RequestHost} in ['example.com']
2. Filter requests with a 404 status code:
${EdgeResponseStatusCode} in ['404']
3. Filter requests with a response time greater than 500 ms:
${EdgeResponseTime} > 500

Multi-Filters Combination

1. Filter requests with a status code of 404 or 500 under specified domains:
${RequestHost} in ['example.com'] and ${EdgeResponseStatusCode} in ['404', '500']
2. Filter non-GET requests with a response time greater than 1000 ms:
not ${RequestMethod} in ['GET'] and ${EdgeResponseTime} > 1000
3. Filter POST requests from the Chinese mainland:
${ClientRegion} in ['CN'] and ${RequestMethod} in ['POST']

Changing Priority with Parentheses

1. Filter non-GET requests from China or the United States:
(${ClientRegion} in ['CN'] or ${ClientRegion} in ['US']) and ${RequestMethod} not in ['GET']
2. Filter requests under specified domains with a 5xx status code or a response time greater than 2000 ms:
${RequestHost} in ['example.com'] and (${EdgeResponseStatusCode} in ['500', '502', '503', '504'] or ${EdgeResponseTime} > 2000)

Security Scenarios

1. Filter requests in Layer 7 Access Logs that trigger specified rules and have an action of Block or Observe:
${SecurityRuleID} in ['2123456789'] and ${SecurityAction} in ['Deny', 'Monitor']
2. Filter requests in Layer 7 Access Logs that trigger the rate-limiting module:
${SecurityModule} in ['RateLimitingCustomRule', 'L7DDoS', 'BandwidthAbuseProtection']
3. Filter requests in Layer 7 Access Logs where any dimension in the client profile is high-risk:
${BotClassAttacker} in ['high'] or ${BotClassMaliciousBot} in ['high'] or ${BotClassProxy} in ['high'] or ${BotClassScanner} in ['high'] or ${BotClassAccountTakeOver} in ['high']
4. Filter requests in managed rule logs where the request path matches a managed rule and the request is ultimately blocked:
${SecurityMatchingPosition} in ['Full request path'] and ${SecurityAction} in ['drop']