Observability
  • Log Service
    • Overview
    • Real-time Logs
      • Real-time Logs Overview
      • Push to EdgeOne Log Analysis
      • Push to Tencent Cloud CLS
      • Push to AWS S3-Compatible COS
      • Push to HTTP Server
    • Log Analysis (Beta)
      • Log Analysis(Beta) Overview
      • How to use Log Analysis filter condition
      • How to Modify Log Analysis Query Time Range
    • Offline Logs
    • Related References
      • Field description
        • L7 Access Logs
        • L4 Proxy Logs
        • Edge Function Running Logs
        • Managed Rule Logs
      • Real-Time Log Push Filter Conditions
      • Custom Log Push Fields
      • Customizing Log Output Formats
  • Data Analysis
    • Overview
    • Analytics
    • Web Security Analysis
    • Data Report
    • Related References
      • Sampling Statistics
      • How to use filter condition
      • How to Modify Query Time Range
      • How to Export Statistical Data and Reports
      • How to Use API to Implement Grouping Aggregation in a Single Call
  • AlarmService
    • Custom Statistical Metrics
Docs Overview/
Observability/
Log Service/
Related References/
Customizing Log Output Formats/

Customizing Log Output Formats

Overview

Supports user-defined log data output formats, including selecting different logging styles (such as JSON Lines or CSV) and defining the prefix and suffix of batches and individual logs, as well as the delimiter between log records or fields. The default format for real-time and offline logs is JSON Lines.
Note:
Currently, only real-time logs - L7 Access Logs support configuring the log output format.

Configuration Item

Log Output Format: Preset output format type for log delivery.
JSON Lines: Fields in a single log are presented as key-value pairs.
CSV: Only field values are shown in a single log entry, not the field names.
Batch Prefixes/Suffixes: Users can define prefixes and suffixes for log batch Definitions. "A batch" refers to a single log push request. Each log delivery batch may contain multiple log entries.
Prefix: A string added before each log delivery batch.
Suffix: A string appended after each log delivery batch.
Single Log Prefixes/Suffixes:
Prefix: A string added before each log record.
Suffix: A string appended after each log record.
Note:
When log sampling or Filtering is not enabled, for domain business, one HTTP request corresponds to one log record.
Log Separator: A string inserted between log records as a separator.
Field Separator: A string inserted between fields within a single log record as a separator.

Sample Configuration

The following are log samples corresponding to different log output formats.

JSON Lines

Configuration Samples

Configuration Item
Value
Log Output Format
JSON Lines
Single Log Prefix
{
Single Log Suffix
}
Log Separator
\n
Field Delimiter
,

Log Output Samples

{"SecurityAction":"Deny","RequestID":"14941044941971548881","RequestTime":"2024-08-12T08:12:15Z","ClientIP":"1.1.1.1"}
{"SecurityAction":"Deny","RequestID":"14941045941971548882","RequestTime":"2024-08-12T08:12:30Z","ClientIP":"2.2.2.2"}

csv

Configuration Samples

Configuration Item
Value
Log Output Format
csv
Log Separator
\n
Field Delimiter
,

Log Output Samples

Deny,14941044941971548881,2024-08-12T08:12:15Z,1.1.1.1
Deny,14941045941971548882,2024-08-12T08:12:30Z,2.2.2.2

JSON Lines Variants

JSON Array Format Configuration Example

Configuration Item
Value
Log Output Format
JSON Lines
Batch Prefix
[
Batch Suffix
]
Single Log Prefix
{
Single Log Suffix
}
Log Separator
,
Field Delimiter
,

JSON Array Format Log Output Sample

[
    {"SecurityAction":"Deny","RequestID":"14941044941971548881","RequestTime":"2024-08-12T08:12:15Z","ClientIP":"1.1.1.1"},
    {"SecurityAction":"Deny","RequestID":"14941045941971548882","RequestTime":"2024-08-12T08:12:30Z","ClientIP":"2.2.2.2"},
    {"SecurityAction":"Allow","RequestID":"14941046941971548883","RequestTime":"2024-08-12T08:12:45Z","ClientIP":"3.3.3.3"}
]

Embedded JSON Object Format Configuration Example

Configuration Item
Value
Log Output Format
JSON Lines
Batch Prefix
{"events":[
Batch Suffix
]}
Single Log Prefix
{"info":{
Single Log Suffix
}}
Log Separator
,
Field Delimiter
,

Embedded JSON Object Format Log Output Sample

{"events": [
        {"info": {"SecurityAction":"Deny","RequestID":"14941044941971548881","RequestTime":"2024-08-12T08:12:15Z","ClientIP":"1.1.1.1"}},
        {"info": {"SecurityAction":"Deny","RequestID":"14941045941971548882","RequestTime":"2024-08-12T08:12:30Z","ClientIP":"2.2.2.2"}}
]}

CSV Variants

CSV Carry Header Format Configuration Example

Configuration Item
Value
Log Output Format
csv
Batch Prefix
SecurityAction,RequestID,RequestTime,ClientIP\n
Log Separator
\n
Field Delimiter
,

CSV Carry Header Format Log Output Sample

SecurityAction,RequestID,RequestTime,ClientIP
Deny,14941044941971548881,2024-08-12T08:12:15Z,1.1.1.1
Deny,14941045941971548882,2024-08-12T08:12:30Z,2.2.2.2
Allow,14941046941971548883,2024-08-12T08:12:45Z,3.3.3.3

TSV Format Configuration Example

Configuration Item
Value
Log Output Format
csv
Log Separator
\n
Field Delimiter
\t

TSV Format Log Output Sample

Deny  14941044941971548881  2024-08-12T08:12:15Z  1.1.1.1
Deny  14941045941971548882  2024-08-12T08:12:30Z  2.2.2.2
Allow  14941046941971548883  2024-08-12T08:12:45Z  3.3.3.3