Customizing Log Output Formats
Overview
Users can customize log output formats, including selecting different logging styles (such as JSON Lines or CSV) and defining the prefix and suffix of batches and single logs, as well as the delimiter between log records or fields. The default format for real-time and offline logs is JSON Lines.
Note:
Currently, only real-time logs - site acceleration logs support configuring the log output format.
Configuration Item
Log output format: Preset output format type for log shipping.
JSON Lines: Fields in a single log are presented as key-value pairs.
CSV: Only field values are presented in a single log, without displaying field names.
Batch prefix & suffix: Users can define prefixes and suffixes for log batches. A batch refers to a single log push request. Each log shipping batch may contain multiple log records.
Prefix: A string added before each log shipping batch.
Suffix: A string appended after each log shipping batch.
Single log record prefix & suffix:
Prefix: A string added before each log record.
Suffix: A string appended after each log record.
Note:
When log sampling or filtering is not enabled, for domain name business, one HTTP request corresponds to one log record.
Log record delimiter: A string inserted between log records as a delimiter.
Field delimiter: A string inserted between fields within a single log record as a delimiter.
Sample Configuration
The following are log samples corresponding to different log output formats.
JSON Lines
Configuration Example
Configuration Item | Value |
Log output format | JSON Lines |
Single log record prefix | { |
Single log record suffix | } |
Log record delimiter | \n |
Field delimiter | , |
Log Output Sample
{"SecurityAction":"Deny","RequestID":"14941044941971548881","RequestTime":"2024-08-12T08:12:15Z","ClientIP":"1.1.1.1"}{"SecurityAction":"Deny","RequestID":"14941045941971548882","RequestTime":"2024-08-12T08:12:30Z","ClientIP":"2.2.2.2"}
CSV
Configuration Example
Configuration Item | Value |
Log output format | csv |
Log record delimiter | \n |
Field delimiter | , |
Log Output Sample
Deny,14941044941971548881,2024-08-12T08:12:15Z,1.1.1.1Deny,14941045941971548882,2024-08-12T08:12:30Z,2.2.2.2
JSON Lines Variants
JSON Array Format Configuration Example
Configuration Item | Value |
Log output format | JSON Lines |
Batch prefix | [ |
Batch suffix | ] |
Single log record prefix | { |
Single log record suffix | } |
Log record delimiter | , |
Field delimiter | , |
JSON Array Format Log Output Sample
[{"SecurityAction":"Deny","RequestID":"14941044941971548881","RequestTime":"2024-08-12T08:12:15Z","ClientIP":"1.1.1.1"},{"SecurityAction":"Deny","RequestID":"14941045941971548882","RequestTime":"2024-08-12T08:12:30Z","ClientIP":"2.2.2.2"},{"SecurityAction":"Allow","RequestID":"14941046941971548883","RequestTime":"2024-08-12T08:12:45Z","ClientIP":"3.3.3.3"}]
Embedded JSON Object Format Configuration Example
Configuration Item | Value |
Log output format | JSON Lines |
Batch prefix | {"events":[ |
Batch suffix | ]} |
Single log record prefix | {"info":{ |
Single log record suffix | }} |
Log record delimiter | , |
Field delimiter | , |
Embedded JSON Object Format Log Output Sample
{"events": [{"info": {"SecurityAction":"Deny","RequestID":"14941044941971548881","RequestTime":"2024-08-12T08:12:15Z","ClientIP":"1.1.1.1"}},{"info": {"SecurityAction":"Deny","RequestID":"14941045941971548882","RequestTime":"2024-08-12T08:12:30Z","ClientIP":"2.2.2.2"}}]}
CSV Variants
CSV with Header Format Configuration Example
Configuration Item | Value |
Log output format | csv |
Batch prefix | SecurityAction,RequestID,RequestTime,ClientIP\n |
Log record delimiter | \n |
Field delimiter | , |
CSV with Header Format Log Output Sample
SecurityAction,RequestID,RequestTime,ClientIPDeny,14941044941971548881,2024-08-12T08:12:15Z,1.1.1.1Deny,14941045941971548882,2024-08-12T08:12:30Z,2.2.2.2Allow,14941046941971548883,2024-08-12T08:12:45Z,3.3.3.3
TSV Format Configuration Example
Configuration Item | Value |
Log output format | csv |
Log record delimiter | \n |
Field delimiter | \t |
TSV Format Log Output Sample
Deny 14941044941971548881 2024-08-12T08:12:15Z 1.1.1.1Deny 14941045941971548882 2024-08-12T08:12:30Z 2.2.2.2Allow 14941046941971548883 2024-08-12T08:12:45Z 3.3.3.3