Push to AWS S3-Compatible COS
EdgeOne Real-time Log Push supports configuration through the console or API, to push logs to AWS S3 Signature Version 4 Authentication Algorithm compatible COS, such as:
Directions
1. Log in to the EdgeOne console and click Site List in the left sidebar. Then click on the site to be configured in the site list, to enter the site details page.
2. On the site details page, click Log Service > Real-time Logs.
3. On the real-time logs page, click Create Push Task.
4. On the log source selection page, enter a task name, select a log type, service area, and domain name/L4 proxy instance requiring log push, and click Next.
Note:
Currently, it only supports pushing site acceleration logs and L4 proxy logs to S3-compatible COS.
5. On the pushing content definition page:
(Required) Check the log fields to be pushed from the predefined field list.
(Optional) Add a custom log field, which supports extracting specified field names from the request headers, response headers, and Cookie headers.
(Optional) Configure the log push filter conditions. Full logs are pushed by default.
(Optional) In advanced configuration, set the sampling ratio. By default, sampling is not enabled and 100% of logs are pushed to the destination.
(Optional) In advanced configuration, set the log output format. The default format is JSON Lines.
6. On the destination selection page, select S3-compatible and click Next.
7. On the destination information page, enter the related destination and parameter information.
Parameter Name | Description |
Endpoint URL | URL not containing a bucket name or path, for example: https://cos.ap-nanjing.myqcloud.com. |
Bucket Region | Region where the bucket is located, for example: ap-nanjing. |
Bucket | Bucket name and log storage directory, for example: your_bucket_name/EO-logs/. No matter whether the directory ends with / or not, it will be correctly parsed and processed. |
File Compression | After checking, log files will be compressed with gzip. |
SecretId | Access key ID used to access the bucket. |
SecretKey | Secret key used to access the bucket. |
8. Click Push, confirm the related cost tips in the pop-up window, and click Confirm Creation.
9. After issuing the real-time log push task, EdgeOne will push a test file to the destination bucket directory to verify the connectivity, for example,
1699874755_edgeone_push_test.txt. The file contains a fixed string "test".File Name Description
Logs are stored in the specified bucket directory in the format
{{UploadTime}}_{{Random}}.log, and archived by date (UTC+00:00) into folders, for example: 20230331/20230331T185917Z_2aadf5ce.log. When gzip compression is enabled, the file name is 20230331/20230331T185917Z_2aadf5ce.log.gz.UploadTime: Upload time of the log file, in ISO-8601 format with UTC+00:00 time zone.
Random: Random characters used to identify different files when there are multiple log files in the same upload time due to large log volume.