Observability
  • Log Service
    • Overview
    • Real-time Logs
      • Real-time Logs Overview
      • Push to Tencent Cloud CLS
      • Push to AWS S3-Compatible COS
      • Push to HTTP Server
    • Offline Logs
    • Related References
      • Field description
        • L7 Access Logs
        • L4 Proxy Logs
      • Real-Time Log Push Filter Conditions
      • Custom Log Push Fields
      • Customizing Log Output Formats
  • Data Analysis
    • Overview
    • Analytics
    • Web Security Analysis
    • Traffic Analysis
    • Cache Analysis
    • Security Analysis
      • Site Security Overview
      • Web Security Analysis
    • L4 Proxy
    • DNS Resolution
    • Related References
      • Sampling Statistics
      • How to use filter condition
      • How to Modify Query Time Range
      • How to Export Statistical Data and Reports
  • AlarmService
    • Custom Statistical Metrics
이 페이지는 현재 영어로만 제공되며 한국어 버전은 곧 제공될 예정입니다. 기다려 주셔서 감사드립니다.
문서 개요/
Observability/
Log Service/
Real-time Logs/
Push to AWS S3-Compatible COS/

Push to AWS S3-Compatible COS

EdgeOne Real-time Log Push can be configured through the console or API to push logs to AWS S3 Signature Version 4 Authentication Algorithm compatible COS, such as:
Tencent Cloud COS
AWS S3
Google Cloud Storage
IBM Cloud Object Storage
Linode Object Storage
Oracle Cloud Object Storage, etc

Operation step

1. Access the EdgeOne console, locate the left-hand menu, and select the Site List. Within this listing, click on the Site requiring configuration to enter its detailed page.
2. On the Site Details Page, click Log Service > Real-time Logs.
3. On the Real-time Logs page, click Create shipping task.
4. On Select the log source Page, fill in the Task name, select the Log type Service area, the Domain name/Layer 4 Proxy Instance/edge function instance for which the logs need to be pushed, and click Next.
Note:
Currently, only Site Acceleration Logs、L4 Proxy Logs and edge function runtime logs (in beta) are supported to be pushed to S3 Compatible COS.
5. On the Define delivery content page:
(Required) Check the log fields to be pushed from the Predefined field list;
(Optional) Add a Custom Log Field, supporting the extraction of specified field names from the Request Header, Response Header, and Cookie Header;
(Optional) Configure the Push Log Filter Criteria, the default is to push full logs;
(Optional) In advanced configuration, set the sampling ratio. By default, sampling is not enabled, and 100% of logs are pushed to the destination.
(Optional) In advanced configuration, set the log output format. The default format is JSON Lines.
6. On the Select a destination address page, select S3 compatible , click Next .
7. On the Destination Information Page, enter the related destination and parameter information.
Parameter name
Description
Endpoint URL
URL without bucket name or path, e.g.: https://cos.ap-nanjing.myqcloud.com.
bucket region
Region where the bucket is located, for example:ap-nanjing.
Bucket
Bucket name and log storage directory, for example:your_bucket_name/EO-logs/. Whether or not you end the directory with /, it will be correctly parsed and processed.
File compression
After checking, gzip will be used to compress log files.
Secret ID
Access Key ID used to access the bucket.
Secret key
Secret key used to access the bucket.
8. Click Ship.
9. After issuing the real-time log push task, EdgeOne will push a test file to the target bucket directory to verify connectivity, for example, 1699874755_edgeone_push_test.txt, with the file content being the fixed string "test".

File Name Description

Logs will be stored in the specified bucket directory in the format {{UploadTime}}_{{Random}}.log, and will be archived by date (UTC+00:00) into folders. For example: 20230331/20230331T185917Z_2aadf5ce.log. When gzip compression is enabled, the file name will be 20230331/20230331T185917Z_2aadf5ce.log.gz.
UploadTime: The upload time of the log file, using ISO-8601 Format, UTC+00:00 Time Zone.
Random: Random characters used to identify different files when there are multiple log files for the same upload time due to large log volume.