Push to AWS S3
EdgeOne Real-time Log Delivery supports pushing logs to S3 buckets in your AWS account, with authentication performed through temporary credentials for IAM roles (AWS STS). You do not need to configure long-term static access keys, and temporary credentials expire automatically, reducing security risks associated with key leakage and rotation. You can configure this through the console or API.
Note:
AWS S3 is an AWS-specific destination that uses IAM roles (Role Assumption + STS temporary credentials) for authentication. It is recommended for customers who use AWS S3 for log storage.
To push logs to other S3-compatible object storage services such as Tencent Cloud COS and Google Cloud Storage, use the S3-compatible object storage destination. For details, see Push to S3-Compatible Object Storage.
Prerequisites
1. You have an AWS account and have created an S3 bucket in the target region for storing real-time logs.
2. You have created an IAM role in AWS IAM for EdgeOne log delivery and configured its Trust policy and Permission policy. For details, see IAM Role Configuration below.
IAM Role Configuration
1. Sign in to the AWS Management Console, go to IAM > Roles, and click Create role.
2. In Step 1 Select trusted entity, set Trusted entity type to AWS account, select Another AWS account, and enter the EdgeOne service account ID in Account ID. The EdgeOne service account ID varies by the service region where your target delivery bucket resides. Enter the corresponding value based on the region where your S3 bucket resides:
If the bucket is in an AWS global region (with the endpoint domain suffix
amazonaws.com), use 361796580773;If the bucket is in the AWS China region (with the endpoint domain suffix
amazonaws.com.cn), use 382884104370.Note:
The AWS global region and the AWS China region are isolated from each other. Make sure to select the corresponding EdgeOne service account ID based on the service region where your bucket resides. If the region does not match the account ID, AssumeRole will be denied, causing the real-time log delivery task creation to fail validation in the console.
3. Select Require external ID, enter your Tencent Cloud root account UIN in External ID, and click Next.
Note:
<External ID> is a security identifier that protects against Confused Deputy attacks. Its value is your Tencent Cloud root account UIN, which can be viewed and copied from the External ID field on the EdgeOne console > Site Details > Real-time Logs > Create delivery task > Destination Details page.4. In Step 2 Add permissions, select Create inline policy to create a new policy that will be directly attached to this role. In the Policy editor, paste the following content and replace
<bucket name> with your actual value:{"Version": "2012-10-17","Statement": {"Sid": "EdgeOneRealtimeLogDelivery","Effect": "Allow","Action": ["s3:PutObject", "s3:AbortMultipartUpload"],"Resource": "arn:aws:s3:::<bucket name>/*"}}
Note:
If the bucket is in the AWS China region, the ARN of
Resource must be changed to arn:aws-cn:s3:::<bucket name>/*.Note:
If you have already created a reusable permission policy in IAM > Policies, you can select Use existing policy in the Add permissions step and select the existing policy instead of creating a new one.
5. After confirming that no errors are displayed at the bottom of the editor, click Next.
6. In Step 3 Name, review, and create, name the role, verify that the Trust policy and Permission policy are correct, configure tags, and then click Create Role to complete the creation.
Example of a Trust policy for the AWS global region:
{"Version": "2012-10-17","Statement": [{"Effect": "Allow","Action": "sts:AssumeRole","Principal": {"AWS": "361796580773"},"Condition": {"StringEquals": {"sts:ExternalId": "<Your Tencent Cloud root account UIN>"}}}]}
Example of a Trust policy for the AWS China region:
{"Version": "2012-10-17","Statement": [{"Effect": "Allow","Action": "sts:AssumeRole","Principal": {"AWS": "382884104370"},"Condition": {"StringEquals": {"sts:ExternalId": "<Your Tencent Cloud root account UIN>"}}}]}
7. After completing the above configuration, go to IAM > Roles, select the role you just created, and record the following information for filling in the destination parameters in the EdgeOne console:
IAM role ARN: in the format of
arn:aws:iam::<your AWS account ID>:role/<role name> (for AWS China accounts, the format is arn:aws-cn:iam::<your AWS account ID>:role/<role name>).Configuration Guide
1. Log in to the EdgeOne console, go to Service Dashboard in the left sidebar, and click the site to configure under Website Security Acceleration.
2. On the Site Details page, click Log Service > Real-time Logs.
3. On the Real-time Logs page, click Create delivery task.
4. On the Select Log Source page, enter the task name, select the log type and service area, and click Next.
5. On the Select Destination page, select AWS S3, and click Next.
6. On the Define Log Content page:
(Required) Select the domain name/layer-4 proxy instance/edge function instance for which logs need to be pushed.
(Required) Check the log fields to be pushed from the predefined field list.
(Optional) Add custom log fields, which support custom log field names and custom log field values using expressions.
(Optional) Configure the filter conditions. By default, all logs are pushed.
(Optional) In Advanced settings, configure the sampling ratio. Sampling is disabled by default, and 100% of logs are pushed to the destination.
(Optional) In Advanced settings, configure the log output format. The default format is JSON Lines.
Click Next.
7. On the Destination Details page, enter the following parameters:
Parameter Name | Description |
Bucket name (required) | Name of the bucket in AWS S3 for storing EdgeOne real-time logs, for example, my-logpush-s3-bucket. The name must be 3 to 63 characters in length and can contain only lowercase letters, digits, periods (.), and hyphens (-), and must start and end with a letter or digit. |
Endpoint domain (required) | Service endpoint domain of the region where the S3 bucket resides, in the format s3.<region>.<domain suffix>.The suffix for AWS global regions is amazonaws.com, for example, s3.us-west-2.amazonaws.com. For the complete list of available options, see the AWS official documentation.The suffix for AWS China regions is amazonaws.com.cn, for example, s3.cn-north-1.amazonaws.com.cn. For the complete list of available options, see the AWS official documentation. |
IAM role ARN (required) | ARN of the IAM role that you created in your AWS account for EdgeOne log delivery. EdgeOne will use this role to request temporary credentials from AWS STS to write to your S3 bucket. The format for AWS global regions is arn:aws:iam::<your AWS account ID>:role/<role name>.The format for AWS China regions is arn:aws-cn:iam::<your AWS account ID>:role/<role name>. |
File compression (optional) | Select whether to compress log files with Gzip. By default, log files are not compressed. |
Log directory (required) | Storage directory path of log objects in the S3 bucket. Use / as the delimiter to create multi-level subdirectories. Only safe characters are allowed (letters, digits, ! - _ . * ' ( )), and the total length cannot exceed 1024 bytes. |
8. Click Ship. EdgeOne will immediately initiate a connectivity precheck, request temporary credentials from AWS STS, and write a test file to the destination bucket:
Precheck passed: The task is created successfully, and real-time log delivery starts.
Precheck failed: Check the trust policy and permission policy of the IAM role and the destination parameter configuration based on the error message.
File Name Description
Logs will be written to the specified directory in the
{{UploadTime}}_{{Random}}.log format and archived into folders by date (UTC+0), for example, 20261028/20261028T185917Z_2aadf5ce.log. When Gzip compression is enabled, the file name will have the .gz suffix appended, for example, 20261028/20261028T185917Z_2aadf5ce.log.gz.UploadTime: The upload time of the log file, in the UTC+0 time zone, with the format
YYYYMMDDTHHmmssZ.Random: A random string used to distinguish different files when multiple log files are generated for the same upload time due to large log volume.
Authentication Process Description
The authentication and writing process of the EdgeOne backend within a log delivery cycle is as follows:
1. Temporary Credential Acquisition: EdgeOne calls
sts:AssumeRole using its service account identity (either 361796580773 for global regions or 382884104370 for China regions, depending on the bucket's partition), with the IAM role ARN and External ID you configured. After AWS verifies the Principal and sts:ExternalId conditions in the trust policy, it returns temporary credentials (temporary Access Key, Secret Key, and Session Token) that are valid for approximately 1 hour. EdgeOne caches the temporary credentials and automatically refreshes them before they expire.2. Log Writing: EdgeOne uses temporary credentials to sign S3 requests with Signature Version 4 and writes logs to the bucket and directory you specified.