How to Synchronize Content between EdgeOne Edge Nodes and the Origin Server?

EdgeOne edge nodes cache source content to a vast network of global acceleration nodes, enabling users to access desired content nearby, avoiding network instability and high latency caused by network congestion, inter-carrier issues, and cross-regional, or cross-border factors. This effectively enhances download speeds, reduces response times, and delivers a seamless user experience.

The Relationship between Origin Site and Content Delivery Network

Origin Server: Refers to the stable business server that users operate on. EdgeOne manages business origin servers in groups called origin server groups. Configured origin server groups can be referenced in functions like adding acceleration domains and Layer 4 proxies. An origin server group is the smallest configuration unit in load balancing, allowing the addition of one or multiple origin servers. When adding multiple origin servers, traffic load can be adjusted by configuring weights.

After your site is connected to EdgeOne, EdgeOne edge nodes will determine whether to cache client requests' resource files responded by the origin server based on the rules configured for caching. Once the edge node caches this file, when another user initiates the same file request, it can be directly responded to by the EdgeOne edge node, effectively avoiding long trips to the origin server. This allows for faster responses to users' requests for the latest files, greatly enhancing the end-user access experience.

Origin Server Group Basic Configuration

Supported Origin Group Types in EdgeOne

HTTP Dedicated: Supports adding IP/domain name origins and object storage origins, and can only be used for site acceleration-related services (e.g., Domain Name Service and rule engine - Modify origin).
Universal: Only supports adding IP/domain name as origin, does not support adding object storage origin, and can be used for site acceleration services (such as Domain Name Service and rule engine) and L4 proxy.

Supported Origin Server Types in EdgeOne

• Object storage origin: Tencent Cloud COS or other object storage buckets compatible with AWS S3.
• IP/domain name origin: Supports IPv4 addresses, IPv6 addresses, and domain names as origins.

Create Origin Server Group

Add Origin Servers

Explanation

Regarding the configuration of weights in the origin group:

1. If a server within the origin group is assigned a weight, all servers within the origin group must also have corresponding weights set. Weights can be integers from 0 to 100. Setting a server's weight to 0 means that no origin requests will be directed to that server. Origin requests will be distributed among the other non-zero-weighted servers based on their respective weight ratios.
2. If you do not set weights, all servers within the origin group should not have weights assigned. In this scenario, if intelligent acceleration is not enabled, EdgeOne will evenly distribute origin requests to each server. If intelligent acceleration is enabled, EdgeOne will choose the best-quality server for each origin request.

Other Origin-pull Configurations

Origin Origin Protocol

Specifies the request protocol used by EdgeOne during origin fetch.

Upstream Timeout

The EdgeOne rule engine allows customization of origin fetch timeout settings. Depending on network conditions and the origin server's processing capacity, it's recommended to set an appropriate fetch request timeout to ensure successful origin fetching. The origin fetch timeout is the duration within which a node, after initiating a fetch request, can consider a lack of response from the origin server as a timeout and actively disconnect.

Currently, EdgeOne supports configuring HTTP response timeout (TCP connection timeout configuration coming soon) with an integer value between 5 and 300. The default value is 15 seconds. After a successful connection between the node and the origin server, if the node sends an HTTP request and doesn't receive any response data within 15 seconds (including no response or interrupted data transmission), the node will consider it an HTTP response timeout.

Control of origin request parameters

By default, when origin-pulling, all query strings and Cookies within the request will be retained. If your business origin only allows specified query strings or Cookie information to be carried in the origin-pull request, you can ensure the normal origin-pull request by deleting the specified origin-pull request parameters.

Redirect Following During Origin-Pull

Under normal circumstances, when the origin returns a 301/302 request, the node will return the status code to the client by default, and the client will redirect to the corresponding resources for access. EdgeOne supports follow-origin redirects. When enabled, if the node receives a 301/302 status code during origin-pull, it will actively follow the redirect (not exceeding the set maximum redirects) to the specified address until the corresponding file is obtained, and then respond to the client with the actual resources, which can improve the user's access response speed.

How to Maintain Content Synchronization Between EdgeOne Nodes and the Origin Server?

Since EdgeOne edge nodes cache a large amount of origin server resources, a problem arises: how to ensure that the resources on EdgeOne edge nodes remain synchronized with the content on the origin server after changes are made to the origin server resources?

Prefetch Cache

When a business releases new resources, the client's first request for these resources may encounter a situation where there is no cache on EdgeOne, resulting in an inability to respond immediately and the need to follow the origin to obtain. The cache prefetch function allows resources to be cached on EdgeOne in advance. In this way, even if the client requests for the first time, it can be directly responded from the cache of EdgeOne without the need to follow the origin. The implementation of cache prefetch is to submit the URLs that need to be pre-warmed, and then cache the resources that match these URLs from the origin to EdgeOne in advance, thereby improving the acceleration effect and mitigating the pressure on the origin.

Usage Scenarios

You may need to use this function in the following scenarios:

1. Newly released content: When your business releases new content or updates existing content, you want to ensure that this content is immediately available on EdgeOne so that client users can access the latest content for the first time, reducing the delay when accessing for the first time. For example, before the game business officially releases a new version of the installation package or upgrade package, the installation package resources can be preheated to EdgeOne. After the official release, users can directly obtain the installation package resources from the node when requesting to download these installation packages, improving the download speed.
2. Large-scale event operations: Before large-scale events, you want to ensure that the key resources of the event have been cached on EdgeOne, which helps to ensure that when the event starts, client users can quickly access the required content, reducing the delay and congestion caused by high traffic.
3. Expected traffic peaks: If you expect a significant increase in website traffic during a specific period (e.g., holiday promotions, news releases, etc.), you can use the cache prefetch function to ensure that key resources have already been cached on the edge nodes. This helps to disperse the pressure of origin-pull requests during peak periods and improve the access speed of client users.

Note

When preheating resources, simulated requests will be made to retrieve the corresponding resources from the origin. If there are many preheating tasks submitted, more origin-pull requests will be generated, and the bandwidth of the origin will increase.
If the preheated resources conflict with the node cache, that is, if EdgeOne has cached identical resources and they have not expired, they will still be valid and will not be overwritten by the preheated resources. If the identical resources have changed, you can purge the corresponding node cache before preheating.

Directions

Scenario One: Preheat cache by inputting content

If you have less content to preheat and it is convenient to input the content directly in the input box, please follow the steps below:

1. Log in to the EdgeOne console, and find the site that needs to be configured.
2. click on the the left menu bar Cache Prefetch.
3. On the Cache Prefetch page, enter the corresponding resource content and click OK.
4. Switch to the History Record tab to view the history records within a specified time range (within one month).

Scenario Two: Batch import preheating cache content by uploading files

If you have more content to preheat or have already placed the content in a file, you can choose to upload the file:

1. Log in to the EdgeOne console, and find the site that needs to be configured.
2. Click on the left menu bar Cache Prefetch.
3. On the Cache Prefetch page, select the "Upload File" method and upload the file, then click Confirm Preheating.
4. Switch to the History Record tab to view the history records within a specified time range (within one month).

Cache Purge

When your resource content is cached to the EdgeOne edge node, during the cache validity period, users accessing the resource will be directly responded by the EdgeOne edge node without triggering a return to the origin. If your origin site updates the resource content at this time, in order to prevent users from still accessing the old resource files, you can manually clear the cached resources in all edge nodes by using the Cache Purge function. After the cache is cleared, when users access the resource, EdgeOne will follow the origin to obtain the latest resource for response.

Use Cases

You may need to use this function in the following scenarios:

• Content update: When you have updated some resources on the origin, but the cache on EdgeOne has not expired, you may want to let the client user see the latest content immediately.
• Error fix: If there are some erroneous contents cached on EdgeOne from your origin, to avoid business risks, you need to purge these erroneous caches immediately to ensure that EdgeOne no longer provides the wrong content.
• Testing and debugging: When developing and debugging a website, you may need to frequently modify and test the content. To ensure that you see the latest modified content rather than the cached old version, you can use the clear the cache function.
• Emergency response: In some emergency situations, such as being attacked or releasing sensitive information, you need to remove the relevant content from EdgeOne immediately.
• Cache Rules adjustment: When you adjust or optimize the EdgeOne Cache Rules, you may need to purge the existing cache to ensure that the new Rules are Effective immediately.

Support Type

EdgeOne clears the cache support based on various types of purging, details as follows:

EdgeOne Cache Purge is divided into direct deletion and mark as expired methods, as follows:

• URL Type and Cache-Tag Type are set to "directly delete" by default, which means directly deleting the cached content. When a user sends a request for resources, EdgeOne will immediately origin-pull the latest resources, increasing the number of origin-pull requests within a short time and weakening the acceleration effect. If a large amount of content is submitted, the origin will be under greater pressure.
• Other purge types are set to "mark as expired" by default, which means that the cache will not be directly deleted, but marked as expired. If the cache node has Last-Modified and Etag headers, the next time a user requests the resource, the node will carry If-None-Match and If-Modified-Since headers for origin-pull verification whether the resource has been updated. The response of 304 or 200 is determined by the origin, generally speaking:
  • If there is no update - the origin returns 304 (Not Modified), then the node continues to use the cache to respond, effectively saving bandwidth.
  • If there is an update - the origin returns 200 (OK), then the node collects the latest resources from the origin and compares the Last-Modified and Content-Length headers of the cached resources and the new resources. If either header value is different, the new resource will overwrite the expired cache on the node.

Directions

Scenario One: Clear cache by entering content

If you have a small amount of content to clear and it is convenient to enter the content directly in the input box, you can follow these steps:

1. Log in to the EdgeOne console, and find the site that needs to be configured.
2. Click on the left menu bar Cache Purge.
3. On the Cache Purge page, select the resource type you want to clear, enter the corresponding resource content, and click OK.
4. Switch to the History Records tab to view the history records of the specified time range (within the last month) and purge type.

Scenario Two: Clear cache by uploading a file for batch import

If you have a large amount of content to clear or have already placed the content in a file, you can choose to upload the file:

1. Log in to the EdgeOne console, and find the site that needs to be configured.
2. Click on the left menu bar Cache Purge.
3. On the Cache Purge page, select the resource type you want to clear, choose the "Upload File" method, upload the file, and click OK.
4. Switch to the History Records tab to view the history records of the specified time range (within the last month) and purge type.

Effective Cache Purge and Cache Prefetch after Content Submission

Cache Purge

Cache Prefetch

Note

• When the cache TTL configured for a file is less than 5 minutes, it is suggested not to use the purge tool, but to wait for the timeout update.
• The actual total time for any type of cache purge mainly depends on the quantity of submitted content, the more content, the longer the waiting time.
• The actual total time for cache prefetch mainly depends on the file size, the larger the file, the longer the waiting time. The prefetch effective time for more large files (≥100MB) may be extended, exceeding 30 minutes.

Additional Information and Tips

For more information on EdgeOne cache configuration, cache rules, cache keys, and related features, you can continue to learn through the documentation. To improve EdgeOne's cache hit rate, please refer here.