🎉 Introducing Edgio v6 which supports Node.js v16. Learn how to upgrade. 🎉
Edgio
Edgio

Predictive Prefetch

Edgio allows you to speed up the user’s browsing experience by prefetching pages and API calls that they are likely to need.

Traffic Shielding

You might think that prefetching will put significant additional load on the infrastructure hosting your APIs. That’s actually not the case! Edgio only serves prefetch requests from the edge cache. It will never make a request to the origin if a prefetch request cannot be served from the edge cache, so your servers will never see an increased load.

Configuring the Service Worker

To enable prefetching, your site’s service worker needs to use the @edgio/prefetch library’s Prefetcher class. If your site doesn’t currently have a service worker, one can easily be created using Google’s Workbox.

Here’s an example service worker based on Workbox:

JavaScript
1import {skipWaiting, clientsClaim} from 'workbox-core';
2import {precacheAndRoute} from 'workbox-precaching';
3import {Prefetcher} from '@edgio/prefetch/sw';
4

5skipWaiting();
6clientsClaim();
7precacheAndRoute(self.__WB_MANIFEST || []);
8

9new Prefetcher().route();

Once you’ve created a service worker, code running in the browser window needs to register the service worker before prefetching can begin. How you do this depends on the front-end framework that you use.

If you’re not using a front-end framework, you can use the install function from @edgio/prefetch to install the service worker. Here’s an example:

JavaScript
1import install from '@edgio/prefetch/window/install';
2

3install();

Configuring Routes for Prefetching

To ensure that excessive prefetch traffic isn’t passed on to your origin, Edgio will serve prefetch requests when a cached response is available at the edge. By default, all prefetch requests will be cached at the edge for 2 minutes (see DEFAULT_MAX_AGE_SECONDS). Additionally, you may configure a route that caches responses at the edge and in the service worker within your router, optionally giving it longer cache time for greater performance. In this example we define a route that caches product API calls for one hour:

JavaScriptroutes.js
1import {Router} from '@edgio/core';
2

3export default new Router()
4  // Here we configure a route for the product API.
5  .get('/api/products/:id.json', ({cache, proxy}) => {
6    // In order to prefetch product data, we must cache responses at the edge and in the service worker.
7    cache({
8      edge: {
9        maxAgeSeconds: 60 * 60, // cache at the edge for one hour
10        staleWhileRevalidateSeconds: 60 * 60 * 24 * 365, // optionally serve stale while retreiving a fresh version from the origin
11      },
12      browser: {
13        serviceWorkerSeconds: 60 * 60, // cache in the browser using the service worker for one hour
14      },
15    });
16  });

Note that if you prefetch a URL without setting browser.serviceWorkerSeconds as shown above, the response will still be prefetched and cached by the service worker with a short TTL (2 minutes by default). You can change the default TTL by setting defaultMaxAgeSeconds when initializing the Prefetcher instance in your service worker:

JavaScript
1const prefetcher = new Prefetcher({defaultMaxAgeSeconds: 60 * 10}); // set the default TTL to 10 minutes

Prefetching a URL

To prefetch a URL, call the prefetch function from @edgio/prefetch/window. Here we prefetch data for a product page using the route we configured in the previous example.

JavaScript
1import {prefetch} from '@edgio/prefetch/window';
2

3prefetch('/api/products/1.json');

Prefetch requests are given the lowest priority. This ensures that they do not block more critical requests like API calls, images, scripts, and navigation.

Optionally is possible to override default TTL or the value of serviceWorkerSeconds defined in routes.js by providing the maxAgeSeconds option to prefetch function call. This option is applied only to that function call and doesn’t affect any other calls made later.

JavaScript
1import {prefetch} from '@edgio/prefetch/window';
2

3prefetch('/api/products/1.json', 'fetch', {
4  maxAgeSeconds: 300, // 5 minutes
5});

All prefetch function options can be found in its API Documentation here.

React

The @edgio/react package provides a Prefetch component that you can wrap around any link to prefetch the link when it becomes visible in the viewport:

JavaScript
1import {Prefetch} from '@edgio/react';
2

3function ProductLink({product}) {
4  return (
5    <Prefetch url={`/api/products/${product.id}.json`}>
6      <a href={`/products/${product.id}`}>{product.name}</a>
7    </Prefetch>
8  );
9}

By default, Prefetch will fetch and cache the URL in the link’s href attribute. If you have a single page app, you most likely want to prefetch the corresponding API call for the page rather than the page’s HTML. The example above shows you how to set the url property to control which URL is prefetched.

Next.js

If you’re using Next.js with getServerSideProps, use createNextDataURL from @edgio/next/client to prefetch the data for the linked page.

JavaScript
1import {Prefetch} from '@edgio/react';
2import Link from 'next/link';
3import {useRouter} from 'next/router';
4import {createNextDataURL} from '@edgio/next/client';
5

6export default function ProductListing({products}) {
7  const {locale} = useRouter(); // you can omit this if you're not using localization
8

9  return (
10    <ul>
11      {products.map((product, i) => (
12        <li key={i}>
13          <Link href={product.url} passHref>
14            <Prefetch
15              url={createNextDataURL({
16                href: product.url,
17                locale, // you can omit this if you're not using localization
18                routeParams: {
19                  // keys must match the param names in your next page routes
20                  // So for example if your product page is /products/[id].js:
21                  id: product.id,
22                },
23              })}>
24              <a>
25                <img src={product.thumbnail} />
26              </a>
27            </Prefetch>
28          </Link>
29        </li>
30      ))}
31    </ul>
32  );
33}
34

35export async function getServerSideProps({params: {id}}) {
36  const products = await fetch(/* fetch from your api */).then((res) =>
37    res.json()
38  );
39

40  return {
41    props: {
42      products,
43    },
44  };
45}

Vue

The @edgio/vue package provides a Prefetch component that you can wrap around any link to prefetch the link when it becomes visible in the viewport:

JSX
1<template>
2  <Prefetch v-bind:url="/api/for/some/page">
3    <router-link v-bind:to="/some/page">Some page</router-link>
4  </Prefetch>
5</template>
6

7<script>
8  import Prefetch from '@edgio/vue/Prefetch'
9  export default {
10    components: {
11      Prefetch,
12    },
13  }
14</script>

By default Prefetch will fetch and cache the URL in the link’s to attribute (for both router-link and nuxt-link). If you have a single page app, you most likely want to prefetch an API call for the page rather than the page’s HTML. The example above shows you how to set the url property to control which URL is prefetched.

Deep Fetching

By default, prefetching only fetches the JSON API data or HTML document for a prefetched page. In order to achieve truly instant page transitions, all of the page’s assets above the fold need to be prefetched as well. These typically include images, CSS, and JavaScript. This is where “deep fetching” comes in. Deep fetching parses the prefetched page and then fetches the important assets of the prefetched page that you specify.

To add deep fetching to your project, add the DeepFetchPlugin to your service worker. The DeepFetchPlugin is then configured with an array of selectors that describe which assets need to be prefetched:

JavaScript
1import {Prefetcher} from '@edgio/prefetch/sw';
2import DeepFetchPlugin from '@edgio/prefetch/sw/DeepFetchPlugin';
3

4new Prefetcher({
5  plugins: [
6    new DeepFetchPlugin([
7      {
8        /* Deep fetching configuration objects go here */
9      },
10    ]),
11  ],
12});

The DeepFetchPlugin can parse both HTML and JSON documents to extract the page assets that must be deep fetched. For Edgio projects that are headless (i.e. the front end communicates with the backend through an API), you’ll typically use the JSON option. However if the backend and front-end endpoints are communicating using HTML responses then you’ll want to use the HTML option. Note that you can mix both HTML and JSON configuration objects in the an array passed to the DeepFetchPlugin.

Deep fetching URLs in JSON responses

For JSON responses, you’ll pass the DeepFetchPlugin an array of DeepFetchJsonConfig interface objects. These DeepFetchJsonConfig objects describe the asset URLs in the JSON response that should be prefetched. For example, the snippet below finds product images to deep fetch for a category page response:

JavaScript
1new DeepFetchPlugin([
2  // parses the category API response to deep fetch the product images:
3  {
4    jsonQuery: 'Bundles.[**].Products:products(Product).MediumImageFile',
5    jsonQueryOptions: {
6      locals: {
7        // filters out null products:
8        products: (input) => input.filter((prod) => prod),
9      },
10    },
11    maxMatches: 10,
12    as: 'image',
13  },
14]);

The jsonQuery syntax is provided by the json-query library. You can test your JSON queries using their JSON-query Tester Sandbox.

Deep Fetching for HTML documents

To deep fetch HTML documents, pass the plugin objects that match the DeepFetchHtmlConfig interface and describe which HTML elements need to be prefetched via CSS selectors.

For example, imagine you’re configuring prefetching for a product page and you want to ensure the main product image is prefetched so that it appears immediately when the page loads. If the main product image is displayed with an HTML img element with a CSS class called product-featured-media, it can be prefetched by adding the following to the DeepFetchPlugin:

JavaScript
1import {Prefetcher} from '@edgio/prefetch/sw';
2import DeepFetchPlugin from '@edgio/prefetch/sw/DeepFetchPlugin';
3

4new Prefetcher({
5  plugins: [
6    new DeepFetchPlugin([
7      {
8        selector: 'img.product-featured-media', // CSS selector syntax - just like you would use with document.querySelector()
9        maxMatches: 1, // limits the number of matched elements to prefetch to 1 per page
10        attribute: 'src', // the attribute holding the URL to prefetching
11        as: 'image', // the type of asset being prefetched
12      },
13    ]),
14  ],
15});

Computing the URL to be prefetched

In the example above the img element’s src attribute contains URL that needs to be prefetched. Sometimes finding the URL to prefetch is not so straightforward. For example, apps sometimes use JavaScript to compute the URL for responsive images based on the user’s device size. In such cases you can provide a callback function which will be passed all matching elements and decide what URLs to prefetch. Here is an example:

TypeScript
1import {Prefetcher, prefetch} from '@edgio/prefetch/sw';
2import DeepFetchPlugin, {
3  DeepFetchCallbackParam,
4} from '@edgio/prefetch/sw/DeepFetchPlugin';
5

6new Prefetcher({
7  plugins: [
8    new DeepFetchPlugin([
9      {
10        selector: 'img.grid-view-item__image',
11        maxMatches: 4,
12        as: 'image',
13        callback: deepFetchResponsiveImages,
14      },
15    ]),
16  ],
17});
18

19function deepFetchResponsiveImages({$el, el, $}: DeepFetchCallbackParam) {
20  const urlTemplate = $el.attr('data-src');
21  const dataWidths = $el.attr('data-widths');
22

23  if (dataWidths && urlTemplate) {
24    const widths = JSON.parse(dataWidths);
25

26    for (let width of widths.slice(0, 2)) {
27      const url = urlTemplate?.replace(/\{width\}/, width);
28      prefetch(url, 'image');
29    }
30  }
31}

Reducing 412s

By default, Edgio will only serve prefetch requests from the edge cache. If a request cannot be served from the cache, a 412 status is returned. This protects your origin servers from additional traffic associated with prefetching. If you’re seeing a surprisingly high number of 412s in your logs:

  1. Ensure that the URLs you’re prefetching match exactly those that are fetched during page navigation. Prefetch URLs will have ?edgio_prefetch=1 whereas the URLs associated with page navigation won’t. That’s okay. The edgio_* query parameters are automatically excluded from the cache key. Just ensure that there are no other differences.
  2. Ensure that cache settings have stale-while-revalidate enabled. For example:
JavaScript
1router.get('/p/:productId', ({cache}) => {
2  cache({
3    edge: {
4      maxAgeSeconds: 60 * 60,
5      staleWhileRevalidateSeconds: 60 * 60 * 24, // this way stale items can still be prefetched
6    },
7  });
8});
  1. Consider increasing edge.maxAgeSeconds. The shorter the cache time to live is, the more prefetches will fail.
  2. Set the includeCacheMisses install option to true. This should be used with caution and is not recommended for use in production because it will significantly increase the traffic to your origin or API servers.
JavaScript
1import install from '@edgio/prefetch/window/install';
2

3// Call the following once when the page loads to allow prefetch requests to be served when responses
4// aren't available in the edge cache:
5install({includeCacheMisses: true});

The cache-manifest.js File

This file is generated at runtime and is used by the Prefetcher class from @edgio/prefetch to add routes to the service worker. The routes ensure that custom cache keys and the serviceWorkerSeconds properties from the cache() settings in your router are propagated to the service worker.

For more information on Prefetcher, serviceWorkderSeconds, and cache(), see Class Prefetcher.