Creating an 11ty collection from a JSON API
By Mike Street
11ty is a fantastic static site generator. One of my favourite elements of it is Global Data files. These allow you to have local JSON and other formats and build collections out of them.
One of the most exciting features is you can use a JavaScript file, which, amongst other things, allows you to fetch a remote JSON API, modify the data and build a collection out of it. I recently did that for my beer review site and thought I would delve into how that was achieved.
Set up data file
In you .eleventy.js
file, you can configure you data
folder location. By default, this is _data
, but you can change it to wherever you wish.
Within that folder you can make .json
or .js
files - the name of the file is the name of your collection. For this blog post, I'll be using the beer
endpoint of my review site (feel free to use it too), so create a beers.js
file in your data folder
_data/beers.json
As a minimum, set up an export from the file where your data will be returned
module.exports = async function() {
};
To test it is working, you can return a JSON array of objects
module.exports = async function() {
return [
{"title": "test"},
{"title": "test2"}
];
};
In your 11ty content file, you can then loop through the collection:
{% for beer in beers %}
{{ beer.title }}<br>
{% endfor %}
Fetch the JSON
The next step is to fetch our JSON instead of returning a static set of data. If this was client side, we could use the natic fetch()
function. Fortunately this can be installed for server-side use:
npm install --save node-fetch
We can then require that in our data file and fetch the JSON. Our function is async
so we can benefit from the async
/await
functionality
const fetch = require('node-fetch');
module.exports = async function() {
let url = "https://beer.mikestreety.co.uk/api/beers.json";
return await fetch(url)
.then(data => data.json())
};
Viewing our beer list now, will show all the beers that have been reviewed.
Cache the JSON
Each time we press save, the data file is processed, meaning it goes off to the API to retrieve the data. This is not only costly on your time, waiting for the server to respond but also costly on the server, having to deal with many requests.
Fortunately, eleventy-cache-assets
plugin exists. This allows you to fetch a JSON api and store it locally. It will then use this data until the cache expires (or you delete the cache folder). More details about the plugin can be found on the 11ty website.
Install the plugin:
npm install --save @11ty/eleventy-cache-assets
.cache
to your .gitignore
file - the last thing you want is to be comitting cached API responses!Next, we can replace our fetch
with the new plugin. The plugin also parses the JSON for us.
const Cache = require("@11ty/eleventy-cache-assets");
module.exports = async function() {
let url = "https://beer.mikestreety.co.uk/api/beers.json";
/* This returns a promise */
return await Cache(url, {
duration: "4h", // save for 4 hours
type: "json" // we’ll parse JSON for you
});
});
Our cache is stored for 4 hours which is more than enough time for our development process to not hammer our API.
Post process the data
Once our JSON is being loaded, we have the power of JavaScript to process the data. We can access our existing filters to utilise them and loop through each of the items to add or modify data for use in our templates.
Following the MVC methodology, your 11ty templates shouldn't do data processing, instead, move that to this data file.
At time of writing, I loop through the beers and create beer and brewery URLs (or slugs) using a slugify filter.
Finished result
This is an example of how I am post-processing the beer
API:
const Cache = require("@11ty/eleventy-cache-assets");
const slugify = require('./../filters/slugify');
module.exports = async function() {
let url = "https://beer.mikestreety.co.uk/api/beers.json";
/* This returns a promise */
const response = await Cache(url, {
duration: "4h", // save for 4 hours
type: "json" // we’ll parse JSON for you
});
let data = response.map(beer => {
beer.slug = `/beer/` + slugify(`${beer.title} ${beer.brewery} ${beer.number}`);
beer.brewery_slug = `/brewery/` + slugify(`${beer.brewery}`);
return beer;
});
return data;
};
The most recent code can be found on Gitlab, along with the brewery data file.
Conclusion
To reiterate, 11ty is such a fantastic platform for building static sites in all shapes and sizes.