Learn how to use getJSON or getCSV to get remote data with Hugo.
In a previous tutorial, we covered the basics of using Hugo data files. There’s one feature of data files we didn’t get a chance to get to — using
getCSV to get remote data. We’ll be going deep into this powerful feature of data files in this tutorial.
Typically, when you are using a data file in Hugo, you’ll put a YAML, JSON, XML, or TOML file in the data directory. But what if you don’t have a local file to hand, and want to download it from somewhere at build time? That’s exactly where remote data files come in.
Hugo has two functions to get remote data:
getJSONfor JSON data
getCSVfor CSV data
On build, it will download the data from the provided endpoint, which you can then iterate over or manipulate however you’d like. For example:
While you could download the response from an API endpoint and add to your Hugo site as a datafile manually, it would mean doing this every time the dataset is updated. Remote data files do this for you automatically, so you never need to worry if you’re using the latest dataset on your site.
There is an immensely wide range of ways you can use this feature, particularly if you combine it with something like scheduled builds which rebuild your site periodically. You can really start to blend the best worlds of static and dynamic.
We’ll go over a few scenarios which covers the tip of the iceberg of the possibilities of remote data files.
We can create and edit a data set in Google Sheets and expose it as a CSV for our Hugo site to consume on build. Let’s have a look at how it works.
First, we create the data set in a Google Sheet. Here I have a list of apples, their price, and associated color:
Now we can publish this publicly as a CSV. First go to File → Share → Publish to web:
And then select Comma-separated values from the dropdown:
This will give you a public URL which is a CSV representation of your Google Sheet. Now we’ll use this on our site:
Let me break down the trickier bits of this snippet:
getCSVtakes a minimum of two parameters. The first is the delimiter for columns in the CSV file which is a “,” in this case. The second is the endpoint for the CSV.
after 1 $apples.
$applesis an array, so we need to use
indexwith the column number to access each item in the row.
Putting this all together will output something like this:
If I change something in the Google Sheet and rebuild the site, you’ll see the changes reflected on the Hugo site. It’s an easy way to keep a data set up-to-date on the website.
Which generates the following:
This is a super easy way to bring a dynamic element to your site. You could even start to combine these methods. For example, you could have a list of repositories in a Google Sheet, then request the contributors list for each one.
Not all APIs are public; sometimes you need to provide authentication details or other headers to make an API request. Fortunately, with both getJSON and getCSV, you can add another parameter with customer headers on the request.
Let’s demonstrate this. The GitHub example above uses the public GitHub API. It’s easy to set up but is limited to 60 requests per hour. If we need more requests, we can run the request on behalf of our own GitHub account using a personal access token:
Hugo will download the remote file on every build, which will have an impact on your build performance. The extent of the impact depends on a number of factors — the size of the data file, the network speeds of the external server, and your local server, to name a few. You can mitigate this impact by caching remote files.
Here’s how it works. In your global config file, set a max age:
A value of
-1 will cache the file forever,
0 turns the cache off, or you can specify a time with:
14h, for example, means that if more than 14 hours have elapsed since your last build, your remote data file will be updated on your next build.
Using remote data files is a Hugo feature that unlocks entirely new ways of thinking about what a static site generator can do.
You’ll now have a solid foundation for further experiments with your own files. Try bringing remote data files into your own projects, and you’ll really be able to push your Hugo sites to the limit.