problem

My sites are built with Hugo and run on Netlify. I write a lot of posts (in 2019, I wrote 66) but I like to spread them out so that they don't all drop at once. To do this I use the date field on my content to schedule publish at a future date. However, Hugo only re-publishes on rebuild so in order for that scheduled content to go out, I need my site to rebuild on a recurring schedule otherwise my content won't be published until the next time I push to my repo which could be days.

Because I host my repos on GitLab, how can I use the built-in GitLab CI to schedule rebuilds on Netlify?

solution

GitLab CI is full featured, so making this happen won't be a problem - I used to use GitLab CI to build and publish my containers when I ran my sites on Kubernetes. Netlify also makes publishing super easy with their publish web hooks, we just need to make sure we keep that hook secret so that bad actors don't DOS (or perhaps worse rack up a large bill) for our accounts.

I'll paste the code I use to push my code to Netlify from GitLab CI below and walk through what's going on and additional steps you'll need to take to get it working for you.

This is my project's .gitlab-ci.yml - it must live in the root of your repo for GitLab to use it and basically it tells GitLab CI what you want it to do.

image: docker:stable
services:
  - docker:dind

stages:
  - deploy

deploy_site:
  stage: deploy
  before_script:
    - apk add --update curl && rm -rf /var/cache/apk/*
  script:
    - curl -X POST -d '{}' $NETLIFY_BUILD_WEBHOOK
  only:
    - schedules 

From top to bottom, this says:

• use a docker:stable image to run this GitLab CI pipeline
• also install the optional docker:dind service - this gives us access to curl later on
• in this pipeline, the stage we want to run is deploy - for more advanced pipelines, there could be multiple stages with multiple steps in each, but this one is v simple so only needs one stage with one step
• define the deploy_site step
  • it's a part of the deploystage
  • before we run this step, we want to install curl
  • the step then uses curl to HTTP post an empty blob to our Netlify web hook which will tell Netlify to rebuild our site - note that we use a GitLab CI variable here so that we can keep that secret out of our code and away from the prying eyes of bad actors
  • finally we say only do this step when this pipeline is run from schedules - we do this because Netlify has built-in push webhooks, so running this on things other than schedules would result in duplicate builds which is suboptimal

You can plop this in your repo and we're almost ready to go. The last things we need to do are set your $NETLIFY_BUILD_WEBHOOK so our pipeline knows what url to post to and create a cron job (a job that runs on a schedule) to run our pipeline as often as we want.

setting $NETLIFY_BUILD_WEBHOOK

To set your $NETLIFY_BUILD_WEBHOOK variable:

First find your webhook on Netlify.

• Go to your Netlify project for your repo
• Go to Deploys > Deploy Settings
• Find the Build Hooks section
• the url there is your build hook!

Now we need to set your variable in your GitLab repo

• Go to your GitLab repo
• Go to Settings > CI/CD
• Find the variables section and click expand
• Create a new variable filling in like so
  • Type: variable
  • key: NETLIFY_BUILD_WEBHOOK
  • value: The url you found in the previous step
  • State: protected - we want to protect this because bad actors could use it to do mean things
• click Save variable at the bottom

creating a cron job / scheduling your pipeline to run

Okay now we have our pipeline all set up but we need something to actually call it. We'll run our pipeline on a schedule using GitLab CI's built-in cron scheduler. To do so:

• Go to your GitLab repo
• Go to CI / CD > Schedules
• Click New Schedule
• Write a description for your schedule (I named mine Scheduled deploy)
• in Interval Pattern select a default pattern or create your own using Cron Syntax - there's a link there for cron syntax so I wont' explain it here
  • A note: Netlify does have limits on number of build minutes you can use before they charge you. While your site probably doesn't take that long to build, you do want to make sure you don't choose rebuilding every minute as that could easily set you over the allotted build minutes and either cost you serious money or prevent you from rebuilding your site til the cycle resets. I put my builds to happen twice a day (every 12 hours) as it balances building when new content might be available with not blowing my build budget but you should choose what works for you.
• Save!

That's it, you now have scheduled builds on Netlify using GitLab CI!