Automating Hugo Deployments with hillpush (Git-Based Workflow)

Overview

This site is a static Hugo site, written on my workstation and served from my home lab. I wanted publishing to be simple:

• Edit content
• Commit the changes
• Push to Git
• Have the live site update without me logging into the web server

The result is a small helper script called hillpush, a self-hosted Gitea instance as the source of truth, and a publish script on the web host that pulls, builds and syncs the site.

Since I first wrote this post, the pipeline has been through a real test: I accidentally deleted the original Gitea server and had to rebuild and re-wire everything. This revision covers the current design and what that rebuild taught me.

Paths and names in the examples are anonymised.

Architecture

There are three roles in the publishing path:

• My workstation, where I write content and run hillpush
• A Gitea server on the internal server VLAN, which holds the site repository
• A build-and-publish host in the DMZ, which pulls from Gitea, runs Hugo and serves the result

The important part is the direction of the arrows. The DMZ host pulls from Gitea using a read-only deploy key. It never holds credentials that can write to the repository. If the public-facing host were compromised, the attacker could read the site source, which is public anyway, but could not push changes back into it.

What hillpush Does

hillpush is run from inside the site repository on my workstation. It:

1. Finds the repository root
2. Fetches from the remote and rebases if anything has changed upstream
3. Prompts for a commit message, then commits and pushes to Gitea over SSH
4. SSHes to the publish host and runs the publish script as the web server’s user

In short, one command takes a change from my editor to the live site.

There is also a safety net. The publish host runs the same publish script on a short timer. If I only run a plain git push, or the SSH trigger fails, the change still goes live within a few minutes. hillpush makes it happen immediately.

The Publish Script

On the publish host, a script owned by the web server’s user does the build:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

#!/bin/bash
set -e

cd /srv/hugo/site

git fetch origin
git reset --hard origin/main
git submodule update --init --recursive

hugo --minify

rsync -a --delete public/ /var/www/site/

A few deliberate choices here:

• Hard reset, not merge. The clone on the server is disposable. Any local edit made there is thrown away on the next run. Git is the only source of truth, so the server can never drift.
• Submodules. The theme is pulled in as a Git submodule, so the build host needs to update it too. I missed this at first and got builds with a stale theme.
• rsync --delete. Pages I remove from the repo also disappear from the web root, instead of lingering as orphaned HTML.
• Runs as the web user. The build doesn’t need root, so it doesn’t get root.

Comments Use the Same Git Server

The site’s per-post comments are stored as Gitea issues, through a small bridge service that holds the API token server-side, so the browser never sees a credential. I’ve written that up separately in Self-Hosted Hugo Comments with a Private Gitea.

This matters here because it changed what the Git server is. When it only held the site repo, it was a convenience: every clone was effectively a backup. Once comments lived in its issue tracker, it held data that existed nowhere else.

Rebuilding After Losing the Git Server

I deleted the original Gitea container by mistake. The site source survived, because my workstation and the publish host both had full clones. The comments did not. Everything posted before the rebuild is gone.

Rebuilding Gitea itself was easy. Re-wiring everything that depended on it took longer:

• The repository owner changed. The new instance has a different owner path, so the remote URL on every clone had to be updated.
• The deploy key had to be re-registered. Deploy keys belong to the repository on a specific instance. A new instance means a new registration.
• API tokens were dead. A token issued by the old Gitea means nothing to the new one. The comments bridge failed with authentication errors until I issued a fresh, scoped token.
• Host keys changed. The publish host refused to pull because the Git server’s SSH host key was different.

Shortly after the rebuild I also moved Gitea to HTTPS-only with a proper certificate. That broke the comments bridge a second time, because it was still talking plain HTTP. Gitea responds to that with a fairly clear 400 error, once you know where to look.

What Worked / What Didn’t

What Worked

• Pull-based publishing from the DMZ. The public host has read-only access to the repo and nothing more.
• Disposable build clone. reset --hard meant I never had to think about the state of the server copy during the rebuild.
• The timer fallback. Even while I was fixing SSH access, any push eventually went live.
• Distributed Git. Every clone was a full backup of the site source.

What Didn’t

• Assuming Git was the backup for everything. It was the backup for the code, but not for issues and comments.
• Implicit dependencies. Tokens, deploy keys, remote URLs and host keys were all tied to the old instance, and nothing listed them in one place.
• Fixing things as the wrong user. I fixed a host-key problem as root on the publish host. That left the web user’s known_hosts file owned by root, which broke the next publish in a new and confusing way.

Tradeoffs

Advantages

• No external CI service and no build runners to maintain
• Every deployment corresponds to a Git commit
• The public host can’t write to the repository
• Works even if the on-demand trigger fails, thanks to the timer

Limitations

• Builds happen on the public-facing host, so Hugo has to be installed there
• The timer runs whether or not anything has changed. That’s cheap for a small site, but not elegant
• The Git server is a single point of failure for comments, so it needs real backups, not just Git clones
• There’s no build preview or staging; main is production

A webhook from Gitea to trigger builds would remove the timer, but it would mean opening a path from the internal network into the DMZ just for that. For a personal site, pulling on a timer is the simpler and safer option.

Lessons Learned

• Know what’s actually stateful. A Git server that also stores issues is a database, not just a remote. Back it up like one.
• Write down every credential that’s tied to an instance. Tokens, deploy keys and host keys all have to be redone after a rebuild. A checklist turns hours of debugging into minutes.
• Check the SSH user before blaming the keys. My Gitea runs under a non-default system user. SSH looked completely broken until I noticed I was connecting as git@ out of habit.
• Do fixes as the user that runs the job. Use sudo -u <service-user> for anything touching a service account’s SSH config. Running as root leaves files that the service can’t read.
• Make the server copy disposable. Hard-resetting to the remote on every build removed a whole category of “why is production different?” problems.
• Small scripts are still enough. Even after the rebuild, I didn’t need a CI platform. I needed better notes about what the scripts depended on.