Converting to Jekyll and Automating Deployments via Google Cloud Build

If you’re visiting benpatterson.io today, things are looking a little unpolished. I converted the site to a jekyll site and opted not to bring over the previous styles.

There were a few reasons for the change. One, was that the previous site was tightly coupled to google app engine and images and entries were stored on google’s services. If I ever thought of migrating, I’d be stuck. I wanted to get my entries back into a markdown format and into github. I also had the app running in three different frameworks. There was an api running on a flask app, a frontend running a vue app an admin console running a react. I learned a lot setting things up but I don’t use any of those technologies on a daily basis. My current job has me programming in Ember and Express. I’d have to ramp up again on three different frameworks if I wanted to make even minor changes.

I also wasn’t happy with the loading speed of the blog. The entries were access through an api call after the vue app loaded. There was a perceivable delay between the events. Google app engine would put the server to sleep when there wasn’t traffic and you might end up looking at a content-less page for few seconds.

Instead of fixing it and relearning the three frameworks, I opted to convert it to a single framework that was a better fit. I chose jekyll because I’ve used it before and could get something together quickly. I like being able to write in markdown and like the idea that I could host the site wherever I wanted after making the conversion.

Converting the Site

The first thing I did was make a branch and copy the code for Quarantine Network into the current repo and start deleting old code.

I deleted the vue app, the flask app and the admin app, any other cruft that was in the repo. Addressing styling later would mean that I could actually achieve the goals of converting the site in the time available. It’d be better to hunt down jekyll themes later.

The next step was converting previous entries and drafts into the jekyll format. For that, I used the api on the deployed flask app. The api had a hardcoded limit of ten entries at a time but it didn’t take long to visit the appropriate pages in the browser five times and create five files. It’s tempting to automate everything but the pragmatic side of brain took control in this situation.

I did throw together a small node.js script for converting the posts into markdown. The text of the entries had been created in a quill editor and had been saved in an html format. To convert that html back into markdown, I used turndown.

process-entries.js

const fs = require('fs');
const moment = require('moment');
const { posts } = require('./recovered/posts-5');
const { kebabCase } = require('lodash');
const TurndownService = require('turndown')
 
const turndownService = new TurndownService()

posts.forEach(({ created, entry, published, title})=>{
    const createdDate = moment(created,  'MMMM DD, YYYY - hh:mm');
    const fileName = `${createdDate.format('YYYY-MM-DD')}-${kebabCase(title)}.markdown`;
    const markdownEntry = `---
title: '${title}'
layout: post
date: '${createdDate.format('YYYY-MM-DD hh:mm:ss')} -0700'
comments: true
---

${turndownService.turndown(entry)}

`

fs.writeFileSync(`${__dirname}/jekyll-app/${ published ? '_posts' : '_drafts'}/${fileName}`, markdownEntry);
})

It wasn’t anything fancy but it got the job done. I’d only uploaded five images to google’s database so I just downloaded and saved those manually. If you’re converting your own blog and it isn’t custom like mine was, there’s likely an existing importer that you can use.

After that, I updated my app.yaml to serve a static site.

runtime: python27
api_version: 1
threadsafe: true

handlers:
  - url: /
    static_files: jekyll-app/_site/index.html
    upload: jekyll-app/_site/index.html
    secure: always

  - url: /(.*)
    static_files: jekyll-app/_site/\1
    upload: jekyll-app/_site/(.*)
    secure: always

I opted to go ahead and deploy it as is. It should have been as simple as using Google’s sdk to deploy it but I had to go back and update, billing, login through a browser and I forgot to build the production version of the site and it left out my analytics code.

Manual deploys are error prone and I didn’t want to have to go through that every time I wanted to post a blog or fix a typo.

Setting up the CI/CD pipeline

I used Google’s Cloud Build to automate deployments. I added a cloudbuild.yaml to automate the build and deploy steps. The quickstart instructions for setting this up are fairly straight forward so I recommend referring to them to get set up.

A few months after I initially posted this, I ran into an issue with builds silently failing. The permissions weren’t adequate anymore. After some research reading through the recommended role for application deployment, I think the solution was that I needed to add a Service Account User to the set of permissions. iam roles

The template seems to be a wrapper around docker’s api but it wasn’t quite the same. It wasn’t quite clear on what was persisted between steps. Installing dependencies on one ruby container didn’t seem to bring the dependencies to the next step but building the site worked fine. I tried all sorts of configs to install dependencies to different folder but ended up doing the entire install and build in a single step.

Here is the yaml file and the associated scripts.

cloudbuild.yaml

steps:
  - name: "ruby:2.7.1"
    entrypoint: /bin/dash
    env:
      - 'JEKYLL_ENV=production'
    args: 
    - "./build.sh"
  - name: "gcr.io/cloud-builders/gcloud"
    args: 
    - "app"
    - "deploy"

./build.sh

cd jekyll-app
# Strip metadata from images and resize them
mogrify -strip -verbose -resize 250000@ assets/images/**/* assets/images/*.jpg assets/images/*.png
bundle install
bundle exec jekyll build --config _config.yml,_config-production.yml

It’s been working pretty well and I’m glad to be able to push to github to deploy. I have a similar setup with AWS for quarantine.network. The only reason I didn’t do the same for this site was because I didn’t want to go through the process of migrating domain names, https certs etc.

Hopefully I can find a few minutes here and there to mess with styles and other features. It is a nice feeling to have the migration done regardless. There’s also comments now, so feel free to leave one.

Advertisement