In case it wasn't obvious, this site runs on Ghost. I made this decision a couple of years ago on a 4 hour plane ride when I was bored, and wanted to kill two birds with one stone: start learning how some of the AWS services worked, and get a website going. At the time I didn't really make the site very easy to maintain, and had to learn a couple of lessons over the years as a result.
Recently, Ghost 1.0 was released, which broke my current setup. Additionally, the SSL certificate I had bought previously was up for renewal. When I bought it, I was not familiar with letsencrypt, so I was basically throwing away $10 a year for a service that I didn't need to be paying for. I decided to take this opportunity to re-engineer the deployment to be ephemeral and easy to maintain.
I had several items that I wanted to implement for my new deployment:
- I wanted to use containers to break the deployment into modular pieces. This would give me the means to blow away a container if one of the pieces of the site wasn't working, instead of having to spend (potentially) hours debugging.
- I wanted to use (as mentioned above) letsencrypt to renew my ssl certificate on a weekly basis.
- My previous deployment used SQLite. I wanted an excuse to check out MariaDB, so I decided to use this as my new backend DB.
- I wanted daily backups using S3.
- I wanted to be able to destroy, update, and rebuild the site automatically on a weekly basis.
- If there was an issue with the system running docker engine, I wanted it to be relatively painless to build a new one. To do this, I decided to use Ansible.
I ended up making the docker work relatively friendly for others to use (in case anyone else wants a Ghost + Nginx + MariaDB + LetsEncrypt docker deployment). The resources I used to get to this point didn't quite address every item I wanted, or didn't fully work. As a result, this part took a bit of time, especially given this was a spare time project I took on right after my wife and I moved a couple of states over.
Once I got the docker stuff figured out, I started working on how to automate standing up the containers and restoring the data. Previously, I've done a fair amount of work with Puppet and have also played around with Ansible. Out of the two, I found Ansible to be a lot more intuitive, and given that most of the logic for the site was going to be in the containers, I didn't see a reason to justify having the overhead of an agent running on the system.
This is what I ended up with. It works to consistently take a base Ubuntu 16.04 box, install Docker, Docker Compose, restore the site data, set up the cronjobs for automated backups, updating, etc. and stand up techvomit in about 6 minutes, give or take. The biggest components for time are the two minutes it takes to set up the folder structure the first time you run the containers, and the four minutes you have to wait for the letsencrypt container to do its thing.
In the future, I'd like to have some logic built in that would have the system rebuild itself in two steps should any issues arise. The first step would be to rebuild the docker containers (
docker-compose up -d --force-recreate). After two attempts of using this method to fix the site, if the problem was not resolved, an email would be sent to the site admin (me), and then the system would be rebuilt using ansible-techvomit and the AWS API. I might employ Terraform for this, but it's probably overkill.
I'm not sure when I'll get to this work, as I'm currently working on learning Go, but I'll come back around to it. I'd also like to find a new theme to run on here as well; this one works great, but isn't really being maintained.
In the meantime, the site runs great I've got good backup/restore methods in place, and updating it is a breeze.