Wordpress to 11ty

Tuesday 20, October 2020

I put off my migration from Wordpress over to 11ty far too long as I was put off by the amount of content I would have to migrate.

Over time I made my Wordpress site do more than just be a blog. At one point I was using it as a micro blog and syndicating out to twitter. I was also importing my instagram posts. I was trying and to good effect owning all my content.

With all this content housed within Wordpress I was put off having to deal with the migration.

Then the other weekend I decided I would just migrate the blog posts and nothing else and see what was involved in the migration process.

11ty touts itself as a "simpler static site generator". It's a simple as adding a markdown file to a directory and running the 11ty command to generate a static site, it is zero config by default. Having played with other static site generators it is certainly the simpliest to get up and running.

With 11ty being so simple it makes you think, am I missing something. I took some time to understand how best to approach setting up my blog.

11ty has many template languages available out the box. When you run the 11ty command it will process them and create outputted HTML. On top of the template languages 11ty also has front matter. This confused me a little at first.

The gist of front matter and template languages are that they co-exist with each other. You use both together, front matter allows you to place some YAML syntax at the top of your source files to provide 11ty with data/config/options. I am using markdown templates for my blog posts, but if you look at one of my posts it doesn't look like a markdown file you may of written or edited before.

–––
	title: "Wordpress to 11ty"
	date: "2020-10-20"
	permalink: "wordpress-to-11ty/"
	layout: "layout/post"
–––
<p>
	I put off my migration from Wordpress over to 11ty far too long
	as I was put off by the amount of content I would have to migrate.
</p>

This is the first part of the markdown file for this blog post. You start of the file with the front matter YAML, using three dashes and end the front matter data with another three dashes. After the second set of dashes you add your content. As I am using markdown files I can either write markdown or straight up plain HTML.

Having understoond how front matter and 11ty work together I begun to work on writing a script to migrate my Wordpress blog posts. Wordpress provides an JSON API to access your data. This may not always be enabled for your instance. It's well documented on Wordpress sites/forums on how to enable it and the endpoints available.

With the API enabled I was able to hit the following URL - http://matthewroach.me/wp-json/wp/v2/posts?page=1&per_page=10&categories_exclude=33,36,82 to get the latest 10 blog posts.
The categories_exclude param is me telling the API to not return posts for the categories, Instagram, Status and Bookshelf.
I also used the per_page param to limit to 10 to test creating my markdown files.

Getting JSON was the first part, transforming the JSON into markdown files with the necessary front matter data for 11ty was the next step.

There is a handy node package json-to-frontmatter-markdown that only does one thing, transform a JavaScript object to markdown with front matter.

By combining the the json to frontmatter markdown module and node's HTTP functionality you can easily make the API request to the Wordpress API and pass the JSON response through the transformAndWriteToFile function to generate a markdown file for each blog post.

The full script I used to fetch and create the 11ty markdown files for all my posts is below.

I only migrated the main post body, title, date and slug (url). I wasn't concerned with the categories/tags I had previously set up. This makes my import script pretty simple. If you would like to mainain the other items I've not it's pretty simple to do.

The script is a node script. I created an index.js file in the root of my 11ty blog and then ran it using node index.js. This processed the files and saved them to src/posts which is configurable by the path option

const http = require('http');
const transformAndWriteToFile = require('json-to-frontmatter-markdown').default;

http.get('http://matthewroach.me/wp-json/wp/v2/posts?page=1&per_page=100&categories_exclude=33,36,82', (response) => {
	let data = '';

	// called when a data chunk is received.
	response.on('data', (chunk) => {
		data += chunk;
	});

	// called when the complete response is received.
	response.on('end', () => {
		let items = JSON.parse(data);
		items.forEach(post => {
			const date = post.date.split('T')[0]; // Split data on T to only get YYYY-MM-DD
			transformAndWriteToFile({
				frontmatterMarkdown: {
					frontmatter: [
						{ title: post.title.rendered },
						{ date: date },
						{ permalink: `${post.slug}/` }, // Must have trailing slash to if you want pretty URLs
						{ layout: "layout/post" }
					],
					body: post.content.rendered
				},
				path: './src/posts', // Location to place the files
				fileName: `${date}-${post.slug}.md`
			});
		});
	});

}).on("error", (error) => {
	console.log("Error: " + error.message);
});

Handling images

Moving text is the simple part. When moving images you'll need to down all the images from your Wordpress blog and update the references in the post.content.rendered part of the script.

My script does not handle any image updating as I previously set up my Wordpress blog to host images on S3 and be served via cloudfront. This allowed my migration to be much smoother.

I did happen to have a couple of posts that some how had images that where being served via the wp-uploads directory. For those posts I downloaded the images from my server and manually uploaded them to the S3 bucket. Making sure to maintian the filenames, this then allowed me to do a find and replace in the markdown files to update the src.

Wordpress API limit

The Wordpress API only allows you to fetch a maximum of 100 posts per request. Rather than writing extra code to loop through and make multiple API requests and collect all the data before processing you can run the script multiple times, and each time I changed the page query string paramenter. I had just over 100 posts so I had to only run the script twice.

Running the script more than once will have no negative impact, if there is a file already in the folder it will just overwrite it