Server Side JavaScript and Static Websites

I’ve experimented with server-side JavaScript in the past, but I finally had an opportunity to dig in and actually build a website with it. I have a lot of experience with JavaScript in general; one of my major professional projects over the past few years was developing a complex web application entirely in JavaScript. For a number of reasons, I had to work only with vanilla JavaScript (no frameworks or libraries like jQuery). But server-side Javsacript is an enitrely different challenge.

JavaScript has been around for almost 25 years, but (despite the vision of its creators at Netscape) for most of that history it lived in the client-side, in your browser. In other words, it was used to manipulate web pages after they were delivered to visitors, but JavaScript didn’t play a role in generating web pages in the first place. In the past ten years or so, projects like Node.js have enabled the creation of rich websites and web applications entirely in JavaScript. Largely as a result, JavaScript has been the most popular language across Github’s¹ millions of code repositories for each of the past six years. It was finally time for me to get in on the trend.

I’ve been building websites for close to twenty years now. When I started as a middle-schooler writing code on clunky iMacs in the computer lab, I made static websites with HTML and CSS. The first big step forward in my progress as a web developer was learning about server-side scripting and databases with the LAMP stack. A “stack” is a set of languages, libraries, and tools used to develop an application. LAMP stands for Linux, Apache, MySQL, and PHP. Linux is the operating system the web server runs on; Apache is the server software, MySQL is the database, and PHP is the scripting language. I learned to develop with the LAMP stack because it was ubiquitous at the time and because my college offered free server space to students, and those servers ran the LAMP stack.

The LAMP stack has some advantages. PHP is realtively easy to learn, and it’s a forgiving language in a lot of ways. SQL databases are powerful and great at storing lots of information. As a result, complex web applications like Facebook² and Wordpress are written in PHP.

PHP generates dynamic web pages. In other words, the scripts on the server generate a web page every time a client requests one. For example, a WordPress blog stores posts in a MySQL database. If the home page shows the last ten blog posts, every time someone visits the home page, a PHP script will connect with the database, pull information on those posts, and then generate HTML³ to send to the visitor’s browser. That’s a good approach, because it means that if we publish a post to the database, it will update across the entire website (i.e. the home page, the unique page for that post, pages for any tags or categories associated with that post, etc.). This approach also makes it easy to reuse elements of a webpage, like a navigation bar or footer, because the PHP script can pull those elements from a separate file. So if we want to add a “Contact” link to the main navigation bar, or update the copyright information in the footer, we just need to make those changes in one place, and they’ll propagate to the rest of the website.

A lot of server-side JavaScript platforms, by contrast, generate static websites. That means the server doesn’t do any generation of these pages, just delivers them, already built, to the browser. That makes static websites faster from the visitor’s perspective, but it makes them harder to update if you’re creating the website.

And that’s where the server-side JavaScript comes in. Instead of relying on a database and server scripts to dynamically generate web pages in response to visitors’ requests, static site generators completely rebuild the website every time something changes. That process allows creators to capture a lot of the advantages of a dynamic website—like reusing page elements and and pushing content updates across the site. That probably sounds like a lot of work, and it is; building the site will typically take several minutes. But once that process is finished, the site will be lightning fast.

Picking a Framework

Once I’d decided to experiment with building a static website, I needed to pick a framework. I landed on Gatsby, which is constructed on top of the React Javascript Library. React in turn is layered on the Node.js JavaScript runtime environment. I also considered Jekyll and Hugo. But Jekyll is built with Ruby, and Hugo with Go, and my desire to focus on server-side JavaScript won out. Additionally, at least compared to Jekyll, Gatsby is newer, faster, and more flexible and extensible. It has a steeper learning curve, but I thought I was up to the challenge, so Gatsby was my choice.

I had a lot to learn. It was particularly important for me to learn about React, because Gatsby’s system of components inherits React’s. I started with this course on React. I found that my background in JavaScript and experience with other web development technologies made learning React straightforward. My site was previously built on WordPress using a custom theme and plugin to add all the functionality and styling I needed. As a result, this post will often compare my experience building this site with Gatsby to my experience using WordPress.

Getting Started

I followed the instructions in the Gatsby tutorial to install Node Version Manager (remember that Node is the basis for React and, in turn, Gatsby). Then I installed Gatsby’s command line interface. I began with Gatsby’s default starter website. The default starter is exactly what it sounds like—a very basic website implemented through Gatsby with few features. There are plenty other Gatsby starters available through Gatsby’s library, including several with more advanced features (like the Gatsby advanced starter, Lumen, and the Hero Blog). But since this was supposed to be a learning opportunity, I started with the most basic and implemented most of the advanced features myself.

One of the things I quickly came to appreciate about Gatsby was how much work it does to optimize a website in the background. First, Gatsby handles things like minification, code-splitting, image optimization, and search engine optimization at compile time. Two more key examples: first, using the Gatsby Manifest plugin, I can specify a few variables—like the site’s icon and theme color—once in the site’s configuration file (gatsby-config.js), and they’ll be applied across the entire site. Second, the Gatsby Offline plugin will transform your website into a progressive web application for superfast loading. There are WordPress plugins that do a lot of these things, but in my experience they don’t work so well. They require a ton of configuration, and even then often don’t do a great job. I think that’s largely because, since WordPress doesn’t compile your website, it doesn’t know where all your code is or which parts of it are used on which page. Gatsby, by contrast, does compile the site and requires explicit imports to include code.

Another thing I really liked about Gatsby was how it encouraged me to modularize my code. Gatsby components are essentially React components, so the work I put in to learn React really helped here. The result was small, configurable website elements that I could use and reuse in a lot of really neat ways. And Gatsby also encourages component-scoped styles, so that CSS styles only apply to their relevant components.

Custom Post Types

One of the things I’ve come to appreciate about Gatsby is that it doesn’t force any structure on your website. I’m used to deploying WordPress-powered sites, and WordPress imposes a pre-imagined structure on your website. But my website requires a number of different types of posts, including photos, travel journal entries, and general posts. In WordPress, I dealt with that by creating a plugin that generated custom post types for photos and journal entries. In Gatsby, I just needed to modify the queries that generate index pages to filter posts by type.

I installed the Gatsby MDX plugin (one of the best things about Gatsby is its incredible plugin library), so I could write React components into my posts as necessary. Most static site generators encourage writing content in Markdown, which is great, but MDX allows significantly more flexibility. For example, the footnotes and images in my posts are all created using JSX, a React-based syntax JavaScript extension. Since I’m using MDX, my GraphQL query for default post types (in my gatsby-node.js file) filters posts like this:

allMdx(
  filter: {frontmatter: {type: {eq: "post"}}}
  sort: { order: ASC, fields: [frontmatter___date] }
  limit: 1000
)

Similarly, the queries for Expeditions entries and photos look for the “type” field in the post frontmatter and filter posts that way. I found this Medium post particularly helpful in building out the infrastructure for custom post types.

Pagination

Another important feature that you can take for granted in blogging software like WordPress is pagination. So if you have hundreds of blog posts but only want to display ten on each page, WordPress will generate all the index pages you need and link them all up very nicely for you. Gatsby doesn’t do that; you need to explicitly tell it how to generate index pages. That means more work up front, but it’s also more flexible; once I figured out how to set up pagination, I was easily able to create separate indices for each of the post types I’d created.

Relying on Gatsby’s documentation, I created a method in my gatsby-node.js file that generated paginated indices:

let createPaginatedIndices = (createPage, numPages, postsPerPage, label, templatePath) => {
  Array.from({ length: numPages }).forEach((_, i) => {
    createPage({
      path: i === 0 ? `/${label}` : `/${label}/${i + 1}`,
      component: path.resolve(templatePath),
      context: {
        limit: postsPerPage,
        skip: i * postsPerPage,
        numPages: numPages,
        currentPage: i + 1,
      },
    })
  })
}

Then I just needed to call createPaginatedIndices for each post type, and pass it the proper parameters.

Tags & Categories

Blogging software like WordPress imposes a lot of structure on your website, but the advantage is that it brings a lot of features. A couple of examples include tags and categories, organizational methods typically used in blogs and websites. Gatsby doesn’t offer those features by default, but it provides pretty solid documentation to roll your own. By combining that documentation with the code from the Gatsby advanced starter template, I was able to create the framework I needed.

I should note here that the use of MDX was important to this process. I used MDX for my general posts and Expeditions travel journal entries, but I used vanilla Markdown for photos. MDX and Markdown come from separate data sources in Gatsby, and they’re identified by their filename extensions (.mdx for MDX and .md for Markdown). As a result, it was easy to construct one query that pulled all general posts and Expeditions entries, and another that pulled only photos, simply by looking for MDX or Markdown files.

Pulling in External Content

If you look at the home page of this site, you’ll notice sections devoted to Instagram⁴ and Last.FM. Those sections pull in my latest activity from those sites. I was excited to learn that Gatsby offers plugins for both Instagram and Last.FM. These plugins handle the API interfaces with those websites for you (when using WordPress, I had to build out the API connectors myself) and pull the data into your Gatsby site’s GraphQL data. This has a couple of big advantages: since the information is pulled into your website at compile time, you don’t have to make requests each time a visitor loads the page, and the page is rendered much faster. Additionally, since the information is stored locally, images can take advantage of Gatsby’s built-in optimization. And it’s more relaible, because the data is copied to your website and you don’t have to load external resources, so no 404 errors or connection timeouts.

The main disadvantage is that you have to recompile the entire Gatsby site every time you want to update data from Instagram or Last.FM. I wasn’t planning on recompiling the site every time I posted a photo or listened to a song. The solution I came up with was to use window.fetch through React’s AJAX integration to pull more recent data from Instagram and Last.FM. Once the AJAX call goes through, the newer data replaces the version stored in my site’s GraphQL data.

Putting it All Together

I added some CSS and structure, largely borrowed from the WordPress version of my website. I also had to convert most of my posts to MDX or Markdown. I was also able to spend a lot of time focusing on styles, in part due to Gatsby’s modular approach. As a result, I fixed and fine-tuned a lot of minor style elements that I’d overlooked in the massive stylesheet I used with WordPress. But once I did all that, the result was a great-looking and incredibly fast version of my website.

Next, I had to figure out to host my website. A lot of shared web hosts will let you upload PHP files and create MySQL databases. But they often don’t give you the sort of command-line access and ability to install your own server-side framework you need to run Node, React, and Gatsby. Fortunately, there are a number of hosts that offer easy ways to deploy your Gatsby website. I picked Netlify for its reputation for performance and ease-of-use, and because its free tier seemed to fit my needs pretty well. Additionally, I can deploy to Netlify directly from Github. Something about writing a post in Markdown and then publishing it pushing to Github makes me feel like a ninja.