Coding

How to Build an Image Processing Service with Express.js and Sharp

Kyle Bedell/May 30, 2018
If you are building an application backend, chances are you need to handle thumbnail image generation as part of the feature set. You have several options. Should I generate the thumbnails on image upload? Should you use a turn key Backend-as-a-Service (BaaS) like imgix.io? Which library should I use? If you're the adventurous type, you may want to roll-your-own. Well, you're in luck. In this tutorial, I am going to teach you how to build your very own image resizing service using Express.js and Sharp, the lightning fast image manipulation library for Node.js. It's actually easier than you might think.
Image scaling services work by proxying a request through a catch-all route to the original resource (also known as a master image). If the image is found, the server will apply a set of image manipulations based on your query parameters. Finally, when processing has finished, the server responds with the manipulated image.
Sound like fun? Great. Let's dive in!
Note: If you get lost or just want to run the final product, I've posted a link to the full source at the end of this tutorial.

Getting Started

You’ll need to have Node.js installed on your system. We're going to use Babel to use all features of ES6.
Create a new directory somewhere on your machine and cd into it.
mkdir imageresizer
cd imageresizer
Use npm init to crate a package.json.
npm init
Save production dependencies with npm i -P.
npm i -P express sharp morgan
...and some dev dependencies with npm i -D.
npm i -D babel-cli babel-core babel-preset-es2015 babel-preset-stage-0 nodemon
Add some simple startup scripts to your package.json.
"scripts": {
  "dev": "nodemon -w src --exec \"babel-node src --presets es2015,stage-0\"",
  "build": "babel src -s -D -d dist --presets es2015,stage-0",
  "start": "node dist",
  "prestart": "npm run -s build",
},
Next create an index.js file, this will be your server entry point.
We need to import a few things.
import axios from 'axios';
import express from 'express';
import http from 'http';
import morgan from 'morgan';
import sharp from 'sharp';
Now we need to setup the server and register a request logging middleware.
let app = express();
app.server = http.createServer(app);

app.use(morgan('dev'));
Before we create the endpoint, we need some helper functions to handle some heavy lifting.

Getting the Storage Location

We’re going to design our service so that we can have various storage locations or customers. You can use subdomains or subdirectories. I want this solution to work on cloud hosting solutions like Google Cloud Functions so I'll use subdirectories.
How does it know where to find customer image assets? That sort of thing should be mapped from an admin panel and stored in a database. Building an admin interface is way beyond the scope of this tutorial so we will just use an object in memory for now. I plan to revisit adding an admin interface in another tutorial. For now, create a config file and name it config.js.
export default {
  port: 8080,
  customers: {
    unsplash: {
      baseUrl: 'https://images.unsplash.com'
    }
  }
}
Back in index.js, import the new module.
import config from './config';
Create a method that gets looks up the account in our config object.
const getCustomerById = (id, db) => db[id] ? db[id] : 'https://localhost';

Fetching the Image

In order to resize an image, we need to fetch it first. I like to use axios for this task, but you can use just about any other streamable HTTP module including request.
Instead of creating a new temporary file on the server everytime we request an image, we can use Node.js streams which cuts down on memory comsumption and file storage tremendously.
Streams are really powerful and efficient for working with large amounts of data. They are worth learning more about. You don't need to know a lot about them at this time, just follow along and you'll be fine.
const download = url => axios({
  method: 'get',
  url,
  responseType: 'stream',
}).then(response => response.data);

Is My Variable a Number?

Checks whether a variable is a number. Speaks for itself.
const isNumeric = n => !isNaN(parseFloat(n)) && isFinite(n);
Moving on...

Image Manipulation

Okay, we now need to create an image manipulation object. Let's write a method called transform() that will accept some parameters and return a manipulated image stream. In other words, we need to create a sharp() object, call a bunch of methods to set a bunch of props and return the sharp object to be consumed by a res stream.
const transform = ({
  blur,
  cropMode,
  height,
  width,
  quality,
}) => {
  const sharpObj = sharp();
  // Width and height set...
  if (Number.isInteger(height) && Number.isInteger(width)) {
    sharpObj.resize(width, height);

  // Only width set...
  } else if (isNumeric(width)) {
    sharpObj.resize(width);

  // Only height set...
  } else if (isNumeric(height)){
    sharpObj.resize(null, height);
  }

  // Blur
  if (isNumeric(blur)) {
    // Clamp between 0.3 and 1000
    sharpObj.blur(Math.min(1000, Math.max(blur, 0.3)));
  }

  // Crop mode
  if (sharp.gravity[cropMode]) {
    sharpObj.crop(sharp.gravity[cropMode]);
  } else if (sharp.strategy[cropMode]) {
    sharpObj.crop(sharp.strategy[cropMode]);
  }

  // JPEG quality
  sharpObj.jpeg({
    quality: isNumeric(quality) ? Math.max(1, Math.min(100, quality)) : 80,
  });
  return sharpObj;
}
Largely the transform() function is just a conditional configuration. isNumeric finally comes into play to perform some simple number validation. Don't forget to return our sharpObj variable at the end, it is going to get piped.

Wire It Up

At last, we have all the supporting functions we need to process an image, we can now create our route. What we are after is a "catch-all" route to allow any number of sub-directories in the image resource path. To do that, we'll setup a wild-card route with the string /:customerId/* That route also provides access to the customer ID via req.params.customerId.
app.get('/:customerId/*', (req, res) => {
  const customer = getCustomerById(req.params.customerId, config.customers);
  const {
    blur,
    crop,
    h,
    w,
    q,
  } = req.query;
  ...
});
We can extract the full imagePath from the req.url with a bit of string manipulation.
const imagePath = req.url.split('/').slice(2).join('/');
Let's set the response mime type.
res.type('jpg');
Next build the resource path and wire up the axios, sharp and res pipes.
const url = `${customer.baseUrl}/${imagePath}`;

// Download the image from the user's master image source.
download(url)
  .then(response => response.pipe(transform({
    blur: parseInt(blur, 10),
    cropMode: crop,
    height: parseInt(h, 10),
    quality: q && parseInt(q, 10),
    width: parseInt(w, 10),
  })).pipe(res))
  .catch(err => {
    res.status(404).send();
  });
In it's entirety, the route should be the following:
app.get('/:customerId/*', (req, res) => {
  const customer = getCustomerById(req.params.customerId, config.customers);
  const {
    blur,
    crop,
    h,
    w,
    q,
  } = req.query;

  // Extract image path from URL
  const imagePath = req.url.split('/').slice(2).join('/');

  // Ignore requests for favicons
  if (imagePath === '/favicon.ico') { return res.status(404); }

  // Output format
  res.type('jpg');

  // Resource path
  const url = `${customer.baseUrl}/${imagePath}`;

  // Download the image from the user's master image source.
  download(url)
    .then(response => response.pipe(transform({
      blur: parseInt(blur, 10),
      cropMode: crop,
      height: parseInt(h, 10),
      quality: q && parseInt(q, 10),
      width: parseInt(w, 10),
    })).pipe(res))
    .catch(err => {
      res.status(404).send();
    });
});

Start your engines...

Log out a helpful message to the console and start the server.
app.server.listen(process.env.PORT || config.port, () => {
  console.log(`Started on port ${app.server.address().port}`);
});
Ah, finally!
Start the server with npm run dev or npm run start.
You can now request resources from Unsplash if you know the original resource URL.
To test it out, make a request to http://localhost:8080/unsplash/photo-1506153456649-ed4ed08d1e0c?w=400&h=400&q=80.
You should see two race cars. Vroom, vroom!

Conclusion

There are always room for improvement. Really, this is just a simple example to get your feet wet and hopefully point you in the right direction. What we haven't covered in this tutorial is security, scaling up, CDN caching, account administration and customer billing. These are topics for another day.
That's it! Don't forget to subscribe to the newsletter to receive JavaScript awesomeness right in your inbox. No spam.
You can find the full source for this tutorial on Github.

Get the Newsletter

Stay up on JavaScript.