Relatively Painless Autoloading in Node.js

It seems a fairly common pattern I encounter in Node-based codebases is an index.js file that simply imports a bunch of other files, and then exports them all in an object to be consumed elsewhere by simply requiring the directory that index file lives in.

Something like this, for example:

import foo from 'foo'
import bar from 'bar'
import baz from 'baz'

export default {
  foo,
  bar,
  baz,
}

Now there's nothing wrong with this, of course.  It just strikes me as a bit of unnecessary tedium that I have to remember to go add my new thing to the import and exported object if I want it to work the way all those other things do.. you know so that I can do stuff like this:

import { foo, bar } from '../../foobarbaz'

Instead, sometimes I like to write relatively painless autoloaders to accomplish the same thing for all files in a directory.

Given the following directory structure:

middleware
  \ index.js
    foo.js
    bar.js
    baz.js

I can use this to accomplish the same as the manual specification of each import/export thing:

import fs from 'fs';

const middlewares = {};

fs.readdirSync(__dirname).forEach(function(file) {
  // Ignore the index file, and anything that is not a .js file
  if (file === 'index.js' || file.substr(file.lastIndexOf('.') + 1) !== 'js') {
    return;
  }

  // Grab all but the file extension for our middleware name
  const middleware = file.substr(0, file.indexOf('.'));
  
  // require it in place in the object we're going to export
  middlewares[`${middleware}`] = require('./' + middleware).default
});

export default middlewares;

This way even if I add a whole bunch of other files to that directory, each will become available, by its name, simply by requiring/importing the base middleware directory.

It OBVIOUSLY doesn't make sense if you just have one or two files to include, and aren't likely to add a lot more, and while this example is a bit trite, it can be greatly expanded from here.

For example, in my current project I'm building up a JSON-RPC api. To keep things neat, I wanted to do some namespacing so that, for example math functions live under a subdirectory math and additionally said namespaces carry over to the actual rpc method names; math.add for example.

As such, I have a methods directory, full of subdirectories for each method namespace I'm going to use (we'll stick with math.add for now), inside that methods directory is an index file to autoload all the other methods, inside each of those subdirectories are files for the individual methods, no other index files.

I'm using tedeh's jayson package... so wrapping each imported thing with jayson.Method()

My index file looks like this:

import fs from 'fs';
import jayson from 'jayson';

const methods = {};

fs.readdirSync(__dirname).forEach(function(file) {
  if (file === 'index.js' || file.substr(file.lastIndexOf('.') + 1) === 'map') {
    return;
  }

  const group = file;
  fs.readdirSync(__dirname + '/' + group).forEach(function(filename) {
    if (filename === 'index.js' || filename.substr(filename.lastIndexOf('.') + 1) !== 'js') {
      return;
    }
    const method = filename.substr(0, filename.indexOf('.'));
    
    methods[`${group}.${method}`] = jayson.Method(require('./' + group + '/' + method).default)
  });
});

export default methods

Basically the same as the simpler example above, here though we instead find directory names and set those as the 'group' and then inside those directories find the individual method files, and append that to the group name, resulting in namespaced methods like math.add as I mentioned before.

In the math directory, I simply have a file called add.js

// This is a simple example implementation to use as a guide
// until actual methods are implemented.
// Export's an object. handler function is _required_
// collect defaults to true and is not necessary
// params should be used to specify default values -- or omitted entirely.
export default {
  handler: function ({ auth, data }, callback) {
    console.log('Hi ', auth.name)
    callback(null, data[0] + data[1])
  },
  collect: true,
  params: {}
}

Works great for my use, and especially as I'll be adding a lot of rpc methods, I didn't want to have to screw around with manually import/requiring them and then adding them all to a big object to export.

I'm sure I could clean it up some, or use glob, for example, but this works, and has never given me any problems, and it's almost identical to the original code I wrote to do this same sort of thing well over 5 years ago.