Haraka SMTP Incoming to Filesystem

I was doing some experimentation with the SMTP protocol using Node.js. I saw a while back the project called Haraka which takes a plugin-based approach and has an interesting list of known users.

For this experiment all I wanted was for the email to come in and get saved to the disk. Luckily Haraka has a bunch of built-in and configurable plugins for checking hosts and other security. Since I’m just taking emails in I don’t have to worry about the more robust security required for outgoing mail. It’s also likely that there is some plugin already made for storing emails directly to disk but I made this other one for my own knowledge purposes.

I started by installing the Haraka npm module globally.

npm install Haraka -g

Next I created a Haraka config project quickly by using the install command.

haraka -i haraka-config-inbound
cd haraka-config-inbound

Haraka also has a plugin generator that I used as well.

haraka -c . -p queue_store

These two files are created from that:

  • /plugins/queue_store.js
  • /plugins/queue_store.md

All I wanted was for emails to be stored by domain. Instead of hard coding the path where they get stored I put it into an ini file. The plugin JS will read the INI file later.

touch config/queue_store.ini
mkdir domains

Here is the content of the INI file ./config/queue_store.ini

[paths]
domains=/Users/tcrowe/haraka-config-incoming/domains

Next I wrote the actual plugin code in JavaScript for node.js. It works by loading your plugin and looking for different members on exports. This plugin is using the hook_queue but that was not technically an event documented on their plugins page.

var path = require('path'),
    fs = require('fs'),
    DSN = require('./dsn'),
    config; // will be set inside the register event

// the register event will grab the config object from the ini file
exports.register = function() {
    var plugin = this;
    config = plugin.config.get('queue_store.ini');
};

// when a message is ready to get queued our event will get called
exports.hook_queue = function(next, connection) {
    var plugin = this,
        trans = connection.transaction,
        message_stream = trans.message_stream,
        domain,
        domainPath,
        fileName,
        filePath,
        storeStream;

    // grab the domain
    domain = trans.rcpt_to[0].host;
    domain = domain.toLowerCase().trim();

    // build a path for that domain
    domainPath = path.join(config.paths.domains, domain);

    // if the directory doesn't exist for this domain yet go create it
    if (!fs.existsSync(domainPath)) {
        fs.mkdirSync(domainPath);
    }

    // build the path for the email file
    fileName = trans.uuid + '.eml';
    fileName = fileName.toLowerCase().trim();
    filePath = path.join(domainPath, fileName);

    // start configuring the streams
    storeStream = fs.createWriteStream(filePath);

    function storeStream_error(err) {
        plugin.logdebug('storeStream error', connection);
        plugin.logdebug(JSON.stringify(err), connection);
        next(DENY, DSN.net_system_congested());
    }

    function storeStream_end() {
        plugin.logdebug('storeStream end', connection);
        next(OK);
    }

    function storeStream_close() {
        plugin.logdebug('storeStream close', connection);
        next(OK);
    }

    function message_stream_error(err) {
        plugin.logdebug('message_stream error');
        plugin.logdebug(JSON.stringify(err), connection);
        next(DENY, DSN.net_system_congested());
    }

    function message_stream_end() {
        plugin.logdebug('message_stream end', connection);
    }

    function message_stream_close() {
        plugin.logdebug('message_stream close', connection);
    }

    // connect the events for our output stream
    storeStream.on('error', storeStream_error);
    storeStream.on('end', storeStream_end);
    storeStream.on('close', storeStream_close);

    // connect the events for the message input stream
    message_stream.on('error', message_stream_error);
    message_stream.on('end', message_stream_end);
    message_stream.on('close', message_stream_close);

    // store the message
    message_stream.pipe(storeStream);
};

The approach I took includes piping the message stream into our output stream. Each file that gets to this step will get saved into a file based on the transaction ID which is a UUID. Headers, attachments, and body text are all lumped into one file.

I think the message is still getting parsed by Haraka but there’s probably some way to disable that. No big deal!

I also made a plugin save the message into CouchDB but it was too time consuming for me to work with the message parts especially when multiple attachments were tested. For the time being I am going to have emails come in and then I can convert them into other formats.

Author Tony Crowe, Salt Lake City, UT
top
home
Attribution
Lato font by Łukasz Dziedzicwww.latofonts.com/team
Roboto font by Christian Robertson, Google Incchristianrobertson.comwww.google.com/fonts
Inconsolata font by Raph Levien, Google Incwww.levien.comwww.google.com/fonts
hexo: fast, simple & powerful blog frameworkhexo.io
ace editorace.c9.io
virtual-domwww.npmjs.com/package/virtual-dom
Black Granite Water Droplets, William Warbyhttps://www.flickr.com/photos/wwarby