Log management

What's going on under the hood

Pino

EmailEngine logs all it's messages to standard output in Pino format. Pino uses JSON structures with some predefined keys so it is kind of human readable but not really. Luckily there are a bunch of tools that can be used to transform EmailEngine logs into more suitable format. The working principle for all these log renderers is the same – you should pipe the standard output from EmailEngine to the log rendering process.

jq

jq is a well known command for processing JSON structures in the command line. It is not EmailEngine specific at all but it can be easily used to make EmailEngine logs slightly more readable.

$ emailengine | jq

pino-pretty

pino-pretty is a command line utility that makes Pino formatted logs more readable. You can install it from the npm registry as a global command.

$ npm install -g pino-pretty
$ emailengine | pino-pretty

eerawlog

eerawlog is a EmailEngine specific log renderer. It is meant to debug IMAP and API call transactions. This is the only thing it's good for so it's not a general log renderer – it tends to throw away everything that is not IMAP specific. Easier to understand how EmailEngine communicates with IMAP servers, hard to see how the application is doing in general.

Additionally eerawlog requires that raw logs are enabled for EmailEngine. By default these are disabled, mostly for two reasons:

  1. raw logs can be quite large as it includes all bytes sent and received, that includes full email message sources
  2. raw logs do not modify secrets. Regular logs are always processed in a way where account credentials and other secrets are filtered out. Raw logs include all bytes transferred between EmailEngine and the server
$ npm install -g eerawlog
$ emailengine --log.raw=true | eerawlog

If you do not specify the --log.raw=true command line argument or do not set EENGINE_LOG_RAW=true environment variable as an alternative then the logs will look completely empty.

pino-gelf

pino-gelf is another useful utility that sends all log messages from EmailEngine to a Graylog server. A neat feature is that pino-gelf can also work in a throughput mode, so that it both sends the logs to Graylog and logs messages to console so we could mix it up with other tools like pino-pretty

$ npm install -g pino-gelf
$ emailengine | pino-gelf log -h graylog.server.com -t | pino-pretty

Something to keep in mind with pino-gelf is that it creates separate log keys only for root level log entry keys. Object values are serialized into JSON strings:

{  
  "version":"1.1",
  "host":"emailengine",
  "short_message":"message entry",
  "full_message":"message entry",
  "timestamp":1481840140.708,
  "level":6,
  "_name":"emailengine",
  "_object_value":"{\"key\":\"value\"}",
}

In this case you can't search for object_value.key field in Graylog. Instead there will be a text field object_value with {"key":"value"} as it's value.