srslog: Sending syslog Messages from Go

I recently created a new open source project called srslog*. It implements the syslog protocol and allows you to send syslog messages from a Go program to a local or remote syslog daemon.

* Why “srslog”, you ask? Well, because it’s my initials, and more importantly, because I’m serious. Pronounced “serious log”. srsly.

Why Fork?

This capability has been available as part of Go’s standard library since its inception and this project is a fork of that package.

I created the fork because, for one thing, a package like this doesn’t need to be part of a language’s standard library. Go is perfectly usable as a language without it — and it would certainly be annoying if you had to upgrade to an entirely new version of the language in order to get new syslog-related features; a perspective Google appears to share:

We have no plans to further develop the log/syslog package in the standard library. We welcome the community to fork it and give it features elsewhere. It was probably a mistake putting it in the standard library to begin with.

Google posted that announcement right around the time that I needed to send TLS-encrypted events to a remote syslog endpoint. This is a supported feature of modern syslog daemons, but was not supported by Go’s syslog package.

As it turns out, the syslog format in Go’s original syslog package was also not compliant with the actual syslog spec. It focused on “compatibility” with existing software over compliance, which resulted in an amalgamation of RFC 3164 and RFC 5424 (that gets rejected by strict daemons).

I created the fork to add features (TLS and RFC compliance) and to separate syslog features from the language itself. Ideally, it would be a drop-in replacement for the original syslog package for anyone who wants to upgrade:

import(
    //"log/syslog"
    syslog "github.com/RackSec/srslog"
)

Dangers of Changing a Public API

As I went through the code, it seemed like some internals were leaking out. Looking at example usage, it all looked like this:

w, err := syslog.Dial("udp", "localhost:514", syslog.LOG_ERR, "testtag")
if err != nil {
    log.Fatal("failed to dial syslog")
}
w.Alert("something is wrong")

The w variable is an instance of syslog.Writer, and LOG_ERR is a syslog.Priority. Based on this code, I assumed that you wouldn’t actually need those to be public. In the interest of having a public API that was as small as possible, I hid the Writer and Priority as internal details.

But, of course, if something is already part of the public API, then someone is using it.

var w *syslog.Writer
var p syslog.Priority
if highPriority {
    p = syslog.LOG_ERR
} else {
    p = syslog.LOG_DEBUG
}
if secureConnect {
    w, _ = syslog.DialWithTLSCert("", "", p, "tag", cert)
} else {
    w, _ = syslog.Dial("", "", p, "tag")
}

Turns out that sort of usage was common, and clearly shows that my attempt to shrink the public API was a mistake.

Even though the code I’d seen hadn’t used these parts of the API, once it’s public then real code will use it. That’s an important thing to remember!

Designing for Expandability

When I added the ability to dial via TLS, one thing I really wanted was to make sure it would be easy to add another dialer in the future.

Originally it could dial either locally, or over the network, by using a conditional:

if network == "" {
    // dial locally
} else {
    // dial remotely
    c, _ = net.Dial(//...
}

I could have added an else/if for TLS support, but adding another dialing method in the future would require a series of conditionals, which would result in code that’s clumsy and difficult to read.

Instead, I created dialer functions:

func (w *Writer) basicDialer() (serverConn, string, error) {}
func (w *Writer) tlsDialer() (serverConn, string, error) {}

and a way to get the one you want based on the network:

type dialerFunctionWrapper struct {
    Name   string
    Dialer func() (serverConn, string, error)
}
func (df dialerFunctionWrapper) Call() (serverConn, string, error) {
    return df.Dialer()
}
func (w *Writer) getDialer() dialerFunctionWrapper {
    dialers := map[string]dialerFunctionWrapper{
        "":        dialerFunctionWrapper{"unixDialer", w.unixDialer},
        "tcp+tls": dialerFunctionWrapper{"tlsDialer", w.tlsDialer},
    }
    dialer, ok := dialers[w.network]
    if !ok {
        dialer = dialerFunctionWrapper{"basicDialer", w.basicDialer}
    }
    return dialer
}

This approach, regardless of your network type, allows you to grab a dialer and call it with the expectation that it’ll behave the way it’s supposed to.

(Note: that dialerFunctionWrapper type is there to allow for testing, it’s not strictly necessary but Go doesn’t have the ability to compare functions for equality.)

But the big benefit of this code is that to add a new dialer, you simply write a new dialer function and add it to the map in getDialer. We’ve avoided an ever-growing chain of conditionals.

Backwards Compatible Compliance

As mentioned above, the original package produced non-compliant syslog messages in the name of compatibility.

There are two potential syslog formats: RFC 3164 and RFC 5424.

RFC 3164 is older, and one major limitation of it is that the timestamp format doesn’t include a year or a timezone.

RFC 5424 rectifies the timestamp problem and adds space for a bit more data in each message, but the data is formatted differently.

Go’s syslog formatted its messages as 3164 but with 5424-style timestamps.

I wanted to add support for compliance with either specification, but without breaking any code that might already be using log/syslog.

type Formatter func(p Priority, hostname, tag, content string) string
func DefaultFormatter(p Priority, hostname, tag, content string) string {
    timestamp := time.Now().Format(time.RFC3339)
    msg := fmt.Sprintf("<%d> %s %s %s[%d]: %s",
        p, timestamp, hostname, tag, os.Getpid(), content)
    return msg
}
func RFC3164Formatter(p Priority, hostname, tag, content string) string {
    timestamp := time.Now().Format(time.Stamp)
    msg := fmt.Sprintf("<%d> %s %s %s[%d]: %s",
        p, timestamp, hostname, tag, os.Getpid(), content)
    return msg
}

I took the code that formatted messages and put it into DefaultFormatter. The new RFC3164Formatter and RFC5424Formatter functions implement those protocols, respectively.

If you do nothing to your code, it will default to using DefaultFormatter — backwards compatibility is maintained.

But if you call w.SetFormatter, you can choose which format you want to produce — compliance is achieved.

End Transmission

Even though I now have a new library to maintain, I think srslog addresses issues missed in the original log/syslog package and, if I may be so bold, is strictly better than the original. If you need to send syslog messages from Go, there’s no reason not to upgrade.

Thanks for reading!