Nuclear Rooster

Graylog2 is the Bee’s Knees

Graylog2 is a log management solution handling log collection, analytics, alerting and more. Graylog2 embodies the DevOps mentality: it combines the best parts of tried-and-true syslog with rich features for application logging, and does it all under the watchful eye of a yeti wearing a birthday hat.

Graylog2 consists of a tastefully designed Rails web-interface and a Java-based TCP/UDP log collector, using MongoDB to store the log messages and application data. Rails is a clear choice for a convenient, easy-to-extend web interface, and Java is efficient for high-traffic message collection and forwarding. The real magic (aside from the core team) is MongoDB, which elegantly ties everything together .

The polyglot approach, another DevOps tactic, utilizes the best tool for the job-at-hand. Graylog2 chooses Ruby and Rails for rapid web UI development, and Java where speed and memory profile are important. With that flexibility in mind, Graylog2 accepts messages over TCP or UDP, whichever protocol suits your needs. MongoDB makes ployglot easy, with a collection of a dozen supported drivers for languages from C to Scala. For Graylog2, this mean Rails and Java can talk to the same data store using updated, documented libraries. Good clean drivers help downright speedy development.

As of version 0.9.5 Graylog2 has ditched MySQL entirely in favor MongoDB, streamlining the installation process and slimming the code base. Reducing the friction during installation is key to driving adoption for any product, and Graylog2's got a few good options for a speedy install. MongoDB is a snap to install, too, and has a user-friendly shell, which makes it familiar for users coming from a SQL background (no required map-reduce or erlang stacktraces, Bob). If you really don't want to install MongoDB, you don't even have to thanks to multiple hosting providers.

The Graylog Extended Log Format (GELF) improves on the standard syslog entry (a 1024 byte string) by adding the ability to specify arbitrary metadata with the log message. This data could be the application environment, server details, debugging data, or anything else. Mongo's document storage handles storing and indexing this data without any fuss (or awkward code forcing a schema upon arbitrary data). No rigid schema means fewer development roadblocks and simpler upgrades, while structured data allows for advanced querying (and indexing). It's the best of both worlds. Check out how Graylog2's 'Quick Filter' exposes these queries:

The collection keeps a fixed number of log entries around, a text-book use-case of MongoDB's capped collection. This means you don't have to worry about cleaning up old entries or overflowing disks. Sure, it isn't that hard write a script to cleanup old entries, but the fewer moving parts, the fewer things to remember, the better (not to mention that capped-collection writes are blazing fast). Another DevOpsy tenant: "Don't use the braintrust". That is, if you are relying on individuals remembering issues and edge-cases, you're headed for trouble. With Graylog2, I have to remember a whole lot less (ssh passwords, log file location, log rotation) to know the state of my apps and servers. Graylog2's use of capped-collections gives me even less to remember (soon, there might even be room left for remembering my childhood).

A true log-management solution, Graylog2 organizes messages into user-defined 'streams', effectively blacklists noisy messages, forwards messages to external resources, and generally makes your logs actionable. Don't send your logs to files so they can die, send them to Graylog2 so they can come to life.

Check out the Graylog2 website or dig into the code. Hope on IRC (#graylog2 on freenode) or the mailing list (http://groups.google.com/group/graylog2?hl=en) if you need a hand getting Graylog2 configured.

YETI PARTY!