October 3rd, 2008


Query Logging on Drizzle

A project I have been working on for a while just hit a significant milestone.

Pluggable Query Logging has just been merged into the main Drizzle development.

The plan is that quickly this will replace the existing query log and slow query log that Drizzle inherited from MySQL.

The legacy logging system is more than a bit byzantine, is full of a lot of complexity to try to get incremental performance with complicated buffering, and yet has several completely unnecessary locks.

In the new plugin logging system, there is a callback right after a query is parsed but before it is executed, and another one right after it is done executing. A plugin registers itself, and will be called at those two points, and is passed the THD, from which it can learn everything about the query that will be / has been executed, and then do whatever it wants with that information.

Of course, since everything is still under active development, details of that interface and API can change at any time.

In addition to the interface, I've written an implementation. It is currently very rough and crude: logging far too much, in an awkward log format, and to just a single predefined file. But I plan on improving it a great deal, until it equals and then surpasses the existing legacy system, which will then be torn out (shrinking the the Drizzle core even more, and adding yet more multiprocessor performance).

Also planned: logging to syslog, and logging to table.