Tuesday, 25 August 2009

Persistence on HornetQ

Persistence on HornetQ is really fast. It is so fast that you would think you are sending non-persistent messages by accident.

Instead of using heavy weighted databases that would provide a bunch of stuff we don't need on HornetQ, we used something faster and still as reliable.

We have written our own circular file Journal that uses either Linux libaio or Java NIO.

Linux libaio is a library that works at the kernel level. We submit writes by sending a DMA Buffer (Direct Memory Access) and a callback interface. The kernel will deal directly with the buffer saving copy time between Java and the disk controller. When the disk is done with the write the callback is returned, and we are sure the data is persisted on the disk.

You may ask, what? At the kernel level?! At the controller level?

Yeah.. that' s the beauty of libaio. It provides system calls to the kernel.

BTW: If you are not a geek who loves programming like I do, you could stop reading this post now :-) Since I will dig a little bit on how it works:

The Journal:

The Journal has a set of pre-allocated files. We keep each file size as close as possible to what would fit on a disk cylinder. We have found a value of 10MiB but that could be different in other systems.

The journal is an append only journal. We always append to the current used file of the pre-allocated set. This way we avoid mechanical movements getting most of the performance possible out of the disk controller.

Deletes are taken as appended records. We add a delete record to the bottom of the file.

We also have a reference counting of records, so when the original file is totally clean (all the records deleted), that file is ready for reuse.

And the journal is transactional also. We have a very nice transactional control, where a commit record is only taken into consideration *if* the entire transaction is on the disk. That gives us ACID control.

SequentialFile interface:

We abstract the disk access through that interface. There are two implementations NIO and AIO. You can select what implementation you want through our configuration. (see User's Manual)

This is a very fast approach already. We work at file level, avoiding disk movements. If you don't have Linux or libaio installed in your system, we default to this 100% Java implementation.

AIO (linux libaio):

We have written a small JNI layer that "talks" to libaio on Linux. The basic write method in java, has this signature:

write(int position, int size, ByteBuffer directBuffer, AIOCallback callback). (More detatils on the javadoc)

The buffer here is sent directly to a libaio method called aio_write. (look at aio_write man page).

Another thread will be polling events out of libaio. As soon as the data is on the disk the JNI layer will execute the callback method.

Instead of performing syncs on the disk (which is a slow operation), we use a concurrent latch. You could have many more transactions being executed in parallel. Instead of blocking the whole system while one sync is being performed, we just write as usual and wait for the callback. Each thread will use the most of the performance available at the disk controller. Instead of waiting an expensive sync operation, each thread will be waiting the callback when the data is safely stored.


Persistence on HornetQ is not only fast but it also scales up when several threads are performing transactions.

This is just one of many of other innovations from HornetQ. We are working hard to make a great software. Feel free to contact us on IRC or our user's forum. We would love to get your feedback.

Monday, 24 August 2009

The Hornet Hatches!

This is an exciting day for messaging at JBoss and Red Hat.

After months of preparation, the middleware messaging team are excited to announce the birth of a new project "HornetQ".

What is HornetQ?

HornetQ is an open source project to build a multi-protocol, embeddable, high performance, clustered, asynchronous messaging system. HornetQ is an example of Message Oriented Middleware (MoM)

HornetQ is designed with usability in mind: We've provided an extensive, easy-to-understand user-manual and quick-start guide and we ship with over 65 ready-to-run examples out of the box, demonstrating everything from simple JMS usage to complex clusters of servers and more exotic functionality.

HornetQ is designed with flexibility in mind: It's elegant POJO based design has minimal third party dependencies: Run HornetQ as a stand-alone messaging broker, run it in integrated in your favourite JEE application server, or run it embedded inside your own application. It's up to you.

HornetQ is designed with performance in mind: Our unique ultra-high performance journal provides never seen before persistent messaging performance. Automatically switching into native mode when running on Linux, it uses asynchronous IO to provide persistent messaging rates that can saturate the write throughput of a disk. Our pluggable transport system uses JBoss Netty out of the box to provide superb performance and scalability on the wire.

HornetQ is licensed using the Apache Software License V 2.0. The ASL 2.0 has fewer restrictions on use than the LGPL, thus providing fewer barriers to adoption. We want HornetQ to be used as widely as possible.

HornetQ has a great feature set that you'd expect of any serious messaging broker.

But... What about JBoss Messaging?

During its development over the last couple of years the HornetQ code-base was worked on under the name JBoss Messaging 2.0

We decided to rename it and separate it as an independent project since it differs in a many ways from JBoss Messaging 1.x and we did not want to confuse the two, quite different, systems. The vast majority of the code base of HornetQ is different to the code base of JBoss Messaging 1.x

So, what happens with JBoss Messaging now? JBoss Messaging 1.x continues to be known under the name of JBoss Messaging and the project is now in maintenance mode only, with all new messaging development happening on the HornetQ project.

Let's go!... to the future

The future has a lot to hold.

What about cloud computing?

Messaging is going to be a key service in the cloud, and our goal is for HornetQ to be the messaging provider of choice in the cloud. It's our view that RESTful APIs will eventually be the preferred API style in clouds. With that in mind we'll be working on implementing a RESTful style API for interoperable messaging.

Since interoperability is high on our list we won't just stop with REST. HornetQ will also be implementing AMQP and native STOMP support to make it a truly multi-lingual messaging system.

Get involved. HornetQ needs you!

The future certainly has lots in store, and there is plenty for us to do. So why not get involved?

HornetQ is a community, open source project and we'd love to hear from you if you'd like to get involved in development, documentation or help in some other way. We have a small team so any help would be fantastic. Join us!

Come, hear the scoop!

To get all the details on this exciting news and more and to see how it fits in the with JBoss' and Red Hat's overall middleware strategy, come and see me and others speak at the Red Hat summit / JBoss World 2009 in Chicago on September 1-4.

See you there!

Any questions?

I've put together a FAQ that should answer most of the common questions about HornetQ.

Here are some more links:

Project web site Project wiki Download User manual Quick start guide Follow us on twitter