The HornetQ Team Blog

Stomp 1.1 Support in HornetQ

2011-10-18T06:09:00.000-07:00

HornetQ now supports the Stomp 1.1 protocol and will be available in the next release, for now though it is part of trunk in the SVN repository.

Stomp 1.1 specification is an update to Stomp 1.0 and is backward compatible. New features include protocol negotiation, heartbeat, the NACK frame and virtual hosting.

An example 'stomp 1.1' is also available in the SVN repository which demonstrates a stomp client that uses one of the new features 'protocol negotiation', lets take a look at the example now.

1. Configuring a HornetQ Stomp Server

First of all we need to configure the server to allow stomp connections by adding a stomp acceptor, this is the same as stomp 1.0.

<acceptor name="stomp-acceptor">
<factory-class>org.hornetq.core.remoting.impl.netty.NettyAcceptorFactory</factory-class>   
<param key="protocol" value="stomp" />
<param key="port" value="61613" />
</acceptor>

2. Connection Negotiation

Once the HornetQ Stomp server is up and running, it is up to the Stomp client to decide which Stomp specification to use to communicate with HornetQ. This is done by connection negotiation. The Stomp1.1 example shows how to do this

// Step 1. Create a TCP socket to connect to the Stomp port
Socket socket = new Socket("localhost", 61613);

// Step 2. Send a CONNECT frame to connect to the server
String connectFrame = "CONNECT\n" +
"accept-version:1.1\n" +
"host:localhost\n" +
"login:guest\n" +
"passcode:guest\n" +
"request-id:1\n" +
"\n" +
END_OF_FRAME;

sendFrame(socket, connectFrame);

In the above code you can see that the client sends a CONNECT stomp frame with the accept-version set to 1.1, this tells the server that the client is using the 1.1 protocol.
Also new to 1.1 is the host header which is used to configure the virtual host to use, although HornetQ supports setting the header it doesn't support virtual hosts. All other headers are standard Stomp 1.0 headers.

If the server accepts the connection request, it will return a 'CONNECTED' stomp with a 'version' header whose value is 1.1. The example prints this to the console:

...

[java] response: CONNECTED

[java] version:1.1

[java] session:1337300467

[java] server:HornetQ/2.2.5 HornetQ Messaging Engine

[java] response-id:1

...

3. Sending and receiving messages.

Once connected, the client can send and receive Stomp messages with the connection. The example illustrates how to send a stomp message:

// Step 3. Send a SEND frame (a Stomp message) to the
// jms.queue.exampleQueue address with a text body
String text = "Hello World from Stomp 1.1 !";
String message = "SEND\n" +
 "destination:jms.queue.exampleQueue\n" +
 "\n" +
 text +
 END_OF_FRAME;
sendFrame(socket, message);

You can see this is very much the same as a Stomp 1.0 client. The message will be sent to the destination 'jms.queue.exampleQueue'. For the mapping of destination headers to HornetQ jms addresses please refer to the HornetQ's User Manual.

Note: Please pay attention to the leading and trailing spaces in the headers. Stomp 1.0 usually trims off the spaces before processing. But in Stomp 1.1, those spaces are preserved. So in Stomp1.1 unnecessary spaces in headers may result in strange errors which is difficult to find.

4. Enabling Heartbeats

One important feature added to Stomp 1.1 is the heartbeats used to monitor the underlying connection. The heart-beating is established at connection time, using a special header 'heart-beat' in CONNECT frame. For example:

CONNECT
accept-version:1.1
host:127.0.0.1
login:guest
passcode:guest
heart-beat:500,1000

Once connected with the above frame, the HornetQ server will make sure that a stomp frame (or a heartbeat byte) is sent to the client every 1 second (1000 milliseconds). Meanwhile the client should send a stomp frame (or a heartbeat byte) for every 500 milliseconds.

HornetQ server will deem a connection to be broken if it hasn't receive a stomp frame from the client via this connection for a time longer than 2 times of the client heart-beat interval (i.e. 2*500). So for the above case, if it hasn't receive any frame within a second, the server will close the connection.

Note: HornetQ specifies a minimum value for both client and server heart-beat intervals. The minimum interval for both client and server heartbeats is 500 milliseconds. That means if a client sends a CONNECT frame with heartbeat values lower than 500, the server will defaults the value to 500 milliseconds regardless the values of the 'heart-beat' header in the frame.

The HornetQ 1.1 implementation also has several performance improvements however we will continue to improve this over the coming months.

For more information read the stomp specification

HornetQ on JBoss AS7

2011-09-01T10:00:00.000-07:00

Now that JBoss AS 7.0.1 has been released which includes messaging and MDB's I thought we would write a quick tutorial on how to get started deploying JMS resources and MDB's.

We have recently blogged about our achievements on SpecJMS and EAP 5.1.2 and of course the version shipped with AS7 has all the same functionality and performance levels that are available in the EAP platform.

This tutorial will demonstrate how HornetQ is configured on AS7, I Will explain the main concepts of how to configure HornetQ server configuration and JMS resources and also provide an example MDB that we can run. So first of all you will need to download AS7 from here.

Make sure you download the 'everything' version as the web profile does not contain messaging or MDB's by default.

In AS7 there's is a single configuration file, either standalone.xml or domain.xml, which is broken into subsystems. These files are pretty much identical although there are differences however this is beyond the scope of this article. For more information on AS7 and its configuration take a look at the AS7 users guide here.

By default the messaging subsystem isn't enabled however a preview configuration is provided that does contain a messaging subsystem. these are standalone-preview.xml and domain-preview.xml, for this tutorial we will use the standalone-preview.xml. To run the preview configuration simply execute the command from the bin directory:

./standalone.sh --server-config=standalone-preview.xml

You should see the HornetQ server started along with some JMS resources, quick wasn't it. Now lets take a closer look at the messaging configuration itself. Each subsystem has its own domain named that is defined by a schema, the schema for the messaging subsystem can be found in docs/schema/jboss-as-messaging_1_0.xsd in the AS7 distribution.

If you search for jboss:domain:messaging in the standalone-preview.xml you will find the HornetQ subsystem configuration.

If you have used HornetQ standalone or in JBoss 6 you will be familiar with some of the configuration. The first part is basically the same as in the hornetq-configuration.xml file. This looks like:

<!-- Default journal file size is 10Mb, reduced here to 100k for faster first boot -->
<journal-file-size>102400</journal-file-size>
<journal-min-files>2</journal-min-files>
<journal-type>NIO</journal-type>
<!-- disable messaging persistence -->
<persistence-enabled>false</persistence-enabled>

<connectors>
<netty-connector name="netty" binding="messaging">
<netty-connector name="netty-throughput" binding="messaging-throughput">
   <param key="batch-delay" value="50">
</netty-connector>
<in-vm-connector name="in-vm" id="0">
</in-vm-connector>

<acceptors>
<netty-acceptor name="netty" binding="messaging">
<netty-acceptor name="netty-throughput" binding="messaging-throughput">
   <param key="batch-delay" value="50">
   <param key="direct-deliver" value="false">
</netty-acceptor>
<in-vm-acceptor name="in-vm" id="0">
</in-vm-acceptor>

<security-settings>
<security-setting match="#">
   <permission type="createNonDurableQueue" roles="guest">
   <permission type="deleteNonDurableQueue" roles="guest">
   <permission type="consume" roles="guest">
   <permission type="send" roles="guest">
</permission>
</permission>

<address-settings>
<!--default for catch all-->
<address-setting match="#">
   <dead-letter-address>jms.queue.DLQ</dead-letter-address>
   <expiry-address>jms.queue.ExpiryQueue</expiry-address>
   <redelivery-delay>0</redelivery-delay>
   <max-size-bytes>10485760</max-size-bytes>
   <message-counter-history-day-limit>10</message-counter-history-day-limit>
   <address-full-policy>BLOCK</address-full-policy>
</address-setting>
</address-settings>

This is the basic server configuration and the configuration of connectors and acceptors. The only difference here to the standalone HornetQ configuration is that the connectors and acceptors use bindings rather than explicitly defining hosts and ports, these can be found in the socket-binding-group part of the configuration.

For more information on configuring the core server please refer to the HornetQ user manual.
The rest of the subsystem configuration is all JMS resources. firstly you will see some JMS connection factories of which there are two types. Firstly basic HornetQ connection factories:

<connection-factory name="RemoteConnectionFactory">
<connectors>
   <connector-ref connector-name="netty"/>
</connectors>
<entries>
   <entry name="RemoteConnectionFactory"/>
</entries>
</connection-factory>

These are basically normal connection factories that would be looked up via any external client and controlled via HornetQ itself. Secondly you will see pooled connection factories, like so:

<pooled-connection-factory name="hornetq-ra">
<transaction mode="xa"/>
<connectors>
   <connector-ref connector-name="in-vm"/>
</connectors>
<entries>
   <entry name="java:/JmsXA"/>
</entries>
</pooled-connection-factory>

These are pooled connection factories and although connect to HornetQ the connections themselves are under the control of the application server. If you have previous experience with older versions of the application server this is the connection factory that would be typically defined in the jms-ds.xml configuration file.

The pooled connection factories also define the incoming connection factory for MDB's, the name of the connection factory refers to the resource adapter name used by the MDB, in previous Jboss application servers this is typically the configuration found in the ra.xml config file that defined the resource adapter.
Lastly you will see some destinations defined like so:

<jms-destinations>
 <jms-queue name="testQueue">
    <entry name="queue/test"/>
 </jms-queue>
 <jms-topic name="testTopic">
    <entry name="topic/test"/>
 </jms-topic>
</jms-destinations>

These are your basic JMS Topics and Queues where entry name is their location in JNDI.
Now lets take a simple MDB example build and deploy it and configure the server for it. A sample MDB and client can be found here and uses Maven to build. Download it and run mvn package to build the application ear file.

The example is a simple request/response pattern so before we deploy the MDB we need to configure 2 queues mdbQueue and mdbReplyQueue like so:

<jms-queue name="mdbQueue">
 <entry name="queue/mdbQueue"/>
</jms-queue>
<jms-queue name="mdbReplyQueue">
  <entry name="queue/mdbReplyQueue"/>
</jms-queue>

now restart (or start) the Application Server and copy the ear file from mdb/mdb-ear/target to the standalone/deployments directory in the AS7 installation. you should now see the mdb deployed. Now we can run the client, simply run the command mvn -Pclient test and the client will send a message and hopefully receive a message in reply.

Congratulations, you have now configured HornetQ and deployed an MDB.

8.2 million messages / second with SpecJMS

2011-07-13T07:35:00.000-07:00

The latest version of HornetQ as part of JBoss EAP 5.1.2 has once again been benchmarked against SPECjms2007 (c).

HornetQ is also included on JBoss AS7, which contains the same improvements we have made to achieve this performance. You will also be able to get the same performance figures with JBoss AS7.

This latest benchmark has outperformed HornetQ's latest publication by a good margin already.

With this latest result HornetQ sustained a load of about 8 million messages per second.

SpecJMS is a peer reviewed benchmark and is the first industry standard and has strict rules as to how each messaging system is configured. This is to make sure that vendors don't cheat when it comes to persistence and transactional requirements etc. SPEC is an independent corporation comprised of representatives from commercial and academic organisations. The corporation creates many industry standard benchmarks for Java amongst others. The benchmark simulates how a Message System would be used in a real world scenario.

The software supports three topologies, Horizontal, Vertical and Freeform. Only Horizontal and Vertical can be used for publishing a result. Freeform can be useful to create a custom workload to test the Messaging System for a given usecase.

Horizontal Topology is where the benchmark scales the number of Topic Subscriptions and Queues whereas a Vertical topology has a fixed number of queues but sends varying volumes of messages depending on the scale.

* The Horizontal results can be found here

* and the Vertical results here.

The scale is configured by setting the BASE configuration property value. At first glance the results can look quite confusing but here is a breakdown of the results:

- What this means in terms of actual performance:

HornetQ sustained a load of about 6 million messages on the vertical topology this can be seen by looking at the following graph

Horizontally HornetQ achieved about 8 million messages per second which is shown on the runtime graph.

The runtime graph is used to show the following:

The expected versus actual message rates. These provide a quickly check the benchmark driver created enough load for the configured scale.
The message sent/received spread. You can expect this because topics are used in the benchmark and many clients will receive a single sent message.
In the Horizontal topology the spread will be greater than Vertical. Horizontal topology by it's nature has a greater distribution of messaging clients.

The system configuration diagram shows the hardware installation used for this result and necessary to get similar levels of performance.

The server was a 2 chip 4 core CPU with 24576 MB of memory and a 1 GbE network interface. The client's, of which there were 4, were 1 chip 4 core CPU with 22528 MB of memory and also a 1 GbE network interface. The HornetQ journal is persisted using a networked mass storage array available to the messaging server.

A few options were set on the JVM which were as follows:

-XX:+UseLargePages - this enables large pages
-XX:LargePageSizeInBytes - set the large page size
-Xms and -Xmx - setting these both to 3800m stops any memory resizing delays

The following changes were made to the HornetQ Server configuration which were as follows

configuration.journal-min-files - this was set to a large number of files
configuration.thread-pool-max-size - increased level of concurrency

For more detailed information about the benchmark see the DesignDocument provided by SPEC. Additionally there is an academic paper detailing the workload characterization in greater detail.

HornetQ 2.2.5 released

2011-06-17T10:29:00.000-07:00

HornetQ 2.2.5.Final will be the first HornetQ release included in the long awaited JBoss AS7. It also contains fixes for paging performance, journal compacting and message priorities amongst others.

HornetQ is rocking out this week

2011-05-05T18:50:00.000-07:00

Since I started working on HornetQ, this was the best week ever.

First the presentation on Judcon had a full room. Even though I suck (at least I think) on presenting, HornetQ shined out by itself as I was showing the new features and the work we have done.

Paging has a new model, more performant and non-blocking. On HornetQ 2.2.2 the syncs on paging are also batched through timers, what really improves performance on page mode also.

The atomic and transparent failover is really enterprise level.

And a lot of cool stuff!

Regarding paging, Drew Dahlke wrote a nice blog entry about how performant is paging on HornetQ on paging:

http://drewdahlke.blogspot.com/2011/05/benchmarking-hornetq-222-paging-mode.html

HornetQ 2.2 Super-HornetQ

2011-03-30T07:09:00.000-07:00

This is the best HornetQ release ever. HornetQ was already cutting edge but is now even better.

It is available here with docs here

This latest release contains the following improvements in functionality

- HornetQ Rest

Thanks to Bill Burke, we have a brand new and cool rest interface that's being released with 2.2.2. Look for a Judcon presentation just about this topic

- New improved failover.

Failover now support multiple backups for live servers and also allows automatic fail back to the original live server.

It also supports using shared file systems for shared journal using distributed locks to handle failover. We also guarantee that the backup server will stay completely passive until the main server crashes avoiding split brain occurring.

- New paging model

The new model now won't lock the address if you have a lazy consumer on a core-queue (or on the Topic Subscription in JMS terms) which previously caused consumer starvation. The system will navigate through page files like a cursor, keeping a soft-cache in memory to avoid duplicated references.

- Large Message Compression

It is now possible to compress the message body of large messages.

On the maintenance front thanks to the JBoss QA guys, their thorough testing means we are more confident then ever of delivering a well tested stable piece of software.

>other improvements include:

- Improvements on the journal reliability.

- Clustering reliability

- XA Integration

On the performance front we have made some optimizations:

- Optimized some non necessary syncs we were doing on the journal

- Optimized syncs on paging. Paging is now also scaling up syncs when many producers are syncing messages.

Also: HornetQ should be available for EAP users really soon, being a viable alternative for enterprise users who require a supportable alternative.

Many thanks for our contributors and for our QA department.

Announcement: Stepping down as project lead

2010-10-08T07:12:00.000-07:00

As of today I'll be stepping down as project lead for the HornetQ project.

This has been a really hard decision for me to make, after all, HornetQ is my baby, and I've invested a lot of mental as well as emotional effort in getting it to where it is today.

The last four years have been really tough. Creating a world class messaging system with a tiny team is a formidable job. But getting this far has taken it's toll on me, and I need to step aside and take a rest for a while.

So what am I going to do next? Not really sure yet, but I'll be taking it easy on sabbatical from Red Hat until the beginning of next year when things should be more clear.

Where does HornetQ go from here? The future is bright for HornetQ. It's the default messaging provider in JBoss AS 6 and 7, and will shortly be available supported in JBoss EAP.

I truly believe HornetQ has the potential to be the world's #1 messaging system. We already know it has the best performance, and now we're rounding off the last few features so we can really position ourselves as a true enterprise class messaging system to go against, and win against any system in the market.

The future is bright :)

This brings me to....

I'd also like to announce that Clebert Suconic will be taking over my role as HornetQ project lead :)

Congratulations Clebert!

Clebert is a nice guy and a talented engineer and knows HornetQ back to front having worked on it from the beginning. HornetQ can't be in better hands than Clebert's.

So, I bid you farewell, and happy messaging!

Tim Fox

=======

HornetQ - the Performance Leader in Enterprise Messaging

2010-09-03T03:22:00.000-07:00

We have just published a report that compares the throughput performance of the majority of the enterprise messaging market, and HornetQ comes out as #1 :) In fact, in several uses cases we pretty much destroy the competition. Read the full report

HornetQ 2.1.2 Released

2010-08-17T23:19:00.001-07:00

We have just released 2.1.2, which contains basically bug fixes and a couple improvements.

The main work done at this release was making sure compacting and the journal is working fine.

The twitter bridge was also made through this release, thanks to our contributor Tomohisa Igarashi.

Case Study - How Last.fm Uses HornetQ for Their Streaming Infrastructure

2010-08-10T06:04:00.000-07:00

We have published a case study about last.fm's use of HornetQ in their infrastructure.

If some of our users have success stories or case studies about HornetQ they want to share, please let us know.

New Stuff on HornetQ 2.1 Final

2010-06-04T13:45:00.000-07:00

HornetQ 2.0 was already fast as mentioned before. And still we have done significant performance improvements on this release.

As a matter of fact, if you are doing light weight message sends you have the option of batching multiple sends and using more of your network resources.

We have also added Stomp to our list of native protocols.

As well the possibility of using WebSockets at the browser

We love what we do, HornetQ is already a very strong messaging server, our performance is astonishing and we are full speed ahead on bringing even more improvements on 2.2

Full speed ahead on 2.2 now!

HornetQ 2.1.0.Final

2010-06-03T16:52:00.000-07:00

I'm glad to announce that HornetQ 2.1.0 is now Final. http://www.jboss.org/hornetq/downloads.html

HornetQ 2.1.0.CR1 is released

2010-05-28T10:33:00.000-07:00

We are glad to announce that we just released 2.1.0.CR1.

Look at the release notes for a complete list of changes:

https://jira.jboss.org/secure/ReleaseNote.jspa?projectId=12310830&version=12314884

Also, if you speak chinese, there's now a Chinese Version for the documentation:

http://www.jboss.org/hornetq/chinesedocs.html

Turbo messaging in JBoss AS 6.0 M3 thanks to HornetQ

2010-05-04T14:03:00.000-07:00

We are glad to announce that HornetQ is now the default (built-in) JMS provider in JBoss AS 6, which has just been released.

We have done many tests to make sure JEE users will be happy with HornetQ in JBoss AS. We believe these are the main benefits for AS users:

- Native IO on Linux

All the HornetQ data is stored in its journal. JBoss AS 6 now bundles the native HornetQ AIO library for Linux users.

Don't worry if you don't use Linux though. HornetQ is still *extremely* fast on Windows, Solaris and other platforms.

- Deployers.

We have integrated HornetQ with the application server deployers. You can have your enterprise application (WAR, EAR.. etc) containing HornetQ deployment descriptors in simple config files which is much simpler than the old formats used by JBoss Messaging and JBoss MQ.

- Administration

We have integrated the JBoss AS admin console with HornetQ. If you don't like XML at all, just use the administrator to maintain your destinations. Simple as click-click

- Enhancements

We have made several enhancements to HornetQ during the release of the JBoss Application Server. HornetQ was already fast before as proven by SpecJMS; an industry standard benchmark yet we have made lots of enhancements. Look at the release notes for more information.

Web Sockets Support For HornetQ

2010-04-29T05:56:00.000-07:00

Web Sockets support has landed in HornetQ trunk and will be available in 2.1.0 release to allow Web browsers to directly exchange messages with HornetQ servers.

See this article for a thorough description of the feature.

With HornetQ 2.0.0, you could send and receive messages from Java applications.
With upcoming 2.1.0 release, we expand it to any environment with a Stomp client and from Web browsers with Web Sockets support.

Enjoy!

Want to see HornetQ at JUDCon 2010?

2010-04-24T03:36:00.001-07:00

If you want to hear a very cool presentation on HornetQ by Diego Naya at JBoss JUDCon 2010, then you can vote for it here! The top voted for presentations will form the third track at the conference.

Free webinar on HornetQ

2010-04-20T04:11:00.000-07:00

The guys at roklee are hosting a free webinar on migration from ActiveMQ to HornetQ. Sign up here

Want to work full-time on HornetQ?

2010-04-14T06:53:00.000-07:00

Rocklee labs, are currently looking to hire a Java senior developer to work full time on the HornetQ project as a community contribution. So.... work full time on a cutting edge open source project AND get paid for it. Doesn't get much better than that

Nice post on ActiveMQ-->HornetQ migration

2010-03-28T09:41:00.000-07:00

Roklee, has posted a nice article on a migration from ActiveMQ to HornetQ

Introduction to HornetQ article on DZone

2010-03-06T04:48:00.000-08:00

If implementing STOMP and websockets support is not enough, Jeff has also found time to write a nice introductory article on HornetQ which is published on Dzone :)

STOMP support and very cool stuff with Web sockets

2010-03-06T04:31:00.000-08:00

Thanks to the work of Jeff Mesnil, HornetQ TRUNK now supports the STOMP protocol natively (that means without having to use StompConnect).

If you're not aware of it, STOMP is a simple, interoperable text based messaging protocol, originally implemented by the ActiveMQ guys.

Sure, it doesn't have some of the features that you'll find in more complex protocols but in many applications it's really good enough.

The great thing about STOMP too is that there are already many STOMP clients available in many different languages, this immediately opens up HornetQ to be accessed by clients in .NET, Python, Ruby etc.

The HornetQ server simultaneous supports all its protocols, so you can, e.g. send a JMS message and consume it as a STOMP message and vice versa, going ahead the same will be true with AMQP.

If STOMP support is not cool enough already, Jeff has done some very cool stuff with web sockets and STOMP.

Jeff has written a simple Javascript library which can speak the STOMP protocol, combine this with web sockets and you can now have your in-browser JavaScript apps talking directly to the HornetQ server.

Jeff demonstrated a very simple prototype with a simple JavaScript chat application that consumed messages from and sent messages to a topic on HornetQ.

Web sockets are new technology so aren't currently available in all browsers, but going ahead this is something that should be much more widely available.

Very cool stuff, and all of this will be available in the up coming 2.1 release :)

HornetQ reaches 800K messages / sec on a commodity server

2010-02-16T10:27:00.000-08:00

I've been hacking away here, and managed to get a throughput of 800K small non persistent messages per second on a single 4 x 2.5GHz core server. The server was only running at 80% utilisation at this point. That means not even one core used :) Now... I just need to think of a way to incorporate this in a non hacky way.....

JBoss HornetQ sets record SPECjms2007 benchmark results

2010-02-12T00:28:00.000-08:00

HornetQ- the new ultra high performance enterprise grade messaging system from JBoss, sets a record breaking score in the SPECjms2007 industry standard benchmark for JMS messaging system performance.

HornetQ 2.0.GA obtained scores up to 307% higher than previously published SPECjms2007 benchmark results, on the same server hardware and operating system set-up.

The peer-reviewed results are available on the spec.org web-site: http://www.spec.org/jms2007/results/jms2007.html

The results were obtained by Kai Sachs and Stefan Appel from an independent research group at the TU Darmstadt, Germany.

Their release announcement can be found here:

http://www.dvs.tu-darmstadt.de/news/specjms2007Results_HornetQ.html

Work is currently occurring on HornetQ 2.1 which includes another round of enhancements to take performance to yet another level.

For more information on HornetQ, please see the web site http://hornetq.org

SPEC® and the benchmark name SPECjms2007® are registered trademarks of the Standard Performance Evaluation Corporation. The results used in the above comparison refer to submissions made on the 17 Sep 2009 and 20 Jan 2010 by TU University Darmstadt

DZone interviews Tim Fox on HornetQ

2010-02-05T01:29:00.001-08:00

DZone presents a short interview on the past, present and future of HornetQ

Using Stomp With HornetQ 2.0.0

2010-01-15T01:08:00.000-08:00

The next release of HornetQ will have native support for Stomp but it already possible to use Stomp with HornetQ 2.0.0 thanks to StompConnect.

I wrote a simple example to show how to configure HornetQ and StompConnect to provide a fully functional Stomp messaging server.

The example code is hosted on GitHub:

git clone git://github.com/jmesnil/hornetq-stomp.git

Enjoy!

HornetQ 2.0.0.GA is released!

2010-01-13T07:54:00.000-08:00

Red Hat Middleware (JBoss) is pleased to announce the release of HornetQ 2.0 GA.

HornetQ is an open source community project to build a multi-protocol, embeddable, ultra high performance, clustered, asynchronous messaging system.

HornetQ can be used to provide messaging functionality from the smallest of applications to empowering the largest of enterprise messaging topologies.

Writing a full-blown, enterprise messaging system is a huge undertaking and it's been a long and windy road to get where we are today. It's taken over 2 years of development and many late nights and weekends to get to this point, but we hope you like what you see.

HornetQ is currently an unsupported community project, but, in the not-too-distant future will be available fully supported by Red Hat as part of a JBoss Enterprise Application Platform subscription. Even sooner than that HornetQ will be available in JBoss Enterprise Application Platform as a technical preview with limited support.

Here's just a taste of why you should consider choosing, or migrating to HornetQ:

* 100% open source software. HornetQ is licenced using the Apache Software License v2.0 to minimise barriers to adoption.
* Jaw-dropping performance. Our unique high performance journal provides astonishing persistent messaging performance. When running on Linux it takes advantage of native code to provide performance unavailable from pure Java. When Linux is not available it still flies at supersonic speed.
* Full feature set. All the features you'd expect in any serious messaging system, and others you won't find anywhere else.
* Usability is key. We ship with over 75 ready-to-run examples demonstrating most aspects of HornetQ functionality.
* Comprehensive documentation. We ship with a in depth 200 page user manual, and a quickstart guide to get you up and running in minutes.
* Written in Java. Runs on any platform with a Java 5+ runtime, that's everything from Windows desktops to IBM mainframes.
* Elegant, clean-cut design with minimal third party dependencies. Run HornetQ stand-alone, run it in integrated in your favourite JEE application server, or run it embedded inside your own product. The choice is yours.
* Solid high availability. We provide a HA solution with automatic client failover so you can guarantee zero message loss or duplication in event of server failure.
* Hugely flexible clustering. Create clusters of servers that know how to load balance messages. Link geographically distributed clusters over unreliable connections to form a global network that powers your core business. Configure routing of messages in a highly flexible way.

Please see the wiki for a complete list of HornetQ features.

What's next for HornetQ? We have lots of exciting things in store including REST support, AMQP support, STOMP support and direct Ajax/Comet/Web sockets support.

I've put together a FAQ that should answer most of the common questions about HornetQ.

Here are some more links:

Project web site
Project blog
Wiki
Download
Documentation
Follow us on twitter
Get your cool HornetQ swag here (T-shirts, mugs etc)

HornetQ 2.0.0.CR3 is released!

2010-01-06T14:14:00.000-08:00

We are very close to GA. This release has a just few minor fixes and improvements as it you could be verified at our release notes. You can download it from the HornetQ Web Site.

HornetQ 2.0.0.CR2 is released!

2009-12-15T01:02:00.000-08:00

This is a candidate release for the forthcoming 2.0.0.GA release (which is due very soon). You can download it from the web site. :)

HornetQ 2.0.0.CR1 is released!

2009-12-07T08:03:00.001-08:00

This is a candidate release for the forthcoming 2.0.0.GA release (which is due very soon). You can download this from the web site. :)

Understanding Connectors & Acceptors

2009-10-16T07:36:00.000-07:00

Connectors and accepors are concepts which often confuses new HornetQ users. Both connectors and acceptors are defined in HornetQ server configuration (hornetq-configuration.xml) but users are often confused about when and why they need to configure them. They are described in the user manual but I have a few drawings which could help the users understand them better.

An acceptor defines which type of connection are accepted by the HornetQ server.

A connector defines how to connect to a HornetQ server. The connector is used by a HornetQ client.

HornetQ defines 2 types of acceptor/connector

invm – this type can be used when both HornetQ client and server run in the same Virtual Machine (invm for Intra Virtual Machine)
netty – this type must be used when HornetQ client and server runs in different Virtual Machines (this connector type uses the netty project to handle the IO)

To communicate, a HornetQ client must use a connector compatible with the server's acceptor.

You can connect from a netty connector to a netty acceptor (if they are configured with the same host and port):

You can not connect from a invm connector to a netty acceptor:

You can not connect from a netty connector to a invm acceptor:

You can not connect from a netty connector on port 5445 to a netty acceptor on port 5446:

By default netty acceptors and connectors uses localhost as the server address. If the HornetQ client is not on the same machine than the server, it will not be able to connect to it.

One source of confusion is that HornetQ connectors are configured on the server. But I wrote that connectors are used by HornetQ clients, not servers! Why should I configure connectors on the server?

There are two reasons to configure connectors in the server:

you want to use JMS & JNDI
you want to communicate between HornetQ servers

Using JMS and JNDI

The standard way to use JMS is to lookup JMS resources (ConnectionFactory and Destination) from JNDI.

Context ctx = new InitialContext();
ConnectionFactory cf = ctx.lookup("/ConnectionFactory")
Connection = cf.createConnection();
// the client is now connected to the JMS server

The ConnectionFactory defines how the JMS client can connect to the JMS server. With HornetQ, this means that the ConnectionFactory implementation will use a connector to connect to the HornetQ Server.

First of all, we must define a "netty" acceptor (in hornetq-configuration.xmlk) so that clients can connect remotely to the server:

<acceptor name="netty">
   <factory-class>org.hornetq.integration.transports.netty.NettyAcceptorFactory</factory-class>
  <!-- by default will accept connection on localhost on port 5445 -->
</acceptor>

Then, we define a "netty" connector (in hornetq-configuration.xmlk)so that JMS clients will know how to connect to the server:

<connector name="netty">
   <factory-class>org.hornetq.integration.transports.netty.NettyConnectorFactory<
   <!-- by default will connect to localhost on port 5445 -->
</connector>

Final step is to configure the JMS ConnectionFactory (in hornetq-jms.xml) so that when it is looked up from JNDI, it uses the "netty" connector to connect to the server:

<connection-factory name="ConnectionFactory">
   <connector-ref connector-name="netty"/>
   <entries>
      <entry name="/ConnectionFactory"/>
   </entries>
</connection-factory>

When the HornetQ server is started, it looks like this:

In JNDI, the HornetQ server has stored the configuration associated to the netty connector with the "/ConnectionFactory" binding.

When the JMS client will look up "/ConnectionFactory", it will also retrieve the netty connector configuration and use it to create a netty connector to connect to the server:

To sum up: if you use JMS with JNDI, you MUST configure a connector to connect to the server itself.

Communication between HornetQ server

The other case when you need to define connectors is when HornetQ servers must communicate. For example, they use core bridges, JMS bridges, diverts, they are in the same cluster...

The important thing to remember is that when two HornetQ servers communicate, one server acts as the client of the other server. In that case, the server acting as the client of the other server MUST define a connector to connect to the other server.

Let's take the example of a JMS bridge: Server #1 will host a core bridge which takes messages from the "source" queue on Server #0 and forwards them to the "target" queue:

Server #0 configuration

Server #0 is a regular HornetQ server, its setup will looks like the "JMS & JNDI" case:

a "netty" acceptor to accept connections from remote clients (one of its clients will be the bridge on Server #1)
a "netty" connector so that clients can connect to it remotely and send messages to the source queue.

Server #1 configuration

Server #1 is a bit more complex. It acts as a HornetQ server with regards to clients consuming from the target queue but it its bridge is acts as a client of Server #0. Its setup requires:

a "netty" acceptor to accept connections from remote clients.
a "netty" connector so that clients can connect to it remotely and receives message from the target queue (as explained in the JMS & JNDI case)
a "source" connector so that the bridge can connect to the Server #0.

Server #1 defines two connectors, "netty" and "source", which serve different purposes: "netty" is used to connect to the server itself (and will be used by its JMS clients) while "source" connector is used to connect to the other server #0 so that the bridge can receive messages from the source queue.

A note on addresses

Both "netty" connectors and acceptors can be configured with a host parameter. However the meaning of this "host" value is not the same for both:

a connector will connect to a single server. Its host parameter must correspond to one of the server address (e.g. localhost or macbook.local or 192.168.0.10)
an acceptor can accept connections from one or many addresses. You can specify a single address (localhost or 192.168.0.10), a list of comma-separated addresses (e.g. 192.168.0.10, 10.211.55.2, 127.0.0.01), or 0.0.0.0 to bind to all the host network interfaces.

Conclusion

Connector configuration can be confusing at first glance but it becomes much more clearer when you follow these simple rules:

If you use JMS with JNDI, you MUST configure a connector to connect to the server itself
If a HornetQ server must communicate with another server, you MUST define a connector to connect to the other server

Welcome to two more committers

2009-10-13T13:12:00.001-07:00

Welcome to two new committers who have earned their commit rights: Kenny MacLeod (skaffman) and Sergej Zizemski (netwater) Welcome!

HornetQ at the London cloud meetup tomorrow

2009-10-13T13:05:00.000-07:00

I'm going to be popping down to the London Cloud Meetup tomorrow (Wednesday 14th Oct) and meeting up with Manik Surtani (Infinspan/JBoss Cache) and Mark Proctor (Drools).

Welcome to a new committer

2009-09-29T03:15:00.001-07:00

We have a new committer on board! :) Diego Naya joins us from Plug Tree Labs where he is the CEO. Diego is based in Argentina. Welcome to Diego!

Want to become a HornetQ committer?

2009-09-25T03:13:00.001-07:00

It's easy - just follow these steps

Back from JBossWorld and on with HornetQ

2009-09-11T06:31:00.000-07:00

Finally I seem to be recovering from an extended period of jet lag after returning from JBossWorld/Red Hat summit in Chicago. The conference was a great success - it was the first time JBossWorld joined forces with the Red Hat guys to create a joint conference. The HornetQ presentation was received very well. We had a good attendance and lots of questions at the end. It's just a shame the presentations weren't video'd - otherwise we could have put it in youtube. Next year I think I will bring along my own video camera and tripod and do it myself! So, what's next? Next stop is HornetQ 2.0 GA. I'm guessing we have around 6 to 8 weeks work to get this out. But of course this depends on how many distractions we are getting. And right now there are a lot ;) Onwards to GA...

A HornetQ Simple Example using Maven

2009-09-01T07:43:00.000-07:00

Since HornetQ can be run embedded and can be run with no little or no thirdparty dependencies, its easy to create a simple project that runs an embedded server or simple client. The example code can be downloaded from here and can be built and run as follows: To build both the embedded server and client run mvn package from the root directory. To run the server run mvn exec:java from the embedded-server directory. To run the client run mvn exec:java from the client directory. Now lets look more closely at both examples in more detail. embedded server Lets start by looking at the dependencies in the pom.xml found under the embedded-server directory. The dependencies needed are configured as follows:

<dependencies>
  <dependency>
     <groupId>org.hornetq</groupId>
     <artifactId>hornetq-core</artifactId>
     <version>2.0.0.GA</version>
     <scope>compile</scope>
  </dependency>
  <dependency>
     <groupId>org.hornetq</groupId>
     <artifactId>hornetq-jms</artifactId>
     <version>2.0.0.GA</version>
     <scope>compile</scope>
  </dependency>
  <dependency>
     <groupId>org.hornetq</groupId>
     <artifactId>hornetq-logging</artifactId>
     <version>2.0.0.GA</version>
     <scope>compile</scope>
  </dependency>
  <dependency>
     <groupId>org.hornetq</groupId>
     <artifactId>hornetq-transports</artifactId>
     <version>2.0.0.GA</version>
     <scope>compile</scope>
  </dependency>
  <dependency>
     <groupId>org.jboss.netty</groupId>
     <artifactId>netty</artifactId>
     <version>3.1.0.GA</version>
  </dependency>
  <dependency>
     <groupId>org.jboss.javaee</groupId>
     <artifactId>jboss-jms-api</artifactId>
     <version>1.1.0.GA</version>
     <scope>compile</scope>
  </dependency>

These dependencies are what you would need to run a HornetQ JMS server now lets look at the code needed to run an embedded server:

public class EmbeddedServer
{
public static void main(String[] args) throws Exception
{
  try
  {
     FileConfiguration configuration = new FileConfiguration();
     configuration.setConfigurationUrl("hornetq-configuration.xml");
     configuration.start();

     HornetQServer server = HornetQServers.newHornetQServer(configuration);
     JMSServerManager jmsServerManager = new JMSServerManagerImpl(server, "hornetq-jms.xml");
     //if you want to use JNDI, simple inject a context here or don't call this method and make sure the JNDI parameters are set.
     jmsServerManager.setContext(null);
     jmsServerManager.start();
     System.out.println("STARTED::");
  }
  catch (Throwable e)
  {
     System.out.println("FAILED::");
     e.printStackTrace();
  }
}
}

You can configure the server via the hornetq-configuration.xml file and configure any JMS objects via the hornetq-jms.xml file. Of course you can do much more than this with your server but this is a good starting point. Consult the HornetQ documentation for more information of what other features are available. client The client example shows how you can easily create a HornetQ JMS client using minimal jars. The dependencies this time are much smaller than the server and contain client jars that HornetQ provides for lightweight client apps.

   <dependency>
     <groupId>org.hornetq</groupId>
     <artifactId>hornetq-core-client</artifactId>
     <version>2.0.0.GA</version>
     <scope>compile</scope>
  </dependency>
  <dependency>
     <groupId>org.hornetq</groupId>
     <artifactId>hornetq-jms-client</artifactId>
     <version>2.0.0.GA</version>
     <scope>compile</scope>
  </dependency>
  <dependency>
     <groupId>org.hornetq</groupId>
     <artifactId>hornetq-transports</artifactId>
     <version>2.0.0.GA</version>
     <scope>compile</scope>
  </dependency>
  <dependency>
     <groupId>org.jboss.netty</groupId>
     <artifactId>netty</artifactId>
     <version>3.1.0.GA</version>
  </dependency>
  <dependency>
     <groupId>org.jboss.javaee</groupId>
     <artifactId>jboss-jms-api</artifactId>
     <version>1.1.0.GA</version>
     <scope>compile</scope>
  </dependency>

The client code in this case is copied from one of the HornetQ examples, running this client will send a message to the server that was started earlier.

public static void main(String[] args) throws Exception
{
   Connection connection = null;
   try
   {
       // Step 1. Directly instantiate the JMS Queue object.
       Queue queue = HornetQJMSClient.createQueue("exampleQueue");

       // Step 2. Instantiate the TransportConfiguration object which contains the knowledge of what transport to use,
       // The server port etc.

       Map connectionParams = new HashMap();
       connectionParams.put(PORT_PROP_NAME, 5445);

       TransportConfiguration transportConfiguration = new TransportConfiguration(NettyConnectorFactory.class.getName(),
                                                                                  connectionParams);

       // Step 3 Directly instantiate the JMS ConnectionFactory object using that TransportConfiguration
       ConnectionFactory cf = HornetQJMSClient.createConnectionFactory(transportConfiguration);

       // Step 4.Create a JMS Connection
       connection = cf.createConnection();

       // Step 5. Create a JMS Session
       Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);

       // Step 6. Create a JMS Message Producer
       MessageProducer producer = session.createProducer(queue);

       // Step 7. Create a Text Message
       TextMessage message = session.createTextMessage("This is a text message");

       System.out.println("Sent message: " + message.getText());

       // Step 8. Send the Message
       producer.send(message);

       // Step 9. Create a JMS Message Consumer
       MessageConsumer messageConsumer = session.createConsumer(queue);

       // Step 10. Start the Connection
       connection.start();

       // Step 11. Receive the message
       TextMessage messageReceived = (TextMessage)messageConsumer.receive(5000);

       System.out.println("Received message: " + messageReceived.getText());
   }
   finally
   {
      if (connection != null)
      {
         connection.close();
      }
   }
}

You can use this template as a starting point for any projects you may want to create that would use HornetQ as a client or as an embedded server.

Persistence on HornetQ

2009-08-25T15:35:00.000-07:00

Persistence on HornetQ is really fast. It is so fast that you would think you are sending non-persistent messages by accident.

Instead of using heavy weighted databases that would provide a bunch of stuff we don't need on HornetQ, we used something faster and still as reliable.

We have written our own circular file Journal that uses either Linux libaio or Java NIO.

Linux libaio is a library that works at the kernel level. We submit writes by sending a DMA Buffer (Direct Memory Access) and a callback interface. The kernel will deal directly with the buffer saving copy time between Java and the disk controller. When the disk is done with the write the callback is returned, and we are sure the data is persisted on the disk.

You may ask, what? At the kernel level?! At the controller level?

Yeah.. that' s the beauty of libaio. It provides system calls to the kernel.

BTW: If you are not a geek who loves programming like I do, you could stop reading this post now :-) Since I will dig a little bit on how it works:

The Journal:

The Journal has a set of pre-allocated files. We keep each file size as close as possible to what would fit on a disk cylinder. We have found a value of 10MiB but that could be different in other systems.

The journal is an append only journal. We always append to the current used file of the pre-allocated set. This way we avoid mechanical movements getting most of the performance possible out of the disk controller.

Deletes are taken as appended records. We add a delete record to the bottom of the file.

We also have a reference counting of records, so when the original file is totally clean (all the records deleted), that file is ready for reuse.

And the journal is transactional also. We have a very nice transactional control, where a commit record is only taken into consideration *if* the entire transaction is on the disk. That gives us ACID control.

SequentialFile interface:

We abstract the disk access through that interface. There are two implementations NIO and AIO. You can select what implementation you want through our configuration. (see User's Manual)

NIO:
This is a very fast approach already. We work at file level, avoiding disk movements. If you don't have Linux or libaio installed in your system, we default to this 100% Java implementation.

AIO (linux libaio):

We have written a small JNI layer that "talks" to libaio on Linux. The basic write method in java, has this signature:

write(int position, int size, ByteBuffer directBuffer, AIOCallback callback). (More detatils on the javadoc)

The buffer here is sent directly to a libaio method called aio_write. (look at aio_write man page).

Another thread will be polling events out of libaio. As soon as the data is on the disk the JNI layer will execute the callback method.

Instead of performing syncs on the disk (which is a slow operation), we use a concurrent latch. You could have many more transactions being executed in parallel. Instead of blocking the whole system while one sync is being performed, we just write as usual and wait for the callback. Each thread will use the most of the performance available at the disk controller. Instead of waiting an expensive sync operation, each thread will be waiting the callback when the data is safely stored.

Conclusion:

Persistence on HornetQ is not only fast but it also scales up when several threads are performing transactions.

This is just one of many of other innovations from HornetQ. We are working hard to make a great software. Feel free to contact us on IRC or our user's forum. We would love to get your feedback.

The Hornet Hatches!

2009-08-24T03:12:00.000-07:00

This is an exciting day for messaging at JBoss and Red Hat.

After months of preparation, the middleware messaging team are excited to announce the birth of a new project "HornetQ".

What is HornetQ?

HornetQ is an open source project to build a multi-protocol, embeddable, high performance, clustered, asynchronous messaging system. HornetQ is an example of Message Oriented Middleware (MoM)

HornetQ is designed with usability in mind: We've provided an extensive, easy-to-understand user-manual and quick-start guide and we ship with over 65 ready-to-run examples out of the box, demonstrating everything from simple JMS usage to complex clusters of servers and more exotic functionality.

HornetQ is designed with flexibility in mind: It's elegant POJO based design has minimal third party dependencies: Run HornetQ as a stand-alone messaging broker, run it in integrated in your favourite JEE application server, or run it embedded inside your own application. It's up to you.

HornetQ is designed with performance in mind: Our unique ultra-high performance journal provides never seen before persistent messaging performance. Automatically switching into native mode when running on Linux, it uses asynchronous IO to provide persistent messaging rates that can saturate the write throughput of a disk. Our pluggable transport system uses JBoss Netty out of the box to provide superb performance and scalability on the wire.

HornetQ is licensed using the Apache Software License V 2.0. The ASL 2.0 has fewer restrictions on use than the LGPL, thus providing fewer barriers to adoption. We want HornetQ to be used as widely as possible.

HornetQ has a great feature set that you'd expect of any serious messaging broker.

But... What about JBoss Messaging?

During its development over the last couple of years the HornetQ code-base was worked on under the name JBoss Messaging 2.0

We decided to rename it and separate it as an independent project since it differs in a many ways from JBoss Messaging 1.x and we did not want to confuse the two, quite different, systems. The vast majority of the code base of HornetQ is different to the code base of JBoss Messaging 1.x

So, what happens with JBoss Messaging now? JBoss Messaging 1.x continues to be known under the name of JBoss Messaging and the project is now in maintenance mode only, with all new messaging development happening on the HornetQ project.

Let's go!... to the future

The future has a lot to hold.

What about cloud computing?

Messaging is going to be a key service in the cloud, and our goal is for HornetQ to be the messaging provider of choice in the cloud. It's our view that RESTful APIs will eventually be the preferred API style in clouds. With that in mind we'll be working on implementing a RESTful style API for interoperable messaging.

Since interoperability is high on our list we won't just stop with REST. HornetQ will also be implementing AMQP and native STOMP support to make it a truly multi-lingual messaging system.

Get involved. HornetQ needs you!

The future certainly has lots in store, and there is plenty for us to do. So why not get involved?

HornetQ is a community, open source project and we'd love to hear from you if you'd like to get involved in development, documentation or help in some other way. We have a small team so any help would be fantastic. Join us!

Come, hear the scoop!

To get all the details on this exciting news and more and to see how it fits in the with JBoss' and Red Hat's overall middleware strategy, come and see me and others speak at the Red Hat summit / JBoss World 2009 in Chicago on September 1-4.

See you there!

Any questions?

I've put together a FAQ that should answer most of the common questions about HornetQ.

Here are some more links:

Project web site Project wiki Download User manual Quick start guide Follow us on twitter