© Copyright IBM Corporation 2008, 2009
ALL RIGHTS RESERVED
The Really Small Message Broker is a small program that sends and receives data in the form of messages to and from applications and devices over TCP/IP network connections. Data from a range of sources, including other applications, other brokers, or sensors and meters that measure their physical environment, is published to the broker. The broker then sends a copy of that data to applications that have registered their interest in that data, by subscribing to it.
The Really Small Message Broker takes up only 50KB of storage space and can run in only 150KB or less of memory (actual figures vary), making it ideal for installing and running on small servers and embedded devices, such as on the Linksys NSLU2 or other low-powered servers.
A useful configuration of a broker is to install it on an internet-connected embedded device in a house, where it can obtain data about the house's electricity readings, for example, and publish them to another, external, broker. The external broker can collect the electricity readings from several houses and publish the data so that applications that produce visualizations of the data can subscribe to receive the data as it is published. You can see an example of such output at Current Cost electricity usage graphs.
You can try out the Really Small Message Broker by installing it on your computer, then sending test messages to the broker. The broker will then publish the messages and you can subscribe to the messages.
The Really Small Message Broker is a server implementation of the MQ Telemetry Transport (MQTT) protocol (version 3). Any client that implements this protocol properly can use this server for sending and receiving messages.
Two IBM products that also implement this protocol are WebSphere® Message Broker and Lotus® Expeditor micro broker. Really Small Message Broker can connect to both of these products using its bridge (see below), allowing messages to be forwarded to and from enterprise messaging systems as well as MQTT clients connected directly to it. Lotus Expeditor micro broker is a Java® server which provides Java Messaging Service (JMS) connectivity as well as MQTT. It is available together with many other components as Lotus Expeditor.
For more information about MQTT, WebSphere Message Broker, and Lotus Expeditor micro broker see the MQTT.org website.
Alternatively, if several types of readings were obtained from each weather station (such as temperature, wind direction, pollen count), the topic names could be constructed as weather/london/temperature, weather/london/wind, weather/london/pollen, weather/southampton/temperature, weather/southampton/wind, and so on. Structuring the topic hierarchies by location is useful if the applications are likely to want to obtain all the weather data about a particular location.
A web site that displays the current weather conditions in London, could specifically subscribe to each of the three topics that relate to London. More flexibly, though, the web site could opt to receive any information about the weather in London by using a wildcard character to subscribe to all three topics (and any others that are subsequently added): weather/london/#
Alternatively, an application (another website, for instance) could opt to receive just the temperature readings for each of the three locations by using a different wildcard character: weather/+/temperature; or to receive any weather data from any location by using a wildcard character to specify a more general level in the topic hierarchy: weather/#
You can download and install the broker and explore the capabilities of publish/subscribe messaging,in particular on small or embedded devices.
Really Small Message Broker runs on the following platforms:
To install a broker:
C:\Program Files\broker or /home/laura/brokerYou are now ready to run the broker.
To start the broker on Windows:
To start the broker on Linux:
cd /home/laura/broker
./broker.ubuntu
nohup ./broker.ubuntu >> /dev/null &
The broker is now ready to publish messages.
To stop the broker cleanly, type Ctrl-C. On Linux, if you prefer, you can just kill the process of the running broker:
ps -ef | grep broker kill -term process_id_of_the_running_broker
When broker is used in an application integration environment, messages are sent to it by client applications; other client applications subscribe to receive the messages. The client applications can be written in any language, but must use the MQTT protocol to communicate with the broker.
Sample client applications in Java and in C are available to download from the MQTT.org website. To send and receive test messages with the broker, you will use the sample Java client application, known as WMQTT Utility.
java -jar wmqttSample.jarYou will use this instance of WMQTT Utility as a subscriber application to the messages you will send.
java -jar wmqttSample.jarYou will use this instance of WMQTT Utility as a publisher application to send messages to the broker.
The message that you entered is published to the broker on the testtopic topic. The broker then sends the message to the subscriber instance of WMQTT Utility which subscribed to the testtopic topic.
Retained messages are also known as last known good values. This name gives a hint of how they are intended to be used. If the value for some quantity (such as temperature) is published on a topic each time that value changes, it might be a long time between updates. When a program subscribes to that topic, it might have to wait a long time before receiving its first message. A retained message on that topic ensures that the client receives the last published value as soon as it subscribes. This value might be used, for example, to put an initial value on a visual display.
The message that you published to the testtopic topic, is not retained by the broker. When the broker received the message that you published, the broker simply sent the message to any connected applications that were already subscribed to the topic. Any applications that were not subscribed to the topic did not receive the message, even if they subsequently subscribed to the topic. This is because the message was not marked as retained by the publisher application.
If a publisher sets the retain flag in a message, the
broker retains the message so that applications subsequently
subscribing to that topic
can receive the retained message immediately. The retained message is
kept by the broker until the next retained message published on that
topic is received by the broker.
You can try this out using WMQTT Utility:
By default, the Really Small Message Broker will not save any retained messages or
subscriptions when it stops, so both retained messsages for all topics and durable
subscriptions for all
clients will be lost. To preserve retained messages and durable subscriptions while a broker restarts,
you can specify the persistence and persistence_location settings in the configuration file.
The autosave
settings (autosave_interval and autosave_on_changes) specify that the
broker will save retained messages and durable subscriptions periodically while it is running.
This provides some protection against power failure or program crashes.
On Linux, sending a HUP signal (for example, kill -hup process_id_of_the_running_broker) to the broker will cause it to save retained message and subscription state immediately if there are any changes since the last save.
Often, some of the data that is being handled by a Really Small Message Broker is needed elsewhere, on another broker. Take the example of a petrol station. There might be a number of sensors, such as how much petrol is in the tanks, if the car wash is working, and so on, and also a number of applications that receive and send messages to keep the petrol station runnning.
Many of those messages are relevant only at the petrol station location. Other messages need to be sent to head office for information, alerting of problems, ordering new stock, and so on. Some data must also come from head office to the petrol station location, such as price changes and order inventory.
The Really Small Message Broker has a bridge which enables the broker to connect to another MQTT-capable broker, and exchange messages with it, in both directions. For more sophisticated architectures, such as peer-to-peer networks, the bridge allows multiple concurrent connections to several brokers.
When you run the Really Small Message Broker, you can specify
certain parameters by entering them in a configuration file called broker.cfg, which is present in the same directory as the broker itself, and which is read when you start the broker.
To configure a bridge connection between two brokers
broker.cfg in the same directory as the local broker.connection connection-name addresses IP_address:port topic parameter
There are several parameters you can set for a bridge connection but only the parameters listed above are required for a basic connection to work. The table in Broker configuration parameters lists the other possible parameters that you can set.
The topic parameter is used to specify what topics are bridged between the local Really Small Message Broker and the remote broker. The format is:
topic pattern direction local_prefix remote_prefix
By setting the local and remote prefixes, you can map a topic on the local broker to a different topic on the remote broker. This is most often used for inserting a local topic or topic tree into a contextually-appropriate place in a remote broker's topic tree.
For example, if you are publishing electricity meter readings on a broker inside your home, you might publish to the power topic. However, if you and many other people are also publishing their electricity readings to a remote broker so that the data can be graphed, or aggregated in some way, it is important that you differentiate your data from the other data. You can do this by mapping a remote prefix, such as meters/laura/, so on the remote broker, the local power topic appears as meters/laura/power.
The topic statement to enable this mapping is:
topic power out "" meters/laura/
Notice how the two quotation marks are used as a placeholder for the local_prefix parameter.
The direction parameter specifies whether the topics are going from the local broker to a remote broker (out), from a remote broker to the local broker (in), or in both directions (both). The default direction, if it is not specified, is out.
The following pattern results in messages on a remote broker with topics matching remote/myprefix/# appearing on the local Really Small Message Broker as local/myprefix/#.
topic myprefix/# in local/ remote/
To remap a topic entirely, use "" for the topic pattern, which results in local topic a becoming remote topic b, and vice versa:
topic "" both a b
Be aware that it is quite easy to generate loops when bridging between two brokers. This can happen if you subscribe to the same topics on the remote broker as you are sending to the remote broker. This most often happens when you use the both setting for the direction in the topic. If you connect two Really Small Message Brokers together, special logic is included to prevent most looping situations, by recognising where messages came from, and not sending them back to the same broker. If you are connecting a Really Small Message Broker to a Lotus Expeditor micro broker, or to a WebSphere Message Broker, loops are not detected and you must be careful to avoid them.
By default, a connection will have the automatic start type. This means that it will try to be connected at all times. It will start soon after the broker starts, and whenever a connection fails it will try to restart after a short interval - about 20 seconds.
The other start types are:
Commands can be issued to the broker by writing them to a special file. That file is "broker.upd" in the same directory as the persistence files, which by default is the working directory when the broker was started. The broker will read this file every five seconds if it exists. Once it has been read it is deleted, so that commands are obeyed only once.
The commands available are:
Two reasons you might want the broker to listen on multiple ports are:
A configuration file which will limit external client connections on port 1883 to 1, and internal client connections on port 1884 to 100 looks like this:
listener 1883 max_connections 1 listener 1884 127.0.0.1 max_connections 100
A configuration file which will stop clients connected to ports on 1883 and 1884 from interfering by separating their topics:
listener 1883 mount_point 1883/ listener 1884 127.0.0.1 mount_point 1884/ listener 1885Clients connected to port 1885 will be able to access all topics and all messages.
The Really Small Message Broker, by default, starts up listening for MQTT connections on port 1883, and uses no persistence mechanism for storing any messages. This default behaviour can be changed by the use of a configuration file. You can also use the configuration file to control other attributes of the broker, and the configuration of the bridge. The parameters that you can set in the configuration files are described in Broker configuration parameters.
Enter each configuration parameter in the configuration file on a separate line, in the following format, where value can be one or more values, as appropriate for the parameter, separated by spaces:
parameter value
Lines starting with a # are treated as comments, and are ignored by the broker.
When the broker starts, it looks for a configuration file called broker.cfg in the same directory as the broker (on Windows, for example, the same directory as the broker.exe file).
If you start the broker from the command line, you can specify the name and location of the configuration file as a parameter when you start the broker. For example, if the configuration file is called testbroker.cfg, enter the following command to start the broker:
./broker testbroker.cfg
As an example, a configuration file might look like this:
# sample configuration on port 1882 with retained persistence in /tmp port 1882 max_inflight_messages 50 max_queued_messages 200 persistence_location /tmp/ retained_persistence true
The following table lists the parameters you can use in the
broker.cfg configuration file to configure the broker. Parameters and
values are case sensitive so, for example, if the possible values are true or false, do not specify True or False.
| Parameter | Description | Default value |
|---|---|---|
| autosave_on_changes | true means that the autosave interval indicates the number of changes. false means that the autosave interval indicates the number of seconds. | false |
| autosave_interval | The length of the autosave interval either in seconds or the number of changes, depending on the autosave_on_changes setting. 0 means no autosave. This relates to the periodic writing to disk of retained messages in the broker. See Saving retained messages. | 1800 (30 minutes) |
| bind_address | The local IP address to bind to for the default listener. Useful when a server has multiple network cards and you want to limit access to be only from one network. Specify 127.0.0.1 to restrict client connections to only those from the same machine as the broker. | (The broker allows connections from all network interfaces.) |
| clientid_prefixes | A list of prefixes for client IDs that are allowed to connect to the broker. Any other connections are rejected. For example, test_ allows only clients with IDs such as test_1 and test_connection to connect. |
(Any client ID is accepted.) |
| connection | The name of the bridge connection, which must be alphanumeric (for example, connection1). This parameter indicates the start of a bridge connection section in the configuration file. This name is combined with the server's hostname to give the clientID on the remote broker. | |
| connection_messages | true means that the client connection and disconnection messages are logged. false means that they are not logged. | true |
| listener port [bind_address] | Creates a new listener with the specified port number and local bind address. This parameter indicates the start of a listener section in the configuration file. | (Listener allows connections from all network interfaces.) |
| max_connections | If greater than 0, the maximum number of active clients which are allowed to be connected at one time to the default listener. | -1 (no limit) |
| max_inflight_messages | The maximum number of persistent (QoS 1 or 2 *) outbound messages that can be in flight (being acknowledged or retried) at a time per client. | 10 |
| max_queued_messages | The maximum number of persistent (QoS 1 or 2 *) messages that can be queued for delivery to each client. Important: if the queue of messages for a client fills up, any subsequent messages for that client are discarded and are not delivered to that client. When the queue is able to accept messages again, normal message delivery resumes. | 100 |
| retry_interval | The number of seconds before broker will retry the sending of an unacknowledged QoS 1 or 2 message. | 20 |
| persistence retained_persistence |
true means that retained messages and durable subscriptions are saved when the broker is shut down and restored when the broker restarts. false means that the retained messages and subscriptions are not saved. See Saving retained messages and subscriptions. | false |
| persistence_location | A string prefix that is used before the names of files that are used by Really Small Message Broker to store retained messages and durable subscriptions (if the value of the retained_persistence parameter is true). The prefix must include the trailing directory separator (/). |
(Use the directory in which the broker is installed.) |
| port | The port number that the default listener uses to listen for MQTT client connections. | 1883 |
| trace_level | The level of tracing required. 30 means normal operation and no tracing, 20 means MQTT protocol packet tracing, 10 means full debug detail. Logs are written to the console and to $SYS/broker/log. | 30 |
* QoS 0, 1, and 2 refer to Quality of Service, or persistence settings in MQTT, and indicate how hard the Really Small Message Broker will try to deliver the message. QoS0 is at most once, QoS1 is at least once, and QoS2 is exactly once. For full details, see the MQTT protocol specification.
Really Small Message Broker can support one or more simultaneous connections to other MQTT-capable message brokers. The configuration file contains sections that define each bridge connection. As an example, a configuration file with configuration settings for two bridge connections might look like this:
# bridge to a broker on localhost port 1884
connection local
addresses 127.0.0.1:1884
topic # both local/ remote/
# bridge to one of two possible brokers
connection remote
addresses 1.2.3.4 5.6.7.8:1884
topic #
The following table lists the parameters that you can configure for the bridge. Any number of connections can be configured (in sections starting with the connection parameter), and the following parameters apply to each connection individually. Connection definitions must follow any broker-wide parameters in the configuration file.
| Parameter | Description | Default value |
|---|---|---|
| address addresses |
A list of brokers to connect to, in the format ip_address:port.
This is a hunt list of brokers for the Really Small Message Broker to connect to. The usual mode of operation (round_robin false) is that the first address is the primary broker. The other brokers are fail-over brokers if the primary is not available, and are tried in turn until one accepts the bridge connection. If the connection fails over to one of the other brokers in the list, the primary broker is retried in the background until it is able to accept connections again, at which time the bridge connection drops and reconnects to the primary broker. In "round-robin" mode (round_robin true), the addresses list is traversed cyclically, connecting to the next broker in the list each time the bridge attempts a reconnection. |
|
| cleansession | Overrides the default setting of clean session, an MQTT protocol setting. true means that all session state is cleaned up on connect and disconnect. Session state includes subscriptions and any queued messages. If the number of addresses is more than 1, then setting cleansession to false only makes sense if the servers at each address can continue interrupted MQTT sessions exactly where they left off. | true if number of addresses is greater than 1, else false |
| idle_timeout | If the start_type is lazy, the number of seconds for which the connection is idle before it is automatically closed. | 60 |
| keepalive_interval | The number of seconds between sending ping requests on a connection when there has been no other traffic. The minimum value that will be used is 5. | 60 |
| notifications | true means that the bridge connection notifications are on. false means that the notifications are off. The notifications are retained messages that are published to a designated topic to indicate the status of each bridge connection at both ends of the bridge (on the local Really Small Message Broker, and on the remote broker). The message is either 1 (connected) or 0 (not connected), and can be checked at any time to determine the status of a bridge connection. The notification_topic parameter determines the topic to which notifications are published. | true |
| notification_topic | The topic on which connection notification messages are published. If it is not set, a topic is constructed which includes the client ID of the bridge connection, which is a combination of the server host name and the name used in the connection parameter. If a topic is specified, all bridge notification messages are published to this topic; it is not possible to distinguish between the statuses of multiple bridge connections. | $SYS/broker/connection/{clientID}/state |
| round_robin | true means that the list of addresses specfied in the addresses parameter is used in a "round robin" fashion (a circular list), rather than treating the first in the list as the primary, and the others as fail-over brokers. false means that the list of addresses is not used in a round robin fashion. | false |
| start_type | automatic, lazy, manual or once. For full details, see Starting Bridge Connections. | automatic |
| threshold | If the start_type is lazy, this is the number of messages waiting to be transmitted which will trigger the automatic starting of the connection. | 10 |
| topic pattern direction local_prefix remote_prefix | Controls the set of topics which are bridged between a Really Small Message Broker and a remote broker. For full details, see Connecting two Really Small Message Brokers together. | |
| try_private | true means that the Really Small Messsage Broker attempts to determine whether a broker that it is connecting to is also a Really Small Message Broker. false means that it will not check. Normally this parameter does not need to be changed because the broker automatically adjusts for connections to a remote broker, but if the bridge can not establish a connection with a particular type of remote broker, setting this parameter to false might solve the problem. | true |
The following table lists the parameters that you can configure for a listener. Any number of listeners can be configured (in sections starting with the listener parameter), and the following parameters apply to each listener individually. Listener definitions must follow any broker-wide parameters in the configuration file.
| Parameter | Description | Default value |
|---|---|---|
| max_connections | If greater than 0, the maximum number of active clients which are allowed to be connected at one time to the listener. | -1 (no limit) |
| mount_point | A string which is prefixed to all topics used by clients connecting to this listener. This can be used to ensure clients on different listeners cannot interfere with each other. |
The broker uses topics prefixed with $SYS/ to publish information about its internal state with retained messages. You must subscribe to a topic pattern starting with $SYS/ to receive these messages. For instance, subscribing to # does not give access to the $SYS topics. Subscribing to $SYS/# subscribes to all the system topics, but none of the non-system topics.
| Topic name | Description |
|---|---|
| $SYS/broker/version | The version number of the broker. |
| $SYS/broker/client count/connected | The number of clients that are currently connected. |
| $SYS/broker/heap/current size | The current heap used, in bytes. |
| $SYS/broker/heap/maximum size | The maximum amount of heap that has been used during the running of the broker. |
| $SYS/broker/log/{severity}/{message_number} | Log messages, where severity is one of D, W, I or E, representing Debug, Informational, Warning or Error. Subscribe to $SYS/broker/log/# to get all log messages. |
| $SYS/broker/uptime | The number of seconds since the broker was started. |
| $SYS/messages/sent | The number of messages sent since the broker was started. |
| $SYS/messages/received | The number of messages received since the broker was started. |
| $SYS/broker/timestamp | The build timestamp of the broker. |
Some common problems and suggested solutions:
The system cannot execute the specified program
or The application has failed to start because its side-by-side configuration is incorrect.
To debug interactions between the Really Small Message Broker and clients or other brokers, set the trace-level parameter to 20 in the broker.cfg file. The broker will then print a message describing each MQTT packet sent and received by the broker.
IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or TM), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at Copyright and trademark information.
Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.
Linux is a trademark of Linus Torvalds in the United States, other countries, or both.
Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both.
Other company, product, or service names may be trademarks or service marks of others.