Forum Home » Fuse Distributions » Fuse ESB

Thread: Problem setting up a clustered ESB

 
This question is answered.

Reply to this Thread Reply to this Thread Search Forum Search Forum Back to Thread List Back to Thread List

Permlink Replies: 6 - Last Post: Sep 26, 2011 5:39 AM Last Post By: dhoyt
dhoyt

Posts: 32
Registered: 03/03/11
Problem setting up a clustered ESB
Posted: Sep 19, 2011 4:45 PM
fuse , jms , configuration , message_broker , servicemix , activemq
 
  Click to reply to this thread Reply
Hello all,

I'm trying to setup clustering of Servicemix (apache-servicemix-4.3.1-fuse-02-05 to be exact). After doing a bit of research, it appears that a "network of brokers" is the best way to accomplish this. I have tried to follow several tutorials online, but I can't seem to get the a consumer on myHost2 to listen on a topic that is being populated by a producer on myHost1 (I am using two very simple camel routes that are publishing and subscribing to an ActiveMQ topic). I can telnet to each server from the other on port 61616, so I know the ports are open, but I am not seeing any communication between them.

As far as I can tell, the only file that needs to be changed is activemq-broker.xml (included below). The only changes I made to the activemq-broker.xml file was to change brokerName (they are different for each instance), added the networkConnectors node, and changed localhost to 0.0.0.0 so it could be reached from other servers. Is there anything else that needs to be done? Any help would be greatly appreciated.

<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
xmlns:cm="http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.0.0"
xmlns:ext="http://aries.apache.org/blueprint/xmlns/blueprint-ext/v1.0.0"
xmlns:amq="http://activemq.apache.org/schema/core">
<ext:property-placeholder />

<broker xmlns="http://activemq.apache.org/schema/core" brokerName="atlas-myHost2-master" dataDirectory="${karaf.data}/activemq/default" useShutdownHook="false">
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic=">" producerFlowControl="true" memoryLimit="1mb">
<pendingSubscriberPolicy>
<vmCursor />
</pendingSubscriberPolicy>
</policyEntry>
<policyEntry queue=">" producerFlowControl="true" memoryLimit="1mb">
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>

<managementContext>
<managementContext createConnector="false"/>
</managementContext>

<persistenceAdapter>
<kahaDB directory="${karaf.data}/activemq/default/kahadb"/>
</persistenceAdapter>

<networkConnectors>
<networkConnector name="master_myHost2_to_myHost1" uri="static://(tcp://myHost1:61616)"/>
</networkConnectors>

<transportConnectors>
<transportConnector name="openwire" uri="tcp://0.0.0.0:61616"/>
<transportConnector name="stomp" uri="stomp://0.0.0.0:61613"/>
</transportConnectors>
</broker>

<bean id="activemqConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="tcp://0.0.0.0:61616" />
</bean>

<bean id="pooledConnectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory">
<property name="maxConnections" value="8" />
<property name="connectionFactory" ref="activemqConnectionFactory" />
</bean>

<bean id="resourceManager" class="org.apache.activemq.pool.ActiveMQResourceManager" init-method="recoverResource">
<property name="transactionManager" ref="transactionManager" />
<property name="connectionFactory" ref="activemqConnectionFactory" />
<property name="resourceName" value="activemq.default" />
</bean>

<reference id="transactionManager" interface="javax.transaction.TransactionManager" />

<service ref="pooledConnectionFactory" interface="javax.jms.ConnectionFactory">
<service-properties>
<entry key="name" value="localhost"/>
</service-properties>
</service>
</blueprint>
iocanel

Posts: 150
Registered: 03/16/10
Re: Problem setting up a clustered ESB
Posted: Sep 21, 2011 8:12 AM   in response to: dhoyt in response to: dhoyt 		 
  Click to reply to this thread Reply
From what I make from you configuration file you are adding a network connector that is connecting to itself, while it should be pointing to the other broker.

so if for instance you have on broker A the following config:

<transportConnector name="openwire" uri="tcp://0.0.0.0:61616"/>

and on broker B:

<transportConnector name="openwire" uri="tcp://0.0.0.0:61617"/>


Then the network connector of broker A should point to 61617.

I hope this helps!
dhoyt

Posts: 32
Registered: 03/03/11
Re: Problem setting up a clustered ESB
Posted: Sep 21, 2011 10:24 PM   in response to: iocanel in response to: iocanel 		 
  Click to reply to this thread Reply
I made that change, but now the broker won't come up. Looking at the log, I see the error listed below. It seems that no matter where I put the networkConnectors element, I run into problems with the following element. Any idea what is causing this? Is the networkConnectors element no longer supported?

org.osgi.service.blueprint.container.ComponentDefinitionException: Unable to validate xml
at org.apache.aries.blueprint.container.Parser.validate(Parser.java:288)7:org.apache.aries.blueprint:0.2.0.incubating
at org.apache.aries.blueprint.container.BlueprintContainerImpl.doRun(BlueprintContainerImpl.java:261)7:org.apache.aries.blueprint:0.2.0.incubating
at org.apache.aries.blueprint.container.BlueprintContainerImpl.run(BlueprintContainerImpl.java:213)7:org.apache.aries.blueprint:0.2.0.incubating
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441):1.6.0_20
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303):1.6.0_20
at java.util.concurrent.FutureTask.run(FutureTask.java:138):1.6.0_20
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98):1.6.0_20
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:207):1.6.0_20
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886):1.6.0_20
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908):1.6.0_20
at java.lang.Thread.run(Thread.java:619):1.6.0_20
Caused by: org.xml.sax.SAXParseException: cvc-complex-type.2.4.a: Invalid content was found starting with element 'destinationPolicy'. One of '{"http://activemq.apache.org/schema/core":persistenceAdapter, "http://activemq.apache.org/schema/core":persistenceFactory, "http://activemq.apache.org/schema/core":persistenceTaskRunnerFactory, "http://activemq.apache.org/schema/core":plugins, "http://activemq.apache.org/schema/core":producerSystemUsage, "http://activemq.apache.org/schema/core":proxyConnectors, "http://activemq.apache.org/schema/core":regionBroker, "http://activemq.apache.org/schema/core":services, "http://activemq.apache.org/schema/core":shutdownHooks, "http://activemq.apache.org/schema/core":sslContext, "http://activemq.apache.org/schema/core":systemUsage, "http://activemq.apache.org/schema/core":taskRunnerFactory, "http://activemq.apache.org/schema/core":tempDataStore, "http://activemq.apache.org/schema/core":transportConnectorURIs, "http://activemq.apache.org/schema/core":transportConnectors, WC}' is expected.
at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:233)[:]


<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
xmlns:cm="http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.0.0"
xmlns:ext="http://aries.apache.org/blueprint/xmlns/blueprint-ext/v1.0.0"
xmlns:amq="http://activemq.apache.org/schema/core">

<!-- Allows us to use system properties as variables in this configuration file -->
<ext:property-placeholder />

<broker xmlns="http://activemq.apache.org/schema/core" brokerName="atlas-erosdapp02-master" dataDirectory="${karaf.data}/activemq/Atlas" useShutdownHook="false">

<networkConnectors>
<networkConnector uri="static://(tcp://erosdapp03:61616)"/>
</networkConnectors>

<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic=">" producerFlowControl="true" memoryLimit="1mb">
<pendingSubscriberPolicy>
<vmCursor />
</pendingSubscriberPolicy>
</policyEntry>
<policyEntry queue=">" producerFlowControl="true" memoryLimit="1mb">
<!-- Use VM cursor for better latency
For more information, see:

http://activemq.apache.org/message-cursors.html

<pendingQueuePolicy>
<vmQueueCursor/>
</pendingQueuePolicy>
-->
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>

<!-- Use the following to configure how ActiveMQ is exposed in JMX -->
<managementContext>
<managementContext createConnector="false"/>
</managementContext>

<persistenceAdapter>
<kahaDB directory="${karaf.data}/activemq/Atlas/kahadb"/>
</persistenceAdapter>

<!-- The transport connectors ActiveMQ will listen to -->
<transportConnectors>
<transportConnector name="openwire" uri="tcp://myhost2:61616"/>
</transportConnectors>
</broker>

<bean id="activemqConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="tcp://0.0.0.0:61616" />
</bean>

<bean id="pooledConnectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory">
<property name="maxConnections" value="8" />
<property name="connectionFactory" ref="activemqConnectionFactory" />
</bean>

<bean id="resourceManager" class="org.apache.activemq.pool.ActiveMQResourceManager" init-method="recoverResource">
<property name="transactionManager" ref="transactionManager" />
<property name="connectionFactory" ref="activemqConnectionFactory" />
<property name="resourceName" value="activemq.Atlas" />
</bean>

<reference id="transactionManager" interface="javax.transaction.TransactionManager" />

<service ref="pooledConnectionFactory" interface="javax.jms.ConnectionFactory">
<service-properties>
<entry key="name" value="localhost"/>
</service-properties>
</service>
</blueprint>
davsclaus

Posts: 1,863
Registered: 10/14/08
Re: Problem setting up a clustered ESB
Posted: Sep 22, 2011 7:02 AM   in response to: dhoyt in response to: dhoyt
Correct
  Click to reply to this thread Reply
The XML tags in the XML file is sorted A..Z. So make sure you position the tags in correct order.
dhoyt

Posts: 32
Registered: 03/03/11
Re: Problem setting up a clustered ESB
Posted: Sep 23, 2011 3:00 AM   in response to: davsclaus in response to: davsclaus
fuse , configuration , message_broker , servicemix , activemq , cluster
 
  Click to reply to this thread Reply
That did it! My XML tags were not in alphabetical order. Below is the config (for myHost1) that worked for me. Thanks for all the help!

<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
xmlns:cm="http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.0.0"
xmlns:ext="http://aries.apache.org/blueprint/xmlns/blueprint-ext/v1.0.0"
xmlns:amq="http://activemq.apache.org/schema/core">

<!-- Allows us to use system properties as variables in this configuration file -->
<ext:property-placeholder />

<broker xmlns="http://activemq.apache.org/schema/core" brokerName="broker-myHost1" dataDirectory="${karaf.data}/activemq/default" useShutdownHook="false">
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic=">" producerFlowControl="true" memoryLimit="1mb">
<pendingSubscriberPolicy>
<vmCursor />
</pendingSubscriberPolicy>
</policyEntry>
<policyEntry queue=">" producerFlowControl="true" memoryLimit="1mb">
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>

<!-- Use the following to configure how ActiveMQ is exposed in JMX -->
<managementContext>
<managementContext createConnector="false"/>
</managementContext>

<networkConnectors>
<networkConnector uri="static://(tcp://myHost2:61616)"/>
</networkConnectors>

<persistenceAdapter>
<kahaDB directory="${karaf.data}/activemq/default/kahadb"/>
</persistenceAdapter>

<!-- The transport connectors ActiveMQ will listen to -->
<transportConnectors>
<transportConnector name="openwire" uri="tcp://0.0.0.0:61616"/>
</transportConnectors>
</broker>

<bean id="activemqConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="tcp://localhost:61616" />
</bean>

<bean id="pooledConnectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory">
<property name="maxConnections" value="8" />
<property name="connectionFactory" ref="activemqConnectionFactory" />
</bean>

<bean id="resourceManager" class="org.apache.activemq.pool.ActiveMQResourceManager" init-method="recoverResource">
<property name="connectionFactory" ref="activemqConnectionFactory" />
<property name="resourceName" value="activemq.default" />
</bean>

<reference id="transactionManager" interface="javax.transaction.TransactionManager" />

<service ref="pooledConnectionFactory" interface="javax.jms.ConnectionFactory">
<service-properties>
<entry key="name" value="localhost"/>
</service-properties>
</service>
</blueprint>
michal.warecki

Posts: 49
Registered: 01/23/11
Re: Problem setting up a clustered ESB
Posted: Sep 24, 2011 3:34 PM   in response to: dhoyt in response to: dhoyt 		 
  Click to reply to this thread Reply
I'm little bit confused about this "ESB Clustering".
Is it really ESB clustering or just ActiveMQ clustering? What if I deploy jaxws web service instead of queue? Will it redirect message to slave Servicemix instance? I think no.
Shouldn't we create ActiveMQ cluster and separate database locking based Servicemix cluster (http://fusesource.com/docs/esb/4.4.1/esb_runtime/ESBRuntimeFailoverJdbc.html)?
dhoyt

Posts: 32
Registered: 03/03/11
Re: Problem setting up a clustered ESB
Posted: Sep 26, 2011 5:39 AM   in response to: michal.warecki in response to: michal.warecki 		 
  Click to reply to this thread Reply
I am doing that as well, but I required an active-active cluster. To accomplish that, I have a network of brokers where each node has a slave. This gives me both clustering and HA. As far as I have seen, clustering of ActiveMQ is what is required to cluster ServiceMix.

Point your RSS reader here for a feed of the latest messages in all forums
Go Back Back to Thread List



Contact Us | About This Site | Privacy Policy | Terms of Use | Site Index

FUSE Open Source Community is a community resource for open source service oriented architecture (SOA) and application integration products and technologies. FUSE offers products based on Apache ServiceMix, Apache ActiveMQ, Apache CXF and Apache Camel which are certified, productized and fully supported by the people who wrote the code. Learn about ActiveMQ with our training, support, and consulting on ActiveMQ performance. FUSE also offers integration software that allows developers to service-enable existing components or build new services with preconfigured enterprise integration patterns. FUSE ESB 4 for enterprise service bus is an open source integration platform for BPEL components that supports the latest emerging standards like OSGi and JBI 2.0. FUSE Services Framework allows Java developers to create reusable RESTful services using front-end programming APIs like JAX-WS. Our powerful message oriented middleware solution, FUSE Message Broker, offers a cost-effective and flexible open source messaging platform. FUSE also works to enhance your SOA using content based routing with FUSE Mediation Router.