[#MB-377] Using small queue prefetch sizes cause all consumers to run as fast as the slowest consumer.

FUSE Message Broker

Using small queue prefetch sizes cause all consumers to run as fast as the slowest consumer.

Created: 17/Jul/08 11:43 AM Updated: 20/Aug/08 09:03 AM

Component/s:

broker

Affects Version/s:

5.0.0.17-fuse, 5.1.0.0-fuse

Fix Version/s:

5.0.0.19-fuse

File Attachments:	1. AMQ-1866.patch (8 kB) 2. AMQ1866.java (8 kB) 3. ESB-343.zip (35 kB)
Environment:	Servicemix 3.3.1.3 on Windows, probably on all platforms.

External Issue URL:

https://issues.apache.org/activemq/browse/AMQ-1866

Description

« Hide

If you have a JMS producer sending messages to multiple consumers and you have the cacheLevel set to 3 or higher, eventually all consumers will run as slow as the slowest consumer.
In the attached projects the SA sends messages to a MockOptimizer (JMS consumer) that has two threads. One sleeps for one second and the other sleeps for ten. Initially the thread sleeping for 1 second will consume most of the messages, but towards the end it will start to consume messages every ten seconds as well.

All

Comments

Change History

Sort Order:

[ Permlink | « Hide ]

Martin Murphy added a comment - 17/Jul/08 11:50 AM

You can of course set the cacheLevel to 1 on the JMS producer to work around this.

[ Permlink | « Hide ]

Hiram Chirino added a comment - 22/Jul/08 12:25 PM

Martin.. that test case seems to do a lot more than what the issue is describing. Any change you can slim it down to just the jms producers and multiple consumers?

[ Permlink | « Hide ]

Hiram Chirino added a comment - 22/Jul/08 12:33 PM

I'm going to try to slim it down. I don't think we need any ESB bits for this test case BTW.

[ Permlink | « Hide ]

Hiram Chirino added a comment - 22/Jul/08 01:26 PM

Moved issue to the MB project.. I have created a new junit the confirms the problem.

[ Permlink | « Hide ]

Hiram Chirino added a comment - 22/Jul/08 01:29 PM

Changing issue title to more clearly reflect the issue.

[ Permlink | « Hide ]

Hiram Chirino added a comment - 22/Jul/08 01:46 PM

Attaching the junit case that confirms the bug.

[ Permlink | « Hide ]

Hiram Chirino added a comment - 25/Jul/08 07:18 AM

The issue revolves around how ActiveMQ 5.x pages and dispatches messages to consumers. Every queue keeps list of paged in messages which defaults to 100 max entires. It round robins dispatching messages between consumers, even the slow consumers. Once the slow consumer gets 100 dispatched messages which it has not yet acked due to it being slow, then no further messages are paged in since the page in list is full.

I am evaluating different ways to implement this but, any change to this code could have repercussions which need to get evaluated.

[ Permlink | « Hide ]

Hiram Chirino added a comment - 29/Jul/08 08:50 AM

https://issues.apache.org/activemq/browse/AMQ-1866 has the latest updates. In short we are still looking for an optimal fix for this. Already two different fix proposoals have been made but were found lacking. We still continuing to to work on the problem.

[ Permlink | « Hide ]

Hiram Chirino added a comment - 07/Aug/08 11:12 AM

Update.. we are now currently evaluating and testing the attached patch. It fixes the slow producer problem but we are trying to verify that it as not caused any other regressions.

[ Permlink | « Hide ]

Gary Tully added a comment - 20/Aug/08 09:03 AM

fix from https://issues.apache.org/activemq/browse/AMQ-1866 is now on fuse fixes branch ready for .19 release.