Effective October 27, 2012, online and email support for FuseSource products will move to Red Hat support channels. For more information, please see the JIRA Migration to Red Hat FAQ.
As of October 27th, please open all new issues in the Red Hat Customer Portal .
Issue Details (XML | Word | Printable)

Key: HQ-70
Type: Improvement Improvement
Status: Resolved Resolved
Resolution: Fixed
Priority: Critical Critical
Assignee: Management Team
Reporter: Pedro Neveu
Votes: 0
Watchers: 2
Operations

If you were logged in you would be able to see more operations.
Fuse HQ

JMX instrumentation and integration with FuseHQ Monitoring of CXF Client Endpoints

Created: 30/Mar/09 05:55 PM   Updated: 24/Nov/09 04:34 PM
Component/s: None
Affects Version/s: None
Fix Version/s: 4.1

File Attachments: None
Image Attachments:

1. screenshot-1.jpg
(303 kB)

2. screenshot-2.jpg
(333 kB)

3. screenshot-3.jpg
(161 kB)
Environment: All versions of FUSEHQ on all platforms
Issue Links:
Fixed By
 


 Description  « Hide
There is a request for FUSEHQ to support JMX instrumentation of CXF client endpoints.

 All   Comments   Change History      Sort Order: Ascending order - Click to sort in descending order
Eoghan Glynn added a comment - 19/Jun/09 03:26 PM
Note the related CXF issue that I noticed when looking into this: https://issues.apache.org/jira/browse/CXF-2305 "PerformanceCounter.Client reports incorrect response times for oneway invocations"

CXF-2305 is not a blocker for the current issue, but it does behoove us to get the CXF problem fixed before declaring victory on HQ-70/DEV-1438.


Eoghan Glynn added a comment - 14/Jul/09 07:58 PM
Using a snapshot os FUSE HQ 4.1, and a bit of tweaking, I could sucessfully view the ClientOperationCounter metrics (Average Response Time, Invocation Count per Minute, Max Response Time and Min Response Time) for a CXF 2.2.3 client.

I suspect the problem encountered by the customer may have been as follows. The JMXServiceURL suggested by the CXF documentation is service:jmx:rmi:///jndi/rmi://localhost:9914/jmxrmi, which differs from the default jmx.url specified in the HQ jmx-plugin.xml, by virtue of the port being 9914 as opposed to 1099. Now the cxf-plugin.xml does not over-ride this default with the CXF-specific value (ironically though the HQ smx-plugin.xml does over-ride the default, but with exactly the same value as the default!). So in order for the CXF beans to be visible, the CXF application must be configured with a uri port to suit the Hyperic plugin, rather than vice versa.

Now there are a number of additional apparent issues with the HQ cxf-plugin.xml that will need to be addressed in order for us to declare victory on the CXF visibility question:

1. Only the "availability" metric is presented for the Bus & WorkQueue MBeans, but not the shutdown() operations on each (should be control operations). Similarly only availabity is presented for the Endpoint MBean, but not the getState(), stop(), start(), getAddress() and getTransportId() operations or the Address, State and TransportId attributes. Not necessarily all the foregoing need to be displayed, but consideration should be given to something more than bare availability, which is a little bit useless on its won for something like a work-queue. So for example, we'd probably want to add at least something like <actions include="shutdown"/> to the WorkQueue <service> element.

2. Multiple CXF processes aren't handled gracefull, so for example starting a client and server on the same host leads to errors in the log such as:

2009-07-14 15:55:00,036 ERROR [ScheduleThread] [ScheduleThread] Metric Value not found: Failed to invoke
getProcCpu[State.Name.sw=java,Args.*.eq=-Dcxf.home=/mnt/disk2/eglynn/kits/apache-cxf-2.2.3-SNAPSHOT]:
Query matched multiple processes (2):
State.Name.sw=java,Args.*.eq=-Dcxf.home=/mnt/disk2/eglynn/kits/apache-cxf-2.2.3-SNAPSHOT

due to the auto-discovery query matching multiple processes (as both satisfy the heuristic of having the cxf.home system property set). So we need a better way of providing uniqueness for multiple CXF processes. For example, perhaps multiple JMX URIs could be defined so as to allow multiple CXF applications running on the same host to be recognized as being separate and also peacefully co-exist without port clashes.

3. After a restart, if there's a different set of beans exposed by the next CXF application auto-discovered (e.g. ClientOperationCounter as opposed to ServerOperationCounter), then the old beans continue to be displayed even if they are now obsolete. This is compounded by the transience of BusID which is burned in the MBean names (so after a restart, we can have beans with an obsolete BusID still displayed as a Hyperic resource).

4. The PerformanceCounter attributes are presented in a flat list, with extremely verbose naming e.g.:

sedna.dublin.emea.iona.com CXF 2.x "{http://apache.org/hello_world_soap_http}SOAPService" "greetMeOneWay" "SoapPort" cxf6302571 ClientOperationCounter
sedna.dublin.emea.iona.com CXF 2.x "{http://apache.org/hello_world_soap_http}SOAPService" "greetMe" "SoapPort" cxf6302571 ClientOperationCounter
sedna.dublin.emea.iona.com CXF 2.x "{http://apache.org/hello_world_soap_http}SOAPService" "sayHi" "SoapPort" cxf6302571 ClientOperationCounter

See the Resources / Group Members panel in the first attached screenshot, where its not at all obvious at first glance that these counts relate to each of the operations on the same service. Even with the Map expanded to give a tree-like presentation (attached screenhot 2), the verbose naming makes it less clear than it could be.

On the other hand, the tree idiom used in jconsole is actually superior from a presentational point of view, as it makes it totally clear that the counts are per-operation, grouped by endpoint type.


Eoghan Glynn added a comment - 14/Jul/09 08:03 PM
Verbose naming of counter attributes in a flat list

Eoghan Glynn added a comment - 14/Jul/09 08:04 PM
Verbose naming of counter attributes in an expanded Map

Eoghan Glynn added a comment - 15/Jul/09 08:49 AM
Attached screenshot-3 showing JConsole tree view with concise naming of leaf nodes, making the service/port/operation hierarchy much more apparent.

Amanda Hall added a comment - 24/Nov/09 04:34 PM
Marking this issue as resolved..the only outstanding piece is in HQ-91 which is being tracked separately.

Amanda