G E N: February 2014

Monday, February 24, 2014

J C O N S O L E

JConsole is shipped as part of the JDK, and doesn't require any specific download or installation. This blog demonstrates how simple it is to browse WebLogic MBeans using JConsole.
In order to connect to a WebLogic MBeanServer, JConsole needs to be started as follow:
$ jconsole -J-Djava.class.path=$JAVA_HOME/lib/jconsole.jar:$JAVA_HOME/lib/tools.jar:$WL_HOME/server/lib/wljmxclient.jar -J-Djmx.remote.protocol.provider.pkgs=weblogic.management.remote -debug
WL_HOME points to the WebLogic binary install.

The above uses WebLogic's thin client classes. In some cases when you access server side classes, or you want to take full advantage of the t3 protocol, you are required to use WebLogic's thick client classes. To do so, you first need to build wlfullclient.jar as follow:
cd $WL_HOME/server/lib
$ java -jar wljarbuilder.jar
You can then start JConsole as follow:
$ jconsole -J-Djava.class.path=$JAVA_HOME/lib/jconsole.jar:$JAVA_HOME/lib/tools.jar:$WL_HOME/server/lib/wlfulclient.jar -J-Djmx.remote.protocol.provider.pkgs=weblogic.management.remote -debug
wlfulclient.jar is not dependent on external jars, and can be copied to client machines. When using the thin client ( wljmxclient.jar ) you cannot just copy wljmxclient.jar to a client's machine as this jar references other jars as part of its manifest Class-Path entry. At the time of this writing, you also need to copy wlclient.jar
$ jconsole -J-Djava.class.path=C:/YYY/java/jdk1.6.0_45/lib/jconsole.jar;C:/Oracle/Middleware/wlserver_10.3/server/lib/wlfullclient.jar -J-Djmx.remote.protocol.provider.pkgs=weblogic.management.remote -debug

-J-Djmx.remote.protocol.provider.pkgs=weblogic.management.remote identifies the package implementing WebLogic's client side connectors code. For instance the 't3' or 'iiop' client connectors.

Finally the -debug flag is extremely handy when a connection failure occurs. In this case the corresponding stack trace is dumped on the console. Once JConsole is started select the Remote process connection, and fill out the target WebLogic's MBeanServer JMX URI, Username and Password fields.

The JMS Service URI: service:jmx:iiop://<hostname>:<port>/jndi/<mbeanserver-type>
$ service:jmx:iiop://cmpd06-po-3p.sys.comcast.net:17001/jndi/weblogic.management.mbeanservers.runtime

The protocol used to communicate with the remote WebLogic process. "iiop" in the above example. WebLogic supports "t3", "iiop" and "rmi". We won't touch further on this topic here.

The WebLogic process to connect to (host & port ). 140.87.10.42:12565 in the above example.
The port value is available from the <WLS_INSTANCE_HOME>/config/config.xml file:
<server>
<name>cmpd06-po-3p</name>
<listen-port>17001</listen-port>
<listen-address>cmpd06-po-3p.sys.comcast.net</listen-address>
</server>
Make sure you look under the correct server if several servers are defined as part of your config.xml. For instance in the above case we are connecting to the server identified as "cmpd06-po-3p".

The MBeanServer to connect to: weblogic.management.mbeanservers.runtime. This identifies WebLogic's "Runtime" MBeanServer. This is the MBeanServer available from any WebLogic process, and that contains both WebLogic and user MBeans. WebLogic also offers two other MBeanServers that are only available from the Domain "AdminServer" process:

The "Domain Runtime" MBeanServer. It aggregates the MBeans registered on the domain's "Runtime" MBeanServers. So as long as a managed server is up, its MBeans can be accessed through the "Domain Runtime" MBeanServer. To connect to that MBeanServer, just use the following JMX URI:
$ service:jmx:iiop://cmpd06-po-3p.sys.comcast.net:17001/jndi/weblogic.management.mbeanservers.domainruntime
"17001" is the iiop port for my "AdminServer" install. Replace with your AdminServer's port as previously explained. Remember; only the AdminServer process runs the "Domain Runtime" MBeanServer.

The "Edit" MBeanServer. It contains the WebLogic Config MBeans that are used to configure the Domain. To connect to that MBeanServer, just use the following JMX URI:
$ service:jmx:iiop://cmpd06-po-3p.sys.comcast.net:17001/jndi/weblogic.management.mbeanservers.edit
"17001" is the iiop port for my "AdminServer" install. Replace with your AdminServer's port as previously explained. Remember; only the AdminServer process runs the "Domain Runtime" MBeanServer.
Note: One can also use "t3" or "rmi" as protocol in place of "iiop"

Finally provide the login and password values for the user associated with the connection. The WebLogic admin user is often used, but other users can be used as well. Depending on the authenticated user and his/her associated roles, operations on MBeans will be granted or denied.

JConsole connected to WebLogic's Runtime MBeanServer: If JConsole fails to connect, look at the stack trace dumped on the console. Make sure you specified the -debug flag when starting JConsole, or you will not see any meaningful information on the console. Common cause of connection failures include: wrong host/port; wrong login/password; jmx.remote.protocol.provider.pkgs not properly specified and incorrect ClassPath. Also note that the above instructions apply to WebLogic server 10.3.1 and above.

JConsole's MBean tree preserves the key order used when building MBeans ObjectNames. The only exception is for the 'type' property which will be included first if present and the 'j2eeType' property which will be included second if present. In general I also use the 'name' property to identify different MBean instances while WebLogic MBeans use 'Type' and 'Name' properties for the same purpose. To ensure that JConsole's MBean tree is build with 'type' or 'Type' as first node and 'name' or 'Name' as its child I use the following command line:
$ jconsole -J-Djava.class.path=$JAVA_HOME/lib/jconsole.jar:$JAVA_HOME/lib/tools.jar:$WL_HOME/server/lib/wljmxclient.jar -J-Djmx.remote.protocol.provider.pkgs=weblogic.management.remote -J-Dcom.sun.tools.jconsole.mbeans.keyPropertyList=type,Type,j2eeType,name,Name -debug

When connecting to WebLogic's "Domain Runtime" MBeanServer, one can also order by MBean 'Location' as follow:
$ jconsole -J-Djava.class.path=$JAVA_HOME/lib/jconsole.jar:$JAVA_HOME/lib/tools.jar:$WL_HOME/server/lib/wljmxclient.jar -J-Djmx.remote.protocol.provider.pkgs=weblogic.management.remote -J-Dcom.sun.tools.jconsole.mbeans.keyPropertyList=Location,type,Type,j2eeType,name,Name -debug

Aside from browsing and editing MBeans, JConsole can be used to monitor WebLogic processes resource usage: Memory, Threads, Class loading. Consult JConsole's documentation for more information.

Configurations for remote JMX access to WebLogic

To celebrate support for MissionControl in the Oracle JDK (7u40 and forwards), this post will illustrate how to access a remote JVM running WebLogic can be accessed using using MissionControl and JConsole.

Unsecure MissionControl setup

This setup can be used to access a remote JVM running WebLogic using MissionControl as the JMX client. The setup is unsecure, i.e. SSL and authentication is disabled. The WebLogic server is assumed to listen on listen address 10.00.00.145.

Server configuration

Add these properties to the WebLogic start arguments:
-Djavax.management.builder.initial=weblogic.management.jmx.mbeanserver.WLSMBeanServerBuilder -Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=7091 -Dcom.sun.management.jmxremote.rmi.port=7091 -Djava.rmi.server.hostname=10.00.00.145 -Dcom.sun.management.jmxremote.local.only=false

The argument javax.management.builder.initial enables usage of the WebLogic platform MBeans. Failure to set this argument will result in a BEA-141277 warning when WebLogic starts.

The com.sun.management.jmxremote.rmi.port argument defines the port where the RMI server and remote objects are exported to by the RMI stack. Used in conjunction with the com.sun.management.jmxremote.port argument, this enables access to the JVM through a single TCP port which is required in a firewall enriched environment.
The arguments java.rmi.server.hostname and com.sun.management.jmxremote.local.only
have been observed to be required if the server is running in a virtualized setup based on VirtualBox&Vagrant.

Client configuration

Create a connection:
Host: 10.00.00.145
Port 7091
Open the JMX Console or the MBean Browser.

Unsecure JConsole setup

This setup can be used to access a remote JVM running WebLogic using JConsole as the JMX client. The setup is unsecure, i.e. SSL and authentication is disabled. The WebLogic server is assumed to listen on listen address 10.00.00.145.

Server configuration

Client configuration

JConsole is configured to use the WebLogic implementation of the network protocols using the jmx.remote.protocol.provider.pkgs system argument. This requires that the libraries wljmxclient.jar and wlclient.jar are located in the WL_HOME directory. Here's a Windows script to start JConsole with the WebLogic network protocol implementations:
set JAVA_HOME=C:\Program Files\Java\jdk1.7.0_45
set WL_HOME=C:\work\jconsole
"%JAVA_HOME%\bin\jconsole.exe" -J-Djava.class.path="%JAVA_HOME%\lib\jconsole.jar;%WL_HOME%\server\lib\wljmxclient.jar" -J-Djmx.remote.protocol.provider.pkgs=weblogic.management.remote -debug
Access WebLogic using the URL: 10.00.00.145:7091
Requires no user credentials since authentication is disabled.

Unsecure JConsole setup with T3

This is a variation of the previous setup. It allows access to the JVM running the WebLogic administration server using the JConsole as the JMX client. The server is accessed using the T3 protocol. Authentication is required with a WebLogic user with administrative privileges. The WebLogic server is assumed to listen on 10.00.00.144:7001

Server configuration

Same server setup as in the previous setup.

Client configuration

JConsole is configured to use the WebLogic implementation of the T3 protocol using the jmx.remote.protocol.provider.pkgs system argument. This requires that the WL_HOME directory to point to the WebLogic home directory. Here's a Windows script to start JConsole with the WebLogic T3 protocol implementation:
set JAVA_HOME=C:\Program Files\Java\jdk1.7.0_45
set WL_HOME=C:\Oracle\Middleware\wlserver
"%JAVA_HOME%\bin\jconsole.exe" -J-Djava.class.path="%JAVA_HOME%\lib\jconsole.jar;%WL_HOME%\server\lib\wljmxclient.jar" -J-Djmx.remote.protocol.provider.pkgs=weblogic.management.remote -debug
Access WebLogic using the URL: service:jmx:t3://10.00.00.144:7001/jndi/weblogic.management.mbeanservers.edit
Requires user credentials despite authentication being disabled.

I can get JDeveloper's external tools capability to do the lifting.

Very simple this. Just choose Tools > External Tools from the menu and press New. Then run through the wizard:

Type is External Program
Program Options > Program Executable = jconsole
Program Options > Arguments = (All on one line of course, reformatted here for readability)

-J-Djava.class.path=${java.path}/lib/jconsole.jar:${java.path}/lib/tools.jar: ${prop:name=weblogic.home}/lib/wljmxclient.jar -J-Djmx.remote.protocol.provider.pkgs=weblogic.management.remote

And that's it. JDeveloper helpfully substitutes both the Java location and the WebLogic location for us saving all that hunting around.

Tip: ${java.path} not expanding correctly (or rather expanding to null). Before running the tool command with the macro make sure that you have a project open and selected as the actual java path is obtained in the context of the JRE version used by the selected project. (Project Properties > Libraries and Classpath > Java SE Version).

Sample Weblogic JMX Client

Our goal is to implement a simple WebLogic JMX client that can be used as a starting point to write any WebLogic JMX client. We will look at the client's code. The classpath used to compile and run the code. The different WebLogic MBeanServers and associated JMX URI's the JMX Client can connect to.

Let's directly dive in the client's code below:

= = = = = = = = = = = = = = = = = = =

package blog.wls.jmx.client;

import javax.management.MBeanServerConnection;

import javax.management.ObjectName;

import javax.management.remote.JMXConnector;

import javax.management.remote.JMXServiceURL;

import javax.management.remote.JMXConnectorFactory;

import java.util.Hashtable;

import java.util.Set;

public class JMXClient {

public static void main(String[] args) throws Exception {

JMXConnector jmxCon = null;

try {

JMXServiceURL serviceUrl = new JMXServiceURL("service:jmx:iiop://127.0.0.1:7001/jndi/weblogic.management.mbeanservers.runtime");

System.out.println("Connecting to: " + serviceUrl);

Hashtable env = new Hashtable();

env.put(JMXConnectorFactory.PROTOCOL_PROVIDER_PACKAGES,"weblogic.management.remote");

env.put(javax.naming.Context.SECURITY_PRINCIPAL, "weblogic");

env.put(javax.naming.Context.SECURITY_CREDENTIALS, "welcome1");

jmxCon = JMXConnectorFactory.newJMXConnector(serviceUrl, env);

jmxCon.connect();

MBeanServerConnection con = jmxCon.getMBeanServerConnection();

Set<ObjectName> mbeans = con.queryNames(null, null);

for (ObjectName mbeanName : mbeans) {

System.out.println(mbeanName);

}

finally {

if (jmxCon != null)

jmxCon.close();

}

= = = = = = = = = = = = = = = = = = =

Let's take a quick look at the above code.

JMXServiceURL serviceUrl = new JMXServiceURL("service:jmx:iiop://127.0.0.1:7001/jndi/weblogic.management.mbeanservers.runtime");

The JMX service URI identifies the following:

The protocol used to communicate with the remote WebLogic process. "iiop" in the above example. WebLogic supports "t3", "iiop" and "rmi". We won't touch further on this topic here.
The WebLogic process to connect to (host & port ). "127.0.0.1:7001" in the above example.The port value is available from the <WLS_INSTANCE_HOME>/config/config.xml file.
The MBeanServer to connect to: " weblogic.management.mbeanservers.runtime". This identifies WebLogic's "Runtime" MBeanServer. This is the MBeanServer available from any WebLogic process, and that contains both WebLogic and user MBeans.

Hashtable env = new Hashtable();
env.put(JMXConnectorFactory.PROTOCOL_PROVIDER_PACKAGES,"weblogic.management.remote");
env.put(javax.naming.Context.SECURITY_PRINCIPAL, "weblogic");
env.put(javax.naming.Context.SECURITY_CREDENTIALS, "welcome1");

The above code specifies properties associated with the connection. "weblogic.management.remote" identifies the package implementing WebLogic's client side connector code. For instance the 't3' or 'iiop' client connector. "weblogic" and "welcome1" are the login/password associated with the user originating the JMX connection.

In the rest of the code we establish the JMX connection, and then retrieve the ObjectNames for all the MBeans registered in the "Runtime" MBeanServer. The ObjectNames are then displayed on the console. We also make sure we release the connection in a finally block to avoid leaking any resource.

The code doesn't contain any WebLogic specific classes, and can be compiled as follow:

$ javac -d . blog/wls/jmx/client/JMXClient.java

Executing the code requires to add WebLogic client side classes to the execution ClassPath. We need to include the WebLogic JMX connector client side classes. Remember we specified those in the environment properties passed to the JMXConnectorFactory. Below is the command I used:

$ java -classpath .:$WL_HOME/server/lib/wljmxclient.jar blog/wls/jmx/client/JMXClient

WL_HOME points to the WebLogic binary install.

The above uses WebLogic's thin client classes. In some cases when you access server side classes, or you want to take full advantage of the t3 protocol, you are required to use WebLogic's thick client classes. To do so, you first need to build wlfullclient.jar as follow:

cd $WL_HOME/server/lib

java -jar wljarbuilder.jar

You can then execute the JMX Client as follow:

$ java -classpath .:$WL_HOME/server/lib/wlfullclient.jar blog/wls/jmx/client/JMXClient

You can also bundle wlfulclient.jar with your client code. When using the thin client you cannot just bundle wljmxclient.jar with you client code, as this jar references other jars as part of its manifest Class-Path entry. At the time of this writing, you also need to bundle wlclient.jar

At this point we are now able to connect to WebLogic's "Runtime" MBeanServer. Each WebLogic process contains a "Runtime" MBeanServer in which local WebLogic MBeans and user-defined application MBeans are registered. WebLogic also offers two other MBeanServers that are only available from the Domain "AdminServer" process:

The "Domain Runtime" MBeanServer. It aggregates the MBeans registered on the domain's "Runtime" MBeanServers. So as long as a managed server is up, its MBeans can be accessed through the "Domain Runtime" MBeanServer. To connect to that MBeanServer, just use the following JMX URI:

service:jmx:iiop://127.0.0.1:7002/jndi/weblogic.management.mbeanservers.domainruntime

"7002" is the iiop port for my "AdminServer" install. Replace with your AdminServer's port as previously explained. Remember; only the AdminServer process runs the "Domain Runtime" MBeanServer.

The "Edit" MBeanServer. It contains the WebLogic Config MBeans that are used to configure the Domain. To connect to that MBeanServer, just use the following JMX URI:

service:jmx:iiop://127.0.0.1:7002/jndi/weblogic.management.mbeanservers.edit

"7002" is the iiop port for my "AdminServer" install. Replace with your AdminServer's port as previously explained. Remember; only the AdminServer process runs the "Domain Runtime" MBeanServer.

Note: One can also use "t3" or "rmi" as protocol in place of "iiop"

The rest of the client's code remains unchanged. Only the JMX URI needs to be changed to connect to different MBeanServers.

References:

Weblogic 12C

Sunday, February 9, 2014

JBoss AS 7 / WildFly in Domain Mode

One of the best features which JBoss AS 7 / Wildfly came up was the domain mode, which is been used when you are looking for a single point of controller over all the servers, which can be running on the same/different boxes.

Most people think that domain mode has to be used when they want clustering which is not true, the main reason domain mode was created to control all the servers which can be in cluster/non-cluster from one point which was not available in the earlier versions of JBoss.

So in this article we would be showing you how can we start JBoss servers in a domain mode in two different boxes which are not in cluster. Once you are able to do it with box-2 you can add other host controller on on different boxes as shown in the below image.

Things to Remember

We need to keep in mind the following 4 things for successfully start JBoss AS 7 in domain mode.
Step 1: Add a name to your host controller in the “host” tag:
           <host name="host1" xmlns="urn:jboss:domain:1.2">

Step 2: Make sure you replace the “<local/>” tag from the “domain-controller” element in the “host.xml” file, when you are creating a host controller:
    <domain-controller>
              <remote host="${jboss.domain.master.address:<IP_ADDRESS-DOMAIN_CONTROLLER>}" port="${jboss.domain.master.port:<NATIVE_PORT-
                                     DOMAIN_CONTROLLER>}" security-realm="ManagementRealm"/>
     </domain-controller>

Step 3: Create a “Management User” in the Domain controller, using the same host name (i.e. “host1″)

Step 4: Now convert that password (i.e password given for the host name in step-3 “host1″) into Base64 encoded password which has to be given in the host controller.
<server-identities>
    
    <secret value="aG9zdDFwYXNzd29yZA=="/>
</server-identities>

Box-1 (Domain Controller – 10.252.168.173)

bhmed-dt-1q.zone7.downingtown.pa.ula.comcast.net [10.252.168.173] - bhmed-dt-1q.cable.comcast.com
On Box-1 we would setup the domain controller, which would be using the “domain.xml” file and the “host.xml” file which defines that on this box we have the domain controller.

Step 1: From “/jboss-as-7.1.1.Final/bin” start the domain controller using the below command
./domain.sh -b 10.252.168.173 -bmanagement 10.252.168.173

Step 2: Now you would have to create users for “Management User”, using the same host names which you are planning to control (i.e. suppose its “host1″)
./add-user.sh
What type of user do you wish to add?
a) Management User (mgmt-users.properties)
b) Application User (application-users.properties)
(a): a
Enter the details of the new user to add.
Realm (ManagementRealm) :
Username : host1
Password : host1password
Re-enter Password : host1password
About to add user 'host1' for realm 'ManagementRealm'
Is this correct yes/no? yes
Added user 'host1' to file '/user/jboss-as-7.1.1.Final/standalone/configuration/mgmt-users.properties'
Added user 'host1' to file '/user/jboss-as-7.1.1.Final/domain/configuration/mgmt-users.properties'

Note: You should NOT use any special characters when you are giving a host name, because during creating a “Managment User” you would get the following error
* Error *
JBAS015239: Only alpha/numeric usernames accepted.
That’s it from domain controller side

Base64 Encrypt Password Generator

There are lot of websites available which would help you generate base64 value from clear text, however you can also use one of our code which we have created, which would do the same for you. We are using commons-codec.jar for this and below Base64EncryptPassword.java and Base64Password.sh files
import org.apache.commons.codec.binary.Base64;
public class Base64EncryptPassword {
     public static void main(String ar[]) throws Exception {
        byte[] encodedByte = org.apache.commons.codec.binary.Base64.encodeBase64(ar[0].getBytes());
        String encodedBase64String = new String(encodedByte);
        System.out.println("\n=========================================================");
        System.out.println("Your Clear Text Password is :     "+ar[0]);
        System.out.println("Base64 Encrypted Password is :     "+encodedBase64String);
        System.out.println("=========================================================\n");
       }
}

Base64Password.sh
#!/bin/sh
export JAVA_HOME=/user/jdk1.6.0_21   #<= change this path
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=commons-codec.jar:.

java Base64EncryptPassword $1

Note: Make sure that you keep all the files commons-codec.jar , Base64EncryptPassword.class and Base64Password.sh in the same folder when you run below command.
/Base64EncryptPassword]$ ./Base64Password.sh host1password
=========================================================
Your Clear Text Password is :     host1password
Base64 Encrypted Password is :     aG9zdDFwYXNzd29yZA==
=========================================================

Box-2 (Host Controller – 10.252.168.174)

bhmed-dt-2q.zone7.downingtown.pa.ula.comcast.net [10.252.168.174] - bhmed-dt-2q.cable.comcast.com
On Box-2 we would setup the host controller, which would NOT be using its “domain.xml” file, but will use the “host.xml” which has the domain controllers details running on Box-1, this way domain controller would be able to controller its hosts.
Step 1: First we would need to give a name for the host controller in the “host” tag
<host name="host1" xmlns="urn:jboss:domain:1.2">

Step 2: As we need to communicate with the Domain controller we would have to make sure we replace the “<local/>” tag from the “domain-controller” element in the “host.xml” file, when you are creating a host controller.
<domain-controller>
    <remote host="${jboss.domain.master.address:10.252.168.173}" port="${jboss.domain.master.port:9999}" security-realm="ManagementRealm"/>
</domain-controller>

Note: You should make sure that you have added and give the correct value for “security-realm” or else you would get the following error during starting the host controller.
[Host Controller] 23:09:43,584 ERROR [org.jboss.remoting.remote.connection] (Remoting "host1:MANAGEMENT" read-1) JBREM000200: Remote connection failed: javax.security.sasl.SaslException: Authentication failed: all available authentication mechanisms failed
[Host Controller] 23:09:43,593 ERROR [org.jboss.as.host.controller] (Controller Boot Thread) JBAS010901: Could not connect to master. Aborting. Error was: java.lang.IllegalStateException: JBAS010942: Unable to connect due to authentication failure.

Step 3: Now the password which was given during creating the “Management User” in domain controller (i.e password given for the host name in step-2 “host1password“) has to be converted into Base64 encoded password which has to be given in the host controller. By doing this domain controller would be able to know that an authorized host is trying to connect.
<server-identities>
    
    <secret value="aG9zdDFwYXNzd29yZA=="/>
</server-identities>

Step 4: All the required configuration has been done, now its time to start your host controller using the below command
./domain.sh -b 10.252.168.174 -bmanagement 10.252.168.174 -Djboss.domain.master.address=10.252.168.173 -Djboss.domain.master.port=9999

Once everything comes up you can see the below logging inside the domain controller console, which means that our configuration works properly as well you can also login into admin console which would be running on “http://10.252.168.173:9090/console” and you would see “host1″ servers in it.

[Host Controller] 22:19:17,250 INFO [org.jboss.as.domain] (domain-mgmt-handler-thread - 1) JBAS010918: Registered remote slave host "host1", JBoss AS