Setting up Hadoop 1.22 on Centos 6.4 with OpenJDK1.6

Before we begin I would like to give a little intro.  We have been working on a lot of “Big Data” and machine learning type projects for some future products and services we intend to offer.  So we like to share some of our initial findings and code into this big realm.  Haven’t heard of Hadoop yet?
As the title indicates we will be installing Hadoop 1.22 on a Centos 6.4 linux server.  This will be a “singe node” install, I will write a separate articles on installing hadoop on a cluster.  If you are just getting into Hadoop this tutorial will help get your first hadoop installation out of the way.

First let’s get started, with some java.

 Installing OpenJDK 1.6

Luckily hadoop works and runs well (from what we have tested) on the openjdk.  No need to download any jdk binaries from Oracle.

yum install java-1.6.0-openjdk.x86_64

Hadoop will run just fine with the vanilla open jdk.  However for maven to run properly we are going need the devel jdk also installed.

yum install java-1.6.0-openjdk-devel.x86_64

Let’s make sure java is registered on your system run.

First let’s see if the correct version is setup in our path.

java -version

java version "1.6.0_28"
OpenJDK Runtime Environment (IcedTea6 1.13.0pre) (rhel-1.66.1.13.0.el6-x86_64)
OpenJDK 64-Bit Server VM (build 23.25-b01, mixed mode)

javac -version

IF you do not see this version of Java, you probably had a previous install of java on your system.  Luckily we are using CentOS so we can easily change this by running the alternatives config.

alternatives -config java
There is 1 program that provides 'java'.
Selection Command
 -----------------------------------------------
 *+ 1 /usr/lib/jvm/jre-1.6.0-openjdk.x86_64/bin/java

If you had any previous versions of java installed make sure the 1.6 JDK is selected.  We will need this version for our later examples.

Now that we have all that java stuff out of the way let’s get down to installing and configuring hadoop.

Note: As with anything there are many ways to do this, Robe de Mariée 2014when we get into installing Hadoop into a cluster it will save you a lot of time.  However sometimes it’s best to do the hard way first to familiarize your self with something new.

Before we begin let’s create some credentials for Hadoop to use.

useradd hadoop

passwd hadoop

You do not have to create a specific user account for hadoop to run properly, of course root works just fine.  You can even enable key based login for your cluster.

Note: For this demo we ran everything as the hadoop account with no key login.

Let’s create a directory for all the hadoop binaries(okay mostly jars) to live in.

Create a Hadoop Directory

mkdir /opt/hadoop

cd /opt/hadoop

wget http://apache.cs.utah.edu/hadoop/common/hadoop-1.2.1/hadoop-1.2.1-bin.tar.gz

Note: Default minimum installs of CentOS don’t include wget.

yum install wget

tar -xzf Hado -C hadoop

Chown -R /opt/hadoop hadoop

 Edit Hadoop Configs

Now that we have everything extracted and proper ownership applied there are a few hadoop configs we will need to change.

 vi conf/core-site.xml

#Add the following inside the configuration tag
<property>
    <name>fs.default.name</name>
    <value>hdfs://localhost:9000/</value>
</property>
<property>
    <name>dfs.permissions</name>
    <value>false</value>
</property>

Edit hdfs-site.xml

 vi conf/hdfs-site.xml

# Add the following inside the configuration tag
<property>
 <name>dfs.data.dir</name>
 <value>/opt/hadoop/hadoop/dfs/name/data</value>
 <final>true</final>
</property>
<property>
 <name>dfs.name.dir</name>
 <value>/opt/hadoop/hadoop/dfs/name</value>
 <final>true</final>
</property>
<property>
 <name>dfs.replication</name>
 <value>2</value>
</property>

Edit mapred-site.xml

 vi conf/mapred-site.xml

# Add the following inside the configuration tag
<property>
        <name>mapred.job.tracker</name>
 <value>localhost:9001</value>
</property>

Edit hadoop-env.sh

 vi conf/hadoop-env.sh

export JAVA_HOME=/usr/lib/jvm/jre-1.6.0-openjdk.x86_64/

Set JAVA_HOME path as per your system configuration for java.

Let’s format our first namenode!

 su - hadoop
 cd /opt/hadoop/hadoop
 bin/hadoop namenode -format

 Start Hadoop

 bin/start-all.sh

Each Service Has It’s Own Status Page

  http://hnode1.vaurent.com:50030/   for the Jobtracker
  http://hnode1.vaurent.com:50070/   for the Namenode
  http://hnode1.vaurent.com:50060/   for the Tasktracker

To stop Hadoop

bin/stop-all.sh

That about sums it all up. I will update this article with links to setting up Maven and running our first Hadoop test with Apache Pig.

Enable SUDO for RHEL & CENTOS

Sudo is an arguably safer alternative to logging in (or using the su command) to the root account. Sudo allows you to partition and delegate superuser commands (functions) without giving a user total "root" power on the system. Here are a few other advantages:

• Privileged commands are logged. It is a simple way to audit who did what at what point in time.
• It is more efficient to use sudo over su, or to log in as root, in reference to keystrokes.
• You don't have to change the root password when an administrator has his root functions revoked, leaves the company, changes roles, etc. The change part is easy, but coordinating the new password with every other administrator can be a hassle.

# Is sudo installed?
Login with the root user.

Let's first determine if the sudo package is installed.

# rpm -q sudo

If the package is not installed, we can retrieve/install it with the following command:

# yum install sudo

# Create a normal user
Create the user and add to the wheel group. The wheel group is usually predefined as the container for administrator accounts.

# useradd -G wheel -c "Test User" testNew

Create a password for the user.

# passwd testNew
Changing password for user testNew.
New UNIX password: P@$$w0rd
Retype new UNIX password: P@$$w0rd
passwd: all authentication tokens updated successfully.

# Or modify an existing user
Add an existing user (the user testMod in my example) to the wheel group.

# usermod -aG wheel testMod

# Modify the sudoers file
Use the visudo command to safely modify the sudoers file.

# visudo

Search for the Allows people in group wheel to run all commands directive and uncomment the second line to enable the wheelgroup to run all commands.

...
## Allows people in group wheel to run all commands
%wheel  ALL=(ALL)       ALL
...

Save the file.

# Test with a privileged command (logged in as a normal user)
We will first attempt to run the visudo command with our normal user account. As expected, the operation will fail.

$ /usr/sbin/visudo
visudo: /etc/sudoers: Permission denied

Now we will run the command within the context of sudo to temporarily elevate the privileges of our normal user.

$ sudo -i visudo
[sudo] password for test: P@$$w0rd

# Verify the command is logged
Check the secure log to verify the event is recorded.

$ sudo grep visudo /var/log/secure
...
Aug  21 20:01:20 centos sudo:     test : TTY=pts/0 ; PWD=/home/test ; USER=root ; COMMAND=/bin/bash -c visudo
...

This is just a single use case of how to implement sudo. I encourage you to check out the man pages and other documentation to see how you can tailor it to your specific environment.

WildFly – A New Improved JBoss Application Server for Linux

As we all know that JBoss AS has been renamed to WildFly.

WildFly 8 is Red Hat‘s Java EE 7 compliant open source application server. The Main features are as below:

Java EE 7 Compatibility: The biggest change in this is that now WildFly 8 is official Java EE7 Certified.
High Performance Web Server: Undertow is new high performance web server written in Java. Now this has been implemented in WildFly 8. This is really designed for high throughput and scalability and can handle millions of connections. Undertow’s lifecycle is completely controlled by the embedding application. This is extremely lightweight with core jar having size of 1MB and embedded server using less than 4MB of heap space. This is really great.
3Port Reduction:  Since it is using Undertow which support for Upgrading HTTP, which will allow multiple protocols to be multiplexed over single HTTP port. WildFly 8 has moved nearly all of its protocols to be multiplexed over two HTTP ports: one is management and another one is application port. This is really a big change and benefit to cloud providers (such as OpenShift) who run hundreds to thousands of instances on a single server. In total, it has two default ports for configuration and they are 9990 (Web Administration Console) and 8080 (Application Console).
Management Role Based Access Control & Auditing: This is the new and interesting thing implemented in WildFly 8. By using this we can create different users and can assign those one to different roles as per requirements. I’ll show you later with screen shots.
Logging:The management API now supports the ability to list and view the available log files on a server. Now, we have attribute called “add-logging-api-dependencies” available for any kind of deployments in which we want to skip container logging. This will disable the adding of the implicit server logging dependencies. We have another option i.e. we can use a jboss-deployment-structure.xml to exclude the logging subsystem. Using this, it will help to stop the logging subsystem from going through any deployment.

We can also make use of another parameter i.e. use-deployment-logging-config for enabling/ disabling processing of logging configuration files within a deployment.
 Note: System Property that we were using for disabling per logging has been deprecated from this version.

Clustering: Again Big change is one clustering. All Features related to Clustering support had been changed in WildFly 8 and these includes as below:

• Distributed web session has been optimized for it with new Java Based Web Server i.e. Undertow.
• mod_cluster support for Undertow.
• Optimized Distributed SSO(Single Sign-On) capabilities and support for Undertow.
• New/optimized distributed @Stateful EJB caching implementation.
• WildFly 8 added some new public clustering API.
• For creating singleton services it provides new public APIs.

 CLI Improvements: CLI Configuration has also been improved. You know All admin love to work on CLI ;). So, now we can create alias for particular server and then can use that alias whenever want to connect to that server using connect command.
There are still lots of enhancements and updates done in WildFly 8. You can check all these at:http://wildfly.org/news/2014/02/11/WildFly8-Final-Released/

Installation of WildFly 8 in Linux

Before moving ahead with Installation make sure that you have Java EE 7 installed on your system. WildFly 8 will not work with previous revisions. Please follow the below guide to install Java EE 7 in the Linux systems (Install JDK/JRE 7u25 in Linux).

Step 1: Downloading WildFly 8

[root@anuppc-02]# wget http://download.jboss.org/wildfly/8.0.0.Final/wildfly-8.0.0.Final.zip
[root@anuppc-02 jboss]# wget http://download.jboss.org/wildfly/8.1.0.CR1/wildfly-8.1.0.CR1.tar.gz
--2014-05-11 19:00:31--  http://download.jboss.org/wildfly/8.1.0.CR1/wildfly-8.1.0.CR1.tar.gz
Resolving download.jboss.org... 23.67.250.131, 23.67.250.122
Connecting to download.jboss.org|23.67.250.131|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 115039521 (110M) [application/x-gzip]
Saving to: `wildfly-8.1.0.CR1.tar.gz'

100%[======================================================================================================>] 115,039,521  975K/s   in 2m 10s

2014-05-11 19:02:41 (867 KB/s) - `wildfly-8.1.0.CR1.tar.gz' saved [115039521/115039521]

[root@anuppc-02 data3]#

Step 2: Extract it

# tar -xvf yourfile.tar [to extract to current directory]
# tar -C /myfolder -zxvf yourfile.tar.gz [ to extract to another directory]
$ tar -zxvf wildfly-8.1.0.CR1.tar.gz

Step 3: Setting Environment variable

Now set some environment variables. You can set these on system wise or within your configuration files. Here I am setting within configuration files standalone.sh and standalone.conf in ‘bin‘ folder.
[root@anuppc-02 data]# cd wildfly-8.0.0.Final
[root@anuppc-02 data]# cd bin/ 
# Add following 2 lines to     standalone.sh / standalone.conf
JBOSS_HOME="/var/cemp/data3/wildfly-8.1.0.CR1"
JAVA_HOME="/usr/java/jdk1.8.0_05"
Note: For whole system wide, you can set it under ‘/etc/profile‘ file.

standalone.xml
<socket-binding-group name="standard-sockets" default-interface="public" port-offset="${jboss.socket.binding.port-offset:0}">
    <socket-binding name="management-http" interface="management" port="${jboss.management.http.port:9990}"/>
    <socket-binding name="management-https" interface="management" port="${jboss.management.https.port:9993}"/>
    <socket-binding name="ajp" port="${jboss.ajp.port:8009}"/>
    <socket-binding name="http" port="${jboss.http.port:9080}"/> <!-- 8080 to 9080 -->
    <socket-binding name="https" port="${jboss.https.port:9443}"/> <!-- 8443 to 9443 -->
    <socket-binding name="txn-recovery-environment" port="4712"/>
    <socket-binding name="txn-status-manager" port="4713"/>
    <outbound-socket-binding name="mail-smtp">
        <remote-destination host="localhost" port="25"/>
    </outbound-socket-binding>
 </socket-binding-group>
</server>

Step 4: Starting WildFly 8

Now start server i.e. for standalone mode use ‘standalone.sh‘ and for domain mode use ‘domain.sh‘.
[root@anuppc-02 bin]# ./standalone.sh
[root@anuppc-02 bin]# ./domain.sh
But, here I am starting in standalone mode. By default it will get start by ‘standalone.xml‘ file, But You can also start with some other configuration using ‘–server-config‘ option.
As below I am starting server with ‘standalone-full-ha.xml‘ and this file is present in “$JBOSS_HOME/standalone(profile)/configuration/”.
[root@anuppc-02 bin]# ./standalone.sh --server-config standalone-full-ha.xml

Step 5: Acessing WildFly 8

Now you can point your browser to ‘http://localhost:9080‘ (if using the default configured http port) which brings you to the Welcome Screen.
From here, you can access WildFly community documentation guides and enhanced web-based Administration Console access.

Step 6: Managing WildFly 8

WildFly 8 provides two administrative consoles for managing running instance:

• web-based Administration Console
• command-line interface

Before connecting to administration console or remotely using the command line, you will need to create a new user using the ‘add-user.sh‘ script in the bin folder.
Next, go to ‘bin‘ directory, set ‘JBOSS_HOME‘ in add-user.sh (if variable is not set on system bases) and create user as below.
[root@anuppc-02 bin]# ./add-user.sh
Once starting the script you will be guided through the process to add a new user:

Sample Output
What type of user do you wish to add?
 a) Management User (mgmt-users.properties)
 b) Application User (application-users.properties)
(a):
Enter the details of the new user to add.
Using realm 'ManagementRealm' as discovered from the existing property files.
Username : admin
The username 'admin' is easy to guess
Are you sure you want to add user 'admin' yes/no? yes
Password recommendations are listed below. To modify these restrictions edit the add-user.properties configuration file.
 - The password should not be one of the following restricted values {root, admin, administrator}
 - The password should contain at least 8 characters, 1 alphanumeric character(s), 1 digit(s), 1 non-alphanumeric symbol(s)
 - The password should be different from the username
Password :
Re-enter Password :
What groups do you want this user to belong to? (Please enter a comma separated list, or leave blank for none)[  ]:
About to add user 'admin' for realm 'ManagementRealm'
Is this correct yes/no? yes
Added user 'admin' to file '/data/wildfly-8.0.0.Final/standalone/configuration/mgmt-users.properties'
Added user 'admin' to file /data/wildfly-8.0.0.Final/domain/configuration/mgmt-users.properties'
Added user 'admin' with groups  to file /data/wildfly-8.0.0.Final/standalone/configuration/mgmt-groups.properties'
Added user 'admin' with groups  to file /data/wildfly-8.0.0.Final/domain/configuration/mgmt-groups.properties'
Is this new user going to be used for one AS process to connect to another AS process?
e.g. for a slave host controller connecting to the master or for a Remoting connection for server to server EJB calls.
yes/no? yes
To represent the user add the following to the server-identities definition
Press any key to continue . . .
Now access the web-based Administration Console at ‘http://localhost:9990/console‘ and enter the new created username and password to directly access the Management Console.

If you prefer to handle your server from the CLI, run the ‘jboss-cli.sh‘ script from the ‘bin‘ directory that offers the same capabilities available via the web-based UI.
[root@anuppc-02 bin]# cd bin
[root@anuppc-02 bin]# ./jboss-cli.sh --connectConnected to standalone controller at localhost:9999

For more information, follow the official WildFly 8 documentation at https://docs.jboss.org/author/display/WFLY8/Documentation.

MBean Servers in Weblogic

Weblogic comes with three own MBeanServers, which are exported via RMI/IIOP as JSR-160 connectors. They can be looked up via a certain JNDI name as shown in the table below. Additionally, there is the ubiquitous PlatformMBeanServer.
MBean Server                                  JNDI Name
Domain Runtime MBean Server          weblogic.management.mbeanservers.domainruntime
Runtime MBean Server                  weblogic.management.mbeanservers.runtime
Edit MBean Server                          weblogic.management.mbeanservers.edit
PlatformMBeanServer                          —

The Runtime MBean Server specifies an individual application server, whereas the Domain Runtime MBean Server exposes the MBeans for all servers in a cluster. The Edit MBean Server is used for accessing and modifying the domain configuration.
Two access modes: There are two ways how the MBeanServers mentioned above can be exported for remote access:

1. Via RMI/IIOP exported by Weblogic. This way, the three Weblogic MBeanServers (those with an JNDI name) can be exported (but not the PlatformMBeanServer directly). Advantage of this export is that it can be enabled with the admin console and that it includes the complete Weblogic security stack.
2. Via RMI/JRMP exported by the JVM. This allows for the PlatformMBeanServer to be exported (but not the other, Weblogic specific MBeanServers). It gets enabled as usual by setting certain java defines as startup options and is secured the JDK way. A forthcoming blog will clarify how to setup security for JDK exported JSR-160 connectors.

Both methods are explained in detail in the following sections.
RMI/IIOP exported by Weblogic: First of all, some configuration items must be set to enable IIOP exported MBeans. In the admin console, check that following properties are set:

• First, IIOP must be enabled: Domain (‘wl_server’) / Environment / Servers -> Server (‘examplesServer’) -> Protocols -> IIOP -> Enable IIOP
• Allow for anonymous read access, if you want to monitor without sending credentials: Domain (‘wl_server’) -> Security -> General -> Anonymous Admin Lookup Enabled
• If you want to secure IIOP access (or want to have write access), set the name and password of the default user: Domain (‘wl_server’) / Environment / Servers -> Server (‘examplesServer’) -> Protocols -> IIOP -> (Advanced) -> “Default IIOP Username” and “Default IIOP Password”

You need to restart the server if you change one of the options above. The JMX service URL for accessing Weblogic JMX connectors via IIOP looks like
service:jmx:rmi:///jndi/iiop://<server address>:<port>/<jndi name>
where the JNDI name is one of the MBeanServer’s JNDI name as described above. For example:
service:jmx:iiop:///jndi/iiop://bhut:7001/weblogic.management.mbeanservers.runtime

RMI/JRMP exported by JVM: As an alternative the usual way for exporting MBeans via RMI/JRMP can be used. This includes to set some java defines as startup options (probably within setDomainEnv.sh) like:
JAVA_OPTIONS="$JAVA_OPTIONS \ 
    -Dcom.sun.management.jmxremote \
    -Dcom.sun.management.jmxremote.port=9999 \
    -Dcom.sun.management.jmxremote.ssl=false \
    -Dcom.sun.management.jmxremote.authenticate=false"
jconsole service:jmx:rmi:///jndi/rmi://target:9999/jmxrmi
Unfortunately, by default, only the PlatformMBeanServer gets exported without any Weblogic specific MBean. However, there is a way to get to the Weblogic MBeans as described in the next section.

Using PlatformMBeanServer for Weblogic MBeans: For monitoring purposes having Weblogic runtime MBeans and JVM MXBeans in different MBeanServers which are exportable in different ways is quite annoying. However, for Weblogic there is a solution by configuring WLS to use the PlatformMBeanServer as it’s MBeanServer. With this configuration, your are able to access Weblogic MBeans from RMI/JRMP Service URLs (as it is used with ‘normal’ JMX clients like jconsole) and Java MXBeans from the RMI/IIOP connector used by Weblogic.

For this to work, you need to set Domain (‘wl_server’) > Configuration > General > (Advanced) > Platform MBeanServer enabled in the admin console for both WLS 9 and 10. Afterwards a server restart is required.

For WLS 10 an additional configuration parameter has to be set in order to let WLS use the PlatformMBeanServer for its runtime MBeans. The attribute PlatformMBeanServerUsed needs to be set to true on the JMXMBean (it is false by default). Unfortunately I didn’t find a way to set this attribute via the admin console, but only via the Weblogic scripting environment WLST. Assuming that your current working directory is you WLS 10 root directory, you should use wlst.sh to fire up the WLST in interactive mode and when the server is not running:
$ common/bin/wlst.sh
.....
wls:/offline> readDomain('samples/domains/wl_server')
wls:/offline/wl_server>cd('JMX')
wls:/offline/wl_server/JMX>ls()
drw-   NO_NAME_0
wls:/offline/wl_server/JMX>cd('NO_NAME_0')
wls:/offline/wl_server/JMX/NO_NAME_0>ls()
-rw-   CompatibilityMBeanServerEnabled               true
-rw-   DomainMBeanServerEnabled                      true
-rw-   EditMBeanServerEnabled                        true
-rw-   InvocationTimeoutSeconds                      0
-rw-   ManagementEJBEnabled                          true
-rw-   Name                                          null
-rw-   Notes                                         null
-rw-   PlatformMBeanServerEnabled                    false
-rw-   PlatformMBeanServerUsed                       false
-rw-   RuntimeMBeanServerEnabled                     true
wls:/offline/wl_server/JMX/NO_NAME_0>set('PlatformMBeanServerUsed','true')
wls:/offline/wl_server/JMX/NO_NAME_0>set('PlatformMBeanServerEnabled','true')
wls:/offline/wl_server/JMX/NO_NAME_0>updateDomain()
wls:/offline/wl_server/JMX/NO_NAME_0>closeDomain()
wls:/offline>exit()
The path to the domain (samples/domains/wl_server), an the name of the JMX-Bean (NO_NAME_0) might differ at your side, but I think, the idea is clear. As shown above you also can set PlatformMBeanServerEnabled via WLST as well.

Fire up the server and you should be ready for accessing WLS MBeans and JDK MXMBeans from within the same MBeanServer on Weblogic 10.

BTW, setting PlatformMBeanServerUsed via jmx4perl itself doesn’t work, because Weblogic needs some extra boilerplate (which I didn’t dive into) before configuration can be changed via JMX:
Part 1: Source:

• How to implement a custom MBean to manage configuration associated with an application.
• How to package the resulting code and configuration as part of the application's ear file.
• How to register MBeans upon application startup, and unregistered them upon application stop (or undeployment).
• How to use generic JMX clients such as JConsole to browse and edit our application's MBean.

Part 2: Source:

• How to add localized descriptions to our MBean, MBean attributes, MBean operations and MBean operation parameters.
• How to specify meaningful name to our MBean operation parameters.
• We also touched on future enhancements that will simplify how we can implement localized MBeans.

Part 3: Source:

• How to localize our MBean based on the calling client's locale as opposed to the server's locale.
• How to take advantage of the default resource mapping provided by WebLogic Server to localize MBean descriptions.
• How to write a simple client that access our MBean using a locale other than the server's locale.
• How to set the locale associated with JConsole.
• In this new blog entry we will demonstrate:

Part 4 : Source 

• How to emit an AttributeChangeNotification each time a new property is added or changed through our MBean.
• How to properly localize our notifications.
• How to observe notification using JConsole.
• How to write a client that subscribes to the notification emitted by our MBean.

J C O N S O L E

JConsole is shipped as part of the JDK, and doesn't require any specific download or installation. This blog demonstrates how simple it is to browse WebLogic MBeans using JConsole.
In order to connect to a WebLogic MBeanServer, JConsole needs to be started as follow:
$ jconsole -J-Djava.class.path=$JAVA_HOME/lib/jconsole.jar:$JAVA_HOME/lib/tools.jar:$WL_HOME/server/lib/wljmxclient.jar -J-Djmx.remote.protocol.provider.pkgs=weblogic.management.remote -debug
WL_HOME points to the WebLogic binary install.

The above uses WebLogic's thin client classes. In some cases when you access server side classes, or you want to take full advantage of the t3 protocol, you are required to use WebLogic's thick client classes. To do so, you first need to build wlfullclient.jar as follow:
cd $WL_HOME/server/lib
$ java -jar wljarbuilder.jar
You can then start JConsole as follow:
$ jconsole -J-Djava.class.path=$JAVA_HOME/lib/jconsole.jar:$JAVA_HOME/lib/tools.jar:$WL_HOME/server/lib/wlfulclient.jar -J-Djmx.remote.protocol.provider.pkgs=weblogic.management.remote -debug
wlfulclient.jar is not dependent on external jars, and can be copied to client machines. When using the thin client ( wljmxclient.jar ) you cannot just copy wljmxclient.jar to a client's machine as this jar references other jars as part of its manifest Class-Path entry. At the time of this writing, you also need to copy wlclient.jar
$ jconsole -J-Djava.class.path=C:/YYY/java/jdk1.6.0_45/lib/jconsole.jar;C:/Oracle/Middleware/wlserver_10.3/server/lib/wlfullclient.jar -J-Djmx.remote.protocol.provider.pkgs=weblogic.management.remote -debug

-J-Djmx.remote.protocol.provider.pkgs=weblogic.management.remote identifies the package implementing WebLogic's client side connectors code. For instance the 't3' or 'iiop' client connectors.

Finally the -debug flag is extremely handy when a connection failure occurs. In this case the corresponding stack trace is dumped on the console. Once JConsole is started select the Remote process connection, and fill out the target WebLogic's MBeanServer JMX URI, Username and Password fields.

The JMS Service URI: service:jmx:iiop://<hostname>:<port>/jndi/<mbeanserver-type>
$ service:jmx:iiop://cmpd06-po-3p.sys.comcast.net:17001/jndi/weblogic.management.mbeanservers.runtime

The protocol used to communicate with the remote WebLogic process. "iiop" in the above example. WebLogic supports "t3", "iiop" and "rmi". We won't touch further on this topic here.

The WebLogic process to connect to (host & port ). 140.87.10.42:12565 in the above example.
The port value is available from the <WLS_INSTANCE_HOME>/config/config.xml file:
<server>
  <name>cmpd06-po-3p</name>
  <listen-port>17001</listen-port>
  <listen-address>cmpd06-po-3p.sys.comcast.net</listen-address>
</server>
Make sure you look under the correct server if several servers are defined as part of your config.xml. For instance in the above case we are connecting to the server identified as "cmpd06-po-3p".

The MBeanServer to connect to: weblogic.management.mbeanservers.runtime. This identifies WebLogic's "Runtime" MBeanServer. This is the MBeanServer available from any WebLogic process, and that contains both WebLogic and user MBeans. WebLogic also offers two other MBeanServers that are only available from the Domain "AdminServer" process:

The "Domain Runtime" MBeanServer. It aggregates the MBeans registered on the domain's "Runtime" MBeanServers. So as long as a managed server is up, its MBeans can be accessed through the "Domain Runtime" MBeanServer. To connect to that MBeanServer, just use the following JMX URI:
$ service:jmx:iiop://cmpd06-po-3p.sys.comcast.net:17001/jndi/weblogic.management.mbeanservers.domainruntime
"17001" is the iiop port for my "AdminServer" install. Replace with your AdminServer's port as previously explained. Remember; only the AdminServer process runs the "Domain Runtime" MBeanServer.

The "Edit" MBeanServer. It contains the WebLogic Config MBeans that are used to configure the Domain. To connect to that MBeanServer, just use the following JMX URI:
$ service:jmx:iiop://cmpd06-po-3p.sys.comcast.net:17001/jndi/weblogic.management.mbeanservers.edit
"17001" is the iiop port for my "AdminServer" install. Replace with your AdminServer's port as previously explained. Remember; only the AdminServer process runs the "Domain Runtime" MBeanServer.
Note: One can also use "t3" or "rmi" as protocol in place of "iiop"

Finally provide the login and password values for the user associated with the connection. The WebLogic admin user is often used, but other users can be used as well. Depending on the authenticated user and his/her associated roles, operations on MBeans will be granted or denied.

JConsole connected to WebLogic's Runtime MBeanServer: If JConsole fails to connect, look at the stack trace dumped on the console. Make sure you specified the -debug flag when starting JConsole, or you will not see any meaningful information on the console. Common cause of connection failures include: wrong host/port; wrong login/password; jmx.remote.protocol.provider.pkgs not properly specified and incorrect ClassPath. Also note that the above instructions apply to WebLogic server 10.3.1 and above. 

JConsole's MBean tree preserves the key order used when building MBeans ObjectNames. The only exception is for the 'type' property which will be included first if present and the 'j2eeType' property which will be included second if present. In general I also use the 'name' property to identify different MBean instances while WebLogic MBeans use 'Type' and 'Name' properties for the same purpose. To ensure that JConsole's MBean tree is build with 'type' or 'Type' as first node and 'name' or 'Name' as its child I use the following command line: 
$ jconsole -J-Djava.class.path=$JAVA_HOME/lib/jconsole.jar:$JAVA_HOME/lib/tools.jar:$WL_HOME/server/lib/wljmxclient.jar -J-Djmx.remote.protocol.provider.pkgs=weblogic.management.remote -J-Dcom.sun.tools.jconsole.mbeans.keyPropertyList=type,Type,j2eeType,name,Name -debug

When connecting to WebLogic's "Domain Runtime" MBeanServer, one can also order by MBean 'Location' as follow: 
$ jconsole -J-Djava.class.path=$JAVA_HOME/lib/jconsole.jar:$JAVA_HOME/lib/tools.jar:$WL_HOME/server/lib/wljmxclient.jar -J-Djmx.remote.protocol.provider.pkgs=weblogic.management.remote -J-Dcom.sun.tools.jconsole.mbeans.keyPropertyList=Location,type,Type,j2eeType,name,Name -debug

Aside from browsing and editing MBeans, JConsole can be used to monitor WebLogic processes resource usage: Memory, Threads, Class loading. Consult JConsole's documentation for more information. 

Configurations for remote JMX access to WebLogic

To celebrate support for MissionControl in the Oracle JDK (7u40 and forwards), this post will illustrate how to access a remote JVM running WebLogic can be accessed using using MissionControl and JConsole.

Unsecure MissionControl setup

This setup can be used to access a remote JVM running WebLogic using MissionControl as the JMX client. The setup is unsecure, i.e. SSL and authentication is disabled. The WebLogic server is assumed to listen on listen address 10.00.00.145.

Server configuration

Add these properties to the WebLogic start arguments:
-Djavax.management.builder.initial=weblogic.management.jmx.mbeanserver.WLSMBeanServerBuilder -Dcom.sun.management.jmxremote.authenticate=false 
-Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=7091 -Dcom.sun.management.jmxremote.rmi.port=7091 -Djava.rmi.server.hostname=10.00.00.145 -Dcom.sun.management.jmxremote.local.only=false

The argument javax.management.builder.initial enables usage of the WebLogic platform MBeans. Failure to set this argument will result in a BEA-141277 warning when WebLogic starts.

The com.sun.management.jmxremote.rmi.port argument defines the port where the RMI server and remote objects are exported to by the RMI stack. Used in conjunction with the com.sun.management.jmxremote.port argument, this enables access to the JVM through a single TCP port which is required in a firewall enriched environment.
The arguments java.rmi.server.hostname and com.sun.management.jmxremote.local.only
have been observed to be required if the server is running in a virtualized setup based on VirtualBox&Vagrant.

Client configuration

Create a connection:
Host: 10.00.00.145
Port 7091
Open the JMX Console or the MBean Browser.

Unsecure JConsole setup

This setup can be used to access a remote JVM running WebLogic using JConsole as the JMX client. The setup is unsecure, i.e. SSL and authentication is disabled. The WebLogic server is assumed to listen on listen address 10.00.00.145.

Server configuration

Add these properties to the WebLogic start arguments:
-Djavax.management.builder.initial=weblogic.management.jmx.mbeanserver.WLSMBeanServerBuilder -Dcom.sun.management.jmxremote.authenticate=false 
-Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=7091 -Dcom.sun.management.jmxremote.rmi.port=7091 -Djava.rmi.server.hostname=10.00.00.145 -Dcom.sun.management.jmxremote.local.only=false

Client configuration

JConsole is configured to use the WebLogic implementation of the network protocols using the jmx.remote.protocol.provider.pkgs system argument. This requires that the libraries wljmxclient.jar and wlclient.jar are located in the WL_HOME directory. Here's a Windows script to start JConsole with the WebLogic network protocol implementations:
set JAVA_HOME=C:\Program Files\Java\jdk1.7.0_45 
set WL_HOME=C:\work\jconsole 
"%JAVA_HOME%\bin\jconsole.exe" -J-Djava.class.path="%JAVA_HOME%\lib\jconsole.jar;%WL_HOME%\server\lib\wljmxclient.jar" -J-Djmx.remote.protocol.provider.pkgs=weblogic.management.remote -debug
Access WebLogic using the URL: 10.00.00.145:7091
Requires no user credentials since authentication is disabled.

Unsecure JConsole setup with T3

This is a variation of the previous setup. It allows access to the JVM running the WebLogic administration server using the JConsole as the JMX client. The server is accessed using the T3 protocol. Authentication is required with a WebLogic user with administrative privileges. The WebLogic server is assumed to listen on 10.00.00.144:7001

Server configuration

Same server setup as in the previous setup.

Client configuration

JConsole is configured to use the WebLogic implementation of the T3 protocol using the jmx.remote.protocol.provider.pkgs system argument. This requires that the WL_HOME directory to point to the WebLogic home directory. Here's a Windows script to start JConsole with the WebLogic T3 protocol implementation:
set JAVA_HOME=C:\Program Files\Java\jdk1.7.0_45
set WL_HOME=C:\Oracle\Middleware\wlserver
"%JAVA_HOME%\bin\jconsole.exe" -J-Djava.class.path="%JAVA_HOME%\lib\jconsole.jar;%WL_HOME%\server\lib\wljmxclient.jar" -J-Djmx.remote.protocol.provider.pkgs=weblogic.management.remote -debug
Access WebLogic using the URL: service:jmx:t3://10.00.00.144:7001/jndi/weblogic.management.mbeanservers.edit
Requires user credentials despite authentication being disabled.

I can get JDeveloper's external tools capability to do the lifting.

Very simple this. Just choose Tools > External Tools from the menu and press New.  Then run through the wizard:

1. Type is External Program
2. Program Options > Program Executable = jconsole
3. Program Options > Arguments =  (All on one line of course, reformatted here for readability)

 -J-Djava.class.path=${java.path}/lib/jconsole.jar:${java.path}/lib/tools.jar: ${prop:name=weblogic.home}/lib/wljmxclient.jar -J-Djmx.remote.protocol.provider.pkgs=weblogic.management.remote 

And that's it.  JDeveloper helpfully substitutes both the Java location and the WebLogic location for us saving all that hunting around.  

Tip: ${java.path} not expanding correctly (or rather expanding to null). Before running the tool command with the macro make sure that you have a project open and selected as the actual java path is obtained in the context of the JRE version used by the selected project. (Project Properties > Libraries and Classpath > Java SE Version).

Sample Weblogic JMX Client

Our goal is to implement a simple WebLogic JMX client that can be used as a starting point to write any WebLogic JMX client. We will look at the client's code. The classpath used to compile and run the code. The different WebLogic MBeanServers and associated JMX URI's the JMX Client can connect to.

Let's directly dive in the client's code below:

= = = = = = = = = = = = = = = = = = =

package blog.wls.jmx.client;

import javax.management.MBeanServerConnection;

import javax.management.ObjectName;

import javax.management.remote.JMXConnector;

import javax.management.remote.JMXServiceURL;

import javax.management.remote.JMXConnectorFactory;

import java.util.Hashtable;

import java.util.Set;

public class JMXClient {

public static void main(String[] args) throws Exception {

        JMXConnector jmxCon = null;

        try {

            JMXServiceURL serviceUrl = new JMXServiceURL("service:jmx:iiop://127.0.0.1:7001/jndi/weblogic.management.mbeanservers.runtime");

            System.out.println("Connecting to: " + serviceUrl);

            Hashtable env = new Hashtable();

            env.put(JMXConnectorFactory.PROTOCOL_PROVIDER_PACKAGES,"weblogic.management.remote");

            env.put(javax.naming.Context.SECURITY_PRINCIPAL, "weblogic");

            env.put(javax.naming.Context.SECURITY_CREDENTIALS, "welcome1");

            jmxCon = JMXConnectorFactory.newJMXConnector(serviceUrl, env);

            jmxCon.connect();

            MBeanServerConnection con = jmxCon.getMBeanServerConnection();

            Set<ObjectName> mbeans = con.queryNames(null, null);

            for (ObjectName mbeanName : mbeans) {

                System.out.println(mbeanName);

            }

        }

        finally {

            if (jmxCon != null)

                jmxCon.close();

        }

    }

}

= = = = = = = = = = = = = = = = = = =

Let's take a quick look at the above code.

JMXServiceURL serviceUrl = new JMXServiceURL("service:jmx:iiop://127.0.0.1:7001/jndi/weblogic.management.mbeanservers.runtime");

The JMX service URI identifies the following:

• The protocol used to communicate with the remote WebLogic process. "iiop" in the above example. WebLogic supports "t3", "iiop" and "rmi". We won't touch further on this topic here.
• The WebLogic process to connect to (host & port ). "127.0.0.1:7001" in the above example.The port value is available from the <WLS_INSTANCE_HOME>/config/config.xml file.
• The MBeanServer to connect to: " weblogic.management.mbeanservers.runtime". This identifies WebLogic's "Runtime" MBeanServer. This is the MBeanServer available from any WebLogic process, and that contains both WebLogic and user MBeans.
• Hashtable env = new Hashtable();
• env.put(JMXConnectorFactory.PROTOCOL_PROVIDER_PACKAGES,"weblogic.management.remote");
• env.put(javax.naming.Context.SECURITY_PRINCIPAL, "weblogic");
• env.put(javax.naming.Context.SECURITY_CREDENTIALS, "welcome1");

The above code specifies properties associated with the connection. "weblogic.management.remote" identifies the package implementing WebLogic's client side connector code. For instance the 't3' or 'iiop' client connector. "weblogic" and "welcome1" are the login/password associated with the user originating the JMX connection.

In the rest of the code we establish the JMX connection, and then retrieve the ObjectNames for all the MBeans registered in the "Runtime" MBeanServer. The ObjectNames are then displayed on the console. We also make sure we release the connection in a finally block to avoid leaking any resource.

The code doesn't contain any WebLogic specific classes, and can be compiled as follow:

$ javac -d . blog/wls/jmx/client/JMXClient.java

Executing the code requires to add WebLogic client side classes to the execution ClassPath. We need to include the WebLogic JMX connector client side classes. Remember we specified those in the environment properties passed to the JMXConnectorFactory. Below is the command I used:

$ java -classpath .:$WL_HOME/server/lib/wljmxclient.jar blog/wls/jmx/client/JMXClient

WL_HOME points to the WebLogic binary install.

The above uses WebLogic's thin client classes. In some cases when you access server side classes, or you want to take full advantage of the t3 protocol, you are required to use WebLogic's thick client classes. To do so, you first need to build wlfullclient.jar as follow:

cd $WL_HOME/server/lib

java -jar wljarbuilder.jar

You can then execute the JMX Client as follow:

$ java -classpath .:$WL_HOME/server/lib/wlfullclient.jar  blog/wls/jmx/client/JMXClient

You can also bundle wlfulclient.jar with your client code. When using the thin client you cannot just bundle wljmxclient.jar with you client code, as this jar references other jars as part of its manifest Class-Path entry. At the time of this writing, you also need to bundle wlclient.jar

At this point we are now able to connect to WebLogic's "Runtime" MBeanServer. Each WebLogic process contains a "Runtime" MBeanServer in which local WebLogic MBeans and user-defined application MBeans are registered. WebLogic also offers two other MBeanServers that are only available from the Domain "AdminServer" process:

• The "Domain Runtime" MBeanServer. It aggregates the MBeans registered on the domain's "Runtime" MBeanServers. So as long as a managed server is up, its MBeans can be accessed through the "Domain Runtime" MBeanServer. To connect to that MBeanServer, just use the following JMX URI:
• service:jmx:iiop://127.0.0.1:7002/jndi/weblogic.management.mbeanservers.domainruntime
• "7002" is the iiop port for my "AdminServer" install. Replace with your AdminServer's port as previously explained. Remember; only the AdminServer process runs the "Domain Runtime" MBeanServer.
• The "Edit" MBeanServer. It contains the WebLogic Config MBeans that are used to configure the Domain. To connect to that MBeanServer, just use the following JMX URI:
• service:jmx:iiop://127.0.0.1:7002/jndi/weblogic.management.mbeanservers.edit 
• "7002" is the iiop port for my "AdminServer" install. Replace with your AdminServer's port as previously explained. Remember; only the AdminServer process runs the "Domain Runtime" MBeanServer.

Note: One can also use "t3" or "rmi" as protocol in place of "iiop"

The rest of the client's code remains unchanged. Only the JMX URI needs to be changed to connect to different MBeanServers.

References:

Weblogic 12C