Monitoring application performance and stuff you can do with NetBeans and VisualVM

    "Patience is a virtue" – probably not applicable when it comes to working with enterprise web applications, where the average web/mobile user expects pages to load in two seconds or less.  The sheer complexity of IT environments handling high transaction volumes and complexity is the norm, and without application performance monitoring, it will be impossible to deliver consistent service levels.

    As enterprise Java technology becomes more and more pervasive, understanding the behavior of the garbage collector becomes extremely important to monitor/manage application performance. Garbage Collection (GC) involves traversing Java heap spaces where application objects are allocated. So, it is important to know how frequently garbage collections are occurring and the time it takes for JVM to free memory no longer utilized by application logic. Heavy or excessive garbage collection will often show up as high CPU (the threads identified in high CPU will be the GC Threads).

    The amount of time that garbage collection takes is related to the number of live objects in the heap space and not necessarily related to the size of a given Java heap size. The popular in-built or open source tools one can leverage to optimize the JVM for throughput and responsiveness are VisualVM (included with JDK 6 update 7 and later) and NetBeans Profiler.

Here is a list of performance monitoring/profiling stuff that can be performed using VisualVM and NetBeans:

1. Capture sophisticated heap profiles using NetBeans profiler. Heap profiling is beneficial for observing frequent garbage collection and provides information about the memory allocation footprint of an application
2. NetBeans CPU Profiling using bytecode instrumentation to profile the performance of an application
3. Memory Profiling using VisualVM to analyze the memory usage of an application 
4. View live heap profiling results and traverse object references using NetBeans HeapWalker
5. Visually observe garbage collector behavior using VisualVM plugin "VisualGC"
6. Monitor surviving generations (or object age); an increasing number over a period of time can be a strong indicator of a memory leak 
7. Monitor Class loading and JIT compilation using VisualGC 
8. The number of threads that are currently active within your application, along with a drilldown of what type of threads that are in use 
9. VisualVM displays real-time, high-level data on thread activity, and thread dumps are very convenient to help diagnose a number of issues such as deadlocks or application hangs
10. Analyze code fragment performance and identifying CPU bottlenecks using NetBeans

Why Distributed Caching Matters ...

     My online shopping experience is limited to electronics and books, and the web site that provides good, fast and secure user experience invariably becomes my vendor of choice. For many web applications, consumers are likely to access the same data many, many times over the course of a day, and they all want fast concurrent access to, say item prices and other information. Behind the scenes, web and grid computing applications may repeatedly retrieve popular data, such as product descriptions, or they may create application-wide data, such as game scores, schedules, or interim stock trading results, all of which have to be accessible to all application servers deployed in a data grid. So, from a business perspective, it is important to recognize that there is an adverse impact on revenue when web page loads slowly and access to information is frustrated.

     Challenges: A rising necessity to share more users’ session data between different Web applications, domains or application servers is a major challenge in designing distributed architecture. With the decline in exponential growth of CPU speed over the past few years, the onus is on the developers to wring out every microsecond of latency and to maximize application throughput. So, the paramount need for deploying high-performant web environments, is to move away from the methodology of having applications query the database or metadata store directly each time data is required to be retrieved, updated or passed around, to one where data lives closer to the application tier. This is where data grid solutions come in, where its innovations in distributed caching make their mark.

     Business Solution:  The main goal of Distributed Caching (or Data Grid) is to provide as much data as possible from memory on every grid node and to ensure data coherency (distributed transactions). By reliably staging application data in memory within the compute grid's memory, data is simultaneously available to all compute nodes with very low latency. This facilitates distributing application’s workloads and the result is many fewer round trips to the database and faster access to data at in-memory speeds.

     Milliseconds translate to significant dollars spent or saved and distributed caching reduces not only the cost but also the risk as compared to scaling up lower data-tiers.For web applications, the in-memory data grid solution enables organizations to economically address data demand usecases involving repetitive reads/writes and session state management. In the era of cloud computing, caching (access to in-memory data) plays a pivotal role in the design of distributed systems.

      Data Grid/Cache Vendors:  Both commercial and open source offerings are available. JBoss Cache, GridGain and EhCache are some of the popular Java open source software for processing in-memory data. On the commercial side, for example, Oracle Coherence lets session states be managed in a variety of caching topologies and enables session data to be stored outside of Java EE application servers. This means that application server heap space is freed up and servers can restart without session data loss.

     Browsing through the product offerings will generally give you an idea of the advantages and solutions readily available for leveraging important application performance gain, and why distributed caching matters.

A look at Android Architecture

Google’ Android documentation (http://developer.android.com/guide/basics/what-is-android.html) has come along pretty good. I now see that there is a cool Android Architecture diagram that shows the major components of the Android OS.

Summarizing the 10 notable aspects of the architecture that every developer should be aware of:

  1. Android is built on portable Linux platform that brings a level of hardware abstraction

  2. Android applications run as separate Linux processes with security delegated to the underlying Linux system
  3. The native libraries are C/C++ libraries and provides necessary services to the Android application layer

  4. Bionic standard C library is rewritten to make it license friendly and purpose-built for tiny, battery-powered devices

  5. Includes Apache Harmony, a modular Java runtime with class libraries

  6. Relies on Linux core kernel for memory management, power management, and networking features

  7. Java VM was replaced with the Dalvik Virtual Machine to focus strictly on mobile devices and avoid licensing

  8.  Dalvik VM executes files in the Dalvik Executable (.dex) format which is optimized for minimal memory footprint

  9.  Includes SQLite, a full featured relational database engine available to all applications

  10.  3D graphics is based on the OpenGL ES 1.0 specification

Android SDKs - Cupcake to Ice Cream Sandwich

My journey with mobile’ leading development platform, Android started back in early 2011 with my first Android app (weekend project) in the marketplace - ‘tennis grand-slam champions’. Ever since, I am keeping a tab on the Android platform and following its feature offerings as time permits.

In the mobile development world, the release cycles are considerably shorter than the traditional enterprise application development. Android’ first stable version ‘Cupcake’ was released in April, 2009 and within couple of years; we have seen many features evolved in ‘Ice Cream Sandwich – Android SDK 4’. Éclair (v2) was better optimized than Donut (v1.6) to take advantage of the hardware for faster processing. Speed improvement was brought in with the integration of Chrome V8 JavaScript engine and JIT optimization in Froyo (v2.2); bluetooth and wi-fi made its way into the networking layer.

Gingerbread (v2.3) was a major release and a considerable number of new features were introduced, that include new UI themes, redesigned keyboards, new copy and paste functionality, improved power management, NFC (Near Field Communication), support for VoIP/SIP calls, new Camera application for accessing multiple cameras and supports extra-large screens. Ice Cream Sandwich (v4) is much sleeker than Gingerbread and Google has designed a font completely from the ground up, which looks exceptional. Another major change is that a lot of the SQL handling is moved from the native to the Java layer.  I am sure there are lots of articles on the web that will give a quick comparison of the feature-sets that were bundled in every SDK release.

           iOS leads the way with quality apps, Android leads in market share; Windows phone has not caught up yet with the mass adoption as its leading competitors. Android and iOS — each offers its own clear take on the mobile OS experience, and I personally feel that the consumers and business users stand to benefit from this healthy competition. Staying tuned on what Android comes up with next …………………

Java Management Extensions and Weblogic Server MBeans

The best part I like about Java Management Extensions (JMX) framework is that it allows developers of applications, services, or devices to make their products manageable in a standard way without having to understand or invest in complex management systems.

JMX has been around as a core JEE API for a long time and is the foundation for everything you can do to manage the Oracle Weblogic Server. The JMX architecture in the context of Weblogic Server as I see it is shown in figure below:

Application components designed with their management interface in mind can typically be written as MBeans. WLS MBeans are 3 types - Domain, Server and Application level MBeans. You can instrument your applications deployed to WLS by providing one or more management beans. For example, the DomainRuntimeMBean provides a federated view of all of the running JVMs in a Weblogic administrative domain. All Weblogic Server MBeans can be organized into one of the following general types - Runtime MBeans or Configuration MBeans based on whether the MBean monitors the runtime state of a server or configures servers and JMX manageable resources.

WLS MBean Server acts as a container for MBeans. The MBean servers available in a Weblogic domain are:

1. Runtime MBeanServer: is the MBeanServer available from any Weblogic process, and contains both Weblogic and user MBeans. Each server in the domain hosts an instance of this MBean server.
2. Domain MBeanServer: This MBeanServer provides federated access to MBeans for domain-wide services. Collocated with the WLS domain admin server process, this server  also acts as a single point of access for MBeans that reside on Managed Servers.  So, as long as a managed server is up, its MBeans can be accessed through the "Domain Runtime" MBeanServer.
3. Edit MBeanServer: Collocated with the WLS domain admin server process, it provides access to pending configuration MBeans and operations that control the configuration of a Weblogic Server domain. No application MBeans can be registered in this MBeanServer. 

The JMX API enables to perform remote management of resources by using JMX technology-based connectors (JMX connectors). Adaptors and connectors make all MBean server operations available to a remote management application. Fusion Applications Control (or commonly known as EM Console) is the primary administration management interface. Developers can programmatically monitor Weblogic Server by using the JMX interface directly in a Java program or by writing scripts using a tool such as WLST.

Here is a good blog article on managing WLS using Jconsole: http://blogs.oracle.com/WebLogicServer/entry/managing_weblogic_servers_with

Please refer the Oracle Weblogic Server documentation for the latest on WLS-MBeans.

An evaluation of Content Management Software Solutions

Content Management Solutions (CMS) have evolved quite a lot in the recent years. There are so many available, all with different features, it can easily be an overwhelming task for any business or a web designer to choose the right solution-fit for their project.

I took upon myself to evaluate the CMS solutions that will be ideal for a small organization, like church for example. From my research of web articles, blogs and support forums the message I get is that a fully featured CMS, that is stable with a clear roadmap for future development/feature enhancements and a commitment to support and enhance the performance aspect is probably the best way to go. Here is the tabulated evaluation matrix of some of the major elements - cost, feature set and hosting options; that were important to me. 

              Table: Content Management Software Evaluation (January, 2012)

Oracle ESS LifeCycle

My first post on Oracle ESS briefly introduced the high level functional aspects of the framework. Here, I will discuss the ESS request execution processing stages - a key concept to understand when working with ESS.

The lifecycle of an ESS Job Execution begins when the client makes a request for the job submission and ends when the server responds with the execution state. The lifecycle consists of three main phases: PreProcess, Execute and PostProcess.

During the PreProcess phase, several actions can take place. For instance, setting request properties for work/output/log directories, verficiation of ESS request file directory, creating application session (a lightweight session object in the Fusion Apps context), database connection initialization, using requested NLS settings, and loading environment properties file for spawned execution are some of the purposes of PreProcessing. PreProcessor will run prior to every job request execution, and when a request is restarted after pausing.  

Executable phase is where the actual job program runs. The program logic can be implemented in any of the supported ESS JobTypes i.e. Java, C, PL/SQL, Host, Perl, SQL, BIP etc. For example, Java job logic runs in the context of the J2ee ESS application hosting the job metadata. On the other hand, PL/SQL program logic written as procedures are executed as an Oracle RDBMS Scheduler job procedure (owned by ESS runtime schema) after the ESS wrapper procedure does some initialization/set up work. In general, for any job type the post-process handler is called only after the ESS mid-tier is notified the request executable has ended.

PostProcessing is performed only if the overall processing of the ESS request gets as far as the executable stage and it completes as success or warning, Post-processing is not done if the pre-processing or executable fails as error, or if the request is cancelled prior to post-processing. The main purpose of PostProcessor is to carry out general cleanup tasks and other actions to reflect the final state of the request before the job request can be deemed as fully complete. PostProcessing runs after the job has completed  its execution to perform defined actions, such as Notification (using Oracle UMS) and storing the request log/output files to the Content Repository (i.e. Oracle UCM Server).

Finally, here are some important points to remember w.r.t the ESS request lifecycle:

• If the pre-process handler returns an error status, the business logic for that request will not be executed for that execution attempt. The request will normally transition to an ERROR state from which it may be retried if RETRIES are configured. It is important to note that pre-processor is not guaranteed to run in the same thread as the executable.
• If post-processing is attempted and returns as error, the overall request state is set to WARNING. Post Processing will only occur after the request has gone to a terminal state.

Please refer the Oracle ESS Developers Guide for more details and latest information.