Re-wire Oracle SOA Datasource to use Active GridLink for RAC

Starting 11gR1 PS5 (11.1.1.6), Oracle SOA Suite platform is enhanced to support Active GridLink for Real Application Clusters (RAC).

WebLogic Server Active GridLink (AGL) for RAC provides the best available support for the RAC features in Oracle Database 11g. The generic data source is the implementation for single database access. The Multi-data source is the native WebLogic middle tier implementation for RAC integration, which does not leverage Oracle Notification Service (ONS).

The GridLink data source is the new Active GridLink implementation and includes the features of generic data sources, taking advantage of Oracle RAC supporting Fast Connection Failover (FCF), Runtime Connection Load-Balancing (RCLB), and XA Affinities. A GridLink data source is created for JDBC connectivity between WebLogic Server and a service targeted to an Oracle RAC cluster. It uses the Oracle Notification Service (ONS) to adaptively respond to state changes in an Oracle RAC instance.

Here are couple of good resources that can help you understand the AGL concepts in more detail:
- What is Active GridLink
- Benefits of AGL data sources

Basically, a data source configuration is always an Active GridLink (AGL) data source configuration if FanEnabled=true or the OnsNodeList is non-null.

Here is a modified version of a python script (borrowed from my colleague), that you can use to rewire the SOA data sources to use Active GridLink for RAC databases. Before executing the script, remember to shutdown all the WLS SOA domain servers; its an offline wlst script.

Creating logical partition on Exalogic X3-2 ECHO vServer using LVM

I switched to using the recently announced Exalogic X3-2 Echo (2.0.6.0.0) guest template for packaging the application software VM. However, on trying to increase the disk space as in the Navstar (2.0.4.0.0) template version using the modifyjeos tool (version 1.1.0-17), I ran into the the following error:

# modifyjeos -f System.img -P App.img 6144 /u01 -S 2048 -n EL_ECHO_MYAPP_2.0_VM_TEMPLATE
Starting VM image reconfiguration...
ERROR: Unknow filesystem.

No detailed messages other than the single line below were logged to /var/log/modifyjeos.log
       ...modifyjeos version 1.1.0-17 starting
 
I later learnt that modifyjeos does not support LVM and this is one of the new feature in Exalogic ECHO (2.0.6.0.0) release. Using Logical Volume Manager (LVM) is recommended to increase/create the virtual disk space. Here are the steps followed to prepare the Exalogic Echo guest VM instance for creating a logical partition to host the application software.


Refer the official Exalogic X3-2 Elastic Cloud Administrator's Guide "Managing LVM Partitions on Guest vServers" for more details on this subject

Configuring SOA Distributed Destination Topics using WLST

The organization of topics and the granularity of subscriptions to them is an important part of a publish/subscribe application’s architecture. A message delivered to a topic is distributed to all consumers that are subscribed to that topic. Ensuring that a clustered application gets exactly one copy of each message, and parallel processing of topics messages may require significant additional configuration, coding, and message hops. To address such limitations, WebLogic JMS provides load balancing and failover for physical JMS destination members (queues or topics) through its "Distributed Destinations" that applications can leverage.

In a typical SOA Suite production cluster deployment, load balancing the JMS topic messages against the SOA composites will be a common requirement. It is to ensure that messages are evenly distributed (load balanced) to SOA cluster members, and to avoid duplication in some scenarios. This can be achieved by configuring the SOA JMS Topic to use a "Distributed Destination" setting the appropriate forwarding policy and other advanced parameters.

Here is a sample WLST offline script:


Note that there in no support to define the UniformDistributed Queue/Topic in weblogic application extension templates' file config/jms/<%jms-module%>-jms.xml (as of the latest SOA PS6 release). Refer the "Fusion Middleware Programming JMS for Oracle WebLogic Server" guide for more details on WLS JMS Distributed Destinations.

Creating Application VM Templates for Exalogic X3-2

 It is a well known fact that one of the primary goal for applications virtualization is to provide enterprise users with a consistent and reliable experience through accelerated deployment. For complex enterprise-class applications like Oracle SOA Suite (or IdM/any other business software), virtualization facilitates rapid provisioning and delivery of optimized application configuration to meet security, high availability and performance standards.

Exalogic Elastic Cloud X3-2 is Oracle' flagship engineered system for running business applications in a highly performant environment. The Exalogic X3-2 software includes a optimized Exabus­ enabled Type I Xen hypervisor i.e. Oracle VM Server, which allows to host virtual servers running Oracle Linux. In this post, I will share my experience creating an application template for deploying on Exalogic.

The application template creation procedure for Exalogic can be split into 2 broad categories:

1. Preparing the Oracle VM Server environment
2. Creating the application VM template

Note that access to an Exalogic instance is not required for the purpose of creating the application templates.You will require the following software though; download from eDelivery.oracle.com

• "Oracle Linux JeOS for Building Oracle VM Templates Media Pack v13 for x86_64 (64 bit)"
• "Oracle Exalogic 2.0.4.0.0 Base Guest Template" for Exalogic Linux x86-64 (64 bit) V35512-01  (requires OVM Server 3.x or later)

First, the steps to configure the Oracle VM Server:

1) Connect to Oracle VM Server 3.x
- it is recommended to use VM Server 3.x installation or later
- ssh username@hostname

2) Configure xend-config.sxp on the OVM Server (OVS) to use bridge networking
- the OVS system must be setup so that we can attach virtual machines to the external network. This is done by creating a virtual switch within dom0 (domain zero i.e. management domain in a OVS env responsible for hardware detection) that takes packets from the virtual machines and forwards them onto the physical network so they can see the internet and other machines on your network. The piece of software we use to do this is called the Linux bridge and its core components reside inside the Linux kernel. In this case the “bridge” is effectively our virtual switch.

# vi /etc/xen/xend-config.sxp
(network-script network-bridge)
(vif-script vif-bridge)

3) Restart OVM Server
# [root@ovmserver BASE]# reboot

4) Configure your OVM Server operating system to use a public Yum repository
- download and install any prerequisite Oracle Enterprise Linux packages not installed on your computer from the public Oracle Yum repository

--- Oracle Linux 4, Update 6 or Newer
# cd /etc/yum.repos.d
# mv Oracle-Base.repo Oracle-Base.repo.disabled
# wget http://public-yum.oracle.com/public-yum-el4.repo

--- Oracle Linux 5
# cd /etc/yum.repos.d
# wget http://public-yum.oracle.com/public-yum-el5.repo

--- Oracle Linux 6
# cd /etc/yum.repos.d
# wget http://public-yum.oracle.com/public-yum-ol6.repo

5) Install the JeOS dependent packages with the following command:
# yum install createrepo

6) Install "Oracle Linux JeOS for Building Oracle VM Templates Media Pack v13 for x86_64 (64 bit)"
     - JeOS is particularly useful and handy when you need to re-size the VM template Root Disk Image
- download the zip files for JeOS base and JeOS base image for required operating system to your OVS system in local directory e.g /u01/JeOS.
        (you will have following rpms in your local directory after uncompressing the zip files)
ovm-modify-jeos-1.1.0-17.el5.noarch.rpm
ovm-template-config-1.1.0-8.el5.noarch.rpm
ovm-el5u8-xvm-jeos-1.1.0-2.el5.x86_64.rpm

- install these rpms

# rpm -ivh ovm-modify-jeos-1.1.0-17.el5.noarch.rpm
# rpm -ivh ovm-template-config-1.1.0-8.el5.noarch.rpm
# rpm -ivh ovm-el5u8-xvm-jeos-1.1.0-2.el5.x86_64.rpm

7) At the end of this step, you will have the base OVM Server environment ready to create the virtual machines (or virtual host servers in Exalogic terminology).

Next, the steps to create the application VM template. Three main phases:

1. Creating VM using Exalogic Base Guest template  
2. Installing and configuring the required application software
3. Preparing the VM template TAR

Phase1

1. From eDelivery.oracle.com download "Oracle Exalogic 2.0.4.0.0 Base Guest Template" for Exalogic Linux x86-64 (64 bit) V35512-01 

2. Extract the Exalogic Guest Template contents to a directory location (say /Exalogic/BASE) on your Oracle VM Server 3.x installation

    # ssh username@ovshostname
     # mkdir -p /Exalogic/BASE
      
    # pwd
   /Exalogic/BASE
   
    BASE]# ls
    System.img  vm.cfg 
    
3. Change the path to the virtual disk in the vm.cfg file, as follows:
        disk = ['file:/OVS/seed_pool/OVM_simple_/System.img,hda,w'] to disk = ['file:/Exalogic/BASE/System.img,hda,w']  

4. In the Exalogic Guest template vm.cfg file, set vif = ['bridge=bond0'], where bond0 is the bridging Ethernet adapter on OVS. It is usually called 'xenbr0'

5. Start the VM with '-c' flag, and press Ctrl+D when it gets to the OVM template configuration stage
# cd /Exalogic/BASE
# xm create vm.cfg -c  
       (alternatively, you can skip the '-c' flag and connect to the VM using the command: xm console <vm-
         domain-Id>)

6. Login as 'root' to the created VM host (lets call it App-VM)
      - the default root password is 'ovsroot'

7. Add the network script ifcfg-eth0 in the App-VM 
- if eth0 already exists, use a different one
      - there are two options, DHCP, and Static IP Address. If you choose to make use of DHCP to automatically assign IP addresses to servers or virtual machines within a network, you must ensure that a DHCP server is set up and available within your VM environment

 # vi /etc/sysconfig/network-scripts/ifcfg-eth0
-----------------------------------------
DEVICE=eth0
            BOOTPROTO=static
            ONBOOT=yes
            IPADDR=10.200.250.300
            NETMASK=255.255.252.0
            GATEWAY=10.200.220.320
            DELAY=0

8. Configure the Network
- Edit the host name in the App-VM /etc/sysconfig/network file
- Edit the host name in /etc/hosts/
- Update /etc/resolv.conf as well

9. Turn off the initial configuration of the OVM template
# vi /etc/sysconfig/ovmd
set INITIAL_CONFIG=no

10. Use ifup to bring eth0 online or reboot VM
# /sbin/ifup eth0

11. Restart the networking service
# /etc/init.d/network restart

Phase2

1. Copy the necessary application software to the App-VM environment created in phase1
- you may need to configure the required JDK as well if your application depends on it. Exalogic guest template OS does not package any Java runtime
2. Install and configure the business applications in the App-VM

Phase3

1. Prepare to create a Template from the running App-VM
- Update ovmd script

# vi /etc/sysconfig/ovmd
Set the "INITIAL_CONFIG" parameter to "yes"
INITIAL_CONFIG=yes

2. Cleanup App-VM host specific OS-network level configuration
- Remove DNS information
    - Remove extraneous SSH information
    - Cleanup network file
# cd /etc/sysconfig
# sed -i '/^GATEWAY/d' network
    - Cleanup hosts file
    - Remove networking information
# cd /etc/sysconfig/network-scripts
# rm -f ifcfg-*eth*
# rm -f ifcfg*ib*
# rm -f ifcfg*bond*
     
# cd /etc/sysconfig/networking/profiles/default
# rm -f ifcfg-*eth*
# rm -f ifcfg*ib*
# rm -f ifcfg*bond*
    - Remove logs
    - Remove kernel messages
# cd /var/log
# rm -f dmesg
# dmesg -c
    - Remove history
# rm -f /root/.bash_history
# history -c

(Note: execute steps 3 through 5 below from the OVM Server host)

3. Shutdown App-VM

# xm list
# xm shutdown <VM_ID>

4. Compact the App-VM template vDisks (optional)

# [root@ovmserver BASE] modifyjeos -f System.img -zero-out-all 

5. Package the vServer with its virtual disk

# tar -pczvf <app_template_name>.tgz <el_base_template_directory>

The TAR VM template <app_template_name>.tgz is now ready to be uploaded/deployed to the Exalogic Elastic Cloud

Hope this helps to get a general idea on what it takes to create application templates on Linux for Exalogic X3-2 system. Feel free to share your experience/comments/tips .

Oracle Enterprise Scheduler (ESS) Notification

Automated electronic notification provides a way to easily alert the business application users of the change in the system status and to stay organized, informed and in-touch 24/7. Oracle ESS provides the ability to notify users of the status of submitted jobs, and the recipient information can be specified via the user interface 'Notification' tab for submitting job requests. Notifications will be sent upon the successful completion of a job, or when a job completes in an error or warning state.

ESS delivers the notifications to the relevant users using the Oracle SOA Suite User Messaging Service (UMS). The UMS Server orchestrates message flows and will route outbound messages from the ESS client application to the appropriate UMS driver. UMS Drivers support widely-adopted messaging protocols and can be integrated with existing infrastructures such as a corporate email servers or XMPP (Jabber) servers.

The ESS Notification post processing action is used to notify users as to the completion status of the request. The Subject and Body of these notifications are preset to standard messages so users will  always get the same information - as to whether a request has succeeded or failed, regardless which job was run. Also, ESS does not force these notifications to be sent via Email - a user may configure their UMS preferences such that their default channel is SMS (or text message), Instant Messaging, etc.

There are two configuration settings that affect how ESS interacts with UMS. These are:
  • NotificationServiceURL
  • SAMLTokenPolicyURI
 
The NotificationServiceURL is of the form http://<host>:<port> and indicates the server where UMS is running. If this is not set correctly, or if the http:// prefix is missing, Notification will not succeed. The SAMLTokenPolicyURI is used if such security has been configured for the UMS server. Notification is designed to handle the case of a null SAMLTokenPolicyURI and continue accordingly in the assumption that the null value indicates no SAMLTokenPolicy is required.

These notification settings are part of the ESSAPP connections configuration. ESS specific connections.xml will contain token expressions which will get materialized with concrete data obtained from the adf-domain-config.xml at runtime. The domain wide configuration file adf-domain-config.xml will maintain all the ADF connection parameters in Metadata Services(MDS) repository, and provides a centralized way to update all connections information used in Fusion Applications.

ESS connections xml attribute values can be modified using Mbeans. The MBeans save the changes in MDS, and the values in MDS override the values in the actual connections.xml file. ESS script 'essAdfDomainConfig.sh' implemented using WLST provides command line options to modify the ESS connections.xml parameters created in the MDS repository (if there is a need to modify the UMS server url properties).

Some important points to note when submitting the job using the ESS Request Submission user interface:
   1. Check User preference for the user/recipient is setup with notification
   2. Make sure that submitter is specifying the username (not email address) of the recipient user that is in ldap in the UI notification tab.
   3. Ensure that ESS hosting application where the job is running includes oracle.sdp.client library reference in weblogic-application.xml.

Please refer the official Oracle documentation for more details on ESS notification, UMS user preferences configuration and troubleshooting.

Configuring Oracle SOA TLOGS JDBC Persistence Store using WLST Offline

In this post, we will look at configuring Oracle SOA to use TLog JDBC persistent stores using WLST (offline).

First, some info on TLogs. Weblogic Server (WLS) holds records of the state of in-flight transactions that are marked to be committed, in a 'TLOG' (Transaction Log) persistent store. The persistent store can either be files on a file-system or a table in database. When an instance is restarted, the TLog files are used to perform the second step of a two-phase commit on a transaction that was in progress.

So, why is this persistent-store configuration important? Configuring a WLS TLOG store to persist transaction logs to a database provides the following benefits (covered in the WLS documentation):

• Leverages replication and High Availability characteristics of the underlying database.
• Simplifies disaster recovery by allowing the easy synchronization of the state of the database and TLOGs.
• Improved Transaction Recovery service migration as the transaction logs to do not need to be migrated (copied) to a new location.

Now onto the sample WLST OFFLINE python script to see how the persistent store configuration can be accomplished. At a high level, the SOA TLOG configuration script enables JDBC Persistent Store for each Weblogic SOA managed server that belongs to the SOA Cluster. The script was verified in a 11g SOA PS6 env and will give you a general idea on what commands need to be executed.

Oracle SOA Suite JMS Deployment Topology

Java Message Service (JMS) messaging architecture relies on a data centric approach for application integration. The Oracle Service Oriented Architecture (SOA) core infrastructure engine (or ESB) implements messaging to enable services to be integrated in a message-based paradigm – both synchronous and asynchronous styles. Hence, for those familiar or working with SOA Suite 11g, it becomes necessary to know the SOA JMS resources that are configured in Weblogic Server.

So, when it comes to understanding JMS deployment in a Weblogic environment, it helps to be aware of the major components of the WebLogic JMS Server architecture. JMS Servers act as management containers for JMS Queue and Topic destinations in JMS Modules, which contains configuration resources such as Queues, Topics, distributed Destinations and Connections Factories. These JMS module resources are grouped and targeted to Weblogic server resources such as JMS servers, server instances, or cluster by a mechanism called SubDeployment. Information on JMS modules' administered objects are placed in a JNDI namespace and are written to the file <product-module-name>--jms.xml in the config\jms subdirectory of the domain directory. The JMS module will be referenced in the domain's config.xml file as a <jms-system-resource> element.

Here is a representation of the JMS resources deployment from a standalone i.e. non-cluster SOA Suite 11g-PS6 deployment, that could be helpful in enhancing your understanding. The persistent store depicted can either be 'File' or a table in database.

Some points to remember for a successful JMS resources deployment and transparent access:

• All JMS server names must be unique within a domain and across all inter-operating domains 
• All JMS store (File or JDBC) names must be unique and have unique MBean names
• All user-defined JMS connection factories targeted to servers/clusters in a domain must have unique JNDI and MBean names
• Queue destinations can use the same name as other Queues on different JMS servers 
•  Topic destinations can also use the same name as other Topics on different JMS servers

Refer the official documentation for latest and greater details on Weblogic JMS architecture.