Saturday Feb 27, 2016

Orchestrate FMW Environments with Docker Compose

In my last article, I explained how the Docker Engine can be used to create Oracle Fusion Middleware environments as containers. The steps described in the article were, however, manual and had to be executed in sequence. This article is a continuation of the previous post and explains how Docker Compose can be used to script the container creation in sequence. For the purpose of this article, we will create a Oracle WebCenter Content environment again. After following the instructions in this article, you wil be able to create the entire environment using a single command.

I have used the following software versions for this article.
Ubuntu 15.10
Docker version 1.10.2, build c3959b1
Docker Compose version 1.7.0dev, build 38dd342

To install the latest Docker Engine, follow the instructions available here. To install the development build of Docker Compose, follow the instructions here and use the URL below instead.
https://dl.bintray.com/docker-compose/master/docker-compose-Linux-x86_64

Howto: Install Dev Branch for Docker Compose

Download the Dockerfiles and the Docker Compose YAML files from the GitHub repository.
cd ~
git clone --branch v1.0.1 https://github.com/justinpaulthekkan/docker-compose-fmw.git ~/build

Download the required installers to create the following directory structure. Click on the links below for more information.
Build the Oracle Database and Oracle WebCenter Content images using the commands below.
# Build the Database image
docker build --shm-size=1g -t oracledb -f ~/build/database/Dockerfile.12.1.0.2.160119 ~/build/database

# Build the WebCenter Content image
docker build --shm-size=1g -t wccontent -f ~/build/wccontent/Dockerfile.12.2.1.0.0 ~/build/wccontent

Edit the ~/build/wccontent.yaml file to view or update your services.
version: "2.0"
services:
 database:
  image: oracledb
  shm_size: 1g
  hostname: database
  user: oracle
  tty: true
  ports:
   - "1521:1521"
  command: /bin/bash -c "/usr/bin/python ~/scripts/main.py -c --start -o /u01/app/oracle -n orcl.justinpaul.com --sid orcl; /bin/bash"

 adminserver:
  image: wccontent
  shm_size: 1g
  hostname: adminserver
  user: oracle
  tty: true
  ports:
   - "7001:7001"
   - "7002:7002"
  links:
   - database
  depends_on:
   - database
  # Sometimes it takes longer for the database to be ready and this container
  # creation will fail. Update the wait_time to a value in seconds as needed.
  # Reduce the value of wait_time also if your database starts up faster.
  command: /bin/bash -c "/usr/bin/python ~/scripts/main.py --wait --wait_time 600 --create_domain -o /u01/app/oracle -f /u01/app/oracle/middleware -h /u01/data/mydomain -c database:1521:orcl -m DEV1 ucm capture wccadf ibr && /u01/data/mydomain/bin/startWebLogic.sh; /bin/bash"

 contentserver:
  image: wccontent
  shm_size: 1g
  user: oracle
  tty: true
  ports:
   - "16200:16200"
   - "17200:17200"
  links:
   - database
   - adminserver
  depends_on:
   - adminserver
  command: /bin/bash -c "/usr/bin/python ~/scripts/main.py --wait --add_servers -o /u01/app/oracle -f /u01/app/oracle/middleware -h /u01/data/mydomain -a adminserver --use_plain ucm && /u01/data/mydomain/bin/startNodeManager.sh; /bin/bash"

 inboundrefinery:
  image: wccontent
  shm_size: 1g
  user: oracle
  tty: true
  ports:
   - "16250:16250"
   - "17250:17250"
  links:
   - database
   - adminserver
  depends_on:
   - contentserver
  command: /bin/bash -c "/usr/bin/python ~/scripts/main.py --wait --add_servers -o /u01/app/oracle -f /u01/app/oracle/middleware -h /u01/data/mydomain -a adminserver --use_plain ibr && /u01/data/mydomain/bin/startNodeManager.sh; /bin/bash"

An important thing to note is that the containers do not wait until the services start completely before reporting themselves as up and running. This poses a problem as the domain cannot be created if the database is not running and the managed servers cannot be configured unless the admin server is running. I have handled this situation by making the services dependent on each other so that they start in a sequence and introducing a wait logic until the ports are open for connection. The database needs some additional wait time after the instance is started and this can be specified using the --wait_time option.

Create the containers by running the following command. You can also choose to create the containers first and start them individually using the create and start subcommands. Read the Docker Compose Command-line Reference for more information on the available subcommands.
# Create a new environment named demo1
docker-compose -p demo1 -f ~/build/wccontent.yaml -d up

# Create another environment named demo2
docker-compose -p demo2 -f ~/build/wccontent.yaml -d up

The containers are created with names in the format <project>_<servicename>_<integer>. To attach to a container to view the standard output of the command that is executing or to start a new shell on the container, use the commands shown below.
# Attach to a container to view the results of the command executed by docker compose
# docker attach demo1_<service>_1
docker attach demo1_adminserver_1

# Open a new shell on a container
# docker exec -it demo1_<service>_1 /bin/bash
docker exec -it demo1_adminserver_1 /bin/bash

Once the admin server and the nodemanager on the managed server containers have started, login to the Admin Console and start the managed servers.

The scripts used under the scenes to create install and configure the environment are available at the links below. View the README files for instructions on how to use these scripts. Use the master branch if you want the latest version of the projects.


NOTE: Please note that these instructions should be used for development and evaluation purposes only, as such they are unsupported and should not be used on production environments.

Monday Feb 22, 2016

Faster FMW Devops using Docker Containers

In this article, I would like to show how Docker (version 1.10.1) can be utilized to create Oracle Fusion Middleware environments. The idea is to create an environment for development or testing and then purge them after their use is complete. An environment would consist of a database and one or more Fusion Middleware services.

For the purpose of this article, we will create a Oracle WebCenter Content environment. We shall use a name (demo) as a prefix to the services and containers we create to identify them. To create more environments, a new name should be used.

Step 1: Download the Dockerfiles to create the base images.

git clone v1.0.0 https://github.com/justinpaulthekkan/docker-compose-fmw.git ~/build

Step 2: Download the required installers to create the following directory structure. Click on the links below for more information.

Step 3: Build the Oracle Database image.

docker build --shm-size=1g --force-rm --rm=true --no-cache -t oracledb -f ~/build/database/Dockerfile.12.1.0.2.160119 ~/build/database

Step 4: Build the Oracle WebCenter Content image.

docker build --shm-size=1g --force-rm --rm=true --no-cache -t wccontent -f ~/build/wccontent/Dockerfile.12.2.1.0.0 ~/build/wccontent

Step 4: Create a Docker network for our new environment.

docker network create demo

Step 5: Create a database container.

docker run --shm-size=1g --name=demo_database --net=demo --net-alias=database -h database -u oracle -Pdt oracledb /bin/bash -c "/usr/bin/python ~/scripts/main.py -c --start -o /u01/app/oracle -h /u01/app/oracle/product/12.1.0/dbhome_1 -n orcl.justinpaul.com --sid orcl; /bin/bash"

Step 6: Create the admin server container.

docker run --shm-size=1g --name=demo_adminserver --net=demo --net-alias=adminserver -h adminserver -u oracle -Pdt wccontent /bin/bash -c "/usr/bin/python ~/scripts/main.py --create_domain -o /u01/app/oracle -f /u01/app/oracle/middleware -h /u01/data/mydomain -c database:1521:orcl -m DEV1 ucm capture wccadf ibr; /u01/data/mydomain/bin/startWebLogic.sh"

Step 7: Create the content server container.

docker run --shm-size=1g --name=demo_contentserver --net=demo --net-alias=contentserver -h contentserver -u oracle -Pdt wccontent /bin/bash -c "/usr/bin/python ~/scripts/main.py --add_servers -o /u01/app/oracle -f /u01/app/oracle/middleware -h /u01/data/mydomain -a adminserver --use_plain ucm; /u01/data/mydomain/bin/startNodeManager.sh"

Step 8: Create the inbound refinery container.

docker run --shm-size=1g --name=demo_inboundrefinery --net=demo --net-alias=inboundrefinery -h inboundrefinery -u oracle -Pdt wccontent /bin/bash -c "/usr/bin/python ~/scripts/main.py --add_servers -o /u01/app/oracle -f /u01/app/oracle/middleware -h /u01/data/mydomain -a adminserver --use_plain ibr; /u01/data/mydomain/bin/startNodeManager.sh"

Step 9: Create the web ui container.

docker run --shm-size=1g --name=demo_adfui --net=demo --net-alias=adfui -h adfui -u oracle -Pdt wccontent /bin/bash -c "/usr/bin/python ~/scripts/main.py --add_servers -o /u01/app/oracle -f /u01/app/oracle/middleware -h /u01/data/mydomain -a adminserver --use_plain wccadf; /u01/data/mydomain/bin/startNodeManager.sh"

Step 10: Create the enterprise capture container.

docker run --shm-size=1g --name=demo_capture --net=demo --net-alias=capture -h capture -u oracle -Pdt wccontent /bin/bash -c "/usr/bin/python ~/scripts/main.py --add_servers -o /u01/app/oracle -f /u01/app/oracle/middleware -h /u01/data/mydomain -a adminserver --use_plain capture; /u01/data/mydomain/bin/startNodeManager.sh"

Step 11: Get the IP Addresses for the containers.

docker inspect --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' demo_database
docker inspect --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' demo_adminserver
docker inspect --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' demo_contentserver
docker inspect --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' demo_inboundrefinery
docker inspect --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' demo_adfui
docker inspect --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' demo_capture

Step 12: Login to the admin server and start all the managed servers. The credentials for the services are given below.

Database SYS Password: welcome1
Database SYSTEM Password: welcome1

Domain Admin Username: weblogic
Domain Admin Password: welcome1

If you would like to connect to any of the containers to view the logs or to perform any manual steps, use the command below.

docker exec [-u root|oracle] -it contaner_name

Naturally the next step is to automate the container creation using docker-compose. Please note that at the time of the writing of this article, the current version of docker-compose is v1.6.0 which does not support the shm_size option. This option will be implemented in the next release of docker-compose, v1.7.0.

The scripts used under the scenes to create install and configure the environment are available here. View the README files for instructions on how to use the scripts.

NOTE: Please note that these instructions should be used for development and evaluation purposes only, as such they are unsupported and should not be used on production environments.

Wednesday Jan 27, 2016

Install Oracle Fusion Middleware using Python

I am obsessed with Python! For the last few weeks, I have spent countless hours porting a lot of my scripts into Python. I have even re-written UNIX scripts in Python. I just can't get enough of it. I have come to love the way it manages my data structures: from sets, lists, dictionaries to even JSON. I would like to share a few scripts that can be used to install Fusion Middleware products and create and drop product schemas.You can also access my GitHub project here.

/usr/bin/python install_fmw.py [-?] -l installers_location -o oracle_home [-j jdk_home] [-f fmw_home] [--rsp_file install_response_file] [--tmp_loc tmp_location] [--os_install_group install_os_group] [wls] [bpm|soa] [wcc] [wcp] [wcs] [ohs]

/usr/bin/python create_schemas.py [-?] -f fmw_home -c db_connect_string -m db_prefix [-w password_file] [--dba_user db_admin_user] [--dba_password db_admin_password] [--dbs_password db_schema_password] [--soa_profile SMALL|MED|LARGE] [--analytics_with_partitioning N|Y] [em] [bpm|soa] [ucm] [capture] [wccadf] [portal] [pagelet portlet] [discussions] [analytics] [sites] [vs] [insights] [sc] [ss]

/usr/bin/python drop_schemas.py [-?] [--all] -f fmw_home -c db_connect_string -m db_prefix [-w password_file] [--dba_user db_admin_user] [--dba_password db_admin_password] [em] [bpm|soa] [ucm] [capture] [wccadf] [portal] [pagelet portlet] [discussions] [analytics] [sites] [vs] [insights] [sc] [ss]

NOTE: Please note that these code snippets should be used for development and testing purposes only, as such it is unsupported and should not be used on production environments.

Monday Dec 28, 2015

Docker Image for Oracle Fusion Middleware 12.2.1

This post shows you how to create a Docker Image with one or more components of Oracle Fusion Middleware 12.2.1 installed. Download the project directory from GitHub here.

I have written a Dockerfile to build an image with Oracle Fusion Middleware components. To use this Dockerfile, you will need to create the following directory structure.

mkdir ~/fmw
cd ~/fmw
mkdir installers

After the directories are setup, all the necessary installers should be downloaded. You may choose to skip the installer files for the products that you do not wish to install in your image; the Dockerfile will simply ignore the install instruction for that product.

./fmw
 - installers
  - silent.rsp (required)
  - jdk-8u66-linux-x64.gz (required)
  - fmw_12.2.1.0.0_infrastructure_Disk1_1of1.zip (required)
  - fmw_12.2.1.0.0_bpmqs_Disk1_1of2.zip (optional)
  - fmw_12.2.1.0.0_bpmqs_Disk1_2of2.zip (optional)
  - fmw_12.2.1.0.0_soaqs_Disk1_1of2.zip (optional)
  - fmw_12.2.1.0.0_soaqs_Disk1_2of2.zip (optional)
  - fmw_12.2.1.0.0_wccontent_Disk1_1of1.zip (optional)
  - fmw_12.2.1.0.0_wcportal_Disk1_1of1.zip (optional)
  - fmw_12.2.1.0.0_wcsites_Disk1_1of1.zip (optional)
  - fmw_12.2.1.0.0_ohs_linux64_Disk1_1of1.zip (optional)
 - Dockerfile.12.2.1.0.0

Create or edit the Dockerfile in your context directory. For this post, the Dockerfile is named as Dockerfile.12.2.1.0.0.

FROM oraclelinux:7
MAINTAINER Justin Paul 

ENV _SCRATCH /tmp/scratch
ENV ORA_HOME /u01/app/oracle
ENV JDK_HOME ${ORA_HOME}/jdk
ENV FMW_HOME ${ORA_HOME}/middleware

COPY installers ${_SCRATCH}/

RUN yum update -y -q && \
 yum install -y -q binutils compat-libcap1 compat-libstdc++-33 \
 compat-libstdc++-33.i686 gcc gcc-c++ glibc glibc-devel glibc-devel.i686 \
 libaio libaio-devel libgcc libgcc.i686 libstdc++ libstdc++.i686 \
 libstdc++-devel libXext libXtst libXi openmotif openmotif22 redhat-lsb \
 sysstat zlib zlib.i686 libX11 libX11.i686 unzip xorg-x11-utils xorg-x11-xauth \
 unzip ksh make ocfs2-tools numactl numactl-devel motif motif-devel && \
 groupadd -g 1000 oinstall && \
 useradd -u 1000 -g 1000 -m oracle && \
 mkdir -p ${ORA_HOME} && \
 chown -R oracle:oinstall ${_SCRATCH} && \
 chown -R oracle:oinstall ${ORA_HOME}

USER oracle

RUN mkdir -p ${JDK_HOME} ${FMW_HOME} && \
 echo "inventory_loc=${FMW_HOME}/oraInventory" > ${_SCRATCH}/oraInst.loc && \
 echo "inst_group=oinstall" >> ${_SCRATCH}/oraInst.loc && \
 tar xzf ${_SCRATCH}/jdk-8u66-linux-x64.gz -C ${JDK_HOME} --strip-components=1 && \
 rm -rf ${_SCRATCH}/jdk-8u66-linux-x64.gz && \
 unzip ${_SCRATCH}/fmw_12.2.1.0.0_infrastructure_Disk1_1of1.zip -d ${_SCRATCH} && \
 ${JDK_HOME}/bin/java -jar ${_SCRATCH}/fmw_12.2.1.0.0_infrastructure.jar \
 -novalidation -silent -responseFile ${_SCRATCH}/silent.rsp \
 -invPtrLoc ${_SCRATCH}/oraInst.loc ORACLE_HOME=${FMW_HOME} \
 INSTALL_TYPE="Fusion Middleware Infrastructure" && \
 rm -rf ${_SCRATCH}/fmw_12.2.1.0.0_infrastructure_Disk1_1of1.zip \
 ${_SCRATCH}/fmw_12.2.1.0.0_infrastructure.jar

RUN [[ -f ${_SCRATCH}/fmw_12.2.1.0.0_bpmqs_Disk1_1of2.zip ]] && \
 [[ -f ${_SCRATCH}/fmw_12.2.1.0.0_bpmqs_Disk1_2of2.zip ]] && \
 unzip ${_SCRATCH}/fmw_12.2.1.0.0_bpmqs_Disk1_1of2.zip -d ${_SCRATCH} && \
 unzip ${_SCRATCH}/fmw_12.2.1.0.0_bpmqs_Disk1_2of2.zip -d ${_SCRATCH} && \
 ${JDK_HOME}/bin/java -jar ${_SCRATCH}/fmw_12.2.1.0.0_bpm_quickstart.jar \
 -novalidation -silent -responseFile ${_SCRATCH}/silent.rsp \
 -invPtrLoc ${_SCRATCH}/oraInst.loc ORACLE_HOME=${FMW_HOME} && \
 rm -rf ${_SCRATCH}/fmw_12.2.1.0.0_bpmqs_Disk1_1of2.zip \
 ${_SCRATCH}/fmw_12.2.1.0.0_bpmqs_Disk1_2of2.zip \
 ${_SCRATCH}/fmw_12.2.1.0.0_bpm_quickstart.jar \
 ${_SCRATCH}/fmw_12.2.1.0.0_bpm_quickstart2.jar && \
 export BPM_INSTALLED=1

RUN [[ -z ${BPM_INSTALLED} ]] && \
 [[ -f ${_SCRATCH}/fmw_12.2.1.0.0_soaqs_Disk1_1of2.zip ]] && \
 [[ -f ${_SCRATCH}/fmw_12.2.1.0.0_soaqs_Disk1_2of2.zip ]] && \
 unzip ${_SCRATCH}/fmw_12.2.1.0.0_soaqs_Disk1_1of2.zip -d ${_SCRATCH} && \
 unzip ${_SCRATCH}/fmw_12.2.1.0.0_soaqs_Disk1_2of2.zip -d ${_SCRATCH} && \
 ${JDK_HOME}/bin/java -jar ${_SCRATCH}/fmw_12.2.1.0.0_soa_quickstart.jar \
 -novalidation -silent -responseFile ${_SCRATCH}/silent.rsp \
 -invPtrLoc ${_SCRATCH}/oraInst.loc ORACLE_HOME=${FMW_HOME} && \
 rm -rf ${_SCRATCH}/fmw_12.2.1.0.0_soaqs_Disk1_1of2.zip \
 ${_SCRATCH}/fmw_12.2.1.0.0_soaqs_Disk1_2of2.zip \
 ${_SCRATCH}/fmw_12.2.1.0.0_soa_quickstart.jar \
 ${_SCRATCH}/fmw_12.2.1.0.0_soa_quickstart2.jar

RUN [[ -f ${_SCRATCH}/fmw_12.2.1.0.0_wccontent_Disk1_1of1.zip ]] && \
 unzip ${_SCRATCH}/fmw_12.2.1.0.0_wccontent_Disk1_1of1.zip -d ${_SCRATCH} && \
 ${JDK_HOME}/bin/java -jar ${_SCRATCH}/fmw_12.2.1.0.0_wccontent_generic.jar \
 -novalidation -silent -responseFile ${_SCRATCH}/silent.rsp \
 -invPtrLoc ${_SCRATCH}/oraInst.loc ORACLE_HOME=${FMW_HOME} \
 INSTALL_TYPE="WebCenter Content" && \
 rm -rf ${_SCRATCH}/fmw_12.2.1.0.0_wccontent_Disk1_1of1.zip \
 ${_SCRATCH}/fmw_12.2.1.0.0_wccontent_generic.jar

RUN [[ -f ${_SCRATCH}/fmw_12.2.1.0.0_wcportal_Disk1_1of1.zip ]] && \
 unzip ${_SCRATCH}/fmw_12.2.1.0.0_wcportal_Disk1_1of1.zip -d ${_SCRATCH} && \
 ${JDK_HOME}/bin/java -jar ${_SCRATCH}/fmw_12.2.1.0.0_wcportal_generic.jar \
 -novalidation -silent -responseFile ${_SCRATCH}/silent.rsp \
 -invPtrLoc ${_SCRATCH}/oraInst.loc ORACLE_HOME=${FMW_HOME} \
 INSTALL_TYPE="WebCenter Portal" && \
 rm -rf ${_SCRATCH}/fmw_12.2.1.0.0_wcportal_Disk1_1of1.zip \
 ${_SCRATCH}/fmw_12.2.1.0.0_wcportal_generic.jar

RUN [[ -f ${_SCRATCH}/fmw_12.2.1.0.0_wcsites_Disk1_1of1.zip ]] && \
 unzip ${_SCRATCH}/fmw_12.2.1.0.0_wcsites_Disk1_1of1.zip -d ${_SCRATCH} && \
 ${JDK_HOME}/bin/java -jar ${_SCRATCH}/fmw_12.2.1.0.0_wcsites_generic.jar \
 -novalidation -silent -responseFile ${_SCRATCH}/silent.rsp \
 -invPtrLoc ${_SCRATCH}/oraInst.loc ORACLE_HOME=${FMW_HOME} \
 INSTALL_TYPE="WebCenter Sites" && \
 rm -rf ${_SCRATCH}/fmw_12.2.1.0.0_wcsites_Disk1_1of1.zip \
 ${_SCRATCH}/fmw_12.2.1.0.0_wcsites_generic.jar

RUN [[ -f ${_SCRATCH}/fmw_12.2.1.0.0_ohs_linux64_Disk1_1of1.zip ]] && \
 unzip ${_SCRATCH}/fmw_12.2.1.0.0_ohs_linux64_Disk1_1of1.zip -d ${_SCRATCH} && \
 ${_SCRATCH}/fmw_12.2.1.0.0_ohs_linux64.bin -jreLoc ${JDK_HOME} \
 -novalidation -silent -responseFile ${_SCRATCH}/silent.rsp \
 -invPtrLoc ${_SCRATCH}/oraInst.loc ORACLE_HOME=${FMW_HOME} \
 INSTALL_TYPE="Colocated HTTP Server (Managed through WebLogic server)" && \
 rm -rf ${_SCRATCH}/fmw_12.2.1.0.0_ohs_linux64_Disk1_1of1.zip \
 ${_SCRATCH}/fmw_12.2.1.0.0_ohs_linux64.bin

RUN rm -rf ${_SCRATCH}

CMD /bin/bash

Once you have your files ready, you can build your image using the command below. You will then have to create a domain manually and then start your managed servers [and optionally, nodemanagers] as separate containers.

cd ~/fmw
docker build -t oracle/fmw:12.2.1.0.0 -f ./Dockerfile.12.2.1.0.0 .

Oracle WebLogic is now certified to run on Docker Containers starting with version 12c (12.1.3.0.0) and you will like the way these Oracle products work with Docker now. There is also a white paper available which describes a WebLogic deployment. The concepts explained this is white paper can be applied to the SOA/BPM and WebCenter managed servers as well. You can read the white paper here.

Sunday Dec 20, 2015

Implement Oracle WebCenter 11g as Docker Containers

This post shows you how to create Docker Images with Oracle WebCenter installed. Download the entire project directory from GitHub here.

Docker offers a container-based virtualization platform for applications and I have been playing with it for a few weeks now to see how I can create a base image with all the Oracle Software installed. Consequently, I wanted to port the WebLogic Admin Server and Managed Servers as Docker containers. As patches become available, I would then update my base image and distribute it. Once a new image is available, the idea is to create new containers for the WebLogic Admin Server and Managed Servers while retaining existing configuration and data. Here is how I did it.

I have used a Ubuntu 15.10 virtual machine for this exercise. You will need plenty of disk space on your virtual machine as Docker creates an intermediate image after every instruction. The instructions for setting up Docker can be found on their website here.

Once the environment is setup and ready to go, the next step is to identify a Docker build context directory and create the necessary folder structure. 

mkdir ~/webcenter
cd ~/webcenter
mkdir jdk wls rcu wcc wcp

After the directories are setup, all the necessary installers should be downloaded. The installers come as a ZIP archive and most of them will have to be extracted in those directories. If you have downloaded the project files from GitHub, you can find the instructions to download and extract the files under the respective directory. You can also click on a directory below to see the instructions. At the end you should have a directory structure shown below. 

./webcenter
 - Dockerfile.11.1.1.9.0
 - domain
  - docker_domain_admin.jar
 - jdk
  - jdk-7u91-linux-x64.tar.gz
 - rcu
  - rcuHome
  - readme_fmw_ps7.htm
 - scripts
  - add_server.py
  - start_server.sh
 - wcc
  - 22060967
  - Disk1
  - Disk2
  - ECM_22249978
  - readme_fmw_ps7.htm
 - wcp
  - 21950042
  - Disk1
  - Disk2
  - Disk3
  - Disk4
  - readme_fmw_ps7.htm
 - wls
  - p20780171_1036_Generic.zip
  - silent.xml
  - wls1036_generic.jar

Create or edit the Dockerfile in your context directory. For this post, the Dockerfile is named as Dockerfile.11.1.1.9.0. 

FROM oraclelinux:6
MAINTAINER Justin Paul

ENV _SCRATCH /tmp/scratch
ENV ORA_HOME /u01/app/oracle
ENV JDK_HOME ${ORA_HOME}/product/jdk
ENV FMW_HOME ${ORA_HOME}/product/fmw
ENV ADM_HOME ${ORA_HOME}/admin

COPY jdk ${_SCRATCH}/jdk/
COPY wls ${_SCRATCH}/wls/
COPY rcu ${_SCRATCH}/rcu/
COPY wcc ${_SCRATCH}/wcc/
COPY wcp ${_SCRATCH}/wcp/
COPY domain ${_SCRATCH}/domain/
COPY scripts ${_SCRATCH}/scripts/

RUN yum update -y -q && \
 yum install -y -q binutils compat-libcap1 compat-libstdc++-33 \
 compat-libstdc++-33.i686 gcc gcc-c++ glibc glibc-devel glibc-devel.i686 \
 libaio libaio-devel libgcc libgcc.i686 libstdc++ libstdc++.i686 \
 libstdc++-devel libXext libXtst libXi openmotif openmotif22 redhat-lsb \
 sysstat zlib zlib.i686 libX11 libX11.i686 unzip xorg-x11-utils xorg-x11-xauth && \
 groupadd -g 1000 oinstall && \
 useradd -u 1000 -g 1000 -m oracle && \
 mkdir -p ${ORA_HOME} && \
 chown -R oracle:oinstall ${_SCRATCH} && \
 chown -R oracle:oinstall ${ORA_HOME}

USER oracle

RUN mkdir -p ${JDK_HOME} && \
 tar xzf ${_SCRATCH}/jdk/jdk-7u91-linux-x64.tar.gz -C ${JDK_HOME} --strip-components=1 && \
 mkdir -p ${FMW_HOME} && \
 ${JDK_HOME}/bin/java -jar ${_SCRATCH}/wls/wls1036_generic.jar \
 -mode=silent -silent_xml=${_SCRATCH}/wls/silent.xml && \
 mv ${_SCRATCH}/rcu/rcuHome ${FMW_HOME}/ && \
 echo "inventory_loc=${FMW_HOME}/oraInventory" > ${_SCRATCH}/oraInst.loc && \
 echo "inst_group=oinstall" >> ${_SCRATCH}/oraInst.loc && \
 ${_SCRATCH}/wcp/Disk1/runInstaller -silent -invPtrLoc ${_SCRATCH}/oraInst.loc \
 -responseFile ${_SCRATCH}/wcp/Disk1/stage/Response/sampleResponse_wls.rsp \
 -jreLoc ${JDK_HOME} -waitforcompletion -force -novalidation \
 MIDDLEWARE_HOME=${FMW_HOME} ORACLE_HOME=${FMW_HOME}/Oracle_WC1 && \
 ${_SCRATCH}/wcc/Disk1/runInstaller -silent -invPtrLoc ${_SCRATCH}/oraInst.loc \
 -responseFile ${_SCRATCH}/wcc/Disk1/stage/Response/sampleResponse_wls.rsp \
 -jreLoc ${JDK_HOME} -waitforcompletion -force -novalidation \
 MIDDLEWARE_HOME=${FMW_HOME} ORACLE_HOME=${FMW_HOME}/Oracle_ECM1 && \
 mkdir -p ${ADM_HOME}/tools/templates && \
 mkdir -p ${ADM_HOME}/tools/scripts && \
 cp ${_SCRATCH}/domain/* ${ADM_HOME}/tools/templates && \
 cp -r ${_SCRATCH}/scripts/* ${ADM_HOME}/tools/scripts

RUN rm -rf ${_SCRATCH}

CMD /bin/bash

Run the following command to build your image. Use --no-cache=true if you do not want to cache intermediate images. 

cd ~/webcenter
docker build -t webcenter:11.1.1.9.0 -f ./Dockerfile.11.1.1.9.0 .

After the image(s) are built, we can start creating the containers. For the purpose of this post, we will keep the admin domain configuration on a shared volume. This volume will be mounted with RW permissions on the admin server container and with R permissions only on the managed server containers.

Run the following commands to create and start your admin server container. Use the -d flag instead of -i to force the container to run as a background process. 

mkdir -p ~/volume/aserver
docker run -i -P --name=adminserver \
-v ~/volume/aserver:/u01/app/oracle/admin/aserver:rw \
-t webcenter:11.1.1.9.0 /bin/sh /u01/app/oracle/admin/tools/scripts/start_server.sh \
adminserver

Find the IP Address of the newly created container using the docker inspect command. 

docker inspect --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' adminserver

After the Admin Server is up and running, login to the Admin Console as weblogic [the password is welcome1]. Update the JDBC data sources for all services that you intend to use. They are setup with dummy values in the domain and these data sources should be configured before you can start and managed server. For example, if you intend to create the UCM managed server container, you must configure the CSDS data source to connect to a valid database.

After the Admin Server configuration is complete, start a managed server container using the following commands. Use the -d flag instead of -i to force the container to run as a background process. Replace abc.def.ghi.jkl with the IP Address of the Admin Server. 

docker run -i -P --name=ucm_mserver1 \
-v ~/volume/aserver:/u01/app/oracle/admin/aserver:ro \
-t webcenter:11.1.1.9.0 /bin/sh /u01/app/oracle/admin/tools/scripts/start_server.sh \
ucm t3://abc.def.ghi.jkl:7001

You can create containers for any managed servers supported by the domain. Replace SERVER_ID with one of [capture, ibr, ipm, irm, ssxa, ucm, urm, collaboration, portlet, spaces, utilities] to create a managed server container for that service. Also remember to replace abc.def.ghi.jkl with the IP Address of the Admin Server. 

docker run -d -P --name=SERVER_ID_1 \
-v ~/volume/aserver:/u01/app/oracle/admin/aserver:ro \
-t webcenter:11.1.1.9.0 /bin/sh /u01/app/oracle/admin/tools/scripts/start_server.sh \
SERVER_ID t3://abc.def.ghi.jkl:7001

If you wish to update the image with the latest patches, update the Dockerfile with the instructions to apply the patches and build the image again or create a new image. After the new image is created, recreate the admin server and managed server containers using the same commands as above. Make sure you use the right image, mount the correct shared volume and change the container names.

NOTE: Please note that these instructions should be used for development and evaluation purposes only, as such they are unsupported and should not be used on production environments.

Monday Jul 27, 2015

Migrating huge content repositories using Archiver

I was recently asked to migrate a WebCenter Content repository to a new instance. They had the content exported locally using the Web Archiver applet and they were copying the archive folder to every destination content server file system. The archive folder was ~300 GB in size and it took around ~6 hours to tarball the archive directory and another ~2 hours to copy the tarball to the destination content server and then another ~6 hours to extract the tarball. And this process was repeated every time we wanter to replicate the repository. 

A easier solution may have been to mount a shared volume on the source content server and export the content there. Then you could just mount this shared volume to any destination server and use it for the import. However, the Archiver web applet only allows you to have the archives under the default location in the instance directory, that is [INSTANCE_DIR]/ucm/cs/archives. The key is to run the Archiver from the console as a standalone application. The standalone version of the Archiver application is required to create new collections or browse the local file system to connect to new collections.
  1. After you have identified a location for your archive collection, say /u02/share/archives/abc, create a new file, collection.hda, under the new directory and enter the following lines. Make sure that the IDC_Name parameter does NOT match the IDC_Name for the content server instance.

    @Properties LocalData
    IDC_Name=wcc-ucm
    blDateFormat=M/d{/yy}{ h:mm[:ss]{ a}}!mAM,PM!tAmerica/New_York
    @end
    @ResultSet Archives
    2
    aArchiveName
    aArchiveDescription
    @end

  2. Start the Archiver utility as a standalone application, [DOMAIN_HOME]/ucm/cs/bin/Archiver, go to Options -> Open Archive Collection… and click Browse Local… to create a new collection location. Browse to the newly created directory and select the collection.hda file we just created, /u02/share/archives/abc/collection.hda.
    Open Archive Collection

  3. In the Browse To Archiver Collection, enter the correct paths for the vault and weblayout directories on the current system and click Ok.
    Browse to Archive Collection

  4. Once the new Archive Collection is added to the list, click Open to make it current. You can now create your new archive and export content into it.
  5. After you have completed your content export/import, make sure that you return to Options -> Open Archive Collection… and make the default Archive Collection active.
To IMPORT this archive collection on another content server, make the shared volume available on the target server and follow steps 2-5 to load the collection for import.

Sunday Mar 22, 2015

Drag and drop file upload using HTML5 (YUI3)

I have seen a lot of ADF applications provide drag and drop file upload capabilities. I was wondering if there was an easy way to have this on the Content Server. If you are familiar with the UI components on the Content Server, you will know that it relies a lot on YUI.

In this example, I will use YUI3 Uploader to provide drag and drop multiple file upload capability on the Content Server. I will create a simple HTML file and check it into the Content Server. Accessing it from the content server using will show a form and you will be able to upload files. If you want to have Content Server like pages, you can modify this HTML file to insert IDOC includes and save it as a HCSP file.
Upload Example Screenshot

I have used the example provided by Yahoo "Example: Multiple Files Uploader with POST Variables and Server Data Retrieval" and modified it to upload files to a Content Server.

First and foremost, you will require the following lines to include YUI3 libraries and stylesheets.
JavaScript and Stylesheet Includes

Second, create a form to capture some metadata. In this example, we will apply these metadata values to all files being uploaded together. You can also move this form to the uploader table to set metadata at the file level.
Metadata Input Form

Third, load the Uploader and the JSON libraries. Also, the Content Server expects the file bytes to be uploaded using the primaryFile field name.
Load YUI Libraries and create Instance

Fourth, set the POST variables or the metadata to be assigned to the files. In this example, we will use the same values for all files. You can have different metadata for individual files too.
Set Metadata as POST Variables

Fifth and finally, once the checkin is complete, we need to parse the JSON data and then display the Content ID.
Process Data returned from the Content Server

You can download the HTML (file_upload.htm) here.

Note:

  • According to the announcement here, Yahoo has stopped development on YUI. However, you should be able to use the concepts used here with other frameworks.
  • If your browser does not support HTML 5, for example IE <= 9, you will need to use the flash component which requires additional configuration not discussed here.
  • This uploader is not supported on iOS devices.

Saturday Mar 21, 2015

WebCenter Content in the Enterprise

I stumbled upon this diagram I created some time back showing the different WebCenter Content modules and how they fit into the enterprise. Since then, the product branding has changed, so made some updates and posted it here.

For the sake of simplicity, only Oracle WebCenter Content products are shown here. Oracle Forms Recognition is not part of the suite but is shown for completeness. There are out-of-the-box integrations/solutions with other Oracle Fusion Middleware products like WebCenter Portal and SOA/BPM Suite which are not shown here.

Saturday Mar 07, 2015

RIDC using Oracle Java Stored Procedures

Have you faced a situation where you have a database table filled with content in a BLOB column and you want to migrate them into the Oracle Content Server. If you have and you want to write a PL/SQL procedure to loop through the records and check-in the content directly into the Content Server, you can use the Oracle JVM that is configured with an Oracle Database. Then you can load your RIDC libraries and your RIDC code into the database as schema objects and then write a Java Stored Procedure.

In this example, I will explain how use a table with content in a BLOB column and check them into the Content Server. We will use two approaches, one to save the file to the disk and then checkin, and the other to keep the file in memory and checkin. Note that the in memory solution should not be used with large files.

The first step is to check if your Oracle Database has Oracle JVM enabled. Usually, it is enabled by default, otherwise, you can find instructions to enable it here.
http://docs.oracle.com/cd/B28359_01/java.111/b31225/chfour.htm#BABEBCJA

The second step is to grant certain permissions to the schema user. In this example, I have created a new user, TEST.

CREATE USER test IDENTIFIED BY <passwd>;
GRANT ALL TO test;
EXEC DBMS_JAVA.GRANT_PERMISSION('TEST', 'java.net.SocketPermission', '*', 'connect, resolve');
EXEC DBMS_JAVA.GRANT_PERMISSION('TEST', 'java.io.FilePermission', '/tmp/file1.txt', 'read, write');

The third step and the most complicated one is to load all the required Java libraries and dependencies as database objects. This will include all the jar files that are in the RIDC library as well as the jars that contain all the classes referenced by the RIDC jars. You can download the RIDC library here. The dependency list is extensive and can be a nightmare to build; but Oracle does provide a tool to make things a bit easier.

The ojvmtc tool enables you to resolve all external references, prior to running the loadjava tool. The ojvmtc tool allows the specification of a classpath that specifies the JARs, classes, or directories to be used to resolve class references. When an external reference cannot be resolved, this tool either produces a list of unresolved references or generated stub classes to allow resolution of the references, depending on the options specified. Generated stub classes throw a java.lang.ClassNotfoundException, if it is referenced at runtime.

Please note that you will need to resolve all the dependencies for the entire modules to work. However, for this article we do not need to resolve all dependencies.

The following command will check for references and will generate stub classes for the classes it did not find. I downloaded and placed the RIDC and other jars in the "jars" directory under my home directory.

<dbhome>/bin/ojvmtc -list ~/jars/oracle.ucm.ridc-11.1.1.jar -classpath ~/jars/*.jar -jar dummy.jar -bootclasspath <dbhome>/jdk/jre/lib/rt.jar:<dbhome>/jdk/jre/lib/jce.jar:<dbhome>/jdk/jre/lib/jsse.jar

Now we are ready to load all these libraries into our schema. Run the following command to load all the jar files.

<dbhome>/bin/loadjava -verbose -resolve -user test/<passwd> ~/jars/*.jar

The fourth step is to write your Java class, compile and load it into your database schema. I will use the two methods below to checkin content.

// This method will take the file path and checkin the file
public static String checkinFile(String fPath) {
String dDocName = "";
try {
manager = new IdcClientManager();
client = (IdcClient<IdcClientConfig, Protocol, Connection>) manager.createClient("idc://10.0.0.5:4444");

IdcContext userContext = new IdcContext("sysadmin");
DataBinder binder = client.createBinder();
binder.putLocal("IdcService", "CHECKIN_NEW");
binder.putLocal("dDocTitle", "Test File - ABC000001");
binder.putLocal("dDocType", "Document");
binder.putLocal("dSecurityGroup", "Public");
binder.putLocal("dDocAccount", "");
binder.addFile("primaryFile", new TransferFile(new File(fPath)));
ServiceResponse response = client.sendRequest(userContext, binder);
DataBinder responsebinder = response.getResponseAsBinder();
dDocName = responsebinder.getLocal("dDocName");
client = null;
manager = null;
} catch (Exception e) {
// Nothing to do
dDocName = e.getLocalizedMessage();
}
return dDocName;
}
// This method will take a BLOB field and checkin in memory
public static String checkinBlob(BLOB bFile, String filename) {
String dDocName = "";
try {
manager = new IdcClientManager();
client = (IdcClient<IdcClientConfig, Protocol, Connection>) manager.createClient("idc://10.0.0.5:4444");

// Read the BLOB and create a Input Stream
// Make sure that the files are small as we are using in memory
byte[] filearray = bFile.getBytes(1, (int) bFile.length());
ByteArrayInputStream stream = new ByteArrayInputStream(filearray);

IdcContext userContext = new IdcContext("sysadmin");
DataBinder binder = client.createBinder();
binder.putLocal("IdcService", "CHECKIN_NEW");
binder.putLocal("dDocTitle", "Test File - ABC000002");
binder.putLocal("dDocType", "Document");
binder.putLocal("dSecurityGroup", "Public");
binder.putLocal("dDocAccount", "");
binder.addFile("primaryFile", new TransferFile(stream, filename, bFile.length()));
ServiceResponse response = client.sendRequest(userContext, binder);
DataBinder responsebinder = response.getResponseAsBinder();
dDocName = responsebinder.getLocal("dDocName");
client = null;
manager = null;
} catch (Exception e) {
// Nothing to do
dDocName = e.getLocalizedMessage();
}

return dDocName;
}

Run the following command to load the class file.

<dbhome>/bin/loadjava -verbose -f -resolve -user test/<passwd> ~/jars/CheckinFileIntoContentServer.class

Oracle DB Commands

The fifth step is to create PL/SQL functions that use the Java methods from the class we just loaded.

  CREATE
      OR
 REPLACE
FUNCTION test.checkinFile(fPath IN VARCHAR2)
  RETURN VARCHAR2
      AS LANGUAGE JAVA
    NAME 'com.oracle.justin.wcc.CheckinFileIntoContentServer.checkinFile(java.lang.String) return java.lang.String';

  CREATE
      OR
 REPLACE
FUNCTION test.checkinBlob(bFile IN BLOB, filename IN VARCHAR2)
  RETURN VARCHAR2
      AS LANGUAGE JAVA
    NAME 'com.oracle.justin.wcc.CheckinFileIntoContentServer.checkinBlob(oracle.sql.BLOB, java.lang.String) return java.lang.String';

The sixth step is to create and insert two rows with content into the table.

CREATE TABLE TEST.blob_table (
  fileid INTEGER NOT NULL,
  filename VARCHAR2(255) NOT NULL,
  filedata BLOB
);

The seventh and the final step is to loop through the table and checkin the files.

  -- Save the file to a temporary location
  -- and checkin the file
  SELECT filename,
         filedata
    INTO filename,
         vblob
    FROM blob_table
   WHERE fileid = 1;
  lbloblen := DBMS_LOB.getlength(vblob);
  lfile := UTL_FILE.FOPEN('TEMP_DIR', 'file1.txt', 'w', 32767);
  WHILE lpos < lbloblen LOOP
    DBMS_LOB.READ(vblob, lamount, lpos, lbuffer);
    UTL_FILE.PUT_RAW(lfile, lbuffer, TRUE);
    lpos := lpos + lamount;
  END LOOP;
  UTL_FILE.FCLOSE(lfile);
  DBMS_OUTPUT.PUT_LINE ('dDocname: ' || CHECKINFILE('/tmp/' || filename));

  -- IN MEMORY file checkin
  SELECT filename,
         filedata
    INTO filename,
         vblob
    FROM blob_table
   WHERE fileid = 2;
   DBMS_OUTPUT.PUT_LINE ('dDocname: ' || CHECKINBLOB(vblob, filename));

After the PL/SQL is executed, you should see two files in your Content Server with metadata that was coded in the Java class.

You can download the source files below:

NOTE: Please note that these code snippets should be used for development and testing purposes only, as such it is unsupported and should not be used on production environments.

Sunday Feb 08, 2015

RIDC using Jython Scripts

Jython is an implementation of Python for the JVM. Jython takes the Python programming language syntax and enables it to run on the Java platform. This allows seamless integration with the use of Java libraries and other Java-based applications. Plus it helps to have a loosely typed interpreted language as it makes writing code much faster.

I have written a set of scripts to execute some core Content Server services to check-in, update, delete content items and to browse, create, delete, link items using FrameworkFolders. You can download the scripts here.

JythonWCCAllExampleScripts.zip

Checkin new Content and a Revision
Update Metadata
Delete a single revision and all revisions
FrameworkFolders: Create Folder
FrameworkFolders: Create Shortcuts
FrameworkFolders: Browse Folders
FrameworkFolders: Copy Items
FrameworkFolders: Move Items
FrameworkFolders: Delete Items
FrameworkFolders: Propagate Metadata
FrameworkFolders: Remove (unfile) content from a folder

NOTE: Please note that these code snippets should be used for development and testing purposes only, as such it is unsupported and should not be used on production environments.

About

Welcome to my blog. I use this site to share my experience as well as tips and tricks on Oracle Fusion Middleware products.

Contributors

Search

Archives
« August 2016
SunMonTueWedThuFriSat
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
   
       
Today