Saturday Dec 06, 2014

Logging Weblogic with Apache Flume on HDFS by Matthias Fuchs

clip_image001

I have read the blog of Mark (http://www.rittmanmead.com) about using flume and hdfs. It sounds very easy and my idea was to integrate it as an extention to Oracle Cloud Control to have an extended tool for detailed log file monitoring and analytic. Here is my experience about it:

Download and Installation

I downloaded all files here and extracted it. I set my java home and started the flume on agent side, configs like Mark’s Blog. On the server side, I used Cloudera Express Edition on one node. Please set dfs replication to 1, if you use only one node

for testing. (/etc/hadoop/conf/hdfs-site.xml) I installed it without flume. I made the tests with starting it from shell, using standard version 1.5.0.1. Be aware to use a user, which is configured inside your HDFS. If not, you can’t login to HDFS or you try kerberos authentication …

Here the config for an managed server log file:

Log file rotation on weblogic

The above configuration works good, if you have no rotation on log files. But what happens, if weblogic rotates the log files? In my case the flume agents stopped to get more data. In flume documentation you also find the possibility to spool folders. For this the archived, rotated log files must be in a seperate folder. If you have all in one, you get the errors as described. So I changed the log configuration in wls – here a managed server of SOA Suite. I added a seperate log rotation directory and disabled auto delete of files. I used a soa server on 12.1.0.3. Read the complete article here.

WebLogic Partner Community

For regular information become a member in the WebLogic Partner Community please visit: http://www.oracle.com/partners/goto/wls-emea ( OPN account required). If you need support with your account please contact the Oracle Partner Business Center.

Blog Twitter LinkedIn Forum Wiki

Wednesday Apr 24, 2013

WebLogic Logging Configuration by René van Wijk

In general, it is a good idea to use log rotation to prevent having to stop a WebLogic server to remove a large log file that is filling up the file system. It is also a good idea to create a separate partition such that the root file system does not fill up. A partition is a contiguous set of blocks on a drive that are treated as if they were independent disks. The default installation of today’s Enterprise Linux distributions uses flexible partitioning layouts by creating one or more logical volumes. Some of the partitions that we might want to consider separating out from root to provide more flexibility and better performance in the environment

  • /home – A file server environment would benefit from separating out /home to its own partition. This is the home directory for all users on the system, if there are no disk quotas implemented, so separating this directory should isolate a user’s runaway consumption of disk space.
  • /tmp – If we are running a high-performance computing environment, large amounts of temporary space are needed during compute time, then released upon completion.
  • /usr – This is where the kernel source tree and Linux documentation (as well as most executable binaries) are located. The /usr/local directory stores the executables that must be accessed by all users on the system and is a good location to store custom scripts developed for our environment. If it is separated to its own partition, then files will not have to be reinstalled during an upgrade or reinstall by simply choosing not to have the partition reformatted
  • /var – The /var partition is important in mail, Web, and print server environments because it contains the log files for these environments and the overall system log. Chronic messages can flood and fill this partition. If this occurs and the partition is not separate from the /, service interruptions are possible. Depending on the environment, further separation of this partition is possible by separating out /var/spool/mail for a mail server or /var/log for system logs.
  • /opt – The installation of some third-party software products, such as WebLogic Server, default to this partition. If not separate, the installation will continue under / and, if there is not enough space allocated, could fail.


Read the full article here.

WebLogic Partner Community

For regular information become a member in the WebLogic Partner Community please visit: http://www.oracle.com/partners/goto/wls-emea ( OPN account required). If you need support with your account please contact the Oracle Partner Business Center.

Blog Twitter LinkedIn Mix Forum Wiki

Search

Archives
« March 2015
SunMonTueWedThuFriSat
21
29
30
31
    
       
Today