Monday Mar 31, 2014

Enhanced Oracle Solaris Cluster Security Framework

Enhanced Oracle Solaris Cluster Security Framework

Besides providing high availability (HA) & reliability to the applications, Oracle Solaris Cluster data services (agents) strive to provide a very secure HA environment by leveraging some of the salient security features implanted in the Oracle Solaris Cluster software. Oracle Solaris Cluster Resource Group Manager (RGM) callback methods such as Start, Stop or Validate execute with a high level of privilege and must be protected against modification by a non-privileged user. These callback methods in turn might execute application programs that often do not require elevated privilege. If an application program is to be executed with elevated privilege, it must similarly be protected against modification by an unprivileged user.

In summary, an agent developer needs to do ownership/permission checks on programs to be executed with elevated privilege; and we want to facilitate the reduction of privilege for executing such programs. For these purposes, several enhancements were introduced in the Oracle Solaris Cluster 4.1 software:

1) New checks on the ownership and permission of RGM callback method executable files.

2) A new Application_user standard property to provide a standard hook for the end user to specify a non-root username for execution of application programs.

3) A new scha_check_app_user command, which implements ownership and permission checks on application executable files.

4) A new cluster property, resource_security, to control the new ownership and permission checks.

This blog will brief you on these enhancement features.

Invoking applications with limited privilege using data services in Oracle Solaris Cluster software

The security enhancement features mentioned above allow Oracle Solaris Cluster software to follow the principle of least privilege and limit any damage that might happen due to accidental or deliberate misuse of unlimited access right bestowed to superuser (root). Oracle Solaris Cluster data services adhere to the principle of least privilege while invoking the applications running in HA mode on Oracle Solaris Cluster software. Since the resource methods that are responsible for interacting with the applications always run with superuser privilege, it is highly recommended from a security perspective that this privilege level be dropped down to the minimum while executing the application program.

It is therefore crucial that the Oracle Solaris Cluster agent methods should run all external programs using a wrapper, to ensure that the external program is executed with the correct username and privileges.

Oracle Solaris Cluster software implements this concept by providing the scha_check_app_user command and properties like Application_user and resource_security. Agent developers can use the Application_user property and the scha_check_app_user command to enforce least-privilege for their own applications. The behaviour of the scha_check_app_user command highly depends on the values of the two properties and The entire picture will be clearer when we visualize this with the help of a simple example described in further text.

For fine details about these features, refer to the man pages for scha_check_app_user(1HA), r_properties(5) and resource_security in cluster(1cl). Meanwhile let us take a quick tour to explore these features by creating a very simple application (referred as "My Example App" in this article) and a minimal skeleton of an agent to bind this application with the Oracle Solaris Cluster HA environment. However, the complexity of agent code is much higher when all the perspectives of an HA environment are taken into consideration. Our goal here is just to understand how to use these security features and hence, only the minimal modules required to bind an application to a resource and start/stop the application/program have been described.

Application_user example: My Example App

Oracle Solaris Cluster software supports myriad applications with its high availability environment and many of them are pretty complex, too. ‘My Example App' on the other hand is a concise sample shell script and will serve as an example of the usage of the Application_user property and the scha_check_app_user command.

My Example App performs the job of sending mail notification to a cluster administrator in scenarios of failover/switchover. Although short, this application can be effectively used to permit a cluster administrator to check emails on a smartphone and see whether a necessary action might be required if a critical resource or resource group goes up/down. The admin can put the My Example App resource in the same resource group of another important resource that needs to be monitored for this purpose.

Please note that this agent doesn't cover all the modules and it's not recommended to run this on a production box.

My Example App modules

- Online module (executes when resource goes up)

/opt/myexapp/online.ksh (Executable permissions: 755 owned by user app_user)

- Offline module (executes when resource goes down)

/opt/myexapp/offline.ksh (Executable permissions: 755 owned by user app_user)

Note: app_user is the UNIX user expected to execute the application. There should be a valid mapping of app_user on all the nodes on which the resource is planned to be configured.

Agent development for My Example App

To create a minimally functional agent for My Example App, we will at least require the Resource Type Registration file, start script and stop script.

 - myexapp RTR file

/opt/myexapp/myexapp.rtr is a Resource Type Registration (RTR) file for this agent. All resources of type myexapp will directly inherit the properties from this file.  

 - Start script

myexapp_start will serve as the start script and will be executed every time the resource goes up.


- Stop script

myexapp_stop will serve as the stop script and will be executed every time the configured resource goes down.


Resource configuration for My Example App

- Register the RTR file:

# /usr/cluster/bin/clresourcetype register -f /opt/myexapp/myexapp.rtr myexapp

- Create the resource group and the resource :
# /usr/cluster/bin/clresourcegroup create myexappRG

# /usr/cluster/bin/clresource create -g myexappRG -t myexapp myexappRS

This completes the configuration phase of the application to be run on Oracle Solaris Cluster software. Now, let us dive deeper to understand the behaviour of the scha_check_app_user command when it is invoked in different scenarios. Refer to scha_check_app_user(1HA) man page for more details.

Behaviour of scha_check_app_user in various scenarios

The following are the descriptions of the most prominent scenarios.

Note: The following guidelines apply to all the example scenarios:

- The user specified as Application_user should be present on all the participating nodes of the cluster.

- The user specified with the -U option is taken as application user ignoring the configured Application_user, file owner or resource_security value.

This completes a comprehensive case study for use of the scha_check_app_user command in a sample application running on Oracle Solaris Cluster software. Now, let us explore one more important feature that helps in storing sensitive information across an Oracle Solaris Cluster configuration.

Handling passwords and sensitive information across Oracle Solaris Cluster

The Oracle Solaris Cluster 4.1 software provides a secure and easy to use mechanism for storing passwords and other sensitive information through private strings. A private string is identified with a unique name, and has an encoded value that can only be obtained by using the scha_cluster_get command.

The clpstring command manages Oracle Solaris Cluster private strings. Private strings are used by resources to store and retrieve private values securely. A typical use might be for an internal password used by an agent. The clps command is the short form of the clpstring command.

Let’s say our agent uses a password string. In order to harness the private strings feature of the Oracle Solaris Cluster security framework for this agent, we are required to bind the private strings with the data service resource.

# /usr/cluster/bin/clpstring create -b RS1 -v pw_str

Enter string value:

Enter string value again:

Private string pw_str is created for the global cluster and is bound to the resource RS1.

Private strings are never exposed to non-privileged users in either obfuscated or clear text form. However, privileged users can then retrieve the private strings by using the scha_cluster_get query as follows:

# /usr/cluster/bin/scha_cluster_get -O PSTRING pw_str

For more detailed information on the clpstring command, refer clpstring(1 CL) man page.

Although the Oracle Solaris Cluster 4.1 software supports these security enhancements, it’s important to note that not all agents are currently using these new features. Some existing agents might execute application programs with elevated privileges, for example, they might be executed as root. So it’s judicious to take necessary steps in validating the contents of such programs installed on a cluster, and to make sure that they are installed with ownership & permissions that prevent a non-privileged user from modifying them.

Posted By:
Tapan Avasthi
Data Services, Availability Engineering, Oracle Solaris Cluster


Oracle Solaris Cluster Engineering Blog


« March 2014 »