Sunday Mar 27, 2011

This blog will move soon

1. ZAO Sun Microsystems to Oracle Hardware move... Complete.

2. This blog move to ... in Progress.

Monday Nov 15, 2010

White paper updated: Running Oracle Real Application Clusters on Oracle VM Server for SPARC 2.0

"Running Oracle Real Application Clusters on Oracle VM Server for SPARC 2.0" white paper was updated to reflect all the Changes and Improvements of Oracle VM Server for SPARC 2.0.

Some key points from white paper:

  • UltraSPARC T2, T2Plus and SPARC T3 processors only supported. T1 is not.
  • Oracle Solaris 10 09/10 (Update 9) or later is required.
  • Running Oracle RAC on Oracle VM Server for SPARC requires the use of Oracle 10g R2 (Oracle or later with the same patch set on all nodes.
  • Link-based IPMP enables you to build IPMP group of private (and/or public) network interfaces within guest domains. No need to have host based routing anymore!
  • With Memory Dynamic Reconfiguration you may add more memory to guest domain while database instances are running. When going down, don't go below SGA+PGA or you may end up with swapping.
  • How to setup Jumbo Frames example given.
  • Pay attention to Availability white papers mentioned in the Reference section.

The authors would like to thank Daniel Dibbets, Sridhar Kn, Rahim Mau, Wataru Miyoshi, Khader Mohiuddin, Ezhilan Narasimhan, Gia-Khanh Nguyen and Uday Shetty.

Thursday Jun 04, 2009

Running Oracle Real Application Clusters (RAC) on Sun Logical Domains (LDoms)

BluePrint of Running Oracle Real Application Clusters (RAC) on Sun Logical Domains is available now. In this blueprint you will find step-by-step configuration example of two nodes in two guest domains located on two separate boxes. Use it as starting point to build your own cluster.

This article discusses running Oracle Real Application Clusters (RAC) on servers configured with Sun™ Logical Domains (LDoms). Sun LDoms virtualization technology allows the creation of multiple virtual systems on a single physical system, and enables fine-grained assignment of CPU and memory resources to an Oracle RAC workload. When deployed on Sun's CoolThreads™ technology-based servers, with up to 256 threads per system, this solution provides a powerful platform for both development and production environments. In development environments, multiple Oracle RAC nodes can be deployed on the same physical server to reduce hardware costs, while a production environment can place each Oracle node on a separate physical server for increased availability.

The authors would like to thank Sridhar Kn, Ezhilan Narasimhan, Gia-Khanh Nguyen, Uday Shetty from Sun Microsystems, as well as Rahim Mau, Khader Mohiuddin from Oracle Corporation for their contributions during the certification of Oracle on Sun Logical Domains.

Saturday Apr 25, 2009

Oracle database 10gR2 is certified with Solaris 10 ZFS

Oracle Database 10gR2 ( and higher patches) Single Instance (not RAC) is certified with Solaris 10 ZFS on SPARC 64-bit. Please read Solaris ZFS_Best_Practices_Guide. You may need to limit ARC cache.

Monday Mar 02, 2009

Configuring Oracle ASM in Solaris Container with Solaris Volume Manager.

We have learned how to configure Oracle in ASM in Solaris Container. For this we setup our container with raw device access. While this works good it is not good idea to expose device names into container. Virtualized OS administrator should not know about devices in global container. And being attached to special device is not giving flexibility. In this exercise we will migrate from raw device onto SMV metadevice.[Read More]

Wednesday Feb 25, 2009

Configuration example of Oracle ASM in Solaris Container.

Configuration example of Oracle ASM in Solaris Container.

In this post I will give you a tip on how to setup Oracle ASM in Solaris Container. The main point of container's configuration is to set proper privileges. Your container should have proc_priocntl (must) and proc_lock_memory (highly recommended) privileges in order to function properly with ASM in it. Use the following as an configuration example when creating container and adjust it for your needs. Please read comments inlined:


# container will be named zone1

# make sure that directory /zones exist and have permissions 700

set zonepath=/zones/zone1

set autoboot=true

set limitpriv=default,proc_priocntl,proc_lock_memory

set scheduling-class="FSS"

set max-shm-memory=16G

# use ip-type exclusive at your wish, non-exclusive is also possible

set ip-type=exclusive

add net

set physical=e1000g1


add fs

set dir=/usr/local

# make sure /opt/zone1/local exist

set special=/opt/zone1/local

set type=lofs


add fs

# mount /distro from global zone into container.

# I have Oracle distribution files there

set dir=/distro

set special=/distro

set type=lofs

add options [ro]


# this device will be used for ASM inside container

add device

set match=/dev/rdsk/c4t600A0B8000562790000005D04998C446d0s0


# permit to use 16 cpu within container

add dedicated-cpu

set ncpus=16-16




Put this confing into zone1.txt and edit this file to adjust your configuration. Then create, install and boot container.

# zonecfg -z zone1 -f zone1.txt

# zoneadm -z zone1 install

# zoneadm -z zone1 boot

When you are done login to newly created container and proceed with installing Oracle and configuring ASM.


  1. Since I have dual-FC connected 6140 array I have it configured with Solaris I/O multipathing feature enabled

# stmsboot -D fp -e

  1. I really like to use VNC to access my lab remotely

Sunday Feb 22, 2009

Oracle Database 10g и Real Application Cluster сертифицированы с Sun Logical Domains (LDoms)

Oracle Database 10g и Real Application Cluster (RAC) отныне официально сертифицированы и поддерживаются на Sun Logical Domains (LDoms). Детали доступны так же в technology support matrix. Используя технологию виртуализации LDoms администратор максимально гибко управляет ресурсами системы и получает максимальный уровень безопасности, одновременно увеличивая уровень использования имеющихся серверов.

Среди множества возможных конфигураций я хотел бы специально отметить возможность создания кластера из нескольких нод, расположенных на одном единственном физическом сервере. Начните создавать свой кластер на чипе прямо сейчас.

Ожидайте, скоро тут появится пример конфигурации.

Oracle Database 10g and Real Application Cluster certified on Sun Logical Domains (LDoms)

Oracle Database 10g and Real Application Cluster (RAC) now officially certified and supported on Sun's Logical Domains (LDoms). For details also check technology support matrix. With LDoms virtualization technology administrators can achieve great level of resource management and security level at the same time having higher utilization of servers.

Among several types of deployment I would like to mention one specifically for developers. By using LDoms now you can have single box with a number of guest domains configured as a cluster nodes. Start with making cluster on a chip.

Stay tuned for real world configuration example.

Sunday Feb 15, 2009

too many cpu for me

When you are creating Oracle Database using templates/examples from the Oracle Database distribution you may fail with ORA-00018: maximum number of sessions exceeded. This can happen on a systems with Niagara chip. It seems that it have too many CPUs (virtual ones). Quick workaround is to turn off most of them at this time:

# psradm -f 16-255

Then create database and turn them back.

# psradm -n 16-255

Your system may have not so many (256) vCPUs. I faced the same problem with 64 vCPU and solution works for me.

The same trick works for Oracle Application Server reporting ORA-00371: not enough shared pool memory. Confirmed by Alexey, such trick is working.

Update: Glenn Fawcett offers another solution, suggesting to set CPU_COUNT on systems that have large number of vcpu's.

Configuration example of Oracle ASM on Solaris.

In this example you will see how to configure Oracle ASM on Solaris.

The following system will be used in this example: Sun T5220 with attached 6140 array, Solaris 10 (10/08) and Oracle 11.

Briefly, Oracle Database installation steps can be performed as five separate steps:

  1. Install Oracle binaries only

  2. Install Oracle patchset

  3. Configuring ASM

  4. Configuring Listener

  5. Database Creation

[Read More]

Friday Jan 23, 2009

Best Practices for Running Oracle Databases in Solaris Containers

Best Practices for Running Oracle Databases in Solaris Containers” now available on Sun/BluePrints site. This document describes Solaris Containers features for using with Oracle databases. You will learn how to setup Container and assign resources to it (scheduler, CPU and memory capping). It tells you which privilege gives you ability to use Dynamic Intimate Shared Memory (DISM) with Oracle. You may find how to setup Container so it will have it's own IP stack. Mounting UFS and ZFS filesystems, devices in Containers and System V Resource Controls also covered.


Solaris Containers provide a very flexible and secure method of managing multiple applications on a single Solaris OS instance. Solaris Containers use Solaris Zones software partitioning technology to virtualize the operating system and provide isolated and secure runtime environments for applications. Solaris Resource Manager can be used to control resource usage, such as capping memory and CPU usage,helping to ensure workloads get required system resources. By utilizing Solaris containers, multiple applications, or even multiple instances of the same application, can securely coexist on a single system, providing potential server consolidation savings.

Oracle 9i R2 and 10g R2 databases have been certified to run in a Solaris container. This paper provides step-by-step directions for creating a non-global zone in a Solaris container that is appropriate for running a non-RAC Oracle database. In addition, it describes special considerations that apply when running an Oracle database within a Solaris container.

Thursday Dec 11, 2008

Monitoring ZFS Statistic on the system loaded with Oracle database

Nothing happens without a reason. My previous post about Monitoring ZFS Statistic is not exception. This week I had T5240 (128 threads, 64GB memory, Solaris 10 10/08 aka U6) with 6140 attached and not loaded with other my projects. So I decided to load the boxes with Oracle on ZFS (binaries, datafiles, logs) and run some tests. Not a benchmarking. Just to check if this can live and breathe. I also wanted to check the battle for the memory between Oracle SGA (set to 32GB), 3800 processes connecting to database (taking approx. 5MB not shared memory each) and ZFS.

On storage I have made single 1TB RAID1 LUN and put ZFS on it. No data/logs of ZFS separation. Keeping it simple.

root@rac05#zpool list
oracle  1016G   243G   773G    23%  ONLINE  -

Arcstat script was modified a little further (lines 82, 177, 229 and 230) so I don't have too many zeros in statistics by going up from bytes to MB. Download new arcstat script and description file for dimstat.

 81 my $version = "0.1";
 82 my $mega = 1024 \* 1024;
 83 my $cmd = "Usage: [-hvx] [-f fields] [-o file] [interval [count]]\\n";
171 sub prettynum {
172     my @suffix=(' ','K', 'M', 'G', 'T', 'P', 'E', 'Z');
173     my $num = $_[1] || 0;
174     my $sz = $_[0];
175     my $index = 0;
176     return sprintf("%s", $num) if not $num =~ /\^[0-9\\.]+$/;
177     return sprintf("\*%d", $sz, $num);
178     while ($num > 1000 and $index < 8) {
179         $num = $num/1000;
180         $index++;
181     }
182     return sprintf("%\*d", $sz, $num) if ($index == 0);
183     return sprintf("%\*d%s", $sz - 1, $num,$suffix[$index]);
184 }
200 sub calculate {
229     $v{"arcsz"} = $cur{"size"}/$mega;
230     $v{"c"} = $cur{"c"}/$mega;
238 }

On the picture you may see results of monitoring:

At 1:01 test was started. It took 1 hour until all processes become ready (1 process was started each 1 second). Then Oracle database was busy between 2:04 and 2:53.

As you can see ZFS is taking more memory if it can, but it stays "polite" by freeing this memory for other processes when it needed.

I will be glad to hear from you if you have any thoughts and experience running Oracle Database on ZFS.

Thanks to Valery for his Perl Syntax highlighting plugin for Netbeans.

Thursday Oct 30, 2008

dimSTAT в примерах

Oracle runs best on Sun series.

На прошлой неделе мне довелось побывать на семинаре Oracle RAC DD4D (deep dive for developers), организованном компанией Oracle в Москве. Семинар крайне интересный и полезный, рекомендую! Огромное спасибо Дмитрию и всем докладчикам!

В числе обсуждённых тем была тема производительности. А конкретно производительности кластера Oracle RAC. Это даже не вопрос, что системные администраторы, администраторы баз данных и пользователи все хотят иметь максимально быструю систему. Но прежде чем приступать к настройке системы необходимо знать, что же с ней происходит. От пользователей вы можете услышать, что работает недостаточно хорошо. В руках администраторов базы есть как минимум стандартные средства мониторинга от Oracle (Enterprise Manager). А что же в руках системного администратора? Множество \*stat команд системы, выдающих столбцы цифр?

Что если нет опыта работы с командами \*stat? Не все помнят ключи запуска этих команд. Необходимо иметь информацию по длительному промежутку времени. В таком случае поможет dimSTAT. В этом посте я покажу несколько картинок из dimSTAT на четырёхнодовом кластере Oracle RAC на Solaris 10.

Основные возможности dim_STAT:

  • web интерфейс

  • Вся собираемая статистика хранится в одном месте

  • Возможность одновременно смотреть насколько собранных статистик

  • Интерактивное (Java) или статическое (PNG) представление

  • Мониторинг в реальном времени

  • Мониторинг нескольких серверов одновременно

  • Анализ по любому промежутку времени

  • Возможность добавления собственных расширений (Add-On)

  • Автоматизация действий

  • Закладки к выбранным статистикам

Здесь вы найдете несколько примеров того, что можно увидеть с помощью dimSTAT. Показанные графики взяты с разных тестов и не коррелируют между собой. Все графики по нескольким нодам, вы же можете смотреть по отдельных хостам и с необходимым уровнем детализации.

Использование CPU в пользовательском контексте.

Использование CPU  в системном и в системном+пользовательском контекстах.

Чтение+запись на диск и Сеть In+Out

Детализация по количетсву сетевых пакетов In и Out:

dimSTAT by examples

Oracle runs best on Sun series.

Last week I was attending Oracle RAC DD4D (deep dive for developers) seminar here in Moscow, Russia. No need to say how useful and full of information this seminar was. Thanks to Dmitry and all presenters!

Among many things we covered performance. Specifically it was Oracle RAC performance. System administrators, DBA and Users all wants to have better and faster system. But before you try to make your system better you have to take a look at your system from all possible angles. Users can tell you about their's personal feelings: “Oh, my program A works so slow”. In DBA's hands there are powerful tools from Oracle and other vendors. But what is in sysadmin's hands? Is it still \*stat commands producing rows of data?

What if you don't have experience looking into \*stat commands output and hardly can remember theirs syntax and even names? In that case all you need is dimSTAT. In this post I will show some screenshots of dimSTAT monitoring four Solaris 10 nodes cluster running Oracle RAC.

The main features of dim_STAT are:

  • A web based user interface

  • All collected data is saved in a central database

  • Multiple data views

  • Interactive (Java) or static graphs (PNG)

  • Real Time monitoring

  • Multi-Host monitoring

  • Post analyzing

  • Statistics integration (Add-On)

  • Professional reporting with automated features

  • One-click STAT-Bookmarks

Here are few examples of what can you see with dimSTAT. All screenshots shown here were taken from different times, so most of them does not correlate. Although shown examples are multi-host you may do analysis per hosts with any details you need.

CPU usage in User context:

CPU Usage in System and User+System:

Storage Read+Write and Network In+Out:

Next two charts will show you network packets In and Out:


Roman (pomah) Ivanov, ISV Engineering. Tips how to run Oracle best on Sun. Performance monitoring and tuning for system administrators. OpenSolaris user experience.


« July 2016