Thursday Dec 12, 2013

Enhanced lights-out patching for Exadata Storage Cells

The recently released 11.2.3.3.0 Oracle Exadata version comes with multiple enhancements in the patchmgr utility. For those who don't know what the patchmgr utility is: the patchmgr utility is a tool Exadata Database Administrators use to apply (or rollback) an update to the Oracle Exadata Storage Cells. The enhanced patchmgr will have an option to send the operator an email for the most significant patch and rollback state changes. This eliminates the need to monitor the screen while the update or rollback is in progress.

In order to send the email you need to specify values for the '-smtp_from' and '-smtp_to' flags. Example as follows:

./patchmgr -cells ~/cell_group -patch -rolling \
           -smtp_from dbma@yourdomain.com  \
           -smtp_to your-email@yourdomain.com

Patchmgr will use the sendmail package which  is installed by default on the Exadata Database Server. It will start sendmail if it's not already started and assumes it's configured to deliver email for the domains you specify. You will recognize the format of the alerts patchmgr sends as they have the same formatting as the ASR emails.

The email you will receive when you enable this option can have the following end states for a cell: 

  • Started
  • Start Failed
  • Waiting
  • Patching
  • Rolling Back
  • Failed
  • Succeeded. 
  • Not Attempted 

The majority of the above states probably is self explanatory, however explaining 'Not Attempted' may help: 'Not Attempted' will be the final state of a cell when patching or rolling back (in a rolling fashion) of the current cell has failed. In that case the patching will stop and the remaining cells (which are not touched yet) are in the state 'Not Attempted'. So for example: imagine you are patching cel1,cel2 and cel3. When cel1 fails patching, then the end state for cel2 and cel3 will be 'Not Attempted'.

The following example will give you the idea: Patching starts in a rolling fashion from 11.2.3.2.0 to 11.2.3.3.0 for cel1,cel2 and cel3. 

Note: this is an example of the type of state changes you will see, actual patch timings are not correct and not relevant for this example. Also actual states or formatting may change in your release.

1. patchmgr was started in a rolling fashion. Initial state for all cells is 'Started'. This should be seen as this initialization phase. You will receive an email as follows:

2. The actual patching has begun since this is in rolling fashion, patchmgr starts with the first cel listed, which is cel1. The other two cells are waiting until cel1 finishes.


3. After some time patching of cel1 has completed. You will receive another status update in your inbox stating cel1 was updated successfully.


4. patchmgr continues with the next cell.


5. When succeeded (or failed) you will receive a state update via mail.


6. Last cell to be patched is cel3:


7. The last email you will receive will give an update stating patching was completed and lists the end state for all cells.

In case patching or rolling back would fail, you would receive a pointer to a release specific MOS note where you may find additional information.

Rene Kundersma 

About

Blog of Rene Kundersma, Principal Member of Technical Staff at Oracle Development USA. I am designing and evaluating solutions and best practices around database MAA focused on Exadata. This involves HA, backup/recovery, migration and database consolidation and upgrades on Exadata. Opinions are my own and not necessarily those of Oracle Corporation. See http://www.oracle.com/technology/deploy/availability/htdocs/maa.htm.

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today