Thursday Jul 16, 2009

784956 Processes

This week we had a customer claiming that they were unable to create more then 60,000 processes. This turned out to be due to them tuning max_nproc, maxuprc and maxpid but not setting segkpsize so the system would run out of “memory” before it ran into the resource limits for process.

Tuning segkpsize to 8G resolved it but I just had to see how many processes I could get running on an M8000.

Using these settings in /etc/system:

set segkpsize=0x300000
set pidmax=999999
set maxuprc=999990
set max_nprocs=999999

and a simple forker program:

#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <unistd.h>

int
main(int argc, char \*\*argv)
{
        pid_t pid;
        int count=0;
        while(count < argc == 2 ? 100 : atoi(argv[1]) &&
            (pid = fork()) != -1) {
                if (pid != 0 ) {
                        /\* Parent \*/
                        if (count % 1000 == 0)
                                printf("%d\\n", count);
                        count++;
                } else {
                        pause();
                        exit(0);
                }
        }
        if (pid < 0)
                perror("fork");
        printf("%d\\n", count);
}

I was slightly disappointed at the result:


$ ./forker 100000
1000
2000
3000
.....
782000
783000
784000
fork: Resource temporarily unavailable
784956
$

Only 784956 processes, plus the ones already running when the system booted. Trying to count them with ps obviously fails but mdb gives the real count.

# ps -e| wc
ksh: cannot fork: too many processes
# 
# echo nproc::print -d | mdb -k  
0t785025
# 

Someone must have managed to get more.

Tuesday Mar 18, 2008

When is a good idea to modify an underlying mirror?

Following on from “When to run fsck” and “When to run quotacheck” here is another:

When to modify the individual sub mirrors that make up a mirrored volume?

Answer: Never.

With the Logical volume manger in Solaris you can build a mirror from two sub mirrors:

# metastat d0
d0: Mirror
    Submirror 0: d10
      State: Okay         
    Submirror 1: d11
      State: Okay         
    Pass: 1
    Read option: roundrobin (default)
    Write option: parallel (default)
    Size: 20482875 blocks (9.8 GB)

d10: Submirror of d0
    State: Okay         
    Size: 20482875 blocks (9.8 GB)
    Stripe 0:
        Device   Start Block  Dbase        State Reloc Hot Spare
        c1d0s0          0     No            Okay   Yes 


d11: Submirror of d0
    State: Okay         
    Size: 20482875 blocks (9.8 GB)
    Stripe 0:
        Device   Start Block  Dbase        State Reloc Hot Spare
        c5d0s0          0     No            Okay   Yes 


Device Relocation Information:
Device   Reloc  Device ID
c1d0   Yes      id1,cmdk@AST3320620AS=____________3QF09GL1
c5d0   Yes      id1,cmdk@AST3320620AS=____________3QF0A1QD
# 

So here we have the mirror “d0” made up of devices “d10” and “d11”. Each of these devices can be addressed in the file system as /dev/md/rdsk/d0 /dev/md/rdsk/d10 and /dev/md/rdsk/d11 respectively. The block devices are also available if you so desire. While being able to address the underlying disk devices that make up a mirror is interesting and potentially useful it is only useful if you really know what you are doing.

Reading from the mirrors is o.k. Writing and that includes just mounting the file system is not. So if the device is idle you can do:


# cmp /dev/md/rdsk/d10 /dev/md/rdsk/d11


#

Which if it returns 01 gives you a feeling of confidence, although if you are this paranoid, and I am, then ZFS is a much better bet.


For example if the mirror contains a file system then mounting one side of the mirror and making modifications is a really really bad idea, even if the mirror is unmounted. Once you have made such a modification you would have to make sure the other side of the mirror had exactly the same change at the block level propagated to it. Realistically the only way to achieve that is for you to detach the other mirror and then reattach it so allow it to resync. If you really know what you are doing there are tricks you could do but I suspect those that really know what they are doing would not get into this mess in the first place.



1 If it does not then you have to look at how the mirror was constructed before you start to worry. If you did “metainit d0 –m d10 d11” or have grown the metadevice then the mirrors will never have been brought into sync. So only the blocks that have been written to since the operation will correctly comapare. Hence this is nothing to worry about. See I told you you do really have to know what you are doing.

About

This is the old blog of Chris Gerhard. It has mostly moved to http://chrisgerhard.wordpress.com

Search

Archives
« April 2014
MonTueWedThuFriSatSun
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
    
       
Today