Monday Mar 09, 2009

Python tips - Handle text file

There are two text files, each are 10 million lines, the size of the text file at about 100M. Now need to know that the two documents there is cross-check the number of lines, in other words, we want to know the the number of lines simultaneously in the two documents exist. Each text file here is unique, so they do not have any duplicate rows. Python set could do this very easy and higher efficient than shell, awk.
#!/usr/bin/python
a = set(open(”data.uniq.1″))
b = set(open(”date.uniq.2″))
print len(a; b)
Here I find a blog in Chinese also description this tips

Monday Oct 27, 2008

Check the OWfontpath's fontpath whether exist

From snv_97 some fonts have been removed. So, when you select some locales such like CCK, you will see garbage display on dtlogin screen. Actually not only dtlogin, all dt application such like dtterm will garbage display. What's root cause? Have a look /usr/dt/config/Xsetup, there is a fuction ADDFONTPATH_LOCAL() to add fonts path:
 ADDFONTPATH_LOCAL() {
        # Combine lines together to make arguments for the xset command
        FP=`/usr/bin/awk '
        BEGIN { fp="fp+ " }
        /\^[     ]\*$/ { fp=" +fp " ; continue }
        { printf("%s%s", fp, $0) ; fp="," } ' $1`
        if [ -n "$FP" ]; then

                eval "$XDIR/xset $FP"
        fi
}

If the font path which be passed by $1(OWfontpath) do not exist. xset will failed. To set X font path successfully, we should make sure all fonts path in OWfontpath exist. That means, if you remove some fonts path, they should be remove from OWfontpath too. Currently from snv_97 to snv_101, some the fonts path, which do not exist, are still listed in OWfontpath. This cause the issue I describe in header of this paper. I wrote a simple tool in python to check which fontpath have been removed, but still in OWfontpath. To display correctly, they need to be removed from OWfontpath.:

###############check_fontpath.py########################
#!/usr/bin/python
import os
import sys

def check_OWfontpath(OWfont):
    '''Check whether the entrie from $OWfont exists.
    '''
    bret = False
    for line in open(OWfont):
        if ',' in line:
            for path1 in line.split(','):
                if (not os.path.exists(path1.strip())):
                    print path1
                    if (bret==False):
                        bret = True
            continue
        if (not os.path.exists(line.strip())):
            print line,
            if (bret==False):
                bret = True
    return bret

def main():
    basedir = '/usr/openwin/lib/locale'
    owfontpath = 'OWfontpath'
    if (not os.path.exists(basedir)):
        print ' %s is not exist. ' % basedir
    for lc in os.listdir(basedir):
        lc_full = basedir + os.path.sep + lc
        if (os.path.isdir(lc_full) and (not os.path.islink(lc_full)) ):
            OWfont = lc_full + os.path.sep + owfontpath
            if (os.path.isfile(OWfont)):
                bRet = check_OWfontpath(OWfont)
                if (bRet==True):
                    print 'Above fonts path do not exist which listed by : %s' % OWfont  
                print   

if __name__ == '__main__':
    main()


About

williamxue

Search

Archives
« March 2015
SunMonTueWedThuFriSat
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
    
       
Today