Monday Oct 21, 2013

Do your filesystems have un-owned files ?

As part of our work for integrated compliance reporting in Solaris we plan to provide a check for determining if the system has "un-owned files", ie those which are owned by a uid that does not exist in our configured nameservice.  Tests such as this already exist in the Solaris CIS Benchmark (9.24 Find Un-owned Files and Directories) and other security benchmarks.

The obvious method of doing this would be using find(1) with the -nouser flag.  However that requires we bring into memory the metadata for every single file and directory in every local file system we have mounted.  That is probaby not an acceptable thing to do on a production system that has a large amount of storage and it is potentially going to take a long time.

Just as I went to bed last night an idea for a much faster way of listing file systems that have un-owned files came to me. I've now implemented it and I'm happy to report it works very well and peforms many orders of magnatude better than using find(1) ever will.   ZFS (since pool version 15) has per user space accounting and quotas.  We can report very quickly and without actually reading any files at all how much space any given user id is using on a ZFS filesystem.  Using that information we can implement a check to very quickly list which filesystems contain un-owned files.

First a few caveats because the output data won't be exactly the same as what you get with find but it answers the same basic question.  This only works for ZFS and it will only tell you which filesystems have files owned by unknown users not the actual files.  If you really want to know what the files are (ie to give them an owner) you still have to run find(1).  However it has the huge advantage that it doesn't use find(1) so it won't be dragging the metadata for every single file and directory on the system into memory. It also has the advantage that it can check filesystems that are not mounted currently (which find(1) can't do).

It ran in about 4 seconds on a system with 300 ZFS datasets from 2 pools totalling about 3.2T of allocated space, and that includes the uid lookups and output.

#!/bin/sh

for fs in $(zfs list -H -o name -t filesystem -r rpool) ; do
        unknowns=""
        for uid in $(zfs userspace -Hipn -o name,used $fs | cut -f1); do
                if [ -z "$(getent passwd $uid)" ]; then
                        unknowns="$unknowns$uid "
                fi
        done
        if [ ! -z "$unknowns" ]; then
                mountpoint=$(zfs list -H -o mountpoint $fs)
                mounted=$(zfs list -H -o mounted $fs)
                echo "ZFS File system $fs mounted ($mounted) on $mountpoint \c"
                echo "has files owned by unknown user ids: $unknowns";
        fi
done
Sample output:

ZFS File system rpool/ROOT/solaris-30/var mounted (no) on /var has files owned by unknown user ids: 6435 33667 101
ZFS File system rpool/ROOT/solaris-32/var mounted (yes) on /var has files owned by unknown user ids: 6435 33667
ZFS File system builds/bob mounted (yes) on /builds/bob has files owned by unknown user ids: 101

Note that the above might not actually appear exactly like that in any future Solaris product or feature, it is provided just as an example of what you can do with ZFS user space accounting to answer questions like the above.

Thursday Jan 24, 2013

Compliance reporting with SCAP

In Solaris 11.1 we added the early stages of our (security) Compliance framework.  We have (like some other OS vendors) selected to use the SCAP (Security Content Automation Protocol) standard from NIST.  There are a number of different parts to SCAP but for Compliance reporting one of the important parts is the OVAL (Open Vulnerability Assesment Language) standard.  This is what allows us to write a checkable security policy and verify it against running systems.

The Solaris 11.1 repository includes the OpenSCAP tool that allows us to generate reports written in the OVAL language (as well as other things but I'm only focusing on OVAL for now).

OVAL is expressed in XML with a number of generic and OS/application specific schema.  Over time we expect to deliver various sample security policies with Solaris to help customers with Compliance reporting in various industries (eg, PCI-DSS, DISA-STIG, HIPAA).

The XML in the OVAL langauge is passed to the OpenSCAP tool for evaluation, it produces either a simple text report of which checks passed and which failed or an XML results file and an optional HTML rendered report.

Lets look at a simple example of an policy written in OVAL.  This contains just one check, that we have configured the FTP server on Solaris to display a banner.  We do this in Solaris 11 by updating /etc/proftpd.conf to add the "DisplayConnect /etc/issue" line - which is not there by default.   So in a default Solaris 11.1 system we should get a "fail" from this policy.

The OVAL for this check was generated by a tool called "Enhanced SCAP Editor (eSCAPe)" which is not included in Solaris.  It could well have been hand edited in your text editor of choice. In a later blog posting I'll attempt to explain more of the OVAL language and give some more examples, including some Solaris specific ones but for now here is the raw XML:

<?xml version="1.0" encoding="UTF-8"?>
<oval_definitions xmlns="http://oval.mitre.org/XMLSchema/oval-definitions-5" 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
xmlns:oval="http://oval.mitre.org/XMLSchema/oval-common-5" 
xmlns:oval-def="http://oval.mitre.org/XMLSchema/oval-definitions-5" 
xmlns:independent-def="http://oval.mitre.org/XMLSchema/oval-definitions-5#independent" 
xsi:schemaLocation="http://oval.mitre.org/XMLSchema/oval-definitions-5 
   oval-definitions-schema.xsd http://oval.mitre.org/XMLSchema/oval-definitions-5#independent 
   independent-definitions-schema.xsd http://oval.mitre.org/XMLSchema/oval-common-5 oval-common-schema.xsd">

  <generator>
    <oval:product_name>Enhanced SCAP Editor</oval:product_name>
    <oval:product_version>0.0.11</oval:product_version>
    <oval:schema_version>5.8</oval:schema_version>
    <oval:timestamp>2012-10-11T10:33:25</oval:timestamp>
  </generator>
  <!--generated.oval.base.identifier=com.oracle.solaris11-->
  <definitions>
    <definition id="oval:com.oracle.solaris11:def:840" version="1" class="compliance">
      <metadata>
        <title>Enable a Warning Banner for the FTP Service</title>
        <affected family="unix">
          <platform>Oracle Solaris 11</platform>
        </affected>
        <description>/etc/proftpd.conf contains "DisplayConnect /etc/issue"</description>
      </metadata>
      <criteria operator="AND" negate="false" comment="Single test">
        <criterion comment="/etc/proftpd.conf contains &quot;DisplayConnect /etc/issue&quot;" 
          test_ref="oval:com.oracle.solaris11:tst:8400" negate="false"/>
      </criteria>
    </definition>
  </definitions>
  <tests>
    <textfilecontent54_test 
        xmlns="http://oval.mitre.org/XMLSchema/oval-definitions-5#independent" 
        id="oval:com.oracle.solaris11:tst:8400" version="1" check="all" 
        comment="/etc/proftpd.conf contains &quot;DisplayConnect /etc/issue&quot;" 
        check_existence="all_exist">
      <object object_ref="oval:com.oracle.solaris11:obj:8400"/>
    </textfilecontent54_test>
  </tests>
  <objects>
    <textfilecontent54_object 
        xmlns="http://oval.mitre.org/XMLSchema/oval-definitions-5#independent" 
        id="oval:com.oracle.solaris11:obj:8400" version="1" 
        comment="/etc/proftpd.conf contains &quot;DisplayConnect /etc/issue&quot;">
      <path datatype="string" operation="equals">/etc</path>
      <filename datatype="string" operation="equals">proftpd.conf</filename>
      <pattern datatype="string" 
         operation="pattern match">^DisplayConnect\s/etc/issue\s$</pattern>
      <instance datatype="int" operation="greater than or equal">1</instance>
    </textfilecontent54_object>
  </objects>
</oval_definitions>

We can evaluate this policy on a given host by using OpenSCAP like this:

$ oscap oval eval ftp-banner.xml 
Definition oval:com.oracle.solaris11:def:840: false
Evaluation done.

As you can see we got the expected failure of the test, but that output isn't very useful, lets instead generate some html output:

$ oscap oval eval --results results.xml --report report.html ftp-banner.xml
Definition oval:com.oracle.solaris11:def:840: false
Evaluation done.
OVAL Results are exported correctly.

Now we have a report.html file which looks like a bit like this:

OVAL Results Generator Information
Schema Version Product Name Product Version Date Time
5.8  cpe:/a:open-scap:oscap 
2013-01-24 14:18:55 
OVAL Definition Generator Information
Schema Version Product Name Product Version Date Time
5.8  Enhanced SCAP Editor  0.0.11  2012-10-11 10:33:25 

System Information
Host Name braveheart 
Operating System SunOS 
Operating System Version 11.1 
Architecture i86pc 
Interfaces
Interface Name net0 
IP Address 192.168.1.1 
MAC Address aa:bb:cc:dd:ee:ff 
OVAL System Characteristics Generator Information
Schema Version Product Name Product Version Date Time
5.8  cpe:/a:open-scap:oscap 
2013-01-24 14:18:55 
Oval Definition Results



 True  



 False  



 Error  



 Unknown  



 Not Applicable  



 Not Evaluated  
OVAL ID Result Class Reference ID Title
oval:com.oracle.solaris11:def:841 true compliance
Enable a Warning Banner for the SSH Service 
oval:com.oracle.solaris11:def:840 false compliance
Enable a Warning Banner for the FTP Service 

As you probably noticed write away the report doesn't match the OVAL I gave above because the report is actually from a very slightly larger OVAL file which checks the banner exists for both SSH and FTP.  I did this purely to cut down on the amount of raw XML above but also so the report would show both a success and failure case.


About

DarrenMoffat

Search

Categories
Archives
« April 2014
MonTueWedThuFriSatSun
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
17
18
19
20
21
22
23
24
25
26
27
28
29
30
    
       
Today