Intrusion detection in Sun Java System Web Server 7.0 update 2 - in experimental stages

Intrusion detection in Sun Java System Web Server 7.0 update 2 - in experimental stages

I have introduced an experimental untested intrusion detection feature in Web Server 7.0 update 2. It is currently an unsupported feature. Basically we can add in server.xml a file name which contains ModSecurity ruleset. Note that this is an experimental feature so please give me feedback about your experiences.

Additions to server.xml

Element
Possible Values Description
<config-file>
Text
This element may be present at the virtual-server level as well as at the server level. Points to a file containing ModSecurity rules. As with all file paths in server.xml it may be an absolute path or a relative path, in which case it is relative to the config directory. The file name component may contain wildcard characters to specify multiple files within the given directory. Multiple config-file elements may be present as well.

Additions to obj.conf

A new AuthTrans SAF, secrule-config, is introduced to control the behavior of the ModSecurity engine. Please make sure that this is the first (topmost) AuthTrans directive in obj.conf.

The following table describes parameters for the secrule-config function.

Parameter Description
engine              
(Optional) Indicates how SecRule directives are processed at request time.
"on" indicates that the directives should be applied.
"off" indicates that the directives should not be applied.
"detection only" indicates that the directives should be evaluated but the result of the evaluation should not be enforced.
The default value is what is set by SecRuleEngine directive (if any) in configuration file(s) specified by <config-file> element. If SecRuleEngine directive is not present, it is "off".
process-request-body (Optional) Indicates whether request bodies are processed when evaluating SecRule directives. When request body processing is enabled, the server will buffer the entire request body in memory, up to the limit defined by SecRequestBodyInMemoryLimit directive (if any) in configuration file(s) specified by <config-file> element. If SecRequestBodyInMemoryLimit directive is not present, it is "131072".
"on" indicates that request bodies should be processed.
"off" indicates that response bodies should not be processed.
The default value is what is set by SecRequestBodyAccess directive (if any) in configuration file(s) specified by <config-file> element. If SecRequestBodyAccess directive is not present, it is "off".
process-response-body (Optional) Indicates whether response bodies are processed when evaluating SecRule directives. When response body processing is enabled, the server will buffer the entire response body in memory, up to the limit defined by SecResponseBodyLimit directive (if any) in configuration file(s) specified by <config-file> element. If SecResponseBodyLimit directive is not present, it is "524288".
"on" indicates that response bodies should be processed.
"off" indicates that response bodies should not be processed.
The default value is what is set by SecResponseBodyAccess  directive (if any) in configuration file(s) specified by <config-file> directive. If SecResponseBodyAccess directive is not present, it is "off".

Example

# Disable SecRule processing in the /docs directory
<Object ppath="/docs/\*">
AuthTrans fn="secrule-config" engine="off"
</Object>

SecRuleEngine, SecRequestBodyAccess and SecRequestBodyAccess will still work in the files specified in <config-file> in server.xml.

Sample :

I changed server.xml to have a new config-file element as shown below:
  <virtual-server>
    <config-file>sample.conf</config-file>
    <name>test</name>
    <host>...</host>
    <http-listener-name>http-listener-1</http-listener-name>
  </virtual-server>

and added this file in <ws7.0u2-server-install-dir>/https-test/config directory

$cat sample.conf
SecRuleEngine On
# Request related
SecRequestBodyAccess On
# default limit is 128 KB (131072)
SecRequestBodyInMemoryLimit  10000
# Variables
SecRule REQUEST_HEADERS "request_headers_match"
SecRule REQUEST_HEADERS_NAMES "request_headers_names_match"
SecRule HTTP_xxx "request_headers_xxx_match" "phase:1"
SecRule REQUEST_HEADERS:yyy "request_headers_yyy_match"  "phase:1"
SecRule REQUEST_HEADERS:/zzz/ "request_headers_zzz_match" "phase:1"
SecRule ARGS "args_match"
SecRule ARGS_NAMES "args_names_match"
SecRule REQUEST_BODY "request_body_match"
SecRule ARGS_COMBINED_SIZE "@gt 1000"
SecRule ENV "env_match"
SecRule QUERY_STRING "query_string_match"
SecRule REQUEST_COOKIES "request_cookies_match"
SecRule REQUEST_COOKIES_NAMES "request_cookies_name_match"
SecRule REQUEST_HEADERS_NAMES "x-aaaaaa.\*" "rev:1,severity:2,msg:'Oops not allwed'"
#Do not match lowercase
SecDefaultAction "deny"
SecRule HTTP_User-Agent "Internet-exprorer"
SecRule HTTP_User-Agent "Mosiac 1\\.\*"
SecRule REQUEST_BODY "X-AAAAAA.\*"
SecRule HTTP_Referer "Powered by Gravity Board" "id:350000,rev:1,severity:2,msg:'Gravity Board Google Recon attempt'"
SecRule REQUEST_URI|REQUEST_BODY "select.\*from.\*information_schema\\.tables"


I started the server. I get this warning at the time of server start up
[11/Apr/2008:12:28:26] info (28100): CORE1116: Sun Java System Web Server 7.0U3 B04/04/2008 12:38
[11/Apr/2008:12:28:27] warning (28101): HTTP3359: An unsupported element config-file is being used. The server may not operate correctly if unsupported features are used.
[11/Apr/2008:12:28:29] info (28101): CORE5076: Using [Java HotSpot(TM) Server VM, Version 1.5.0_12] from [Sun Microsystems Inc.]
[11/Apr/2008:12:28:43] info (28101): HTTP3072: http-listener-1: http://test.sun.com: ready to accept requests
[11/Apr/2008:12:28:43] info (28101): CORE3274: successful server startup

Now I send these two requests :
$telnet 0 1894
GET / HTTP/1.0
foo: request_headers_match

HTTP/1.1 403 Forbidden
Server: Sun-Java-System-Web-Server/7.0
Date: Fri, 11 Apr 2008 07:01:58 GMT
Content-length: 142
Content-type: text/html
Connection: close

<HTML><HEAD><TITLE>Forbidden</TITLE></HEAD>
<BODY><H1>Forbidden</H1>
Your client is not allowed to access the requested object.
</BODY></HTML>

$telnet 0 1894
GET / HTTP/1.0
foo: bar

HTTP/1.1 200 OK
Server: Sun-Java-System-Web-Server/7.0
Date: Fri, 11 Apr 2008 07:02:07 GMT
Content-type: text/html
Last-modified: Fri, 04 Apr 2008 14:24:09 GMT
Content-length: 12038
Etag: "2f06-47f63a09"
Accept-ranges: bytes
Connection: close

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">

<html>
....
  </body>
</html>

The first request got denied as the header value matched with "SecRule REQUEST_HEADERS "request_headers_match"" rule in sample.conf as shown below :
$tail -f ../logs/errors
[11/Apr/2008:12:31:58] security (28166): for host 127.0.0.1 trying to GET /index.html while trying to GET /, HTTP8005: SecRule directive in file /export1/ws/https-test/config/ms.conf at line 7 matched returning status code 403


Note that this is an untested feature and may have bugs. Please let me know if you find any.

Currently Supported ModSecurity Directives

Due to time constraints not all of the ModSecurity directives are supported in this release. This section documents the supported subset. Note that the supported rules are based on ModSecurity 2.0.

The terms and interfaces given below are taken from ModSecurity 2.0 documentation
http://www.modsecurity.org/documentation/modsecurity-apache/2.0.0-rc-4/modsecurity2-apache-reference.html

Some of these keywords like VARIABLES etc. are abstract quantities and not elements.

Directive
Values Description
SecRuleEngine On
Off
DetectionOnly
server initialization
Default value is "off"

Directive
Values Description
SecRule VARIABLES
"
[@OPERATOR]
Text
regular expression or parameters to pass to the operator
"
[ACTIONS]

VARIABLES [&!]VARIABLE[:/regular-expression/]|
[&!]VARIABLE[:name]|
[&!]VARIABLE[:regular-expression]...

&
should count the number of variables in the array.
!
x|!x:y examine all x but y should not be checked.
|
concatenate variables
:name
a particular value
:/regular_expression/ or :'/regular_expression/' matches regular expression


Values Description
VARIABLE
REQUEST_HEADERS
REQUEST_HEADERS_NAMES
REQUEST_HEADERS:yyy
or
REQUEST_HEADERS:/yyy/
Where yyy is any applicable request header name.
ARGS Contains request body if SecRequestBodyAccess is set to on.
ARGS_NAMES

ARGS_COMBINED_SIZE
AUTH_TYPE
ENV

QUERY_STRING

HTTP_yyy
TIME, TIME_EPOCH, TIME_DAY, TIME_HOUR, TIME_MIN,  TIME_SEC, TIME_WDAY, TIME_MON, TIME_YEAR
REMOTE_HOST, REMOTE_PORT, REMOTE_USER, REMOTE_ADDRESS
PATH_INFO
SERVER_NAME, SERVER_PORT, SERVER_ADDR
SCRIPT_BASENAME, SCRIPT_FILENAME

REQUEST_COOKIES
REQUEST_COOKIES_NAMES
REQUEST_FILENAME

REQUEST_BASENAME
REQUEST_BODY

REQUEST_LINE
REQUEST_METHOD
REQUEST_PROTOCOL
REQUEST_URI, REQUEST_URI_RAW
RESPONSE_BODY
RESPONSE_STATUS
RESPONSE_HEADERS

RESPONSE_HEADERS_NAMES
RESPONSE_PROTOCOL





Values
OPERATOR rx
eq
ge
gt
le
lt
validateByteRange



 Values Description
ACTIONS
ACTION[:xxx], ACTION[:xxx] ...



 Values
ACTION
allow
msg
id
rev
severity
log
deny
status
phase
t
skip
chain


Values Description
phase
[1..4]
phase:1 - Request headers stage
phase:2 - Request body stage
phase:3 - Response headers stage
phase:4 - Response body stage

Default value is phase:2
Note that operations on "phase:5 (Logging stage)" are not supported. If encountered, these are ignored and a log message is recorded.


Values Description
t
lowercase
Transformation functions to perform on the variables before operator is executed (this includes request body)
urlDecode
none
compressWhitespace
removeWhitespace
replaceNulls
removeNulls

Directive
Values Description
SecDefaultAction ACTIONS
For a SecRule, if the previous SecDefaultAction directive is present, those actions takes into effect.

If none of these SecDefaultAction directives are present before a SecRule (in that file or files loaded before it), default SecDefaultAction directive with ACTIONS
"log,deny,status:403,phase:2,t:replaceNulls,t:compressWhitespace,t:lowercase" is internally added.

For the following directives, in case the multiple directives are present (in one or multiple files), last directive's value takes the precedence.
Directive Values Description
SecRequestBodyAccess
On
Off
Whether the server should parse request body or not.
Default value is "off"

Directive Values Description
SecRequestBodyInMemoryLimit integer

Configures the maximum request body size server will store in memory. By default the limit is 128 KB (131072)



Directive Values Description
SecResponseBodyAccess On
Off
Whether the server should parse response body or not. Default value is "off"

Directive Values Description
SecResponseBodyLimit integer

Configures the maximum response body size that will be accepted for buffering. Anything over this limit will be rejected with status code 500 Internal Server Error. This setting will not affect the responses with MIME types that are not marked for buffering. By default this limit is configured to 512 KB. (524288)


Directive Values Description
SecResponseBodyMimeType
strings

Configures which MIME types are to be considered for response body buffering. The default value is text/plain text/html.


Directive Values Description
SecResponseBodyMimeTypesClear -
Clears the list of MIME types considered for response body buffering, allowing to start populating the list from scratch.



Please look at my next blog on this topic also.


Comments:

wow, pretty neat.

Posted by guest on April 12, 2008 at 02:54 PM IST #

Meena, this feature seems to be having some trouble with standard ReGex syntax from the Core Set v1.6.1. For example, this line in modsecurity_crs_20_protocol_violations.conf:

SecRule REQUEST_HEADERS:'/(Content-Length|Transfer-Encoding)/' "," "phase:2,t:none,deny,log,auditlog,status:400,msg:'HTTP Request Smuggling Attack.',id:'950012',tag:'WEB_ATTACK/REQUEST_SMUGGLING',severity:'1'"

throws these errors when an attempt is made to load it:

config (32208): HTTP8008: Unknown variable Transfer-Encoding)/' in SecRule directive in file /export/WS7/third-party/mod_sec_rules/modsecurity_crs_20_protocol_violations.conf at line 28
config (32208): HTTP8010: Unknown action auditlog,status:400,msg:'HTTP Request Smuggling Attack.',id:'950012',tag:'WEB_ATTACK/REQUEST_SMUGGLING',severity:'1' in directive in file /export/WS7/third-party/mod_sec_rules/modsecurity_crs_20_protocol_violations.conf at line 28
config (32208): HTTP8010: Unknown action tag:'WEB_ATTACK/REQUEST_SMUGGLING',severity:'1' in directive in file /export/WS7/third-party/mod_sec_rules/modsecurity_crs_20_protocol_violations.conf at line 28

The first error is complaining about this construct:

REQUEST_HEADERS:'/(Content-Length|Transfer-Encoding)/'

but as near as I can tell that construct is a valid regex grouping with an OR operator.

The other two errors seem to be complaining about auditlog and its variables being unknown. That's easy enough to live without, but breaking with the regex expression is kinda painful.

Posted by DzM on November 22, 2008 at 10:50 PM IST #

Thanx, Found one culprit. Will fix in next release.

In file "iplanet/ias/server/src/cpp/iws/netsite/lib/libsecrule/SecRec.cpp" line 540 regexp_started != regexp_started;

Should Be regexp_started = !regexp_started;

One BIG limitation with our server is we compare values in pblocks for headers or any variables. Our pblocks contain only lowercase name value pairs. So I do not recommend ACTION t:none (that removes default ACTION t:lowercase).

Anyways why would you only stop Content-Length? You would also want to stop "content-length"or any other lowercase upper case combination.

On my Web Server built with above fix :
$cat ms.conf
SecRuleEngine On
SecRule REQUEST_HEADERS:'/(content-length|transfer-encoding)/' "," "phase:2,t:none,deny,log,status:400,msg:'HTTP Request Smuggling Attack.',id:'950012'"

$telnet 0 1894
HEAD / HTTP/1.0
content-length: ,

HTTP/1.1 400 Bad request
Server: Sun-Java-System-Web-Server/7.0
Date: Sun, 23 Nov 2008 10:33:06 GMT
Content-length: 147
Content-type: text/html
Connection: close

Posted by meena on November 23, 2008 at 08:12 AM IST #

CR6775403 "Intrusion detection : REQUEST_HEADERS:'/ regular expression /' doesn't work" is fixed in 7.0 update 5.

Posted by meena on November 18, 2009 at 08:16 AM IST #

Thanks for a lifesaving feature :)

Will the setvar action be implemented any time soon?

Posted by Steffen Fiksdal on February 10, 2010 at 10:49 AM IST #

I have an issue with the following directive:
SecRule REQUEST_BODY "!&test=([A-Za-z0-9]|%3D|%2[BF])+" combined with a large HTTP POST with a payload of 4000 bytes.

The web server seems to hang and even core dump during the process of checking this rule with a large payload.

It is probably not the regexp that provokes this behaviour, rather the payload size?

Hope you can help me further, because I am stuck...

The last line in the log before a new child process is spawned:
Output function magnus-internal/process-response-secrule returned an error

Posted by Steffen Fiksdal on February 11, 2010 at 01:11 PM IST #

I tried increasing the thread stack size from the default 262144 bytes to 2621440 bytes, but it still fails.

Posted by Steffen Fiksdal on February 12, 2010 at 05:59 AM IST #

pcre seems to use a \*lot\* of stack size. I increased the thread stack size to 8MB, and the web server instance survived.

prcre can be built without using stack for recursion, but it will give a performance hit..

Posted by Steffen Fiksdal on February 12, 2010 at 08:47 AM IST #

We recommend you not to use regular expressions that eat up a lot of stack.

You can try :
SecRuleEngine On
SecRequestBodyAccess On
SecRule REQUEST_BODY "!\^[A-Za-z0-9&=%\\.]+$"
SecRule REQUEST_BODY "%([\^23]|2[\^bBfF]|3[\^dD])"

Posted by Meena Vyas on February 25, 2010 at 07:09 AM IST #

Thanks for your help Meena.

I have now changed my regular expressions so that they do not use that much stack.

To prevent a SIGSEGV from happening the libpcre has compile time options for limiting the number of match function recursions (--with-match-limit-recursion).I do not know if this option is set in the libpcre shipped with sjsws, but if it is set it must be a very high number. It might be worth considering putting in/modifying the compile time restriction in order to stop people like me making lousy regexps killing the server :)

I do not know what will be a "good" number though...

Regards,
Steffen

Posted by Steffen Fiksdal on February 26, 2010 at 04:59 AM IST #

Bringing Mod Security functionality to Sun Web Server is an excellent bonus! Are there and plans to have this feature supported?

DetectionOnly is working great (allowing matched requests to pass through) but matched rules seemed to be logged with the log level “fine”. Is it possible to modify the log level in DetectionOnly mode?

Thank you!

Posted by Eric Achelis on March 19, 2010 at 07:35 PM IST #

Thank you for using Our Web Server. Please contact Oracle/Sun support. Can you tell more details about your company?

Right now it logs it at LOG_VERBOSE level. You can file a request to change log level to what is in severity in each SecRule.

Posted by meena on March 22, 2010 at 09:17 AM IST #

severity and error log files' log level mapping is as follows :

0 = LOG_CATASTROPHE
1,2 = CRITICAL
3 = LOG_SECURITY
4 = LOG_WARN;
5,6 = LOG_INFORM
7 = LOG_VERBOSE

8 or any other value- We should have some error in error log stating that severity is not valid.

Posted by mena on February 09, 2011 at 07:34 AM IST #

We are working with the ModSecurity community on ports to different platforms.

http://www.modsecurity.org/projects/

Would you like us to list your module?

Posted by Ryan Barnett on March 21, 2011 at 02:09 PM IST #

Post a Comment:
  • HTML Syntax: NOT allowed
About

Meena Vyas

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today