Tuesday Aug 23, 2011

Grizzly 2.0: HttpServer API. Asynchronous HTTP Server - Part II

In my previous blog "Asynchronous HTTP Server - Part I", we discussed the possibility of delegating HTTP request processing to a custom thread in order to not effect entire HTTP server stability and response time. An example of where this might be useful is when dealing with tasks, like database requests, which may take relatively long time to complete.

We saw that by using Grizzly's ability to suspend/resume HTTP request processing, we'd be able to move a possible bottleneck from Grizzly HttpServer service thread-pool to an application specific custom thread-pool. This is very important, cause it lets us isolate applications inside the HttpServer and avoid situations when clients of one application block clients of another application and potentially the entire server.

Now we will try to go further and see how we deal with another potential bottleneck, client<->server communication using blocking streams, without having to resort to the use of an additional thread pool. Here are some usecases (may be you'll be able to come with more of them):

  • Client<->server communication is slow. We spend significant amount of time waiting for data to become available or for the outbound socket buffer to be flushed so that we can continue writing. This slowness might be caused by underlying network architecture (for ex. relatively slow, high-latency mobile networks), or DoS-like attack.
  • Client or/and server is sending large amounts of data (for ex. large file upload/download). This operation usually can be time consuming even if client<->server communication is fast. Given this, it would only take a couple of clients uploading/downloading files, to occupy all available threads and make server unresponsive to other clients.
  • Combination of two above.

The Grizzly 2.x HttpServer API has the concept of NIO streams (non-blocking streams), which are pretty similar to standard java.io streams, but have ability to check whether next read/write operation can be executed without blocking. For example:

-----------------------------------------------
public class MyHttpHandler extends HttpHandler {
    @Override
    public void service(Request request, Response response) throws Exception {
        NIOInputStream nioInputStream = request.getInputStream();
        System.out.println("Number of bytes we can read without blocking is: " + nioStream.readyData());

        NIOOutputStream nioOutputStream = response.getOutputStream();
        System.out.println("I'm able to write 8K without blocking: " + nioOutputStream.canWrite(8192));
    }
}
-----------------------------------------------

The methods in the sample above can help you figure out if next I/O operation going to block or not. Additionally Grizzly provides a notification mechanism, that may be used in order to be notified when next read/write operation can be executed without blocking.

The general schema for non-blocking reading may look like:

-----------------------------------------------
final NIOInputStream nioInputStream = request.getInputStream();

// Register ReadHandler, which is going to be notified
// when at least 1 byte is available for reading
nioInputStream.notifyAvailable(new ReadHandler() {

    @Override
    public void onDataAvailable() throws Exception {
        processAvailable();

        // Reregister this ReadHandler to be notified 
        // when next chunk of data gets available
        nioInputStream.notifyAvailable(this);
    }

    @Override
    public void onAllDataRead() throws Exception {
        // Process the last chunk of data
        processAvailable();
        // Complete request processing
        // (preparing and sending response back to the client)
        complete();
    }

    @Override
    public void onError(Throwable t) {
        // Error occurred when reading data
        handleError(t);
        complete();
    }
});
-----------------------------------------------

* It's possible to set the minimum number of bytes you require to become available before notifying ReadHandler.

A similar approach is available for non-blocking writing using NIOOutputStream.

Usually we register Read/WriteHandler in the HttpHandler.service(...) method, but as we understand the actual notification may come from Grizzly asynchronously in a different thread, so before registering Read/WriteHandler we have to suspend HTTP request processing (so Grizzly won't finish it after exiting HttpHandler.service() method), and don't forget to resume it once processing is done.

-----------------------------------------------
public class MyHttpHandler extends HttpHandler {
    @Override
    public void service(final Request request, final Response response) throws Exception {
        // ReadHandler might be called asynchronously, so
        // we have to suspend the response
        response.suspend();
        final NIOInputStream nioInputStream = request.getInputStream();

        nioInputStream.notifyAvailable(new ReadHandler() {

            @Override
            public void onDataAvailable() throws Exception {
                processAvailable();
                nioInputStream.notifyAvailable(this);
            }

            @Override
            public void onAllDataRead() throws Exception {
                processAvailable();                
                complete();
            }

            @Override
            public void onError(Throwable t) {
                handleError(t);
                complete();
            }

            private void complete() {
                // Complete HTTP request processing
                ...................
                // Don't forget to resume!!!
                response.resume();
            }
        });
    }
}
-----------------------------------------------

PS: While the samples above demonstrate how the NIO streams feature may be used in binary mode via NIOInputStream and NIOOutputStream abstractions. The same set of methods/features could be used in character mode, using NIOReader and NIOWriter.

For more information please use following docs and samples:

Monday Aug 08, 2011

Grizzly 2.0: HttpServer API. Asynchronous HTTP Server - Part I

In my previous blog entry I described basic Grizzly HttpServer API abstractions and offered a couple of samples to show how one can implement light-weight Servlet-like web application.

Here I'll try to show how we can process HTTP requests asynchronously within HttpHandler, in other words implement asynchronous HTTP application.

What do we mean by "asynchronous"?

Normally HttpServer has a service thread-pool, whose threads are used for HTTP requests processing, which includes following steps:

  1. parse HTTP request;
  2. execute processing logic by calling HttpHandler.handle(Request, Response);
  3. flush HTTP response;
  4. return service thread to a thread-pool.
Normally the steps above are executed sequentially in a service thread. Using "asynchronous" feature it's possible to delegate execution of steps 2 and 3 to a custom thread, which will let us release the service thread faster.

Why would we want to do that?

As it was said above, the service thread-pool instance is shared between all the HttpHandlers registered on HttpServer. Assume we have application (HttpHandler) "A", which  executes a long lasting task (say a SQL query on pretty busy DB server), and application "B", which serves static resources. It's easy to imagine the situation, when couple of application "A" clients block all the service threads by waiting for response from DB server. The main problem is that clients of application "B", which is pretty light-weight, can not be served at the same time because there are no available service threads. So it might be a good idea to isolate these applications by executing application "A" logic in the dedicated thread pool, so service threads won't be blocked.

Ok, let's do some coding and make sure the issue we've just described is real :)


-----------------------------------------------
HttpServer httpServer = new HttpServer();

NetworkListener networkListener = new NetworkListener("sample-listener", "127.0.0.1", 18888);

// Configure NetworkListener thread pool to have just one thread,
// so it would be easier to reproduce the problem
ThreadPoolConfig threadPoolConfig = ThreadPoolConfig
        .defaultConfig()
        .setCorePoolSize(1)
        .setMaxPoolSize(1);

networkListener.getTransport().setWorkerThreadPoolConfig(threadPoolConfig);

httpServer.addListener(networkListener);

httpServer.getServerConfiguration().addHttpHandler(new HttpHandler() {
    @Override
    public void service(Request request, Response response) throws Exception {
        response.setContentType("text/plain");
        response.getWriter().write("Simple task is done!");
    }
}, "/simple");

httpServer.getServerConfiguration().addHttpHandler(new HttpHandler() {
    @Override
    public void service(Request request, Response response) throws Exception {
        response.setContentType("text/plain");
        // Simulate long lasting task
        Thread.sleep(10000);
        response.getWriter().write("Complex task is done!");
    }
}, "/complex");

try {
    server.start();
    System.out.println("Press any key to stop the server...");
    System.in.read();
} catch (Exception e) {
    System.err.println(e);
}
-----------------------------------------------

In the sample above we create, initialize and run HTTP server, which has 2 applications (HttpHandlers) registered: "simple" and "complex". To simulate long-lasting task in the "complex" application we're just causing the current thread to sleep for 10 seconds.

Now if you try to call "simple" application from you Web browser using URL: http://localhost:18888/simple - you see the response immediately. However, if you try to call "complex" application http://localhost:18888/complex - you'll see response in 10 seconds. That's fine. But try to call "complex" application first and then quickly, in different tab, call the "simple" application, do you see the response immediately? Probably not. You'll see the response right after "complex" application execution completed. The sad thing here is that service thread, which is executing "complex" operation is idle (the same situation is when you wait for SQL query result), so CPU is doing nothing, but still we're not able to process another HTTP request.

How we can rework the "complex" application to execute its task in custom thread pool? Normally application (HttpHandler) logic is encapsulated within HttpHandler.service(Request, Response) method, once we exit this method, Grizzly finishes and flushes HTTP response. So coming back to the service thread processing steps:

  1. parse HTTP request;
  2. execute processing logic by calling HttpHandler.handle(Request, Response);
  3. flush HTTP response;
  4. return service thread to a thread-pool.

we see that it wouldn't be enough just to delegate HTTP request processing to a custom thread on step 2, because on step 3 Grizzly will automatically flush HTTP response back to client at the state it currently is. We need a way to instruct Grizzly to not do 3 automatically on the service thread, instead we want to be able to perform this step ourselves once asynchronous processing is complete.

Using Grizzly HttpServer API it could be achieved following way:

  • HttpResponse.suspend(...) to instruct Grizzly to not flush HTTP response in the service thread;
  • HttpResponse.resume() to finish HTTP request processing and flush response back to client.
So asynchronous version of the "complex" application (HttpHandler) will look like:

-----------------------------------------------
httpServer.getServerConfiguration().addHttpHandler(new HttpHandler() {
    final ExecutorService complexAppExecutorService =
        GrizzlyExecutorService.createInstance(
            ThreadPoolConfig.defaultConfig()
            .copy()
            .setCorePoolSize(5)
            .setMaxPoolSize(5));
            
    @Override
    public void service(final Request request, final Response response) throws Exception {
                
        response.suspend(); // Instruct Grizzly to not flush response, once we exit the service(...) method
                
        complexAppExecutorService.execute(new Runnable() {   // Execute long-lasting task in the custom thread
            public void run() {
                try {
                    response.setContentType("text/plain");
                    // Simulate long lasting task
                    Thread.sleep(10000);
                    response.getWriter().write("Complex task is done!");
                } catch (Exception e) {
                    response.setStatus(HttpStatus.INTERNAL_SERVER_ERROR_500);
                } finally {
                    response.resume();  // Finishing HTTP request processing and flushing the response to the client
                }
            }
        });
    }
}, "/complex");
-----------------------------------------------

* As you might have noticed, "complex" application uses Grizzly ExecutorService implementation. This is the preferred approach, however you can still use own ExecutorService.

The three most important steps in the code above are marked red:

  1. Suspend HTTP response processing: response.suspend()
  2. Delegating task to the custom thread pool: complexAppExecutorService.execute(...)
  3. Resuming HTTP response processing: response.resume()

Now, using your browser, you can make sure "simple" and "complex" applications are not affecting each other, and the "simple" application works just fine when the "complex" application is busy.

About

oleksiys

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today