Non-blocking I/O using Servlet 3.1: Scalable applications using Java EE 7 (TOTD #188)


Servlet 3.0 allowed asynchronous request processing but only traditional I/O was permitted. This can restrict scalability of your applications. In a typical application, ServletInputStream is read in a while loop.
public class TestServlet extends HttpServlet {
    protected void doGet(HttpServletRequest request, HttpServletResponse response)
         throws IOException, ServletException {    
ServletInputStream input = request.getInputStream();
       byte[] b = new byte[1024];
       int len = -1;
       while ((len = input.read(b)) != -1) {
          . . .
       }
   }
}
If the incoming data is blocking or streamed slower than the server can read then the server thread is waiting for that data. The same can happen if the data is written to ServletOutputStream.

This is resolved in Servet 3.1 (JSR 340, to be released as part Java EE 7) by adding event listeners - ReadListener and WriteListener interfaces. These are then registered using ServletInputStream.setReadListener and ServletOutputStream.setWriteListener. The listeners have callback methods that are invoked when the content is available to be read or can be written without blocking.

The updated doGet in our case will look like:

AsyncContext context = request.startAsync();
ServletInputStream input = request.getInputStream();
input.setReadListener(new MyReadListener(input, context));

Invoking setXXXListener methods indicate that non-blocking I/O is used instead of the traditional I/O. At most one ReadListener can be registered on ServletIntputStream and similarly at most one WriteListener can be registered on ServletOutputStream. ServletInputStream.isReady and ServletInputStream.isFinished are new methods to check the status of non-blocking I/O read. ServletOutputStream.canWrite is a new method to check if data can be written without blocking.

 MyReadListener implementation looks like:

@Override
public void onDataAvailable() {
try {
StringBuilder sb = new StringBuilder();
int len = -1;
byte b[] = new byte[1024];
while (input.isReady()
&& (len = input.read(b)) != -1) {
String data = new String(b, 0, len);
System.out.println("--> " + data);
}
} catch (IOException ex) {
Logger.getLogger(MyReadListener.class.getName()).log(Level.SEVERE, null, ex);
}
}

@Override
public void onAllDataRead() {
System.out.println("onAllDataRead");
context.complete();
}

@Override
public void onError(Throwable t) {
t.printStackTrace();
context.complete();
}

This implementation has three callbacks:
  • onDataAvailable callback method is called whenever data can be read without blocking
  • onAllDataRead callback method is invoked data for the current request is completely read.
  • onError callback is invoked if there is an error processing the request.
Notice, context.complete() is called in onAllDataRead and onError to signal the completion of data read.

For now, the first chunk of available data need to be read in the doGet or service method of the Servlet. Rest of the data can be read in a non-blocking way using ReadListener after that. This is going to get cleaned up where all data read can happen in ReadListener only.

The sample explained above can be downloaded from here and works with GlassFish 4.0 build 64 and onwards.

The slides and a complete re-run of What's new in Servlet 3.1: An Overview session at JavaOne is available here.

Here are some more references for you:

Comments:

Nice.

Posted by guest on November 28, 2012 at 01:36 AM PST #

Thanks Arun,
Nice post!!
excited enough to get started with servlet 3.1

Posted by Amit Phaltankar on November 28, 2012 at 03:13 AM PST #

when I read this first time first thought was "are they trying to do something similar to nodejs ? is this something like acknowledgement of nodejs way of doing IO" :)
I think java really will be amazing when Lambda Project a.k.a. JSR 335 will be available.

Posted by Akshay Ransing on November 28, 2012 at 11:28 AM PST #

When you say the first chunk needs to be read in the doGet method, what does this (temporary) implementation look like?

Posted by Jim Cheesman on November 28, 2012 at 12:47 PM PST #

Jim,

It'll be similar to the implementation in onDataAvailable. But that does not follow DRY.

The EG has agreed that this code need to be specified at one place and that would be onDataAvailable method only. So this is only an interim work around, probably for the next few builds of GlassFish only.

Posted by Arun Gupta on November 28, 2012 at 12:51 PM PST #

The title reads '...Scalable applications using Java EE 7', does it mean that jee7 will be made to run on multiple core when the app is deployed? Just like the other dynamic languages(Scala,Groovy etc)

Posted by guest on November 29, 2012 at 07:45 AM PST #

Scalability comes from the non-blocking I/O as compared to blocking I/O earlier. No special configuration is required in the application server to run across multiple cores.

Posted by Arun Gupta on December 02, 2012 at 06:04 PM PST #

Regarding:

For now, the first chunk of available data need to be read in the doGet or service method of the Servlet. Rest of the data can be read in a non-blocking way using ReadListener after that. This is going to get cleaned up where all data read can happen in ReadListener only.

Might be tidier to read the first block some time between the ReadListener being created and the first callback being made. I.e. MyReadListener.create(input, context); have create instantiate and do the first blocking read. That way it's all outside of doGet and to adapt for when it's all async, all the code is in one place (instead of across all your servlets)

Posted by joel on December 19, 2012 at 09:40 AM PST #

I'm a bit confused about the claim that the servlet 3 spec requires blocking io.

ServletInputStream input = request.getInputStream();
byte[] b = new byte[1024];
int len = -1;
while ((len = input.read(b)) != -1) {
. . .
}

where does it say in the spec that input.read(b) has to block (as in wait for data to arrive in the socket's receive buffer)? InputStream is essentially an interface, why can't the ServletInputStream returned be "backed" by e.g. a byte[] or a ByteBuffer or some other abstraction which has been filled from the original http request by a non-blocking read(s) of the client socket?
This would not change the semantics in any way.
I think you're confusing non-blocking with asynchronous reads (which is what your example code demonstrates).
Could you point to a specific section of the Servlet 3.0 spec that proscribes what I've suggested?

Posted by guest on March 24, 2013 at 12:41 PM PDT #

Post a Comment:
Comments are closed for this entry.
About

profile image
Arun Gupta is a technology enthusiast, a passionate runner, author, and a community guy who works for Oracle Corp.


Java EE 7 Samples

Stay Connected

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today