LimitedInputStream: how to secure against OutOfMemoryErrors
By bblfish on Sep 15, 2005
When reading data off the web one has to always to be ready for the worst. One of the situations that is perhaps too often ignored is that the file one may want to read, may be huge, if not infinite. When parsing xml with SAX this could just lead to method that never returns, but when working with a DOM this could also lead to a huge data structure being created, and the fatal java.lang.OutOfMemoryError being thrown. To guard against this sitation I yesterday wrote a special LimitedInputStream class. You just use it like this:
Document document = builder.parse( new LimitedInputStream( rsd.openConnection().getInputStream(), 100 \* 1024) );If the file you download exceeds the size in bytes passed to the constructor an IOException is thrown. In my case I am trying to parse Real Simple Discoverability xml files, which ought to be very small (much less than the 100k I allow for).
Very simple, but I am sure, very useful. This has probably been done before, but anyway, it's released under a BSD licence, so anyone can use it. Feedback appreciated.
Update: I just realised that I could have used the URLConnection.getContentLength() method to do this! Duh.----
Update: Tim Bray tells me that the Content Length is not a reliable indicator of the length of the message, and is not allways present. So it is still a good precautionary step to use the LimitedInputStream or the LimitInputStream developed by the jxta group (thanks to Mike Duigou).