Streaming a stream

I have another dirty trick for you. Let's imagine that you have a document server with following sophisticated interface.

public interface DocumentServer {
	/**
	 * Stores a document.
	 * @return document id
	 */
	public String storeDocument(InputStream in);
	
	/**
	 * Loads document with given id.
	 * @param id
	 * @return
	 */
	public InputStream loadDocument(String id);
}

It works well, until you realize, that you want to call the DocumentServer remotely, for example using Spring HttpInvoker.

HttpInvoker is a nice small tool, that lets you make remote method invocations over HTTP without much pain. It creates dynamic proxy, serializes all attributes using Java serialization, calls remote proxy, which deserializes arguments, calls the implementation and sends the result back in the same way. It is nice, simple and it works without any problem. You can configure it in five minutes. The downside is, that you have to have Spring Java application on both ends of the wire and all arguments that you send over the wire have to be Serializable.

And that's exactly the trouble we have with our DocumentServer. There is no standard InputStream implementation that implements serializable. Lets think about it. We need to have a stream that is able to stream itself into an ObjectOutputStream. Sounds strange, but it is quite easy. We can simply write a wrapper, that implements writeObject(ObjectOutputStream) method and that takes all the bytes from the underlying stream and writes them to the ObjectOutputStream. It is simple and it does not consume much memory.

Deserialization is much harder. We have to implement readObject(ObjectInputStream in) method, load the data and store them somewhere for later use. We can store them in the memory or in a temporary file. If the streams are larger, temporary file is better choice but we have to delete them after use and the implementation gets messy.

The good news is, that we do not have to implement it, it's already done by RMIIO library. They provide SerializableInputStream class that does exactly what we need. You just have to ignore Javadoc comment saying:

An additional layer around a RemoteInputStream which makes it Serializable and an InputStream. In general, this extra layer is not necessary and I do not recommend using this class. However, in the odd case where the callee really wants to get something which is already an InputStream, this class can be useful.

The only thing we have to do is to wrap all the streams into SerializableInputStream like this

new SerializableInputStream(new DirectRemoteInputStream(inputStream));

It can be done by hand written proxy, by dynamically generated proxy or by AOP.

Of course there are other alternatives. The easiest one is to create something like ByteArayInputStream that implements Serializable. If the streams are small enough, it's the best solution.

We can also try to use Hessian, but I had some troubles when method signatures were more complicated. And of course there is still a possibility to implement own remoting mechanism. After all, HTTP is meant to transport streams. But it would require lot of boilerplate code and I do not like boilerplate code.

Tags: , , , , ,

3 Responses to “Streaming a stream”

  1. Michael Zhong Says:

    Hi, I'm confused with the RemoteInputStreamMonitor when sending a file from a client to a server and want to know how many bytes have been sent. Would you like to show a simple example about how to use it? Thank you very very much :)

  2. Lukáš Křečan Says:

    If you need just counting of bytes, you can use commons-io library https://commons.apache.org/io/api-1.4/org/apache/commons/io/output/CountingOutputStream.html

  3. wendel castro Says:

    I need one simple example, transfer file 5GB rmi, my example transfer 2GB....thank you very much

Leave a Reply

Current day month ye@r *