Category Archives: Articles in English

REST Fire

Let me introduce you to my latest pet project – REST Fire. It’s basically a thin wrapper around Apache HTTP client, providing simple DSL for REST service testing. You can test your REST APIs like this:

        fire().get()
                .to("https://www.google.com/search?q=rest-fire")
                .withHeader("Accept", "text/html")
        .expectResponse()
                .havingStatusEqualTo(200)
                .havingHeader("Content-Type", hasItem(startsWith("text/html")))
                .havingBody(containsString("rest-fire"));

And that’s basically all. If you know REST Assured library, you may be asking why I just do not use REST Assured and write something new instead. Apart from NIH syndrome, I have two reasons.

First of all, I do not like REST Assured syntax, where you define the request, then you validate the response and then again you send the request. REST Fire is more straightforward, just fire a requst and validate the response. The syntax is strongly inspired by Jadler, so if you want to mock backends and test your API at the same time, you do not have to switch between two different styles.

Secondly, REST fire is built on top of Apache HTTP client and lets you configure whatever you want. It’s even capable to use our exotic authentication mechanism.

Usage of Apache HTTP client has its downsides too. It is hard to simulate invalid requests. Apache HTTP client tries to make sure that the requests are valid and if you need to test reactions on invalid requests, you are out of luck.

So if you want to try REST fire, visit its GitHub page for more details and let me know what you think about it.

Asynchronous servlet processing

Welcome to new episode of fun with threads. Today we will cover the exciting topic of asynchronous servlet processing. In the last episode we have learned that it’s not a problem to start more than 10,000 threads in modern JVM, today we will play with asynchronous “threadless” model.

The first important thing thing to note is that we still need a thread everytime our code is doing something. If you have your code in front of you, you always need a thread to move between the lines of the code. But if the code is waiting for something like IO, you do not need a thread and we can leverage that. We have to realize that the vast majority of Java applications act as a middleman – they just toss data from one side to another. From database to HTML, from proprietary back-end to SOAP WS, from REST service to another REST service. You get the picture. I usually do not write chat applications that need to keep a connection open and wait for an event. I usually end-up writing a classical request-response applications that just need to handle lot of requests.

Let’s examine how such applications work. What are the characteristics of such applications? First thing to notice is that they are waiting for most of the time. They are waiting for an incoming request. When the request comes, they do some processing, validation and transformation and then call a back-end. And than they wait again, this time for the response. When the response arrives, our application does some transformation and sends the response back. And then it waits again.

Request processing

Of course it’s application dependent, but in most cases there are few milliseconds of work followed by tens or hundreds of milliseconds of waiting time. You do not have to block a thread when you are waiting. Even though the threads are pretty cheap, we do not want to waste them.

But as is usually the case, there are some tradeoffs. The traditional threaded model is easier to reason about, the asynchronous model is harder to grasp and easier to mess-up. I am not talking about shared state and similar stuff, I am just talking about the fact that my brain works better when I can read the code in a linear way an not thinking about jumping here and there.

Take the following example of a naive proxy using HTTP client 4

log("Servlet no. {} called.", number);
HttpGet backendRequest = createBackendRequest(req, number);
HttpResponse result = client.execute(backendRequest);
copyResultToResponse(result, resp);
log("Servlet no. {} processed.", number);
log("Servlet no. {} returning.", number);

Full source code

It’s evident what’s going on. I create a back-end request, then I execute it and then process the result. It’s straightforward.

Now let’s take a look at the asynchronous version. The easiest way is to delegate the asynchronous processing to libraries. On the top we will use Servlet 3 asynchronous processing, on the back-end side we will use HttpClient asychronous capabilities. Luckily both features play well together, so I can use this code.

log("Servlet no. {} called.", number);
HttpGet backendRequest = createBackendRequest(req, number);
//start async processing
final AsyncContext asyncContext = req.startAsync(req, resp);
client.execute(backendRequest, new FutureCallback<HttpResponse>() {
       @Override
       public void completed(HttpResponse result) {
              ServletResponse response = asyncContext.getResponse();
              copyResultToResponse(result, response);
              log("Servlet no. {} processed.", number);
              asyncContext.complete();
      }
      // error handling removed for brevity
           
});
log("Servlet no. {} returning.", number);

Full source code

Well it’s kind of readable. I can imagine that readability could be better in modern languages. What I do not like is that’s hard to grasp the code. Try it for yourself, what will be the order of log messages? It can be

a) Servlet no. 1 called. – Servlet no. 1 processed. – Servlet no. 1 returning.
b) Servlet no. 1 called. – Servlet no. 1 returning. – Servlet no. 1 processed.
c) Servlet no. 1 returning. – Servlet no. 1 called. – Servlet no. 1 processed.

What’s your answer? It’s of course the second one. The first one is for the synchronous version and the third is just made-up. Why the second version? It’s easy. By calling req.startAsync(req, resp) I say that I am starting the asynchronous processing. Then I execute back-end call but I put the response processing to a callback. So it’s processed after the back-end response arrives. Now I have finished the request execution and the request processing thread provided by the servlet container finishes the execution. Of course it has to unroll the whole stack. So it goes through all your filters as if the processing has already finished.

But it has not. It’s still waiting for the back-end response. Once the response arrives, the HttpClient calls the callback using its own thread. It has to, we have already returned the servlet thread to the container.

In the callback we process the response and by calling asyncContext.complete() we say to the servlet container that we have really finished. When we leave the callback, HttpClient will use the same thread to process another back-end response.

So in reality, the log from asynchronous call is

ServletThread - Servlet no. 1 called.
ServletThread - Servlet no. 1 returning.
HttpClientThread - Servlet no. 1 processed.

It’s easy to understand it once you get used to it. But you have to be aware of few downsides. The first is, that servlet filters do not work as you are used to. Do you want to do a response postprocessing in a filter? You are out of luck. You can use asyncContext.dispatch() method, but it adds even more complexity. Better solution may be to use framework support like the one in Spring MVC that can help you with the task.

The other downside is that you have to think more about the threads and sizing of thread pools. For example, if we have more time-consuming response processing, we have to add more threads to the HttpClient thread pool.

The third downside is that if you do not have support in the client library as we had in HttpClient, you have to take care about the threading yourself. Not only the library needs to support asynchronous calls, it has to support callbacks. If it uses Futures, you have to figure out how to get the result from the future and process it. Again, you will end up with some threading work.

So, if the asynchronous model is so complicated why would we want to use it? Why do not we just use the old synchronous model? The truth is that the synchronous model make sense for most of the use-cases. It is able to process few thousands of parallel requests and it is usually enough for most sites.

But if you have really massive load and you have to utilize you hardware as much as possible, asynchronous processing performs much better. In my test, I recursively call a servlet which in turns calls itself which calls itself until some limit is reached. Jetty is able to process 20,000 connections in less than 15 seconds (after some warm-up when it creates all the connections needed and then just reuses them). Keep in mind that under this test all connections are kept open. So effectively, in the middle of the test I have 20,000 connections open on servlet side and 20,000 connections open on the HTTP client side. All of this with 30 threads including JVM housekeeping threads. All of this on my five-year-old laptop with less than 1.5GB of heap. Pretty cool.

You can try it for yourself the source code is available here.

How NOT to implement caching and optimistic locking in REST

Today I have encountered an interesting REST API. It looked like this one

GET /objects/{objectId} % Object resource - returns 302 (Found) redirecting to the latest version
GET, PUT /objects/{objectId}/versions/{versionId} % Version resource - getting, changing current status of the object

We have two connected resources. I will call the first one the object resource. If you call GET on the object resource /object/123 you always get redirected to the other resource – version resource. It contains the most recent version /object/123/versions/1. This resource is immutable, a new version is created every time someone updates the resource.

At first glance such API has lot of advantages. First of all, if you use a good HTTP library, the redirect from the object resource to its most recent version happens automatically. You do not have to care about it. You just send request to the object URI and get the response. Moreover, you can cache the most recent version of the document. Next time when you want to retrieve it, you just send request to /object/123, get the version resource URI and if the resource has not changed you can get the most recent version from the cache. The caching can be provided by standard HTTP proxies, server or the client. The version resource is immutable, so it can be cached indefinitely.

Not only that, you get optimistic locking for free. You send the PUT request to the most recent version. If the object has been modified by someone else in the meantime, you get an error. Cool.

The problem is that such API does not make sense if you do not need access to old versions of the object. If you do not keep track of changes and have only current version of the resource, this API is plainly wrong.

Why? The first problem is what to do if someone wants to access an older version of the object. Return an error? We do not have the old value any more, so it has to be an error. But now the resource is not immutable. It’s inconsistent and unpredictable. We can get an error one time and an old version from cache another time.

What’s worse, by having to do two HTTP calls to get the status of the resource, we can get to a race condition. Imagine the following scenario. Both client A and client B call GET /objects/123. Both of them get redirected to /objects/123/versions/1. Client A gets the resource representation, modifies it and sends PUT to /objects/123/versions/1 thus changing its state. Client B is kind of slow so the redirected call to GET /objects/123/versions/1 will be sent after client A changed the value. Therefore client B gets an error because he is trying to access an old version that is no more accessible. The redirect is no longer transparent! The client has to be prepared for such situations and manually implement a retry mechanism just to get the resource representation!

There are other shortcomings. The main motivation for such solution is to leverage HTTP caching. But we have to always call the object resource first to get the URI of the latest version. Only after that a cache can be used. So we sometimes end-up with 2 HTTP calls instead of one. If the resource had been implemented in a simple way, only one HTTP request would have been needed.

The biggest problem of this solution is that it attempts to solve in a clever way something that has been solved long time before in HTTP standard.

I am talking about ETag. We do not need the version resource. We can return the resource representation directly from the object resource /object/123. If we want to leverage caching and optimistic locking, we can return ETag header which describes current state of the object. Basically it will have exactly the same value as versionId we have used in the our version resource.

Now if the client or a proxy has an old value in the cache, it can send GET to /object/123 with If-None-Match: ETagValue HTTP header. If the resource state has not changed, the server will send 304 (Not Modified) and no HTTP body. If the resource has changed, new representation will be returned. We have to always do only one HTTP request. If the server value is the same as cached value, no response body is sent in the response so it saves some bandwidth and processing power.

Optimistic locking (a.k.a Conditional Update) can be done in similar way. Just send PUT to /object/123 with If-Match: ETagValue header. If the ETag value correspond to the resource state, the update will be performed. If the resource has been changed in the meantime, the ETag value will not correspond and the server will return 412 (Precondition Failed).

It does everything as the versioned API, but better. What’s more, it is described in HTTP standard so there is high chance that it will be supported by HTTP client libraries, proxy servers and application frameworks.