How NOT to implement caching and optimistic locking in REST

Today I have encountered an interesting REST API. It looked like this one

GET /objects/{objectId} % Object resource - returns 302 (Found) redirecting to the latest version
GET, PUT /objects/{objectId}/versions/{versionId} % Version resource - getting, changing current status of the object

We have two connected resources. I will call the first one the object resource. If you call GET on the object resource /object/123 you always get redirected to the other resource – version resource. It contains the most recent version /object/123/versions/1. This resource is immutable, a new version is created every time someone updates the resource.

At first glance such API has lot of advantages. First of all, if you use a good HTTP library, the redirect from the object resource to its most recent version happens automatically. You do not have to care about it. You just send request to the object URI and get the response. Moreover, you can cache the most recent version of the document. Next time when you want to retrieve it, you just send request to /object/123, get the version resource URI and if the resource has not changed you can get the most recent version from the cache. The caching can be provided by standard HTTP proxies, server or the client. The version resource is immutable, so it can be cached indefinitely.

Not only that, you get optimistic locking for free. You send the PUT request to the most recent version. If the object has been modified by someone else in the meantime, you get an error. Cool.

The problem is that such API does not make sense if you do not need access to old versions of the object. If you do not keep track of changes and have only current version of the resource, this API is plainly wrong.

Why? The first problem is what to do if someone wants to access an older version of the object. Return an error? We do not have the old value any more, so it has to be an error. But now the resource is not immutable. It’s inconsistent and unpredictable. We can get an error one time and an old version from cache another time.

What’s worse, by having to do two HTTP calls to get the status of the resource, we can get to a race condition. Imagine the following scenario. Both client A and client B call GET /objects/123. Both of them get redirected to /objects/123/versions/1. Client A gets the resource representation, modifies it and sends PUT to /objects/123/versions/1 thus changing its state. Client B is kind of slow so the redirected call to GET /objects/123/versions/1 will be sent after client A changed the value. Therefore client B gets an error because he is trying to access an old version that is no more accessible. The redirect is no longer transparent! The client has to be prepared for such situations and manually implement a retry mechanism just to get the resource representation!

There are other shortcomings. The main motivation for such solution is to leverage HTTP caching. But we have to always call the object resource first to get the URI of the latest version. Only after that a cache can be used. So we sometimes end-up with 2 HTTP calls instead of one. If the resource had been implemented in a simple way, only one HTTP request would have been needed.

The biggest problem of this solution is that it attempts to solve in a clever way something that has been solved long time before in HTTP standard.

I am talking about ETag. We do not need the version resource. We can return the resource representation directly from the object resource /object/123. If we want to leverage caching and optimistic locking, we can return ETag header which describes current state of the object. Basically it will have exactly the same value as versionId we have used in the our version resource.

Now if the client or a proxy has an old value in the cache, it can send GET to /object/123 with If-None-Match: ETagValue HTTP header. If the resource state has not changed, the server will send 304 (Not Modified) and no HTTP body. If the resource has changed, new representation will be returned. We have to always do only one HTTP request. If the server value is the same as cached value, no response body is sent in the response so it saves some bandwidth and processing power.

Optimistic locking (a.k.a Conditional Update) can be done in similar way. Just send PUT to /object/123 with If-Match: ETagValue header. If the ETag value correspond to the resource state, the update will be performed. If the resource has been changed in the meantime, the ETag value will not correspond and the server will return 412 (Precondition Failed).

It does everything as the versioned API, but better. What’s more, it is described in HTTP standard so there is high chance that it will be supported by HTTP client libraries, proxy servers and application frameworks.

Spring and string list injection

This week, I have been bitten by a surprising behavior of Spring type conversion. Let’s have the following bean and Spring config.

public class Bean {
    public void setArray(String[] values) {
        System.out.println("array: " + Arrays.toString(values));
        System.out.println(values.length);
    }

    public void setList(List<String> values) {
        System.out.println("list: " + values);
        System.out.println(values.size());
    }
}

<bean class="net.javacrumbs.Bean">
    <property name="array" value="item1, item2"/>
    <property name="list" value="item1, item2"/>
</bean>

We are trying to inject a String containing comma delimited list of values. The question is whether the String will be automatically converted to list of multiple items in array/list or only one item string will be injected. Both options make sense. What do you think? What will be the length of the array/list? One or two?

The answer is surprising. The array value will be split to two Strings whereas the List will have only one element. The output will be:

array: [item1, item2]
2
list: [item1, item2]
1

It’s an easy error to make but hard one to spot. If you log the values, the output is undistinguishable. If you look at the output above, the values are exactly the same. What’s worse you usually do not have unit tests to test Spring configuration. In my case the bug got as far as production environment. On test environments we have got only one value in the property so the bug went unnoticed. On the production environment we have set a comma separated list of values. And of course, I have expected the array behavior while using list parameter. Silly me.

Manažerské strasti

Udělal jsem velikou chybu. Nechal jsem se uvrtat do role podobné projektovému manažeru. Jak byste asi čekali moc mi to nejde ani mě to nebaví. Tak si tu chci napsat co to obnáší, abych už příště podobnou bláhovost neudělal.

Schůze
Kalendář někoho kdo komanduje lidi je úplně jiný než člověka co dělá nějakou užitečnou práci. Pěkně je to popsáno tady. Jako manažer prostě potřebujete komunikovat s mnoha různými lidmi. Z čehož plyne oblíbená manažerská kratochvíle – schůzky. Jako programátor je musíte tiše trpět, jako manažer je svoláváte. A to i několik za den. Snažím se je omezovat na minimum a na co nejkratší čas, ale i tak je jich dost. Navíc mám pořád blbý pocit, že ostatní vytrhávám od práce.

Nuda
Nejhorší ale je, že schůzky jsou na té práci nejzajímavější. Ano, čtete správně. Schůzky jsou zajímavé. Jako manažer totiž nemáte moc jiné práce. Teda pokud nepočítám těch pár mailů. Ono to plyne z definice. Řízení lidí je hlavně o komunikaci. Takže buď mailujete, plánujete schůze nebo na nich sedíte. Pokud ale neřídíte víc projektů na jednou, tak mezi tím máte spoustu volného času. Takže sedíte v práci a nemáte do čeho píchnout.

Programování a řízení nejde dohromady
To že má člověk volný čas se dá brát jako pozitivum. Ve volných chvílích se dá udělat něco užitečného. Třeba programovat. Zkusil jsem to, a zjistil jsem, že to nezvládám. Jedna nebo druhá činnost tím trpí. U mě bohužel trpí obojí. Když programuji, tak nechci být vyrušován. Natož pak sám sebou. Takže manažerské aktivity odkládám nebo jinak zanedbávám. Největší legrace pak je, když v hlavě řešíte programátorský problém a přitom sedíte na schůzi, kterou máte řídit.

Odpovídáte za věci, které nemůžete ovlivnit
Řídící role sebou nese také odpovědnost. A to i bohužel za věci, které nemůžete ovlivnit. Jako programátor můžete s klidem říci, není to hotové, protože Franta nedodal XYZ i když jsem mu to každý den připomínal. Jako manažer za to nesete zodpovědnost, a to i přesto, že nemáte žádnou pravomoc Frantovi říkat co má dělat. Musíte se pokoušet o řešení, i když víte, že žádné neexistuje. Je to prostě vaše práce.

Když to tak po sobě čtu, tak nevidím důvod, proč by člověk, kterého baví programování, chtěl dělat manažera. Já to tenkrát vzal, protože jsem si to chtěl zkusit. Je to zajímavá zkušenost, ale mě osobně to nezaujalo.