Java crumbs » Articles in English

Maven, Eclipse, WTP and project dependencies

Lukáš Křečan — Mon, 05 Sep 2011 19:35:45 +0000

If you are working with Maven, Eclipse and WTP, you can encounter different problems.

One of them is connected to project dependencies. Imagine that you have an in-house library A and a web application B that depends on A. Sometimes, when I run the webapp in a server using WTP, it can not find classes (or resources) from B. It's quite confusing if you get a ClassNotFoundException even though the class is apparently there. It happens only if you are using m2eclipse extras. If you have different configuration this solution will not probably help you.

Ok, so you have m2extras and you are getting ClassNotFoundException. What next? You have to check file A/.settings/org.eclipse.wst.common.component (I use the Navigator View to access the file)

If you see something like this

you have found the issue. The source-path attribute is wrong. It's necessary to set the value to /src/main/java. Sometimes the path is not just wrong, but it's completely missing. In such case just add a new line with corresponding configuration and everything should work as expected.

Mock socket

Lukáš Křečan — Sat, 27 Aug 2011 12:56:47 +0000

Few months ago I have written a small tool that mocks network sockets in Java. Now I have some time to describe it, so here you are.

Let's imagine you want to test network communication in Java. It's not easy, you have to start some server on the other side, configure its responses and somehow verify that the data you send are correct.

With mock-socket it's incredibly easy.

import static net.javacrumbs.mocksocket.MockSocket.*;
...

//prepare mock
byte[] dataToWrite = new byte[]{5,4,3,2};
expectCall().andReturn(emptyResponse());

//do test
Socket socket = SocketFactory.getDefault().createSocket("example.org", 1234);
IOUtils.write(dataToWrite, socket.getOutputStream());
socket.close();

//verify data sent
assertThat(recordedConnections().get(0), data(is(dataToWrite)));
assertThat(recordedConnections().get(0), address(is("example.org:1234")));

You see, just statically import MockSocket class, prepare the mock, execute the test and verify the data. The library just removes the default Java socket implementation and place a mock implementation in its stead.

Of course, this example does not have much sense. It just tests that Java socket implementation works. But imagine that you implement some non-trivial network library. A test library can be handy.

Moreover, there is a HTTP extension which can be used if you want to test some HTTP client. Let's say a JSON REST client. In such case, you can write this.

import static net.javacrumbs.mocksocket.http.HttpMockSocket.*;

...

//prepare mock
expectCall()
  .andWhenRequest(uri(is("/test/something.do")))
     .thenReturn(response().withStatus(404))
  .andWhenRequest(uri(is("/test/other.do")))
    .thenReturn(response().withContent("Text")).thenReturn(response().withContent("Text"));

//do your test
...

//verify
assertThat(recordedConnections(), hasItem(header("Accept", is("text/plain"))));

Ain't great? You can do much more, please take a look at the project page if you are interested.

Converting JSON to XML

Lukáš Křečan — Wed, 06 Jul 2011 11:12:48 +0000

I have been working on a validation tool that would allow us to validate JSON messages. I could not any usable JSON validator in Java so I have decided to use RelaxNG. It had one catch. RelaxNG is designed for XML validation, not JSON.

That's the reason why I have decided to convert JSON to XML. Before we dive into the implementation, let's think about the problem in general. Both JSON and XML ale tree structures and they are quite similar. In fact, I have discovered only the following differences.

JSON supports multiple top-level elements. In XML we can have only one root element. While {"a":1, "b":2} is a valid JSON document, 12 is not valid XML.
JSON supports arrays, XML does not.
JSON has null keyword, in XML we several options how to deal with nulls. We can either represent it by an empty element or using xsi:nil attribute.
JSON has support for boolean and numeric types. In JSON {"a":1} has different meaning than {"a":"1"} (with double quotes). It's not possible to represent such difference in the XML.

All of the problems were acceptable for my needs. There would be more problems when going in the other direction, from XML to JSON, but I did not need it. The only thing I have needed was to convert from

{"root":{
	"data1":[
		[1,2,3],
        	[4,5,6]
	],
	"data2":null,
	"data3":"2011-05-30T10:00:00",
	"data4":
	{
		"a":1,
     		"b":2
	}
}
}



	
		
			1
			2
			3
		
		
			4
			5
			6
		
	
	
	2011-05-30T10:00:00
	
		1
		2

There already is a project Jettison, that is able to convert between JSON and XML but it does not handle JSON arrays properly. Moreover, I have encountered some bugs like this one. It's trivial to fix but it's there for more than half a year. Strange.

One nice Saturday morning I have decided to implement it. And I have found that it's incredibly easy. It's just matter of using Jackson pull parser and generating SAX events based on it. It's about 200 lines of code that you can enjoy here here.

Using Jing for Relax NG validation

Lukáš Křečan — Mon, 02 May 2011 21:21:51 +0000

I just ant to jot down how to programmatically use Jing for Relax NG validation. It has a simple API but I could not find any description besides the JavaDoc. It took me some time to figure out how to use it.

If you want to validate a XML file, just use ValidationDriver. If you have a DOM Document or StAX source, just do the following

//we are using Relax NG compact format
SchemaReader schemaReader = CompactSchemaReader.getInstance();

//schema can be reused, it's thread safe
Schema schema = schemaReader.createSchema(ValidationDriver.fileInputSource(new File("your_schema.txt)), PropertyMap.EMPTY);

//can use different error handler here (try DraconianErrorHandler http://www.thaiopensource.com/relaxng/api/jing/com/thaiopensource/xml/sax/DraconianErrorHandler.html)
ErrorHandler seh = new ErrorHandlerImpl();
PropertyMapBuilder  builder = new PropertyMapBuilder();
builder.put(ValidateProperty.ERROR_HANDLER, seh);

//Validator is NOT thread safe
Validator validator = schema.createValidator(builder.toPropertyMap());

Source source = ...//your XML source

TransformerFactory.newInstance().newTransformer().transform(source, new SAXResult(validator.getContentHandler()));

And that's it.

Visualizing Fork/Join

Lukáš Křečan — Sun, 27 Mar 2011 18:20:16 +0000

Last Friday I have participated on GPars Hackathon. Together with Jety we have picked the task "Catchy visual demos". Neither of us had used Swing for several years and we are newbies in both Groovy and GPars. But Václav helped us to solve several Groovy mean tricks and at the end we managed to create really nice demo. It's a visualization of Fork/Join based merge-sort. You can enjoy it here. You can really see how fork/join aka jsr166y works. Cool.

THE DEMO (alternatively you can download and execute this jar)

The source code can be downloaded here although it's much less enjoyable than the application itself. I have to confess that I enjoyed it so much that I have rewritten it to Java and in fact that's what you are really looking at. In Java it's less elegant, but you do not have to download 5Mb of Groovy libraries. I also hoped that in Java it would run in sandbox without special permissions. Unfortunately the jsr166 library uses sun.misc.Unsafe class which is not only unsafe but also from nonpublic API so it is not possible to run the demo in the sandbox. That's why you have to trust my self-signed certificate. The Java source can be downloaded from GitHub.

WS testing

Lukáš Křečan — Tue, 08 Mar 2011 17:32:04 +0000

This post recaps some results of my experiments with WS testing. In last few months I have been working on several projects – Spring WS Test, Spring WS 2.0 testing support and finally on Smock. All of them reflect different approaches to WS testing. Today I am going recapitulate my findings focusing on testing producer/server side code. Consumer/client side will be covered in a following post.

Simple service, simple test

You may wonder what's the problem? Testing web services is easy. We just need to call the endpoint end verify the response. For a simple web service we can write a simple test like this

@Test
public void testPlus()
{
	assertEquals(3, endpoint.plus(1, 2));
}

The test is same as if I was testing a normal method.

Web Services tend to be complex

The trouble is that web services tend to be complex. Normal Java methods usually have few relatively simple parameters. On the other hand, web service request or response could be quite complicated which makes the service endpoint hard to test.

GetFlightsRequest request = new GetFlightsRequest();
request.setFrom("PRG");
request.setTo("DUB");
request.setServiceClass(ServiceClass.BUSINESS);
request.setDepartureDate(DatatypeFactory.newInstance().newXMLGregorianCalendarDate(2011, 02, 23, 0));
//more setter calls could be here

GetFlightsResponse response = endpoint.getFlights(request);

List flights = response.getFlight();
assertEquals(1, flights.size());
assertEquals(ServiceClass.BUSINESS, flights.get(0).getServiceClass());
//more assertions here

Such tests are bearable only until certain level of complexity. From certain level it's real pain. I really hate to call twenty setters to construct several levels of nested XML hierarchy and then to call dozens of getters to validate the response.

To define XML use ... the XML
The problem is that we want to create or validate an XML structure using wrong language. Java is just not suitable for the task. The best language to describe XML is, surprise, surprise, the XML. Would not it be easier to create XML files containing the request and response and just use them in the test?

GetFlightsRequest request = JAXB.unmarshal(getStream("request1.xml"), GetFlightsRequest.class);

GetFlightsResponse response = endpoint.getFlights(request);

DOMResult domResponse = new DOMResult();
JAXB.marshal(response, domResponse);

XMLUnit.setIgnoreWhitespace(true);
XMLAssert.assertXMLEqual(getDocument("response1.xml"), (Document)domResponse.getNode());

It's much better. This test will look exactly the same way regardless the complexity of the service. But it still have some shortcomings. The biggest one is that usually testing using only one request is not sufficient. I'd like to test the service using different inputs. In such case I have to create lot of XML files that differ just in few elements. It's a maintenance nightmare. If you change the service, you also have to change all the XML files.

Smock comes to help
Up until now we managed to test without any special framework except XMLUnit. Now we will start to use my precious Smock library. The test above can be rewritten to this form.

Map params = new HashMap();
params.put("from", "DUB");
params.put("to", "JFK");
params.put("serviceClass", "economy");
GetFlightsRequest request = createRequest(
					withMessage("request-context-groovy.xml")
					.withParameters(params), GetFlightsRequest.class);

GetFlightsResponse response = endpoint.getFlights(request);

validate(response, request)
	.andExpect(message("response-context-groovy.xml")
	.withParameter("serviceClass", "economy"));

The test is quite similar as the previous one. The difference is that it uses statically imported method createRequest to create a request and validate method to validate the response. The main advantage is that Smock methods support templates. So we can set just values we want to, the rest can be driven by the template. Moreover we do not need to deal with the request structure, parameters can be set into a simple flat map.


	${from}
	${to}
	2001-01-01
	${serviceClass}

Templates are even more useful when comparing the response


	
		OK1324
		2011-02-19T10:00:00
		
			${GetFlightsRequest.from}
			${IGNORE}
			${IGNORE}
		
		2011-02-19T12:00:00
		
			${GetFlightsRequest.to}
			${IGNORE}
			${IGNORE}
		
		${serviceClass}

We can ignore some elements, we can compare values based on the request elements and so on. Moreover, when we are using Smock library, we can leverage Spring WS 2.0 testing capabilities that Smock builds on. We can for example validate that the response is valid regarding to its schema.

validate(response).andExpect(validPayload(resource("xsd/messages.xsd")));

Cool, isn't it?

Testing the configuration
Sometimes unit test are not sufficient. Sometimes we want to test the configuration, interceptors, error handling and so on. No problem. If you are using Spring WS 2, you can use the built-in testing library. If you need templates or if you are using different framework like Axis 2, you can use Smock. The test looks similar.

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration("file:src/main/webapp/WEB-INF/spring-ws-servlet.xml")
public class GroovyEndpointTest {

	private MockWebServiceClient wsMockClient;

	@Autowired
	public void setApplicationContext(ApplicationContext context)
	{
		wsMockClient = createClient(context);
	}

	@Test
	public void testResponseTemplate() throws Exception {
		Map params = new HashMap();
		params.put("from", "DUB");
		params.put("to", "JFK");
		params.put("serviceClass", "economy");
		wsMockClient.sendRequest(
			withMessage("request-context-groovy.xml")
			  .withParameters(params))
			.andExpect(message("response-context-groovy.xml")
			  .withParameter("serviceClass", "economy"));
	}

The only difference is that we have to bootstrap the application context. Having done this, we can use the mock web service client to call our services. Even though this kind of tests is more complicated, we can use it to test object-XML mapping, endpoint resolution, error handling and configuration in general.

We have seen several different methods of WS testing. Each of them have advantages and disadvantages. If you keep you services simple, your tests are simple too. If the service is more complex, it makes sense to use XML for the test. Either using your own helper methods, Spring WS 2.0 test support or Smock.

All the samples could be downloaded from here.

Where does this class came from?

Lukáš Křečan — Mon, 01 Nov 2010 21:18:09 +0000

Today I have been debugging some nasty problem on Weblogic. The application worked on one version but did not work on an older one. Apparently some classes were not compatible. But it's quite difficult to debug such problem. You need to find out from which JAR given class was taken from. It's not trivial in enterprise environment. The class can be part of J2SE, application server and your EAR at the same time. Especially if the class has something to do with XML or web services. End if you have such class, you need some trick to determine which version was actually used.

The trick is simple yet not widely known. In fact I have encountered it in a Scala related article. You just call this code from JSP or some other suitable place.

YourClass.class.getProtectionDomain().getCodeSource().getLocation()

And that's all, this command will return path to your mystery class JAR.

Kindlology

Lukáš Křečan — Fri, 01 Oct 2010 10:49:13 +0000

Most readers get as far as the Future Semiconditionally Modified Subinverted Plagal Past Subjunctive Intentional before giving up; and in fact in later editions of the book all pages beyond this point have been left blank to save on printing costs.

Hitchhiker's Guide to the Galaxy

I am new Kindle user and the more I use it, the more I like it. Although it probably will never have the same feeling as reading a book there are two features that gives it some edge.

It's always with me. Even if I do not have it in my backpack, I do have my Android cell phone with Kindle app installed. And what's best, the application knows where I have stopped reading and I can simply continue.
Highlights and bookmarking are much better. Even if I have pencil when reading a book the highlights are hard to find. I love to quote authorities. But it's always hard to find the right page even though I have underlined it. Kindle solves it quite well. You can highlight whatever you want, you can search the notes and highlights, you can access it on-line and you can even tweet it right from the device. You can even see paragraphs frequently highlighted by other readers! (Although I am thinking about switching this feature off, it's a bit disturbing.)

But I do not want to praise Kindle here, I'd like to think about the consequences. There is the other side of the coin. To be able to implement this features, Amazon has to know about every page turn you make. It brings some privacy issues, but I do not want to talk about that neither.

I'd like to imagine how such data could be used for “scientific” purposes. I'd like to know what's my reading speed. Is it lower in the evening? Is it faster when reading thrillers? What's European reading speed? Is it higher than American? Do I skip paragraphs? Do other readers skip the same paragraphs? Is there some part of given book that force people to take a break? Do people make more notes at the beginning of the book? Is there a book that readers usually do not finish? What's the finish rate in general? Are there some books that people just buy but do not even start to read?

You see, there is lot of questions that we can finally find answers for. The only think we need is to persuade Amazon to publish the data and some PhD students to do the statistics. Even without that the data give some interesting result right now. You can see that readers make highlights in fiction books like Eat Pray and Love. I would not expect that. You can see it yourself on http://kindle.amazon.com

PS: I am not the first one who had the same idea, here is one of my predecessor's articles.

Writing fluent API

Lukáš Křečan — Sat, 10 Jul 2010 13:12:19 +0000

In recent weeks I have been working with Arjen Poutsma on a brand new testing support for Spring Web Services. At least Arjen has been working, I have been just generating crazy ideas and he kept explaining me why we can't use them.

But it was quite interesting experience and I have learned a lot about fluent API design. Since it was my first fluent API and I am not an API expert, please take everything I mention here with grain of salt.

The task is simple. We need a way how to configure a mock. This mock has to validate that requests generated by our application are correct and after that the mock has to return some response. We want to write something like this:

Computer, when someone is trying to send a request, verify that the request contains value “a”, has header “header” and then respond with value “b”.

The requirements for the API are following:

It has to be simple for users. Mainly it has to be friendly to GUI addicts like me, that rely on code-completion all the time.
The library itself has to be simple
The API should be as fluent as possible
It has to be extensible

In the next part, I will try to describe several variants of the solution, each of them fulfilling the requirements differently. Please keep in mind that the code is simplified, it focuses solely on the API, not on the implementation. Moreover, the classes have been renamed to show what their main purpose is.

All characters appearing in this work are fictitious. Any resemblance to real persons, living or dead, is purely coincidental.

Interface based version

code
The first variant is based mainly on interfaces. The test itself looks like this

	@Test
	public void testMock()
	{
		Mock mock = new Mock();
		mock.whenConnecting().expectValue("a").and().expectHeader("header").andRespondWithValue("b");

		//test code

		mock.verify();
	}

There is one entry point, class Mock

public class Mock {

	public RequestExpectations whenConnecting()
	{
		return new MockInternal();
	}

	public void verify()
	{
		//do something
	}
}

It just returns an internal class that implements RequestExpectations interface.

public interface RequestExpectations {

    ResponseActions expectValue(String value);

    ResponseActions expectHeader(String name);
}

All methods of this interface return ResponseActions interface

public interface ResponseActions {

	RequestExpectations and();

	void andRespondWithValue(String value);

	void andRespondWithError();

}

Please note that and() method allows chaining of request expectation.

This API is really IDE friendly. Code completion works like a charm. It's quite simple both from outside and inside. But this API has one big flaw. It's not extensible. Imagine that we would like to add an expectation that verifies a digital signature. We would have to add a method to the RequestExpectations interface. It is not possible neither for third parties nor for us, since this change would break backward compatibility.

Static factory

code
We can split the functionality and allow others to provide their own validators and response generators. We just have to change the interfaces

public interface RequestExpectations {

    ResponseActions expect(RequestMatcher matcher);
}

and

public interface ResponseActions {

	RequestExpectations and();

	void andRespond(ResponseCallback callback);
}

Now we have made the API extensible, but we have to provide an easy way how to create RequestMatchers and ResponseCallbacks. One way how to do it is a statically imported static factory.

public class StaticFactory {
	public static RequestMatcher value(String value)
	{
		return new RequestMatcher() {
			public void match(Object someParams) {
				//do something
			}
		};
	}
	public static ResponseCallback withValue(String value)
	{
		return new ResponseCallback() {
			public void doWithResponse(Object someParams) {
				//do something
			}
		};
	}
}

The test then can look like this.

import static net.javacrumbs.fluentapi2.StaticFactory.*;

public class MockTest {

	@Test
	public void testMock()
	{
		Mock mock = new Mock();
		mock.whenConnecting().expect(value("a")).and().expect(header("header")).andRespond(withValue("b"));

		//test code

		mock.verify();
	}
}

Such API is also fluent. It even resembles EasyMock API. The issue here is that code completion does not work as well as before. In Eclipse you have to type first letter of the method name before Ctrl+Space starts working. In Idea you have to press Ctrl+Shift+Space. But the advantage is that everyone can provide their own factory and extend the library seamlessly and consistently.

Super-static variant

code
We can go event further. We can get rid of RequestExpectations interface, remove Mock class and do statically most of the stuff. The test can look like this

import static net.javacrumbs.fluentapi3.StaticFactory.*;

public class MockTest {

	@Test
	public void testMock()
	{
		expect(value("a")).andExpect(header("header")).andRespond(withValue("b"));

		//test code

		verify();
	}
}

It's simple and elegant. We even got rid of the strange and() method. The downside is that we usually have to use some ThreadLocals internally. But it looks almost like EasyMock. Cool.

Inheritance

code
Some people (like me) do not like static methods. It does not feel right, it's not object oriented programming. Can we do something similar, but dynamic? Yes, we can.

We just have to use inheritance. Just delete static keywords from the methods in StaticFactory class. We will also rename the class itself because it's not static any more.

We get

public class MockTest extends MockFactory{

	@Test
	public void testMock()
	{
		expect(value("a")).andExpect(header("header")).andRespond(withValue("b"));

		//test code

		verify();
	}
}

The downside is, that we have to extend our class from MockFactory. Sometimes it's not possible. No problem, we can use a trick I have seen in Apache Camel DSL.

public class MockTest{
	@Test
	public void testMock()
	{
		MockFactory mockFactory = new MockFactory()
		{
			protected void configure() {
				expect(value("a")).andExpect(header("header")).andRespond(withValue("b"));
			};
		};

		//test code

		mockFactory.verify();
	}
}

It's not as elegant as the direct inheritance, but it works. If we are brave enough, we can even use a closure.

public class MockTest extends MockFactory{
	@Test
	public void testMockClosure()
	{
		MockFactory mockFactory = new MockFactory()
		{{
				expect(value("a")).andExpect(header("header")).andRespond(withValue("b"));
		}};

		//test code

		mockFactory.verify();
	}
}

(I bet you did not know that Java has closures already).

So I have shown you four variants, which one is the best? I do not know, each of them has it's pros and cons. I personally like the last one the most. But the super static variant is not bad neither. Or the static? Or some mix between them? Some other? You have to make your own opinion.

References:
http://martinfowler.com/bliki/FluentInterface.html
http://martinfowler.com/dslwip/InternalOverview.html
http://camel.apache.org/routes.html
http://easymock.org/

Spring WS Security on both client and server

Lukáš Křečan — Mon, 07 Jun 2010 11:17:13 +0000

Recently, I have been playing with Spring WS with WS-Security. I just want to write down how it works. Do not except anything special, just simple example of basic security operations.

The example

We want to implement both client and server side. The client will sign the message, encrypt some part of it and add a timestamp. To make it more complex and real-life like we will sign the message using private key with alias “client” and encrypt the message using public key called “server”. Server will validate that the request is valid and will just sign the response using his key called “server”. Please note that I have picked Wss4j implementation because the configuration seemed to be easier than Xws.

Client

It's easy to do configure client interceptor like this (full configuration).

...

As you can see, there is nothing special. We just define which actions to take and properties. The only confusing part is, that key alias is defined as “securementUsername”.

Whit this configuration we will get following SOAP message.


	
		
			
				
				
					
						
							
								CN=Test Server,OU=Test
								1275904530
							
						
					
				
				
					fwFM7ShJ1xd7dTGrkh0410sTmW92OPB1q1fpzB21XFIe36siDDJWGgbw5B94yjmGK2YaPOWLb7cpVTYPzc9VUDs7Jc42CtrhT2H6eZ7CDiA60Ugz+qi2UyyfMDK6Vrdj9J68rij5P12AiBeTnd2wlhI29+71XbUpD5weHDHjMtQ=
					
				
				
					
				
			
			
				
					
					
					
						
							
						
						
						AU9utUgz5RylYCRDUAO0JWM48kM=
					
				
				
					NHjjgpb9/alUOq50CqPKLcdYrp7edYdKJDNvIhh+2OAhYdDvZmD1qGsVKd1H9oKPF17uaF2Sv3aY
					0le6BrvzVx3n2+nYYlHwAWlzBk7wsBt4vLll6q6juLCP+siupTIb1PeZDf3WrAbHUQh5oqjD6cZB
					Sc89pDspWRABQ8wPxYE=

				
					
						
							
								CN=Lukas Krecan,OU=Test
								1275900789
							
						
					
				
			
			
				2010-06-07T11:03:35.749Z
				2010-06-07T11:08:35.749Z
			
		
	
	
		
			
				
					
					
						
							
						
					
					
						81TEtUhHXo6iZeAmYrtYlm2ObAqOBpjfzf2VOVUg4Hs=
						
					
				
			
			2

Server config

To configure server, you have to define Spring WS server interceptor like this (full example).

No surprise here neither. The response will look like this.


	
		
			
				
					
					
					
						
							
						
						
						hEdDfxM6Nfs62Jxe8EOsELCDtUk=
					
					
						
							
						
						
						TTSRri5KJqXeMJfjzXyVmUewPxc=
					
				
				
					V5by3bOoGQNajfs7i9xQ+cbAqIkI0NS9N9FQlLb/dAuQfguE7jKRP9iypOeRLHCPr7g3BNg+NCrX
					6YcgDQ0TfXNhdL00AmoEfDmWSNvIVNE49kZEn3Ji/RW4VtdEiV79VD7Vuay0YAYGo9DSQvzq3FP6
					YEhfzfMqvfbWMdEKcO8=

				
					
						
							
								CN=Test Server,OU=Test
								1275904530
							
						
					
				
			
			
		
	
	
		
			3

As we have seen it's possible to configure WS-Security without much hassle. To learn more, visit the official Spring WS reference. You can download full example here.