Development of a new service

From WebLab Wiki
Jump to navigationJump to search

Goal of the tutorial

In this chapter we will describe the development of a new web service with respect to the WebLab model. The development of web services is possible in several languages, in this tutorial we chose to use JAVA. Moreover in each language, and especially in JAVA, multiple solutions exists to implement a web service. In this tutorial we will rely on CXF[1] which is one of the reference implementation of the implementation of the JAX-WS APIs (Java API for XML Web Services). Finally we will use maven[2] as project configuration management and building automation tool. The objective of the new service will be to detect the language of text embedded in WebLab document. To do this we will use an existing API called 'the cngram library'[3]. Next sections will detail :

  1. Creation of a Java application scheme using Maven and WebLab Archetype Plugin
  2. Implementation of the web service class
  3. Packaging and test of the web service

This service will be added into the demo chain in the next chapter.

Maven configuration

First of all, if it's not done, you have to install Maven (see maven documentation[4]). Then, you have to modifying the file <maven directory>/conf/settings.xml. If you already have a profile in your settings, then you just have to add this in the repositories markup :

[...]
<repositories>
[...]
    <repository>
        <id>ow2</id>
        <url>http://maven.ow2.org/maven2</url>
        <name>ow2</name>
    </repository>
    <repository>
	<id>ow2-snapshot</id>
	<url>http://maven.ow2.org/maven2-snapshot/</url>
	<name>ow2-snapshot</name>
    </repository>
    <repository>
       <id>weblab</id>
       <url>http://maven.weblab-project.org/</url>
       <name>weblab</name>
    </repository>
    <repository>
       <id>weblab-snapshot</id>
       <url>http://maven-snapshot.weblab-project.org/</url>
	<name>weblab-snapshot</name>
     </repository> 
</repositories>
[...]

If not, you have to create a new profile in the profiles mark-up, and activate the profile in the settings mark-up, such as below :

[...]
<profiles>
   <profile>
      <id>AnyIDYouWant</id> <!-- must be the same as below -->
      <activation>
         <activeByDefault>true</activeByDefault>
      </activation>
      <repositories>
         <repository>
            <id>ow2</id>
            <url>http://maven.ow2.org/maven2</url>
            <name>ow2</name>
         </repository>
         <repository>
	    <id>ow2-snapshot</id>
	    <url>http://maven.ow2.org/maven2-snapshot/</url>
	    <name>ow2-snapshot</name>
         </repository>
         <repository>
            <id>weblab</id>
            <url>http://maven.weblab-project.org/</url>
            <name>weblab</name>
         </repository>
         <repository>
	    <id>weblab-snapshot</id>
            <url>http://maven-snapshot.weblab-project.org/</url>
	    <name>weblab-snapshot</name>
         </repository> 
      </repositories>
      <pluginRepositories>
         <pluginRepository>
            <id>ow2</id>
            <url>http://maven.ow2.org/maven2</url>
            <name>ow2</name>
         </pluginRepository>
         <pluginRepository>
            <id>ow2-snapshot</id>
	    <url>http://maven.ow2.org/maven2-snapshot/</url>
	    <name>ow2-snapshot</name>
	 </pluginRepository>
         <pluginRepository>
            <id>weblab</id>
            <url>http://maven.weblab-project.org/</url>
            <name>weblab</name>
         </pluginRepository>
         <pluginRepository>
	    <id>weblab-snapshot</id>
            <url>http://maven-snapshot.weblab-project.org/</url>
	    <name>weblab-snapshot</name>
         </pluginRepository> 
      </pluginRepositories>
   </profile>
</profiles>
<activeProfiles>
    <activeProfile>AnyIDYouWant</activeProfile> <!-- must be the same as above -->
</activeProfiles>

To invoke the plugin by its prefix weblab you have to add group identifier (i.e.: mvn weblab:generate ...):

<settings ...>
[...]
    <pluginGroups>
    [...]
        <pluginGroup>org.ow2.weblab.tools.maven</pluginGroup>
    </pluginGroups>
</settings>

With this configuration, Maven can retrieve libraries needed to the next steps of this tutorial.

Creation of a Java web application

Thanks to Maven and to the WebLab Archetype Plugin (for more details, see WebLab Archetype Plugin README in #Extra documentation), this task is extremely easy and is reduce to a simple command:

mvn org.ow2.weblab.tools.maven:weblab-archetype-plugin:1.1:generate -DgroupId=org.tutorial.ws -DartifactId=langDetectorService \
-DserviceName=LangDetectorService -DwebLabInterface=analyser -Dversion=1.0-SNAPSHOT

The interactive creation plugin will ask you to confirm the configuration you defined (i.e.. groupID, artifactID, archetype and version). Just type "y" and valid. The plugin will then generate the following folders and files:

langDetectorService/ 
|-- pom.xml 
`-- src 
    |-- main
    |   |-- java
    |   |   `-- org
    |   |       `-- tutorial
    |   |           `-- ws
    |   |               `-- LangDetectorService.java
    |   `-- webapp
    |       `-- WEB-INF
    |           |-- cxf-servlet.xml
    |           `-- web.xml
    `-- test
        `-- java
            `-- org
                `-- tutorial
                    `-- ws
                        `-- LangDetectorService.java

Now we will update the XML file called pom.xml that contains information about the project and configuration details used by Maven in order to build the project. Open the file langDetectorService/pom.xml, which should look like this:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
	<modelVersion>4.0.0</modelVersion>

	<parent>
		<groupId>org.ow2.weblab.webservices</groupId>
		<artifactId>parent</artifactId>
		<version>1.2.5</version>
	</parent>


	<groupId>org.tutorial.ws</groupId>
	<artifactId>langDetectorService</artifactId>
	<version>1.0-SNAPSHOT</version>

	<packaging>war</packaging>
	<name>langDetectorService</name>

	<description>This is a project generated WebLab Maven plug-in</description>

	<build>
		<finalName>${project.artifactId}</finalName>
	</build>

	<dependencies>
		<dependency>
			<groupId>log4j</groupId>
			<artifactId>log4j</artifactId>
			<scope>runtime</scope>
			<optional>true</optional>
		</dependency>
		<dependency>
			<groupId>org.ow2.weblab.core.helpers</groupId>
			<artifactId>resource-validator</artifactId>
			<scope>test</scope>
		</dependency>
	</dependencies>

</project>

To adapt this file to our needs, you have to add dependencies for the language detection API. To do this, you can copy the lines provided below under the <dependencies></dependencies> mark-ups. It will enables Maven to load the jar libraries and all its dependencies.

[...]
<dependencies>
    [...]
    <dependency>
        <groupId>de.spieleck.app.ngramj</groupId>
        <artifactId>cngram</artifactId>
        <version>1.0-0.060327</version>
    </dependency>
</dependencies>
[...]

The next step is the actual implementation of the service. In the WebLab platform, services interfaces have been standardized in a small set of generic interfaces that rely on the same data exchange model. They define the method that are used by the multiple components that can be used in a multimedia processing chain (see on WebLab documentation on the wiki[5] for more information). Thus a "WebLab-compliant" service need to implement one of these generic interfaces. For our new service, we will use the Analyser interface. A UML description of this interface can be found in WebLab documentation whereas the WSDL itself can be found in the source repository. This WSDL has been used to generate the JAVA interface (through CXF) which will be used in our example. It includes the JAVA classes associated with elements of the data exchange model.

Implementation of the Web service class

The implementation itself will be quite easy. Our new service is an Analyser, so we have to implement this interface which force us to add only one unique method. It is called to parse resources and named process(ProcessArgs args). It receive a Resource as input and answer a Resource as output (the ProcessArgs could optionally contain an extra object called UsageContext but it will be ignored for this tutorial).

In order to implement the language detector, you just have to edit your-service-path/src/main/java/org/tutorial/ws/Langdetectorservice.java in order to fill it like this (this is an example of implementation for the tutorial).

package org.tutorial.ws;
 
import java.io.IOException;
import java.util.List;

import javax.jws.WebService;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.ow2.weblab.core.extended.factory.ExceptionFactory;
import org.ow2.weblab.core.extended.util.ResourceUtil;
import org.ow2.weblab.core.model.Document;
import org.ow2.weblab.core.model.Resource;
import org.ow2.weblab.core.model.Text;
import org.ow2.weblab.core.services.Analyser;
import org.ow2.weblab.core.services.InvalidParameterException;
import org.ow2.weblab.core.services.analyser.ProcessArgs;
import org.ow2.weblab.core.services.analyser.ProcessReturn;
import org.purl.dc.elements.DublinCoreAnnotator;

import de.spieleck.app.cngram.NGramProfiles;
 
/**
 * Basic service that annotate Text unit with the code of the detected language
 * @author WebLab team
 */
@WebService(endpointInterface = "org.ow2.weblab.core.services.Analyser")
public class LangDetectorService implements Analyser {
 
	private final NGramProfiles nps;
	private final NGramProfiles.Ranker ranker;
	private final Log log;
 
	/**
	 * Simple constructor that initiate the language profiles for NGramJ
	 */
	public LangDetectorService() throws IOException {
		this.nps = new NGramProfiles();
		this.ranker = nps.getRanker();
		this.log = LogFactory.geLog(this.getClass());
	}
 
	@Override
	public ProcessReturn process(final ProcessArgs args) throws InvalidParameterException {
		// check we received a valid request
		this.checkArgs(args);
		// Get text units in the Document received
		final List<Text> texts = ResourceUtil.getSelectedSubResources(args.getResource(), Text.class);
		final StringBuilder sb = new StringBuilder();
		final Resource res = args.getResource();
		// Concatenation of the document text sections
		for (final Text text : texts) {
			if (text.getContent() == null || text.getContent().isEmpty()) {
				this.log.debug("Text '" + text.getUri() + "' has no content; ignoring it.");
				continue;
			}
			sb.append(text.getContent());
			sb.append("\n\n\n"); // insert line break to separate successive texts
		}
		// detecting of the language of the whole text
		final String textLanguage = this.detectLanguage(sb.toString());
		// integration of the annotation dc:language in the document
		this.log.info("Detected language code '" + textLanguage + "' for Document '" + res.getUri() + "'.");
		final DublinCoreAnnotator dca = new DublinCoreAnnotator(res);
		dca.writeLanguage(textLanguage);

		final ProcessReturn pr = new ProcessReturn();
		pr.setResource(res);
 
		return pr;
	}
 
	/**
	 * Detecting the language of the text content passed as a String with NGramJ
	 * @param text the text content
	 * @return a string that contains the char code of the detected language (i.e. the name of the ngp file that contains the profile of the language in ngramj jar) or und if the language is unknown
	 */
	private synchronized String detectLanguage(final String text) {
		final String retVal;
		ranker.reset();
		ranker.account(text);
		NGramProfiles.RankResult res = ranker.getRankResult();
 		if (res.getScore(0) > 0.1) {
			retVal = res.getName(0);
		} else {
			retVal = "und";
		}
		return retVal;
	}
 

	/**
	 * Check the request sent to the service, mostly for null content.
	 * @param args the arguments sent to the service
	 * @throws InvalidParameterException
	 */
	private void checkArgs(final ProcessArgs args) throws InvalidParameterException {
		if (args == null) {
			throw ExceptionFactory.createInvalidParameterException("ProcessArgs was null.");
		}
		final Resource res = args.getResource();
		if (res == null) {
			throw ExceptionFactory.createInvalidParameterException("Resource was null.");
		}
		if (!(res instanceof Document)) {
			throw ExceptionFactory.createInvalidParameterException("Resource was not a document.");
		}
	}
}

That's all folks, now the code is ready, let's test it.

Compilation, packaging and test

The next step is to package our web service into a war file in order to deploy it on a web server and to test it.

/!\ Do not forget to replace the annotation in the test code in file LangDetectorServiceTest.java

[...]
	/**
	 * Multi-thread tests
	 * 
	 * TODO Delete expected exception when method process will be implemented.
	 * For the moment, method throws an exception instead of return a ProcessReturn.
	 */
	@Test(expected = ExecutionException.class)
	public void testAnalyserMultiThread() throws Exception {
		final List<Document> docs = this.getDocuments();
		final List<Callable<Resource>> analysers = new ArrayList<Callable<Resource>>();
[...]

by

[...]
	/**
	 * Multi-thread tests
	 * 
	 * TODO Delete expected exception when method process will be implemented.
	 * For the moment, method throws an exception instead of return a ProcessReturn.
	 */
	@Test
	public void testAnalyserMultiThread() throws Exception {
		final List<Document> docs = this.getDocuments();
		final List<Callable<Resource>> analysers = new ArrayList<Callable<Resource>>();
[...]

then you can use Maven and type the below command to package your service:

mvn package

This command will compile the project and create the war file of your web service under the directory langDetectorService/target. You can now deploy it on your tomcat server (we assume you used the tomcat web server provided with the demo) : simply copy the resulting WAR file from langDetectorService/target/langDetectorService.war to <tomcat-home>/webapps/. To ensure the service is correctly deployed on tomcat, check the log (open <tomcat-home>/logs/catalina.out) or go to the service welcome page URL http://localhost:8080/langDetectorService. If the service is correctly statred you should see :

ServiceWelcomeCXF.png

Welcome page of the service correctly deployed on tomcat

To test this service, you have to install SoapUI (see SoapUI documentation[6]). Then simply :

  1. Create a new SoapUI project ;
  2. Copy the URL of the service in the 'Initial WSDL/WADL' box (i.e. http://localhost:8080/langDetectorService/analyser?wsdl) ;
  3. Edit the 'process' request and copy the request provided hereafter ;
  4. Send the request and check the results.

<soapenv:Envelope
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:analyser="http://weblab.ow2.org/core/1.2/services/analyser"
xmlns:model="http://weblab.ow2.org/core/1.2/model#" >
    <soapenv:Header/>
    <soapenv:Body>
        <analyser:processArgs>
            <resource xsi:type="model:Document" uri="weblab://aaa/1">
                <mediaUnit xsi:type="model:Text" uri="weblab://aaa/1#0">
                    <content>The WebLab platform is the generic name of the development and execution environment
                        platform provided by EADS in several research projects (Vitalas, WebContent, e-WokHub, Citrine...)
                        involving the process of several types of media.</content>
                </mediaUnit>
            </resource>
        </analyser:processArgs>
    </soapenv:Body>
</soapenv:Envelope>

You can see here the result of this query :

SoapUI result.png

Test of the langDetectorService using SoapUI

Moreover if you check the logs of your Apache Tomcat server (default logger write in <tomcat-home>/logs/catalina.out), you should see :

2014-05-28 14:50:14,983 INFO  org.ow2.weblab.service.LangDetectorService - Detected language code 'en' for Document 'weblab://aaa/1'.

So that's the end of the service tutorial. The next section will provide you some insight that may be useful when trying to go further and implement your own service.

Going further

After this tutorial is achieved, the next step is to implement your own service in accordance to any service specification. Hopefully, the procedure is really similar to the proposed tutorial so we will only provide you the information needed to your own whatYouWantProject.

Building custom project

First of all, you need to build a new Maven project using your own names. So the command can be change to:

mvn org.ow2.weblab.tools.maven:weblab-archetype-plugin:generate
-DgroupId=anyGroupId
-DartifactId=newNameOfTheProject
-DserviceName=newNameOfService
-DwebLabInterface=anyWebLabinterfaces #separate by ","
-Dversion=anyVersion

Note that this will create a new folder which will be named after the project name and that the group and artifact ID will be in the pom.xml. Moreover, you do not have to complete every field, the Maven command can ask you details during execution.

Using an IDE

There are plenty of maven plugins in order to enable the use of your service project in your favorite IDE (including Eclipse, Netbeans, JDeveloper and more ...). There will be also a WebLab Plugin for Eclipse which will be released soon, there will be an other tutorial for this.

Custom implementation

To implement the service, just create a new class in the project (in the package you want) and make it implementing the right interface previously generated. In the tutorial it was org.ow2.weblab.core.services.Analyser, but if you change it, it may use another name (and probably package). Remember, that you also need to add the annotation concerning the web service implementation like:

@WebService(endpointInterface = "package.plus.name.of.interface")

The class is now specifying that it implements the service interface both to Java (with classing implements statement) and jaxws (with annotation). You should now implement the needed methods in accordance to the interface definition. This is where you can do "what you want". Which means that this is the place of your code which can call your own external applications, use your own libraries... Just make sure that it achieve the right process.

When you develop your own service, there are two possibilities:

  • add your own code to the existing project created with the tutorial procedure. It means that your business classes will be directly in the project. Thus you can easily package the application.
  • package your code in a JAR. It will allow your code to be independent from the service implementation and thus to be easily reusable in other application. However, you should follow the procedure to add correctly your jar as a dependency in the project (see hereafter).

In both cases, SoapUI will only test that the communication with the service is valid (i.e. it can handle the arguments and provide response) and that the arguments and output are valid or not against the data exchange model. However no validation can be made on the content of the response to test if the process that should be done by your service has been correctly done. You should either valid the results manually or test it outside the soapUI (using JUnit for instance).

Adding dependencies to the project

If your project depends on external libraries (classic Java libraries or your own libraries), two step are required to use the benefits of Maven. First you need to install the jar libraries into your local repository and then declare the dependence in your project POM file.

As explained in the Maven documentation, libraries are identified by their group id, artifact id and version number. Thus for each libraries you depends on you should choose those references. For instance if you have a myLibrary.jar, you can choose this configuration:

  • groupId = my.organisation.group
  • artifactId = myLibrary
  • version = 1.0

Then you have to install it in your local repository using the Maven command:

mvn install:install-file -Dfile=path/to/your/jar/myLibrary.jar
-DgroupId=my.organisation.group -DartifactId=myLibrary
-Dversion=1.0 -Dpackaging=jar -DgeneratePom=true

Then, add the following dependency to the POM file of your project:

[...]
<dependency>
    <groupId>my.organisation.group</groupId>
    <artifactId>myLibrary</artifactId>
    <version>1.0</version>
</dependency>
[...]

Finally, run the following command to ensure that Maven correctly take this new dependency in account:

mvn clean compile

If you have several dependencies, you can install all of them, add all dependencies to the POM file and execute only one time the previous command.

If the JAR you want to install depends itself on other libraries, the easiest way is to include them directly in the JAR itself. The other procedure can be to convert your entire project to the Maven structure, but this task is out of the scope of this tutorial. However, as already mentioned, one can look into Maven documentation for more information on Maven dependencies management. You may have set maven to work ``offline if you follow the steps proposed in the tutorial. It may be time to go back online and configure some repositories if you want to use extra dependencies.

Packaging and tests

Nothing change here except that the provided arguments should be in accordance with the new interface implemented (and perhaps your own service pre-conditions).

Delivery

To deliver your component embedded in a web service, you have to provide of course the WAR file, ready to be deployed on PEtALS application server. However some more data are needed to ensure it will be integrable easily. A complete description of the services provided by the component is needed in order to build the correct processing chain without misunderstanding on any component functionality. A dedicated sheet is provided and should be filled by partners for each components they will develop. Then a test kit should be provided in order to be able to replay the internal tests that have been made to validate the component functionality. It involves data to build requests and valid the results of each service methods:

  • SOAP request samples,
  • SOAP responses (one for each requests),
  • configuration procedures,
  • testing data (i.e. multimedia content for most of the services) provided in input and/or expected results.

Without those information and data, the integration of any component can be very time consuming and one should take car on provided the right information in them.

Extra documentation

For more information, follow the links hereafter: