Stanford:QaGuide

From Protege Wiki

Jump to: navigation, search

Contents

How-to for architects, test and project managers

Target audience and scope

In this section you find how tos which are related to convert projects from ANT to Maven, to write pom.xml files etc.

Comprehensive example

There is a project that shows everything related to build and testing:

  • how to write a pom.xml
  • how to write and integrate unit tests (JUnit and TestNG)
  • how to write and integrate integration tests (selenium)

Please visit https://bmir-gforge.stanford.edu/svn/hudson-test/Hudson-Test.

How to create a new or convert an existing project

This steps steps have to be done once per project)

Establish or change to Maven directory structure

The following is copied from http://maven.apache.org/guides/introduction/introduction-to-the-standard-directory-layout.html. It documents the directory layout expected by Maven and the directory layout created by Maven. Please try to conform to this structure as much as possible; however, if you can't these settings can be overridden via the project descriptor.

Path Description
src/main/java Application/Library sources
src/main/resources Application/Library resources
src/main/filters Resource filter files
src/main/assembly Assembly descriptors
src/main/config Configuration files
src/main/webapp Web application sources
src/test/java Test sources
src/test/resources Test resources
src/test/filters Test resource fiter files
src/site Site
LICENSE.txt Project's license
NOTICE.txt Notices and attributions required by libraries that the project depends on
README.txt Project's readme

In the table the most relevant paths are marked bold.

Additionally there are at the top level

  • pom.xml
  • other optional files as properties, maven.xml or build.xml (if using Ant).


In order to get to the new file structure:

  • Add folder inside /src: /src/main/java and /src/test/java
  • Move /src/edu respectively /test/edu folders into new folders.
  • Delete folder test-lib
  • Rename “war” to “webapp” and move folder under /src. (here I ran into problems. Really the best way?)
  • Add folder /src/main/resources/config. Add to this folder configuration.properties file.
  • Add under src/test/java following folders: /client/ and /selenium??
  • Delete files in root folder with exception of src. Delete e.g. /etc/,


To move files/directories you can use Tortoise as described in http://stackoverflow.com/questions/62570/how-do-i-move-a-file-or-folder-from-one-folder-to-another-in-tortoisesvn.

Hint: In the developer guide there is a description how to import a project from SVN and immediatelly update the src-folder to the expected structure.

How to upload required artifacts (typicall jar-files) to Nexus repository

There are multiple ways of adding resources to the repository

  1. Direct copy (not recommended)
    • Copy resources directly to Nexus subfolder e.g. to /var/nexus/sonatype-work/nexus/storage/thirdparty/edu/stanford/bmir/core/icd.
    • You need to put the file the right directory and give it the name indicating the version number, e.g. /edu/Stanford/bmir/core/protégé-1.0.jar
    • Goto nexus and right click on third party repository: reindex respectively rebuild metadata
  2. Uploaded file automatically in due course of a build process (this is recommended for "own" code as Snapshots and Releases).
    • There is nothing to do besides adding the <distributionManagement> tag to the pom.xml and use the deploy plugin. The PC specific settings.xml has to point to the repository as described in the adminstrators guide in section TODO
  3. Uploaded file using the Nexus interface (recommended only for third party artifacts). This option is described in the following:
Screenshot how to upload jars to Nexus (step 1)
upload an artifact to Nexus (part 1): Select repository
Screenshot how to upload jars to Nexus (step 2)
upload an artifact to Nexus (part 2): Define GAV parameters
Screenshot how to upload jars to Nexus (step 3)
upload an artifact to Nexus (part 3): Reindexing
Screenshot how copy the dependency snippet for pom.xml
copy the dependency snippet for pom.xml


For each artifact (jar) you can later copy the dependency information for Maven (into the pom.xml). The code snipped look like:

 <dependency>
   <groupId>edu.stanford.bmir.core</groupId>
   <artifact>protege</artifact>
   <version>1.0</version>
 </dependency>


TODO: define naming convention for groupId. Currently discussed are the following

  • protege3
  • protege4
  • webprotege
  • owl


Further reading

How to write the Maven script (pom.xml)

In our case the pom.xml typically will have the following structure:

  • Some general information e.g.
 <modelVersion>4.0.0</modelVersion>
 <groupId>edu.stanford.smi.protegex</groupId>
 <artifactId>dlquery</artifactId>
 <packaging>jar</packaging>
 <name>protege-plugin-dlquery</name>
 <version>0.0.3-SNAPSHOT</version>
  • Properties as the Java version
  • the <distributionManagement> tag which is necessary if artifacts have to deployed to a repository
  • the <dependency/> tags (an example was given above)
  • the build part with its plugins (e.g. to compile, test, deploy)
  • the reporting part with its plugins (e.g. PMD, Findbugs, JDepend)


It is recommended to use an existing pom.xml as template.


Further hints and readings:

How to add project to Hudson

Projects can be added from the Hudson administration interface
  • Go to Hudson page (e.g. http://bmir-hudson1.stanford.edu).
  • Add new job. Select name (e.g. WebProtégé”) and type of project (“Build a maven2 project”).
  • Select in section “Source Code Management” the option “Subversion” and add SVN server URL (e.g. http://smi-protege.stanford.edu/repos/protege/web-protege/branches/who-maven/). Keep “Use update” checked. No local module directory. (no password needed?)
  • Determine build interval respectively trigger. Currently we use “Build whenever a SNAPSHOT dependency is built”.
  • Optionally enable email notification.
  • Activate Sonar in “Post-Build-Activities” (Sonar needs to be installed first)

FAQs

Related to reporting

My code coverage statistics aren’t updated. What can I do?

Please change version in pom.xml to next snapshot respectively release number. Don’t forget the configuration.properties file.

I do not get code coverage statistics for my integration tests. What can I do?

Currently nothing, this is the way maven handles the build process.

My build fails since the sonar plugin was not found! What can I do?

Your Hudson build fails. In the console output you find this stacktrace:

  org.apache.maven.lifecycle.LifecycleExecutionException: The plugin 'org.codehaus.mojo:sonar-maven-plugin' does not exist or no valid version could be found
	at org.apache.maven.lifecycle.DefaultLifecycleExecutor.verifyPlugin(DefaultLifecycleExecutor.java:1569)
	at org.apache.maven.lifecycle.DefaultLifecycleExecutor.getMojoDescriptor(DefaultLifecycleExecutor.java:1851)
	at org.apache.maven.lifecycle.DefaultLifecycleExecutor.segmentTaskListByAggregationNeeds(DefaultLifecycleExecutor.java:462)
	at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:175)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328)

Then you most probably ran into a Maven bug which is here.

What can you do? As the plugin update is not working as it is supposed to do, please remove the following two directories:

  1. Maven local repository: sudo rm -r /usr/local/data/hudson/.m2/repository/org/codehaus/mojo/sonar-maven-plugin
  2. Nexus repository: sudo rm -r /usr/local/data/sonatype-work/nexus/storage/central/org/codehaus/mojo/sonar-maven-plugin

Then run the build again. Now the repositories are updated and your build should run successfully.

If this fails do addtionally the following:

  1. ssh to bmir-hudson1.standford.edu
  2. Switch user: sudo su - hudson
  3. if Maven path is not yet set:
    • export M2_HOME=/usr/local/apache-maven-2.2.1
    • export PATH=$PATH:$HOME/bin:$JAVA_HOME/bin:$M2_HOME/bin
  4. Run one project from command line
    • switch to one project cd /usr/local/data/hudson/jobs/Example-Project/workspace
    • mvn -U sonar:sonar The -U forces the update. Now the maven plugin should be downloaded

Related to pom.xml

How can copy output to a specific directory?

Remark: In the perfect world you are deploying your artifacts to a local or a remote repository. You are not supposed to work with specific target directories. Nevertheless there may be occasions where it is necessary to do so:

Step 1: Use Resource Plugin

Add copy plugin to your pom.xml (in build section). Example:

  <!--  Copy the stuff to the desired place -->
  <plugin>
    <artifactId>maven-resources-plugin</artifactId>
    <version>2.4.3</version>
    <executions>
      <execution>
        <id>copy-resources</id>
        <!-- here the phase you need -->
        <phase>test</phase>
        <goals>
          <goal>copy-resources</goal>
        </goals>
        <configuration>
          <outputDirectory>${outputdir}</outputDirectory>
          <resources>
            <resource>
              <directory>src</directory>
              <filtering>true</filtering>
            </resource>
          </resources>
        </configuration>
      </execution>
    </executions>
  </plugin>

Step 2: Parametrize output directory

Set the variable (outputdir) either in the properties-section of the pom.xml:

  <properties>
    <maven.compiler.source>1.6</maven.compiler.source>
    <maven.compiler.target>1.6</maven.compiler.target>
    <outputdir>testoutput/testdirectory</outputdir>
  </properties>

Or in an external properties file. This requires the Maven properties plugin:

  <plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>properties-maven-plugin</artifactId>
    <version>1.0-alpha-1</version>
    <executions>
      <execution>
        <phase>initialize</phase>
        <goals>
          <goal>read-project-properties</goal>
        </goals>
        <configuration>
          <files>
            <file>src/main/resources/example.properties</file>
          </files>
        </configuration>
      </execution>
    </executions>
  </plugin>

In this (latter) case there is a src/main/resources/example.properties with an entry

 outputdir = testoutput2/testdirectory

Further reading:

How can I set the version number in the manifest of OSGI bundles?

Further reading

How can I set the version number of the build number

Do it the default way

The easiest way is to do nothing. Maven automatically increases the version number for snapshot releases. You even would have to turn it off explicitly in the pom.xml:

	<distributionManagement>
		<repository>
			<id>releases</id>
			<url>${nexus.path}/bmir-release</url>
		</repository>
 
		<snapshotRepository>
			<id>snapshots</id>
			<name>Internal Snapshots</name>
			<url>${nexus.path}/snapshots</url>
			<uniqueVersion>false</uniqueVersion> <!-- This is the important line-->
		</snapshotRepository>
	</distributionManagement>

Do it explicitly

Another way to determine the build number is by using the Maven build number plugin. This is quite forward:

  <build>
    <!-- The final name uses the build namber plugin. You can either us the repository id,
    an incremental number or a timestamp. Or combinations of those -->
    <finalName>${project.artifactId}-${project.version}-r${buildNumber}</finalName>
 
    <plugins>
      <!-- This plugin is needed to autoincrement the version number -->
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>buildnumber-maven-plugin</artifactId>
        <version>1.0-beta-3</version>
        <executions>
          <execution>
            <phase>validate</phase>
            <goals>
              <goal>create</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <!-- the "0" refers to item[0] which is the timestamp -->
          <format>{0,date,yyyy-MM-dd-HH-mm-ss}</format>
          <items>
            <item>timestamp</item>
          </items>
        </configuration>		
      </plugin>
 
      <!-- and now all the other plugins -->
    </plugins>
  </build>


The resulting file name could be sth. like dlquery-0.0.3-r2010-11-15-10-36-20.jar. The dlquery is the artifact id, 0.0.3 the version, both read from the pom.xml (and not generated by the version plugin).

The plugin also provides you the option to include this version in manifest files (jar, war).

Additional ways to determine the build number:

  • The plugin supports also to retrieve the version number from the current repository revision.
  • You also can include the branch (${scmBranch}) into the version number.
  • The plugin supports also autoincrement.
  • You could use a property file which you can access in the <finalName> tag.

Further reading


How and where to I decide if I want to make a snapshot release or a "real" release?

Maven picks up your decision from the version. If your ends with -SNAPSHOT then it is a snapshot release, otherweise are productive release.

Example for a snapshot release:

  <project xmlns="http://maven.apache.org/POM/4.0.0">
	<modelVersion>4.0.0</modelVersion>
	<groupId>protege4</groupId>
	<artifactId>xmlcatalog</artifactId>
	<packaging>jar</packaging>
	<name>xmlcatalog</name>
	<version>1.0.0-SNAPSHOT</version>
	...
  </project>

"I get a return error 400" or "I the deploy of artifacts to Nexus fails. What can I do?"

There are two typical errors:

  1. the Maven setting.xml file (either locally or on Hudson) does not contain the right credentials.
  2. you are trying to deploy to a release repository. Per default it is not allowed to "overwrite" artifacts in a release repository. It is possible in Snapshot repositories. What type a repository has, you can find out by selecting the repository in Nexus (login first). The configuration tab will tell you.

What are the repositories I'm supposed to deploy to?

We decided to work with two repositories only:

  1. BMIR-Release
  2. Snapshot


How do I assemple a complex project with dependencies?

Please use the assembly plugin.

Further reading:

Personal tools