Stanford:DeveloperGuide

From Protege Wiki

Jump to: navigation, search

Contents

In a nutshell

You work as you typically work just using Eclipse to develop and test. There are only a few things you need to consider:

  • The initial setup of your Eclipse project is done with Maven.
  • Prefer TestNG over JUnit in order to write unit tests.
  • We encourage you to write UI tests using the Selenium framework.
  • We encourage you to deploy your artifacts not just to the file system (local Maven repository) but also to the central Nexus repository, at least if you have the privileges to do so.

And don’t forget to update and commit to/from on a daily basis.

How to start from scratch

Downloads and Installs

Use the JDK instead of the JRE

JAVA

  • Download and install java.
  • Set environment setting JAVA_HOME pointing to the java folder, e.g. C:\Program Files\Java\jdk1.6.0_21\bin.

Maven

  • Download Maven from http://maven.apache.org/download.html und chose apache-maven-2.2.1.
  • Add system environment variable M2_HOME pointing to new directory apache-maven-2.2.1
  • Add to Path ;%M2_HOME%\bin
  • Modify either ${user.home}/.m2/settings.xml or ${M2_HOME}/conf/settings.xml: Path to Nexus and you operating system. There is an example page for a settings file.

If you update to Maven 3, there is no need to update the scripts (pom.xml).

Eclipse

  "C:\Program Files\eclipse-jee-helios-SR1-win32\eclipse\eclipse.exe" -vm "C:\Programme\Java\jdk1.6.0_21\bin\javaw.exe" 

Firefox & GWT plugin (only for GWT UI testing)

After download and install of Firefox we recommend adding a new profile. Rationale: Selenium typically initiates a new Firefox profile which is missing the GWT Firefox plugin. Therefore we generate a new one:

  • Open command window and go to Firefox directory
  • Execute firefox -P. Plugin manager shows up.
  • Add new profile and chose a directory, e.g. one Protégé related.
  • Start Firefox and start eclipse. Open page http://127.0.0.1:8888/WebProtege.html?gwt.codesvr=127.0.0.1:9997. This prompts you to install the GWT plugin.
  • Save the profile

Selenium (only for GWT UI testing)

Please refer to chapter in section administration. TODO add link.

Start server with java –jar selenium-server.jar -firefoxProfileTemplate <Path2Profile>. <Path2Profile> is the folder you were choosing when creating the new profile.

Example:

 C:\Users\DrCJ\selenium-server-1.0.3>java -jar selenium-server.jar  -firefoxProfileTemplate c:\Users\DrCJ\firefoxprofile

Start working with a project e.g. with Protégé

Option 1: import project using Subclipse (preferred)

Pretty straight forward

  • Import project from repository using Subclipse.
    • File > Import > SVN > SVN Projects. Import desired project and branch.
    • Use “Create Project wizard”, use “Next” instead of “Finish” button and add a new source folder “src/main/java” and remove “src” from build path.
    • Copy imported files from src to “src/main/java” folder. There will be problems, since libraries are missing. Do not tweak build path, this will be done automatically in the next step.
  • Execute mvn eclipse:eclipse from command line
  • Refresh project (F5)

To update classpath (e.g. if there is a new jar (as defined in the pom.xml)) just repeat these steps.

If there are still libraries missing (which are obviously defined in your pom.xml), please consult with your project manager or upload the artifacts to the respository as described in the project manager guide.

import project using Subclipse (video tutorial)

Maven can be used to check out the project
Maven can be used to check out the project
Or use SVN to download. Then call mvn eclipse:eclipse and import project
Or use SVN to download. Then call mvn eclipse:eclipse and import project
make project to become a GWT project
make project to become a GWT project
Set the webapp folder for the GWT compiled code
Set the webapp folder for the GWT compiled code

Option 2: import project from command line (using a SVN client)

  • Download project from repository with Tortoise or other (command line based) SVN client
  • Execute mvn eclipse:eclipse from command line
  • Refresh project (F5)

To update classpath (e.g. if there is a new pom.xml) just repeat these steps.

Option 3: using the m2eclipse plugin

Chose “Project” from the “File” menu, select Maven and then “Checkout maven Projects from SCM”.

Configure GWT plugin

  • Select Project Properties and check under Google “Use Google App Engine”
  • In project properties set plugin as active and change WAR directory to be /src/main/webapp (there is no longer a /war directory!)

Questions that may arise during development

How do I work with multiple projects in parallel

To given an example: We work with projects A and B where B depends on A. (An example would be A = Protégé, B = WebProtégé.)

Assuming you would like to make a change to project A which is required to continue working with project B:

  1. Make the change in project A
  2. Go to the command line in project A's directory (e.g. in /home/<USER_NAME>/workspace/projectA) and enter mvn install [1], [2]. This will update your local Maven repository (M2_REPO) which typically is located in /home/.m2/repository respectively in c:\user\<username>\.m2\repository. [1]
  3. Go to the command line of project B and enter mvn eclipse:eclipse [2]. This will update the classpath and (re)load the artifact of project A.
  4. Continue developing in project B.

Actually there are two variants:

  1. Variant 1: you do not change the version number of the artifact of project A. In this case you do not have to modify the version number in project A's pom.xml. This is an advantage, another advantage is that you do not need to update the dependency section in the pom.xml of project B.
  2. Variant 2: you change the version number of the artifact of project A in the project A' pom.xml. This requires that you update the version in the dependency section in project B's pom.xml. The advantage of this variant is that you easily can switch back to the old version of project A's artifact, you even can have multiple versions in your repository.

The dependency section in the pom.xml the text above is refering to looks like:

  <dependency>
    <groupId>protege</groupId>
    <artifactId>protege-core</artifactId>
    <version>3.4.2</version>
  </dependency>


[1]: If you want to skip the unit tests use mvn install -DskipTests=true instead.

[2]: Alternatively you can use the m2eclipse plugin.

How do I deploy my artifacts to the (central) Nexus repository

First you have to have write privileges to BMIR's Nexus repository (http://bmir-hudson1.stanford.edu/nexus). Typically these privileges are only granted to a closed group of developers. Both, these privileges (-> credentials) and the path to the repository have to be entered to your Maven settings file (<MAVEN_INSTALL_DIR>/conf/settings.xml). There is a complete example of this file available.

As soon you execute mvn deploy the Nexus repository is updated. Please keep in mind that your artifact either is stored in the snapshot or in the release repository dependent on the version number in your pom.xml: If the version number ends with SNAPSHOT the snapshot directory is addressed, otherwise the release repository.

Keep in mind that...

  • the SNAPSHOT repository allows the redeployment of an artifact (with the same version number), the release repository doesn't,
  • a deployment (mvn deploy) to the SNAPSHOT repository will autoincrement the version number of your artifact (if not explicitly turned off in the pom.xml), a deployment to the release repository won't.
  • there is no autoincrement of the version number during install (mvn install),
  • there is no need to install a local Nexus repository on your machine.

How to update classpath

Just get latest pom.xml and execute mvn eclipse:eclipse. This will update (not overwrite) your classpath. Press refresh in Eclipse. There is not even a restart necessary.

How to write unit test case (TestNG, JUnit) and UI test cases (FEST, QFS)

This section is about standard unit tests, i.e. not about writing UI tests/integrations tests using Selenium.

How to re-use existing JUnit test cases

Option 1: Convert JUnit test cases to TestNG test cases

You either can do that programmatically as described in http://testng.org/doc/migrating.html or you do that manually. TestNG tests look pretty much the same then JUnit tests. What has to be done typically includes:

  • Change import to org.testng.annotations.*; and import static org.testng.Assert.*;
  • Add annotation @BeforeClass to setup-method.
  • Add annotation @Test to your test methods

Option 2: Keep JUnit tests cases as they are and run it from TestNG

In the /src/test/resources/unit-tests.xml there is a section with junit=true. Just add your tests there.

Example (mind second set of tests)

  <!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
  <suite name="ProtegeTestSuite" verbose="1">
    <test name="TestNG-Tests">
      <classes>
        <class name="edu.stanford.bmir.protege.web.server.WatchedEntitiesCacheTest"></class>
        <class name="client.TestUnitProtege"></class>
      </classes>
    </test>
    <test name="JUnit-Tests (Example)" junit="true">
      <classes>
        <class name="client.JUnitExample"></class>
      </classes>
    </test>
  </suite>

There are (theoretically) two options how to include JUnit tests cases to the continuous integration.

  1. Make sure that your test classes are in the designated folder /src/test/java and that they are following the naming convention *Test.java
  2. Specify the classes in the xml file in /src/test/resources

We decided to go for the latter option. All types of specification as test parameters, grouping of tests and running tests in parallel have to be defined in theses TestNG-XML-files (and not(!) in the pom.xml). In the pom.xml we deliberatly disabled the automatic execution of JUnit tests in the line

  <plugin>  
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <configuration>
      <skip>true</skip>
    </configuration>
    ...
  </plugin>

Write new TestNG test cases

The vest resource to start is http://www.testng.org. Nevertheless the following lines might be helpful:

It is recommended to use the TestNG plugin for Eclipse since it adds all libraries to your classpath. Then writing a new test case is straight forward:

  • write a standard Java class. Even there is no naming convention required it is recommended to call the class XXXTest, whereas XXX stands for the class under test.
  • Write some test methods. Even there is no naming convention required it is recommended to call the method testXXX, whereas XXX stands for the method to be tested. However the test methods require the annotation @Test.
  • Optionally have methods which have to be executed one before the test class is executed respectively which are executed before each test method. Add annotations @BeforeClass, @AfterClass, @BeforeMethod, respectively @AfterMetehod
  • Fix imports
  import junit.framework.Assert;
  import org.testng.annotations.AfterClass;
  import org.testng.annotations.BeforeClass;
  import org.testng.annotations.Parameters;
  import org.testng.annotations.Test;
  • Optional enhancements
    • Add dependencies between methods e.g. @Test (dependsOnMethods = {"openICDTree"}) where openICDTree is the name of the other method.
    • Move test parameters from test code to the configuration file, e.g.
  <test name="Navigation 1">
    <parameter name="username"  value="protege"/>
    <parameter name="username"  value="secret"/>
    <classes>
      <class name="selenium.TestICat"></class>
    </classes>
  </test>
  • Refer to these parameters in the Java code
  @Test
  @Parameters({ "username", "password" })
  public void login(String username, String password) throws InterruptedException {...}
  • Add new test case to the test-configuration xml as shown above.

How to write UI test cases (web) with Selenium

General

The UI tests use the Selenium test framework. You either can write the tests from scratch or use the Selenium IDE, a Firefox Plugin to record actions and to generate Java Code (TestNG classes).

The biggest challenge with writing UI tests is the identification of UI components since one can not relay on the id of the component. These ids are generated by GWT dynamically.

Example Code

s. example.

How to locate UI elements

s. example.

Resources

Comprehensive example

There is a project that shows everything related to build and testing:

  • how to write a pom.xml
  • how to write and integrate unit tests (JUnit and TestNG)
  • how to write and integrate integration tests (selenium)

Please visit https://bmir-gforge.stanford.edu/svn/hudson-test/Hudson-Test.

How to write UI test cases (Swing) using FEST

The tutorial on how to write FEST tests will be written upon decision for/against FEST. So far please refer to the official documentation: http://docs.codehaus.org/display/FEST/Getting+Started

The "Example-Project" (Hudson) also contains a demo application using FEST: https://bmir-gforge.stanford.edu/svn/hudson-test/Hudson-Test

How to write UI test cases (Swing) using QF-Test

The tutorial on how to write QFS tests will be written upon decision for/against QFS. So far please refer to the official documentation: http://www.qfs.de (tutorials in English available).

Other questions

How to switch to ICD11 with all data (database)

Follow instructions given in http://groups.google.com/group/icat-users/web/accessing-the-icat-content-programmatically?hl=en&_done=%2Fgroup%2Ficat-users%3Fhl%3Den%26&hl=en

On Windows the import command might be slightly different than described in the website. Use instead

 mysql --user=protege --password=protege < c:\protege_2010-11-01_04h02m.Monday.sql

In summary you need to

  • import database (and check that protege DB acutally exists afterwards)
  • add user
  • grant rights to the user

In order to do so, make use of the follwing commands:

 mysql -u root
 mysql> -u root connect to localhost;
 mysql> show databases;
 mysql> use protege;
 mysql> show tables;
 mysql> CREATE USER protege IDENTIFIED BY 'protege';
 mysql> GRANT SELECT, INSERT, UPDATE, DELETE on *.* to 'protege'@'localhost';

More useful commands: http://www.pantz.org/software/mysql/mysqlcommands.html


How to fix unit tests when dependencies fail.

The most common reason for the unit tests to fail is that there is a dependency problem. Specifically, a library on which the test depends has changed and the source code for the component being tested does not compile. A typical example of this type of problem occurred on September 8, 2011 to the org.protege.editor.owl project due to a dependency problem with the protege-owl-extensions library. Looking at the recent changes at that date we see:

   23714. refactor in the protege-owlapi-extensions. — Tim Redmond / detail
   23713. refactor in the protege-owlapi-extensions. — Tim Redmond / detail
   23712. refactor in the protege-owlapi-extensions. — Tim Redmond / detail
   23710. Bug fix from András Förhécz who did all the work.  Annotation
          properties were not getting reselected when a refactor
          occurred. — Tim Redmond / detail 

The key word in these log statements is "refactor". This suggests that the new version of org.protege.editor.owl may not compile with the old version of the protege-owlapi-extensions library that is being downloaded by maven. We therefore need to compile and include a new version of the protege-owlapi-extensions library in the maven repository (bmir-hudson1.stanford.edu/sonar) and point the maven build script for org.protege.editor.owl to this new version of the library.

The first step is to deal with the protege-owlapi-extensions library. The key to fixing this is the version information in the pom.xml file for the protege-owlapi-extensions library. This version information looks like this:

   <version>3.2.0.SNAPSHOT</version>
   < !-- <version>3.2.2.2011_04_07_1523</version> -->

(I added a space before the '!' to make the wiki happy.) Usually I hate commented code but in this case the comment is so useful that I keep it in. (This is the exception that proves the rule - in some sense the commented code is not dead.) To make a release I need to

  1. comment out the snapshot version
  2. uncomment and update the released version information
  3. commit the changes and run a build in bmir-hudson1.
  4. restore the snapshot version and commit this back
  5. run another build on bmir-hudson1 to make sure things are still good

Usually when I update the release version information for this type of situation, I only change the date and time information (2011_04_07_1523). But in this case, the major/minor/micro version numbers follow the owl api versions so I will also update the micro version number. So I modify the pom.xml to look like this:

   < !-- <version>3.2.0.SNAPSHOT</version> -->
   <version>3.2.4.2011_09_08_1008</version>


I commit this and run a build on bmir-hudson1. This ensures that the built version of this library is in the maven repository and you can go check this by going to bmir-hudson1/sonar. If I run this build twice I will get errors because I can't overwrite the release in the maven repository. Therefore it is important to modify the pom.xml to use snapshots and commit that.

Now I am ready to fix the pom.xml for the org.protege.editor.owl. The key line in this file is the dependency on the protege-owlapi-extensions:

       <dependency>
           <groupId>protege4</groupId>
           <artifactId>protege-owlapi-extensions</artifactId>
           <version>3.2.2.2011_04_05_1744</version>
       </dependency>

This needs to be updated and committed. Once this is done the build problems are fixed.


How to deal with snapshot dependencies

When a project has dependencies that are snapshots, that project cannot be built in isolation from the other project builds. This may mean that the nightly builds will either fail or that they won't build and test in the same way that they do on a developers machine. The argument has been made that this means that all the plugins should be tested in one big monolithic test build but this clearly has problems of its own. These problems can be fixed, however, by building and installing the desired snapshot on the hudson build machine. After the snapshot has been installed the builds that depend on that snapshot should hopefully work. In this write-up, I will focus on the simpler problem of what to do when the snapshot is not found. The case where the wrong version of the snapshot is being found may require more care with checking out the right version of the snapshot sources.

In this case, the console output from the from the failing project will include lines such as the following:

[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Can not execute Sonar

Embedded error: Missing:
----------
1) edu.stanford.protege:org.protege.common:jar:4.2.1-SNAPSHOT

  Try downloading the file manually from the project website.

  Then, install it using the command: 
      mvn install:install-file -DgroupId=edu.stanford.protege -DartifactId=org.protege.common -Dversion=4.2.1-SNAPSHOT -Dpackaging=jar -Dfile=/path/to/file

  Alternatively, if you host your own repository you can deploy the file there: 
      mvn deploy:deploy-file -DgroupId=edu.stanford.protege -DartifactId=org.protege.common -Dversion=4.2.1-SNAPSHOT -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]

  Path to dependency: 
  	1) edu.stanford.protege:org.coode.dlquery:bundle:1.1.0-SNAPSHOT
  	2) edu.stanford.protege:org.protege.common:jar:4.2.1-SNAPSHOT

2) edu.stanford.protege:org.protege.editor.core.application:jar:4.2.1-SNAPSHOT

  Try downloading the file manually from the project website.

  Then, install it using the command: 
      mvn install:install-file -DgroupId=edu.stanford.protege -DartifactId=org.protege.editor.core.application -Dversion=4.2.1-SNAPSHOT -Dpackaging=jar -Dfile=/path/to/file

  Alternatively, if you host your own repository you can deploy the file there: 
      mvn deploy:deploy-file -DgroupId=edu.stanford.protege -DartifactId=org.protege.editor.core.application -Dversion=4.2.1-SNAPSHOT -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]

  Path to dependency: 
  	1) edu.stanford.protege:org.coode.dlquery:bundle:1.1.0-SNAPSHOT
  	2) edu.stanford.protege:org.protege.editor.core.application:jar:4.2.1-SNAPSHOT

3) edu.stanford.protege:org.protege.editor.owl:jar:4.2.1-SNAPSHOT

  Try downloading the file manually from the project website.

  Then, install it using the command: 
      mvn install:install-file -DgroupId=edu.stanford.protege -DartifactId=org.protege.editor.owl -Dversion=4.2.1-SNAPSHOT -Dpackaging=jar -Dfile=/path/to/file

  Alternatively, if you host your own repository you can deploy the file there: 
      mvn deploy:deploy-file -DgroupId=edu.stanford.protege -DartifactId=org.protege.editor.owl -Dversion=4.2.1-SNAPSHOT -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]

  Path to dependency: 
  	1) edu.stanford.protege:org.coode.dlquery:bundle:1.1.0-SNAPSHOT
  	2) edu.stanford.protege:org.protege.editor.owl:jar:4.2.1-SNAPSHOT

These errors indicate that hudson cannot find the following three snapshot files:

             edu.stanford.protege:org.protege.common:jar:4.2.1-SNAPSHOT
             edu.stanford.protege:org.protege.editor.core.application:jar:4.2.1-SNAPSHOT
             edu.stanford.protege:org.protege.editor.owl:jar:4.2.1-SNAPSHOT

To make the builds work these three snapshots need to be installed on the hudson account on the hudson machine. This can be arranged in the following steps:

Login to bmir-hudson1.stanford.edu

You will need to talk to one of the administrators to arrange ssh and sudo access on the bmir-hudson1 machine. Then you can ssh to bmir-hudson1.stanford.edu to get a command line prompt there. You will next need to become hudson. If you have sudo access you don't need hudson's password to do this (he probably doesn't have one). You can simply execute the command:

               sudo su hudson

Prepare the sources

First checkout the sources for the projects whose snapshots are needed. In the example above, I checked out the org.protege.common, org.protege.editor.core.application and org.protege.editor.owl projects. When I did this I created a directory under /tmp (e.g., mkdir /tmp/workspace followed by cd /tmp/workspace) and did the checkouts there. Properly speaking you should checkout the "right version" of the code but when I did this I was simply sloppy and checked out the latest.

Then you need to edit the pom.xml file. If you are inexperienced with unix text editors, there is an editor called nano that might work for you. First you need to make sure that the version of the artifact being created by the pom.xml corresponds to the artifact that is needed by the failing build. In this case, all three snapshots needed to be 4.2.1-SNAPSHOT. Again properly speaking, if you checked out the right version of the sources this shouldn't be necessary. Second, you will need to disable the gpg plugin. If you search forward for gpg, you will find a plugin execution for gig that you should delete.

Install the snapshots

With luck all that needs to be done now is mvn install. This should install the snapshots and the failing build should succeed. I just did these steps and the status is all blue.

Personal tools