About deors

senior technology architect in accenture, with a passion for java related stuff, celtic music and the best sci-fi, among other thousand things!

Writing Your Own SonarQube Plug-ins – Part 3: Creating Widgets

In part 1 of this series here, I explained the basics of SonarQube plug-ins and how to start writing your own plug-in. In part 2 here, I explained how to define custom metrics and sensor classes to collect metric data (measures).

In part 3, I will show the basics of how to create custom widgets to present to users the information we are collecting from analysed projects.

Widgets in SonarQube are typically written in Ruby on Rails. SonarQube platform comes with a lot of existing widgets, views, parts, etc. that can be combined into dashboards, which will present different set of information based on roles, but also will allow to drilldown measures or issues from top-level projects to source code.

Part 3 – Creating Widgets

The objective for the widget we would like to create is to simply show collected project-level information in project’s dashboard: which type of project we have configured in the IDE, which dependencies exist with other projects, etc.

In part 1, when creating the plug-in class, we instructed SonarQube to find a widget definition contained in IDEMetadataDashboardWidget class.

A widget class is very simple to create:

  • Create a class extending org.sonar.api.web.AbstractRubyTemplate and implementing org.sonar.api.web.RubyRailsWidget.
  • Annotate the class with @org.sonar.api.web.UserRole. Add as parameter the roles that will have access to this widget: Admin, User, CodeViewer.
  • Optionally annotate the class with @org.sonar.api.web.Description. Add as parameter the description for the widget, as it will be showed in SonarQube dashboard configuration view. Tell the users what is your widget meant to do!
  • Implement getId() and getTitle() methods. Simply return a string with the widget id and title. The id should be unique and the title will later be showed in SonarQube dashboard configuration view.
  • Implement getTemplatePath() method. This method will return a string with the path where the Ruby page can be found. The path should be relative to the plug-in project classpath (starting with a forward slash). During development, however, it is useful to set the path as an absolute path in your file system. Doing that will allow for automatic refresh of changes in the widget while you are coding and testing it. But don’t forget to revert back to the relative path when you are ready to deploy the plug-in!

It’s really simple – it takes more time to explain it than to actually code it ;-). See below the content of the widget class, without Javadocs to simplify the post.

@UserRole(UserRole.USER)
@Description("Shows IDE metadata configuration for the project, including project type, language support and configured frameworks")
public class IDEMetadataDashboardWidget extends AbstractRubyTemplate implements RubyRailsWidget {
    public String getId() {
        return "idemetadata";
    }
    public String getTitle() {
        return "IDE Metadata";
    }
    protected String getTemplatePath() {
        // uncomment next line for change reloading during development
        //return "c:/projects/xxxxx/src/main/resources/xxxxx/sonar/idemetadata/idemetadata_widget.html.erb";
        return "/xxxxx/sonar/idemetadata/idemetadata_widget.html.erb";
    }
}

Creating the View

Now it’s time to actually create the view. Views are created as Ruby on Rails pages. They are simple: even if you have not coded in Ruby on Rails before, you are probably familiarised with other templating engines so the jump to Rails should not be a challenge. Evenmore, the large amount of existing views that are available inside SonarQube are excellent to learn and apply patterns to your own views.

At the end, is HTML, styles and JavaScript what you are delivering to the clients, so just think with open mind what you can do with this mix of technologies. Do you want to use a JavaScript library because you like some of its visual components? You can. Are you a fan of Google charts and graphs? Use them. Are you a jQuery ninja? Unleash that power on your benefit.

If you don’t know where to start, pick an existing widget and make changes here and there, look at the existing JavaScript libraries to interact with SonarQube API, and you will be ready in no time.

Some pieces of advice that can help you to ease that journey:

  • You can place your own static resources (like images) in src/main/resources/static folder. Then you can refer to them from the widget with this code (note the reference to the plug-in id):
url_for_static(:plugin => 'idemetadata', :path => 'check-round-yes-small.png')
  • If you want to place an image in any place of the view, use the image_tag() function:
image_tag(url_for_static(:plugin => 'idemetadata', :path => 'check-round-yes-small.png'), :size => '16x16', :style => 'vertical-align:middle')
  • View consistency is very important. Don’t confuse your users and leverage the standard SonarQube styles. They are available to custom widgets as well and will help you to provide a consistent user interface.

Below you can see a short snippet of the view:

<div class="dashbox">
    <h3>Project name (as configured in the IDE):</h3>
    <p>
        <span class="big">
            <%= format_measure('ide_prj_name') %>
        </span>
    </p>
</div>
<div class="dashbox">
    <table class="width100">
        <tbody>
            <tr>
                <td>
                    <h3>Project type and active frameworks:</h3>
                    <table>
                        <tbody>
                            <tr>
                                <td>
                                    <%= check_image('ide_is_java') -%>
                                    <span>Java VM</span>
                                </td>
                            </tr>
...

This image shows how the configuration view looks like when you are going to add a view into a dashboard:

sonar-plugin-1

And the images below are a couple of examples on how the view looks like once placed in a project’s dashboard, showing the results of the plugin IDE metadata files scan and analysis:

sonar-plugin-2

 sonar-plugin-3

What’s next

Next, in part 4, I will explain how execute the plug-in in development mode and how to distribute and install it in your production SonarQube instance.

Writing Your Own SonarQube Plug-ins – Part 2: Defining Custom Metrics and Sensors

In part 1 of this series here, I explained the basics of SonarQube plug-ins and how to start writing your own plug-in.

A plug-in may consist on a series of Metrics, Sensors, Decorators and Widgets, although not all of them are required in any single plug-in.

Metric classes define custom metrics, Sensors are used to scan projects and calculate metric values (measures, in SonarQube terminology), Decorators are used to calculate derived values for a given project resource (typically using several measures to do the calculation) and Widgets are used to present information in SonarQube dashboard.

When designing a new SonarQube plug-in, it’s important to differentiate between Sensors and Decorators. A Sensor is executed once per analysis. It typically scans the project folder structure recursively, calculating metric values, i.e. with the help of an external tool, and stores the information in SonarQube database. A Decorator is executed once per resource, after all Sensors are executed. Decorators can query for existing metric values, calculate derived values and store them in SonarQube database. Decorators are typically used to calculate aggregated values or to calculate metrics that rely on values coming from multiple sources.

Part 2 – Defining Custom Metrics and Sensors

The plug-in we are developing during these posts is a simple one that collects information from IDE configuration files, i.e. whether the project is configured as a Web project, an EJB project, etc.

The plug-in will rely on a EclipseAnalyzer class to do the hard work (details not interesting for the purpose of these posts).

A Metric class will define the set of metrics that this plug-in will gather. A Sensor class will execute the EclipseAnalyzer on the context of the analysis, and store the results using SonarQube API (no direct database access is allowed, an it’s much simpler this way).

Let’s view a bit more in detail of each one.

Defining a Metric Class

A Metric class in SonarQube API is very simple to define: create a class implementing the interface org.sonar.api.measures.Metrics. This interface only specifies one method to be implemented: List<org.sonar.api.Metric> getMetrics(). That is, a simple method returning a list of Metric objects.

Each Metric object returned will represent a Metric entity in SonarQube model. Each Metric can store value of a certain data type (boolean, string, integer values, floating-point values, percentages, etc.), be qualitative or quantitative, aggregated automatically in some direction (upward or downward), and other characteristics.

The simplest way to define a Metric is to use the Metric builder pattern, as can be seen in code fragment below:

public static final Metric IDE_IS_JAVA =
    new Metric.Builder(
        "ide_is_java", // metric identifier
        "Java project", // metric name
        Metric.ValueType.BOOL) // metric data type
    .setDescription("Whether the project is configured as Java in the IDE")
    .setQualitative(false)
    .setDomain(CoreMetrics.DOMAIN_GENERAL)
    .create();

After defining as many metrics as needed, the implementation of the getMetrics() method is straightforward:

public List<Metric> getMetrics() {
    return Arrays.asList(
        IDE_PRJ_NAME, IDE_IS_JAVA, IDE_IS_EAR, IDE_IS_EJB, IDE_IS_WEB,
        IDE_IS_GWT, IDE_IS_GAE, IDE_IS_GROOVY, IDE_IS_GRAILS,
        IDE_IS_PDE, IDE_IS_JET, IDE_DEPENDENCIES, IDE_CLASSPATH);
}

Defining a Sensor Class

A Sensor class in SonarQube API is also very simple to define: create a class implementing the interface org.sonar.api.batch.Sensor and implement the two specified methods.

The first one is boolean shouldExecuteOnProject(org.sonar.api.resources.Project), that tells SonarQube analyser whether it should execute on any particular project or not. The Project object provides the information needed to take decision: the project name, language, analysis type, the list of modules it contains, etc. In our case, it will always return true.

The second one is void analyse(org.sonar.api.resources.Project, org.sonar.api.batch.SensorContext). This is the method where all the magic happens: inside it the project resources will be scanned, metric values calculated and stored in SonarQube database – hopefully with the help of other classes and methods.

To get access to the project directory structure, the recommended way is to leverage the dependency injection mechanism of the Plexus container where SonarQube analysis runs. Let’s define a constructor this way and Plexus will do the rest for us:

public IDEMetadataSensor(org.sonar.api.scan.filesystem.ModuleFileSystem fileSystem) {
    this.fileSystem = fileSystem;
}

Independently of the mechanism used to calculate metric values, once they are ready, they will be store using the saveMeasure(org.sonar.api.measures.Measure) method of interface org.sonar.api.batch.SensorContext.

A Measure is created using a simple constructor with metric id and value. The sensor context will determine the resource to which the metric value belongs to.

For example:

Measure measure = new Measure(IDEMetadataMetrics.IDE_PRJ_NAME, projectInfo.getProjectName());
sensorContext.saveMeasure(measure);
measure = new Measure(IDEMetadataMetrics.IDE_IS_JAVA, projectInfo.isJavaProject() ? 1d : 0d); // for boolean values, 1 is true, 0 is false
sensorContext.saveMeasure(measure);

When a measure (or a violation if our plug-in is profiling code) needs to apply to a specific resource inside the project, i.e. a Java class, you can use the sensorContext.getResource() method to obtain the right place and use it to create and save the measure using the saveMeasure(Resource, Measure) method.

Finally, when in doubt, it’s always useful to access SonarQube plug-in source files on GitHub, and take a look to Checkstyle or FindBugs plug-ins.

What’s next

Next, in part 3, I will explain how to create widgets and finally in pat 4 I will explain how to execute the plug-in in development mode and how to install it in your production SonarQube instance.

Writing Your Own SonarQube Plug-ins – Part 1: Preparing the Project

Since a few weeks ago I’ve been spending some free time on writing my own custom plug-in for SonarQube (formerly Sonar, www.sonarqube.org). SonarQube is a code analysis tool and dashboard, that lets you not only inspect code as you go, but also publish and track all metrics in a dashboard, manage your quality reviews, define action plans, and much more.

Custom plug-ins enhance SonarQube functionality in multiple ways: a plug-in may support static analysis of new languages, add new metrics to the tool, or change the way that information is presented (widgets, views) and managed (action plans, peer reviews).

Creating new plug-ins is not complex, given that you know what to do – documentation is not as complete as it should, you get pieces and bits here and there, but at the end you need to take a look to existing sources and make all the connections. Based on my own experiences I’ve decided to write a few posts documenting the process, with the hope that it can be of help to others. :-)

Part 1 – Preparing the Project

Now you have an idea for a plug-in – what’s next? You need to setup a project that will hold all plug-in code and configuration. SonarQube plug-in projects can be easily configured using Maven, with the added benefit that you leverage the well known project folder structure and dependency management mechanism.

A SonarQube plug-in has three requirements, in terms of Maven configuration, that makes them different to a regular Java project:

  1. A special type of packaging, sonar-plugin. This custom packaging is used to fine tune the project life-cycle without requiring us to tweak plug-in configuration within the pom.xml file.
  2. A dependency with SonarQube plug-in API. Artefact coordinates are: org.codehaus.sonar:sonar-plugin-api:3.7.3 - replace version with the one that is right for your plug-in, taking into consideration compatibility.
  3. A build plug-in that will take care of the specifics of packaging the plug-in for deployment into a SonarQube installation. The plug-in coordinates are: org.codehaus.sonar:sonar-packaging-maven-plugin:1.7 - 1.7 is the latest one. Configuration for the SonarQube packaging plug-in should include at least the plug-in key (an unique identifier for this plug-in) and the plug-in main class (more about this later).

Putting this all together, the POM for the plug-in project looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">

    <groupId>com.wordpress.deors</groupId>
    <artifactId>tools.sonarqube.idemetadata</artifactId>
    <packaging>sonar-plugin</packaging>
    <version>1.0-SNAPSHOT</version>

    <dependencies>
        <dependency>
            <groupId>org.codehaus.sonar</groupId>
            <artifactId>sonar-plugin-api</artifactId>
            <version>3.7.3</version>
        </dependency>
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>org.codehaus.sonar</groupId>
                <artifactId>sonar-packaging-maven-plugin</artifactId>
                <version>1.7</version>
                <extensions>true</extensions>
                <configuration>
                    <pluginKey>idemetadata</pluginKey>
                    <pluginClass>deors.tools.sonarqube.idemetadata.IDEMetadataPlugin</pluginClass>
                    <pluginName>Sonar IDE Metadata plugin</pluginName>
                    <pluginDescription>Gathers and displays information from IDE metadata files, including project type (based on natures/facets) and dependencies.</pluginDescription>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

The Plug-in Main Class

The entry point for every SonarQube plug-in is a class that extends the org.sonar.api.SonarPlugin abstract class. A plug-in is modelled as a list of extensions (implementations of the interface org.sonar.api.Extension).

The SonarQube plug-in main class unique purpose is to define which extensions are contributed by the plug-in – widgets, sensors, decorators, metrics. In this case I was building a plug-in that contributed a set of custom metrics, a sensor (component that scans the code, gathers the metrics and stores them in SonarQube) and a widget (a dashboard view used to present the gathered information to users). The final code was like this (package, import and Javadocs omitted):

public class IDEMetadataPlugin extends SonarPlugin {
    public List<Class<? extends Extension>> getExtensions() {
        return Arrays.asList(
            IDEMetadataMetrics.class,
            IDEMetadataSensor.class,
            IDEMetadataDashboardWidget.class);
    }
}

Very simple definition, and once done, we are ready to go.

Going forward, in part 2 I will explain how to define metrics and the sensor class, in part 3 I will explain how to create widgets and finally in pat 4 I will explain how to execute the plug-in in development mode and how to install it in your production SonarQube instance.

Next-generation IDEs talk at OpenSlava 2013 conference

On October 11th, 2013 I participated in the first ever OpenSlava conference (www.openslava.sk).

openslava

Located in Bratislava, Slovakia, this conference was devoted to the latest and greatest around Java and open source technologies. With talks about dynamic languages, Node.js, data architectures or automated infrastructure provisioning with Chef, OpenSlava tried to establish itself as a reference in the region, and I really thing it exceeded all expectations.

Excellent organization, impressive selection of topics and warm and crowded reception by academicals, students and professionals from the region. I fell honoured to have been part of it.

The talk I contributed with was “Next-generation IDEs”. A journey trough IDE features and characteristics from the point of view of three people: Lisa, an undergraduate student; Stefan, and enterprise developer with 2 years IT experience; and Adam, a hardcore developer with many years of experience, passionate about coding and contributor to open source projects. While each of them focus on the IDE with different priorities, their collective experiences and needs will help us to determine which characteristics a Next-generation IDE would have.

Deck and session recordings will be made public on conference site shortly, but in the meantime you can get the slides directly in the link below:

OpenSlava – Next-Gen IDEs v2 – Jorge Hidalgo

Edit: Session recording is now available at YouTube: http://www.youtube.com/watch?v=QipY0vcgVA8

Check OpenSlava channel for many other awesome presentations. Don’t miss them!

Idiom for Browser-Selectable Selenium Tests

For some time I’ve wanted to share an idiom I personally use and recommend when building Selenium Tests. This idiom allows to control which browsers are used to run the tests without needing to update test sources or configuration.

The simple ideas behind this idiom are:

  • Test code and configuration should not depend on the test environment.
  • Tests can be executed in any given browser, independently from others.
  • To change the browsers used for test execution, it is not needed to update test sources or configuration.
  • Selenium Grid URL and application URL are also configurable.
  • Both environment variables and Java system variables can be used.
  • All settings have sensible defaults.

I call this idiom ‘Browser-Selectable Tests’. I promise I keep thinking on a better name :-)

Continue reading

Assorted Tips – Oracle VM VirtualBox

With this post I start a new series with short tips about tools and frameworks. My main motivation is to add a “safety net” to my OneNote notebook full of notes and, who knows, maybe they are useful to others as well.

Oracle VM VirtualBox

Through the command-line you can issue a number of commands that can be useful for advanced users. (For the rest of the post let’s assume you are in a command prompt where VirtualBox is installed.)

Headless VMs

You can use command-line to start a VM in headless mode (you will need to enable remote desktop to access it, be warned!):

VBoxHeadless -startvm <vm-name>

You can also save the state of a running VM using the following command:

VBoxManage controlvm <vm-name> savestate

(Use double quotes to surround the VM name if it has spaces.)

Continue reading

Selenium WebDriver: Waiting for an application to be fully loaded

While working with Selenium WebDriver to automate web application tests in multiple browsers and application platforms, we found out some cases in which the application under test was not fully loaded when our tests executed, causing the test with the first browser to fail (but not the subsequent ones).

For example, when using Cargo to provision an embedded JBoss container, the ‘server ready’ flag was sent once the HTTP services were available, but the application was not loaded until first request hit the server.

After unsuccessfully trying with some Cargo settings, we considered adapting the test script to handle this.

To our surprise, the WebDriver API provided us with a very elegant, minimally disruptive, way of handling with this situation.

In previous posts we were simply using a pattern like this one to load a page, find a link and click on it:

driver.get(baseUrl);
driver.findElement(By.linkText("Find owner")).click();

In those posts we also used another pattern for waiting a page to be loaded after a link click or form submit:

(new WebDriverWait(driver, 5)).until(new ExpectedCondition<Boolean>() {
    public Boolean apply(WebDriver d) {
        return d.getCurrentUrl().startsWith(baseUrl + "/owners/search");
    }
});

Up to here, nothing new. WebDriverWait and ExpectedCondition allows for defining a wide range of conditions, like waiting for a new page to become available, waiting for a new field to be enabled or waiting for some AJAX response to be received. Combining the previous two snippets, we can write a condition that reads as: try loading this page until it contains a link with the text “Find owner”, but do not wait for more than five seconds. Here is the code we used:

// wait for the application to get fully loaded
WebElement findOwnerLink = (new WebDriverWait(driver, 5)).until(new ExpectedCondition<WebElement>() {
    public WebElement apply(WebDriver d) {
        d.get(baseUrl);
        return d.findElement(By.linkText("Find owner"));
    }
});

findOwnerLink.click();

With this small change in code, the test script waits until the application is fully loaded and the first page in the test sequence is available.

... Cargo output provisioning the server and JBoss starting up

[INFO] [talledLocalContainer] 09:20:06,004 INFO [org.jboss.as.server.deployment] (MSC service thread 1-1) JBAS015876: S
tarting deployment of "petclinic.war"
[INFO] [talledLocalContainer] 09:20:06,004 INFO [org.jboss.as.server.deployment] (MSC service thread 1-3) JBAS015876: S
tarting deployment of "cargocpc.war"
[INFO] [talledLocalContainer] 09:20:07,779 INFO [org.jboss.web] (MSC service thread 1-5) JBAS018210: Registering web co
ntext: /cargocpc
[INFO] [talledLocalContainer] JBoss 7.1.1.Final started on port [8180]

... JBoss Cargo adapter sends the 'ready' flag at this point

[INFO]
[INFO] --- maven-failsafe-plugin:2.8.1:integration-test (integration-test) @ org.springframework.samples.petclinic-rhc -
--
[INFO] Failsafe report directory: C:\projects\deors.demos\petclinic\org.springframework.samples.petclinic-rhc\target\fai
lsafe-reports
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running org.springframework.samples.petclinic.it.NewPetFirstVisitIT

... Test script is waiting!

[INFO] [talledLocalContainer] 09:20:12,521 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-1) JBAS01
0404: Deploying non-JDBC-compliant driver class com.mysql.jdbc.Driver (version 5.1)
[INFO] [talledLocalContainer] 09:20:12,605 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/pe
tclinic]] (MSC service thread 1-2) Initializing Spring root WebApplicationContext
[INFO] [talledLocalContainer] 09:20:12,608 INFO [org.springframework.web.context.ContextLoader] (MSC service thread 1-2
) Root WebApplicationContext: initialization started
[INFO] [talledLocalContainer] 09:20:12,648 INFO [org.springframework.web.context.support.XmlWebApplicationContext] (MSC
 service thread 1-2) Refreshing Root WebApplicationContext: startup date [Fri Jan 11 09:20:12 CET 2013]; root of context
 hierarchy
[INFO] [talledLocalContainer] 09:20:12,704 INFO [org.springframework.beans.factory.xml.XmlBeanDefinitionReader] (MSC se
rvice thread 1-2) Loading XML bean definitions from ServletContext resource [/WEB-INF/classes/applicationContext-jdbc.xm
l]
[INFO] [talledLocalContainer] 09:20:13,037 INFO [org.springframework.beans.factory.xml.XmlBeanDefinitionReader] (MSC se
rvice thread 1-2) Loading XML bean definitions from ServletContext resource [/WEB-INF/classes/applicationContext-dataSou
rce.xml]

... Rest of JBoss and Pet Clinic initialization - test is waiting for the app to be available

[INFO] [talledLocalContainer] 09:20:16,743 INFO [org.jboss.as.server] (DeploymentScanner-threads - 2) JBAS018559: Deplo
yed "petclinic.war"
[INFO] [talledLocalContainer] 09:20:16,744 INFO [org.jboss.as.server] (DeploymentScanner-threads - 2) JBAS018559: Deplo
yed "cargocpc.war"
Tests run: 6, Failures: 0, Errors: 0, Skipped: 5, Time elapsed: 11.745 sec

... The test executed successfully (some browsers skipped)

Out of curiosity, this does not happen with Cargo and Tomcat/Jetty, but it did no harm to add the wait!

Happy testing!

P.S.: More on WebDriver waits here:  http://seleniumhq.org/docs/04_webdriver_advanced.jsp