Writing Your Own SonarQube Plug-ins – Part 4: Testing and Deploying

In part 1 of this series here, I explained the basics of SonarQube plug-ins and how to start writing your own plug-in. In part 2 here, I explained how to define custom metrics and sensor classes to collect metric data (measures). In part 3 here, I showed the basics of how to create custom widgets to present to users the information we are collecting from analysed projects.

Now finally in part 4, I will show how to test and deploy your plug-ins, and some tips to make development and testing workarounds quicker, like configuring JRebel to speed up development.

EDIT: Source code for the simple plug-in shown during these posts can be obtained from GitHub here: https://github.com/deors/deors.plugins.sonarqube.idemetadata

Part 4 – Testing and Deploying

If you have been following this series of blog posts, you already have a simple structure for a plug-in, hopefully doing something useful and showing valuable insights from your code and configuration.

So what’s next? How can we test and deploy our plug-ins? The simple naïve answer is: package and deploy it to a working SonarQube instance, as you would do with any other plug-in.

Packaging the plug-in is as easy as executing Maven’s package goal. The Maven configuration, governed by the sonar-plugin packaging type, will ensure that the right classes and configuration files are packaged in the right way. The resulting Jar file will be ready for deployment into a SonarQube instance.

Deploying the plug-in is also an easy task. If you are a seasoned SonarQube user, you probably know how to install and update public plug-ins from SonarQube update center. Unfortunately this way is only available for plug-ins listed in the public SonarQube directory, and is not suitable for our own in-house, internal-only plug-ins. In our case, we would need to manually copy the packaged plug-in file to our SonarQube instance, in the folder extensions\downloads, and then restart SonarQube. This is actually what the update center does: downloading a plug-in file from Internet and place it in downloads folder, so it is picked up automatically in next restart.

Although the process above is simple and easy to automate with a script, it is lengthy. Moreover, it needs a local SonarQube instance that you need to manage, and that it’s subject to get polluted by other plug-ins or just by normal user usage. Fortunately, SonarQube provides with a better alternative way to test a plug-in.

SonarQube Development Mode

To prevent developer’s sanity, SonarQube provides with a development mode. This development mode will automate the deploy process as a Maven plug-in, downloading a SonarQube instance from the internet if needed and running it as a child process of the Maven process.

To launch the development mode, just issue this command:

mvn install org.codehaus.sonar:sonar-dev-maven-plugin:1.7:start-war -Dsonar.runtimeVersion=3.7.3

You may setup an alternate version of SonarQube to host and run the plug-in as needed. The SonarQube instance spawned by this command will run on default settings: listening on port 9000, with embedded H2 database.

Once ready, just launch a SonarQube analysis, either using SonarQube Maven plug-in or using SonarQube Runner standalone analysis. The analysis will connect by default to the server in development mode and run your plug-in as part of the analysis process.

Optimized Testing with JRebel

Although SonarQube development mode is nice and very convenient, it has a fundamental problem – it is slow! It takes quite a few minutes to have the environment prepared, and if you find a bug or want to improve something, and you need to change some Java code, you have to drop it and start the process again.

Fortunately, JRebel comes to a rescue and can dramatically speed up the process. The same that JRebel monitors Java web containers and injects changes in Java bytecodes directly into the running JVM that hosts the web container, without loosing state – effectively saving you the few (or many) minutes needed to redeploy and/or restart the web container – JRebel can monitor our plug-in under development.

To configure JRebel, just add JRebel configuration and agent to the developer mode launch script.

JRebel configuration is just a simple rebel.xml file under src/main/resources folder, which content is set to monitor the right folder where Java classes are compiled into:

<?xml version="1.0" encoding="UTF-8"?>
<application xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.zeroturnaround.com" xsi:schemaLocation="http://www.zeroturnaround.com http://www.zeroturnaround.com/alderaan/rebel-2_0.xsd">
    <classpath>
        <dir name="C:/projects/xxxxx/target/classes"/>
    </classpath>
</application>

Then, add the JRebel agent to the development mode launch command:

mvn install org.codehaus.sonar:sonar-dev-maven-plugin:1.7:start-war -Dsonar.runtimeVersion=3.7.3 -Dsonar.containerArgs=-javaagent:C:\java\tools\jrebel-5.5.2\jrebel.jar -Drebel.log=true

This is possible thanks to sonar.containerArgs command-line options, kindly provided by ZeroTurnaround support team as a merge request into open-sourced SonarQube development mode plug-in. Who said vendor support does not work?

Once launched with JRebel, the development mode may remain active during the full development and testing session, without needing to redeploy or restart it again, and picking changes done in plug-in classes.

Evenmore, if the SonarQube analysis is launched with JRebel agent active, each subsequent SonarQube analysis process that is executed will pick changes in the plug-in, effectively making the plug-in code to be ‘reloaded’ automatically, without needing to redeploy it again. To do that, don’t forget to add this to the SonarQube analysis command:

mvn sonar:sonar -javaagent:c:\java\tools\jrebel-5.5.2\jrebel.jar

Conclusion

It is easy to get started with custom plug-in development in SonarQube if you know where to start. Although open-source plug-ins are a good source of useful information, the online documentation lacks of a good plug-in 101 guide. I really hope that this short blog post series has been of help, it allows you to connect all the pieces that confirms a plug-in, and encourages you to conceive and bring to life your own custom plug-ins, either analysers, reporters or integrators with other existing tools.

Happy coding!

Advertisements

Writing Your Own SonarQube Plug-ins – Part 3: Creating Widgets

In part 1 of this series here, I explained the basics of SonarQube plug-ins and how to start writing your own plug-in. In part 2 here, I explained how to define custom metrics and sensor classes to collect metric data (measures).

In part 3, I will show the basics of how to create custom widgets to present to users the information we are collecting from analysed projects.

EDIT: Source code for the simple plug-in shown during these posts can be obtained from GitHub here: https://github.com/deors/deors.plugins.sonarqube.idemetadata

Widgets in SonarQube are typically written in Ruby on Rails. SonarQube platform comes with a lot of existing widgets, views, parts, etc. that can be combined into dashboards, which will present different set of information based on roles, but also will allow to drilldown measures or issues from top-level projects to source code.

Part 3 – Creating Widgets

The objective for the widget we would like to create is to simply show collected project-level information in project’s dashboard: which type of project we have configured in the IDE, which dependencies exist with other projects, etc.

In part 1, when creating the plug-in class, we instructed SonarQube to find a widget definition contained in IDEMetadataDashboardWidget class.

A widget class is very simple to create:

  • Create a class extending org.sonar.api.web.AbstractRubyTemplate and implementing org.sonar.api.web.RubyRailsWidget.
  • Annotate the class with @org.sonar.api.web.UserRole. Add as parameter the roles that will have access to this widget: Admin, User, CodeViewer.
  • Optionally annotate the class with @org.sonar.api.web.Description. Add as parameter the description for the widget, as it will be showed in SonarQube dashboard configuration view. Tell the users what is your widget meant to do!
  • Implement getId() and getTitle() methods. Simply return a string with the widget id and title. The id should be unique and the title will later be showed in SonarQube dashboard configuration view.
  • Implement getTemplatePath() method. This method will return a string with the path where the Ruby page can be found. The path should be relative to the plug-in project classpath (starting with a forward slash). During development, however, it is useful to set the path as an absolute path in your file system. Doing that will allow for automatic refresh of changes in the widget while you are coding and testing it. But don’t forget to revert back to the relative path when you are ready to deploy the plug-in!

It’s really simple – it takes more time to explain it than to actually code it ;-). See below the content of the widget class, without Javadocs to simplify the post.

@UserRole(UserRole.USER)
@Description("Shows IDE metadata configuration for the project, including project type, language support and configured frameworks")
public class IDEMetadataDashboardWidget extends AbstractRubyTemplate implements RubyRailsWidget {
    public String getId() {
        return "idemetadata";
    }
    public String getTitle() {
        return "IDE Metadata";
    }
    protected String getTemplatePath() {
        // uncomment next line for change reloading during development
        //return "c:/projects/xxxxx/src/main/resources/xxxxx/sonar/idemetadata/idemetadata_widget.html.erb";
        return "/xxxxx/sonar/idemetadata/idemetadata_widget.html.erb";
    }
}

Creating the View

Now it’s time to actually create the view. Views are created as Ruby on Rails pages. They are simple: even if you have not coded in Ruby on Rails before, you are probably familiarised with other templating engines so the jump to Rails should not be a challenge. Evenmore, the large amount of existing views that are available inside SonarQube are excellent to learn and apply patterns to your own views.

At the end, is HTML, styles and JavaScript what you are delivering to the clients, so just think with open mind what you can do with this mix of technologies. Do you want to use a JavaScript library because you like some of its visual components? You can. Are you a fan of Google charts and graphs? Use them. Are you a jQuery ninja? Unleash that power on your benefit.

If you don’t know where to start, pick an existing widget and make changes here and there, look at the existing JavaScript libraries to interact with SonarQube API, and you will be ready in no time.

Some pieces of advice that can help you to ease that journey:

  • You can place your own static resources (like images) in src/main/resources/static folder. Then you can refer to them from the widget with this code (note the reference to the plug-in id):
url_for_static(:plugin => 'idemetadata', :path => 'check-round-yes-small.png')
  • If you want to place an image in any place of the view, use the image_tag() function:
image_tag(url_for_static(:plugin => 'idemetadata', :path => 'check-round-yes-small.png'), :size => '16x16', :style => 'vertical-align:middle')
  • View consistency is very important. Don’t confuse your users and leverage the standard SonarQube styles. They are available to custom widgets as well and will help you to provide a consistent user interface.

Below you can see a short snippet of the view:

Project name (as configured in the IDE):

<div class="dashbox"> <table class="width100"> <tbody> <tr> <td> <h3>Project type and active frameworks:</h3> <table> <tbody> <tr> <td> <%= check_image('ide_is_java') -%> <span>Java VM</span> </td> </tr> ...

This image shows how the configuration view looks like when you are going to add a view into a dashboard:

sonar-plugin-1

And the images below are a couple of examples on how the view looks like once placed in a project’s dashboard, showing the results of the plugin IDE metadata files scan and analysis:

sonar-plugin-2

 sonar-plugin-3

What’s next

In the next and final post of this series, the part 4, I will explain how to execute the plug-in in development mode and how to install it in your production SonarQube instance, with the help of JRebel to speed up development.

Writing Your Own SonarQube Plug-ins – Part 2: Defining Custom Metrics and Sensors

In part 1 of this series here, I explained the basics of SonarQube plug-ins and how to start writing your own plug-in.

EDIT: Source code for the simple plug-in shown during these posts can be obtained from GitHub here: https://github.com/deors/deors.plugins.sonarqube.idemetadata

A plug-in may consist on a series of Metrics, Sensors, Decorators and Widgets, although not all of them are required in any single plug-in.

Metric classes define custom metrics, Sensors are used to scan projects and calculate metric values (measures, in SonarQube terminology), Decorators are used to calculate derived values for a given project resource (typically using several measures to do the calculation) and Widgets are used to present information in SonarQube dashboard.

When designing a new SonarQube plug-in, it’s important to differentiate between Sensors and Decorators. A Sensor is executed once per analysis. It typically scans the project folder structure recursively, calculating metric values, i.e. with the help of an external tool, and stores the information in SonarQube database. A Decorator is executed once per resource, after all Sensors are executed. Decorators can query for existing metric values, calculate derived values and store them in SonarQube database. Decorators are typically used to calculate aggregated values or to calculate metrics that rely on values coming from multiple sources.

Part 2 – Defining Custom Metrics and Sensors

The plug-in we are developing during these posts is a simple one that collects information from IDE configuration files, i.e. whether the project is configured as a Web project, an EJB project, etc.

The plug-in will rely on a EclipseAnalyzer class to do the hard work (details not interesting for the purpose of these posts).

A Metric class will define the set of metrics that this plug-in will gather. A Sensor class will execute the EclipseAnalyzer on the context of the analysis, and store the results using SonarQube API (no direct database access is allowed, an it’s much simpler this way).

Let’s view a bit more in detail of each one.

Defining a Metric Class

A Metric class in SonarQube API is very simple to define: create a class implementing the interface org.sonar.api.measures.Metrics. This interface only specifies one method to be implemented: List<org.sonar.api.Metric> getMetrics(). That is, a simple method returning a list of Metric objects.

Each Metric object returned will represent a Metric entity in SonarQube model. Each Metric can store value of a certain data type (boolean, string, integer values, floating-point values, percentages, etc.), be qualitative or quantitative, aggregated automatically in some direction (upward or downward), and other characteristics.

The simplest way to define a Metric is to use the Metric builder pattern, as can be seen in code fragment below:

public static final Metric IDE_IS_JAVA =
    new Metric.Builder(
        "ide_is_java", // metric identifier
        "Java project", // metric name
        Metric.ValueType.BOOL) // metric data type
    .setDescription("Whether the project is configured as Java in the IDE")
    .setQualitative(false)
    .setDomain(CoreMetrics.DOMAIN_GENERAL)
    .create();

After defining as many metrics as needed, the implementation of the getMetrics() method is straightforward:

public List<Metric> getMetrics() {
    return Arrays.asList(
        IDE_PRJ_NAME, IDE_IS_JAVA, IDE_IS_EAR, IDE_IS_EJB, IDE_IS_WEB,
        IDE_IS_GWT, IDE_IS_GAE, IDE_IS_GROOVY, IDE_IS_GRAILS,
        IDE_IS_PDE, IDE_IS_JET, IDE_DEPENDENCIES, IDE_CLASSPATH);
}

Defining a Sensor Class

A Sensor class in SonarQube API is also very simple to define: create a class implementing the interface org.sonar.api.batch.Sensor and implement the two specified methods.

The first one is boolean shouldExecuteOnProject(org.sonar.api.resources.Project), that tells SonarQube analyser whether it should execute on any particular project or not. The Project object provides the information needed to take decision: the project name, language, analysis type, the list of modules it contains, etc. In our case, it will always return true.

The second one is void analyse(org.sonar.api.resources.Project, org.sonar.api.batch.SensorContext). This is the method where all the magic happens: inside it the project resources will be scanned, metric values calculated and stored in SonarQube database – hopefully with the help of other classes and methods.

To get access to the project directory structure, the recommended way is to leverage the dependency injection mechanism of the Plexus container where SonarQube analysis runs. Let’s define a constructor this way and Plexus will do the rest for us:

public IDEMetadataSensor(org.sonar.api.scan.filesystem.ModuleFileSystem fileSystem) {
    this.fileSystem = fileSystem;
}

Independently of the mechanism used to calculate metric values, once they are ready, they will be store using the saveMeasure(org.sonar.api.measures.Measure) method of interface org.sonar.api.batch.SensorContext.

A Measure is created using a simple constructor with metric id and value. The sensor context will determine the resource to which the metric value belongs to.

For example:

Measure measure = new Measure(IDEMetadataMetrics.IDE_PRJ_NAME, projectInfo.getProjectName());
sensorContext.saveMeasure(measure);
measure = new Measure(IDEMetadataMetrics.IDE_IS_JAVA, projectInfo.isJavaProject() ? 1d : 0d); // for boolean values, 1 is true, 0 is false
sensorContext.saveMeasure(measure);

When a measure (or a violation if our plug-in is profiling code) needs to apply to a specific resource inside the project, i.e. a Java class, you can use the sensorContext.getResource() method to obtain the right place and use it to create and save the measure using the saveMeasure(Resource, Measure) method.

Finally, when in doubt, it’s always useful to access SonarQube plug-in source files on GitHub, and take a look to Checkstyle or FindBugs plug-ins.

What’s next

Next, in part 3, I will explain how to create widgets and finally in part 4 I will explain how to execute the plug-in in development mode and how to install it in your production SonarQube instance, with the help of JRebel to speed up development.