Installing Sonar in OpenShift as a DIY application

Note: this is an excerpt extracted from my talk at Red Hat Developer Day London. You can see more about the talk in my post here:

http://deors.wordpress.com/2012/10/03/developer-day/

Sonar is a popular code profiler and dashboard that excels when used along a Continuous Integration engine:

  • Seamless integration with Maven.
  • Leverages best-of-breed tools as Checkstyle, PMD or FindBugs.
  • Configurable quality profiles.
  • Re-execution of tests and test code coverage (UT, IT).
  • Design Structure Matrix analysis.
  • Flexible and highly customisable dashboard.
  • Actions plans / peer reviews.
  • Historic views / run charts.
  • Can be used with Java, .Net, C/C++, Groovy, PHP,…

 

Assuming you already have an OpenShift account (see my previous post here for details: http://deors.wordpress.com/2011/12/13/red-hat-openshift/), the steps to install Sonar in OpenShift as a DIY (do-it-yourself) application are the following:

  • Create a DIY application
  • Add MySQL cartridge
  • Unzip Sonar distribution
  • Update action hooks
  • Upload the changes
  • Configure some existing build job in Jenkins

1. Create a DIY application

To create a DIY application simply execute a command like this in your console (the same can be achieved using OpenShift web console):

>rhc app create -a sonar -t diy-0.1

After a few seconds, you should get some important information about the new application, so take note of them. In my case they were something like this (note that this corresponds to an OpenShift origin installation):

UUID: 4cd9eacb71b34401b81ea712352ffccb
Git URL: ssh://4cd9eacb71b34401b81ea712352ffccb@sonar-demo.example.com/~/git/sonar.git/
Public URL: http://sonar-demo.example.com/

2. Add MySQL cartridge

Next step is to add a MySQL cartridge to the application we’ve just created. Sonar will use this database to store the information collected during analysis, so take note of database URL, user and password. Let’s create the database using the following command:

>rhc app cartridge add -a sonar -c mysql-5.1

In my case, the database information was something like this:

Root User: admin
Root Password: VyikSHzlLvjx
Database Name: sonar
Connection URL: mysql://127.0.251.129:3306/

3. Unzip Sonar distribution

For next step, download a blank Sonar installation from Sonar’s web site (http://www.sonarsource.org/). In my case I was working with Sonar 3.1.1. You can start from an existing Sonar installation if you like.

Next, unzip Sonar Zip file so all files and folders are placed inside a sonar-x.x.x folder, like this:

Once Sonar is unzipped, edit the configuration file conf/sonar.properties so it points to the MySQL database just created:

...
sonar.web.host:      127.0.251.129
sonar.web.port:      8080
sonar.web.context:   /
sonar.jdbc.username: admin
sonar.jdbc.password: VyikSHzlLvjx
sonar.jdbc.driver:   com.mysql.jdbc.Driver
sonar.jdbc.url:      jdbc:mysql://127.0.251.129:3306/sonar?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true
...

4. Update action hooks

Next step is to update action hooks so OpenShift broker knows how to start and stop the Sonar process. We will leverage the scripts that came with Sonar, which means we will be using Jetty as the container where Sonar will be executed.

First, update start action hook that can be found in folder .openshift/action_hooks:

#!/bin/bash
cd $OPENSHIFT_REPO_DIR/sonar-3.1.1/bin/linux-x86-64
chmod 755 wrapper
sh ./sonar.sh start

Next, update stop action hook in the same folder:

#!/bin/bash
cd $OPENSHIFT_REPO_DIR/sonar-3.1.1/bin/linux-x86-64
sh ./sonar.sh stop

5. Upload the changes

Now that we are ready to go, commit and push the changes back to OpenShift. After a few minutes (first time Sonar will initialize the database schema and master data) you should be able to access Sonar dashboard.

6. Configure some existing build job in Jenkins

To verify that everything is working fine and, more important, to do something useful with Sonar, you may want to add Sonar analysis to any of your existing jobs in Jenkins. For this to work, you will need  to pass the right configuration properties referring to Sonar web dashboard and database. In my case it was something like this:

# leave all initial content of Jenkins build script up to when
# Maven is executed to package the app for OpenShift

mvn --global-settings $OPENSHIFT_MAVEN_MIRROR clean test
export SONAR_JDBC="-Dsonar.jdbc.url=jdbc:mysql://127.0.251.129:3306/sonar?useUnicode=true&characterEncoding=utf8"
export SONAR_DRIVER=-Dsonar.jdbc.driver=com.mysql.jdbc.Driver
export SONAR_DBUSER=-Dsonar.jdbc.username=admin
export SONAR_DBPASS=-Dsonar.jdbc.password=VyikSHzlLvjx
export SONAR_URL=-Dsonar.host.url=http://sonar-demo.example.com
export SONAR_OPTS="$SONAR_JDBC $SONAR_DRIVER $SONAR_DBUSER $SONAR_DBPASS $SONAR_URL"
mvn --global-settings $OPENSHIFT_MAVEN_MIRROR sonar:sonar $SONAR_OPTS

# rest of original Jenkins build script follows

Launch your job manually or commit some code and… magic! Sonar is now analysing your application and giving you valuable insights!

And this is just the beginning: you can install additional Sonar plug-ins and make more elaborate build cycles:

You may for example, use the Build Breaker plug-in. Build Breaker let’s you flag a build as a failure if some quality thresholds are not met. For example, if you analyse the application using a quality profile that sets a minimum code coverage of 60%, or a maximum average complexity of 4.0, the Sonar build will break (and the code will not be deployed into OpenShift) if some of these thresholds are not met.

You may as well launch integration tests (i.e. Selenium, Arquillian) and get code coverage for integration tests as well.

And why not add performance metrics? Using JMeter and the Sonar JMeter plug-in you can run some load tests in your app and add results to Sonar dashboard!

Agile with style!

About these ads

14 thoughts on “Installing Sonar in OpenShift as a DIY application”

  1. I followed your post, but error occured.
    Service Temporarily Unavailable

    The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.

      1. When ?
        the error pops up after I start the server.
        Logs ?
        Actually, I didn’t find any logs in server.
        would you mind tell me where are log files stored ?

      2. Hi, logs should be stored inside Sonar directory, so you can ssh and cat/tail the logs from the console.

        As a suggestion, you may want to tweak Sonar’s log configuration so you can use rhc app tail command.
        First, update the wrapper configuration (file conf/wrapper.conf) this way:

        -> beginning of file
        # Java Additional Parameters
        wrapper.java.additional.1=-Djava.awt.headless=true
        wrapper.java.additional.2=-XX:MaxPermSize=128m
        wrapper.java.additional.3=-DOPENSHIFT_LOG_DIR=%OPENSHIFT_LOG_DIR%

        # Log file to use for wrapper output logging.
        wrapper.logfile=%OPENSHIFT_LOG_DIR%/sonar.log

        -> end of file

        Second, update logback configuration (file conf/logback.xml) replacing all occurences of “${SONAR_HOME}/logs” by “${OPENSHIFT_LOG_DIR}” (of course remove the quotes).

        Once done, restart Sonar. Since now logs are stored in OPENSHIFT_LOG_DIR folder, you can run rhc app tail -a sonar to tail the logs from your PC without the need to ssh+cd+tail:

        c:\java>rhc app tail -a sonar -p xxxxxxxxxx
        Attempting to tail files: sonar/logs/*
        Use ctl + c to stop
        ==> sonar/logs/sonar.log <==
        2012.11.23 17:08:24 INFO org.sonar.INFO Register quality models done: 1 ms
        2012.11.23 17:08:24 INFO org.sonar.INFO Register filters…
        2012.11.23 17:08:24 INFO org.sonar.INFO Register filters done: 94 ms
        2012.11.23 17:08:24 INFO org.sonar.INFO Register dashboards…
        2012.11.23 17:08:24 INFO org.sonar.INFO Register dashboards done: 188 ms
        2012.11.23 17:08:24 INFO org.sonar.INFO Start components done: 34087 ms
        INFO | jvm 1 | 2012/11/23 17:08:45 | JRuby limited openssl loaded. http://jruby.org/openssl
        INFO | jvm 1 | 2012/11/23 17:08:45 | gem install jruby-openssl for full support.
        2012.11.23 17:09:05 INFO org.sonar.INFO Loading webservice /api/plugins/MotionchartWebService
        INFO | jvm 1 | 2012/11/23 17:09:06 | 2012-11-23 17:09:06.163:INFO::Started SelectChannelConnector@127.0.251.129:8080
        ==> sonar/logs/profiling.log <==

  2. I got the log as below, seems Sonar used some ports which were blocked by OpenShift.

    STATUS | wrapper | 2012/11/26 04:37:37 | –> Wrapper Started as Daemon
    FATAL | wrapperp | 2012/11/26 04:37:37 | unable to bind listener to any port in the range 32000-32999. (Permission denied)
    STATUS | wrapper | 2012/11/26 04:37:37 | <– Wrapper Stopped
    sonar.log (END)

    1. Mmmm, this sounds to me like the port in sonar.properties is not well configured. You should have something like this:


      sonar.web.host: -IP-
      sonar.web.port: 8080
      sonar.web.context: /
      sonar.jdbc.username: admin
      sonar.jdbc.password: -PASSWORD-
      sonar.jdbc.driver: com.mysql.jdbc.Driver
      sonar.jdbc.url: jdbc:mysql://-IP-:3306/sonar?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true

      1. Hi, not sure why this is happening to you and not to me (need to investigate a bit further) but the issue seems to be related to the Java Service Wrapper process default port assignments. See here: http://wrapper.tanukisoftware.com/doc/english/prop-port.html

        According to this, you can change the range of ports in wrapper.conf file using the properties that appear in the link above.

        Hope this helps.

  3. I have the same problem – “unable to bind listener to any port in the range 32000-32999. (Permission denied)”. As far as I can tell the Java service wrapper tries to open the communication port on localhost. This not now allowed anymore. All ports must be opened against $OPENSHIFT_INTERNAL_IP. I was not able to find a configuration option to specify the host. There are wrapper.port, wrapper.port.min and wrapper.port.max, but none of them help me in this case. I am wondering when you created your app? I think this change might have been quite recent and when you created your app you still could connect to localhost. That’s really the only explanation I have atm.
    On a side note, you should probably change your action scripts to somehow filter the right IP addresses into the configuration. Using hard coded IP addresses won’t work all the time. See also https://openshift.redhat.com/community/page/openshift-environment-variables

    1. Yes this can be the case. Since recently I’ve been working with OpenShift Origin and the issue seems not to be present there.

      I’m working on an alternate start action hook that, instead of making use of the wrapper script it just launches the embedded container, that is, reproduce what Sonar’s startup script does but without the wrapper layer.
      If you want to push this on your own, just follow these lines from the wrapper.conf file to build the start script:

      # set the classpath with these:
      wrapper.java.classpath.1=../../lib/*.jar
      wrapper.java.classpath.2=../../
      wrapper.java.classpath.3=../../extensions/jdbc-driver/derby/*.jar
      wrapper.java.classpath.4=../../extensions/jdbc-driver/mysql/*.jar
      wrapper.java.classpath.5=../../extensions/jdbc-driver/oracle/*.jar
      wrapper.java.classpath.6=../../extensions/jdbc-driver/postgresql/*.jar
      wrapper.java.classpath.7=../../extensions/jdbc-driver/mssql/*.jar

      # launch this class
      java -cp $THE_CLASSPATH org.sonar.application.StartServer -DOPENSHIFT_LOG_DIR=$OPENSHIFT_LOG_DIR

  4. I’m facing a connection timeout error on Jenkins build due to the value on sonar.host.url. The Jenkins cartridge runs on another application on openshift, but sonar DIY application is on the same domain as the application with Jenkins cartridge.
    – Have you a real example for sonar.host.url ?
    -The changes mentioned about start/stop scripts must have only the specified content or the content shown here must be appended to the existing one?
    -When I pushed some updates, got the following log about application restart:
    Total 5 (delta 0), reused 2 (delta 0)
    remote: Stopping MySQL 5.1 cartridge
    remote: Stopping DIY cartridge
    remote: CLIENT_RESULT: Application is already stopped
    remote: Starting sonar…
    remote: Failed to start sonar.
    remote: Building git ref ‘master’, commit b7321b2
    remote: Preparing build for deployment
    remote: Deployment id is 7d458afb
    remote: Activating deployment
    remote: Starting MySQL 5.1 cartridge
    remote: Starting DIY cartridge
    remote: Starting sonar…
    remote: Failed to start sonar.
    remote: ————————-
    remote: Git Post-Receive Result: success
    remote: Activation status: success
    remote: Deployment completed with status: success

    1. While running on OpenShift Origin, the sonar.host.url I have is the same that appears in the post: http://sonar-demo.example.com
      Re the start/stop scripts for Sonar, the specified content is the full content of the scripts. In which version of OpenShift are you deployed the application?

      1. Hello deors, thanks for the answer.
        I solved the previous question when I found this project on github (https://github.com/wenhao/openshift-sonar) which creates a sonar over a jbossews server. Following the instructions and doing some other tricks, my sonar console was up. Then I run the jenkins job again. Following the console results, sonar is running fine, but suddenly it stops on the following error (with build failure):

        [INFO] [22:07:52.483] Java bytecode scan done: 6752 ms
        [INFO] [22:07:52.842] Package design analysis…
        [INFO] [22:07:55.296] Package design analysis done: 2454 ms
        [INFO] [22:07:57.482] Sensor JavaSquidSensor done: 22263 ms
        [INFO] [22:07:57.483] Sensor FindbugsSensor…
        [INFO] [22:07:57.483] Execute Findbugs 2.0.1…
        [INFO] [22:08:45.321] Findbugs output report: /var/lib/openshift/5323050c5973cab03c0005e1/app-root/data/jobs/GeFiMA DEV free-style/workspace/target/sonar/findbugs-result.xml

        /tmp/hudson1746132169675279201.sh: line 25: 57044 Killed mvn –global-settings $settingsPath sonar:sonar $SONAR_OPTS
        Build step ‘Execute shell’ marked build as failure
        Finished: FAILURE

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s