Tuesday, August 28, 2012

How to manage Quartz remotely

Option 1: JMX

Many people asked can they manage Quartz via JMX, and that the documentation on this is not clear enough to help them get started. So, let me highlight couple ways you can do this.

Yes you can enable JMX in quartz with the following in quartz.properties

org.quartz.scheduler.jmx.export = true

After this, you use standard JMX client such as $JAVA_HOME/bin/jconsole to connect and manage remotely.

Option 2: RMI

Another way to manage quartz remotely is to enable RMI in Quartz. If you use this, you basically run one instance of Quartz as RMI server, and then you can create second Quartz instance as RMI client. These two can talk remotely via a TCP port.

For server scheduler instance, you want to add these in quartz.properties

org.quartz.scheduler.rmi.export = true
org.quartz.scheduler.rmi.createRegistry = true
org.quartz.scheduler.rmi.registryHost = localhost
org.quartz.scheduler.rmi.registryPort = 1099
org.quartz.scheduler.rmi.serverPort = 1100

And for client scheduler instance, you want to add these in quartz.properties

org.quartz.scheduler.rmi.proxy = true
org.quartz.scheduler.rmi.registryHost = localhost
org.quartz.scheduler.rmi.registryPort = 1099

The RMI feature is mentioned in Quartz doc here. Quartz doesn't have a client API, but use the same org.quartz.Scheduler for both server and client. It's just the configuration are different. By different configuration, you get very different behavior. For server, your scheduler is running all the jobs, while for client, it's simply a proxy. Your client scheduler instance will not run any jobs! You must be really careful when shutting down client because it does allow you to bring down the server!

These configurations have been highlighted in the MySchedule project. If you run the webapp, you should see a screen like this demo, and you will see it provided many sample of quartz configurations with these remote managment config properties.

If configure with RMI option, you can actually still use MySchedule web UI to manage the Quartz as proxy. You can view and drill down jobs, and you can even stop or shutdown remote server!

Based on my experience, there is a down side of using Quartz RMI feature though. That is it creates a single point of failure. There is no fail over if your RMI server port is down!

Monday, August 27, 2012

Taming those Java editors on MacOSX

Today, I learned that you can open multiple Eclipse instances in MacOSX like this

$ open -n -a Eclipse

This comes pretty handy if you want a separate workspace per large project that has dozen or so modules (try importing the Camel or SpringFramework project source and you will see what I mean!)

The -a is for name of an application to open, and -n means to open with new instance.

Finding a programmer friendly text editor on MacOSX

Despite I use Eclipse for most of my large Java projects, I still often find myself the need of a plain text editor. I use plain text editor for most of my scripting programming. I also use it for notes taking, text file browsing, or even writing Blog post in markdown format!

I guess for most programmers, choosing an Editor is a very personally thing. So I am not here to convert anybody. However, for those who never heard of jEdit, you ought to give it a try. It's open source, full features, programmers friendly text editor that works across many of OS systems consistenly. It works because it's Java based! If you use one of decent computer now a day, the performance is excellent as well.

However, for some strange reason, the jedit installer on MacOSX doesn't comes with a normal java command line script wrapper. The only thing I can see is the actual binary under /Applications/jEdit.app/Contents/MacOS/jedit. We can certainly run this as full jedit command, but then it always liter the console with strange GUI warnings. So a better way to run it is actually to use the open command.

$ open -a jedit $HOME/.bash_profile

Or you can add this in your .bash_profile file:

function jedit(){ open -a jedit "$@" ; }

Now you can use jedit command on the terminal.

Some of my favorite features of jEdit

  • It's open source and has very active community of users.
  • It works across MacOSX, Windows and Linux Desktop.
  • Support many common programming language syntax highlights.
  • Support multi columns editing.
  • Support rich feature search (reg ex, hyper-search, all buffers search, search and replace etc.)
  • Support multi buffers tabs editing.
  • Supports Macro recording and scripting.
  • Support plugin extension and there are many external plugins available (eg: XML formater, Diff viewer, SFTP client/editing, etc).

Openning other stuff on MacOSX

The little open command can open not just application, but it can also open directories or files by extension associations. For example, to configure any .properties files to be open by jEdit, try the the following.

  1. Use Finder to browse and find one of properties
  2. Right click, "Open With" > "Other..."
  3. Choose "jEdit" under Applications folder.
  4. Enable "Always Open With", then click Open button.

This change should affect your Terminal open command as well, and if you run open my.properties, your jEdit should pop up.

You can find more options by running it without any options, and it will print you full help page.

Usage: open [-e] [-t] [-f] [-W] [-R] [-n] [-g] [-h] [-b <bundle identifier>] [-a <application>] [filenames] [--args arguments]
Help: Open opens files from a shell.
      By default, opens each file using the default application for that file.  
      If the file is in the form of a URL, the file will be opened as a URL.
Options: 
      -a                Opens with the specified application.
      -b                Opens with the specified application bundle identifier.
      -e                Opens with TextEdit.
      -t                Opens with default text editor.
      -f                Reads input from standard input and opens with TextEdit.
      -F  --fresh       Launches the app fresh, that is, without restoring windows. Saved persistent state is lost, excluding Untitled documents.
      -R, --reveal      Selects in the Finder instead of opening.
      -W, --wait-apps   Blocks until the used applications are closed (even if they were already running).
          --args        All remaining arguments are passed in argv to the application's main() function instead of opened.
      -n, --new         Open a new instance of the application even if one is already running.
      -j, --hide        Launches the app hidden.
      -g, --background  Does not bring the application to the foreground.
      -h, --header      Searches header file locations for headers matching the given filenames, and opens them.

Tuesday, August 21, 2012

myschedule-2.4.4 is out!

For those of you who are using Quartz, you might find MySchedule useful. I've made another minor release to upgrade it to latest Quartz 2.1.6 and few minor fixes. Enjoy!

Sunday, August 19, 2012

Getting started with Apache Camel using Groovy

From their site, it says the Apache Camel is a versatile open-source integration framework based on known Enterprise Integration Patterns. It might seem like a vague definition, but I want to tell you that this is a very productive Java library that can solve many of typical IT problems! You can think of it as a very light weight ESB framework with "batteries" included.

In every jobs I've been to so far, folks are writing their own solutions in one way or another to solve many common problems (or they would buy some very expensive enterprisy ESB servers that takes months and months to learn, config, and maintain). Things that we commonly solve are integration (glue) code of existing business services together, process data in a certain workflow manner, or move and transform data from one place to another etc. These are very typical need in many IT environments. The Apache Camel can be used in cases like these; not only that, but also in a very productive and effective way!

In this article, I will show you how to get started with Apache Camel along with just few lines of Groovy script. You can certainly also start off with a full Java project to try out Camel, but I find Groovy will give you the shortest working example and learning curve.

Getting started with Apache Camel using Groovy

So let's begin. First let's see a hello world demo with Camel + Groovy.

@Grab('org.apache.camel:camel-core:2.10.0')
@Grab('org.slf4j:slf4j-simple:1.6.6')
import org.apache.camel.*
import org.apache.camel.impl.*
import org.apache.camel.builder.*

def camelContext = new DefaultCamelContext()
camelContext.addRoutes(new RouteBuilder() {
    def void configure() {
        from("timer://jdkTimer?period=3000")
            .to("log://camelLogger?level=INFO")
    }
})
camelContext.start()

addShutdownHook{ camelContext.stop() }
synchronized(this){ this.wait() }

Save above into a file named helloCamel.groovy and then run it like this:

$ groovy helloCamel.groovy
388 [main] INFO org.apache.camel.impl.DefaultCamelContext - Apache Camel 2.10.0 (CamelContext: camel-1) is starting
445 [main] INFO org.apache.camel.management.ManagementStrategyFactory - JMX enabled.
447 [main] INFO org.apache.camel.management.DefaultManagementLifecycleStrategy - StatisticsLevel at All so enabling load performance statistics
678 [main] INFO org.apache.camel.impl.converter.DefaultTypeConverter - Loaded 170 type converters
882 [main] INFO org.apache.camel.impl.DefaultCamelContext - Route: route1 started and consuming from: Endpoint[timer://jdkTimer?period=3000]
883 [main] INFO org.apache.camel.impl.DefaultCamelContext - Total 1 routes, of which 1 is started.
887 [main] INFO org.apache.camel.impl.DefaultCamelContext - Apache Camel 2.10.0 (CamelContext: camel-1) started in 0.496 seconds
898 [Camel (camel-1) thread #1 - timer://jdkTimer] INFO camelLogger - Exchange[ExchangePattern:InOnly, BodyType:null, Body:[Body is null]]
3884 [Camel (camel-1) thread #1 - timer://jdkTimer] INFO camelLogger - Exchange[ExchangePattern:InOnly, BodyType:null, Body:[Body is null]]
6884 [Camel (camel-1) thread #1 - timer://jdkTimer] INFO camelLogger - Exchange[ExchangePattern:InOnly, BodyType:null, Body:[Body is null]]
...

The little script above is simple but it presented few key features of Camel Groovyness. The first and last section of the helloCamel.groovy script are just Groovy featuers. The @Grab annotation will automatically download the dependency jars you specify. We import Java packages to use its classes later. At the end we ensure to shutdown Camel before exiting JVM through the Java Shutdown Hook mechanism. The program will sit and wait until user press CTRL+C, just as a typical server process behavior.

The middle section is where the Camel action is. You would always create a Camel context to begin (think of it as the server or manager for the process.) And then you would add a Camel route (think of it as a workflow or pipeflow) that you like to process data (Camel likes to call these data "messages"). The route consists of a "from" starting point (where data generated), and one or more "to" points (where data going to be processed). Camel calls these destination 'points' as 'Endpoints'. These endpoints can be expressed in simple URI string format such as "timer://jdkTimer?period=3000". Here we are generating timer message in every 3 secs into the pipeflow, and then process by a logger URI, which will simply print to console output.

After Camel context started, it will start processing data through the workflow, as you can observe from the output example above. Now try pressing CTRL+C to end its process. Notice how the Camel will shutdown everything very gracefully.

7312 [Thread-2] INFO org.apache.camel.impl.DefaultCamelContext - Apache Camel 2.10.0 (CamelContext: camel-1) is shutting down
7312 [Thread-2] INFO org.apache.camel.impl.DefaultShutdownStrategy - Starting to graceful shutdown 1 routes (timeout 300 seconds)
7317 [Camel (camel-1) thread #2 - ShutdownTask] INFO org.apache.camel.impl.DefaultShutdownStrategy - Route: route1 shutdown complete, was consuming from: Endpoint[timer://jdkTimer?period=3000]
7317 [Thread-2] INFO org.apache.camel.impl.DefaultShutdownStrategy - Graceful shutdown of 1 routes completed in 0 seconds
7321 [Thread-2] INFO org.apache.camel.impl.converter.DefaultTypeConverter - TypeConverterRegistry utilization[attempts=2, hits=2, misses=0, failures=0] mappings[total=170, misses=0]
7322 [Thread-2] INFO org.apache.camel.impl.DefaultCamelContext - Apache Camel 2.10.0 (CamelContext: camel-1) is shutdown in 0.010 seconds. Uptime 7.053 seconds.

So that's our first taste of Camel ride! However, we titled this section as "Hello World!" demo, and yet we haven't seen any. But you might have also noticed that above script are mostly boiler plate code that we setup. No user logic has been added yet. Not even the logging the message part! We simply configuring the route.

Now let's modify the script little bit so we will actually add our user logic to process the timer message.

@Grab('org.apache.camel:camel-core:2.10.0')
@Grab('org.slf4j:slf4j-simple:1.6.6')
import org.apache.camel.*
import org.apache.camel.impl.*
import org.apache.camel.builder.*

def camelContext = new DefaultCamelContext()
camelContext.addRoutes(new RouteBuilder() {
    def void configure() {
        from("timer://jdkTimer?period=3000")
            .to("log://camelLogger?level=INFO")
            .process(new Processor() {
                def void process(Exchange exchange) {
                    println("Hello World!")
                }
            })
    }
})
camelContext.start()

addShutdownHook{ camelContext.stop() }
synchronized(this){ this.wait() }

Notice how I can simply append the process code part right after the to("log...") line. I have added a "processor" code block to process the timer message. The logic is simple: we greet the world on each tick.

Making Camel route more concise and practical

Now, do I have you at Hello yet? If not, then I hope you will be patient and continue to follow along for few more practical features of Camel. First, if you were to put Camel in real use, I would recommend you setup your business logic separately from the workflow route definition. This is so that you can clearly express and see your entire pipeflow of route at a glance. To do this, you want to move the "processor", into a service bean.

@Grab('org.apache.camel:camel-core:2.10.0')
@Grab('org.slf4j:slf4j-simple:1.6.6')
import org.apache.camel.*
import org.apache.camel.impl.*
import org.apache.camel.builder.*
import org.apache.camel.util.jndi.*

class SystemInfoService {
    def void run() {
        println("Hello World!")
    }
}
def jndiContext = new JndiContext();
jndiContext.bind("systemInfoPoller", new SystemInfoService())

def camelContext = new DefaultCamelContext(jndiContext)
camelContext.addRoutes(new RouteBuilder() {
    def void configure() {
        from("timer://jdkTimer?period=3000")
            .to("log://camelLogger?level=INFO")
            .to("bean://systemInfoPoller?method=run")
    }
})
camelContext.start()

addShutdownHook{ camelContext.stop() }
synchronized(this){ this.wait() }

Now, see how compact this workflow route has become? The Camel's Java DSL such as "from().to().to()" for defining route are so clean and simple to use. You can even show this code snip to your Business Analysts, and they would likely be able to verify your business flow easily! Wouldn't that alone worth a million dollars?

How about another demo: FilePoller Processing

File polling processing is a very common and effective way to solve many business problems. If you work for commercial companies long enough, you might have written one before. A typical file poller would process incoming files from a directory and then process the content, and then move the file into a output directory. Let's make a Camel route to do just that.

@Grab('org.apache.camel:camel-core:2.10.0')
@Grab('org.slf4j:slf4j-simple:1.6.6')
import org.apache.camel.*
import org.apache.camel.impl.*
import org.apache.camel.builder.*
import org.apache.camel.util.jndi.*

class UpperCaseTextService {
    def String transform(String text) {
        return text.toUpperCase()
    }
}
def jndiContext = new JndiContext();
jndiContext.bind("upperCaseTextService", new UpperCaseTextService())

def dataDir = "/${System.properties['user.home']}/test/file-poller-demo"
def camelContext = new DefaultCamelContext(jndiContext)
camelContext.addRoutes(new RouteBuilder() {
    def void configure() {
        from("file://${dataDir}/in")
            .to("log://camelLogger")
            .to("bean://upperCaseTextService?method=transform")
            .to("file://${dataDir}/out")
    }
})
camelContext.start()

addShutdownHook{ camelContext.stop() }
synchronized(this){ this.wait() }

Here you see I defined a route to poll a $HOME/test/file-poller-demo/in directory for text files. Once it's found it will log it to console, and then process by a service that transform the content text into upper case. After this, it will send the file into $HOME/test/file-poller-demo/out directory. My goodness, reading the Camel route above probably express what I wrote down just as effective. Do you see the benefits here?

What's the "batteries" included part.

If you've used Python programming before, you might have heard the pharase that they claim often: Python has "batteries" included. This means their interpreter comes with a rich of libaries for most of the common programming need. You can often write python program without have to download separated external libraries.

I am making similar analogies here with Apache Camel. The Camel project comes with so many ready to use components that you can find just about any transport protocals that can carry data. These Camel "components" are ones that support different 'Endpoint URI' that we have seen in our demos above. We have simply shown you timer, log, bean, and file components, but there are over 120 more. You will find jms, http, ftp, cfx, or tcp just to name a few.

The Camel project also has an option for you to define route in declarative xml format. The xml is just an extension of a Spring xml config with Camel's namespace handler added on top. Spring is optional in Camel, but you can use it together in a very powerful way.

Saturday, August 18, 2012

How to integrate TimeMachine Scheduler in Spring based application

One of the goals for TimeMachine Scheduler is to make use of POJO as friendly as possible so that we can integrate into any IoC container easily. I've prepared a demo that will show you how to integrate TimeMachine with Spring based application.

We will mainly focus on the Spring xml config part in this article, so if you don't feel like checking out the demo project source code in details, then simply grab the binary package of the timemachine-spring-demo.zip and follow along.

Introducing a tiny Spring Server

If you don't already have a Spring project going, then take a look at my little Spring server. It can bootstrap any spring configuration xml file and then sits and wait for a CTRL+C to end. Unzip the demo and try it out like this.

$ cd timemachine-spring-demo
$ bin/run-spring config/hello-spring.xml

You should see a "Hello World!" message printed. Hit CTRL+C to end it. The xml just declares a simple HelloService bean like this.

<beans xmlns="http://www.springframework.org/schema/beans"
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:schemaLocation="http://www.springframework.org/schema/beans
     http://www.springframework.org/schema/beans/spring-beans-3.0.xsd">

    <bean id="helloService" class="timemachine.spring.HelloService" init-method="run">
        <property name="name" value="World"></property>
    </bean>

</beans>

That's your typical Spring declaration file. With this tiny Spring server, you can quickly experiment any POJOs wired up in any way you wish and put them in action.

Setting up TimeMachine Scheduler beans

In Java code, you can easily create an instance of the TimeMachine scheduler as follow:

SchedulerFactory schedulerFactory = new SchedulerFactory("config/scheduler.properties");
Scheduler scheduler = schedulerFactory.createScheduler();
scheduler.start();
// Then we would also need to call scheduler.destroy() before we exit the Java program. (eg: in shutdownHook)

You can easily translate that into Spring xml config like in config/timemachine-pojo-spring.xml:

<beans xmlns="http://www.springframework.org/schema/beans"
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:schemaLocation="http://www.springframework.org/schema/beans
     http://www.springframework.org/schema/beans/spring-beans-3.0.xsd">

    <bean id="schedulerFactory" class="timemachine.scheduler.SchedulerFactory">
        <constructor-arg value="config/scheduler.properties"></constructor-arg>
    </bean>
    <bean id="scheduler" class="timemachine.scheduler.Scheduler" 
        factory-bean="schedulerFactory" factory-method="createScheduler"
        init-method="start" destroy-method="destroy">
    </bean>

</beans>

The benefit of using Spring xml is that your scheduler lifecycles would be automatically taken care of without setting any shutdown hook. Running above should give you a plain scheduler started, and pressing CTRL+C should shutdown gracefully.

Going further with a custom SchedulerFactoryBean

Now above is not much excitement using Spring since you would still configure the scheduler through config/scheduler.properties. You certainly can load jobs and add custom services there. But if we are going to use Spring, we might as well take full advantage of it to create the Job Definition and Schedule inside the xml config! To do this, I created a simple timemachine.scheduler.spring.SchedulerFactoryBean in my demo project, and you can try it out like this:

<beans xmlns="http://www.springframework.org/schema/beans"
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:schemaLocation="http://www.springframework.org/schema/beans
     http://www.springframework.org/schema/beans/spring-beans-3.0.xsd">

    <bean id="scheduler" class="timemachine.scheduler.spring.SchedulerFactoryBean">
        <property name="configPropsUrl" value="config/scheduler.properties"></property>
        <property name="autoStart" value="true"></property>
        <property name="autoAddJobDef" value="true"></property>
    </bean>
    <bean id="jobDef01" class="timemachine.scheduler.JobDef">
        <property name="jobTaskClassName" value="timemachine.scheduler.jobtask.ScriptingJobTask"></property>
        <property name="props">
            <map>
                <entry key="scriptEngineName" value="Groovy"></entry>
                <entry key="scriptText" value="println('Hello World.')"></entry>
            </map>
        </property>
        <property name="schedules">
            <list>              
                <bean class="timemachine.scheduler.schedule.CronSchedule">
                    <property name="expression" value="0/3 * * * * ?"></property>
                </bean>
            </list>
        </property>
    </bean>

</beans>

Above will create not only the scheduler but also auto manage the lifecycles for you without you explicitly declare it. The factory bean will also auto detect any JobDef and Schedule bean types and add them into the scheduler instance. This would make your scheduler config all in the Spring xml, and it's well integrated.

There is also another similar sample in config/timemachine-spring.xml you may try. It will invoke an external groovy script instead of inline text.

Going extra mile on exposing TimeMachine Scheduler through JMX

The TimeMachine doesn't support any JMX instrumentation, however when running with the Spring, you can easily expose any Java interface to the local MBean Server Platform using Spring's MBeanExporter. For example in config/timemachine-jmx-spring.xml you will see:

<beans xmlns="http://www.springframework.org/schema/beans"
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:schemaLocation="http://www.springframework.org/schema/beans
     http://www.springframework.org/schema/beans/spring-beans-3.0.xsd">

    <bean id="scheduler" class="timemachine.scheduler.spring.SchedulerFactoryBean">
        <property name="configPropsUrl" value="config/scheduler.properties"/>
    </bean> 
    <bean id="exporter" class="org.springframework.jmx.export.MBeanExporter">
        <property name="assembler">
            <bean class="org.springframework.jmx.export.assembler.InterfaceBasedMBeanInfoAssembler">
                <property name="managedInterfaces">
                    <list>
                        <value>timemachine.scheduler.Scheduler</value>
                    </list>
                </property>
            </bean>
        </property>
        <property name="beans">
            <map>
                <entry key="timemachine.scheduler:name=Scheduler" value-ref="scheduler"/>
            </map>
        </property>
    </bean>

</beans>

Above will start an empty scheduler, and expose the timemachine.scheduler.Scheduler interface to the JMX local server. You may use $JAVA_HOME/bin/jconsole to connect and will see all the methods automatically exposed. There few methods that contains custom java objects that exporter will not able to convert properly, but at least you will get all the lifecycles and some simple getters that available to you.

Wednesday, August 15, 2012

How to make an executable WAR file

If you ever used Jenkin CI server, you have probably seen their super easy one-line instruction to get started: java -jar jenkins.war. Now that's one awesome way to get your users to try out your product!

In this article, I will show you how to make a WAR file self executable just like above. Before I do that, let me tell you a little web app that I made, and then I will show how I converted it into runnable war.

The Requirement: Taking Notes Efficiently

I tend take a lot of notes when I do my work. Instead of using the crapy Windows Notepad.exe, I would setup my cygwin .bash_profile like this:

function e() { /cygdrive/C/apps/notepad++.exe $(cygpath -w "$@") ; }
function n() { F="$HOME/notes/$(date "+%Y%m%d-%H%M%S").markdown"; touch $F; e $F; }

I happen to have Notepad++ installed, but you can use just about any text editor you like. With that setup, then anywhere in my terminal I can simply type n to have a GUI editor pop open, and I will have my note file ready to record notes. The file name would have a timestamp date, and I like to record them with simple Markdown syntax for easy viewing later. Since I write them in plain text, it's easy and fast to search using grep MYSTUFF ~/notes/*. Once I found it, I can quickly edit it by e ~/notes/20120814-000000.markdown.

This method of taking notes is fast and portable between my Linux and Windows systems (well, I have to choose a different editor between OS but not big deal). Now since I already recorded my notes in markdown, there is no way I can see the pretty result unless I use an online converter. So I thought it would be awesome if I can have a web app that let me record these notes, and yet give me a nice quick Markdown to html view.

Introducing the webnotepad.war

To improve my notes taking, I created little web app called webnotepad.war. It's really simply one page web app that record notes in Markdown syntax, preview it, and allow you to search it. It also let you edit old notes too. The whole thing is just a Servlet class, and it doesn't even use JSP.

However, after I have this little web app, it bothers me that I would always need an app server to run it. (I can run mvn tomcat7:run directly from my source project, but it would require me be online to satisfy initial Maven build.) So here is a perfectly example use case that would be super nice to make this war file self executable. In fact I have succeeded in this little experiement. You can download it and try it out yourself with java -jar webnotepad.war, and then browse http://localhost:8080. If you get port conflict, try --httpPort=8081.

Of course you continue to deploy webnotepad.war into any app server too.

Making war file executable

Before you want to make war executable, you want to build and package your normal web application first. The trick to make the war file runnable is that you want additionally add a Main-Class in the META-INF/MANIFEST.MF in the war file. And from this we need to load and run an embedded Servlet server with the self war file deployed. I noticed Jenkins uses one called Winstone container server. I was amazed to find that this server only has 300K in size! It support full Servlet 2.5 spec! It can optionally support JSP with Jasper which will cost you up to 3MB in size. That's still a very small price to pay compare to any Servlet container out there!

Making a Main-Class WinstoneMain.java is not hard. However making Maven to package everything is pretty tricky. Now my webnotepad.war doesn't need Jasper, but if you do, you specially want these jars outside of WEB-INF/lib because you don't want your normal app server to load these! Else you will get very odd errors.

In my example I added into WEB-INF/lib-winstone, and then the WinstoneMain would extract these and then load the winstone.Launcher using a custom class loader. Also, we need to use some Winstone options to start a web server properly. I use this webnotepad/pom.xml to package it all up the war file. You can easily use my example because I have create them under a separate profile and you just need add to your web module/pom.xml. In my project demo you would run mvn package -Pdist to generate the executable war file.

NOTE: I noticed Winstone project didn't show much activity since last release since 2008. But I was happy to find that there is a fork project on GoogleCode Winstone that shows activities. I didn't use this for my demo, but it looks promising.

Tuesday, August 7, 2012

How to generate Redmine WAR application to run with Java

If you survey around Java open source land, it's really disappointing in not able to find a decent project management web app that we can use. The Java war file deployment in an app server is easy to use, and it would give us good performance. Especially if we already in a Java shop, having a project management tool that deploys in an existing app server would be easier to maintain.

I have found the Ruby's Redmine to be a very good ONE-STOP-SHOP for most of your project development need. It has nice clean web interface, and the features are loaded: projects, users, issue tracking, wiki, source control viewer, documents, and it even has time tracking.

Altough the application is written in Ruby on Rails, you can run this with the JRuby perfectly fine on any Java app server. I've just done this and able to run it on JBossAS7.1.0 with jruby-1.6.7.2 and redmine-2.03.

For the impatients, you can actually grab my generated redmine-2.0.3.war from my sandbox. I even provided an MySQL sql dump for you to get started without have to go through the building the war file. If you want to change the DB or any other settings, you can simply unpack the war file and edit the config files, then re-jar the war backup. This is still easier than building the war from scratch.

Here is how I build the war file. First step is setup a database. I picked MySQL.

mysql> create database redmine character set utf8;
mysql> create user 'redmine'@'localhost' identified by 'redmine123';
mysql> grant all privileges on redmine.* to 'redmine'@'localhost';

Then prepare the Redmine with JRuby. Download the redmine-2.0.3.zip and unzip and then cd into this directory. Then run the following:

$ $JRUBY_HOME/bin/gem install bundler
$ $JRUBY_HOME/bin/bundle install --without development test rmagick postgresql sqlite
$ cp config/database.yml.example config/database.yml
$ # edit config/database.yml
$ # type in your database connection info for "production" env.
$ $JRUBY_HOME/bin/rake generate_secret_token
$ RAILS_ENV=production $JRUBY_HOME/bin/rake db:migrate
$ RAILS_ENV=production $JRUBY_HOME/bin/rake redmine:load_default_data

Now create the war file.

$ $JRUBY_HOME/bin/gem install warbler
$ $JRUBY_HOME/bin/warble config
$ # edit config/warble.rb
$ # replace: config.dirs = %w(app config lib log vendor tmp extra files)
$ # add: config.gems += ["activerecord-jdbcmysql-adapter", "jruby-openssl", "i18n", "rack"]
$ $JRUBY_HOME/bin/warble

After this, you will see redmine-2.0.3.war in the directory. Now you can drop it into any app server and you may kick start any projects!

Saturday, August 4, 2012

TimeMachine jars size and dependencies

From the TimeMachine Scheduler download page you will see that the latest 1.2.1 release zip package size is around 13M. It may seem large for a scheduler application, but let's examine it closely to see what's in there. Here is how the zip package looks like inside

timemachine-scheduler-1.2.1
+-- bin
+-- lib
+-- config

The config directory contains many sample configurations and scripts to help new users get started. The bin contains few startup wrapper scripts that let you start the scheduler. The lib contains the libraries for the scheduler application. I want to go into more details on these.

+-- lib
-rw-r--r--@  1 zemian  staff   445288 Jun 20 23:20 antlr-2.7.7.jar
-rw-r--r--@  1 zemian  staff   610790 Jun 20 23:26 c3p0-0.9.1.2.jar
-rw-r--r--@  1 zemian  staff   109043 Jun 20 23:20 commons-io-1.4.jar
-rw-r--r--@  1 zemian  staff   279193 Jun 20 23:20 commons-lang-2.5.jar
-rw-r--r--@  1 zemian  staff   313898 Jun 20 23:26 dom4j-1.6.1.jar
-rw-r--r--@  1 zemian  staff  6156866 Aug  2 22:05 groovy-all-2.0.0.jar
-rw-r--r--@  1 zemian  staff  1267149 Jun 20 23:26 h2-1.3.163.jar
-rw-r--r--@  1 zemian  staff    81271 Jun 20 23:26 hibernate-commons-annotations-4.0.1.Final.jar
-rw-r--r--@  1 zemian  staff  4407494 Jun 20 23:26 hibernate-core-4.1.3.Final.jar
-rw-r--r--@  1 zemian  staff   102661 Jun 20 23:26 hibernate-jpa-2.0-api-1.0.1.Final.jar
-rw-r--r--@  1 zemian  staff   648253 Jun 20 23:26 javassist-3.15.0-GA.jar
-rw-r--r--@  1 zemian  staff    60768 Jun 20 23:26 jboss-logging-3.1.0.GA.jar
-rw-r--r--@  1 zemian  staff    11209 Jun 20 23:26 jboss-transaction-api_1.1_spec-1.0.0.Final.jar
-rw-r--r--@  1 zemian  staff   481535 Jun 20 23:20 log4j-1.2.16.jar
-rw-r--r--@  1 zemian  staff    25496 Jun 20 23:20 slf4j-api-1.6.1.jar
-rw-r--r--@  1 zemian  staff     9753 Jun 20 23:20 slf4j-log4j12-1.6.1.jar
-rw-r--r--@  1 zemian  staff    46885 Aug  2 22:37 timemachine-hibernate-1.2.1.jar
-rw-r--r--@  1 zemian  staff   154925 Aug  2 22:36 timemachine-scheduler-1.2.1.jar

The core of the scheduler timemachine-scheduler-1.2.1.jar would only need slf4j-api-1.6.1.jar, commons-io-1.4.jar and commons-lang-2.5.jar to run. These would support full scheduler with in memory data store. As you can see these add up to only about 500K!

To be able to see useful logging output, you need an SLF4j impl. You may use built-in JDK's logger if you choose, or you may use slf4j-log4j12-1.6.1.jar and log4j-1.2.16.jar. I personally think the JDK logger is horrible to use and configure, so I package up log4j as option for you.

A Java scripting engine is only optional in TimeMachine, and JDK already comes with JavaScript. To be practical, I also package the Groovy engine groovy-all-2.0.0.jar. It's much more easy to use. But the full groovy engine is costing us 6M in size!

The TimeMachine allows you to switch data store service. We provided the HibernateDataService service in timemachine-scheduler-1.2.1.jar. This service would require the Hibernate dependencies, which is pretty heavy. They are: hibernate-*.jar, jboss-*.jar, javassist-3.15.0-GA.jar, antlr-2.7.7.jar, and dom4j-1.6.1.jar. These can add up to another 5M in size!

Now with the Hibernate data service, you would also need a database driver that match to your database. I have included a H2Database, a Java database that provides full SQL database service including its driver in h2-1.3.163.jar. This alone is about 1.2M in size.

When you working with Hibernate and Database, you should always use a Database connection pool. I packaged the c3p0-0.9.1.2.jar implementation which Hibernate can automatically detects.

I hope these information can help you decide what to remove or add in case you want to customize your scheduler to fit into your application. You may also visit the project to see the Maven site dependencies report that has all the details. This also means that if you use Maven to bring in TimeMachine scheduler as dependency, you will automatically gets the minimal because I have carefully set all the optional items not to be included with the proper Maven scopes.

Thursday, August 2, 2012

A better java shell script wrapper

In many Java projects, you often see wrapper shell script to invoke the java command with its custom application parameters. For example, $ANT_HOME/bin/ant, $GROOVY_HOME/bin/groovy, or even in our TimeMachine Scheduler you will see $TIMEMACHINE_HOME/bin/scheduler.sh.

Writing these wrapper script is boring and error prone. Most of the problems come from setting the correct classpath for the application. If you're working on an in-house project for a company, then you can get away with hardcoding paths and your environment vars. But for open source projects, folks have to make the wrapper more flexible and generic. Most of them even provide a .bat version of it. Windows DOS is really a brutal and limited terminal to script away your project need. For this reason, I often encourage others to use Cygwin as much as they can. It at least has a real bash shell to work with. Another common problem with these wrappers is it can quickly get out of hand and have too many duplication of similar scripts liter every where in your project.

In this post, I will show you a Java wrapper script that I've written. It's simple to use and very flexible for running just about any Java program. Let's see how it's used first, and then I will print its content at the bottom of the post.

Introducing the run-java wrapper script

If you take a look at $TIMEMACHINE_HOME/bin/scheduler.sh, you will see that it in turns calls a run-java script that comes in the same directory.

DIR=$(dirname $0)
SCHEDULER_HOME=$DIR/..
$DIR/run-java -Dscheduler.home="$SCHEDULER_HOME" timemachine.scheduler.tool.SchedulerServer "$@"

As you can see, our run-java can take -D options. Not only this, it can also take -cp option as well! What's more is that you can specify these options even after the main class! This makes the run-java re-wrappable by other script, and still be able to add additional system properties and classpath.

For examples, the TimeMachine comes with Groovy library, so instead of downloading it's full distribution again, you can simply invoke the groovy like this

$TIMEMACHINE_HOME/bin/run-java groovy.ui.GroovyMain test.groovy

You can use run-java in any directory you're in, so it's convenient. It will resolve it's own directory and load any jars in the lib directory automatically. Now if you want Groovy to run with more additional jars, you can use the -cp option like this:

$TIMEMACHINE_HOME/bin/run-java -cp "$HOME/apps/my-app/lib/*" groovy.ui.GroovyMain test.groovy

Often times things will go wrong if you are not careful with Java classpath, but with run-java script you can perform a dry run first:

RUN_JAVA_DRY=1 $TIMEMACHINE_HOME/bin/run-java -cp "$HOME/apps/my-app/lib/*" groovy.ui.GroovyMain test.groovy

You would run the above all in single line on a command prompt. It should print out your full java command with all options and arguments for you to inspect.

There are many more options to the script, which you can find out more by reading the comments in it. The current script will work on any Linux bash or on a Windows Cygwin terminal.

Using run-java during development with Maven

Above examples are assuming you are in a released project structure such as this:

$TIMEMACHINE_HOME
  +- bin/run-java
  +- lib/*.jar

But what about during development? A frequent use case is that you want to be able to run your latest compiled classes under target/classes without have to package up or release the entire project. You can use our run-java in these scenario as well. First, simply add bin/run-java in your project, then you run mvn compile dependency:copy-dependencies that will generate all the jar files into target/dependency. That's all. The run-java will automatically detect these directories and create the correct classpath to run your main class.

If you use Eclipse IDE for development, then your target/classes will be always up-to-date, and the run-java can be a great gem to have in your project even for development.

Get the run-java wrapper script now