Up and Running with Obsidian Scheduler in 5 Minutes

One of the best things about Obsidian Scheduler is how quick and easy it is to get it running jobs. With other tools, you might have to set aside an afternoon to get it going, but with Obsidian, trust me, it won’t take long at all.

To show you how easy it really is, I’m going to walk you through an example of a setting up a simple job that will do a basic health check to make sure our website is working.
.

Step 1 – Download and Run the Installer

Elapsed Time: 0 minutes

  • First, just grab the latest Obsidian installation package from our download page.
  • Extract the zip file, and double-click on the Obsidian-Install-x.x.x.jar file. (We assume you have Java installed.)
  • Go through the installer, make sure the Winstone installation option is checked, and then fill in your basic database connection info. You should point it to a database that already exists. The screenshots below show the Winstone option and a sample MySQL database connection.

winstone

dbinfo

Step 2 – Start Obsidian

Elapsed Time: 2 minutes

  • Navigate to the installation directory you selected in Step 1, and run the following on the command line, and Obsidian will be merrily on its way.
webObsidian.sh scheduler start
.. or for our Windows friends
webObsidian.bat scheduler start

Step 3 – Log In

Elapsed Time: 3 minutes

  • Navigate in your browser to http://localhost:8080 and enter admin and changeme for the login credentials.
  • Congrats! You have a functioning Obsidian instance that you can execute jobs in!

log in

Step 4 – Configure our Health Check Job

Elapsed Time: 3.5 minutes

  • Click on the Jobs tab, then on Add Job right below that.
  • Configure your health check job by filling in the following fields:
    • Nickname: Website Health Check
    • Job Class: com.carfey.ops.job.script.GroovyJob
    • Defined Parameters > script:
      jobContext.saveJobResult('url', url)
      new URL(url).getText()
    • Custom Parameters > Click Add Custom Parameter, then enter:
      • Name: url
      • Value: http://obsidianscheduler.com (or some other URL)
    • Initial Schedule > Schedule: * * * * * (this will run every minute, and you can omit the effective and end times)
  • Click Save. After you save it will look something like the screenshot below.
  • The job will start running every minute! Easy huh?

add job

job setup

Step 5 -You’re Done! Monitor Your Job

Elapsed Time: 5 minutes

  • Click on the Job History tab, and refresh now and then to see your jobs running!

All that’s left to do is watch your jobs execute! Below are what a success case and a failure might look like.

success

failure

Of course, we could do a lot better than this as far as messaging goes, but we’ll leave that to you. We could also add some conditional notifications to get alerted of any job failures (which would require setting up SMTP).

So there it is: we do everything we can to make your lives easier, but some things like writing the jobs you’ll unfortunately have to do yourself!

Happy scheduling!

Developing a Reusable File: Cleanup Schedulable Job

In my last post, I delivered on a reusable, file archival schedulable job. I promised to next look at doing something similar for file cleanup.

File cleanup needs are driven by similar, if not identical, factors as those for file archival. Reiterating those, they are – limits of physical space, business or compliance requirements for retention and information availability for support teams. Again, it makes sense to allow for flexible configuration of our job for differences and even changes in those requirements so that we won’t have to go back to the drawing board each time.

Configuration Parameters

Some of those configuration parameters are:

  • Directory
  • Recursive Y/N?
  • Filename pattern
  • Minimum/Maximum Size
  • Minimum time since modification

Again, this will be an Obsidian schedulable job adaptable to any scheduling platform you desire. Here is the resulting Java source code available under the MIT open source licence for you do to as you wish. FileCleanupJob.java.

Job Code

The basic algorithm is as follows:

  • Iterate over files in the directory
  • Determine if the filename mask applies
  • Determine if the file matches any other criteria specified
  • Delete the file.

Here’s what our primary cleanup method ends up looking like:

protected void processDirectory(final Context context, String dir) throws Exception {
  boolean recursive = Boolean.TRUE.equals(context.getConfig().getBoolean(RECURSIVE));
  List files = new ArrayList();
  Directory d = new Directory(dir);
  List fileList = recursive ? d.listFilesRecursively() : d.listFiles();
  
  for (com.carfey.jdk.io.File file : fileList) {
    File ff = new File(file.getAbsolutePath());
    if (shouldDelete(context, ff)) {
      files.add(ff);
    }
  }
  for (File f : files) {
    deleteMatchingFile(context, f);
    checkInterrupted();
  }
}

You’ll notice the job even supports multiple directories, so you don’t have to configure this job multiple times for different directories if the other configuration criteria are all the same. This job is also designed with customization in mind. All the available features and its usage are detailed on our wiki. Try this job out in your own free instance of Obsidian Scheduler.

Developing a Reusable File: Archival Schedulable Job

DevOps and full-stack have been popular topics in our industry for a number of years now. Unfortunately, they don’t always mean the same thing to every organization or even individuals within an organization. Rather than debate what they are or what they should be, this post will work through a solution created by the development team to either facilitate a frequently executed task performed by the operations team or provide a solution worthy of use by developers functioning in a part-time operations role – file archival.

File archival needs are typically driven by a few factors – limits of physical space, business or compliance requirements for retention and information availability for support teams. Rather than develop a solution that is fixed to a given set of requirements, we want a reusable, flexible solution that can be used and adapted without a need for new development.

Configuration Parameters

This schedulable job will need to accept configuration parameters. They are:

  • Source Directory with optional filename pattern OR Source File(s)
  • Archival Directory
  • Rename pattern
  • Compress Y/N?
  • Delete original Y/N?

We will develop our job as an Obsidian schedulable job but you can adapt this easily to any scheduling platform you desire. In fact, here is the resulting Java source code available under the MIT open source licence for you do to as you wish. FileArchiveJob.java.

Job Code

The basic algorithm is as follows.

  • Iterate over files
  • Ensure archive directory exists
  • Archive file – applying compression if selected
  • Delete original if selected

Since our job is an Obsidian job, we use a source job’s results as the most flexible and powerful mechanism to get your archival list.

Here’s what our main archival method ends up looking like:

protected void processFile(Context context, File f) throws ParameterException, IOException, DBException {
  boolean gzip = Boolean.TRUE.equals(context.getConfig().getBoolean(GZIP));
  String newName = determineArchiveFilename(f);
  for (String dirPath : context.getConfig().getStringList(ARCHIVE_DIR)) {
    File dir = new File(dirPath);
    if (!dir.exists()) {
      dir.mkdirs(); 
    } 
    if (!dir.isDirectory()) {
      throw new RuntimeException("Archive directory does not exist and could not be created: " + dir.getAbsolutePath());
    }
    File archiveFile = new File(dir, newName);
    if (!Boolean.TRUE.equals(context.getConfig().getBoolean(OVERWRITE)) && archiveFile.exists()) {
      throw new RuntimeException("File already exists and overwrite is disabled: " + archiveFile.getAbsolutePath());
    }
    InputStream src = null;
    OutputStream dest = null;
    FileOutputStream fos = null;
    try {
      src = new BufferedInputStream(new FileInputStream(f));
      fos = new FileOutputStream(archiveFile);
      if (gzip) {
        dest = new BufferedOutputStream(new GZIPOutputStream(fos));
      } else {
        dest = new BufferedOutputStream(fos);
      }
      IOUtil.copyStream(src, dest, 4096, false);
      dest.flush();
      dest.close();
      context.saveJobResult("archiveFile", archiveFile.getAbsolutePath());
    } finally {
      IOUtil.closeQuietly(src);
      IOUtil.closeQuietly(dest);
      IOUtil.closeQuietly(fos);
    }
  }
}

And there’s a lot more this job offers. It is designed with customization in mind and has some additional optional features detailed on our wiki. Let us know what you think or take this job for a spin in your own free instance of Obsidian Scheduler.

Next up, we’ll take a look at a reusable file cleanup job.

The Mouse is a Programmer’s Enemy

One of the first programming management books I was encouraged to read was Peopleware – Productive Projects and Teams. It was a great read and I try to re-read it every once in a while.

One of the topics covered is actually a term that comes from psychology – flow. Flow carries the idea of being completely mentally immersed in a task.

There are a lot of things that can break us out of flow or prevent us from ever entering that sate that are out of our control. But I want to focus on something that is completely within our control and that could be interrupting our flow hundreds of times per day.

The Mouse

Reaching for the mouse (trackpad, touchpad, etc.) is a very natural instinct for many of us. But removing one of our hands from the keyboard actually can disrupt our thinking and hamper productivity – albeit not to the extent of many of the other distractions we contend with. Modern IDEs are so feature rich, that if you are deeply involved in a development task, know your requirements and have a well-thought out design that meets those requirements, you can perform much of the development without having your fingers leave your keyboard and maintain your blissful state of flow.

Keyboard Shortcuts

Most of us would agree that using the mouse to perform frequent operations like copy/cut/paste/save/undo/redo is unnecessary. But we can go much further in our effort to keep our minds focused and increase our productivity.

I won’t bother outlining what the shortcuts are since every IDE has its own set of built-in keyboard shortcuts, most allow for customization of these and some languages may have some shortcut concepts that don’t apply elsewhere. What I will do is outline some of the shortcuts that you should know for your IDE and, in brief, how they will benefit you.

I work in Java most often day-to-day, so some of these may apply more strictly to Java developers.

Refactor – rename / move

Don’hesitate to rename variables, files or move them if it’s in your best interest to do so.

Generate

Generate entire code files, variables, implementation shells.

Open resource

Get to that code or resource file by name.

Open selection

Open the item your cursor is on.

Find references

Find all code uses of the item your cursor is on.

Show hierarchy

Display class hierarchy of selected item.

Indent / Outdent

Keep that code looking beautiful.

Comment / Toggle Comment

Quickly and easily handle code blocks to comments.

Cleanup / Format

More code beautification. Can even be used to resolve code problems.

Add Import

Java import of a specific class.

Run/Debug

Quickly relaunch last launched or the code that’s open in your editor.

Set/Toggle Breakpoint – Step Into / Step Out / Step Over / Run to Line / Resume

Debugger shortcuts.

Quick Fix / Quick Assist / Suggest

Super powerful! Code problem? Your IDE might know how to fix it! Also, save on pointless typing with assist/suggest features.

Duplicate lines

Need to perform another similar operation to an existing block of code? Line duplication without copy/paste!

Templates

Repetitive typing tasks simplified.


In Conclusion

These are just some of my favourites. There are dozens, even hundreds more available. Interested in more information?
IDEA Keyboard Shortcuts
Eclipse Keyboard Shortcuts
Have a favourite that I haven’t listed? Leave a comment and let us know!

Scheduling a Job in Quartz Versus Obsidian

We frequently compare Quartz and Obsidian in our blog, and today we’re going to see the difference in how you would schedule a job for recurring execution in both pieces of software.

A Quick Note on Job Configuration

Before we show the API that you use for Quartz and Obsidian, first I’ll mention that using an API typically isn’t the best approach to scheduling a job. Quartz provides a mechanism to configure jobs via XML, and Obsidian provides a full administration and monitoring web application for you to schedule jobs.

However, some use cases definitely suggest API integration, so lets move forward with that.

Quartz Example

Let’s see what scheduling a job to run every half hour looks like in Quartz.


// First, create the scheduler
SchedulerFactory sf = new StdSchedulerFactory();
Scheduler scheduler = sf.getScheduler();

// Build up the job detail
JobDetail job = JobBuilder.newJob(HelloWorld.class)
    .withIdentity("HelloWorldJob", "HelloWorldGroup")
    .build();

// Add some configuration
job.getJobDataMap().put("planet", "Earth");

// Create the scheduler
CronScheduleBuilder schedule = CronScheduleBuilder.cronSchedule("* 0/30 * * * ?");

// Create the trigger
CronTrigger trigger = TriggerBuilder.newTrigger()
    .withIdentity("HelloWorldTrigger", "HelloWorldGroup")
    .withSchedule(schedule)
    .build();

// Schedule the job with the created trigger.
scheduler.scheduleJob(job, trigger);

scheduler.start(); // This is how you start the scheduler itself if you need to

// Later, we can shut down the scheduler 
scheduler.shutdown();

As you can see, first you have to get a handle on the scheduler instance (or create it), then you create the JobDetail, CronScheduleBuilder and CronTrigger, and then you can finally schedule the job itself.

There are a few seemingly superfluous steps here, and some extraneous properties like job identities, trigger names, trigger groups, etc. that you have to provide, but this is the basic template you’d use.

Obsidian Example

Now let’s see how it’s done in Obsidian. We are using the same job class (let’s assume it satisfies both Quartz and Obsidian’s job interfaces), and we’ll use the same every-half-hour schedule.

	
// Create our configuration parameters
List<ConfigurationParameter> parameters = Arrays.asList(
         new ConfigurationParameter().withName("planet")
                                     .withType(ParameterType.STRING)
                                     .withValue("Earth")
);

// Set up the job configuration
JobCreationRequest request = new JobCreationRequest()
	.withNickname("HelloWorld")
	.withJobClass(HelloWorld.class.getName())
        .withState(JobStatus.ENABLED)
	.withSchedule("* 0/30 * * *")
        .withRecoveryType(JobRecoveryType.LAST) // how to recover failed jobs
	.withParameters(parameters);

// Now actually save this configurations, which becomes active on all schedulers.
JobDetail addedJob = new JobManager().addJob(request, "Audit User");
System.out.println("Added new job: " + addedJob );

// If you need to start an embedded scheduler, use this line to initialize it.
SchedulerStarter scheduler = SchedulerStarter.get(SchedulerMode.EMBEDDED);

// Later, we can gracefully shut down the scheduler 
scheduler.shutDown();

As you can see, Obsidian keeps things simple and does away with extraneous properties which don’t help you develop and manage your jobs. You simply create a JobCreationRequest with the required fields above, including any ConfigurationParameters, and call JobManager.addJob() with the job configuration and an optional audit user, which is used to track changes.

This API call saves your configuration to the Obsidian database, so it is instantly propagated to all schedulers in your cluster, and it outlives restarts. Many of our users take advantage of this API to perform one-time initialization of job schedules, after which they use our powerful web application to make changes as they are required.

This API was carefully designed to expose the powerful features our users demand, while still remaining simple to use. This sample will give you the basic template for how to schedule a job in Obsidian, but if you need more detail or want to use other extended features, you can check out our full Embedded API documentation.

If you have any questions about Quartz versus Obsidian, or perhaps about Obsidian features, please leave a note here or contact us and we’ll be happy to reply.

Comparing Quartz, cron4j and Obsidian Scheduler

We’ve all worked on projects that required us to do very basic tasks at periodic intervals. Perhaps we chose a basic ScheduledThreadPoolExecutor. If we’re already using Spring, maybe we tried their TaskExecutor/TaskScheduler support.

But once we encounter any number of situations such as an increased quantity of tasks, new interdependencies between tasks, unexpected problems in task execution or the like, we will likely start to consider a more extensive scheduling solution.

Our website has a fairly exhaustive feature comparison of the most commonly used Java schedulers, so we won’t go into that in this post, but we do encourage you to take a look.

Features aside, are there other criteria that should come into play? Factors such as development team responsiveness to feature requests and bug reports certainly can be critical for many organizations. If you head over to the Quartz Download page, you’ll see that they haven’t had a release in over a year, despite there being many active unresolved issues. Cron4j hasn’t had a release in over 2 years. While Spring has made some changes to the design of their TaskExecutor/TaskScheduler support in recent releases, their true priorities lie elsewhere as they have not really done much to expand the feature set.

Obsidian Scheduler on the other hand is actively maintained, actively supported (with free online support!) and responsive to our user community. In the past year, we’ve averaged a release per month delivering a blend of features, enhancements and fixes, proof that we’re a nimble and responsive organization. We encourage you to give Obsidian a try today!

Java Enterprise Software Versus What it Should Be

A lot of developers end up in the Java “enterprise” world at some point in their careers. I know the term alone conjures up all kinds of reactions, and rightly so. Often environments where lots of interesting technical challenges exist end up being those that nobody wants to work on because they are brittle, difficult to work on and just no fun. Often the problems that crop up in big projects are due to management, but I’ve seen developers make lots of bad decisions that lead to awful pieces of software, all in the name of “enterprise”.

What is Enterprise?

You could argue that the term can mean just about anything, and that’s true, but for the sake of this article I’m going to define it in a way that I think lines up with the common usage. The average enterprise project has these attributes:

  • typically in a large corporate environment
  • multiple layers of management/direction involved
  • preference for solutions from large vendors like Red Hat, IBM or Microsoft
  • preference for well-known, established (though sometimes deficient) products and standards
  • concerns about scaling and performance

Now that I’ve defined what type of project we are talking about, let’s see what they usually end up looking like.

The Typical Enterprise Java Project

Most of us have seen the hallmarks of enterprise projects. It would help if we had an example, so let’s pretend it’s an e-commerce platform with some B2B capabilities. Here’s what it might look like:

  • EJB3 plus JPA and JSF – these fit a “standard” and everyone use them, so it’s safe choice.
  • SOAP – it’s standard and defines how things like security should work, so there’s less to worry about.
  • JMS Message-Driven Beans – it fits into the platform and offers reliability and load-balancing.
  • Quartz for job scheduling – a “safe” choice (better the enemy you know than the devil you don’t).
  • Deployed on JBoss – it has the backing of a large company and paid support channels.

Now, the problem with a project like this isn’t necessarily the individual pieces of technology selected. I definitely have issues with some of the ones in my example, but the real issue is how the choices are made and what the motivations are for using certain technologies.

The stack of software above is notoriously more difficult to manage and work with compared to other choices. Development will take longer to get off the ground, changes will be more difficult to make as requirements evolve, and the project will ultimately end up more complicated than other possible solutions.

Enterprise Decision-Making

The goals that enterprise project usually fixate on when making choices are:

  • Low-risk technology – make a “safe” choice that won’t result in blow-back, even if it is known to have serious drawbacks.
  • Standards obsession – worrying more about having well-defined specifications like EJB3 or SOAP than offering the simplest solution that does the job effectively.
  • Need for paid support with an SLA, often without concern to the quality or timeliness of responses.
  • Designing out of fear of unknown future requirements.

These goals aren’t bad ones, with the exception of the last one, but they tend to overshadow the real goals of every software project. The primary objective of all software projects is to deliver a project that:

  • is on-time;
  • meets requirements;
  • is reliable;
  • performs well; and
  • is easy to maintain and extend.

These should be what decision-makers focus on in software projects, whether small or large. It’s obvious that sometimes special organizational needs factor into the choices that are made, but fundamentally good choices generally apply in all types of organizations.

So what if we were to reimagine our project with those goals in mind instead?

A Reimagined Enterprise Project

First, a little disclaimer: there are many ways one can go in any project, and I won’t claim that the technologies below are better that the ones mentioned previously. Tools need to be evaluated according to your needs, and everyone’s are different.

What I will try to do is demonstrate an example technology stack along with the reasoning for each choice. This will show how well designed systems can be built which survive in the enterprise world without succumbing to the bad choices that are often made.

Here’s the suggested stack:

  • Spring MVC using Thymeleaf – stable history, lots of development resources, quick development and flexibility. Don’t be afraid of using platforms or libraries, but try to avoid too much “buy-in” to their stack that you might regret.
  • Simple database layer using jOOQ for persistence where useful. This lets us manage performance in a more fine-grained way, while still getting easy database interaction and avoiding ORM pitfalls.
  • REST using Jackson JSON processor – REST and JSON are both popular because they are easy-to-use and understand, cheap to develop, use simple standards and are familiar to developers. Lock-in isn’t much of a problem either – we could easily switch to another JSON processor without much difficulty, unlike SOAP. This can be easily secured with SSL and basic authentication.
  • JMS messaging using JSON-encoded messages on ActiveMQ – loose coupling, reliability and load-balancing, without the nastiness of being stuck with Message-Driven Beans.
  • Obsidian Scheduler – simple-to-use, offers excellent monitoring and reduces burden on developers. Once again, the goal is to simplify and reduce costs where possible.
  • Deployed on Tomcat – no proprietary features used. This helps us follow standards, avoid upgrade issues and keep things working in the future. Who needs support with SLAs when things aren’t inexplicably breaking all the time?

I think the stack and corresponding explanations above help give an idea to what an enterprise project can be if it’s approached from the right angle. The idea is to show that even enterprise projects can be simple and be nimbly built – bloated frameworks and platforms aren’t a necessary part, and rarely offer any significant tangible benefit.

In Closing

Recent trends in development to technologies like REST have been very encouraging and inroads are being made into the enterprise world. Development groups are realizing that simplicity leads to reliability and cost-effective solutions long as the underlying technology choices meets the performance, security, etc. needs of the project.

The software world moves quickly, and is showing promising signs that it’s moving in the right direction. I can only hope that one day the memories of bloated enterprise platforms fade into obscurity.

Why You Still Shouldn’t Use Quartz Scheduler

When I first wrote on this topic, I lamented that the de facto standard for scheduling jobs in Java applications is use of the Quartz scheduling library. Why? At the time, I explained:

  • Enabling your application to schedule jobs with Quartz takes significant code and/or configuration.
  • Changes in job schedule or other parameters require code/configuration changes with a corresponding software build/test/deploy cycle.
  • Quartz is just a library and not a scheduling solution. It has no built-in UI for configuration or monitoring, no useful and reasonably searchable logging or historical review, no support for multiple execution nodes, no administration interface, no alerts or notifications, not to mention buggy recovery mechanisms for failed jobs and missed jobs.

Quartz hasn’t changed much in the two years since I wrote that post. A single minor release and two maintenance releases have done nothing to address its pain points.

It’s true that Quartz has some free or even commercial add-ons to address some of these issues, but these require additional configuration, add complexity to your software environment and are by nature afterthoughts and will never be able to fundamentally change Quartz’ nature – a library. Obsidian Scheduler has been designed from the ground up to address all of these shortcomings and more and is free to use for a single scheduling instance.

Consider some reasons why Quartz could disappoint you should you choose this “standard” for your project.

Scheduling Configuration

Quartz requires code or XML configuration or some combination thereof to schedule jobs. This has multiple drawbacks, the biggest of which is this: verifying your code or configuration changes requires either deploying your software environment in its target environment (WAR, EAR, standalone Spring application, etc.) or some appropriate test harness before finally verifying the results. If there is a severe issue, you must then decipher an API exception message and attempt to make the necessary changes and then restart the verification cycle. This process is long and tedious, especially if you have to make multiple passes to correct issues.

Obsidian on the other hand requires no configuration or code to make scheduling changes or other job execution parameters. These changes are validated and applied in real-time or can even themselves be scheduled ahead of time. You can even see a preview of when jobs will run, so you aren’t left wondering if you got your schedule correct. You are free to do this using the UI or you can use the REST API.

Configure Job Conflict Management Chaining

Execution Context

Quartz is a library. This means that it is nothing more than an API. It does not provide an execution context. Why is an execution context important? An execution context is required to interact with your scheduling environment in real-time. Any number of things you might like to do are not possible because of no execution context. Disable a job? Change a schedule? See what jobs are running? Subscribe to scheduling event? Some of these are not possible without an execution context while others are impractical. Even if you have an execution context in your application already, such as Spring, you need to expose the Quartz API functions in your context and then make them available for invocation in some UI or expose them in some new or existing context API.

Obsidian includes an execution context that can either run as standalone or embedded into an existing context (such as Spring). This execution context provides many of Obsidian’s useful features, including clustering, interactive scheduling changes via GUI or REST, real-time monitoring, scheduler pausing (global or host-specific) and so on.

Operational Roles

With Quartz, everything relating to job scheduling becomes a function of the development team. Job failed or didn’t trigger when expected? Developer must investigate in the log files. Scheduling change? Developer must make the change, test it and support pushing the change to the target environments. Need to disable the scheduler? More code changes.

With Obsidian Scheduler, we make it possible to reduce the burden on the development team and allocate functions to more appropriate owners. Developers really shouldn’t have any more access to your live environment than is absolutely necessary. If they do need an access account to Obsidian for support, give them a read-only account that allows them to support the operations and business groups in their work. Or give them the limited-read role if your job configurations contain sensitive information such as IDs or passwords. Business teams or other appropriate designates could be given write access to make job scheduling and parameter changes or submit ad-hoc jobs. Your operations staff can be given the admin role to configure the Obsidian run-time itself (thread pool sizes, system recovery, etc.)

If you’ve ever used Quartz then you’ve likely suffered through these and other issues. Download Obsidian today and experience how productive and efficient job scheduling can be!

Integrating Spring and Obsidian Scheduler

The use of Spring framework has woven its way into a large percentage of Java projects of all sorts. It has grown well beyond a dependency injection framework to a large interconnected technology stack providing “plumbing” in a large number of areas. Since it’s so often a part of our lives, we’ll discuss Obsidian’s support of Spring and how to quickly integrate your use of Spring into Obsidian Scheduler.

Version Indifferent

First, Obsidian does not include any Spring libraries and is not dependent on Spring to run. Given that Spring is a very active project and has frequent releases, and at times projects have constraints that require using older versions of certain libraries, we felt it would be best if Obsidian just worked with whatever version of Spring you’re using. Obsidian supports versions 2.5 and later of Spring.

Configuring Job Components

The first step in integrating Obsidian Scheduler is often already in place in your project. Obsidian expects to find @Component annotations or any Spring or custom extensions of those annotations (such as @Service and @Repository) on your job classes. If you’re using these annotations to wire your classes, you likely don’t have to do anything else to complete this step. If you’re not using these annotations to wire your classes, you will need to add the annotation to serve as a Marker to Obsidian. If you happen to have multiple instances of a given bean class in your Spring context, Obsidian will need to know how to uniquely identify the job within the Spring context. In these cases, use value="" in the annotation to provide the bean name. For example:

@Component("mySpringComponent")
public class SpringComponent implements SchedulableJob {

Providing the Spring Context to Obsidian

The next step is to let Obsidian know about your Spring context. Included with Obsidian is an implementation of ApplicationContextAware. If you’re using classpath scanning in Spring, you can just add the Obsidian package to your configuration. Here is a sample:

<context:component-scan base-package="com.my.package, com.carfey.ops.job.di"/>

If you’re wiring explicitly in a configuration XML file, here is a sample configuration:

<bean id="obsidianAware" class="com.carfey.ops.job.di.SpringContextAware"/>

Startup & Shutdown Options

The third step, which is optional and depends on your desired usage, relates to starting and stopping Obsidian. If you’re configuring Spring integration for use in the Obsidian standalone Web Administration application (with no scheduler running), you would not include this configuration. In all other cases, such as embedded scheduler or Web Administration application with bundled scheduler, you can use this sample:

<bean id="obsidianStarter" class="com.carfey.ops.job.di.SpringSchedulerStarter"/>

This bean should wired as late as possible in the Spring context startup, using depends-on or other appropriate strategies. This class does not possess a @Component annotation since it cannot be auto-wired via Spring classpath scanning.

Now your Add/Edit Job list in the Web Administration UI, will include all implementations of com.carfey.ops.job.SchedulableJob that you’ve wired through Spring. If you’re using Spring 3.0 or later, even jobs that use Obsidian’s annotations com.carfey.ops.job.SchedulableJob.Schedulable and com.carfey.ops.job.SchedulableJob.ScheduledRun are included. See our wiki for more information on implementing jobs using the interface or the annotations.

Good to Go!

You now have complete Spring integration with Obsidian. All job instances that loaded via Spring annotations will be loaded from the Spring container when performing validation and execution. Note that these could be either prototypes or singletons according to your needs, though we recommend prototypes if you have any kind of internal state within the job.

Of course Spring is not the only dependency injection framework available in Java. On the roadmap for Obsidian is similar integration with Google Guice. Stay tuned! Get in touch with us if you have any questions on Spring integration or any other Obsidian Scheduler features.

Reusable Jobs with Obsidian Chaining

A lot of Obsidian users write custom jobs in Java and leverage their existing codebase to performs things like file archive or file transfer right inside the job. While it’s great to reuse that existing code you have, sometimes users end up implementing the same logic in multiple jobs, which isn’t ideal, since it means extra QA and potential for inconsistencies and uncaught bugs.

Fortunately, Obsidian’s configurable chaining support combined with using job results (i.e. output) lets developers write a single job as a reusable component and then chain to it where required.

To demonstrate this, we will go through a fairly common type of situation where you have a job which generates zero or more files which must be transferred to a remote FTP site and which also must be archived. We could chain to an FTP job which chains to an archive job, but for the sake making this example simpler, we will bundle them into the same job.

File Generating Job

First, we’ll demonstrate how to save job results in our source job to make them available to our chained job. Here’s the execute() method of the job:

public void execute(Context context) throws Exception {
   // grab the configured FTP config key for the job 
   // and pass it on to the chained FTP/archive job
   context.saveJobResult("ftpConfigKey", 
         context.getConfig().getString("ftpConfigKey"));

   for (File file : toGenerate) {
      // ... some code to generate the file

      // when successful, save the output 
      // (multiple saved to the same name is fine)
      context.saveJobResult("generatedFile", file.getAbsolutePath());
   }
}

Pretty simple stuff. The most interesting thing here is the first line. To make the chained FTP/archive job really reusable, we have configured our file job with a key which we can use to load the appropriate FTP configuration used to transfer the files. We then pass this configuration value onto the FTP job as a job result, so that we don’t have to configure a separate FTP job for every FTP endpoint. However, configuring a separate FTP job for each FTP site is another option available to you, in which case you wouldn’t have to configure the file job with the config key or include that first line.

Next we’ll see how to access this output in the FTP/archive job and after that, how to set up the chaining configuration.

FTP/Archive Job

This job has two key features:

  1. It loads FTP config based on the FTP key passed in by the source job.
  2. It iterates through all files that were generated and deals with them accordingly.

Note that all job results keep their Java type when loaded in the chained job, though they are returned as List<Object>. Primitives are supported as output values, as well as any type that has a public constructor that takes a String (toString() is used to save the values).

public void execute(Context context) throws Exception {
   Map<String, List<Object>> sourceJobResults = context.getSourceJobResults();
   List<Object> fullFilePaths = sourceJobResults.get("generatedFile");
   
   if (fullFilePaths != null) {
      if (sourceJobResults.get("ftpConfigKey") == null) {
         // ... maybe fail here depending on your needs
      }
      String ftpConfigKey = (String) sourceJobResults.get("ftpConfigKey").get(0);
      FTPConfig config = loadFTPConfig(ftpConfigKey);
	  
      for (Object filePath : fullFilePaths) {
         File f = new File((String) filePath);
         // ... some code to transfer and archive the file

         // Note that this step ideally can deal with already processed files
         // in case we need to resubmit this job after failing half way through.
      }
   }
}

Again, this is pretty simple. We grab the saved results from the source job and build our logic around it. As mentioned in the comments, one thing to consider in an implementation like this is handling if the job fails after processing only some of the results. You may wish to just resubmit the failed job in a case like that, so it should be able to re-run without causing issues. Note that this isn’t an issue if you only ever have a single file to process.

Chaining Configuration

Now that the reusable jobs are in place, here’s how to set up the chaining. Here’s what it looks like in the UI:

Chaining Configuration

We use conditional chaining here to indicate we only need to chain the job when values for generatedFile exist. In addition, we ensure that an ftpConfigKey value is set. The real beauty of this is that Obsidian tracks why it didn’t chain a job if it doesn’t meet the chaining setup. For example, if the ftpConfigKey wasn’t setup, the FTP/Archive job would still have a detailed history record with the “Chain Skipped” state and the detailed message like this:

Chain Result

Note that in this example it’s not required that we do conditional chaining since our FTP/archive job handles when there are no values for generatedFile, but it’s still a good practice in case you have notifications that go out when a job completes. It also makes your detailed history more informative which may help you with troubleshooting. If you don’t wish to use conditional chaining, you could simply chain on the Completed state instead.

Conclusion

Obsidian provides powerful chaining features that were deliberately designed to maximize productivity and reliability. Our job is to make your life easier as a developer, operator or system administrator and we are continually searching for ways to improve the product and provide values to our user.

If you have any questions about the examples above, let us know in the comments.