Right Outer Join

22 September 2011

Date Queries in MongoDB Reports

Filed under: JasperReports Server — Tags: , , , — mdahlman @ 12:21

First Dates with MongoDB

Background

You need some software before you can make use of these ideas:

You should probably start by reading my previous articles before reading this one:

Date queries in MongoDB reports

This idea actually applies to all types of Java objects, not just Dates. But most data types likes Strings and simple numbers get converted magically to what you probably want. Dates are probably the most common object that is too complex to be handled by this automatic conversion process.

Let’s take what is perhaps the simplest possible example. I want a query that will return only the documents for which a specified field, status_date, is later than some date. It’s a simple JSON-style query in the MongoDbQuery language of the Jaspersoft MongoDB connector:

{ 
  'collectionName' : 'active_users',
  'findQuery' : { 'status_date' : {'$gte':$P{StartDate}} }
}

It’s easy enough to understand even if it’s a bit verbose with curly braces in curly braces in curly braces. My report has a java.util.Date parameter which gets passed into the query. In earlier versions of the connector—as pointed out by one astute reader—there was no way to do this. Attempts to embed syntax like this into the query resulted in JSON parsing exceptions: { ‘status_date’ : {‘$gte’: new ISODate(’2011-09-15T00:00:00.000Z’)’ } }

Sample Data

In case you’re relatively new to MongoDB, like me, it can smooth your path to testing to have a sample data set and sample report that are known to work well together. You can insert a these few documents on the MongoDB shell and then use the sample report listed below to get something that ought to work with minimal effort. Then you’re ready to start writing your own reports.

The first document is listed on many lines to make it easier to parse it by eye to see what’s in it. The second two documents are compressed onto a single line. The final “find” command is useful to confirm that the earlier inserts worked as expected.

db.active_users.save(
{
"status_date" : ISODate("2011-09-01T07:48:58Z"),
"id" : "123",
"events" : {
"ACHIEVEMENT_Minor" : 1,
"ACHIEVEMENT_Major" : 1,
"ACHIEVEMENT_Ultimate" : 0
},
"install_source" : {
"detail" : "campaign - Free iPad",
"vt1" : "email",
"vt2" : "facebook"
},
"rev" : {
"net_total" : 700
}
}
)
db.active_users.save({"status_date" : ISODate("2011-09-09T22:17:27Z"),"id" : "456","events" : {"ACHIEVEMENT_Minor" : 1,},"install_source" : {"detail" : "campaign - Free iPad","vt1" : "email","vt2" : "facebook"},"rev" : {"net_total" : 450}})
d = new Date()
db.active_users.save({"status_date" : d,"id" : "789","events" : {"ACHIEVEMENT_Minor" : 1,},"install_source" : {"detail" : "campaign - Like us on Facebook","vt1" : "email","vt2" : "facebook"},"rev" : {"net_total" : 222}})
db.active_users.find()

Sample Report

Reminder

The idea for this article (and for this new feature to the Jaspersoft MongoDB Connector) came from a reader comment. Please let me know when you run into questions, but for general questions about the MongoDB connector please post to the JasperReports Server forum instead of posting a comment here. That’s a better location for asking and answering questions.

 

3 March 2010

JasperReports Second Dates

Filed under: iReport, JasperReports, JasperServer — Tags: , , , , — mdahlman @ 18:11

JasperReports Second Dates – Meet the Relatives

The Problem

I want a simple way to run a report for “Yesterday” or “Next Month”. It’s easy enough to parameterize the StartDate and EndDate; therefore the user can get whatever dates she wants. But what about schedules? If I know that the report always runs for “Last Month” then it is easy enough to put the date calculations into the SQL query. But what if my data source isn’t SQL? What if I want to use the same report for a weekly summary and a monthly summary?

The Solution

  1. Create a parameter named DateRange that lets users select the appropriate values like “Tomorrow”, “Next week”, etc.
  2. Create other parameters to convert DateRange into the corresponding StartDate and EndDate values. These parameters will rely on Java Commons Lang (specifically the DateUtils class) to calculate the dates.
  3. Create an input control in JasperServer associated with DateRange so users can select an appropriate value. Do not create input controls for StartDate and EndDate since these get calculated automatically.

The idea is simple enough, but the details are not trivial. I began by creating a simple table to query. It has a single date field populated with one value for each day in 2010. Using this my report can use this simple query:

SELECT the_date
FROM date_helper
WHERE
    the_date >= $P{StartDate}
and the_date <  $P{EndDate}

One of the most common problems with date ranges is missing values because truncating time values from a date. For example, if I plug “today” as the StartDate and the EndDate in the query above then I will get an empty result set. That’s because “today” really means “today at midnight”. So when I want data from “today” I need to set StartDate to “today” and set EndDate to “tomorrow”. It’s easy enough to do… but it’s easy enough to forget.

Next I defined the list of DateRanges that I want to make available. I settled on this sample:

  • Today
  • Yesterday
  • Tomorrow
  • This Week
  • Last Week
  • Next Week
  • This Month
  • Last Month
  • Next Month

I then defined the main logic for the date calculation. This logic should go in a separate Java class that gets compiled into a JAR file which can be shared by all the reports on the server. It should not be in the .jrxml file. I repeat: it should not be in the .jrxml file. Having said that, I put the logic in the .jrxml file. I considered this the simplest way to make the sample easily accessible by others. But if you plan to use this sample code, do yourself a favor and move it into a reusable JAR file.

The logic for the date calculations is simple. The hardest part was calculating the relative dates based on weeks. Java provides no simple way to determine the day of the week for a given Date. You can format a Date to return “Sun”, “Mon”, “Tue”, etc. But you cannot just get 0, 1, 2, etc. (I will be eternally grateful to the reader that tells me I’m wrong and shows how I can do it.) Therefore I needed to embed verbose logic like this:

DateFormatUtils.format(new Date(),"E").equals("Sun")
? 0
: DateFormatUtils.format(new Date(),"E").equals("Mon")
  ? 1
  : DateFormatUtils.format(new Date(),"E").equals("Tue")
    ? 2
    : DateFormatUtils.format(new Date(),"E").equals("Wed")
      ? 3
      : DateFormatUtils.format(new Date(),"E").equals("Thu")
        ? 4
        : DateFormatUtils.format(new Date(),"E").equals("Fri")
          ? 5
          : 6

Yikes. It’s easily understandable, but it’s painful.

Having done that, the report is trivially simple. The samples below show the results for “This Week” and “This Month”. Notice how the EndDate for ThisMonth is actually the first date of next month, but that returns only rows from this month.

The report calculates the start and end of "This Week"

The report calculates the start and end of "This Week"

The report calculates the start and end of "This Month"

The report calculates the start and end of "This Month"

When running the report in iReport you need to type in a value like “last week”. After you upload it to JasperServer and create the appropriate input control then you can more easily pick the relevant value from a list.

I have attached my .jrxml file along with the SQL needed to populate my sample table. Note that you can run the report without having the table. You would just need to remove the SQL query. If you do that then the report will still run. Obviously it won’t retrieve any data if it has no query, but it still calculates the StartDate and EndDate. You’ll see those displayed in the title band.

To make things easier I also attached a copy of the report exported from JasperServer. This exported copy includes both the .jrxml files as well as the input control that allows you to select from the predefined list of values.

There are plenty of ways to improve this sample. First, move the date calculation out of the report and into a separate JAR file. Second, there are all sorts of other date ranges you might want to include: fiscal periods, years, quarters, etc.

RelativeDatesJRXML_20100303 RelativeDatesExport_20100303 Browse attachments to all Right Outer Join articles

(The links above are .odt files because WordPress won’t let me attach .zip files. You need to download them and then manually change the filename to *.zip.)

9 September 2009

JasperReports First Dates

Filed under: JasperReports — Tags: , , , — mdahlman @ 16:09

Date Calculations in JasperReports

The Problem

When working with JasperServer, JasperReports, and iReport it’s common to want to perform date calculations. This is surprisingly hard to do in Java expressions. In pseudo code the desired result can be extremely easy to express. For example, “MyDate + 1 day” or “The first day of the month before the month in which MyDate occurs”. The problem is that while Java has all of the pieces you need to perform such calculations, it requires you to instantiate Calendar objects and use these to operate on Dates. But this isn’t possible in a simple JR parameter expression. So what can you do?

My standard solution was to push these date calculations to the SQL query. This works for many purposes, but it has drawbacks. For one thing SQL has lots of date functions, but they are painfully verbose for some needs. Worse, there are cases where I need the date calculation in the report but I don’t need it in the query. Frankly, I feel dirty adding an extra column to my result set just to calculate what the first day of this month is. And of course pushing the calculation to the SQL query isn’t an option at all when your data source isn’t a SQL database.

The Solution

I was recently introduced to the awkwardly named “Lang” component of the Apache Commons project. It’s homepage beautifully summarizes what it does. “The standard Java libraries fail to provide enough methods for manipulation of its core classes. The Lang Component provides these extra methods.” Somehow it manages to both subtly mock and mightily contribute back to the Java world simultaneously. Guillaume, thanks for the introduction; I’m sure Lang and I will become close friends.

It has a class called DateUtils with methods like addDays, parseDate, round, truncate, and setDays. Are you already salivating? These static functions allow you to perform all sorts of new data calculations with ease.

So let’s get started. I created a report that uses iReport’s built-in “Empty datasource” so it will be easy for any iReport user to work with it. I ran it on 9 Sept 2009, and it looks like this:

Date manipulations using Commons Lang's DateUtils

Date manipulations using Commons Lang's DateUtils

Do you see it? “MyDate + 1 day” There it is! It gets better. “The first day of the month before the month in which MyDate occurs” That’s there too! Look, it’s already tomorrow in New Zealand. Maybe they’ve already read my post even though I’m not quite done in my timezone. But I digress.

Notice "One import set"

Notice "One import set"

The syntax is easy to follow. For example MyDate + 1 day is just “DateUtils.addDays($P{MyDate},1)”. Don’t forget that you need to import the relevant package. In iReport click on the root report element, and in its properties you’ll find Imports. Add “org.apache.commons.lang.time.*” (without the quotes) as an import. If you don’t add this import then you would need to use the syntax “org.apache.commons.lang .time.DateUtils.addDays ($P{MyDate},1)”. It works… but it’s a lot harder to read. Thanks to CBox for reminding me of this.

I have attached the report here so that anyone can give it a try (or browse all Right Outer Join files). As always, I renamed it from .jrxml to .odt because WordPress won’t let me upload .jrxml files. Please rename it after downloading.

If anyone discovers useful examples that ought to make it into this report, please let me know. I’ll be happy to update the report and re-post it.

The Ignored Stuff

As useful as this is, I have ignored some highly useful related ideas. For example, what if I want to deploy a report to JasperServer so that the user can schedule it with appropriate start and end dates? It’s easy for them to choose specific start and end dates… but how can they select “the start of the month” or “the end of the year”? There’s no easy way. Using the techniques mentioned above it’s possible, but it’s certainly not trivial. Maybe I’ll get a chance to address this in another post soon.

Blog at WordPress.com.