Everything Code

Chrome and java applet plugin on ubuntu tails

2011-10-18T18:13:00.001+02:00

To enable do this:

sudo ln -sf /usr/lib/jvm/java-6-sun/jre/lib/amd64/libnpjp2.so /usr/lib/firefox-addons/plugins/libnpjp2.so
sudo ln -sf /usr/lib/jvm/java-6-sun/jre/lib/amd64/libnpjp2.so /usr/lib/mozilla/plugins/libnpjp2.so
mkdir ~/.mozilla/plugins
cd ~/.mozilla/plugins/
ln -s /usr/lib/firefox-addons/plugins/libnpjp2.so

More info here http://www.google.com/support/forum/p/Chrome/thread?tid=3ef7b4f7b08118ed&hl=en

Collection of noSQL presentations and terms

2011-01-17T11:27:00.005+02:00

Some great noSQL resources

A book club for noSQL papers http://nosqlsummer.org/papers with difficulty ratings and local meetings everyone can attend.

noSQL concepts that you really do want to understand: Bloom Filter, Vector Clocks, Gossip Protocol, Dynamo, Paxos, MapReduce, CAP, Eventual Consistency, Column Storage, Consistent Hashing, Hinted Handoff, Read Repair, Write Ahead. Also make sure to know what: Hadoop, GFS, Cassandra, Mongo are.

Lectures:

http://www.infoq.com/presentations/Riak-Core - Andy Gross review of Dynamo (and Riak)
http://nosqltapes.com/ - Collection of lectures about the noSQL ecosystem, plus the technical aspects.

Recursive delete utility for version enabled S3 bucket

2011-01-05T02:30:00.004+02:00

Deleting S3 bucket that contains anything more then the most trivial collection of keys (>1000) is a never ending saga. Deleting S3 bucket that has versioning enabled is with today's tools simply not possible.

We had 2 buckets that were taking up storage space: the first had ~300,000,000 keys, the second ~2500 but with versioning enabled and keys that contain "\n" (new line), "//" (double slash) and " " (space) in their names. Trying to use tools such as jets3, boto, bucket explorer, aws s3 console, s3fox on either of these buckets failed in the most pathetic/exotic of ways.

I quickly understood that this requires a custom implementation.

Doing some googling I found this http://mark.koli.ch/2010/09/recursively-deleting-large-amazon-s3-buckets.html excellent post proving some pseudo-code for multi threaded delete operation against S3. Good, the correct post in the correct time! Based on Mark's code I've implemented BucketDestroy which has the sole purpose of doing everything it takes to delete an S3 bucket, a seek & destroy S3 bucket commando if you like.

I've enhanced Mark code with the ability to delete not just bucket keys but bucket key versions, this is required because Amazon will not let you delete S3 bucket that has old versions of a Key.

The code is provided below, I've also hosted it on google code where you can find maven based setup instructions and the latest version.

Enjoy!

import java.util.List;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.atomic.AtomicInteger;

import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.services.s3.AmazonS3Client;
import com.amazonaws.services.s3.model.DeleteVersionRequest;
import com.amazonaws.services.s3.model.S3VersionSummary;
import com.amazonaws.services.s3.model.VersionListing;


/**
 * Efficient delete of version enabled bucket.
 * 
 * This code will delete all keys from gives bucket and all previous versions of given key.
 * This is done in an efficient multi-threaded ExecutorService environment
 * 
 * Launching:
 * 
 * $java -Daws.key=AK7895IH1234X2GW12IQ -Daws.secret=1234567890123456789012345678901234456789 BucketDestroy "bucketName"

 * @author Maxim Veksler <maxim@vekslers.org>
 * @author Mark S. Kolich http://mark.koli.ch/2010/09/recursively-deleting-large-amazon-s3-buckets.html
 * 
 * @version 0.1 - First pseudo code version, Written by Mark S. Kolich. 
 * @version 0.2 - Maxim Veksler - Fix paging. Implement deletion of versioned keys.
 * 
 * For the latest version see http://code.google.com/p/bucket-destroy/
 */
public class BucketDestroy {
    private static final String AWS_ACCESS_KEY_PROPERTY = "aws.key";
    private static final String AWS_SECRET_PROPERTY = "aws.secret";
    private static AmazonS3Client s3;
    
    public static void main(String[] args) {

        final String accessKey = System.getProperty(AWS_ACCESS_KEY_PROPERTY);
        final String secret = System.getProperty(AWS_SECRET_PROPERTY);
        if (accessKey == null || secret == null) {
            System.err.println("You're missing the -Daws.key and -Daws.secret required VM properties.");
            System.exit(100);
        }

        if (args.length < 1) {
            System.err.println("Missing required program argument: bucketName.");
            System.exit(100);
        }
        
        // S3 Initialization
        s3 = new AmazonS3Client(new BasicAWSCredentials(accessKey, secret));


        // Instantiate BucketDestroy and call destroyBucket() - Bucket deleting beast!
        new BucketDestroy().destroyBucket(args[0]);
    }

    /**
     * Do everything it takes to delete amazon S3 bucket. 
     * 
     * Currently it means delete all versions of bucket keys.
     * 
     * TODO: 
     *  - Support distributed operation by doing shuffle on first level of bucketName/../
     *  - Seek a method to obtain MFA to delete MFA enabled bucket
     *  - Implement starvation prevention for deleting threads (do not wait until next s3.listVersions finishes)
     */
    public void destroyBucket(final String bucketName) {
        // Set up a new thread pool to delete 20 objects at a time.
        ExecutorService _pool = Executors.newFixedThreadPool(20);

        // Get counter, just for status
        final AtomicInteger ATOMIC_INTEGER = new AtomicInteger();
    
        // List all key in the bucket
        VersionListing versionListing = s3.listVersions(bucketName, "");
        List<S3VersionSummary> versionSummaries = versionListing.getVersionSummaries();
        while(versionSummaries != null && versionSummaries.size() > 0) {
            final CountDownLatch latch = new CountDownLatch(versionSummaries.size());
            for(final S3VersionSummary objectSummary : versionSummaries) {
                _pool.execute(new Runnable() {
                    @Override
                    public void run() {
                        String keyName = null;
                        String versionId = null;
                        try {
                            keyName = objectSummary.getKey();
                            versionId = objectSummary.getVersionId();
                            
                            String info = ">>>>   INFO: " + ATOMIC_INTEGER.incrementAndGet() + " deleting: (" + bucketName + "/" + keyName + "@" + versionId + ")";
                            System.out.println(info);
                            
                            s3.deleteVersion(new DeleteVersionRequest(bucketName, keyName, versionId));
                        } catch (Exception e) {
                            String err = ">>>> FAILED delete: (" + bucketName + "/" + keyName + "@" + versionId + ")";
                            System.err.println(err);
                        } finally {
                            latch.countDown();
                        }
                    }
                });
            }
            
            // After sending current batch of delete tasks we block until Latch reaches zero, this 
            // allows to not over populate ExecutorService tasks queue. 
            try {
                latch.await();
            } catch (Exception exception) {
            }

            // Paging over all S3 keys...
            versionListing = s3.listNextBatchOfVersions(versionListing);
            versionSummaries = versionListing.getVersionSummaries();
            
        }
        
        _pool.shutdown();

        try {
            // This code fails with HTTP 400 from aws API 
//            SetBucketVersioningConfigurationRequest setBucketVersioningConfigurationRequest = new SetBucketVersioningConfigurationRequest(bucketName, new BucketVersioningConfiguration().withStatus(BucketVersioningConfiguration.OFF));
//            s3.setBucketVersioningConfiguration(setBucketVersioningConfigurationRequest);
            // So we do this:
            s3.deleteBucket(bucketName);
            System.out.println(">>>>   INFO: bucket " + bucketName + " deleted!");
        } catch (Exception e) { 
            System.err.println("Failed to ultimately delete bucket: " + bucketName);
            e.printStackTrace();
        }

    }
    
}

Accessing hadoop web interface on AWS

2010-08-09T14:24:00.005+03:00

Hadoop exports status interface via HTTP, using DNS names.
Amazon uses internal DNS names in their AWS cloud.

Therefor, should you request to access Hadoop status interface from outside the cloud (like your office pc... you will fail).

This guide suggest a solution (using a claver work around) to this problem.

Problem Description

Hadoop builds status html pages & links based on current machine hostname. This hostname represents internal machine DNS name which is resolvable from within the AWS cloud using AWS internal DNS server:

cat /etc/resolv.conf 
nameserver 172.16.0.23
domain ec2.internal
search ec2.internal

the problem here comes into existence when trying to access the status page from your development box (Office, home, mobile, telepathic...) here your machine will try to resolve ip-10-202-30-119.ec2.internal using your ISP DNS server, which will obviously fail.

Solution

The solution to the above problem is to use a combination of several cool tricks in the basis of which stands SOCKS5 proxy. Let's get started:

Install Firefox 3 http://www.mozilla.com/firefox/
FoxyProxy Standard is a proxy manipulation utility for firefox, install it from https://addons.mozilla.org/en-US/firefox/addon/2464/
Make the following configurations to FoxyProxy
1. Right click on foxy proxy Icon and choose options
2. Choose "Add New Proxy"
  
  General
  Proxy Name = AWS Internal
  
  Proxy Details
  Select * Manual proxy configuration
  Host or IP Address = localhost Port = 6666
  [v] SOCKS proxy? (x) SOCKS v5
  
  URL Patterns
  Create 2 patterns: *ec2.internal*, *domu*.internal*
  * Tip: You might want to consider a 3rd patten if you are for example from Israel... *pandora.com*
3. Make sure you have selected "Use proxies based on their pre-defined patterns and priorities" from the FoxyProxy options menu.
This step diverts for Windows & Linux users
Windows
//TODO// Configure putty with dynamic tunning.

Linux
```
nohup ssh -CqTnN -D 6666 ubuntu@ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com
```

OK... now try to access http://ip-10-202-30-119.ec2.internal:50070/dfshealth.jsp you should be getting a nice GUI allowing you to browse your HDFS file system.

How it works

You are creating using SSH protocol a socks proxy tunned over ssh. From one hand your ssh client accepts SOCKS5 proxy requests on port 6666 (localhost) and transfers them to the host inside AWS cloud. Your firefox is set to do requests for *patten* which matches internal host names inside AWS cloud over a proxy (using foxyproxy). So what happens is the following:

Firefox: "Hmmm, I need to do HTTP GET for ip-10-202-30-119.ec2.internal" - Who should I be talking to? OH! I know, foxyproxy tells me I should delegate this to my friends, the socks5 proxy at localhost:6666, cool.
Firefox: Hey localhost:6666 (socks5 proxy), do me a favor, get me the content for ip-10-202-30-119.ec2.internal:50070
localhost:6666: Hmmm, ok.
localhost:6666 -> ssh -> ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com: HTTP GET http://ip-10-202-30-119.ec2.internal:50070
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com: Hey, I know him. let me do a quick http get on his ass
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com: HTTP GET 10.202.30.119:50070
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com -> ssh -> localhost:6666 -> Firefox

This completes the cycle. At this point (skipped a few for clerity) you, the user, can the context of the HTTP GET request.

Mission accomplished.

Note that other DNS address (not from the aws ec2 cloud) are not being served over the socks proxy, for example if you go to http://google.com you get a direct hit by the browser - Which is exactly what you want to happens because traffic that goes via proxy has it's costs (bandwidth, latency).

Hope you enjoyed this, comments (as always) are welcome.
Maxim.

Ubuntu perfect setup for Java Developers

2010-07-31T21:33:00.008+03:00

MySQL General Query Log Filtering using grep and regular expressions

2010-07-13T12:56:00.004+03:00

MYSQL has a very useful feature called Query Log which allows you among other things to debug your application code by viewing DB DML's submitted live from the application straight into DB back-end (Low level, the best!).

To configure this little joy you will need to play a bit with your mysql server settings.

Edit your my.cnf file (Under debian that is /etc/mysql/my.cnf) in the section of

[mysqld]

add

log=/var/log/mysql/query.log

This is explained in greater details at this post.

Now the thing is query log as the name suggests logs everything that has SQL smell on it and I do mean EVERYTHING (besides CRUD operations you will see: commit/rollbacks, show tables, use DB-XYZ and co.) - Which is mostly junk when the only thing you really need is to trace your hibernate based Console save to get the back-end integration going... To summarize in a catch phrase: You wan't to filter out the log spam when debugging your application (vs. your replication ;).

To be more efficient and be able to look at the logs in real time I've created a fast and dirty regex. The following chain of piped grep's will leave you only with INSERT, UPDATE, DELETE statements which is exactly what I was after (Your mileage might vary).

One would use something like this:

tailf /var/log/mysql/query.log | grep -E '[[:space:]]+[[:digit:]]+[[:space:]]Query' | grep -ivE 'Query([[:space:]])+(/\*.*\*/)?(select|set|show|commit|rollback|use)'

The command works by constantly (tailf) pushing events from mysql query.log throw the 2 greps which do some smarty ass filtering to leave you with the interesting part. If you need other command (SELECT for example) you can remove them from the second grep section. If you want to see just a single table add another 3'rd grep to the chain and filter just by table name.

Comments are welcome.

Enjoy,
Maxim.

Eclipse 3.6 Helios - Final version available for download

2010-06-19T13:08:00.001+03:00

I don't know what's so special about this build from eclipse.org foundation but for some reason the excitement is high.

Well friends, without further adieu I give you the path to the final release download location 4 days before the official release schedule

Check it out from here http://mirrors.xmission.com/eclipse/technology/epp/downloads/release/helios/R/

Enjoy.

In the working for automatic User Agent Detecting algorithem

2010-04-01T10:27:00.002+03:00

I'm thinking about writing a tool that will extract information from User-Agent strings.

In the mean time, here is a quite comprehensive list of tools I've been able to find around the net that can assist me in this task

Tone of links for known User Agent strings

DB:
http://www.useragentstring.com/
http://www.user-agents.org/index.shtml?moz
http://www.icehousedesigns.com/useragents/spiderlist.php
http://www.botsvsbrowsers.com/
http://useragentstring.com/pages/useragentstring.php

Auto detection tools:
http://nerds.palmdrive.net/useragent/code.html
http://stackoverflow.com/questions/1005153/auto-detect-mobile-browser-via-user-agent
http://mdbf.codeplex.com/
http://detectmobilebrowser.com/
http://stackoverflow.com/questions/2508214/regexp-that-matches-user-agents-of-end-user-browsers-but-not-crawlers-with-90
http://stackoverflow.com/questions/927552/parsing-http-user-agent-string
http://www.djangosnippets.org/snippets/267/
http://browsers.garykeith.com/downloads.asp
http://user-agent-string.info/
http://pypi.python.org/pypi/httpagentparser
http://www.handsetdetection.com/
http://www.tnl.net/ua/OS/Cross-Platform
http://stackoverflow.com/questions/1005153/auto-detect-mobile-browser-via-user-agent
http://api.jquery.com/jQuery.browser/#jQuery.browser.version2
http://www.quirksmode.org/js/detect.html

INFO:
http://www.nczonline.net/blog/2010/01/12/history-of-the-user-agent-string/

If you know any other resources that can help please post in comments.
I will update this post as soon as I have some news on the subject.

Thank you,
Maxim.

Resetting cached password of Subclipse

2010-01-20T10:47:00.002+02:00

I was asked to document a FAQ about Subclipse and cached passwords (my turn to play with the IT hat!). Here is the email (kept here for future reference).

Hello Friends,

Eclipse subversion plugin (Subclipse) will cache the username & password of the last user that did a successful commit with your eclipse installation.
This can be problematic if this user is not you, it turns out that until further notice all your svn commits will be under different username.

To resolve this issue, delete the following 2 folders:
rm -rf /opt/eclipse3.3/configuration/org.eclipse.core.runtime/.keyring
rm -rf ~/.subversion/
Then restart eclipse.

An alternative solution would be to kindly ask the busy IT department to change the svn password of the user under which your commits are being executed, then on first commit attempt Eclipse should prompt your for username & password, which will allow you to input your information.

Reference: http://nlp.cs.byu.edu/~rah67/wordpress/?p=4

Maxim.

Running Average calculation implementation in Java

2010-01-17T18:45:00.001+02:00

Quick blog and store note:

Here is a code to calculate average series of length N without the problem of overflow that naive average calculation could face.

/**
* Copyright (c) 2010, Maxim Veksler 
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without modification, 
* are permitted provided that the following conditions are met: 
* 
* Redistributions of source code must retain the above copyright notice, this list
* of conditions and the following disclaimer.
* 
* Redistributions in binary form must reproduce the above copyright notice, 
* this list of conditions and the following disclaimer in the documentation
* and/or other materials provided with the distribution. Neither the name
* of the http://bytecoded.blogspot.com/ nor the names of its contributors
* may be used to endorse or promote products derived from this software 
* without specific prior written permission.
* 
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
* THE POSSIBILITY OF SUCH DAMAGE.
*/

/**
* Calculate average of arbitrary length series of doubles, running average ensures 
* result won't overflow.
* 
* Algorithm: CELLCOUNT represents number of cells calculated so far. 
* For each : cell
*  avg =  ((CELLCOUNT-1) * avg) + ((1/CELLCOUNT) * avg)
* 
* @param series vararg list of parameters (array of doubles).
* @return Average calculated from supplied series.
*/
public static double movingAverage(double... series) {
if(series.length < 1) {
   return 0;
  } 

  double avg = series[0];
  for(int i = 1; i < series.length; i++) {
   avg = (avg * ((double)i/(i+1))) + (series[i] * ((double)1/(i+1)));
  }
  
  return avg;
 }

Jasper Reports - Subreport usage demonstration

2009-12-15T13:59:00.007+02:00

This post is part of Jasper Reports demo documentation project, please read that post for information on downloading the discussed demo files

jasperreports-3.7.0/demo/samples/subreport

This demonstration will teach you how to reuse sub-reports for enhancing report visual layout. Subreports among other things allow you to achieve high level of data presentation complexity with relatively small overhead.

a Subreport in Jasper terminology is single jrxml report file which is designed to be included in a master report, the master report should pass parameters to the subreport (For ex. rowId or street address) which are accessed by the subreport using $P{"parameterName"} notation. Optional feature for subreport is return parameters which allow passing information from calculations done within the subreport back to the master report, under the hood these are variables which are defined in the master report and passed to the subreport element (we will see an example of this later in the post).

Lets start with the basics setup required to get the report demo running:

First you will need to start the hsqldb database.

Next open all 3 reports in iReports.

MasterReport.jrxml
ProductReport.jrxml
AddressReport.jrxml

Test the report is working:

In iReports, assuming you followed the hsqlsdb setup guide you should select "jdbc:hsqldb:hsql://localhost" as your datasource.
Make sure you have focus on the MasterReport tab
Now click "Preview"

In the popup windows select "Use Default"

This should bring a semi-working report that looks something like this:

To configure a fully functional we will need to do some adjustment steps, as we are operating the report from iReport instead from Java code.

MasterReport has has a parameter called ProductsSubreport of type net.sf.jasperreports.engine.JasperReport this parameter can be used in production environment to dynamically supply the subreport Class Instance that will be used by one of this subreport elements (more on this later). For now, lets change this to supply a fixed sub report element so that we can preview the full report in iReports as Jasper team designed it.

Locate in iReports the element's:

MasterReport -> Details 1 -> $P{ProductsSubreport} then set the following fields:
- Subreport Expression = "ProductReport.jasper"
- Expression Class = java.lang.String
MasterReport -> Parameters -> ProductsSubreport then set the following fields:
- Use as a prompt = [ ] (unselected)

Now if you try to preview the report again the following will be shown:

Good! The report is fully operation. Lets start analyzing.

Understanding Report logical Structure

This report prints product orders grouped by city. The report layout is as following:

== Report Title =
Heading + Total number of orders
== Page Title ==
Page Heading
== Details ==
= First page =
City name COUNTER: Number of address ordered in this city
= Next n pages =
City name cont

Then 2 lists order side by side displaying:
First list = Products ordered by people in current city
Second list = Address that made orders in current city

== Page Footer ==
Page X out of XXX

Understanding Report JRXML structure

MasterReport.jrxml

Master Report :: Query
The main report is based on the following SQL query: SELECT City FROM Address GROUP BY City ORDER BY City which should self explentionary.

Master Report -> Title
This section contains static text and a graphical horizontal bar.

Pay attention to the Text Field that as part of his expression uses the dynamically calculated variable $V{ProductTotalPrice} this variable is returned from the sub reports called during report filling phase and the results are summarized by the engine. More on this later.

Master Report -> Page Header
Nothing special here. Static text that will be printed at the start of each page.

Master Report -> Details 1

The heart of the report.

It is important to understand that a details band can grow beyond single page size, Jasper has special treatment for these situation via settings which can be set on each element as we shall see next:

On the first page we want to print "CityName" COUNTER Addresses, on the following pages for the same city OTOH all we want to print is "CityName" cont

The idea is that if a city band grows to several pages we will reprint the "CityName" and the text "Cont"

TextField $F{City} has the setting Print Repeated Values and Print When Details Overflows which causes it repeat on each page, even when the band for current city grows beyond 1 page size.

TextField $V{CityAddressCount} + " addresses" has the setting Print Repeated Values which will allow this field to be printed even if 2 cities have the same number of addresses but note this Print When Details Overflows is turned off which causes this text to be printed only on the first page of the band !

Label (continued) has the setting Print Repeated Values turned off and Print When Details Overflows enabled. This cause this label to be printed only on overflowed bands (meaning only on 2nd pages and following...).

Subreport $P{ProductsSubreport} is the embedding of the ProductReport subreport. Note this report gets the Database connection from the MasterReport, we also pass a parameter to this report City = $F{City}. This report returns a variable, the way this is implement in Jasper Reports is as following: We define a variable in the master report ProductTotalPrice, then we pass this variable to the subreport which sets values into this variable. Then, after the subreport evaluation has ended in the MasterReport we can use this variables value. In the present context we use this method to calculate a summary (TOTAL) of the price of all the products being evaluated by the subreport. Note that as part of the definition of the variable on the subreport element we configure Calculation Type = Sum this allows the engine to do the summary of this variable (which is passed to many subreport's - A subreport is evaluated for each city). See Variables section for another note on the ProductTotalPrice variable.

Master Report -> Variables

ProductTotalPrice this variable is used by the MasterReport to display the sum of the price of all products that have been evaluated by the subreport ProductReport note this variable has the property Calculation = System meaning that it is being manipulated separately (not by the engine). This is actually a little hack (IMHO) because we do sum it using the engine only we do it on the Subreport $P{ProductsSubreport} variable configuration properties. Just in case you wondered: Yes! it works.

ProductReport.jrxml

To be continued...

AddressReport.jrxml

To be continued...

Jasper Reports hsqldb database setup

2009-12-14T18:44:00.019+02:00

This post is part of Jasper Reports demo documentation project, please read that post for information on downloading the discussed demo files

Jasper Report distribution comes with a sample database populated with data which proves to be as useful as it can be for running the supplied demo reports / testing concepts before implementing in production environments.

To use this database you will need to

Have it up and running
Configure the connection in iReports

On a linux box this is straight forward as explained below (Win/OSX and others should have similar guide lines).

Have HSqlDB it up and running

// Demonstrated for iReports, JasperReports v3.7.0

Open up a terminal windows and issue the following commands:

$ cd JasperReports/jasperreports-3.7.0/demo/hsqldb/
$ ant

You will see output similar to this

Buildfile: build.xml

runServer:
[java] [Server@15c7850]: [Thread[main,5,main]]: checkRunning(false) entered
[java] [Server@15c7850]: [Thread[main,5,main]]: checkRunning(false) exited
[java] [Server@15c7850]: Startup sequence initiated from main() method
[java] [Server@15c7850]: Loaded properties from [/home/maximv/Desktop/JasperReports/jasperreports-3.7.0/demo/hsqldb/server.properties]
[java] [Server@15c7850]: Initiating startup sequence...
[java] [Server@15c7850]: Server socket opened successfully in 4 ms.
[java] [Server@15c7850]: Database [index=0, id=0, db=file:test, alias=] opened sucessfully in 607 ms.
[java] [Server@15c7850]: Startup sequence completed in 612 ms.
[java] [Server@15c7850]: 2009-12-14 18:32:33.542 HSQLDB server 1.8.0 is online
[java] [Server@15c7850]: To close normally, connect and execute SHUTDOWN SQL
[java] [Server@15c7850]: From command line, use [Ctrl]+[C] to abort abruptly

That's it. Keep the terminal window open otherwise the DB will shutdown. The database can be accessed locally using the

Path: jdbc:hsqldb:hsql://localhost
Username: sa
Password: "" (blank, no password).

Configure the HSqlDB connection in iReports

In iReports the setup is as follows:

Click the new data source connection button:
Click the new connection
Configure connection to the hsqldb datasource

That is

JDBC Driver = HSQLDB (server) (org.hsqldb.jdbcDriver)
JDBC URL = jdbc:hsqldb:hsql://localhost

Username = sa
Password = ([V] Save password)
Test the connection is working.

After configuring the connection you should click the "Test" button, if everything goes well the following popup windows will confirm this.

Jasper Reports demo documentation project

2009-12-14T18:14:00.011+02:00

Jasper Reports is one of several well known and well established open source reporting frameworks.

One thing it lacks badly is good documentation, I believe it's a safe claim that anything beyond the regular most basic report structure will throw you straight into googling, example files and forum posts. iReport will help, documentation will be somewhat helpful as well, but do be prepared for the moment when even these paths fail, that's when you will stand at the face of either commercial support or diving into the code of the engine.

Downloading the demo jrxml files

To help the community a little bit, I'm starting a little documentation project for the examples supplied with Jasper Reports source distribution which can be obtained by checking out the source code or by downloading the latest dist from here. All this in the hope to lower the learning curve and allow easier adoption by casual report dev folks.

Several more source of information for Jasper Report Developers

JasperReports 3.7.0 - Schema Reference
Designing reports using JasperReports 3.5
Generally speaking StackOverflow jasper-reports tag

Jasper Reports Demo files Index

Written guides are hyper-linked - This is work in progress I will be editing this post as more guides are published.

[maximv@mldev2 JasperReports]$ find jasperreports-3.7.0/demo/samples/ -maxdepth 1 -mindepth 1 -type d | sort

jasperreports-3.7.0/demo/samples/alterdesign
jasperreports-3.7.0/demo/samples/antcompile
jasperreports-3.7.0/demo/samples/barbecue
jasperreports-3.7.0/demo/samples/barcode4j
jasperreports-3.7.0/demo/samples/batchexport
jasperreports-3.7.0/demo/samples/beanshell
jasperreports-3.7.0/demo/samples/charts
jasperreports-3.7.0/demo/samples/chartthemes
jasperreports-3.7.0/demo/samples/crosstabs
jasperreports-3.7.0/demo/samples/csvdatasource
jasperreports-3.7.0/demo/samples/datasource
jasperreports-3.7.0/demo/samples/ejbql
jasperreports-3.7.0/demo/samples/fonts
jasperreports-3.7.0/demo/samples/genericelement
jasperreports-3.7.0/demo/samples/groovy
jasperreports-3.7.0/demo/samples/hibernate
jasperreports-3.7.0/demo/samples/horizontal
jasperreports-3.7.0/demo/samples/hyperlink
jasperreports-3.7.0/demo/samples/i18n
jasperreports-3.7.0/demo/samples/images
jasperreports-3.7.0/demo/samples/jasper
jasperreports-3.7.0/demo/samples/java1.5
jasperreports-3.7.0/demo/samples/javascript
jasperreports-3.7.0/demo/samples/jcharts
jasperreports-3.7.0/demo/samples/jchartscomponent
jasperreports-3.7.0/demo/samples/jexcelapi
jasperreports-3.7.0/demo/samples/jfreechart
jasperreports-3.7.0/demo/samples/landscape
jasperreports-3.7.0/demo/samples/list
jasperreports-3.7.0/demo/samples/markup
jasperreports-3.7.0/demo/samples/mondrian
jasperreports-3.7.0/demo/samples/nopagebreak
jasperreports-3.7.0/demo/samples/noreport
jasperreports-3.7.0/demo/samples/noxmldesign
jasperreports-3.7.0/demo/samples/openflashchart
jasperreports-3.7.0/demo/samples/pdfencrypt
jasperreports-3.7.0/demo/samples/poi-3.5
jasperreports-3.7.0/demo/samples/printservice
jasperreports-3.7.0/demo/samples/query
jasperreports-3.7.0/demo/samples/rotation
jasperreports-3.7.0/demo/samples/scriptlet
jasperreports-3.7.0/demo/samples/shapes
jasperreports-3.7.0/demo/samples/stretch
jasperreports-3.7.0/demo/samples/styledtext
jasperreports-3.7.0/demo/samples/subreport
jasperreports-3.7.0/demo/samples/table
jasperreports-3.7.0/demo/samples/tableofcontents
jasperreports-3.7.0/demo/samples/templates
jasperreports-3.7.0/demo/samples/text
jasperreports-3.7.0/demo/samples/unicode
jasperreports-3.7.0/demo/samples/virtualizer
jasperreports-3.7.0/demo/samples/webapp
jasperreports-3.7.0/demo/samples/xlsdatasource
jasperreports-3.7.0/demo/samples/xlsformula
jasperreports-3.7.0/demo/samples/xmldatasource

Blog Template Hacking

2009-01-31T13:55:00.008+02:00

I will probably be hacking the blog template from time to time, you can find the source of it hosted at this google code repository.

For those few that are not familiar with concept of Revision Control I will describe briefly why I choose to use it here, which should also explain the general idea behind the concept.

I will be hacking the template of the blog, this means I will be modifying the "code" of my blog, the reason I want to do this is simple: It affects the way you see it. This "code" is parsed by blogger backend server and rendered into HTML which is later parsed by your browser and displayed to your eyes (which by the way goes another parsing and interpretation by your brain before you can actually "see"/ "read" my blog). Parsing is the process of "understanding" a known format.

Now, due to the known fact that there is no software without bugs. Hacking surly means introducing bugs, considering the fact that I've never edited blogger templates before- The bug factor is even higher. Further more, bugs are here forever or as the saying goes "To err is human, to fix it divine" (A remix of Alexander Pope famous quote "To err is human, to forgive divine" by your humble).

To tackle this bug fixing process in a more profound process, one of the best techniques is going back step by step attempting to isolate what exactly caused the wrong behavior (the bug). This is where revision control system comes in- It allows you to retrieve older versions of the file, further more it allows you to display a difference between 2 versions of the same file in an easily accessible interface. This fits into the previously introduced metaphor of bug tackling: start from something that worked, search revision by revision until you find what broke it, understand why it's broken, fix the problem, ensure the fix works. Some folks forgot the import step of Learn from the error and think how to avoid it next time.

You can open your own Google code repository at the following link Google Code, as for document revisioning, you can do the same thing using Google Docs which allows out of the box "History" of your document modifications.

TheServerSide bytecode article summary

2009-01-29T01:43:00.017+02:00

Lets summarize what can be learned from the really brilliant TheServerSide article "The working developer's Guide to Java Bytecode"

The JVM is open to host different languages among them JRuby, Groovy and Jython all these translate to JVM bytecode for runtime. Java is translated in exactly the same process to bytecode using javac.
Bytecode is the assembly language of the JVM, you might as well think of the JVM as a globally available instruction set.
The translation of bytecode to human readable form is encouraged by Sun. Fact is Sun provides a tool called javap that allows you to do just that in an easily accessible way.
Bytecode works by calling instructions with their relevant data, for example a reference to a memory location that contains a string. What's interesting about this idea is that methods are also called by string reference- so for example if your default code is designed to call a method by the instruction invokevirtual #4 you could very replace that with invokevirtual #7 at runtime, this opens a huge range of possibilities for the application.
Inside the JVM the stack is divided into slots, each slot size is 4bytes (32bits).
The size of a bytecode instruction is 1 byte. Which makes this the simplest assembly language in the world. In fact the grand total of bytecode instructions existing is 224!
The method by which the JVM passes data between instructions is by putting and popping data into the stack. The calling context, before instructing the JVM to execute a bytecode instruction loads data into the stack, the instruction in turn once called pops data out of the stack. The amount of loaded data by the caller must match the amount that would be popped otherwise we have a Buffer Overflow, something that does not exist in the JVM (As far as I know).

I've just covered the first section of the article, you are encouraged to read the whole thing. Please by all means don't retain yourself from commenting with ideas and requests regarding additional covering you would like to read. Also, I should mention that people wishing to participate in the writing of this blog are welcome to do.

Please contact me at maxim.veksler@gmail.com for further discussion.

Bytecode Learning Path

2009-01-25T01:55:00.006+02:00

Continuing with the learning path, A good knowledge basis must be built.

Lets start with a few relevant reading material :

First wikipedia article explaining what java bytecode is all about.
The actual specification of Java bytecode, kindly engineered by Sun.
The working developer's Guide to Java Bytecode, an article by TheServerSide.
IBM article discussing Java byte and why you need it.
Bytecode Basics the article at JavaWorld that was the trigger for my passion.
A JVM Language Summit collection of presentations from InfoQ.

Continuing with some books that cover this topic from different angles. I hope that in due time I would be able to lay a better review about them... after I read them.

And here are some projects that deal with Java bytecode manipulation

These should give you a good starting point in exploring the Byte coded wonders of the JVM. Welcome to the world of "Real Programmers".

Last but not least, a link to some more tools and related artifacts for folks looking to toy with Bytecode http://www.artima.com/insidejvm/resources/index.html

Java Byte Code Assembly

2009-01-25T01:40:00.002+02:00

I have decided to learn how to write Java bytecode, this issue fascinates me.

I think it's amazing to be able to understand what goes under the hood of the code, this will allow me to see a much bigger picture of the Java world. Also, with knowledge comes power - Being able to read assembly means being able to write it... and that is all I agree to say about this. For more information ask phrack.org.

This blog will document and hopefully share with you my dear reader interesting ideas, projects and hacks I learn along the way.

Enjoy, have fun and be nice!

Maxim Veksler.