Displaying a Videos Sequence on a Java Component and what’s the deal with SwingUtilities.invokeLater()?

October 21st, 2011

I kind of struggled a bit with displaying a video sequence on top of a self-written Java component based on Swing and Graphics2D. Let’s say we have some VideoDecoder with a public interface such as:

public class VideoDecoder {

    public VideoDecoder(VideoPanel panel, String videoFile) { ... }
    public void run() { ... }
}

So the VideoDecoders run() method happily decodes the video sequence and pushes the frames into the VideoPanel which then simply displays the frame. The most simple version of such a VideoPanel is probably:

public class VideoPanel extends JPanel
{
    private Image image;

    @Override
    public void paint(Graphics g)
    {
         if(this.image != null) {
            g.drawImage(this.image, 0, 0, null);
        }
    }

    public void setImage(Image image) {
        this.image = image;
        repaint();
    }
}

So once it comes to interaction with GUI components you usually do not want to call a method like run() which blocks until it is finished from some Action Handler to avoid freezing your GUI. So I initially came up with some code that I kind of borrowed from my GUI designer:

            SwingUtilities.invokeLater(new Runnable()
            {
                @Override
                public void run()
                {
                    VideoDecoder decoder = new VideoDecoder(videoPanel, filename);
                    decoder.run();
                }
            });

So the thing with SwingUtilities.invokeLater() is, that we actually queue a job that will be run by swing on the event dispatch thread. So once our job gets started it hangs the GUI because they share the same thread. Therefore there will be no update on the video panel until the decoder is done.

Simple solution, run the decoder as a separate thread. Change the class definition to:

public class VideoDecoder extends Thread { ... }

and modifiy the Action Handler to:

VideoDecoder decoder = new VideoDecoder(videoPanel, filename);
decoder.start();

Of course you have to take care of not creating multiple instances of the decoder or other ways of handling asynchronous data access to the video panel.


Lesson Learned:

Do not use SwingUtilities.invokeLater() if your job either blocks, or takes up a significant amount of time to complete. Instead run it on a separate thread and save yourself a lot of trouble!

NaN, Not anything Nice

October 14th, 2011

So I end up two days searching for some strange bug changing the results of my classification code randomly. The results look valid but change on multiple calls. I expect a race condition and keep checking my multi-threaded code, lots of fun.

I finally end up with a piece of that normalizes a histogram and find that I do not check the
divisor of a quotient for 0. NaN showing up in my further computation leading to unexpected results.

As long as we are working with integers types, most languages will complaint with an exception but
as soon as floating types are used the NaN and Infinty are included in the types and are handled
by the hardware as such, no complaints!!

Some things to know about our enemy are:

Double.NaN == Double.NaN // false
Doube.NaN != Double.NaN // true
Double.NaN - Double.NaN = Double.NaN
Double.Nan - 0 = Double.NaN

So NaN is not NaN and NaN is NaN. Take care as NaN does not respect math rules (yeah, it’s not a number :))

The Return of The Blog

October 13th, 2011

First of all, thank you for all comments and emails we received regarding project bamby. I think an apology from our side is due at this point. We both have been very busy getting into our new jobs. Hence there was no blogging done.

As project bamby was part of our research at the university our main interest was to build a proof of concept code regarding our Registration and Decoding algorithms. That’s why we are currently putting no more effort into this project. Nevertheless we have quite a few cool ideas regarding the barcode detection and decoding that might find their way into an implementation at a later point. Im thinking a lot about wavelets here.

So what’s planned to be covered inside the lemqi blog anyways you might ask ? Easy answer: “Whatever comes to be interesting in the world of computer science.”

Nevertheless this blog is not planned to be of entirely technical format. We might also blog about funny stories happening in our work or just mention our thoughts about certain facts in the field of computer science.

So long!

Optimizing for speed

November 14th, 2008

As we are quite confident with the recognition rate, we’re doing some refactoring and optimization now. Because of the raw processing power modern GPU’s offer, we’re experimenting with HLSL Shaders for image processing.

The first tests showed a significant performance boost compared to similar things done on cpu.
Altough we’re just evaluating this method, we think that we can improve recognition performance dramatically.

Below a screenshot of a sobel filter implemented in a Pixel Shader applied to a 6 megapixel image:

Little comparison of barcode recognition components

November 10th, 2008

to get a little impression on the recognition quality of our barcode library (called Project Bamby) we decided to compare the recognition rate of several publicly available commercial and non-commercial barcode recognition tools/libraries.

For evaluation we’ve chosen 30 different sample images with variable recognition difficulty and tested wheter the different components are able to locate and decode the barcode.
These images can be viewed by clicking on the image number on the left side of the table below.

To save space we denote the different components with letters from A to K whereas

Image A B C D E F G H I J
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

And finally a small chart that shows an overall recognition rate comparison including false positives.

The results may be a little bit biased towards Project Bamby as some of the pictures we used to compare are the same used while developing. Therefore this library may have a little advantage (although we cannot imagine very different images that may not be recognized).

Thresholding for fun and profit!

November 10th, 2008

Thresholding is one of the vital steps in many preprocessing scenarios. The idea is to distinguish between foreground and background pixels to extract information. The big problem is to decide a threshold for each pixel without losing information or introducing noise. We have tested a couple of thresholding methods for our project within the spatial domain. We tested thresholding algorithms that were proposed within the domain of character recognition.

Local Thresholding

Local Threshdolding is a class of thresholding algorithms that calculate a single threshold for every pixel within an image. The advantage is that uneven illumination can be handled better than within a global thresholding algorithm. The idea is that a certain pixel threshold is calculated in respect of the pixel statistics surrounding the pixel of interest.  The boundary is usually called a local window.

One problem when using local thresholding is to find a good size for this local window. It should span 1-2 characters within OCR applications to be efficient. This is not so easy with barcode recognition. As we do not know (and do not want) the scaling of the image, and hence the barcode size, we can not guarantee that the local window spans foreground and background pixels. Because the pixel thresholds are calculated in respect of the statistics of surrounding pixels, light barcode pixels might be classified as background pixels if the local windows is too small (covering only foreground (barcode) pixels). This results in a hole within the barcode line, which certainly is not wanted.

Global Thresholding

Global thresholding algorithms calculate a common thresholding value for every pixel within the image. Uneven illumination can introduce noise and false pixel classifications. Images of bad quality (through glare or blur) introduce some unwanted effects as well. So global binarization algorithms do not seem to be a very good idea on the first impression (related to barcode localization). But what if we use the binarization to classify between edge and none edge pixels? This of course involves some way of edge detection (like using the sobel operator). Or course the edge detection operator might be fooled by glare. But it will only detect the edges of the glarred spot and introduce a much smaller error. The thresholding then does not classify between foreground or background pixels, but between edge and non edge pixels.

Summary

Local thresholding can handle bad quality images better than Global thresholding but it’s tough to decide a good local window size. If the window is too small barcode pixels might get lost, if the window is too big the algorithm speed decreases as well as the threshold qualities. A good tradeoff is to use global thresholding on a preprocessed images to classify between edge and non-edge pixels.

Local Binarization 1

Local Binarization on grayalue Image

Local Binarization 2

Local Binarization on grayvalue Image

Global Binarization of Edge Image

Global Binarization on Edge Image

Barcodes here, Barcodes there …

November 3rd, 2008

… Barcodes everywhere.

Sorry about the long gap since the last post, summer break and a new very interesseting research project took some of the spare time. I hope i will be able to give some more info about the Robot Kinematics as well as some Java3D Gotchas (and annoying library issues) soon :)

As mentioned before, our current (almost done) research project is about Barcode Localization and Decoding. We developed a very stable localization method using classical image processing methods as well as applied statistics. Our decoding uses a very robust method for line classification using paramterizable non-linear membership functions.

We plan to improve to optimize the code further and run a study comparing the effectiveness of public available barcode recognition software. Later we will cover some topics of interest relating to the development of this methods.

A few Screenshots of the current version:

Decoded Image without Localization Visualization Decoded Image with Localization Information
Decoded Barcode without Localization Visualization Decoded Image with Localization Information

Yet Another Way to say Hello World!

May 28th, 2008

#define t typedef
main( void )
{extern puts;t char *a;t a*b;t b*c;t c*d;t int(*q)(const a);t int
i;q n;t char o;i f;a z;b y;c x;d w;d v;o e;
f^=f;n=(q)&puts;z=&e;*z=f;*(z+--f)=0x64;*(z+--f)=*(z+f+1)+0x8;
*(z+--f)=*(z+f+1)+0x6;*(z+--f)=*(z+f+1)-0x3;
*(z+--f)=*(z+f+1)+0x8;*(z+--f)=*(z+f+1)-87;
*(z+--f)=*(z+f+1)+0xc;*(z+--f)=*(z+f+1)+0x43;
*(z+--f)=*(z+f+1)-0x3;*(z+--f)=*(z+f+1)+0x0;
*(z+--f)=*(z+f+1)-0x7;*(z+--f)=*(z+f+1)+0x3;
z=((&e)+f);y=&z;x=(c)*y;w=&x;v=(d)*w;n((a)v);
}

LemQi Blog goes online

May 28th, 2008

The first “test” version of the new LemQi Blog is online. There are still a lotof design issues to make.

This post ist actually just a test of the management system of this blog engine.

Have a lot of fun!