Image Component Library (ICL)
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines
Public Member Functions | Private Attributes
icl::filter::ColorSegmentationOp Class Reference

Class for fast LUT-based color segmentation. More...

#include <ColorSegmentationOp.h>

Inheritance diagram for icl::filter::ColorSegmentationOp:
icl::filter::UnaryOp icl::utils::Configurable

List of all members.

Public Member Functions

 ColorSegmentationOp (icl8u c0shift=2, icl8u c1shift=2, icl8u c2shift=2, core::format fmt=core::formatYUV) throw (utils::ICLException)
 Creates a new instance of this class with given segmentation parameters.
 ~ColorSegmentationOp ()
 Destructor.
virtual void apply (const core::ImgBase *src, core::ImgBase **dst)
 main apply function
icl8u classifyPixel (icl8u r, icl8u g, icl8u b)
 classifies a pixel in given rgb format
void clearLUT (icl8u value=0)
 clears the whole segmentation LUT
void setSegmentationShifts (icl8u c0shift, icl8u c1shift, icl8u c2shift)
 sets the segmentation shifts
void setSegmentationFormat (core::format fmt) throw (utils::ICLException)
 sets the internally used segmentation format
const icl8ugetSegmentationShifts () const
 returns the pointer to the 3 internally used segmentation shifts
core::format getSegmentationFormat () const
 returns the current internally used segmentation format
const core::Img8ugetSegmentationPreview ()
 returns in internally managed image of the segmentation result
void lutEntry (icl8u a, icl8u b, icl8u c, icl8u rA, icl8u rB, icl8u rC, icl8u value)
 given a,b and c in segFormat, this function fills the LUT within a sub-volume with given radii
void lutEntry (core::format fmt, int a, int b, int c, int rA, int rB, int rC, icl8u value) throw (utils::ICLException)
 given a,b and c in format fmt, this function fills the LUT within a sub-volume with given radii
void load (const std::string &filename)
 loads the segmentation LUT only (no other parameters)
void save (const std::string &filename)
 saves the segmentation LUT only (no other parameters)
const core::Img8ugetLUTPreview (int xDim, int yDim, icl8u zValue)
 this also creates a preview for the current segmentation LUT
const core::Img8ugetColoredLUTPreview (int xDim, int yDim, icl8u zValue)
 this also creates a colored preview for the current segmentation LUT
const std::vector< core::Color > & getClassMeanColors ()
 returns a mean color for all used class labels (in rgb format)
const icl8ugetLUT () const
 returs the internal lut data
icl8ugetLUT ()
 returns the internal lut data
void getLUTDims (int &w, int &h, int &t) const
 returns the lut-sizes

Private Attributes

core::format m_segFormat
 format, that is used for internal segmentation
core::Img8u m_inputBuffer
 internal image in depth8u and segmentation format
core::Img8u m_outputBuffer
 internal buffer holding the output image
core::Img8u m_segPreview
 internal buffer for providing a preview of the current segmentation
core::Img8u m_lastDst
 last used destination image
icl8u m_bitShifts [3]
 bit shifts for all 8-Bit channels
LUT3D * m_lut
 color classification lookup table

Detailed Description

Class for fast LUT-based color segmentation.

General Information

Color segmentation is a very common issue in computer vision applications. The ColorSegmentationOp class implements a common and very efficient LUT based segmentation algorithm. It can be used to apply any possible segmentation where the classification of one pixel is only influenced by that pixels color value

The Segmentation Algorithm

Please consider an input color image I. In a first step, I is converted into depth8u and the specified segmentation-color-format (this can be set in the constructor or with the corresponding setter-function ColorSegmentationOp::setSegmentationFormat). This conversion is skipped if the given image parameters are already correct. Now, every pixel pI = (A,B,C) (e.g. if the segmentation format is formatRGB, pI=(R,G,B)) is classified by applying a lookup transformation on it. Basically, we use a lookup table of the whole segmentation color space for this. This implies that size of the lookup table becomes 256*256*256. In order to reduce the amount of cache misses when picking elements of this 16MB LUT, it is possible to adjust the number of bits that are used for each color channel. In the example above, 8 bits (256 values) are used. If the number of bits are shifted (using the constructors channel-shift parameters or the setter function ColorSegmentationOp::setSegmentationShifts), the size of the lookup table becomes smaller and therefore, the lookup operation becomes faster (see Benchmarks). Another advantage of using the channel-shifts in order to reduce the bits used to represent a certain image channel is that is also provides a better generalization. Consider the following example (see Example)

Example

Segmentation format: YUV, Shifts = (8,0,0).
In this case, the Y (brightness) information is not used for segmentation at all. Therefore the pixel classification does not depend on the pixels brightness, which is very useful for color-segmentation. The size of the used lookup table becomes 256*256 which is 65KB. As the noise level of common cameras is still quite high, it might also be possible to use only the 7 (or even only the 6) most significant bits for the two color channels U and V without a significant reduction of the segmentation quality.

How to Define the Segmentation Lookup Table

Another important question that has not been answered yet is, the lookup table can be set in order to achieve a certain segmentation result. Consider you want to segment a yellow ball in front of a black background. Now, you simply have to set the lookup table values for all possible colors of this ball to a value that is different from 0 (so maybe 200). Then the segmentation result will be white where the ball was, and black (value 0) otherwise. But how can you set up the lookup table values? Of course, it might be quite hard to find all possible colors of the ball, but it might be easy to pick some of its color e.g. by mouse. If you once have these color prototype pixels, you can now use one of the the ColorSegmentationOp::lutEntry functions in order to set up all corresponding lookup table entries to your class label (which was 200 in the example above). The lutEntry methods also provide the functionality to not only set up the prototype-pixel's class-labels, but also the class-lables of all pixels within the vicinity of the prototype pixels. So sometimes, if your 'to-be-segmented'-yellow ball is quite homogeneously yellow, it can be sufficient to add only a single average yellow prototype pixel with high radii for the single color components.
If you don't want to implement mouse handling in order to be able to click at your objects to get color-prototypes, you can also use the ICL-example tool 'icl-color-picker'.

Using more than one Class

By adding color prototypes for different objects, but with different class labels, you can also use this class to segment several objects at once. In the next step, you will possibly use an instance of ICLCV/RegionDetector to extract the segemented image regions. Here, you can then obtain the corresponding class label by using the ImageRegion's getVal method. Please note that always the last lut-entry is used if your prototype-entries overlap.

Restrictions

Maybe it is now, if not before necessary to mention, that the ColorSegmentationOp can only be set up to classify pixels into 255 valid classes. But actually, this should not become a problem at all as the classification quality usually restricts the number of classes to a maximum of about 10.

Benchmarks


Constructor & Destructor Documentation

Creates a new instance of this class with given segmentation parameters.

Parameters:
c0shiftnumber of least significant bits that are removed from channel0
c1shiftnumber of least significant bits that are removed from channel1
c2shiftnumber of least significant bits that are removed from channel2
fmtinternally used segmentation format (needs to have 3 channels)

Destructor.


Member Function Documentation

virtual void icl::filter::ColorSegmentationOp::apply ( const core::ImgBase src,
core::ImgBase **  dst 
) [virtual]

main apply function

Implements icl::filter::UnaryOp.

classifies a pixel in given rgb format

clears the whole segmentation LUT

returns a mean color for all used class labels (in rgb format)

const core::Img8u& icl::filter::ColorSegmentationOp::getColoredLUTPreview ( int  xDim,
int  yDim,
icl8u  zValue 
)

this also creates a colored preview for the current segmentation LUT

the method behaves like getLUTPreview, except, it tints the resulting lut-slice with mean colors of that class

returs the internal lut data

The data order is depth-major row-major i.e. an index(x,y,z) is estimated by (x + w*y + w*h * z)

returns the internal lut data

Please be careful with this method :-)

void icl::filter::ColorSegmentationOp::getLUTDims ( int &  w,
int &  h,
int &  t 
) const

returns the lut-sizes

w = 1+(0xff >> bitShift[0]) etc.

const core::Img8u& icl::filter::ColorSegmentationOp::getLUTPreview ( int  xDim,
int  yDim,
icl8u  zValue 
)

this also creates a preview for the current segmentation LUT

Here, you can choose, which dimension shall become the resulting images with and height. The remaining 3rd dimension is slices with given zValue

returns the current internally used segmentation format

returns in internally managed image of the segmentation result

The image has the dimensions 2^(8-shift0) x 2^(8-shift1) and it has 2^(8-shift3) channels

returns the pointer to the 3 internally used segmentation shifts

void icl::filter::ColorSegmentationOp::load ( const std::string &  filename)

loads the segmentation LUT only (no other parameters)

void icl::filter::ColorSegmentationOp::lutEntry ( icl8u  a,
icl8u  b,
icl8u  c,
icl8u  rA,
icl8u  rB,
icl8u  rC,
icl8u  value 
)

given a,b and c in segFormat, this function fills the LUT within a sub-volume with given radii

void icl::filter::ColorSegmentationOp::lutEntry ( core::format  fmt,
int  a,
int  b,
int  c,
int  rA,
int  rB,
int  rC,
icl8u  value 
) throw (utils::ICLException)

given a,b and c in format fmt, this function fills the LUT within a sub-volume with given radii

void icl::filter::ColorSegmentationOp::save ( const std::string &  filename)

saves the segmentation LUT only (no other parameters)

sets the internally used segmentation format

void icl::filter::ColorSegmentationOp::setSegmentationShifts ( icl8u  c0shift,
icl8u  c1shift,
icl8u  c2shift 
)

sets the segmentation shifts


Member Data Documentation

bit shifts for all 8-Bit channels

internal image in depth8u and segmentation format

last used destination image

color classification lookup table

internal buffer holding the output image

format, that is used for internal segmentation

internal buffer for providing a preview of the current segmentation


The documentation for this class was generated from the following file:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines