Image Component Library (ICL)
|
Class for fast LUT-based color segmentation. More...
#include <ColorSegmentationOp.h>
Public Member Functions | |
ColorSegmentationOp (icl8u c0shift=2, icl8u c1shift=2, icl8u c2shift=2, core::format fmt=core::formatYUV) throw (utils::ICLException) | |
Creates a new instance of this class with given segmentation parameters. | |
~ColorSegmentationOp () | |
Destructor. | |
virtual void | apply (const core::ImgBase *src, core::ImgBase **dst) |
main apply function | |
icl8u | classifyPixel (icl8u r, icl8u g, icl8u b) |
classifies a pixel in given rgb format | |
void | clearLUT (icl8u value=0) |
clears the whole segmentation LUT | |
void | setSegmentationShifts (icl8u c0shift, icl8u c1shift, icl8u c2shift) |
sets the segmentation shifts | |
void | setSegmentationFormat (core::format fmt) throw (utils::ICLException) |
sets the internally used segmentation format | |
const icl8u * | getSegmentationShifts () const |
returns the pointer to the 3 internally used segmentation shifts | |
core::format | getSegmentationFormat () const |
returns the current internally used segmentation format | |
const core::Img8u & | getSegmentationPreview () |
returns in internally managed image of the segmentation result | |
void | lutEntry (icl8u a, icl8u b, icl8u c, icl8u rA, icl8u rB, icl8u rC, icl8u value) |
given a,b and c in segFormat, this function fills the LUT within a sub-volume with given radii | |
void | lutEntry (core::format fmt, int a, int b, int c, int rA, int rB, int rC, icl8u value) throw (utils::ICLException) |
given a,b and c in format fmt, this function fills the LUT within a sub-volume with given radii | |
void | load (const std::string &filename) |
loads the segmentation LUT only (no other parameters) | |
void | save (const std::string &filename) |
saves the segmentation LUT only (no other parameters) | |
const core::Img8u & | getLUTPreview (int xDim, int yDim, icl8u zValue) |
this also creates a preview for the current segmentation LUT | |
const core::Img8u & | getColoredLUTPreview (int xDim, int yDim, icl8u zValue) |
this also creates a colored preview for the current segmentation LUT | |
const std::vector< core::Color > & | getClassMeanColors () |
returns a mean color for all used class labels (in rgb format) | |
const icl8u * | getLUT () const |
returs the internal lut data | |
icl8u * | getLUT () |
returns the internal lut data | |
void | getLUTDims (int &w, int &h, int &t) const |
returns the lut-sizes | |
Private Attributes | |
core::format | m_segFormat |
format, that is used for internal segmentation | |
core::Img8u | m_inputBuffer |
internal image in depth8u and segmentation format | |
core::Img8u | m_outputBuffer |
internal buffer holding the output image | |
core::Img8u | m_segPreview |
internal buffer for providing a preview of the current segmentation | |
core::Img8u | m_lastDst |
last used destination image | |
icl8u | m_bitShifts [3] |
bit shifts for all 8-Bit channels | |
LUT3D * | m_lut |
color classification lookup table |
Class for fast LUT-based color segmentation.
Color segmentation is a very common issue in computer vision applications. The ColorSegmentationOp class implements a common and very efficient LUT based segmentation algorithm. It can be used to apply any possible segmentation where the classification of one pixel is only influenced by that pixels color value
Please consider an input color image I. In a first step, I is converted into depth8u and the specified segmentation-color-format (this can be set in the constructor or with the corresponding setter-function ColorSegmentationOp::setSegmentationFormat). This conversion is skipped if the given image parameters are already correct. Now, every pixel pI = (A,B,C) (e.g. if the segmentation format is formatRGB, pI=(R,G,B)) is classified by applying a lookup transformation on it. Basically, we use a lookup table of the whole segmentation color space for this. This implies that size of the lookup table becomes 256*256*256. In order to reduce the amount of cache misses when picking elements of this 16MB LUT, it is possible to adjust the number of bits that are used for each color channel. In the example above, 8 bits (256 values) are used. If the number of bits are shifted (using the constructors channel-shift parameters or the setter function ColorSegmentationOp::setSegmentationShifts), the size of the lookup table becomes smaller and therefore, the lookup operation becomes faster (see Benchmarks). Another advantage of using the channel-shifts in order to reduce the bits used to represent a certain image channel is that is also provides a better generalization. Consider the following example (see Example)
Segmentation format: YUV, Shifts = (8,0,0).
In this case, the Y (brightness) information is not used for segmentation at all. Therefore the pixel classification does not depend on the pixels brightness, which is very useful for color-segmentation. The size of the used lookup table becomes 256*256 which is 65KB. As the noise level of common cameras is still quite high, it might also be possible to use only the 7 (or even only the 6) most significant bits for the two color channels U and V without a significant reduction of the segmentation quality.
Another important question that has not been answered yet is, the lookup table can be set in order to achieve a certain segmentation result. Consider you want to segment a yellow ball in front of a black background. Now, you simply have to set the lookup table values for all possible colors of this ball to a value that is different from 0 (so maybe 200). Then the segmentation result will be white where the ball was, and black (value 0) otherwise. But how can you set up the lookup table values? Of course, it might be quite hard to find all possible colors of the ball, but it might be easy to pick some of its color e.g. by mouse. If you once have these color prototype pixels, you can now use one of the the ColorSegmentationOp::lutEntry functions in order to set up all corresponding lookup table entries to your class label (which was 200 in the example above). The lutEntry methods also provide the functionality to not only set up the prototype-pixel's class-labels, but also the class-lables of all pixels within the vicinity of the prototype pixels. So sometimes, if your 'to-be-segmented'-yellow ball is quite homogeneously yellow, it can be sufficient to add only a single average yellow prototype pixel with high radii for the single color components.
If you don't want to implement mouse handling in order to be able to click at your objects to get color-prototypes, you can also use the ICL-example tool 'icl-color-picker'.
By adding color prototypes for different objects, but with different class labels, you can also use this class to segment several objects at once. In the next step, you will possibly use an instance of ICLCV/RegionDetector to extract the segemented image regions. Here, you can then obtain the corresponding class label by using the ImageRegion's getVal method. Please note that always the last lut-entry is used if your prototype-entries overlap.
Maybe it is now, if not before necessary to mention, that the ColorSegmentationOp can only be set up to classify pixels into 255 valid classes. But actually, this should not become a problem at all as the classification quality usually restricts the number of classes to a maximum of about 10.
icl::filter::ColorSegmentationOp::ColorSegmentationOp | ( | icl8u | c0shift = 2 , |
icl8u | c1shift = 2 , |
||
icl8u | c2shift = 2 , |
||
core::format | fmt = core::formatYUV |
||
) | throw (utils::ICLException) |
Creates a new instance of this class with given segmentation parameters.
c0shift | number of least significant bits that are removed from channel0 |
c1shift | number of least significant bits that are removed from channel1 |
c2shift | number of least significant bits that are removed from channel2 |
fmt | internally used segmentation format (needs to have 3 channels) |
Destructor.
virtual void icl::filter::ColorSegmentationOp::apply | ( | const core::ImgBase * | src, |
core::ImgBase ** | dst | ||
) | [virtual] |
main apply function
Implements icl::filter::UnaryOp.
classifies a pixel in given rgb format
void icl::filter::ColorSegmentationOp::clearLUT | ( | icl8u | value = 0 | ) |
clears the whole segmentation LUT
const std::vector<core::Color>& icl::filter::ColorSegmentationOp::getClassMeanColors | ( | ) |
returns a mean color for all used class labels (in rgb format)
const core::Img8u& icl::filter::ColorSegmentationOp::getColoredLUTPreview | ( | int | xDim, |
int | yDim, | ||
icl8u | zValue | ||
) |
this also creates a colored preview for the current segmentation LUT
the method behaves like getLUTPreview, except, it tints the resulting lut-slice with mean colors of that class
const icl8u* icl::filter::ColorSegmentationOp::getLUT | ( | ) | const |
returs the internal lut data
The data order is depth-major row-major i.e. an index(x,y,z) is estimated by (x + w*y + w*h * z)
returns the internal lut data
Please be careful with this method :-)
void icl::filter::ColorSegmentationOp::getLUTDims | ( | int & | w, |
int & | h, | ||
int & | t | ||
) | const |
returns the lut-sizes
w = 1+(0xff >> bitShift[0]) etc.
const core::Img8u& icl::filter::ColorSegmentationOp::getLUTPreview | ( | int | xDim, |
int | yDim, | ||
icl8u | zValue | ||
) |
this also creates a preview for the current segmentation LUT
Here, you can choose, which dimension shall become the resulting images with and height. The remaining 3rd dimension is slices with given zValue
core::format icl::filter::ColorSegmentationOp::getSegmentationFormat | ( | ) | const [inline] |
returns the current internally used segmentation format
returns in internally managed image of the segmentation result
The image has the dimensions 2^(8-shift0) x 2^(8-shift1) and it has 2^(8-shift3) channels
const icl8u* icl::filter::ColorSegmentationOp::getSegmentationShifts | ( | ) | const [inline] |
returns the pointer to the 3 internally used segmentation shifts
void icl::filter::ColorSegmentationOp::load | ( | const std::string & | filename | ) |
loads the segmentation LUT only (no other parameters)
void icl::filter::ColorSegmentationOp::lutEntry | ( | icl8u | a, |
icl8u | b, | ||
icl8u | c, | ||
icl8u | rA, | ||
icl8u | rB, | ||
icl8u | rC, | ||
icl8u | value | ||
) |
given a,b and c in segFormat, this function fills the LUT within a sub-volume with given radii
void icl::filter::ColorSegmentationOp::lutEntry | ( | core::format | fmt, |
int | a, | ||
int | b, | ||
int | c, | ||
int | rA, | ||
int | rB, | ||
int | rC, | ||
icl8u | value | ||
) | throw (utils::ICLException) |
given a,b and c in format fmt, this function fills the LUT within a sub-volume with given radii
void icl::filter::ColorSegmentationOp::save | ( | const std::string & | filename | ) |
saves the segmentation LUT only (no other parameters)
void icl::filter::ColorSegmentationOp::setSegmentationFormat | ( | core::format | fmt | ) | throw (utils::ICLException) |
sets the internally used segmentation format
void icl::filter::ColorSegmentationOp::setSegmentationShifts | ( | icl8u | c0shift, |
icl8u | c1shift, | ||
icl8u | c2shift | ||
) |
sets the segmentation shifts
icl8u icl::filter::ColorSegmentationOp::m_bitShifts[3] [private] |
bit shifts for all 8-Bit channels
internal image in depth8u and segmentation format
last used destination image
LUT3D* icl::filter::ColorSegmentationOp::m_lut [private] |
color classification lookup table
internal buffer holding the output image
format, that is used for internal segmentation
internal buffer for providing a preview of the current segmentation