.. _package-rst-vision: ==================== Package rst.vision ==================== In computer science, image processing is any form of signal processing for which the input is an image. This package contains data types for representing images (:py:class:`Image `) or things which can be recognized in images (e.g. :py:class:`Face `). .. seealso:: Wikipedia article containing the definition above http://en.wikipedia.org/wiki/Image_processing Messages ======== .. container:: mess4ge-multi .. container:: mess4ge-graph .. digraph:: message_graph fontname="Arial"; fontsize=11; stylesheet="../_static/corlab.css"; node [fontsize=11] node [fontname="Arial"] edge [fontsize=11] edge [fontname="Arial"] "16" [label=<

Faces

Face

faces

UINT32

width

UINT32

height

>,shape=box,style=filled,fillcolor="white"]; "17" [label=<

Face

BoundingBox

region

FLOAT64

confidence

>,shape=box,style=filled,fillcolor="white"]; "18" [label=<

BoundingBox

Vec2DInt

top_left

UINT32

width

UINT32

height

UINT32

image_width

UINT32

image_height

>,shape=box,style=filled,fillcolor="white"]; "19" [label=<

Vec2DInt

INT32

>,shape=box,style=filled,fillcolor="white"]; "7" [label=<

LocatedXYZImage

SimpleXYZImage

image

CameraPose

camera

ViewFrustum

frustum

>,shape=box,style=filled,fillcolor="white"]; "14" [label=<

ViewFrustum

FieldOfView

fov

FLOAT32

minimal_distance

FLOAT32

maximal_distance

>,shape=box,style=filled,fillcolor="white"]; "15" [label=<

FieldOfView

FLOAT32

horizontal_aov

FLOAT32

vertical_aov

>,shape=box,style=filled,fillcolor="white"]; "9" [label=<

CameraPose

CoordinateFrame

coordinate_frame

Pose

pose

>,shape=box,style=filled,fillcolor="white"]; "11" [label=<

Pose

Translation

translation

Rotation

rotation

>,shape=box,style=filled,fillcolor="white"]; "13" [label=<

Rotation

FLOAT64

ASCII-STRING

frame_id

>,shape=box,style=filled,fillcolor="white"]; "12" [label=<

Translation

FLOAT64

ASCII-STRING

frame_id

>,shape=box,style=filled,fillcolor="white"]; "10" [label=<

CoordinateFrame

CAMERA_IMAGE_FRAME

CAMERA_X_UP_FRAME

CAMERA_Y_UP_FRAME

LASER_FRAME

SCREEN_FRAME

>,shape=box,style=filled,fillcolor="white"]; "8" [label=<

SimpleXYZImage

UINT32

width

UINT32

height

FLOAT32

>,shape=box,style=filled,fillcolor="white"]; "2" [label=<

Images

Image

images

>,shape=box,style=filled,fillcolor="white"]; "3" [label=<

Image

OCTET-VECTOR

data

UINT32

width

UINT32

height

UINT32

channels

Depth

depth

ColorMode

color_mode

DataOrder

data_order

>,shape=box,style=filled,fillcolor="white"]; "6" [label=<

DataOrder

DATA_SEPARATE

DATA_INTERLEAVED

>,shape=box,style=filled,fillcolor="white"]; "5" [label=<

ColorMode

COLOR_GRAYSCALE

COLOR_RGB

COLOR_BGR

COLOR_YUV

COLOR_YUV422

>,shape=box,style=filled,fillcolor="white"]; "4" [label=<

Depth

DEPTH_8U

DEPTH_16U

DEPTH_32F

>,shape=box,style=filled,fillcolor="white"]; "1" [label=<

LaserScan

FLOAT32

scan_values

FLOAT32

scan_angle

>,shape=box,style=filled,fillcolor="white"]; "16":faces -> "17" []; "17":region -> "18" []; "18":top_left -> "19" []; "7":frustum -> "14" []; "7":camera -> "9" []; "7":image -> "8" []; "14":fov -> "15" []; "9" -> "10" [dir=both,arrowtail=odiamond]; "9":pose -> "11" []; "9":coordinate_frame -> "10" []; "11":rotation -> "13" []; "11":translation -> "12" []; "2":images -> "3" []; "3" -> "6" [dir=both,arrowtail=odiamond]; "3" -> "5" [dir=both,arrowtail=odiamond]; "3" -> "4" [dir=both,arrowtail=odiamond]; "3":data_order -> "6" []; "3":color_mode -> "5" []; "3":depth -> "4" []; .. container:: mess4ge-list .. container:: messages * :ref:`LaserScan ` * :ref:`Images ` * :ref:`LocatedXYZImage ` * :ref:`SimpleXYZImage ` * :ref:`Faces ` * :ref:`Face ` * :ref:`Image ` .. container:: clearer clearer: should be made invisible via css .. _message-rst-vision-laserscan: Message LaserScan ----------------- .. container:: message-rst-vision-laserscan-multi .. container:: message-rst-vision-laserscan-documentation .. py:class:: rst.vision.LaserScan An object of this type represents a single laser scan carried out by an appropriate planar laser range-finder. .. codeauthor:: Leon Ziegler .. py:attribute:: scan_values :type: array of :py:class:`FLOAT32` **Unit**: meter A set of values coming from the laserscanning device. .. py:attribute:: scan_angle :type: :py:class:`FLOAT32` **Unit**: radian The angular range that is covered by the scan. This value depends on the used hardware and equals the sensor's one-dimensional field of view. The values from :py:attr:`scan_values ` are evenly distributed over this angular range. .. container:: message-rst-vision-laserscan-source :download:`Download this file ` .. literalinclude:: //home/jenkins/workspace/rst-manual-0.11-merge-simulator/rst-manual/../rst-proto/proto/stable/rst/vision/LaserScan.proto :lines: 11-29 :language: protobuf :emphasize-lines: 7-7,17-17 .. _message-rst-vision-images: Message Images -------------- .. container:: message-rst-vision-images-multi .. container:: message-rst-vision-images-documentation .. py:class:: rst.vision.Images TODO A collection of images. .. codeauthor:: Johannes Wienke .. py:attribute:: images :type: array of :py:class:`rst.vision.Image` TODO .. container:: message-rst-vision-images-source :download:`Download this file ` .. literalinclude:: //home/jenkins/workspace/rst-manual-0.11-merge-simulator/rst-manual/../rst-proto/proto/stable/rst/vision/Images.proto :lines: 12-19 :language: protobuf :emphasize-lines: 6-6 .. _message-rst-vision-locatedxyzimage: Message LocatedXYZImage ----------------------- .. container:: message-rst-vision-locatedxyzimage-multi .. container:: message-rst-vision-locatedxyzimage-documentation .. py:class:: rst.vision.LocatedXYZImage A simple point cloud represented in 2D structure with information from where it was taken. By adding information about the pose of the camera and its perceived view frustum, one can reconstruct the location and the circumstances under which the provided point cloud was captured. .. codeauthor:: Leon Ziegler .. py:attribute:: image :type: :py:class:`rst.vision.SimpleXYZImage` The background model. .. py:attribute:: camera :type: :py:class:`rst.geometry.CameraPose` The camera's pose in 3d space. .. py:attribute:: frustum :type: :py:class:`rst.geometry.ViewFrustum` The camera's view frustum. .. container:: message-rst-vision-locatedxyzimage-source :download:`Download this file ` .. literalinclude:: //home/jenkins/workspace/rst-manual-0.11-merge-simulator/rst-manual/../rst-proto/proto/stable/rst/vision/LocatedXYZImage.proto :lines: 19-36 :language: protobuf :emphasize-lines: 6-6,11-11,16-16 .. _message-rst-vision-simplexyzimage: Message SimpleXYZImage ---------------------- .. container:: message-rst-vision-simplexyzimage-multi .. container:: message-rst-vision-simplexyzimage-documentation .. py:class:: rst.vision.SimpleXYZImage **Constraint**: ``.width * .height = len(.x)`` **Constraint**: ``.width * .height = len(.y)`` **Constraint**: ``.width * .height = len(.z)`` Saves 3D data (X, Y, Z values) which have a 2D ordering. This kind of depth-data is produced by e.g. Kinect or Swissranger cameras. :py:attr:`x `, :py:attr:`y `, :py:attr:`z ` are 1D arrays which save separately the X, Y, Z values of the 3D points. .. codeauthor:: Agnes Swadzba .. py:attribute:: width :type: :py:class:`UINT32` **Constraint**: ``value > 0`` **Unit**: pixel Width of the underlying 2D ordering structure. .. py:attribute:: height :type: :py:class:`UINT32` **Constraint**: ``value > 0`` **Unit**: pixel Height of the underlying 2D ordering structure. .. py:attribute:: x :type: array of :py:class:`FLOAT32` The values are ordered by linearizing the underlying 2D structure row-wise. .. py:attribute:: y :type: array of :py:class:`FLOAT32` See :py:attr:`x `. .. py:attribute:: z :type: array of :py:class:`FLOAT32` See :py:attr:`x `. .. container:: message-rst-vision-simplexyzimage-source :download:`Download this file ` .. literalinclude:: //home/jenkins/workspace/rst-manual-0.11-merge-simulator/rst-manual/../rst-proto/proto/stable/rst/vision/SimpleXYZImage.proto :lines: 19-51 :language: protobuf :emphasize-lines: 8-8,16-16,21-21,26-26,31-31 .. _message-rst-vision-faces: Message Faces ------------- .. container:: message-rst-vision-faces-multi .. container:: message-rst-vision-faces-documentation .. py:class:: rst.vision.Faces A list of faces with information about the dimensions of a source image used by a face detector. This allows interpretation of the bounding box data contained in the Face types. .. codeauthor:: Sebastian Wrede .. py:attribute:: faces :type: array of :py:class:`rst.vision.Face` List of faces. .. py:attribute:: width :type: :py:class:`UINT32` **Unit**: pixel Width of the source image. .. py:attribute:: height :type: :py:class:`UINT32` **Unit**: pixel Height of the source image. .. container:: message-rst-vision-faces-source :download:`Download this file ` .. literalinclude:: //home/jenkins/workspace/rst-manual-0.11-merge-simulator/rst-manual/../rst-proto/proto/stable/rst/vision/Faces.proto :lines: 15-34 :language: protobuf :emphasize-lines: 6-6,12-12,18-18 .. _message-rst-vision-face: Message Face ------------ .. container:: message-rst-vision-face-multi .. container:: message-rst-vision-face-documentation .. py:class:: rst.vision.Face An object of this type represents a human face detected in an image. The region of the image which corresponds to the face is represented as a :py:class:`BoundingBox ` . .. codeauthor:: Johannes Wienke .. py:attribute:: region :type: :py:class:`rst.geometry.BoundingBox` TODO Bounding box for the detection. .. py:attribute:: confidence :type: :py:class:`FLOAT64` **Unit**: TODO TODO Detection confidence factor. .. container:: message-rst-vision-face-source :download:`Download this file ` .. literalinclude:: //home/jenkins/workspace/rst-manual-0.11-merge-simulator/rst-manual/../rst-proto/proto/stable/rst/vision/Face.proto :lines: 15-28 :language: protobuf :emphasize-lines: 6-6,12-12 .. _message-rst-vision-image: Message Image ------------- .. container:: message-rst-vision-image-multi .. container:: message-rst-vision-image-documentation .. py:class:: rst.vision.Image An uncompressed image with a certain pixel format, color and depth. The binary intensity data is contained in the :py:attr:`data ` field. The remaining fields contain information about the interpretation, that is dimensions (:py:attr:`width ` x :py:attr:`height `), color (:py:attr:`color_mode `, :py:attr:`channels `), depth (:py:attr:`depth `) and channel layout (:py:attr:`data_order `, :py:attr:`channels `), of that data. Suggested interpretation of RSB timestamps: create Image grab time .. todo:: to be more precisely defined. e.g. what does this mean for rolling shutter cameras? .. codeauthor:: Johannes Wienke .. py:attribute:: data :type: :py:class:`OCTET-VECTOR` **Constraint**: ``len(value) = (.width * .height * .channels * .depth) / 8`` Stores the actual pixel data of the image. The number of bytes and their interpretation depend on the values of the remaining fields. .. py:attribute:: width :type: :py:class:`UINT32` **Unit**: pixel The number of pixels in each row of the image. .. py:attribute:: height :type: :py:class:`UINT32` **Unit**: pixel The number of pixels in each column of the image. .. py:attribute:: channels :type: :py:class:`UINT32` **Unit**: number The number of color channels in the image. The interpretation of these channels depends on :py:attr:`color_mode `. .. py:attribute:: depth :type: :py:class:`rst.vision.Image.Depth` The number of bits used to encode .. py:attribute:: color_mode :type: :py:class:`rst.vision.Image.ColorMode` Colorspace and layout of the color channels of the image. Determines pixel layout in combination with :py:attr:`data_order `. .. py:attribute:: data_order :type: :py:class:`rst.vision.Image.DataOrder` Layout of the color planes of the image. Determines pixel layout in combination with :py:attr:`color_mode `. .. container:: message-rst-vision-image-source :download:`Download this file ` .. literalinclude:: //home/jenkins/workspace/rst-manual-0.11-merge-simulator/rst-manual/../rst-proto/proto/stable/rst/vision/Image.proto :lines: 25-167 :language: protobuf :emphasize-lines: 101-101,107-107,113-113,122-122,127-127,134-134,141-141 .. _message-rst-vision-image-depth: Message Depth ------------- .. container:: message-rst-vision-image-depth-multi .. container:: message-rst-vision-image-depth-documentation .. py:class:: rst.vision.Image.Depth Depth of one color channel. Reflecting the definitions from OpenCV's IplImage. .. py:attribute:: DEPTH_8U = 8 *Not documented* .. py:attribute:: DEPTH_16U = 16 *Not documented* .. py:attribute:: DEPTH_32F = 32 *Not documented* .. container:: message-rst-vision-image-depth-source :download:`Download this file ` .. literalinclude:: //home/jenkins/workspace/rst-manual-0.11-merge-simulator/rst-manual/../rst-proto/proto/stable/rst/vision/Image.proto :lines: 32-36 :language: protobuf :emphasize-lines: 2-2,3-3,4-4 .. _message-rst-vision-image-colormode: Message ColorMode ----------------- .. container:: message-rst-vision-image-colormode-multi .. container:: message-rst-vision-image-colormode-documentation .. py:class:: rst.vision.Image.ColorMode Colorspace(s) and layout of color channel(s). .. py:attribute:: COLOR_GRAYSCALE = 0 The image consists of only one channel containing intensity values. MUST be used with :py:attr:`channels ` = 1. .. py:attribute:: COLOR_RGB = 1 The image has three color channels containing the red, green and blue color components (note the order of the channels, not to be confused with :py:attr:`COLOR_BGR `). MUST be used with :py:attr:`channels ` = 3. .. py:attribute:: COLOR_BGR = 2 The image has three color channels containing the blue, green and red color components (note the order of the channels, not to be confused with :py:attr:`COLOR_RGB `). MUST be used with :py:attr:`channels ` = 3. .. py:attribute:: COLOR_YUV = 4 The image has three color channels, encoded in the YUV colorspace, and uses the YUVYUV pixel layout. MUST be used with :py:attr:`channels ` = 3. .. py:attribute:: COLOR_YUV422 = 8 The image has three color channels, encoded in the YUV colorspace, and uses the YUYV pixel layout. MUST be used with :py:attr:`channels ` = 3. .. container:: message-rst-vision-image-colormode-source :download:`Download this file ` .. literalinclude:: //home/jenkins/workspace/rst-manual-0.11-merge-simulator/rst-manual/../rst-proto/proto/stable/rst/vision/Image.proto :lines: 41-85 :language: protobuf :emphasize-lines: 9-9,18-18,27-27,35-35,43-43 .. _message-rst-vision-image-dataorder: Message DataOrder ----------------- .. container:: message-rst-vision-image-dataorder-multi .. container:: message-rst-vision-image-dataorder-documentation .. py:class:: rst.vision.Image.DataOrder Describes storage layout of pixels with respect to channels. .. py:attribute:: DATA_SEPARATE = 0 Separate planes. Example for :py:attr:`COLOR_RGB `: .. parsed-literal:: R R R ... (:py:attr:`width ` x :py:attr:`height ` times) G G G ... (:py:attr:`width ` x :py:attr:`height ` times) B B B ... (:py:attr:`width ` x :py:attr:`height ` times) .. py:attribute:: DATA_INTERLEAVED = 1 Interleaved image. Example for :py:attr:`COLOR_RGB `: .. parsed-literal:: R G B R G B R G B ... (:py:attr:`width ` x :py:attr:`height ` times) .. container:: message-rst-vision-image-dataorder-source :download:`Download this file ` .. literalinclude:: //home/jenkins/workspace/rst-manual-0.11-merge-simulator/rst-manual/../rst-proto/proto/stable/rst/vision/Image.proto :lines: 90-116 :language: protobuf :emphasize-lines: 14-14,25-25