Package rst.vision

Not documented

Messages

• LocatedFace
• CalibrationMatrices
• AugmentedPointCloudSet3DFloat
• AugmentedPointCloud3DFloat
• HeadObjects
• HeadObject
• Face
• CameraMatrix3x4

clearer: should be made invisible via css

Message LocatedFace

class rst.vision.LocatedFace

This type extends the regular Face type with a 3D location.

Code author: Leon Ziegler <lziegler@techfak.uni-bielefeld.de>

face

Type :rst.vision.Face

The face as a part of an image.

location

Type :rst.geometry.Translation

The 3d location of the face.

orientation

Type :rst.geometry.Rotation

The 3d orientation of the face. The provided rotation should be passed as a relative rotation from the default pose when the person looks straight ahead. In this case the axes of the corresponding coordinate system are defined as follows:

• x axis: points in the direction of the person’s view
• y axis: points to the person’s right side
• z axis: points downwards

Download this file

message LocatedFace {

    /**
     * The face as a part of an image.
     */
    required vision.Face face = 1;

    /**
     * The 3d location of the face.
     */
    optional geometry.Translation location = 2;

    /**
     * The 3d orientation of the face. The provided rotation should be passed
     * as a relative rotation from the default pose when the person looks
     * straight ahead. In this case the axes of the corresponding coordinate
     * system are defined as follows:
     *
     * * x axis: points in the direction of the person's view
     * * y axis: points to the person's right side
     * * z axis: points downwards
     */
    optional geometry.Rotation orientation = 3;
}

Message CalibrationMatrices

class rst.vision.CalibrationMatrices

A pair of 2D Points.

Todo

comment seems wrong

Code author: Johannes Wienke <jwienke@techfak.uni-bielefeld.de>

left

Type :rst.vision.CameraMatrix3x4

Not documented

right

Type :rst.vision.CameraMatrix3x4

Not documented

Download this file

message CalibrationMatrices {

    required CameraMatrix3x4 left = 1;

    required CameraMatrix3x4 right = 2;

}

Message AugmentedPointCloudSet3DFloat

class rst.vision.AugmentedPointCloudSet3DFloat

A set of augmented 3D Point clouds.

Code author: Jan Moringen <jmoringe@techfak.uni-bielefeld.de>

clouds

Type :array of rst.vision.AugmentedPointCloud3DFloat

Empty collection of clouds is allowed. The order of cloud objects is not significant.

Download this file

message AugmentedPointCloudSet3DFloat {

    /**
     * Empty collection of clouds is allowed.
     * The order of cloud objects is not significant.
     */
    repeated AugmentedPointCloud3DFloat clouds = 1;

}

Message AugmentedPointCloud3DFloat

class rst.vision.AugmentedPointCloud3DFloat

Associates a set of scalar values (such as color information) to each point of a point cloud.

Code author: Johannes Wienke <jwienke@techfak.uni-bielefeld.de>

points

Type :rst.geometry.PointCloud3DFloat

Geometry of the point cloud augmented by the data in channels.

channels

Type :array of rst.vision.AugmentedPointCloud3DFloat.Channel

An optional set of scalar information to attach to each point of the point cloud.

Download this file

message AugmentedPointCloud3DFloat {

    /**
     * One channel of color information attached to a set of 3D points.
     *
     * Each channel contains a vector of the same size as the original point
     * cloud and attaches the value of one color channel to each point of the
     * target point cloud.
     *
     * The following conventions for channel names and respective values exist:
     *
     * "r", "g", "b"
     *
     *   RGB values represented between 0 and 1.
     *
     * "intensity"
     *
     *   Grayscale image value. Between 0 and 1.
     *
     * "u", "v"
     *
     *   Forms YUV color information in combination with an
     *   "intensity" channel. Between 0 and 1.
     *
     * Inspired by the design of the sensor_msgs/ChannelFloat32 ROS
     * message type.
     */
    message Channel {

        /**
         * The name of the color channel.
         */
        required string name = 1;

        /**
         * The actual values for the represented color channel for
         * each point of the target point cloud.
         *
         * N-th value is associated to n-th point in @ref .points.
         */
        // @constraint(len(value) = len(.points.points))
        repeated float values = 2 [packed = true];

    }

    /**
     * Geometry of the point cloud augmented by the data in @ref
     * .channels.
     */
    required .rst.geometry.PointCloud3DFloat points = 1;

    /**
     * An optional set of scalar information to attach to each point
     * of the point cloud.
     */
    repeated Channel channels = 2;

}

Message Channel

class rst.vision.AugmentedPointCloud3DFloat.Channel

One channel of color information attached to a set of 3D points.

Each channel contains a vector of the same size as the original point cloud and attaches the value of one color channel to each point of the target point cloud.

The following conventions for channel names and respective values exist:

“r”, “g”, “b”

RGB values represented between 0 and 1.

“intensity”

Grayscale image value. Between 0 and 1.

“u”, “v”

Forms YUV color information in combination with an “intensity” channel. Between 0 and 1.

Inspired by the design of the sensor_msgs/ChannelFloat32 ROS message type.

name

Type :ASCII-STRING

The name of the color channel.

values

Type :array of FLOAT32

Constraint: len(value) = len(.points.points)

The actual values for the represented color channel for each point of the target point cloud.

N-th value is associated to n-th point in points.

Download this file

    message Channel {

        /**
         * The name of the color channel.
         */
        required string name = 1;

        /**
         * The actual values for the represented color channel for
         * each point of the target point cloud.
         *
         * N-th value is associated to n-th point in @ref .points.
         */
        // @constraint(len(value) = len(.points.points))
        repeated float values = 2 [packed = true];

    }

Message HeadObjects

class rst.vision.HeadObjects

A list of head objects.

Code author: Vasil Khalidov <vasil.khalidov@idiap.ch>

head_objects

Type :array of rst.vision.HeadObject

Not documented

Download this file

message HeadObjects {

    repeated vision.HeadObject head_objects = 1;

}

Message HeadObject

class rst.vision.HeadObject

Focus on image coordinate systems (vision-based).

A head object represented by its ID, location in the image and head rotation angles.

id : head object identifier

Todo

what does the above mean?

Code author: Vasil Khalidov <vasil.khalidov@idiap.ch>

tracking_info

Type :rst.tracking.TrackingInfo

Not documented

region

Type :rst.geometry.BoundingBox

Head location in the input image.

position

Type :rst.math.Vec3DFloat

Not documented

pose

Type :rst.math.Vec3DFloat

<Could not resolve reference to .x> -> pan

Pan head rotation angle. Positive: person looks to her or his right side.

<Could not resolve reference to .y> -> tilt

Tilt head rotation angle. Positive: person looks up.

<Could not resolve reference to .z> -> roll

Roll head rotation angle. Positive: person’s head rolled to his right shoulder.

faces

Type :array of rst.vision.HeadObject.LabeledFace

Todo

semantic description

Repeated field because in cases of stereo processing there might be faces from both cameras.

speaking_probability

Type :FLOAT32

Not documented

vfoa_target

Type :OCTET-VECTOR

String description of the visual focus of attention of this head.

identity

Type :rst.classification.ClassificationResult

Not documented

gender

Type :rst.classification.ClassificationResult

Not documented

age

Type :rst.classification.ClassificationResult

Not documented

Download this file

message HeadObject {

    optional tracking.TrackingInfo tracking_info = 1;

    /**
     * Head location in the input image.
     */
    optional geometry.BoundingBox region = 2;

    optional math.Vec3DFloat position = 3;

    /**
     * @ref .x -> pan
     *
     *   Pan head rotation angle. Positive: person looks to her or his
     *   right side.
     *
     * @ref .y -> tilt
     *
     *   Tilt head rotation angle. Positive: person looks up.
     *
     * @ref .z -> roll
     *
     *   Roll head rotation angle. Positive: person's head rolled to
     *   his right shoulder.
     */
    optional math.Vec3DFloat pose = 4;

    message LabeledFace {

        required bytes label = 1;

        required vision.Face face = 2;

    }

    /**
     * @todo "semantic description"
     *
     * Repeated field because in cases of stereo processing there
     * might be faces from both cameras.
     */
    repeated LabeledFace faces = 6;

    optional float speaking_probability = 7;

    /**
     * String description of the visual focus of attention of this
     * head.
     */
    optional bytes vfoa_target = 8;

    optional classification.ClassificationResult identity = 9;

    optional classification.ClassificationResult gender = 10;

    optional classification.ClassificationResult age = 11;

}

Message LabeledFace

class rst.vision.HeadObject.LabeledFace

Not documented

label

Type :OCTET-VECTOR

Not documented

face

Type :rst.vision.Face

Not documented

Download this file

    message LabeledFace {

        required bytes label = 1;

        required vision.Face face = 2;

    }

Message Face

class rst.vision.Face

An object of this type represents a human face detected in an image.

The region of the image which corresponds to the face is represented as a BoundingBox .

Code author: Johannes Wienke <jwienke@techfak.uni-bielefeld.de>

region

Type :rst.geometry.BoundingBox

TODO Bounding box for the detection.

confidence

Type :FLOAT64

Unit: TODO

TODO Detection confidence factor.

Download this file

message Face {

    /**
     * TODO Bounding box for the detection.
     */
    required geometry.BoundingBox region = 1;

    /**
     * TODO Detection confidence factor.
     */
    // @unit(TODO)
    optional double confidence = 2;

}

Message CameraMatrix3x4

class rst.vision.CameraMatrix3x4

Matrix for Camera Matrix Parameters K(R, t).

Code author: Johannes Wienke <jwienke@techfak.uni-bielefeld.de>

c11

Type :FLOAT32

First row.

c12

Type :FLOAT32

Not documented

c13

Type :FLOAT32

Not documented

c14

Type :FLOAT32

Not documented

c21

Type :FLOAT32

Second row.

c22

Type :FLOAT32

Not documented

c23

Type :FLOAT32

Not documented

c24

Type :FLOAT32

Not documented

c31

Type :FLOAT32

Third row.

c32

Type :FLOAT32

Not documented

c33

Type :FLOAT32

Not documented

c34

Type :FLOAT32

Not documented

Download this file

message CameraMatrix3x4 {

    /**
     * First row.
     */
    required float c11 = 1;
    required float c12 = 2;
    required float c13 = 3;
    required float c14 = 4;

    /**
     * Second row.
     */
    required float c21 = 5;
    required float c22 = 6;
    required float c23 = 7;
    required float c24 = 8;

    /**
     * Third row.
     */
    required float c31 = 9;
    required float c32 = 10;
    required float c33 = 11;
    required float c34 = 12;

}