The Goal

The goal is to provide a 3-Layered segmentation of a motion.

The first layer describes the main segments of the motion. These are often labels you would use to explain the motion. Like at first the person jumps and then he walks. Jumping and Walking would be the first level segments for a motion like that

The second layer describes the sub-segments of its previous layer. Example: Walking consists of left step and right step segments, Jumping consists of a propulsion phase a flying phase and a landing phase.

The third layer describes the second layer in mostly atomic segments. Example: a step would be segmentable into lifting the foot and placing the foot

Sometimes there are cases where the second and third layer are the same because the second layer is not segmentable anymore. Example: Flying Phase of a jump. In this case, you just type the same thing as in the second layer

You might also be interested in the article, which proposes this kind of segmentation.

Click to view the corresponding passage out of
A Framework for Evaluating Motion Segmentation Algorithms.

As already mentioned before, one major problem with evaluations of works on motion segmentation is that there is no generally applicable ground truth, because segmentation algorithms are usually designed and fine-tuned for specific use cases. For example, one algorithm may be designed as an activity recognition component in a smart home environment, while another algorithm may be used on a robot which is capable of learning new motions by demonstration of a human teacher. Each of these use cases come with certain granularities for which the given algorithm performs best – e.g. the smart home algorithm would not be able to distinguish motion primitives as well as the one used on a robot that depends on it in order to learn new movements.

We therefore propose a Motion Segmentation Point Hierar- chy, in which each ground truth segmentation point is tagged with a granularity. In total there are three granularities and they are defined as follows:

  • Rough granularity: Used to distinguish activities.
    Example: Jumping vs. walking
  • Medium granularity: Used to distinguish actions.
    Example: Stepping with left vs. right foot
  • Fine granularity: Used to distinguish motion primitives.
    Example: Lift foot for step vs. set down foot

A motion recording labelled with these tagged segmentation points could look as depicted in Figure 4.

We make use of the granularities while evaluating the segmentation quality of an algorithm. For each granularity, the evaluation process will be performed once, starting with only the ground truth segmentation points tagged as rough. After that, the ground truth segmentation points tagged as medium


will be added and the evaluation process will be repeated. In the last stage, the remaining ground truth segmentation points, tagged as fine, will be added and again, the evaluation process will be performed. Therefore, for each measure evaluating the segmentation quality, we receive three scores, one for each granularity.

While the introduction of different granularities increases the complexity of labelling the ground truth, the Motion Segmentation Point Hierarchy serves as a guideline for annotators, making the labelling process more intuitive after a brief instruct.

Please try to make your labels and segments as accurate as possible

For further instructions have a look at the Tutorial Videos and the Controls

Tutorial Videos

Controls

If you click into the Scene, you can use the left Mouse button to rotate the Camera and the Mouse wheel to zoom in and out.

To just play the motion you can use the Play Button to start playing the motion or use the Slider to jump to a specific time inside the motion

If you want to segment the motion into more segments, you can click one of the Sliders and position it at the point where you want the segment to start/end

As soon as you started dragging a Slider a frame counter will appear. It will show you the current frame your segment is currently ending in.

While dragging a Slider, you can hit Space to make the frame adjustment more accurate. You can do this to set your segment endings to a more specific frame

To open a new Hierarchy just click the segment you want to sub-segment

This will open the Slider for the next Hierarchy

The currently selected segment becomes Blue

If you double-click a segment, a dialog for naming it will be opened.

If a segment and all its sub-segments are named, it becomes Green to indicate that you named and segmented all Hierarchies of that segment

If you want to remove a segment, you have to select the segment and click the Remove Button next to the Slider.

Note: each Hierarchy has an own Remove Button and you can only remove a segment if there is more than one segment in that Hierarchy

If a segment and all of the segments in the lower Hierarchies are named the segment will be shown in Green. This means that the segment is fully segmented

When all segments are finished you are able to submit your segmentation

Note: You have to segment the motion in all 3 Hierarchies