gesture-recognition Documentation


gr_license

Syntax:

gr_license();

Returns: (string) The license string

Description:

The Gesture Recognition Engine for GameMaker Studio 2, uses internally the $P Algorithm.

The $P Point-Cloud Recognizer algorithm used in this Gesture Recognition Engine, is distributed under the "New BSD License". It means that if you use this extension you:


gr_template_add

Syntax:

gr_template_add(name, data_type, points_data);
Argument name Type Description
name string The name of the gesture. You can create multiple gestures with the same name.
data_type string A string that specify the type of data provided in points_data argument. It can be "path", "array_path", "array_points", "array_raw", "list_points" or "list_raw".
points_data array|list|path The data that will be determinate the points of the path.

Returns: (Gesture) The new created gesture

Description:

Creates and adds a gesture to the Recognition system as a template gesture. Check gr_gesture_create to know more about the arguments.


gr_template_delete_all

Syntax:

gr_template_delete_all([dont_destroy_gestures]);
Argument name Type Description
dont_destroy_gestures boolean If true, the gestures will be only removed from the template list without destroying them

Description:

Removes all gestures from the templates list. By default, it also calls gr_gesture_destroy on each gesture to free the memory. If for some reason you only want to delete the gestures from the list but without destroying them, you can pass true as the optional dont_destroy_gestures argument.


gr_template_find_by_name

Syntax:

gr_template_find_by_name(name);
Argument name Type Description
name string The name of the gesture that you are looking.

Returns: (Gesture) The gesture

Description:

Finds and returns the first added template gesture with the specified name.


gr_template_find_by_index

Syntax:

gr_template_find_by_index(index);
Argument name Type Description
index integer The index starting from 0 of the gesture that you are looking.

Returns: (Gesture) The gesture

Description:

Finds and returns the first added template gesture with the specified name.


gr_template_number

Syntax:

gr_template_number();

Returns: (integer) The number of total templates on the system

Description:

gr_template_number();


gr_template_get_all

Syntax:

gr_template_get_all([exclude_disabled ]);
Argument name Type Description
exclude_disabled bool If true, only the enabled gestures will be returned

Returns: (Array) An array with all the gestures added

Description:

Returns an array with all the template gestures added to the system


gr_gesture_create

Syntax:

gr_gesture_create(name, gesture_data_type, points_data);
Argument name Type Description
name string The name of the gesture. You can create multiple gestures with the same name.
gesture_data_type string A string that specify the type of data provided in points_data argument. It can be "path", "array_path", "array_points", "array_raw", "list_points" or "list_raw".
points_data array|list|path The data that will be determinate the points of the path.

Returns: (Gesture) The new created gesture

Description:

Creates a gesture with the specified name ussing the points_data specified. Note: if you want to add the gesture to the recognition system you must use gr_add instead. You can use one of the following strings in the data_type argument to create you gesture:

Warning, if you pass a ds_list as the points_data argument, the gesture will be created and the list will be destroyed.


gr_gesture_set_enabled

Syntax:

gr_gesture_set_enabled(gesture, enabled);
Argument name Type Description
gesture Gesture The gesture
enabled bool True if enabled

Description:

Sets the enabled propery of a specified gesture. If a gesture is disabled, then is ignored by the system when performing the gesture recognition process.


gr_gesture_get_enabled

Syntax:

gr_gesture_get_enabled(gesture);
Argument name Type Description
gesture Gesture The gesture

Returns: (bool) True if enabled

Description:

Gets the enabled propery of a specified gesture. If a gesture is disabled, then is ignored by the system when performing the gesture recognition process.


gr_gesture_get_name

Syntax:

gr_gesture_get_name(gesture);
Argument name Type Description
gesture Gesture The gesture

Returns: (string) The gesture name

Description:

Gets the name of a gesture


gr_gesture_get_point

Syntax:

gr_gesture_get_point(gesture, index);
Argument name Type Description
gesture Gesture The gesture
index integer The point index

Returns: (Point) The specified point

Description:

Gets the name of a gesture

Info: Gestures are preprocesed and stored diferently than you spect. So it's probably that you encounter that the number of points that you get inside a gesture is different than the number of points you provided when created the gesture


gr_gesture_points_number

Syntax:

gr_gesture_points_number(gesture);
Argument name Type Description
gesture Gesture The gesture

Returns: (integer) The number of points inside the gesture

Description:

Gets the number of points inside a gesture.

Info: Gestures are preprocesed and stored diferently than you spect. So it's probably that you encounter that the number of points that you get inside a gesture is different than the number of points you provided when created the gesture


gr_gesture_destroy

Syntax:

gr_gesture_destroy(gesture);
Argument name Type Description
gesture Gesture The gesture to destroy

Returns: (undefined) The value undefined. See example.

Description:

Destroys the specified gesture and free up memory. Use it as: my_gesture = gr_gesture_destroy(my_gesture);

Example:

my_gesture = gr_gesture_destroy(my_gesture);

gr_point_create

Syntax:

gr_point_create(x, y, stroke);
Argument name Type Description
x real The x coordinate of the point.
y real The y coordinate of the point.
stroke integer The stroke index for the specified point startig from zero.

Returns: (Point) The point instance

Description:

Creates a point and returns it. If you are creating a gesture with multiple strokes, you can pass the stroke index. If not, simply pass 0 (zero) for the stroke parameter.


gr_point_get_x

Syntax:

gr_point_get_x(point);
Argument name Type Description
point Point The point

Returns: (real) The X coordinate of the specified point

Description:

Returns the X coordinate of the specified point

Example:

// This code will manually draw each point of the recorded gesture
var size = gr_record_points_number(record);
for (var i = 0; i < size; i++) {
var point = gr_record_get_point(record, i);
var xx = gr_point_get_x(point);
var yy = gr_point_get_y(point);
draw_circle(xx, yy, 3, false);
}

gr_point_get_y

Syntax:

gr_point_get_y(point);
Argument name Type Description
point Point The point

Returns: (real) The Y coordinate of the specified point

Description:

Returns the Y coordinate of the specified point

Example:

// This code will manually draw each point of the recorded gesture
var size = gr_record_points_number(record);
for (var i = 0; i < size; i++) {
var point = gr_record_get_point(record, i);
var xx = gr_point_get_x(point);
var yy = gr_point_get_y(point);
draw_circle(xx, yy, 3, false);
}

gr_point_get_stroke

Syntax:

gr_point_get_stroke(point);
Argument name Type Description
point Point The point

Returns: (integer) The stroke index of the specified point

Description:

Returns the Stroke index of the specified point


gr_point_distance

Syntax:

gr_point_distance(point1, point2);
Argument name Type Description
point1 Point The first point
point2 Point The second point

Returns: (real) The distance between two points

Description:

Returns the distance between two points


gr_recognition_async_process

Syntax:

gr_recognition_async_process(result, [time_to_spend ]);
Argument name Type Description
result Result The recognition result object to process
time_to_spend real The minimum number of milliseconds to spend in the async recognition process

Returns: (boolean) True if the recognition has ended. If you pass undefined or an invalid "result"

Description:

Process the recognition object asynchronous. If the recognition is in progress, the funtion returns false. When the recognition is ready, the function returns true. Then, you can use the normal gr_recognition_get functions to get the results of the recognition.

The second argument, specifies how many milliseconds should be spent computing the result. If you don't pass this argument, it is automatically calculated based on the room_speed.

Example:

/// Global mouse up event
/// Pass true to activate the async mode
recognition = gr_record_recognize(record, true);

/// Step event
if (gr_recogtition_async_is_ready(recognition)) {
var ready = gr_recognition_async_process(recognition);
if (ready) {
var name = gr_recognition_get_name(recognition, 0);
recognition = undefined; // reset the result
show_message("You have drawn a " + name );
}
}

/// Draw event
var ready = gr_recognition_async_is_ready(recognition);
var total = gr_recognition_async_total(recognition);
var number = gr_recognition_async_total(number);
var progress = floor(number / total * 100);
draw_text(x, y, "Recognition is " + (ready ? "ready" : "in progress"));
draw_text(x, y, "Progress: " + string(progress) + "%");

gr_recognition_async_is_ready

Syntax:

gr_recognition_async_is_ready(result);
Argument name Type Description
result Result The recognition result object

Returns: (boolean) True if the results are ready. False if the job is in progress. If the result object is not async, then it always return true

Description:

Returns true if the recognition is already computed, and is ready. You can use in in the step event to check the recognition state. Check the example below.

INFO: if you pass a value that is not a Recognition, the function returns false.


gr_recognition_async_total

Syntax:

gr_recognition_async_total(result);
Argument name Type Description
result Result The recognition result object

Returns: (integer) The number of total templates to be processed by the async recognition system

Description:

gr_recognition_async_total(result);


gr_recognition_async_number

Syntax:

gr_recognition_async_number(result);
Argument name Type Description
result Result The recognition result object

Returns: (integer) The number of templates processed by the async recognition system

Description:

If the recognition is not async, then the total number of templates processed will be returned


gr_recognition_get_name

Syntax:

gr_recognition_get_name(result, [index]);
Argument name Type Description
result Result The recognition result object
index integer The index of the recognition starting from zero

Returns: (string) The gesture template name of the result with the specified index

Description:

Returns the name of the recognized pattern gesture for that recognition result with the specified index. For example, to get the name of the most similar pattern acording to the recognition results you can use var name = gr_recognition_get_name(result, 0);. To get the second most similar pattern, use 1, and so on. The index must be between ´0´ and ´gr_recognition_get_number(recognition) - 1´, and they are ordered by the accuracy.

If you don't specify the index, then the index 0 (the most accurate result) is assumed.


gr_recognition_get_accuracy

Syntax:

gr_recognition_get_accuracy(result, [index]);
Argument name Type Description
result Result The recognition result object
index integer The index of the recognition starting from zero

Returns: (real) The accuracy of the result with the specified index

Description:

Returns the accuracy (as a real from 0.00 to 1.00) of the recognized pattern gesture for that recognition result with the specified index. For example, to get the accuracy of the most similar pattern acording to the recognition results you can use var accuracy = gr_recognition_get_accuracy(result, 0);. To get the second most similar pattern, use 1, and so on. The index must be between ´0´ and ´gr_recognition_get_number(recognition) - 1´, and they are ordered by the accuracy.

If you don't specify the index, then the index 0 (the most accurate result) is assumed.


gr_recognition_to_map

Syntax:

gr_recognition_to_map(result);
Argument name Type Description
result Result The recognition result object

Returns: (ds_map) A ds_map

Description:

Creates and returns a new ds_map where each key is the name of the template gesture and the value is the accuracy score of that gesture template.

Example:

var map = gr_recognition_to_map(result);
var s = map[? "triangle"];
show_debug_message("The accuracy score for the triangle gesture is " + string(s));

gr_recognition_get_number

Syntax:

gr_recognition_get_number(result);
Argument name Type Description
result Result The recognition result object

Returns: (integer) The number of total recognition results

Description:

Returns the number of total recognition results. Normally, this is equal to the total number of template gestures on the system at the time of the recognition ocurred.


gr_path_resample

Syntax:

gr_path_resample(points_list, n);
Argument name Type Description
points_list ds_list A ds_list with points
n integer The number of points

Returns: (ds_list) A new resampled ds_list

Description:

Note: This function is used internally by the system, but is public in case you want to use it for something

Resample a figure of ds_list of an arbitrary number of points and returns a new ds_list with the new number of points. The coordinates of each point are interpolated in the resampling process.


gr_path_length

Syntax:

gr_path_length(points_list);
Argument name Type Description
points_list ds_list A ds_list with points

Returns: (real) length traversed by a point path

Description:

Returns the length of a path


gr_path_distance

Syntax:

gr_path_distance(points_list1, points_list2);
Argument name Type Description
points_list1 ds_list A ds_list with points
points_list2 ds_list A ds_list with points

Returns: (real) average distance between corresponding points in two paths

Description:

Returns the average distance between correspoinding points in two paths.


gr_path_centroid

Syntax:

gr_path_centroid(points_list);
Argument name Type Description
points_list ds_list A ds_list with points

Returns: (Point) The centroid of the path

Description:

Calculates the centroid of a path of points


gr_path_scale

Syntax:

gr_path_scale(points_list);
Argument name Type Description
points_list ds_list A ds_list with points

Returns: (ds_list) The same point_list

Description:

Normalizes the size of a path of points


gr_path_translate

Syntax:

gr_path_translate(points_list, translate_to);
Argument name Type Description
points_list ds_list A ds_list with points
translate_to Point The location to translate the centroid to

Returns: (ds_list) The same point_list

Description:

Translates points' centroid to the specified location in space


gr_draw_points_list

Syntax:

gr_draw_points_list(points_list, x, y, scale_x, scale_y);
Argument name Type Description
points_list ds_list A ds_list with Point objects. You can create new points with gr_point_create.
x real The x position
y real The y position
scale_x real The horizontal scale. 1 = no scale.
scale_y real The vertical scale. 1 = no scale.

Returns: (nothing) Nothing

Description:

Draw the list of points on the screen. This is an advanced function, normally, you want to use gr_draw_gesture to draw a gesture, or gr_draw_record to draw a record. To configure the colour and width of the line, you MUST use first gr_draw_config.


gr_draw_gesture

Syntax:

gr_draw_gesture(gesture_or_name, x, y, width, height);
Argument name Type Description
gesture_or_name Gesture|string The gesture instance of the name of the gesture
x real The x position of the top left corner of the gesture
y real The y position of the top left corner of the gesture
width real The width of the gesture
height real The height of the gesture

Description:

Draws a path of the given gesture on the screen. To configure the colour and width of the line, you MUST use first gr_draw_config.


gr_draw_record

Syntax:

gr_draw_record(record);
Argument name Type Description
record Record The record to draw

Description:

Draws a path of the given gesture record on the screen To configure the colour and width of the line, you MUST use first gr_draw_config.


gr_draw_config

Syntax:

gr_draw_config(width, color, round_corners);
Argument name Type Description
width real The width of the stroke
color real THe color of the stroke
round_corners boolean If true, the drawing methods will use circles to create a round corners effect

Description:

Configure the width, color and corners mode of the another gr_draw_ methods. You MUST call this before calling any other method since there is no default value for the width, color and corners.


gr_record_create

Syntax:

gr_record_create();

Returns: (Record) The new record instance

Description:

Creates a new record instance. A record, represents a User recorded gesture. Is composed by a set of points. The points are usually captured using a mouse, touchscreen, or another input device and then are shown on the screen. One record, can have multiple strokes. At least one stroke with two points are needed to be able to recognize any gesture.

Example:

// create
record = undefined;

// global mouse presed event
gr_record_destroy(record); // Destroy the previous gesture record
record = gr_record_create(); // And creates a new gesture record

// global mouse down event
if (record != undefined) {
gr_record_add_point(record, mouse_x, mouse_y);
}

// global mouse released event
if (record != undefined) {
result = gr_record_recognize(record);
// Then, do stuff with the result of the recognition
}

gr_record_destroy

Syntax:

gr_record_destroy(record);
Argument name Type Description
record Record The record to destroy

Returns: (undefined) Returns undefined (see example to understand why)

Description:

gr_record_destroy(record);


gr_record_add_point

Syntax:

gr_record_add_point(record, x, y, [stroke_number]);
Argument name Type Description
record Record The record to add the point
x real The point x coordinate (usually, you want to use mouse_x)
y real The point y coordinate (usually, you want to use mouse_y)
stroke_number integer The stroke number, by default is stroke 0

Description:

gr_record_add_point(record, x, y, [stroke_number = 0]);


gr_record_draw

Syntax:

gr_record_draw(	record, line_width, line_col, line_alpha, bullets_rad, bullets_col, bullets_alpha);
Argument name Type Description
record Record The record you want to draw
line_width real The line width
line_col color The line color
line_alpha real The line alpha
bullets_rad real The radius of each bullet (vertex point)
bullets_col color The color of each bullet (vertex point)
bullets_alpha real The alpha of each bullet (vertex point)

Returns: (nothing) Nothing

Description:

Draws a gesture record on the screen. You can choose your own line width, color and alpha. Also, you can choose a radius, color and alpha to draw a circle or bullet in each vertex of the recorded gesture. This is a very simple code to draw the gesture, if you need a more advanced drawing, you can use the functions gr_record_get_point to iterate over the points of the recording and draw it yourself.


gr_record_points_number

Syntax:

gr_record_points_number(record);
Argument name Type Description
record Record The record

Returns: (integer) The number of points on the record (for all strokes)

Description:

Get the number of points inside the gesture record (for all strokes)


gr_record_get_point

Syntax:

gr_record_get_point(record, i);
Argument name Type Description
record Record The record
i integer The point index

Returns: (Point) The point number i inside the recording

Description:

Get the specified point in the specified gesture record. You can use the gr_point functions to get info about the point.

Example:

// This code will manually draw each point of the recorded gesture
var size = gr_record_points_number(record);
for (var i = 0; i < size; i++) {
var point = gr_record_get_point(record, i);
var xx = gr_point_get_x(point);
var yy = gr_point_get_y(point);
draw_circle(xx, yy, 3, false);
}

gr_record_recognize

Syntax:

gr_record_recognize(record, [async]);
Argument name Type Description
record Record The record object to recognize
async boolean If true, the recognition will be made asynchronous

Returns: (Recognition) The recognition result object

Description:

Recognize the current record and return a Recognition result object. You must use the ´gr_recognition_*´ functions to get the available information about the result. The recognition process is made synchornously. It means, that if you have A LOT of templates to be recognized, this function can be slow and freeze the game. If notice that your game slows, then you need to use the async mode.

The async mode let you split the recognition process over multiple game frames (steps). To use the async mode, pass true as the second argument. To learn more about the async mode, check gr_recognition_async_process();

NOTE: The algorithm ussed was the $P algorithm. I (Javier "Ciberman" Mora) am not the creator of the $P algorithm. The gesture recognition engine for GMS2 is a rewrite for GMS of the $P algorithm with a lot of useful aditions. The algorithm itself is released under the "New BSD License". Check the script gr_license for more info.

WARNING: the passed record will be affected. So if you want to preserve the original point list, you must duplicate it first.