Vision processing mode

RAM Processing submodule of OCAtari

class ocatari.vision.game_objects.GameObject(x, y, w, h, *args)

The Parent Class of every detected object in the Atari games (Vision Processing mode)

property category

The Category of class name of the game object (e.g. Player, Ball, …).

Type:

str

property center

The center of the bounding box of the object.

Type:

(int, int)

closest_object(others)

Returns the closest object from others, based on manathan distance between the center of both objects.

Returns:

(Index, Object) from others

Return type:

int

property dx

current_x - previous_x.

Type:

int

Type:

The pixel movement correponding to

property dy

current_y - previous_y.

Type:

int

Type:

The pixel movement correponding to

property h

The height/vertical size of the object (in pixels).

Type:

int

property h_coords

History of coordinates, i.e. current (x, y) and previous (x, y) position.

Type:

[(int, int), (int, int)]

is_on_top(other)

Returns True if this and another gameobject overlap.

Returns:

True if objects overlap

Return type:

bool

manathan_distance(other)

Returns the manathan distance between the center of both objects.

Returns:

Manathan distance (in pixel)

Return type:

int

property properties

All the properties of the object in a list.

Returns:

The properties of the object.

Return type:

list

property w

The width/horizontal size of the object (in pixels).

Type:

int

property x

The x positional coordinate on the image (on the horizontal axis).

Type:

int

property xy

Both (x, y) positional coordinates in a tuple.

Type:

(int, int)

property xywh

The (x, y, w, h) positional and width coordinates.

Type:

(int, int, int, int)

property y

The y positional coordinate on the image (on the vertical axis).

Type:

int

ocatari.vision.utils.assert_in(observed, expected, tol)

Asserts if the observed point is equal to the expected one with a given tolerance. True if ||observed - expected|| <= tol, with || the maximum over the two dimensions.

Parameters:

• observed ((int, int)) – The observed value point (e.g. (x,y), (w,h))

• expected ((int, int)) – The expected value point (also (x,y), (w,h))

• tol (int or (int, int)) – A given tolerance.

Returns:

True if points within the tolerance

Return type:

bool

ocatari.vision.utils.color_analysis(image, bbox, exclude=[])

Returns a Counter of all the detected colors in the bounding box

Parameters:

• image (np.array) – The image to mark the point on

• bb ((int, int, int, int)) – The bouding box where to perform the color analysis

• exclude (list of (int, int, int)) – A list of color to exclude

Returns:

A collections.Counter of the detected colors

Return type:

list of (int, int, int)

ocatari.vision.utils.find_mc_objects(image, colors, size=None, tol_s=10, position=None, tol_p=2, min_distance=10, closing_active=True, closing_dist=3, minx=0, miny=0, maxx=160, maxy=210, all_colors=True)

Finds the multicolors objects in the image.

This functions is used to detect object in e.g. Atlantis (depicted bellow).

Parameters:

• image (np.array) – The image to mark the point on

• colors (list of (int, int, int)) – The colors of the object

• size (int or (int, int)) – presupposed size of the targeted object (to detect)

• tol_s (int or (int, int)) – tolerance on the presupposed size of the targeted object

• size – presupposed size of the targeted object (to detect)

• tol_s – tolerance on the presupposed size of the targeted object

• position (int or (int, int)) – presupposed position of the targeted object (to detect)

• tol_p (int or (int, int)) – tolerance on the presupposed position of the targeted object

• min_distance (int) – tolerance on the presupposed position of the targeted object

• closing_active (bool) – If true, gathers in one bounding box the instances that are less than closing_dist

• closing_dist (int) – The closing distance, for the under which two (or more) instances are merged into one bounding box.

• minx (int) – minimum x position where the object can be located

• miny (int) – minimum y position where the object can be located

• maxx (int) – maximum x position where the object can be located

• maxy (int) – maximum y position where the object can be located

• all_colors (bool) – If True, only return the object if every given color in colors is present in the image

Returns:

a list of tuple boxing boxes

Return type:

list of (int, int, int, int)

ocatari.vision.utils.find_objects(image, color, size=None, tol_s=10, position=None, tol_p=2, min_distance=10, closing_active=True, closing_dist=3, minx=0, miny=0, maxx=160, maxy=210)

Finds the single colored objects in the image.

Parameters:

• image (np.array) – The image to mark the point on

• color (list of (int, int, int)) – The color of the object

• size (int or (int, int)) – presupposed size of the targeted object (to detect)

• tol_s (int or (int, int)) – tolerance on the presupposed size of the targeted object

• position (int or (int, int)) – presupposed position of the targeted object (to detect)

• tol_p (int or (int, int)) – tolerance on the presupposed position of the targeted object

• closing_active (bool) – If true, gathers in one bounding box the instances that are less than closing_dist away.

• closing_dist (int) – The closing distance, for the under which two (or more) instances are merged into one bounding box.

• minx (int) – minimum x position where the object can be located

• miny (int) – minimum y position where the object can be located

• maxx (int) – maximum x position where the object can be located

• maxy (int) – maximum y position where the object can be located

Returns:

a list of tuple boxing boxes

Return type:

list of (int, int, int)

ocatari.vision.utils.find_objects_in_color_range(image, color_min, color_max, size=None, tol_s=10, position=None, tol_p=2, min_distance=10, closing_active=True, closing_dist=3, minx=0, miny=0, maxx=160, maxy=210)

Finds the single colored objects in the image.

Parameters:

• image (np.array) – image to mark the point on

• color_min ((int, int, int)) – lower bound of the color spectrum

• color_max ((int, int, int)) – upper bound of the color spectrum

• size (int or (int, int)) – presupposed size of the targeted object (to detect)

• tol_s (int or (int, int)) – tolerance on the presupposed size of the targeted object

• size – presupposed size of the targeted object (to detect)

• tol_s – tolerance on the presupposed size of the targeted object

• position (int or (int, int)) – presupposed position of the targeted object (to detect)

• tol_p (int or (int, int)) – tolerance on the presupposed position of the targeted object

• closing_active (bool) – If true, gathers in one bounding box the instances that are less than closing_dist

• closing_dist (int) – The closing distance, for the under which two (or more) instances are merged into one bounding box.

• minx (int) – minimum x position where the object can be located

• miny (int) – minimum y position where the object can be located

• maxx (int) – maximum x position where the object can be located

• maxy (int) – maximum y position where the object can be located

Returns:

a list of tuple boxing boxes

Return type:

list of (int, int, int)

ocatari.vision.utils.find_rectangle_objects(image, color, max_size=None, minx=0, miny=0, maxx=160, maxy=210)

Finds rectangle objects with a given maximum size.

Parameters:

• image (RGB np.array) – The image in which the objects are displayed

• color (list of (int, int, int)) – The color of the object

• max_size ((int, int)) – The maximum size of the objects

• miny (int) – minimum y position where the object can be located

• maxx (int) – maximum x position where the object can be located

• maxy (int) – maximum y position where the object can be located

Returns:

a list of tuple boxing boxes

Return type:

list of (int, int, int)

ocatari.vision.utils.find_rope_segments(image, color, seg_height=(2, 5), minx=0, miny=0, maxx=160, maxy=210)

Finds the rope segments (max rope width of 1) of a displayed rope.

Parameters:

• image (np.array) – The image to mark the point on

• color (list of (int, int, int)) – The color of the object

• seg_height (int or (int, int)) – interval in which segments are considered

• minx (int) – minimum x position where the object can be located

• miny (int) – minimum y position where the object can be located

• maxx (int) – maximum x position where the object can be located

• maxy (int) – maximum y position where the object can be located

Returns:

a list of tuple boxing boxes

Return type:

list of (int, int, int)

ocatari.vision.utils.iou(bb, gt_bb)

Computes the intersection over union between two bounding boxes.

Parameters:

• bb ((int, int, int, int)) – The bouding box of the detected object in (x, y, w, h) format

• gt_bb ((int, int, int, int)) – The ground truth bouding box

ocatari.vision.utils.make_darker(color, col_precent=0.8)

Return a darker color.

Parameters:

• color ((int, int, int)) – upper bound of the color spectrum

• col_precent (float) – the luminosity reduction coefficient (between 0 and 1)

Returns:

the darker color

Return type:

(int, int, int)

ocatari.vision.utils.mark_bb(image_array, bb, color=(255, 0, 0), surround=True)

Marks a bounding box on the image.

Parameters:

• image_array (RGB np.array) – The image to mark the point on

• bb ((int, int, int, int)) – The bouding box of the detected object in (x, y, w, h) format

• color ((int, int, int)) – The rgb values of the point.

• surround (bool) – If True, place the bouding box with an offset of 1 pixel to surround the object

ocatari.vision.utils.mark_point(image_array, x, y, color=(255, 0, 0), size=1, show=False, cross=True)

Marks a point on the image at the (x,y) position (inplace)

Parameters:

• image_array (RGB np.array) – The image to mark the point on

• x (int) – The x coordinate of the point

• y (int) – The y coordinate of the point

• color ((int, int, int)) – The rgb values of the point.

• size (int) – The size of the point

• show (bool) – If True, shows the image using matplotlib

• cross (bool) – If True, places a diagonal cross, else place a square

ocatari.vision.utils.most_common_color(image, exclude_black=True)

Returns the most common color in the image.

Parameters:

exclude_black (bool) – If True, exclude the black color from the taken into account, defaults to True

ocatari.vision.utils.showim(image)

Display the given in a matplolib.pyplot plot.

Parameters:

image (np.array) – The image to mark the point on