datasetinsights.io

datasetinsights.io.bbox

class datasetinsights.io.bbox.BBox2D(label, x, y, w, h, score=1.0)

Bases: object

Canonical Representation of a 2D bounding box.

label

string representation of the label.

Type

str

x

x pixel coordinate of the upper left corner.

Type

float

y

y pixel coordinate of the upper left corner.

Type

float

w

width (number of pixels)of the bounding box.

Type

float

h

height (number of pixels) of the bounding box.

Type

float

score

detection confidence score. Default is set to score=1. if this is a ground truth bounding box.

Type

float

Examples

Here is an example about how to use this class.

>>> gt_bbox = BBox2D(label='car', x=2, y=6, w=2, h=4)
>>> gt_bbox
"label='car'|score=1.0|x=2.0|y=6.0|w=2.0|h=4.0"
>>> pred_bbox = BBox2D(label='car', x=2, y=5, w=2, h=4, score=0.79)
>>> pred_bbox.area
8
>>> pred_bbox.intersect_with(gt_bbox)
True
>>> pred_bbox.intersection(gt_bbox)
6
>>> pred_bbox.union(gt_bbox)
10
>>> pred_bbox.iou(gt_bbox)
0.6
property area

Calculate area of this bounding box

Returns

width x height of the bound box

intersect_with(other)

Check whether this box intersects with other bounding box

Parameters

other (BBox2D) – other bounding box object to check intersection

Returns

True if two bounding boxes intersect, False otherwise

intersection(other)

Calculate the intersection area with other bounding box

Parameters

other (BBox2D) – other bounding box object to calculate intersection

Returns

float of the intersection area for two bounding boxes

iou(other)

Calculate intersection over union area with other bounding box

\[IOU = \frac{intersection}{union}\]
Parameters

other (BBox2D) – other bounding box object to calculate iou

Returns

float of the union area for two bounding boxes

union(other, intersection_area=None)

Calculate union area with other bounding box

Parameters

  • other (BBox2D) – other bounding box object to calculate union

  • intersection_area (float) – pre-calculated area of intersection

Returns

float of the union area for two bounding boxes

class datasetinsights.io.bbox.BBox3D(translation, size, label, sample_token, score=1, rotation: pyquaternion.quaternion.Quaternion = Quaternion(1.0, 0.0, 0.0, 0.0), velocity=(nan, nan, nan))

Bases: object

Class for 3d bounding boxes which can either be predictions or ground-truths. This class is the primary representation in this repo of 3d bounding boxes and is based off of the Nuscenes style dataset.

property back_left_bottom_pt

Back-left-bottom point.

Type

Returns

Type

float

property back_left_top_pt

Back-left-top point.

Type

float

property back_right_bottom_pt

Back-right-bottom point.

Type

float

property back_right_top_pt

Back-right-top point.

Type

float

property front_left_bottom_pt

Front-left-bottom point.

Type

float

property front_left_top_pt

Front-left-top point.

Type

float

property front_right_bottom_pt

Front-right-bottom point.

Type

float

property front_right_top_pt

Front-right-top point.

Type

float

property p
list of all 8 corners of the box beginning with the the bottom

four corners and then the top

four corners, both in counterclockwise order (from birds eye view) beginning with the back-left corner

Type

Returns

datasetinsights.io.bbox.group_bbox2d_per_label(bboxes)

Group 2D bounding boxes with same label.

Parameters

bboxes (list[BBox2D]) – a list of 2D bounding boxes

Returns

a dictionary of 2d boundign box group. {label1: [bbox1, bboxes2, …], label2: [bbox1, …]}

Return type

dict

datasetinsights.io.download

class datasetinsights.io.download.TimeoutHTTPAdapter(timeout, *args, **kwargs)

Bases: requests.adapters.HTTPAdapter

send(request, **kwargs)

Sends PreparedRequest object. Returns Response object.

Parameters

  • request – The PreparedRequest being sent.

  • stream – (optional) Whether to stream the request content.

  • timeout (float or tuple or urllib3 Timeout object) – (optional) How long to wait for the server to send data before giving up, as a float, or a (connect timeout, read timeout) tuple.

  • verify – (optional) Either a boolean, in which case it controls whether we verify the server’s TLS certificate, or a string, in which case it must be a path to a CA bundle to use

  • cert – (optional) Any user-provided SSL certificate to be trusted.

  • proxies – (optional) The proxies dictionary to apply to the request.

Return type

requests.Response

datasetinsights.io.download.checksum_matches(filepath, expected_checksum, algorithm='CRC32')

Check if the checksum matches

Parameters

  • filepath (str) – the doaloaded file path

  • expected_checksum (int) – expected checksum of the file

  • algorithm (str) – checksum algorithm. Defaults to CRC32

Returns

True if the file checksum matches.

datasetinsights.io.download.compute_checksum(filepath, algorithm='CRC32')

Compute the checksum of a file.

Parameters

  • filepath (str) – the doaloaded file path

  • algorithm (str) – checksum algorithm. Defaults to CRC32

Returns

the checksum value

Return type

int

datasetinsights.io.download.download_file(source_uri: str, dest_path: str, file_name: Optional[str] = None)

Download a file specified from a source uri

Parameters

  • source_uri (str) – source url where the file should be downloaded

  • dest_path (str) – destination path of the file

  • file_name (str) – file name of the file to be downloaded

Returns

String of destination path.

datasetinsights.io.download.get_checksum_from_file(filepath)

This method return checksum of the file whose filepath is given.

Parameters

filepath (str) – Path of the checksum file. Path can be HTTP(s) url or local path.

Raises

ValueError – Raises this error if filepath is not local or not HTTP or HTTPS url.

datasetinsights.io.download.validate_checksum(filepath, expected_checksum, algorithm='CRC32')

Validate checksum of the downloaded file.

Parameters

  • filepath (str) – the doaloaded file path

  • expected_checksum (int) – expected checksum of the file

  • algorithm (str) – checksum algorithm. Defaults to CRC32

Raises

ChecksumError if the file checksum does not match. –

datasetinsights.io.exceptions

exception datasetinsights.io.exceptions.ChecksumError

Bases: Exception

Raises when the downloaded file checksum is not correct.

exception datasetinsights.io.exceptions.DownloadError

Bases: Exception

Raise when download file failed.

exception datasetinsights.io.exceptions.InvalidTrackerError

Bases: Exception

Raises when unknown tracker requested .

datasetinsights.io.gcs

class datasetinsights.io.gcs.GCSClient(**kwargs)

Bases: object

This class is used to download data from GCS location and perform function such as downloading the dataset and checksum validation.

GCS_PREFIX = '^gs://'
KEY_SEPARATOR = '/'
download(*, url=None, local_path=None, bucket=None, key=None)

This method is used to download the dataset from GCS.

Parameters

  • url (str) – This is the downloader-uri that indicates where the dataset should be downloaded from.

  • local_path (str) – This is the path to the directory where the download will store the dataset.

  • bucket (str) – gcs bucket name

  • key (str) – object key path

  • Examples – 

    >>> url = "gs://bucket/folder or gs://bucket/folder/data.zip"
>>> local_path = "/tmp/folder"
>>> bucket ="bucket"
>>> key ="folder/data.zip" or "folder"

get_most_recent_blob(url=None, bucket_name=None, key=None)

Get the last updated blob in a given bucket under given prefix

Parameters

  • bucket_name (str) – gcs bucket name

  • key (str) – object key path

upload(*, local_path=None, bucket=None, key=None, url=None, pattern='*')

Upload a file or list of files from directory to GCS

Parameters

  • url (str) – This is the gcs location that indicates where

  • dataset should be uploaded. (the) –

  • local_path (str) – This is the path to the directory or file

  • the data is stored. (where) –

  • bucket (str) – gcs bucket name

  • key (str) – object key path

  • pattern – Unix glob patterns. Use **/* for recursive glob.

  • Examples –

    For file upload:
    >>> url = "gs://bucket/folder/data.zip"
>>> local_path = "/tmp/folder/data.zip"
>>> bucket ="bucket"
>>> key ="folder/data.zip"
    For directory upload:
    >>> url = "gs://bucket/folder"
>>> local_path = "/tmp/folder"
>>> bucket ="bucket"
>>> key ="folder"
>>> key ="**/*"

class datasetinsights.io.BBox2D(label, x, y, w, h, score=1.0)

Bases: object

Canonical Representation of a 2D bounding box.

label

string representation of the label.

Type

str

x

x pixel coordinate of the upper left corner.

Type

float

y

y pixel coordinate of the upper left corner.

Type

float

w

width (number of pixels)of the bounding box.

Type

float

h

height (number of pixels) of the bounding box.

Type

float

score

detection confidence score. Default is set to score=1. if this is a ground truth bounding box.

Type

float

Examples

Here is an example about how to use this class.

>>> gt_bbox = BBox2D(label='car', x=2, y=6, w=2, h=4)
>>> gt_bbox
"label='car'|score=1.0|x=2.0|y=6.0|w=2.0|h=4.0"
>>> pred_bbox = BBox2D(label='car', x=2, y=5, w=2, h=4, score=0.79)
>>> pred_bbox.area
8
>>> pred_bbox.intersect_with(gt_bbox)
True
>>> pred_bbox.intersection(gt_bbox)
6
>>> pred_bbox.union(gt_bbox)
10
>>> pred_bbox.iou(gt_bbox)
0.6
property area

Calculate area of this bounding box

Returns

width x height of the bound box

intersect_with(other)

Check whether this box intersects with other bounding box

Parameters

other (BBox2D) – other bounding box object to check intersection

Returns

True if two bounding boxes intersect, False otherwise

intersection(other)

Calculate the intersection area with other bounding box

Parameters

other (BBox2D) – other bounding box object to calculate intersection

Returns

float of the intersection area for two bounding boxes

iou(other)

Calculate intersection over union area with other bounding box

\[IOU = \frac{intersection}{union}\]
Parameters

other (BBox2D) – other bounding box object to calculate iou

Returns

float of the union area for two bounding boxes

union(other, intersection_area=None)

Calculate union area with other bounding box

Parameters

  • other (BBox2D) – other bounding box object to calculate union

  • intersection_area (float) – pre-calculated area of intersection

Returns

float of the union area for two bounding boxes

datasetinsights.io.create_dataset_downloader(source_uri, **kwargs)
This function instantiates the dataset downloader

after finding it with the source-uri provided

Parameters

  • source_uri – URI used to look up the correct dataset downloader

  • **kwargs –

Returns: The dataset downloader instance matching the source-uri.