data

Loading images and other data from the file-system.

Package Contents

Classes

ImageLoader	Loads images from the file-system.
LoadImagesModule	A Lightning data-module for loading images recursively from a file-system directory.

class data.ImageLoader

Loads images from the file-system.

image_directory :str

The image directory to load images recursively from.

image_size :Tuple[int, int]

The size (height, width) to resize all images to.

extension :str = jpg

The extension (without a leading period) that all image files must match.

rgb :bool = True

When true, images are always loaded as RGB. when false, they are loaded as grayscale.

batch_size :int = 16

How many images should be in a batch.

num_workers :int = 1

The number of workers for the data-loader.

load_images(self) → torch.utils.data.DataLoader

Load all images recursively from a directory.

Returns

a data-loader with all the images that match the extension recursively in a directory.

load_images_split_two(self, ratio_validation: float = 0.3) → Tuple[torch.utils.data.DataLoader, torch.utils.data.DataLoader]

Load all images recursively from a directory, and split into training and validation batches.

Parameters

ratio_validation – a number between 0 and 1 determining linearly how many elements belong in the validation set e.g. 0.4 would try and place 40% approximately of elements into the second partition.

Returns

the loaded images, split into training and validation data respectively.

load_images_split_three(self, ratio_validation: float = 0.3, ratio_test: float = 0.2) → Tuple[torch.utils.data.DataLoader, torch.utils.data.DataLoader, torch.utils.data.DataLoader]

Load all images recursively from a directory, and split into training, validation and test batches.

Parameters

• ratio_validation – a number between 0 and 1 determining linearly how many elements belong in the validation set e.g. 0.4 would try and place 40% approximately of elements into the second batch.

• ratio_test – a number between 0 and 1 determining linearly how many elements belong in the validation set e.g. 0.2 would try and place 20% approximately of elements into the third batch.

Returns

the loaded images, split into training and validation data and test data respectively.

class data.LoadImagesModule(image_directory: str, image_size: int, extension: str = 'jpg', rgb: bool = True, batch_size: int = 16, num_workers: int = 1, ratio_validation: float = 0.3, ratio_test: float = 0.2)

Bases: pytorch_lightning.LightningDataModule

A Lightning data-module for loading images recursively from a file-system directory.

prepare_data(self) → None

Overrides pl.LightningDataModule.

setup(self, stage: Optional[pytorch_lightning.trainer.states.TrainerFn] = None)

Overrides pl.LightningDataModule.

train_dataloader(self)

Overrides pl.LightningDataModule.

val_dataloader(self)

Overrides pl.LightningDataModule.

test_dataloader(self)

Overrides pl.LightningDataModule.

predict_dataloader(self)

Overrides pl.LightningDataModule.