hyrax.data_sets.hyrax_cifar_dataset

Attributes

logger

Classes

HyraxCifarBase	Base class for Hyrax Cifar datasets
HyraxCifarDataset	Map style CIFAR 10 dataset for Hyrax
HyraxCifarIterableDataset	Iterable style CIFAR 10 dataset for Hyrax

Module Contents

logger

class HyraxCifarBase(config: dict, data_location: pathlib.Path = None)

Base class for Hyrax Cifar datasets

data_location

training_data

cifar

id_width = 0

get_image(idx)

Get the image at the given index as a NumPy array.

get_label(idx)

Get the label at the given index.

get_index(idx)

Get the index of the item.

get_object_id(idx)

Get the object ID for the item.

ids()

This is the default IDs function you get when you derive from hyrax Dataset

Returns:

A generator yielding all the string IDs of the dataset.

Return type:

Generator[str]

class HyraxCifarDataset(config: dict, data_location: pathlib.Path = None)

Bases: HyraxCifarBase, hyrax.data_sets.data_set_registry.HyraxDataset, torch.utils.data.Dataset

Map style CIFAR 10 dataset for Hyrax

This is simply a version of CIFAR10 that is initialized using Hyrax config with a transformation that works well for example code.

We only use the training split in the data, because it is larger (50k images). Hyrax will then divide that into Train/test/Validate according to configuration.

__init__()

Overall initialization for all DataSets which saves the config

Subclasses of HyraxDataSet ought call this at the end of their __init__ like:

from hyrax.data_sets import HyraxDataset
from torch.utils.data import Dataset

class MyDataset(HyraxDataset, Dataset):
    def __init__(config):
        <your code>
        super().__init__(config)

If per tensor metadata is available, it is recommended that dataset authors create an astropy Table of that data, in the same order as their data and pass that metadata_table as shown below:

from hyrax.data_sets import HyraxDataset
from torch.utils.data import Dataset
from astropy.table import Table

class MyDataset(HyraxDataset, Dataset):
    def __init__(config):
        <your code>
        metadata_table = Table(<Your catalog data goes here>)
        super().__init__(config, metadata_table)

Parameters:

• config (dict, Optional) – The runtime configuration for hyrax

• metadata_table (Optional[Table], optional) – An Astropy Table with 1. the metadata columns desired for visualization AND 2. in the order your data will be enumerated.

• object_id_column_name (Optional[str], optional) – The name of the column containing object IDs. If None, uses the default from config or creates one from the ids() method.

__len__()

__getitem__(idx)

class HyraxCifarIterableDataset(config: dict, data_location: pathlib.Path = None)

Bases: HyraxCifarBase, hyrax.data_sets.data_set_registry.HyraxDataset, torch.utils.data.IterableDataset

Iterable style CIFAR 10 dataset for Hyrax

This is simply a version of CIFAR10 that is initialized using Hyrax config with a transformation that works well for example code. This version only supports iteration, and not map-style access

We only use the training split in the data, because it is larger (50k images). Hyrax will then divide that into Train/test/Validate according to configuration.

__init__()

Overall initialization for all DataSets which saves the config

Subclasses of HyraxDataSet ought call this at the end of their __init__ like:

from hyrax.data_sets import HyraxDataset
from torch.utils.data import Dataset

class MyDataset(HyraxDataset, Dataset):
    def __init__(config):
        <your code>
        super().__init__(config)

If per tensor metadata is available, it is recommended that dataset authors create an astropy Table of that data, in the same order as their data and pass that metadata_table as shown below:

from hyrax.data_sets import HyraxDataset
from torch.utils.data import Dataset
from astropy.table import Table

class MyDataset(HyraxDataset, Dataset):
    def __init__(config):
        <your code>
        metadata_table = Table(<Your catalog data goes here>)
        super().__init__(config, metadata_table)

Parameters:

• config (dict, Optional) – The runtime configuration for hyrax

• metadata_table (Optional[Table], optional) – An Astropy Table with 1. the metadata columns desired for visualization AND 2. in the order your data will be enumerated.

• object_id_column_name (Optional[str], optional) – The name of the column containing object IDs. If None, uses the default from config or creates one from the ids() method.

__iter__()