pickle_dataset

PickleDataset

Bases: InMemoryDataset

A dataset from a pickle file.

PickleDataset reads entries from pickled pandas data-frames. The root directory of the pickle file may be accessed using dataset.parent_path. This may be useful if the file contains relative path information that you want to feed into, say, an ImageReader Op.

Parameters:

Name	Type	Description	Default
file_path	str	The (absolute) path to the pickle file.	required

Source code in fastestimator/fastestimator/dataset/pickle_dataset.py

@traceable()
class PickleDataset(InMemoryDataset):
    """A dataset from a pickle file.

    PickleDataset reads entries from pickled pandas data-frames. The root directory of the pickle file may be accessed
    using dataset.parent_path. This may be useful if the file contains relative path information that you want to feed
    into, say, an ImageReader Op.

    Args:
        file_path: The (absolute) path to the pickle file.
    """
    def __init__(self, file_path: str) -> None:
        df = pd.read_pickle(file_path)
        self.parent_path = os.path.dirname(file_path)
        super().__init__(df.to_dict(orient='index'))