cheesechaser.datapool.danbooru

This module provides data pool classes for managing and accessing Danbooru image datasets.

It includes classes for handling both original and WebP versions of Danbooru images, as well as classes for accessing the newest additions to the dataset. The module utilizes Hugging Face’s file system for data storage and retrieval.

Classes:

DanbooruDataPool: Main class for accessing Danbooru image data.
DanbooruStableDataPool: Class for accessing a stable version of Danbooru data.
DanbooruNewestDataPool: Class for accessing both stable and newest Danbooru data.
DanbooruWebpDataPool: Class for accessing WebP versions of Danbooru images.
DanbooruNewestWebpDataPool: Class for accessing both stable and newest WebP Danbooru data.

The module uses various utility functions and classes from the hfutils package for interacting with the Hugging Face file system.

DanbooruDataPool

class cheesechaser.datapool.danbooru.DanbooruDataPool(data_revision: str = 'main', idx_revision: str = 'main')[source]

Main class for accessing Danbooru image data.

This class provides access to the Danbooru dataset using default repository IDs.

Parameters:

data_revision (str) – The revision of the data repository to use. Defaults to ‘main’.
idx_revision (str) – The revision of the index repository to use. Defaults to ‘main’.

__init__(data_revision: str = 'main', idx_revision: str = 'main')[source]

DanbooruStableDataPool

class cheesechaser.datapool.danbooru.DanbooruStableDataPool[source]

Class for accessing a stable version of Danbooru data.

This class uses specific revisions of the data and index repositories to provide access to a stable version of the Danbooru dataset.

__init__()[source]

DanbooruNewestDataPool

class cheesechaser.datapool.danbooru.DanbooruNewestDataPool[source]

Class for accessing both stable and newest Danbooru data.

This class combines access to both the stable Danbooru dataset and the newest additions, providing a comprehensive view of the data.

__init__()[source]

mock_resource(resource_id, resource_info) → AbstractContextManager[Tuple[str, Any]][source]

Provide a context manager for accessing a resource.

This method attempts to retrieve the resource from both the stable and newest data pools, returning the first successful match.

Parameters:

resource_id (Any) – The ID of the resource to retrieve.
resource_info (Any) – Additional information about the resource.

Returns:

A context manager yielding a tuple of (temporary directory, resource info).

Return type:

ContextManager[Tuple[str, Any]]

Raises:

ResourceNotFoundError – If the resource is not found in either pool.

DanbooruWebpDataPool

class cheesechaser.datapool.danbooru.DanbooruWebpDataPool(data_revision: str = 'main', idx_revision: str = 'main')[source]

Class for accessing WebP versions of Danbooru images.

This class provides access to WebP-formatted Danbooru images, which are typically smaller in file size while maintaining good quality.

Parameters:

data_revision (str) – The revision of the data repository to use. Defaults to ‘main’.
idx_revision (str) – The revision of the index repository to use. Defaults to ‘main’.

__init__(data_revision: str = 'main', idx_revision: str = 'main')[source]

DanbooruNewestWebpDataPool

class cheesechaser.datapool.danbooru.DanbooruNewestWebpDataPool[source]

Class for accessing both stable and newest WebP Danbooru data.

This class combines access to both the stable WebP-formatted Danbooru dataset and the newest WebP additions, providing a comprehensive view of the WebP data.

__init__()[source]

mock_resource(resource_id, resource_info) → AbstractContextManager[Tuple[str, Any]][source]

Provide a context manager for accessing a WebP resource.

This method attempts to retrieve the WebP resource from both the stable and newest WebP data pools, returning the first successful match.

Parameters:

resource_id (Any) – The ID of the resource to retrieve.
resource_info (Any) – Additional information about the resource.

Returns:

A context manager yielding a tuple of (temporary directory, resource info).

Return type:

ContextManager[Tuple[str, Any]]

Raises:

ResourceNotFoundError – If the resource is not found in either WebP pool.