cheesechaser.datapool.e621

This module provides classes for accessing and managing E621 image data.

It includes classes for handling both original and WebP-formatted images, as well as stable and newest datasets. The module uses Hugging Face’s file system for data storage and retrieval.

Classes:

  • BaseE621DataPool: Base class for E621 data pools.

  • E621DataPool: Main class for accessing E621 image data.

  • E621StableDataPool: Class for accessing a stable version of E621 data.

  • E621NewestPartialDataPool: Class for accessing the newest partial E621 data.

  • E621NewestDataPool: Class for accessing both stable and newest E621 data.

  • E621WebpDataPool: Class for accessing WebP versions of E621 images.

  • E621NewestPartialWebpDataPool: Class for accessing the newest partial WebP E621 data.

  • E621NewestWebpDataPool: Class for accessing both stable and newest WebP E621 data.

The module uses several constants to define default repository IDs and revisions.

E621DataPool

class cheesechaser.datapool.e621.E621DataPool(data_revision: str = 'main', idx_revision: str = 'main', hf_token: str | None = None)[source]

Main class for accessing E621 image data.

This class provides access to the E621 dataset using default repository IDs.

Parameters:

  • data_revision (str) – The revision of the data repository to use. Defaults to ‘main’.

  • idx_revision (str) – The revision of the index repository to use. Defaults to ‘main’.

  • hf_token (Optional[str]) – Optional Hugging Face token for authentication.

__init__(data_revision: str = 'main', idx_revision: str = 'main', hf_token: str | None = None)[source]

E621StableDataPool

class cheesechaser.datapool.e621.E621StableDataPool(hf_token: str | None = None)[source]

Class for accessing a stable version of E621 data.

This class uses specific revisions of the data and index repositories to provide access to a stable version of the E621 dataset.

Parameters:

hf_token (Optional[str]) – Optional Hugging Face token for authentication.

__init__(hf_token: str | None = None)[source]

E621NewestDataPool

class cheesechaser.datapool.e621.E621NewestDataPool(hf_token: str | None = None)[source]

Class for accessing both stable and newest E621 data.

This class combines access to both the stable E621 dataset and the newest additions, providing a comprehensive view of the data.

Parameters:

hf_token (Optional[str]) – Optional Hugging Face token for authentication.

__init__(hf_token: str | None = None)[source]
mock_resource(resource_id, resource_info, silent: bool = False) → AbstractContextManager[Tuple[str, Any]][source]

Provide a context manager for accessing a resource.

This method attempts to retrieve the resource from both the stable and newest data pools, returning the first successful match.

Parameters:

  • resource_id (Any) – The ID of the resource to retrieve.

  • resource_info (Any) – Additional information about the resource.

  • silent (bool) – If True, suppresses progress bar of each standalone files during the mocking process.

Returns:

A context manager yielding a tuple of (temporary directory, resource info).

Return type:

ContextManager[Tuple[str, Any]]

Raises:

ResourceNotFoundError – If the resource is not found in either pool.

E621WebpDataPool

class cheesechaser.datapool.e621.E621WebpDataPool(data_revision: str = 'main', idx_revision: str = 'main', hf_token: str | None = None)[source]

Class for accessing WebP versions of E621 images.

This class provides access to WebP-formatted E621 images, which are typically smaller in file size while maintaining good quality.

Parameters:

  • data_revision (str) – The revision of the data repository to use. Defaults to ‘main’.

  • idx_revision (str) – The revision of the index repository to use. Defaults to ‘main’.

  • hf_token (Optional[str]) – Optional Hugging Face token for authentication.

__init__(data_revision: str = 'main', idx_revision: str = 'main', hf_token: str | None = None)[source]

E621NewestWebpDataPool

class cheesechaser.datapool.e621.E621NewestWebpDataPool(hf_token: str | None = None)[source]

Class for accessing both stable and newest WebP E621 data.

This class combines access to both the stable WebP-formatted E621 dataset and the newest WebP additions, providing a comprehensive view of the WebP data.

Parameters:

hf_token (Optional[str]) – Optional Hugging Face token for authentication.

__init__(hf_token: str | None = None)[source]
mock_resource(resource_id, resource_info, silent: bool = False) → AbstractContextManager[Tuple[str, Any]][source]

Provide a context manager for accessing a WebP resource.

This method attempts to retrieve the WebP resource from both the stable and newest WebP data pools, returning the first successful match.

Parameters:

  • resource_id (Any) – The ID of the resource to retrieve.

  • resource_info (Any) – Additional information about the resource.

  • silent (bool) – If True, suppresses progress bar of each standalone files during the mocking process.

Returns:

A context manager yielding a tuple of (temporary directory, resource info).

Return type:

ContextManager[Tuple[str, Any]]

Raises:

ResourceNotFoundError – If the resource is not found in either WebP pool.