cheesechaser.datapool.rule34

This module provides data pool classes for accessing Rule34 image data.

It contains two classes:

  1. Rule34DataPool: For accessing the full Rule34 dataset.

  2. Rule34WebpDataPool: For accessing the WebP-formatted Rule34 dataset with 4M pixel images.

Both classes inherit from IncrementIDDataPool and provide easy access to the respective datasets stored in Hugging Face repositories. These classes simplify the process of retrieving and working with Rule34 image data, allowing users to easily integrate this data into their projects or research.

Note

The datasets deepghs/rule34_full and deepghs/rule34-webp-4Mpixel is gated, you have to get the access of it before using this module.

Rule34DataPool

class cheesechaser.datapool.rule34.Rule34DataPool(revision: str = 'main', hf_token: str | None = None)[source]

A data pool class for accessing the full Rule34 dataset.

This class inherits from IncrementIDDataPool and is configured to access the full Rule34 dataset stored in the ‘deepghs/rule34_full’ repository. It provides methods to retrieve image data based on image IDs.

Parameters:

revision (str) – The revision of the dataset to use, defaults to ‘main’.

Note:

This class uses a base level of 4 for file organization, which means the images are stored in a directory structure with 4 levels of subdirectories.

__init__(revision: str = 'main', hf_token: str | None = None)[source]

Initialize the Rule34DataPool.

Parameters:

  • revision (str) – The revision of the dataset to use, defaults to ‘main’.

  • hf_token (Optional[str]) – Hugging Face authentication token, defaults to None.

Rule34WebpDataPool

class cheesechaser.datapool.rule34.Rule34WebpDataPool(revision: str = 'main', hf_token: str | None = None)[source]

A data pool class for accessing the WebP-formatted Rule34 dataset with 4M pixel images.

This class inherits from IncrementIDDataPool and is configured to access the WebP-formatted Rule34 dataset stored in the ‘deepghs/rule34-webp-4Mpixel’ repository. It provides methods to retrieve WebP-formatted image data based on image IDs.

Parameters:

  • revision (str) – The revision of the dataset to use, defaults to ‘main’.

  • hf_token (Optional[str]) – Hugging Face authentication token, defaults to None.

Note:

This class uses a base level of 3 for file organization, which means the images are stored in a directory structure with 3 levels of subdirectories. Authentication may be required to access this dataset.

__init__(revision: str = 'main', hf_token: str | None = None)[source]

Initialize the Rule34WebpDataPool.

Parameters:

  • revision (str) – The revision of the dataset to use, defaults to ‘main’.

  • hf_token (Optional[str]) – Hugging Face authentication token, defaults to None.