cheesechaser.datapool.yande

This module provides data pool implementations for Yande image data.

It extends the IncrementIDDataPool class to specifically handle Yande image data, utilizing predefined Hugging Face repositories for data storage and indexing.

The module includes two main classes: 1. YandeDataPool: For managing original Yande image data. 2. YandeWebpDataPool: For managing WebP-formatted Yande image data.

Note

The datasets deepghs/yande_full and deepghs/yande-webp-4Mpixel are gated, you have to get access to them before using this module.

YandeDataPool

class cheesechaser.datapool.yande.YandeDataPool(revision: str = 'main', hf_token: str | None = None)[source]

A data pool class for managing Yande image data.

This class extends IncrementIDDataPool to provide a specialized implementation for handling Yande image data. It uses a predefined Hugging Face repository for both data storage and indexing.

Parameters:

  • revision (str) – The revision of the data to use, defaults to ‘main’.

  • hf_token (Optional[str]) – Optional Hugging Face authentication token.

Variables:

  • data_repo_id – The Hugging Face repository ID for data storage.

  • idx_repo_id – The Hugging Face repository ID for indexing.

Usage:
>>> yande_pool = YandeDataPool()
>>> yande_pool_with_token = YandeDataPool(hf_token='your_token_here')
__init__(revision: str = 'main', hf_token: str | None = None)[source]

Initialize the YandeDataPool.

Parameters:

  • revision (str) – The revision of the data to use, defaults to ‘main’.

  • hf_token (Optional[str]) – Optional Hugging Face authentication token for accessing private repositories.

This constructor initializes the YandeDataPool by calling the parent class constructor with specific parameters for the Yande data repository. It sets up both the data and index repositories to use the same Hugging Face repository and revision.

YandeWebpDataPool

class cheesechaser.datapool.yande.YandeWebpDataPool(revision: str = 'main', hf_token: str | None = None)[source]

A data pool class for managing WebP-formatted Yande image data.

This class extends IncrementIDDataPool to provide a specialized implementation for handling WebP-formatted Yande image data. It uses a predefined Hugging Face repository for both data storage and indexing.

Parameters:

  • revision (str) – The revision of the data to use, defaults to ‘main’.

  • hf_token (Optional[str]) – Optional Hugging Face authentication token.

Variables:

  • data_repo_id – The Hugging Face repository ID for data storage.

  • idx_repo_id – The Hugging Face repository ID for indexing.

Usage:
>>> yande_webp_pool = YandeWebpDataPool()
>>> yande_webp_pool_with_token = YandeWebpDataPool(hf_token='your_token_here')
__init__(revision: str = 'main', hf_token: str | None = None)[source]

Initialize the YandeWebpDataPool.

Parameters:

  • revision (str) – The revision of the data to use, defaults to ‘main’.

  • hf_token (Optional[str]) – Optional Hugging Face authentication token for accessing private repositories.

This constructor initializes the YandeWebpDataPool by calling the parent class constructor with specific parameters for the WebP-formatted Yande data repository. It sets up both the data and index repositories to use the same Hugging Face repository and revision.