API documentation

Checkpoints

This module provides a CheckpointManager class for managing the saving and loading of checkpoints during PyTorch model training.

The CheckpointManager handles:

  • Saving and loading the state of models, optimizers, and metrics.

  • Registering and evaluating performance criteria to determine if a model’s performance has improved, enabling automated saving of the best-performing model checkpoints.

  • Resuming training from a specific checkpoint.

Usage:

  1. Initialize the CheckpointManager with a PyTorch model, optimizer, and metric manager.

  2. Register criteria for tracking and evaluating metrics.

  3. Use the save and load methods to manage checkpoints during training.

  4. Call evaluate_criteria to automatically evaluate and save the best-performing checkpoints.

Dependencies:

  • PyTorch (torch)

class congrads.checkpoints.CheckpointManager(network: Module, optimizer: Optimizer, metric_manager: MetricManager, save_dir: str = 'checkpoints', create_dir: bool = False)

Bases: object

A class to handle saving and loading checkpoints for PyTorch models and optimizers.

Parameters:

  • network (torch.nn.Module) – The network (model) to save/load.

  • optimizer (torch.optim.Optimizer) – The optimizer to save/load.

  • metric_manager (MetricManager) – The metric manager to restore saved metric states.

  • save_dir (str) – Directory where checkpoints will be saved. Defaults to ‘checkpoints’.

  • create_dir (bool) – Whether to create the save_dir if it does not exist. Defaults to False.

Raises:

  • TypeError – If a provided attribute has an incompatible type.

  • FileNotFoundError – If the save directory does not exist and create_dir is set to False.

evaluate_criteria(epoch: int)

Evaluate the defined criteria for model performance metrics during training.

Parameters:

epoch (int) – The current epoch number.

Compares the current metrics against the previous best metrics using predefined comparators. If a criterion is met, saves the model and the corresponding best metric values.

load(filename: str)

Load a checkpoint and restores the state of the network, optimizer and best_metrics.

Parameters:

filename (str) – Name of the checkpoint file.

Returns:

A dictionary containing the loaded checkpoint information (epoch, loss, etc.).

Return type:

dict

register(metric_name: str, comparator: ~typing.Callable[[~torch.Tensor, ~torch.Tensor], ~torch.Tensor] = <built-in method gt of type object>)

Register a criterion for evaluating a performance metric during training.

Stores the comparator to determine whether the current metric has improved relative to the previous best metric value.

Parameters:

  • metric_name (str) – The name of the metric to evaluate.

  • comparator (Callable[[Tensor, Tensor], Tensor], optional) – A function that compares the current metric value against the previous best value. Defaults to a greater-than (gt) comparison.

Raises:

TypeError – If a provided attribute has an incompatible type.

resume(filename: str = 'checkpoint.pth', ignore_missing: bool = False) → int

Resumes training from a saved checkpoint file.

Parameters:

  • filename (str) – The name of the checkpoint file to load. Defaults to “checkpoint.pth”.

  • ignore_missing (bool) – If True, does not raise an error if the checkpoint file is missing and continues without loading, starting from epoch 0. Defaults to False.

Returns:

The epoch number from the loaded checkpoint, or 0 if

ignore_missing is True and no checkpoint was found.

Return type:

int

Raises:

  • TypeError – If a provided attribute has an incompatible type.

  • FileNotFoundError – If the specified checkpoint file does not exist.

save(epoch: int, filename: str = 'checkpoint.pth')

Save a checkpoint.

Parameters:

  • epoch (int) – Current epoch number.

  • filename (str) – Name of the checkpoint file. Defaults to ‘checkpoint.pth’.

Constraints

This module provides a set of constraint classes for guiding neural network training by enforcing specific conditions on the network’s outputs.

The constraints in this module include:

  • Constraint: The base class for all constraint types, defining the interface and core behavior.

  • ImplicationConstraint: A constraint that enforces one condition only if another condition is met, useful for modeling implications between network outputs.

  • ScalarConstraint: A constraint that enforces scalar-based comparisons on a network’s output.

  • BinaryConstraint: A constraint that enforces a binary comparison between two neurons in the network, using a comparison function (e.g., less than, greater than).

  • SumConstraint: A constraint that enforces that the sum of certain neurons’ outputs equals a specified value, which can be used to control total output.

  • PythagoreanConstraint: A constraint that enforces the Pythagorean theorem on a set of neurons, ensuring that the square of one neuron’s output is equal to the sum of the squares of other outputs.

These constraints can be used to steer the learning process by applying conditions such as logical implications or numerical bounds.

Usage:

  1. Define a custom constraint class by inheriting from Constraint.

  2. Apply the constraint to your neural network during training to enforce desired output behaviors.

  3. Use the helper classes like IdentityTransformation for handling transformations and comparisons in constraints.

Dependencies:

  • PyTorch (torch)

class congrads.constraints.BinaryConstraint(operand_left: str | Transformation, comparator: Callable[[Tensor, Number], Tensor], operand_right: str | Transformation, name: str = None, monitor_only: bool = False, rescale_factor: Number = 1.5)

Bases: Constraint

A constraint that enforces a binary comparison between two neurons.

This class ensures that the output of one neuron satisfies a comparison operation with the output of another neuron (e.g., less than, greater than, etc.). It uses a comparator function to validate the condition and calculates adjustment directions accordingly.

Parameters:

  • operand_left (Union[str, Transformation]) – Name of the left neuron or a transformation to apply.

  • comparator (Callable[[Tensor, Number], Tensor]) – A comparison function (e.g., torch.ge, torch.lt).

  • operand_right (Union[str, Transformation]) – Name of the right neuron or a transformation to apply.

  • name (str, optional) – A unique name for the constraint. If not provided, a name is auto-generated in the format “<neuron_name_left> <comparator> <neuron_name_right>”.

  • monitor_only (bool, optional) – If True, only monitor the constraint without adjusting the loss. Defaults to False.

  • rescale_factor (Number, optional) – Factor to scale the constraint-adjusted loss. Defaults to 1.5.

Raises:

TypeError – If a provided attribute has an incompatible type.

Notes

  • The neuron names must be defined in the descriptor mapping.

  • The constraint name is composed using the left neuron name, comparator, and right neuron name.

calculate_direction(prediction: dict[str, Tensor]) → Dict[str, Tensor]

Calculates adjustment directions for neurons to better satisfy the constraint.

Parameters:

prediction (dict[str, Tensor]) – Model predictions for the neurons.

Returns:

Dictionary mapping neuron layers to tensors specifying the adjustment direction for each neuron.

Return type:

Dict[str, Tensor]

Raises:

NotImplementedError – If not implemented in a subclass.

check_constraint(prediction: dict[str, Tensor]) → tuple[Tensor, int]

Evaluates whether the given model predictions satisfy the constraint.

Parameters:

prediction (dict[str, Tensor]) – Model predictions for the neurons.

Returns:

A tuple where the first element is a tensor indicating whether the constraint is satisfied (with True for satisfaction, False for non-satisfaction, and torch.nan for irrelevant results), and the second element is an integer value representing the number of relevant constraints.

Return type:

tuple[Tensor, int]

Raises:

NotImplementedError – If not implemented in a subclass.

class congrads.constraints.Constraint(neurons: set[str], name: str = None, monitor_only: bool = False, rescale_factor: Number = 1.5)

Bases: ABC

Abstract base class for defining constraints applied to neural networks.

A Constraint specifies conditions that the neural network outputs should satisfy. It supports monitoring constraint satisfaction during training and can adjust loss to enforce constraints. Subclasses must implement the check_constraint and calculate_direction methods.

Parameters:

  • neurons (set[str]) – Names of the neurons this constraint applies to.

  • name (str, optional) – A unique name for the constraint. If not provided, a name is generated based on the class name and a random suffix.

  • monitor_only (bool, optional) – If True, only monitor the constraint without adjusting the loss. Defaults to False.

  • rescale_factor (Number, optional) – Factor to scale the constraint-adjusted loss. Defaults to 1.5. Should be greater than 1 to give weight to the constraint.

Raises:

  • TypeError – If a provided attribute has an incompatible type.

  • ValueError – If any neuron in neurons is not defined in the descriptor.

Note

  • If rescale_factor <= 1, a warning is issued.

  • If name is not provided, a name is auto-generated, and a warning is logged.

abstract calculate_direction(prediction: dict[str, Tensor]) → Dict[str, Tensor]

Calculates adjustment directions for neurons to better satisfy the constraint.

Parameters:

prediction (dict[str, Tensor]) – Model predictions for the neurons.

Returns:

Dictionary mapping neuron layers to tensors specifying the adjustment direction for each neuron.

Return type:

Dict[str, Tensor]

Raises:

NotImplementedError – If not implemented in a subclass.

abstract check_constraint(prediction: dict[str, Tensor]) → tuple[Tensor, int]

Evaluates whether the given model predictions satisfy the constraint.

Parameters:

prediction (dict[str, Tensor]) – Model predictions for the neurons.

Returns:

A tuple where the first element is a tensor indicating whether the constraint is satisfied (with True for satisfaction, False for non-satisfaction, and torch.nan for irrelevant results), and the second element is an integer value representing the number of relevant constraints.

Return type:

tuple[Tensor, int]

Raises:

NotImplementedError – If not implemented in a subclass.

descriptor: Descriptor = None
device = None
class congrads.constraints.ImplicationConstraint(head: Constraint, body: Constraint, name=None, monitor_only=False, rescale_factor=1.5)

Bases: Constraint

Represents an implication constraint between two constraints (head and body).

The implication constraint ensures that the body constraint only applies when the head constraint is satisfied. If the head constraint is not satisfied, the body constraint does not apply.

Parameters:

  • head (Constraint) – The head of the implication. If this constraint is satisfied, the body constraint must also be satisfied.

  • body (Constraint) – The body of the implication. This constraint is enforced only when the head constraint is satisfied.

  • name (str, optional) – A unique name for the constraint. If not provided, the name is generated in the format “{body.name} if {head.name}”. Defaults to None.

  • monitor_only (bool, optional) – If True, the constraint is only monitored without adjusting the loss. Defaults to False.

  • rescale_factor (Number, optional) – The scaling factor for the constraint-adjusted loss. Defaults to 1.5.

Raises:

TypeError – If a provided attribute has an incompatible type.

calculate_direction(prediction: dict[str, Tensor]) → Dict[str, Tensor]

Calculates adjustment directions for neurons to better satisfy the constraint.

Parameters:

prediction (dict[str, Tensor]) – Model predictions for the neurons.

Returns:

Dictionary mapping neuron layers to tensors specifying the adjustment direction for each neuron.

Return type:

Dict[str, Tensor]

Raises:

NotImplementedError – If not implemented in a subclass.

check_constraint(prediction: dict[str, Tensor]) → tuple[Tensor, int]

Evaluates whether the given model predictions satisfy the constraint.

Parameters:

prediction (dict[str, Tensor]) – Model predictions for the neurons.

Returns:

A tuple where the first element is a tensor indicating whether the constraint is satisfied (with True for satisfaction, False for non-satisfaction, and torch.nan for irrelevant results), and the second element is an integer value representing the number of relevant constraints.

Return type:

tuple[Tensor, int]

Raises:

NotImplementedError – If not implemented in a subclass.

class congrads.constraints.PythagoreanIdentityConstraint(a: str | Transformation, b: str | Transformation, rtol: float = 1e-05, atol: float = 1e-08, name: str = None, monitor_only: bool = False, rescale_factor: Number = 1.5)

Bases: Constraint

A constraint that enforces the Pythagorean identity: a² + b² ≈ 1, where a and b are neurons or transformations.

This constraint checks that the sum of the squares of two specified neurons (or their transformations) is approximately equal to 1. The constraint is evaluated using relative and absolute tolerance (rtol and atol) and is applied during the forward pass.

Parameters:

  • a (Union[str, Transformation]) – The first input, either a neuron name (str) or a Transformation.

  • b (Union[str, Transformation]) – The second input, either a neuron name (str) or a Transformation.

  • rtol (float, optional) – The relative tolerance for the comparison (default is 0.00001).

  • atol (float, optional) – The absolute tolerance for the comparison (default is 1e-8).

  • name (str, optional) – The name of the constraint (default is None, and it is generated automatically).

  • monitor_only (bool, optional) – Flag indicating whether the constraint is only for monitoring (default is False).

  • rescale_factor (Number, optional) – A factor used for rescaling (default is 1.5).

Raises:

TypeError – If a provided attribute has an incompatible type.

calculate_direction(prediction: dict[str, Tensor]) → Dict[str, Tensor]

Calculates adjustment directions for neurons to better satisfy the constraint.

Parameters:

prediction (dict[str, Tensor]) – Model predictions for the neurons.

Returns:

Dictionary mapping neuron layers to tensors specifying the adjustment direction for each neuron.

Return type:

Dict[str, Tensor]

Raises:

NotImplementedError – If not implemented in a subclass.

check_constraint(prediction: dict[str, Tensor]) → tuple[Tensor, int]

Evaluates whether the given model predictions satisfy the constraint.

Parameters:

prediction (dict[str, Tensor]) – Model predictions for the neurons.

Returns:

A tuple where the first element is a tensor indicating whether the constraint is satisfied (with True for satisfaction, False for non-satisfaction, and torch.nan for irrelevant results), and the second element is an integer value representing the number of relevant constraints.

Return type:

tuple[Tensor, int]

Raises:

NotImplementedError – If not implemented in a subclass.

class congrads.constraints.ScalarConstraint(operand: str | Transformation, comparator: Callable[[Tensor, Number], Tensor], scalar: Number, name: str = None, monitor_only: bool = False, rescale_factor: Number = 1.5)

Bases: Constraint

A constraint that enforces scalar-based comparisons on a specific neuron.

This class ensures that the output of a specified neuron satisfies a scalar comparison operation (e.g., less than, greater than, etc.). It uses a comparator function to validate the condition and calculates adjustment directions accordingly.

Parameters:

  • operand (Union[str, Transformation]) – Name of the neuron or a transformation to apply.

  • comparator (Callable[[Tensor, Number], Tensor]) – A comparison function (e.g., torch.ge, torch.lt).

  • scalar (Number) – The scalar value to compare against.

  • name (str, optional) – A unique name for the constraint. If not provided, a name is auto-generated in the format “<neuron_name> <comparator> <scalar>”.

  • monitor_only (bool, optional) – If True, only monitor the constraint without adjusting the loss. Defaults to False.

  • rescale_factor (Number, optional) – Factor to scale the constraint-adjusted loss. Defaults to 1.5.

Raises:

TypeError – If a provided attribute has an incompatible type.

Notes

  • The neuron_name must be defined in the descriptor mapping.

  • The constraint name is composed using the neuron name, comparator, and scalar value.

calculate_direction(prediction: dict[str, Tensor]) → Dict[str, Tensor]

Calculates adjustment directions for neurons to better satisfy the constraint.

Parameters:

prediction (dict[str, Tensor]) – Model predictions for the neurons.

Returns:

Dictionary mapping neuron layers to tensors specifying the adjustment direction for each neuron.

Return type:

Dict[str, Tensor]

Raises:

NotImplementedError – If not implemented in a subclass.

check_constraint(prediction: dict[str, Tensor]) → tuple[Tensor, int]

Evaluates whether the given model predictions satisfy the constraint.

Parameters:

prediction (dict[str, Tensor]) – Model predictions for the neurons.

Returns:

A tuple where the first element is a tensor indicating whether the constraint is satisfied (with True for satisfaction, False for non-satisfaction, and torch.nan for irrelevant results), and the second element is an integer value representing the number of relevant constraints.

Return type:

tuple[Tensor, int]

Raises:

NotImplementedError – If not implemented in a subclass.

class congrads.constraints.SumConstraint(operands_left: list[str | Transformation], comparator: Callable[[Tensor, Number], Tensor], operands_right: list[str | Transformation], weights_left: list[Number] = None, weights_right: list[Number] = None, name: str = None, monitor_only: bool = False, rescale_factor: Number = 1.5)

Bases: Constraint

A constraint that enforces a weighted summation comparison between two groups of neurons.

This class evaluates whether the weighted sum of outputs from one set of neurons satisfies a comparison operation with the weighted sum of outputs from another set of neurons.

Parameters:

  • operands_left (list[Union[str, Transformation]]) – List of neuron names or transformations on the left side.

  • comparator (Callable[[Tensor, Number], Tensor]) – A comparison function for the constraint.

  • operands_right (list[Union[str, Transformation]]) – List of neuron names or transformations on the right side.

  • weights_left (list[Number], optional) – Weights for the left neurons. Defaults to None.

  • weights_right (list[Number], optional) – Weights for the right neurons. Defaults to None.

  • name (str, optional) – Unique name for the constraint. If None, it’s auto-generated. Defaults to None.

  • monitor_only (bool, optional) – If True, only monitor the constraint without adjusting the loss. Defaults to False.

  • rescale_factor (Number, optional) – Factor to scale the constraint-adjusted loss. Defaults to 1.5.

Raises:

  • TypeError – If a provided attribute has an incompatible type.

  • ValueError – If the dimensions of neuron names and weights mismatch.

calculate_direction(prediction: dict[str, Tensor]) → Dict[str, Tensor]

Calculates adjustment directions for neurons to better satisfy the constraint.

Parameters:

prediction (dict[str, Tensor]) – Model predictions for the neurons.

Returns:

Dictionary mapping neuron layers to tensors specifying the adjustment direction for each neuron.

Return type:

Dict[str, Tensor]

Raises:

NotImplementedError – If not implemented in a subclass.

check_constraint(prediction: dict[str, Tensor]) → tuple[Tensor, int]

Evaluates whether the given model predictions satisfy the constraint.

Parameters:

prediction (dict[str, Tensor]) – Model predictions for the neurons.

Returns:

A tuple where the first element is a tensor indicating whether the constraint is satisfied (with True for satisfaction, False for non-satisfaction, and torch.nan for irrelevant results), and the second element is an integer value representing the number of relevant constraints.

Return type:

tuple[Tensor, int]

Raises:

NotImplementedError – If not implemented in a subclass.

Core

This module provides the CongradsCore class, which is designed to integrate constraint-guided optimization into neural network training. It extends traditional training processes by enforcing specific constraints on the model’s outputs, ensuring that the network satisfies domain-specific requirements during both training and evaluation.

The CongradsCore class serves as the central engine for managing the training, validation, and testing phases of a neural network model, incorporating constraints that influence the loss function and model updates. The model is trained with standard loss functions while also incorporating constraint-based adjustments, which are tracked and logged throughout the process.

Key features: - Support for various constraints that can influence the training process. - Integration with PyTorch’s DataLoader for efficient batch processing. - Metric management for tracking loss and constraint satisfaction. - Checkpoint management for saving and evaluating model states.

Modules in this package provide the following:

  • Descriptor: Describes variable layers in the network that are subject to constraints.

  • Constraint: Defines various constraints, which are used to guide the training process.

  • MetricManager: Manages and tracks performance metrics such as loss and constraint satisfaction.

  • CheckpointManager: Manages saving and loading model checkpoints during training.

  • Utility functions to validate inputs and configurations.

Dependencies:

  • PyTorch (torch)

  • tqdm (for progress tracking)

The CongradsCore class allows for the use of additional callback functions at different stages of the training process to customize behavior for specific needs. These include callbacks for the start and end of epochs, as well as the start and end of the entire training process.

class congrads.core.CongradsCore(descriptor: Descriptor, constraints: list[Constraint], loaders: tuple[DataLoader, DataLoader, DataLoader], network: Module, criterion: _Loss, optimizer: Optimizer, metric_manager: MetricManager, device: device, checkpoint_manager: CheckpointManager = None, epsilon: Number = 1e-06)

Bases: object

The CongradsCore class is the central training engine for constraint-guided neural network optimization. It integrates standard neural network training with additional constraint-driven adjustments to the loss function, ensuring that the network satisfies domain-specific constraints during training.

Parameters:

  • descriptor (Descriptor) – Describes variable layers in the network.

  • constraints (list[Constraint]) – List of constraints to guide training.

  • loaders (tuple[DataLoader, DataLoader, DataLoader]) – DataLoaders for training, validation, and testing.

  • network (Module) – The neural network model to train.

  • criterion (callable) – The loss function used for training and validation.

  • optimizer (Optimizer) – The optimizer used for updating model parameters.

  • metric_manager (MetricManager) – Manages metric tracking and recording.

  • device (torch.device) – The device (e.g., CPU or GPU) for computations.

  • checkpoint_manager (CheckpointManager, optional) – Manages checkpointing. If not set, no checkpointing is done.

  • epsilon (Number, optional) – A small value to avoid division by zero in gradient calculations. Default is 1e-10.

Note

A warning is logged if the descriptor has no variable layers, as at least one variable layer is required for the constraint logic to influence the training process.

fit(start_epoch: int = 0, max_epochs: int = 100, on_epoch_start: Callable[[int], None] = None, on_epoch_end: Callable[[int], None] = None, on_train_start: Callable[[int], None] = None, on_train_end: Callable[[int], None] = None) → None

Train the model for a given number of epochs.

Parameters:

  • start_epoch (int, optional) – The epoch number to start the training with. Default is 0.

  • max_epochs (int, optional) – The number of epochs to train the model. Default is 100.

  • on_epoch_start (Callable[[int], None], optional) – A callback function that will be executed at the start of each epoch.

  • on_epoch_end (Callable[[int], None], optional) – A callback function that will be executed at the end of each epoch.

  • on_train_start (Callable[[int], None], optional) – A callback function that will be executed before the training starts.

  • on_train_end (Callable[[int], None], optional) – A callback function that will be executed after training ends.

test_step(prediction: dict[str, Tensor], loss: Tensor) → Tensor

Evaluate constraints during test and log satisfaction metrics.

Parameters:

  • prediction (dict[str, Tensor]) – Model predictions for variable layers.

  • loss (Tensor) – The base loss computed by the criterion.

Returns:

The unchanged base loss.

Return type:

Tensor

train_step(prediction: dict[str, Tensor], loss: Tensor) → Tensor

Adjust the training loss based on constraints and compute the combined loss.

Parameters:

  • prediction (dict[str, Tensor]) – Model predictions for variable layers.

  • loss (Tensor) – The base loss computed by the criterion.

Returns:

The combined loss (base loss + constraint adjustments).

Return type:

Tensor

valid_step(prediction: dict[str, Tensor], loss: Tensor) → Tensor

Evaluate constraints during validation and log satisfaction metrics.

Parameters:

  • prediction (dict[str, Tensor]) – Model predictions for variable layers.

  • loss (Tensor) – The base loss computed by the criterion.

Returns:

The unchanged base loss.

Return type:

Tensor

Datasets

This module defines several PyTorch dataset classes for loading and working with various datasets. Each dataset class extends the torch.utils.data.Dataset class and provides functionality for downloading, loading, and transforming specific datasets.

Classes:

  • BiasCorrection: A dataset class for the Bias Correction dataset focused on temperature forecast data.

  • FamilyIncome: A dataset class for the Family Income and Expenditure dataset.

  • NoisySines: A dataset class that generates noisy sine wave samples with added Gaussian noise.

Each dataset class provides methods for downloading the data (if not already available), checking the integrity of the dataset, loading the data from CSV files or generating synthetic data, and applying transformations to the data.

Key Methods:

  • __init__: Initializes the dataset by specifying the root directory, transformation function, and optional download flag.

  • __getitem__: Retrieves a specific data point given its index, returning input-output pairs.

  • __len__: Returns the total number of examples in the dataset.

  • download: Downloads and extracts the dataset from

    the specified mirrors.

  • _load_data: Loads the dataset from CSV files and applies transformations.

  • _check_exists: Checks if the dataset is already downloaded and verified.

Each dataset class allows the user to apply custom transformations to the dataset through the transform argument to allow pre-processing and offers the ability to download the dataset if it’s not already present on the local disk.

class congrads.datasets.BiasCorrection(root: str | Path, transform: Callable, download: bool = False)

Bases: Dataset

A dataset class for accessing the Bias Correction dataset.

This class extends the Dataset class and provides functionality for downloading, loading, and transforming the Bias Correction dataset. The dataset is focused on temperature forecast data and is made available for use with PyTorch. If download is set to True, the dataset will be downloaded if it is not already available. The data is then loaded, and a transformation function is applied to it.

Parameters:

  • root (Union[str, Path]) – The root directory where the dataset will be stored or loaded from.

  • transform (Callable) – A function to transform the dataset (e.g., preprocessing).

  • download (bool, optional) – Whether to download the dataset if it’s not already present. Defaults to False.

Raises:

RuntimeError – If the dataset is not found and download is not set to True or if all mirrors fail to provide the dataset.

property data_folder: str

Returns the path to the folder where the dataset is stored.

Returns:

The path to the dataset folder.

Return type:

str

download() → None

Downloads and extracts the dataset.

This method attempts to download the dataset from the mirrors and extract it into the appropriate folder. If any error occurs during downloading, it will try each mirror in sequence.

Raises:

RuntimeError – If all mirrors fail to provide the dataset.

mirrors = ['https://archive.ics.uci.edu/static/public/514/']
resources = [('bias+correction+of+numerical+prediction+model+temperature+forecast.zip', '3deee56d461a2686887c4ae38fe3ccf3')]
class congrads.datasets.FamilyIncome(root: str | Path, transform: Callable, download: bool = False)

Bases: Dataset

A dataset class for accessing the Family Income and Expenditure dataset.

This class extends the Dataset class and provides functionality for downloading, loading, and transforming the Family Income and Expenditure dataset. The dataset is intended for use with PyTorch-based projects, offering a convenient interface for data handling. This class provides access to the Family Income and Expenditure dataset for use with PyTorch. If download is set to True, the dataset will be downloaded if it is not already available. The data is then loaded, and a user-defined transformation function is applied to it.

Parameters:

  • root (Union[str, Path]) – The root directory where the dataset will be stored or loaded from.

  • transform (Callable) – A function to transform the dataset (e.g., preprocessing).

  • download (bool, optional) – Whether to download the dataset if it’s not already present. Defaults to False.

Raises:

RuntimeError – If the dataset is not found and download is not set to True or if all mirrors fail to provide the dataset.

property data_folder: str

Returns the path to the folder where the dataset is stored.

Returns:

The path to the dataset folder.

Return type:

str

download() → None

Downloads and extracts the dataset.

This method attempts to download the dataset from the mirrors and extract it into the appropriate folder. If any error occurs during downloading, it will try each mirror in sequence.

Raises:

RuntimeError – If all mirrors fail to provide the dataset.

mirrors = ['https://www.kaggle.com/api/v1/datasets/download/grosvenpaul/family-income-and-expenditure']
resources = [('archive.zip', '7d74bc7facc3d7c07c4df1c1c6ac563e')]
class congrads.datasets.NoisySines(length, amplitude=1, frequency=10.0, noise_std=0.05, bias=0, random_seed=42)

Bases: Dataset

A PyTorch dataset generating samples from a causal sine wave with added noise.

Parameters:

  • length (int) – Number of data points in the dataset.

  • amplitude (float) – Amplitude of the sine wave.

  • frequency (float) – Frequency of the sine wave in Hz.

  • noise_std (float) – Standard deviation of the Gaussian noise.

  • bias (float) – Offset from zero.

The sine wave is zero for times before t=0 and follows a standard sine wave after t=0, with Gaussian noise added to all points.

Descriptor

This module defines the Descriptor class, which is designed to manage the mapping between neuron names, their corresponding layers, and additional properties such as constant or variable status. It provides a way to easily place constraints on parts of your network, by referencing the neuron names instead of indices.

The Descriptor class allows for easy constraint definitions on parts of your neural network. It supports registering neurons with associated layers, indices, and optional attributes, such as whether the layer is constant or variable.

Key Methods:

  • __init__: Initializes the Descriptor object with empty mappings and sets for managing neurons and layers.

  • add: Registers a neuron with its associated layer, index, and optional constant status.

class congrads.descriptor.Descriptor

Bases: object

A class to manage the mapping between neuron names, their corresponding layers, and additional properties (such as min/max values, output, and constant variables).

This class is designed to track the relationship between neurons and layers in a neural network. It allows for the assignment of properties (like minimum and maximum values, and whether a layer is an output, constant, or variable) to each neuron. The data is stored in dictionaries and sets for efficient lookups.

neuron_to_layer

A dictionary mapping neuron names to their corresponding layer names.

Type:

dict

neuron_to_index

A dictionary mapping neuron names to their corresponding indices in the layers.

Type:

dict

constant_layers

A set of layer names that represent constant layers.

Type:

set

variable_layers

A set of layer names that represent variable layers.

Type:

set

add(layer_name: str, index: int, neuron_name: str, constant: bool = False)

Adds a neuron to the descriptor with its associated layer, index, and properties.

This method registers a neuron name and associates it with a layer, its index, and optional properties such as whether the layer is an output or constant layer.

Parameters:

  • layer_name (str) – The name of the layer where the neuron is located.

  • index (int) – The index of the neuron within the layer.

  • neuron_name (str) – The name of the neuron.

  • constant (bool, optional) – Whether the layer is a constant layer. Defaults to False.

Raises:

  • TypeError – If a provided attribute has an incompatible type.

  • ValueError – If a layer or index is already assigned for a neuron or a duplicate index is used within a layer.

Metrics

This module defines the Metric and MetricManager classes, which are used to track and aggregate performance metrics during model training or evaluation in machine learning. These classes support the accumulation of metric values, aggregation using customizable functions (such as mean), and resetting of the metrics.

Classes:

  • Metric: A class that tracks and aggregates a specific metric over multiple samples, allowing for accumulation, aggregation, and resetting of values.

  • MetricManager: A class that manages and tracks multiple metrics during model training or evaluation, supporting registration, accumulation, aggregation, and resetting of metrics.

Key Methods:

  • Metric.__init__: Initializes a metric with a specified name and optional accumulator function (defaults to nanmean).

  • Metric.accumulate: Accumulates a new value for the metric, typically a tensor of model output or performance.

  • Metric.aggregate: Aggregates the accumulated values using the specified accumulator function.

  • Metric.reset: Resets the accumulated values and sample count for the metric.

  • MetricManager.__init__: Initializes a manager for multiple metrics.

  • MetricManager.register: Registers a new metric with a name, group, and optional accumulator function.

  • MetricManager.accumulate: Accumulates a new value for the specified metric.

  • MetricManager.aggregate: Aggregates all metrics in a specified group.

  • MetricManager.reset: Resets all registered metrics in a specified group.

Each class provides functionality to efficiently track, aggregate, and reset metrics during the training and evaluation phases of machine learning tasks, supporting flexible aggregation strategies and group-based management of metrics.

class congrads.metrics.Metric(name: str, accumulator: ~typing.Callable[[...], ~torch.Tensor] = <built-in method nanmean of type object>)

Bases: object

A class that tracks and aggregates a specific metric over multiple samples.

This class allows the accumulation of values, their aggregation using a specified function (e.g., mean), and the ability to reset the metrics. It is typically used to track performance metrics during training or evaluation processes in machine learning.

Parameters:

  • name (str) – The name of the metric.

  • accumulator (Callable[..., Tensor], optional) – A function used to

  • values (aggregate)

name

The name of the metric.

Type:

str

accumulator

The function used to aggregate values.

Type:

Callable[…, Tensor]

values

A list to store accumulated values.

Type:

list

sample_count

The count of accumulated samples.

Type:

int

accumulate(value: Tensor) → None

Accumulates a new value for the metric.

Parameters:

value (Tensor) – The new value to accumulate, typically a tensor of model output or performance.

aggregate() → Tensor

Aggregates the accumulated values using the specified accumulator function.

Returns:

The aggregated result of the accumulated values.

Return type:

Tensor

reset() → None

Resets the accumulated values and sample count for the metric.

class congrads.metrics.MetricManager

Bases: object

A class to manage and track multiple metrics during model training or evaluation.

This class allows registering metrics, accumulating values for each metric, and recording the aggregated values. It also supports the reset of metrics after each epoch or training step.

metrics

A dictionary of registered metrics.

Type:

dict[str, Metric]

groups

A dictionary mapping metric names to groups.

Type:

dict[str, str]

accumulate(name: str, value: Tensor) → None

Accumulates a new value for the specified metric.

Parameters:

  • name (str) – The name of the metric.

  • value (Tensor) – The new value to accumulate.

aggregate(group: str) → dict[str, Tensor]

Aggregates all metrics in a group using the accumulators specified during registration.

Parameters:

group (str) – The name of the group.

Returns:

A dictionary with the metric names and the

corresponding aggregated values of the selected group.

Return type:

dict[str, Tensor]

register(name: str, group: str, accumulator: ~typing.Callable[[...], ~torch.Tensor] = <built-in method nanmean of type object>) → None

Registers a new metric with the specified name and accumulator function.

Parameters:

  • name (str) – The name of the metric to register.

  • group (str) – The name of the group to assign the metric to.

  • accumulator (Callable[..., Tensor], optional) – The function used to aggregate values for the metric (defaults to nanmean).

reset(group: str) → None

Resets all registered metrics in a group.

Parameters:

group (str) – The name of the group.

Networks

This module defines the MLPNetwork class, which constructs and operates a multi-layer perceptron (MLP) neural network model. The MLP network consists of an input layer, multiple hidden layers, and an output layer. It allows for configurable hyperparameters such as the number of input features, output features, number of hidden layers, and the dimensionality of the hidden layers.

Classes:

  • MLPNetwork: A neural network model that implements a multi-layer perceptron with customizable layers and dimensionalities.

Key Methods:

  • __init__: Initializes the MLP network with specified input size, output size, number of hidden layers, and hidden layer dimensionality.

  • forward: Performs a forward pass through the network, returning both the input and output of the model.

  • linear: Creates a basic linear block consisting of a Linear layer followed by a ReLU activation function.

The MLPNetwork class constructs a fully connected neural network with multiple hidden layers, providing flexibility in designing the network architecture. It can be used for regression, classification, or other machine learning tasks that require a feedforward neural network structure.

class congrads.networks.MLPNetwork(n_inputs, n_outputs, n_hidden_layers=3, hidden_dim=35)

Bases: Module

A multi-layer perceptron (MLP) neural network model consisting of an input layer, multiple hidden layers, and an output layer.

This class constructs an MLP with configurable hyperparameters such as the number of input features, output features, number of hidden layers, and the dimensionality of hidden layers. It provides methods for both building the model and performing a forward pass through the network.

Parameters:

  • n_inputs (int, optional) – The number of input features. Defaults to 25.

  • n_outputs (int, optional) – The number of output features. Defaults to 2.

  • n_hidden_layers (int, optional) – The number of hidden layers. Defaults to 2.

  • hidden_dim (int, optional) – The dimensionality of the hidden layers. Defaults to 35.

forward(data)

Performs a forward pass through the network.

Parameters:

data (Tensor) – The input tensor to be passed through the network.

Returns:

A dictionary containing the ‘input’ (original input) and ‘output’ (predicted output) of the network.

Return type:

dict

static linear(in_features, out_features)

Creates a basic linear block with a linear transformation followed by a ReLU activation function.

Parameters:

  • in_features (int) – The number of input features.

  • out_features (int) – The number of output features.

Returns:

A sequential module consisting of a Linear layer and ReLU activation.

Return type:

nn.Module

Transformations

This module defines the abstract base class Transformation and two specific transformations: IdentityTransformation and DenormalizeMinMax. These transformations are used to apply operations to neuron data.

Classes:

  • Transformation: An abstract base class for transformations that can be applied to neuron data. Subclasses must implement the __call__ method to apply the transformation.

  • IdentityTransformation: A subclass of Transformation that returns the input data unchanged.

  • DenormalizeMinMax: A subclass of Transformation that denormalizes input data based on specified minimum and maximum values.

Key Methods:

  • __call__(data: Tensor) -> Tensor: Abstract method in the Transformation class that must be implemented by subclasses to apply a transformation to the input data.

  • __init__(neuron_name: str): Initializes the transformation with the associated neuron name.

  • IdentityTransformation.__call__(data: Tensor) -> Tensor: Returns the input data without applying any transformation.

  • DenormalizeMinMax.__call__(data: Tensor) -> Tensor: Denormalizes the input data by scaling it based on the specified min and max values.

The Transformation class is intended as a base class for creating custom transformations for neuron data, while the IdentityTransformation is used when no transformation is desired, and DenormalizeMinMax is used for reversing the normalization process by using a min-max scaling approach.

class congrads.transformations.DenormalizeMinMax(neuron_name: str, min: Number, max: Number)

Bases: Transformation

A transformation that denormalizes data based on a specified min and max value.

This transformation scales the data by the range of the min and max values, then adds the min value to denormalize it back to the original scale.

Parameters:

  • neuron_name (str) – The name of the neuron associated with the transformation.

  • min (Number) – The minimum value to scale the data.

  • max (Number) – The maximum value to scale the data.

__call__(data

Tensor) -> Tensor: Applies the denormalization to the given data by scaling it with the min and max values.

class congrads.transformations.IdentityTransformation(neuron_name: str)

Bases: Transformation

A transformation that returns the data unchanged (identity transformation).

Inherits from the Transformation class and implements the __call__ method to return the input data as is.

class congrads.transformations.Transformation(neuron_name: str)

Bases: ABC

Abstract base class for transformations applied to neuron data.

Parameters:

neuron_name (str) – The name of the neuron associated with the transformation.

__call__(data

Tensor) -> Tensor: Applies the transformation to the provided data. Must be implemented by subclasses.

Utils

This module provides utility functions and classes for managing data, logging, and preprocessing in machine learning workflows. The functionalities include logging key-value pairs to CSV files, splitting datasets into training, validation, and test sets, preprocessing functions for various data sets and validator functions for type checking.

class congrads.utils.CSVLogger(file_path: str, overwrite: bool = False, merge: bool = True)

Bases: object

A utility class for logging key-value pairs to a CSV file, organized by epochs. Supports merging with existing logs or overwriting them.

Parameters:

  • file_path (str) – The path to the CSV file for logging.

  • overwrite (bool) – If True, overwrites any existing file at the file_path.

  • merge (bool) – If True, merges new values with existing data in the file.

Raises:

  • ValueError – If both overwrite and merge are True.

  • FileExistsError – If the file already exists and neither overwrite nor merge is True.

add_value(name: str, value: float, epoch: int)

Adds a value to the logger for a specific epoch and name.

Parameters:

  • name (str) – The name of the metric or value to log.

  • value (float) – The value to log.

  • epoch (int) – The epoch associated with the value.

load()

Loads data from the CSV file into the logger.

Converts the CSV data into the internal dictionary format for further updates or operations.

save()

Saves the logged values to the specified CSV file.

If the file exists and merge is enabled, merges the current data with the existing file.

static to_dataframe(values: dict[tuple[int, str], float]) → DataFrame

Converts a dictionary of values into a DataFrame.

Parameters:

values (dict[tuple[int, str], float]) – A dictionary of values keyed by (epoch, name).

Returns:

A DataFrame where epochs are rows, names are columns, and values are the cell data.

Return type:

pd.DataFrame

static to_dict(df: DataFrame) → dict[tuple[int, str], float]

Converts a DataFrame with epochs as rows and names as columns back into a dictionary of the format {(epoch, name): value}.

congrads.utils.preprocess_BiasCorrection(df: DataFrame) → DataFrame

Preprocesses the given dataframe for bias correction by performing a series of transformations.

The function sequentially:

  • Drops rows with missing values.

  • Converts a date string to datetime format and adds year, month, and day columns.

  • Normalizes the columns with specific logic for input and output variables.

  • Adds a multi-index indicating which columns are input or output variables.

  • Samples 2500 examples from the dataset without replacement.

Parameters:

df (pd.DataFrame) – The input dataframe containing the data to be processed.

Returns:

The processed dataframe after applying the transformations.

Return type:

pd.DataFrame

congrads.utils.preprocess_FamilyIncome(df: DataFrame) → DataFrame

Preprocesses the given Family Income dataframe by applying a series of transformations and constraints.

The function sequentially:

  • Drops rows with missing values.

  • Converts object columns to appropriate data types and removes string columns.

  • Removes certain unnecessary columns like ‘Agricultural Household indicator’ and related features.

  • Adds labels to columns indicating whether they are input or output variables.

  • Normalizes the columns individually.

  • Checks and removes rows that do not satisfy predefined constraints (household income > expenses, food expenses > sub-expenses).

  • Samples 2500 examples from the dataset without replacement.

Parameters:

df (pd.DataFrame) – The input Family Income dataframe containing the data to be processed.

Returns:

The processed dataframe after applying the transformations and constraints.

Return type:

pd.DataFrame

congrads.utils.split_data_loaders(data: Dataset, loader_args: dict = None, train_loader_args: dict = None, valid_loader_args: dict = None, test_loader_args: dict = None, train_size: float = 0.8, valid_size: float = 0.1, test_size: float = 0.1, split_generator: Generator = None) → tuple[DataLoader, DataLoader, DataLoader]

Splits a dataset into training, validation, and test sets, and returns corresponding DataLoader objects.

Parameters:

  • data (Dataset) – The dataset to be split.

  • loader_args (dict, optional) – Default DataLoader arguments, merges with loader-specific arguments, overlapping keys from loader-specific arguments are superseded.

  • train_loader_args (dict, optional) – Training DataLoader arguments, merges with loader_args, overriding overlapping keys.

  • valid_loader_args (dict, optional) – Validation DataLoader arguments, merges with loader_args, overriding overlapping keys.

  • test_loader_args (dict, optional) – Test DataLoader arguments, merges with loader_args, overriding overlapping keys.

  • train_size (float, optional) – Proportion of data to be used for training. Defaults to 0.8.

  • valid_size (float, optional) – Proportion of data to be used for validation. Defaults to 0.1.

  • test_size (float, optional) – Proportion of data to be used for testing. Defaults to 0.1.

  • split_generator (Generator, optional) – Optional random seed generator to control the splitting of the dataset.

Returns:

A tuple containing three DataLoader objects: one for the training, validation and test set.

Return type:

tuple

Raises:

ValueError – If the train_size, valid_size, and test_size are not between 0 and 1, or if their sum does not equal 1.

congrads.utils.validate_callable(name, value, allow_none=False)

Validate that a value is callable function.

Parameters:

  • name (str) – Name of the argument for error messages.

  • value – Value to validate.

  • allow_none (bool) – Whether to allow the value to be None. Defaults to False.

Raises:

TypeError – If the value is not callable.

congrads.utils.validate_comparator_pytorch(name, value)

Validate that a value is a callable PyTorch comparator function.

Parameters:

  • name (str) – Name of the argument for error messages.

  • value – Value to validate.

Raises:

TypeError – If the value is not callable or not a PyTorch comparator.

congrads.utils.validate_iterable(name, value, expected_element_types, allowed_iterables=(<class 'list'>, <class 'set'>), allow_none=False)

Validate that a value is an iterable (e.g., list, set) with elements of the specified type(s).

Parameters:

  • name (str) – Name of the argument for error messages.

  • value – Value to validate.

  • expected_element_types (type or tuple of types) – Expected type(s) for the elements.

  • allowed_iterables (tuple of types) – Iterable types that are allowed (default: list and set).

  • allow_none (bool) – Whether to allow the value to be None. Defaults to False.

Raises:

TypeError – If the value is not an allowed iterable type or if any element is not of the expected type(s).

congrads.utils.validate_loaders() → None

TODO: implement function

congrads.utils.validate_type(name, value, expected_types, allow_none=False)

Validate that a value is of the specified type(s).

Parameters:

  • name (str) – Name of the argument for error messages.

  • value – Value to validate.

  • expected_types (type or tuple of types) – Expected type(s) for the value.

  • allow_none (bool) – Whether to allow the value to be None. Defaults to False.

Raises:

TypeError – If the value is not of the expected type(s).