Introduction
Gin-config is a lightweight configuration framework developed by Google that provides a practical way to configure parameters of Python functions and classes. Rather than requiring explicit configuration files in formats like YAML or JSON, Gin uses its own simple .gin configuration syntax combined with Python decorators. This approach allows you to bind parameter values to any function or class without modifying its signature or adding boilerplate code.
Gin is especially valuable in machine learning research, where experiments involve many hyperparameters spread across data loading, model architecture, training loops, and evaluation. Instead of threading parameters through every function call, Gin lets you specify values in a centralized configuration file.
In this guide, you will learn how to make functions configurable with Gin, write .gin configuration files, and organize experiment configurations for reproducible research.
Overview
Key features of Gin-config:
- Decorator-based: Mark any function or class as configurable with
@gin.configurable .ginfile format: A simple, purpose-built syntax for binding parameter values- Hierarchical scoping: Configure the same function differently in different contexts using scopes
- No schema required: Configuration is derived directly from Python function signatures
- Composable configs: Import and override configurations across multiple
.ginfiles
Use cases:
- Machine learning experiment configuration
- Hyperparameter management for research
- Configuring complex pipelines with many components
- Reproducible experiment tracking
Getting Started
Install Gin-config using pip:
pip install gin-config
Here is a complete working example demonstrating the core workflow:
import gin
@gin.configurable
def train_model(learning_rate=0.001, batch_size=32, epochs=10):
"""A training function whose parameters can be set via gin."""
print(f"Training with lr={learning_rate}, batch_size={batch_size}, epochs={epochs}")
# Parse a gin configuration string (in practice, you would use a .gin file)
gin.parse_config("""
train_model.learning_rate = 0.01
train_model.batch_size = 64
train_model.epochs = 20
""")
# Call the function - parameters are injected by gin
train_model()
# Output: Training with lr=0.01, batch_size=64, epochs=20
Core Concepts
The @gin.configurable Decorator
Any function or class can be made configurable by applying the @gin.configurable decorator:
import gin
@gin.configurable
class Optimizer:
def __init__(self, learning_rate=0.001, momentum=0.9):
self.lr = learning_rate
self.momentum = momentum
@gin.configurable
def build_model(num_layers=3, hidden_size=256, activation="relu"):
print(f"Building model: {num_layers} layers, hidden={hidden_size}, act={activation}")
return {"layers": num_layers, "hidden": hidden_size, "activation": activation}
The .gin File Format
Gin uses its own configuration format. Create a file called config.gin:
# config.gin
build_model.num_layers = 5
build_model.hidden_size = 512
build_model.activation = "gelu"
Optimizer.learning_rate = 0.0003
Optimizer.momentum = 0.95
Then load it in Python:
import gin
gin.parse_config_file("config.gin")
model = build_model() # Uses values from config.gin
opt = Optimizer() # Uses values from config.gin
Scoped Configurations
Gin supports scopes for configuring the same function differently in different contexts:
# In the .gin file
train/Optimizer.learning_rate = 0.001
finetune/Optimizer.learning_rate = 0.00001
with gin.config_scope("train"):
train_opt = Optimizer() # lr = 0.001
with gin.config_scope("finetune"):
ft_opt = Optimizer() # lr = 0.00001
Binding Parameters Programmatically
You can also bind parameters directly in Python:
gin.bind_parameter("build_model.num_layers", 8)
Practical Examples
Example 1: Configuring a Machine Learning Experiment
import gin
@gin.configurable
def load_data(dataset_name="mnist", split="train", batch_size=32):
print(f"Loading {dataset_name} ({split}), batch_size={batch_size}")
return {"dataset": dataset_name, "split": split, "batch_size": batch_size}
@gin.configurable
def create_model(architecture="resnet", num_classes=10, dropout=0.1):
print(f"Creating {architecture} with {num_classes} classes, dropout={dropout}")
return {"arch": architecture, "classes": num_classes}
@gin.configurable
def train(model, data, epochs=10, learning_rate=0.001):
print(f"Training for {epochs} epochs at lr={learning_rate}")
# experiment.gin would contain:
gin.parse_config("""
load_data.dataset_name = "cifar10"
load_data.batch_size = 128
create_model.architecture = "resnet50"
create_model.num_classes = 10
create_model.dropout = 0.2
train.epochs = 50
train.learning_rate = 0.0001
""")
data = load_data()
model = create_model()
train(model, data)
Example 2: Composing Configurations with Gin File Imports
Create a base configuration base.gin:
# base.gin
create_model.architecture = "resnet18"
create_model.num_classes = 10
train.epochs = 100
train.learning_rate = 0.001
Create an override for a specific experiment large_model.gin:
# large_model.gin
include "base.gin"
create_model.architecture = "resnet101"
train.learning_rate = 0.0003
gin.parse_config_file("large_model.gin")
# Inherits base.gin values, but overrides architecture and learning rate
Best Practices
- Use
.ginfiles for experiment configs: Keep your configuration in.ginfiles alongside your code for reproducibility. Check them into version control. - Keep configurable functions focused: Apply
@gin.configurableto functions with meaningful parameters, not every function in your codebase. - Use
gin.operative_config_str(): After running an experiment, call this to get a string of all gin parameters that were actually used, which is useful for logging. - Leverage scopes for variants: Use scoped configurations instead of duplicating code when you need different parameter sets for the same function.
- Clear state between experiments: Call
gin.clear_config()when running multiple experiments in the same process.
Common pitfalls:
- Gin does not support YAML, JSON, or TOML. It uses its own
.ginformat exclusively. - Forgetting to call
gin.parse_config_file()before invoking configurable functions will result in default values being used silently. - The
@gin.configurabledecorator must be applied before the function is referenced elsewhere.
Conclusion
Gin-config offers an elegant approach to configuration management that is particularly well-suited for machine learning research. By binding configuration values directly to function parameters through a simple decorator and a lightweight file format, Gin eliminates the boilerplate of passing parameters through deep call stacks while maintaining full transparency into what values are being used.
Resources:
Powered by Jekyll & Minimal Mistakes.