Repast4Py: Distributed Agent-Based Modeling for Large-Scale Social Simulations on HPC Clusters

Agent-based models (ABMs) of social systems — opinion cascades, labor market dynamics, urban migration flows — routinely require millions of interacting agents to produce statistically meaningful results. Desktop tools hit memory and CPU walls long before reaching that scale. Repast4Py, the Python-native distributed extension of the Repast family, bridges the gap: it runs on everything from a laptop to a 10,000-core supercomputer cluster using MPI parallelism, while keeping the model logic in readable Python.

What Repast4Py Is (and Is Not)

Repast4Py is not a replacement for Repast Simphony (the Java/GUI tool already covered in this series). It is a separate library — repast4py on PyPI — designed from the ground up for headless, script-driven, high-performance computing (HPC) workflows. Key characteristics:

• MPI-based parallelism via mpi4py: the agent population is partitioned across ranks; each rank owns a spatial sub-domain or a subset of the network.
• Shared ghost layers: agents near partition boundaries are mirrored as read-only "ghost" copies on neighboring ranks, enabling local interaction rules without global communication.
• Continuous and discrete spaces: repast4py.space provides 2-D/3-D continuous spaces and discrete grids, both MPI-aware.
• Network support: repast4py.network wraps NetworkX graphs with distributed edge management.
• Pure Python model logic: NumPy, SciPy, pandas, and PyTorch are all importable inside agent step functions.

Core Architecture: Ranks, Contexts, and Ghost Agents

Every Repast4Py simulation is structured around a Context — the container that holds all agents a rank "owns" — and one or more Projections (spaces or networks) that define how agents relate to each other.

from repast4py import core, space, schedule, logging
from mpi4py import MPI

comm = MPI.COMM_WORLD
rank = comm.Get_rank()

# Create a shared continuous 2-D space across all ranks
box = space.BoundingBox(0, 500, 0, 500, 0, 0)
shared_space = space.SharedCSpace(
    "opinion_space", bounds=box,
    borders=space.BorderType.Sticky,
    occupancy=space.OccupancyType.Multiple,
    buffer_size=2, comm=comm, tree_threshold=100
)

When an agent moves near a partition boundary, Repast4Py automatically synchronizes its state to the ghost layer of the adjacent rank before the next tick — a detail that would require hundreds of lines of hand-written MPI code in a bare-metal implementation.

Practical Example: Bounded-Confidence Opinion Dynamics at Scale

The Deffuant-Weisbuch model (covered separately in this series) is a canonical opinion dynamics benchmark. Running it with 5 million agents on a single machine is impractical; Repast4Py makes it routine.

class OpinionAgent(core.Agent):
    TYPE = 0

    def __init__(self, a_id, rank, opinion):
        super().__init__(a_id, OpinionAgent.TYPE, rank)
        self.opinion = opinion  # float in [0, 1]

    def step(self, shared_space, epsilon=0.25, mu=0.5):
        # Query neighbors within interaction radius
        neighbors = shared_space.get_agents(
            space.ContinuousPoint(self.opinion, 0),
            radius=epsilon
        )
        for neighbor in neighbors:
            if abs(self.opinion - neighbor.opinion) < epsilon:
                self.opinion += mu * (neighbor.opinion - self.opinion)

With 64 MPI ranks on a modest cluster, this scales to ~10 million agents with sub-minute wall-clock time per 1,000 ticks — a 40× speedup over a single-process Mesa implementation on the same hardware.

Data Collection and Logging

Repast4Py's logging module writes agent-level and aggregate data to CSV files in a rank-aware manner, then merges them post-run:

agent_logger = logging.TabularLogger(
    comm, "output/agents.csv",
    ["tick", "agent_id", "rank", "opinion"]
)

def log_agents(tick):
    for agent in context.agents():
        agent_logger.log_row(tick, agent.uid[0], rank, agent.opinion)
    agent_logger.write()

For ensemble runs (parameter sweeps), Repast4Py integrates with swift-t and EMEWS (Extreme-scale Model Exploration with Swift) — a workflow layer that launches thousands of independent MPI jobs and feeds results back to a central database for surrogate modeling or Bayesian optimization.

Installation and HPC Deployment

# On a local machine (Linux/macOS)
pip install repast4py

# On an HPC cluster (e.g., SLURM)
module load python/3.11 openmpi/4.1
pip install --user repast4py

# Run with 32 MPI ranks
mpirun -n 32 python opinion_model.py

Repast4Py ships pre-built wheels for Linux x86-64 with OpenMPI and MPICH. On Cray systems, users typically compile from source against the system MPI. The official documentation includes ALCF Theta and OLCF Summit build recipes.

When to Choose Repast4Py vs. Alternatives

Repast4Py is the right choice when: (1) the model logic is already in Python, (2) the target machine is a CPU cluster without GPUs, and (3) the team needs tight integration with HPC workflow managers like EMEWS or Parsl.

Limitations to Know

• No built-in GUI: visualization requires post-processing with matplotlib, Gephi, or ParaView. There is no live dashboard equivalent to NetLogo's world view.
• Debugging distributed models is hard: race conditions in ghost synchronization are subtle. The development team recommends always validating on a single rank first.
• Windows is unsupported: MPI on Windows is possible but not officially tested by the Repast team.

Further Resources

• Repast4Py GitHub Repository — source, issues, and examples
• Repast4Py API Reference
• EMEWS Project — HPC workflow integration for model exploration
• Argonne Leadership Computing Facility Tutorials — HPC-specific build and run guides
• Collier, N., & North, M. (2013). Parallel agent-based simulation with Repast for High Performance Computing. SIMULATION, 89(10), 1215–1235.

Repast4Py occupies a unique niche: it delivers genuine HPC scalability without abandoning the Python ecosystem that most social simulation researchers already work in. For teams ready to move beyond desktop-scale ABMs, it is the most accessible on-ramp to distributed agent-based modeling available today.