Cumulative Development in the Age of AI

Introduction

Software development has always depended on cumulativity: the ability to build incrementally on prior work rather than repeatedly starting from scratch. Progress in engineering and R&D depends on developers being able to return to old code, understand what was done and why, and extend it meaningfully. This applies at multiple scales: individually, when you revisit your own repository months later, and interpersonally, when you inherit someone else's codebase long after the original context has faded.

We have entered a more declarative era of programming in which AI systems make rapid prototyping increasingly easy. When the sole objective is a functional artifact, this paradigm is often sufficient. Sustained progress, however, depends on the ability to make incremental changes and to build cumulatively on prior work, whether produced by oneself, by collaborators, or by tools.

AI-assisted development introduces new scales of cumulative work. Developers now collaborate with AI agents that make autonomous edits and generate pull requests, and emerging multi-agent systems increasingly establish and extend projects with limited human oversight. Yet current workflows offer little support for preserving intent, context, and continuity as these systems evolve, leaving cumulative development fragile despite accelerating production.

The question therefore becomes urgent: how do we design human-centered practices for AI-assisted software engineering? How do we ensure that the code produced today, by humans and agents alike, can be understood, modified, and extended tomorrow? GitHub serves as a multi-use developer platform that supports cumulative work by preserving the core functionality of Git as a local version history system while providing a web-based interface for navigating code and project state. Through its privacy controls, GitHub can function solely as an interface for individual developers managing their own work. At the same time, it enables collaboration by allowing projects to be shared with known collaborators and organized within teams and institutions. More broadly, GitHub has become the backbone of open-source software development, where contributors may not know one another and repository owners retain discretion over whether and how external contributions are incorporated.

This essay argues that effective collaboration in software engineering hinges on the ability to share sufficient context through interfaces such as a shared repository. This context is traditionally conveyed through explicit mechanisms like developer-provided documentation, including README files, as well as through structural features of a codebase that a contributor must interpret in order to make progress. In AI-assisted development, however, an originating author's ability to explicate an initial artifact and provide meaningful context is increasingly constrained. As a result, it becomes harder for individuals to extend largely AI-generated code when the initial author has not taken the time to understand, interpret, and distill the system themselves.

At the same time, AI-assisted tooling makes it easier than ever to extend others' work for one's own development goals without reintroducing that code into the public ecosystem as a shared artifact. In these cases, external codebases function primarily as contextual input for sovereign agents rather than as collaboratively maintained systems. The risk is not a decline in code production, but a decline in cumulative collaboration. What was once a chain of software extension increasingly resembles a series of truncated links, where work no longer meaningfully builds on what came before.

The Downsides of Short-Chained Software Engineering

Software development can be understood as the process of producing a desired artifact under a set of constraints, most notably a limited budget. This structure is shared by both human and machine actors. For humans, the budget is defined by time, attention, and cognitive effort. For machines, it is defined by context length, computation, and latency. In both cases, a fixed budget must be allocated across the same underlying stages: acquiring context, forming a plan for modification, and executing that plan.

While context often appears to be freely available, it is not. For human developers, acquiring context requires sustained effort to read, interpret, and mentally model a complex system. This cost grows with system size and with the degree to which prior decisions and intent are undocumented or implicit. For machine learning models, context can be loaded more mechanically as input, with costs that are easier to quantify in terms of tokens and compute. As a result, machines can often ingest large amounts of context cheaply, while humans cannot.

The asymmetry becomes more pronounced during synthesis. For humans, decoding context involves forming an intent for extension and planning how to modify the system to achieve that goal. Execution is comparatively slow and error-prone, but synthesis is the primary bottleneck. For models, decoding is largely defined by generative output. When these outputs must be interpreted, validated, and integrated by humans for human-defined goals, the overall development pipeline becomes constrained by human synthesis. In fully agentic systems, this bottleneck is partially avoided because both decoding and execution are delegated to machines.

AI-assisted development tools therefore change not only the speed of execution but the allocation of budget across the development pipeline. When systems are optimized to reach functional end states under tight human budgets, they naturally favor shortcuts that reduce immediate synthesis costs. While this enables rapid prototyping, it can also produce artifacts that are difficult to understand, modify, or extend over time. The result is a tension between achieving short-term goals and preserving the capacity for cumulative progress.