Introduction

At GitHub, we operate under a unique premise: our own source code is hosted on github.com. This self-hosting model makes us our own biggest customer, allowing us to test internal changes before rolling them out to users. However, it introduces a critical challenge: if github.com were to go down, we would lose access to our own source code, creating a circular dependency that complicates recovery.

To mitigate this risk, we maintain a mirror of our code for fixing forward and keep built assets ready for rollback. But even with these safeguards, additional circular dependencies can emerge—particularly during deployment scripts. This article explores how we leveraged eBPF (Extended Berkeley Packet Filter) to monitor and block such dependencies, improving deployment safety.

Types of Circular Dependencies

Consider a MySQL outage: GitHub cannot serve release data from repositories. To resolve the incident, we need to deploy a configuration change to the affected MySQL nodes by running a deploy script. Let’s examine the circular dependencies that could hinder this process.

Direct Dependency

The MySQL deploy script attempts to download the latest release of an open source tool from GitHub. Because GitHub is unavailable during the outage, the script cannot complete, directly blocking the deployment.

Hidden Dependency

The script uses a servicing tool already present on the machine’s disk. However, this tool checks GitHub for updates before running. If it cannot reach GitHub, the script may fail or hang, depending on how the tool handles the update check.

Transient Dependency

The script calls an internal service (e.g., a migrations service) via an API. This service, in turn, tries to fetch a new binary from GitHub. The failure propagates back to the deploy script, cascading the outage.

Traditional Approaches and Their Limitations

Previously, the responsibility fell on each team to manually review their deployment scripts and identify circular dependencies. This approach was error-prone and often missed hidden or transient dependencies, especially in complex deployments. We needed a more systematic way to enforce safety.

How eBPF Solves the Problem

When designing our new host-based deployment system, we evaluated eBPF as a solution. eBPF allows us to attach small programs to kernel hooks—for example, to monitor network calls or file access—and take actions like logging or blocking. This enables us to selectively intercept any call that might create a circular dependency.

Implementation in Deployment Scripts

We use eBPF to enforce policies such as: “Block all outgoing HTTP requests to github.com during deployment unless explicitly allowed.” This prevents direct dependencies. For hidden dependencies, we can inspect tool behavior at runtime and block update checks. For transient dependencies, we trace the call chain across services and block any propagation that leads to GitHub.

Getting Started with eBPF

Writing your own eBPF programs requires familiarity with C and kernel concepts. We recommend starting with bpftrace for simple tracing, then moving to libbpf for more complex solutions. The key is to identify the system calls that correspond to your dependencies (e.g., connect(), openat()) and write filters accordingly.

Conclusion

eBPF has proven instrumental in automatically preventing circular dependencies during deployments at GitHub. While our mirror and rollback assets handle the primary circular dependency of self-hosting, eBPF closes the gap for script-induced dependencies. We encourage teams facing similar challenges to explore eBPF as a powerful tool for deployment safety.