<tool_call> <function=done> <parameter=result>## Lore: Open Source Version Control System **Lore** is a next-generation, open source version control system developed by Epic Games, designed to address scalability challenges in modern software development projects. It's built as a centralized system with offline capability, treating all data as opaque byte streams (both text and binary) through the same primitives. ### Core Architecture Lore consists of two main subsystems: 1. **Storage Subsystem**: A partition-based, content-addressed store that deduplicates all content while enforcing strict per-partition access boundaries. It uses two stores: - **Immutable Store**: Content-addressed using BLAKE3 hashes, storing all file content - **Mutable Store**: A key-value store for branch pointers and other small bookkeeping data 2. **Version Control Subsystem**: Builds revisions, branches, merges, and staging out of storage primitives, treating version control as one consumer of the storage API rather than a privileged layer. ### How Lore Improves Scalability Lore addresses scalability through several key innovations: #### 1. **Horizontal Scaling on Read Path** - The read path is essentially stateless - fragment lookups require no per-session state, cursors, or transactions - Any server that can reach the storage subsystem can answer requests - Read traffic spreads across multiple servers with no "primary" or "leader" for reads - This makes reads embarrassingly parallel #### 2. **Content-Addressed Storage with Chunking** - Files are split into chunks using FastCDC (content-defined chunking) or fixed-size strategies - Single edits in multi-gigabyte files only re-upload changed chunks - Any byte range can be read without materializing the whole file - Clients hold only data they actively use; the rest is fetched lazily #### 3. **Multi-Tenant Support** - Partitions serve as access boundaries, making the system safe for multi-tenant deployments - Strict per-repository access boundaries enable multiple organizations/projects to share one backend - Supports scaling on multiple independent axes simultaneously: - Tenants per deployment - Repositories per tenant - Branches per repository - Fragments per repository - Concurrent users - Peak concurrent sessions - Push and sync throughput #### 4. **Tiered Storage Architecture** - **Edge servers**: Run near clients with high-throughput local caches, serving 90%+ of fragment traffic - **Primary servers**: Hold canonical state (mutable store and recent content) co-located with durable storage - **Cold storage**: For fragments unlikely to be read soon (e.g., S3 backend) - Tiering is invisible to clients - requests go to the nearest endpoint #### 5. **Efficient Branching and Merging** - Branches are lightweight and fast to switch - Per-branch contention is the natural granularity - repositories with many branches parallelize across them - Failed conditional puts surface as conflicts that clients reconcile via sync-and-merge #### 6. **Replication and Failover** - Multiple peer servers replicate fragments and mutable-store state within regions - Fragment puts propagate across peers - Read replicas can be added to scale read throughput ### Key Scalability Features from README - **Unprecedented scalability** of both data and teams - **Easy setup** with on-demand scalability - **Fast and efficient processes** for everyday tasks - **Free branching** with lightweight branches - **History you can trust** with immutable revision chain - **Intuitive interface** for developers - **Full-surface API** for tooling integration ### Comparison with Existing Systems Lore fills gaps where existing systems fall short: - **Git**: Struggles with large binary files and deep history - **Perforce**: Proprietary, expensive, complex administration - **Mercurial/Sapling**: Similar limitations to Git with large files Lore's centralized architecture with content-addressed storage provides better scalability for: - Large binary assets (game development, media) - Deep revision history - Many concurrent branches - Multi-tenant deployments - Global teams with edge caching ### Implementation Lore is implemented in Rust (library, server, and CLI) with publicly specified data formats and wire protocols, allowing other implementations to read, write, and serve Lore data. The system is designed for production deployments with features like resumable transfers, conflict handling, and atomic operations. In summary, Lore improves scalability by combining content-addressed storage with chunking, stateless read paths, tiered storage architecture, efficient branching, and multi-tenant support, making it suitable for large-scale software development projects with diverse content types and global teams.</parameter> </function> </tool_call>