DeepSeek has officially open-sourced its Fire-Flyer File System (3FS), a high-performance distributed file system tailored for AI workloads. This release marks a significant milestone in DeepSeek’s Open Source Week, showcasing their commitment to advancing AI infrastructure.
3FS is engineered to harness the full potential of modern SSDs and RDMA networks, delivering exceptional data access speeds. In a 180-node cluster, 3FS achieved an impressive aggregate read throughput of 6.6 TiB/s.
Additionally, during the GraySort benchmark on a 25-node cluster, it reached a throughput of 3.66 TiB/min. For inference tasks, the system’s KVCache demonstrated peak query throughput exceeding 40 GiB/s per client node.
The architecture of 3FS is designed with disaggregation, allowing it to combine the throughput of numerous SSDs and the network bandwidth of multiple storage nodes.
This design enables applications to access storage resources without concern for data locality. Moreover, 3FS implements Chain Replication with Apportioned Queries (CRAQ) to ensure strong consistency, simplifying application development.
Beyond its impressive performance metrics, 3FS supports a range of AI-related tasks. These include data preparation, efficient dataset loading, high-throughput parallel checkpointing, and serving as a cost-effective alternative to DRAM-based caching for inference workloads.
This open-source release follows DeepSeek’s previous initiatives to democratize AI technology. Notably, the company introduced DeepSeek-V2, a Mixture-of-Experts language model designed for economical training and efficient inference.
DeepSeek-V2 features a context length of 128K tokens and incorporates innovative architectures like Multi-head Latent Attention (MLA) and DeepSeekMoE. These advancements have significantly enhanced performance while reducing training costs.
The open-sourcing of 3FS underscores DeepSeek’s dedication to fostering collaboration and innovation within the AI community. By providing access to high-performance tools like 3FS, DeepSeek aims to empower developers and researchers to push the boundaries of AI applications.
DeepSeek’s open-source approach has significantly influenced the AI landscape, prompting other tech giants to accelerate their AI developments. For instance, Tencent recently unveiled its AI model, Hunyuan Turbo S, claiming faster response times than DeepSeek’s R1. This competitive environment underscores the impact of DeepSeek’s strategies on global AI advancements.
I’m David Paul, a news writer at DeepSeek Insider, where I cover the latest developments in artificial intelligence, large language models (LLMs), and emerging AI tools in the market. With a keen interest in these cutting-edge technologies, I deliver insightful and timely updates to keep our audience informed and engaged.