Edge0 Open-Sources a 145B Model for Local Frontier AI

Introduction

Running a capable language model locally usually means accepting a trade-off: smaller models fit on consumer hardware, while larger models offer better reasoning and coding but demand more memory, bandwidth, and power. Edge0’s new release attempts to change that equation rather than simply shrinking a cloud-trained model. The company has open-sourced Edge0-145B and Edge0-120B alongside an inference framework, deployment tools, and an application.

Key points

• A large model aimed at personal computers: Edge0 reports that Edge0-145B can run on a compatible laptop with about 29GB of peak memory. Its stated measurements include more than 1,100 tokens per second for prompt processing, over 80 tokens per second for decoding, typical power usage of roughly 120W, and support for 128K to 256K context. These are vendor-reported figures and will vary with hardware and configuration.
• Sparse experts with shared weights: The model has 145B total parameters, but approximately 10B participate in each inference step. Groups of 64 experts share two large matrices, while each expert retains a smaller set of parameters for its individual behavior. This reduces the amount of data that must be stored and moved.
• A storage and compute pipeline: The runtime coordinates GPU memory, system memory, and SSD access. Frequently used experts can remain in GPU memory, while others are loaded on demand. The CPU and GPU handle different parts of the workload in parallel, and speculative decoding is used to spread weight-loading costs across multiple output tokens.
• Designed for multi-step work: The demonstrations go beyond chat. Edge0 describes workflows that plan, write, execute, and debug code, then visually inspect results in projects such as 3D scenes, games, hardware programs, and PCB designs.
• Benchmark claims require context: The company publishes results for DeepSWE 1.1, GPQA-Diamond, IFBench, Terminal-Bench 2.1, and AA-LCR, comparing them with selected cloud models. Those results apply to named models and evaluation settings; they do not establish universal superiority.

Why it matters

If the reported measurements can be independently reproduced, the main contribution is not only the model’s parameter count. Edge0 is treating architecture, training, weight loading, and runtime scheduling as one local-inference problem. That approach could make advanced coding and agent workflows more practical for users who cannot send repositories, design files, or other sensitive data to a cloud service.

Local execution is not free, however. A user still needs suitable hardware, storage, electricity, and software maintenance. The material also comes from the project itself and relies heavily on internal testing. Community validation will therefore matter as much as the headline specifications. Compatibility, installation complexity, licensing, long-task reliability, and performance on ordinary machines will determine whether the release becomes a meaningful alternative to cloud APIs.

More broadly, Edge0’s launch reflects a shift in on-device AI. The question is no longer only how to make a model smaller, but how to coordinate sparse computation, memory tiers, and heterogeneous processors so that a much larger model can operate within consumer constraints.

Source: Hacker News