Meet the world's smallest AI supercomputer — it packs 'doctorate-level intelligence', its makers say, and can fit into your pocket

The portable computing powerhouse is capable of running 120-billion-parameter LLMs, roughly three times larger than GPT-3, without needing to access the internet or the cloud. When you purchase through links on our site, we may earn an affiliate commission. Here’s how it works. Get the world’s most fascinating discoveries delivered straight to your inbox.Become a Member in Seconds Unlock instant access to exclusive member features.You are now subscribedYour newsletter sign-up was successfulWant to add more newsletters?Delivered DailyDaily NewsletterSign up for the latest discoveries, groundbreaking research and fascinating breakthroughs that impact you and the wider world direct to your inbox.Once a weekLife's Little MysteriesFeed your curiosity with an exclusive mystery every week, solved with science and delivered direct to your inbox before it's seen anywhere else.Once a weekHow It WorksSign up to our free science & technology newsletter for your weekly fix of fascinating articles, quick quizzes, amazing images, and moreDelivered dailySpace.com NewsletterBreaking space news, the latest updates on rocket launches, skywatching events and more!Once a monthWatch This SpaceSign up to our monthly entertainment newsletter to keep up with all our coverage of the latest sci-fi and space movies, tv shows, games and books.Once a weekNight Sky This WeekDiscover this week's must-see night sky events, moon phases, and stunning astrophotos. Sign up for our skywatching newsletter and explore the universe with us!Join the clubGet full access to premium articles, exclusive features and a growing list of member rewards.A U.S. startup has developed what it claims is the world’s smallest artificial intelligence (AI) supercomputer. Packed with high-performance hardware and plenty of RAM, company representatives say it can run "Ph.D. intelligence" AI models — despite being compact enough to tuck into your pocket. This means they're capable of autonomous problem solving, abstract reasoning and strategic planning.The "AI Pocket Lab," as its creators at Tiiny AI have branded the device, is capable of running a complex 120-billion-parameter large language model (LLM) locally, without any reliance on internet connectivity. You would ordinarily need data-center-class infrastructure to run these systems, and it opens up the possibility of local expert-level coding capabilities, document assessment and refinement, or multi-step reasoning.It's built around a 12-core ARM processor, of the kind commonly found in smartphones, laptops and tablets. Despite its tiny frame — the device measures just 5.59 × 3.15 × 1.00 inches (14.2 × 8 × 2.53 cm) — it packs 80 GB of LPDDR5X RAM. Most current laptops come with between 8 GB and 32 GB RAM, by way of comparison.A massive 48 GB of the Pocket Lab's RAM is also reserved exclusively for the neural processing unit (NPU), a chip optimized for AI-related computations. Both Intel and AMD have been manufacturing processors for a few years that include dedicated NPUs to handle AI workloads and to meet Microsoft's 40 trillion operations per second (TOPS) threshold to run AI features on Windows 11.The Pocket Lab qualifies as a supercomputer (rather than a standard mini-PC or workstation) because of its computational power, capable of running workloads — specifically local inference on 100 billion-plus parameter language models — that normally require multi-GPU, data-center-class systems. Current models the device can run include GPT-OSS 120B, large Phi models and high-parameter Llama family models.This is part of a recent push towards edge computing for AI, in an attempt to reduce some of the power constraints and environmental impact of distributed AI processing.While it's a far cry from rivaling the world's most powerful supercomputers, the AI Pocket Lab is capable of delivering 190 TOPS of computing power between its NPU and CPU. It represents another step towards miniaturization in the wake of Nvidia's recently announced Project Digits mini PC. While it doesn't pack the same horsepower as the Nvidia project, it's a fraction of the size.Get the world’s most fascinating discoveries delivered straight to your inbox.To pack so much power into such an unassuming chassis, the Tiiny AI team leaned on a number of technologies and optimizations. Key among them was something the company calls TurboSparse — an innovation that allows massive LLMs to run faster on more limited hardware by ensuring a system only calls on the parts of a model that it needs at any given moment. While traditional models use every parameter for each word of processing/output, a TurboSparse model only uses specific parameters per step.—'Quantum AI' algorithms already outpace the fastest supercomputers, study says—New supercomputing network could lead to AGI, scientists hope, with 1st node coming online within weeks—China is building a constellation of AI supercomputers in space — and just launched the first piecesAnother important feature is PowerInfer, which allows for heterogeneous scheduling of the device’s CPU, GPU and NPU. This means that each processor is only given the workload that it's most capable of handling, which makes the entire system more efficient overall and reduces power draw. PowerInfer also includes intelligent power management, deciding when full power is necessary and when it's possible to use less, in part by eliminating unnecessary calculations.The implications of a miniature AI supercomputer go beyond reducing our reliance on environmentally harmful data centers. It's a boon to privacy, with users able to deploy the power of a sophisticated LLM without being connected to the internet and without their data being processed in the cloud by third parties, while enabling AI access in fieldwork situations such as remote research stations, or on ships or aircraft out of connectivity range.Alan is a freelance tech and entertainment journalist who specializes in computers, laptops, and video games. He's previously written for sites like PC Gamer, GamesRadar, and Rolling Stone. If you need advice on tech, or help finding the best tech deals, Alan is your man.You must confirm your public display name before commentingPlease logout and then login again, you will then be prompted to enter your display name.