📄 中文摘要
作者购买了一台IBM POWER8 S824服务器，配备16个核心、128个硬件线程和768GB的DDR3内存。这是一款2014年的数据中心级PowerPC系统，最初价格超过3万美元，但由于市场需求低迷，作者以远低于该价格的成本购得。POWER8服务器的主要缺点在于其软件兼容性差、功耗高和重量大。然而，这款服务器具备独特的指令集，其中的vec_perm指令能够在一个周期内完成GPU需要80个操作才能实现的功能。这一特性使得POWER8在特定类型的LLM推理优化中表现出色。
我在768GB的IBM POWER8服务器上运行LLM（比你想象的更快）
出处: I Run LLMs on a 768GB IBM POWER8 Server (And It's Faster Than You Think)
发布: 2026年2月13日
📄 English Summary
I Run LLMs on a 768GB IBM POWER8 Server (And It's Faster Than You Think)
The author acquired an IBM POWER8 S824 server featuring 16 cores, 128 hardware threads, and 768GB of DDR3 RAM. This datacenter-class PowerPC system from 2014 originally cost over $30,000, but the author purchased it for significantly less due to low demand for POWER8 servers. These servers face challenges such as poor software compatibility, high power consumption, and substantial weight. However, they possess a unique instruction set, notably the vec_perm instruction, which can perform in one cycle what would take 80 operations on a GPU. This capability makes the POWER8 particularly effective for specific types of LLM inference optimization.