MAGNET:通过去中心化自研究和BitNet训练实现自主专家模型生成
📄 中文摘要
MAGNET(模型自主增长网络)是一种去中心化系统,旨在在普通硬件上自主生成、训练和服务领域专家语言模型。MAGNET集成了四个主要组件:(1)自研究,自动化数据集生成、超参数探索、评估和基于错误的迭代的自主机器学习研究流程;(2)BitNet b1.58三元训练,支持通过bitnet.cpp在没有GPU硬件的情况下进行CPU本地推理;(3)基于DiLoCo的分布式合并,实现领域专家的高效通信聚合;(4)在HOOTi EVM链上进行链上贡献跟踪。通过三个案例研究验证了自研究的有效性:视频安全分类(平衡准确率为0.9287)。
📄 English Summary
MAGNET: Autonomous Expert Model Generation via Decentralized Autoresearch and BitNet Training
MAGNET (Model Autonomously Growing Network) is a decentralized system designed for the autonomous generation, training, and serving of domain-expert language models on commodity hardware. It integrates four key components: (1) autoresearch, an autonomous ML research pipeline that automates dataset generation, hyperparameter exploration, evaluation, and error-driven iteration; (2) BitNet b1.58 ternary training, enabling CPU-native inference via bitnet.cpp without GPU hardware; (3) DiLoCo-based distributed merging for communication-efficient aggregation of domain specialists; and (4) on-chain contribution tracking on the HOOTi EVM chain. The effectiveness of autoresearch is validated through three case studies, including video safety classification with a balanced accuracy of 0.9287.
Powered by Cloudflare Workers + Payload CMS + Claude 3.5
数据源: OpenAI, Google AI, DeepMind, AWS ML Blog, HuggingFace 等