📄 中文摘要
高斯过程 (GP) 因其强大的建模能力在机器学习领域备受关注，但其计算复杂度随数据量呈立方增长，严重限制了其在大规模数据集上的应用。本研究提出了一种名为“投影似然”（PL）的新型训练目标，旨在通过利用数据在低维线性投影下的信息来提高GP学习的效率。PL 的核心思想是将高维数据投影到低维子空间，然后在这个低维空间中构建和训练GP模型。通过这种方式，可以显著降低GP训练的计算成本，同时尽可能保留原始数据中的关键信息。我们推导了与PL相关的信息损失的闭式表达式，这为量化投影操作对模型性能的影响提供了理论基础。经验证据表明，通过在单位球面上进行随机投影，可以有效减少这种信息损失，从而在保持模型准确性的同时实现计算效率的提升。与精确GP训练以及其他现有的近似GP方法相比，PL在准确性和计算效率方面均展现出卓越的性能。具体而言，PL方法能够在大规模数据集上实现与精确GP相媲美的预测精度，而训练时间却大幅缩短。这种效率的提升对于实际应用中处理海量数据至关重要。PL方法的操作流程包括：首先，选择一个合适的低维子空间维度；其次，通过随机投影或数据驱动的方式生成投影矩阵；接着，在投影后的低维数据上构建和优化GP模型；最后，利用训练好的GP模型进行预测。该方法为解决大规模GP学习的计算瓶颈提供了一条有效途径，使其能够更好地应用于需要高精度预测但又受限于计算资源的实际场景。
📄 English Summary
Efficient Gaussian process learning via subspace projections
Gaussian processes (GPs) are powerful models in machine learning, but their cubic computational complexity with respect to data size severely restricts their application to large datasets. This work introduces a novel training objective, termed “projected likelihood” (PL), designed to enhance the efficiency of GP learning by leveraging information from lower-dimensional linear projections of the data. The core idea of PL involves projecting high-dimensional data into a lower-dimensional subspace, and subsequently constructing and training the GP model within this reduced space. This approach significantly decreases the computational cost of GP training while aiming to preserve critical information from the original data. A closed-form expression for the information loss associated with PL is derived, providing a theoretical foundation to quantify the impact of the projection operation on model performance. Empirical evidence demonstrates that random projections on the unit sphere can effectively mitigate this information loss, thereby achieving computational efficiency improvements while maintaining model accuracy. Compared to exact GP training and other existing approximate GP methods, PL exhibits superior performance in terms of both accuracy and computational efficiency. Specifically, the PL method achieves prediction accuracy comparable to exact GPs on large datasets, with substantially reduced training times. This efficiency gain is crucial for practical applications dealing with massive amounts of data. The operational procedure of the PL method includes: first, selecting an appropriate lower-dimensional subspace; second, generating projection matrices either through random projections or data-driven approaches; subsequently, constructing and optimizing the GP model on the projected lower-dimensional data; finally, utilizing the trained GP model for predictions. This method offers an effective solution to the computational bottleneck in large-scale GP learning, enabling its broader application in real-world scenarios that demand high-accuracy predictions but are constrained by computational resources.