向内扩展,而非向上扩展?使用 GPT-5 和脆弱提示测试厚重引用上下文分析
📄 中文摘要
该研究测试了大型语言模型(LLMs)是否能够通过对单一复杂案例进行厚重的文本基础解读,支持解释性引用上下文分析(CCA),而非通过扩展类型标签来实现。研究强调了提示敏感性分析作为一种方法论问题,通过在平衡的 2x3 设计中变化提示框架和支撑。以 Chubin 和 Moitra(1975)中的脚注 6 及 Gilbert(1977)的重构作为探针,实施了一个两阶段的 GPT-5 流程:首先进行引用文本的表面分类和期望传递,随后使用引用和被引用的完整文本进行跨文档的解释性重构。在 90 次重构中,模型生成了 450 个不同的假设。通过细读和归纳分析,研究揭示了模型在处理复杂文本时的能力与局限性。
📄 English Summary
Scaling In, Not Up? Testing Thick Citation Context Analysis with GPT-5 and Fragile Prompts
This study tests whether large language models (LLMs) can support interpretative citation context analysis (CCA) by scaling in thick, text-grounded readings of a single hard case rather than scaling up typological labels. It foregrounds prompt sensitivity analysis as a methodological issue by varying prompt scaffolding and framing in a balanced 2x3 design. Using footnote 6 in Chubin and Moitra (1975) and Gilbert's (1977) reconstruction as a probe, a two-stage GPT-5 pipeline is implemented: a citation-text-only surface classification and expectation pass, followed by cross-document interpretative reconstruction using the citing and cited full texts. Across 90 reconstructions, the model produces 450 distinct hypotheses. Close reading and inductive analysis reveal the model's capabilities and limitations in handling complex texts.
Powered by Cloudflare Workers + Payload CMS + Claude 3.5
数据源: OpenAI, Google AI, DeepMind, AWS ML Blog, HuggingFace 等