CreativeInstruct: Scalably Teaching LLMs to Balance Quality, Creativity, and Diversity
Source: https://arxiv.org/abs/2608.07460
Author: Ananya Sahu, Mohit Bansal, Elias Stengel-Eskin
Date: 2026-08-07
Category: models
Type: paper
Quality: 4/5
Tags: llm-training, creativity, diversity, instruction-tuning, arxiv
English Summary
CreativeInstruct is a scalable instruction-tuning method that teaches LLMs to balance creative, base-model-like generations with the quality of post-trained models, by learning to inject special [StartCreativity] spans that bias generation toward creativity. It introduces a structural diversity metric based on graph edit distance capturing narrative-level variation. On narrative generation, CreativeInstruct matches or exceeds the diversity of multi-model baselines without requiring multiple models at inference time. Human evaluators rated CreativeInstruct generations as more creative than post-trained LLMs' generations in 70.3% of cases. GRPO applied to a CreativeInstruct checkpoint improves by ~4% on AMC and ~5% on MATH over the same training on the post-trained checkpoint.
中文概要
CreativeInstruct 是一种可扩展的指令微调方法，通过注入 [StartCreativity] 标签使 LLM 在生成质量与创造性之间取得平衡论文引入了基于图编辑距离的结构化多样性指标，能捕捉纯词法指标遗漏的叙事层面差异人工评估中 70.3% 的情况下 CreativeInstruct 生成被评为更具创造力，且将创造性模型作为 RL 子底时，GRPO 在 AMC 上提升约 4%在 MATH 上提升约 5%
*Added via external-scan on 2026-08-11*