Skip to content

Which LLMs can support the longest outputs, currently?

Generation Details

Date LLM Access Method
Nov 2024 GPT-4 Web UI

Large Language Models (LLMs) vary in their ability to generate long outputs, depending on their architecture, training, and fine-tuning. The models that support the longest outputs typically have undergone specialized training or utilize advanced techniques to handle extended context windows. Here are some of the LLMs known for supporting the longest outputs:

1. LongWriter

LongWriter is specifically designed for generating ultra-long outputs. It can produce coherent text outputs of over 10,000 words in a single generation. This model was developed by researchers at Tsinghua University and Zhipu AI, who addressed the common limitation in LLMs by training LongWriter with a dataset containing documents ranging from 2,000 to 32,000 words. The model uses a pipeline called AgentWrite to break down long-generation tasks into subtasks, which allows it to maintain coherence over long text spans[7][9].

2. Claude 2.1 (Anthropic)

Claude 2.1, developed by Anthropic, is another model that supports long outputs. It has a context window of up to 200,000 tokens, which allows it to process and generate extended text sequences. While the exact maximum output length may vary based on task complexity and prompt structure, Claude's large context window enables it to generate much longer responses than many other models[6].

3. GPT-4 Turbo

OpenAI's GPT-4 Turbo supports an output limit of around 4,096 tokens in its default configuration. Although this is significantly shorter than LongWriter or Claude 2.1, GPT-4 Turbo is commonly used for general-purpose tasks where extremely long outputs are not required[6]. However, recursive prompting techniques can be used to extend its output length in multiple steps[3].

4. Aurelian (LLaMA-based)

Aurelian is a fine-tuned version of LLaMA-2 with a context window of up to 32K tokens. It is designed for long-form writing tasks such as story generation and can produce outputs exceeding 2,000 tokens in one go. While it may not reach the lengths of LongWriter, Aurelian is optimized for generating longer scenes or chapters in creative writing contexts[3][4].

Challenges with Long Outputs

Despite advancements in handling long contexts and generating extended outputs, most LLMs still face challenges:

  • Coherence: Maintaining logical consistency over very long outputs remains difficult as models may lose track of earlier parts of the text.
  • Repetition: Some models tend to repeat phrases or ideas when generating very long responses.
  • Computational Complexity: Generating long outputs requires significant computational resources and memory.

In conclusion, LongWriter currently leads in terms of maximum output length (10,000+ words), followed by models like Claude 2.1 with its large context window (200K tokens). Other models like Aurelian and GPT-4 Turbo can also generate relatively long texts but are more limited compared to specialized models like LongWriter[7][9].

The above text was generated by a large language model (LLM) and its accuracy has not been validated. This page is part of 'LLMs-on-LLMs,' a Github repository by Daniel Rosehill which explores how curious humans can use LLMs to better their understanding of LLMs and AI. However, the information should not be regarded as authoritative and given the fast pace of evolution in LLM technology will eventually become deprecated. This footer was added at 16-Nov-2024.