LLM: Large Language Model

• Architecture: Transformer (Vaswani et al 2017), decoder-only in most models
• Training: pretrain on text corpus (TB) + RLHF + instruction tuning
• Parameters (2026): small 8B, mid 70B, frontier 400B+ (sparse MoE equivalent of 1.8T)
• Inference cost: $0.15-15 per 1M tokens (API) vs self-host (A100/H100/B100)
• Context window 2026: Claude 1M stable, Gemini 2M, GPT-5 also 1M

# OpenAI API call
curl https://api.openai.com/v1/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "model": "gpt-5",
    "messages": [{"role": "user", "content": "Explain TCP vs UDP"}]
  }'

• RAG
• Transformer
• Fine-tuning

Large Language Models (LLMs) are primarily built on the transformer architecture, which utilizes self-attention mechanisms to process and generate text. This architecture allows LLMs to weigh the importance of different words in a sentence relative to each other, enabling them to capture context more effectively than previous models.

The training process involves two main phases: pre-training and fine-tuning. During pre-training, the model is exposed to a vast corpus of text, learning to predict the next word in a sentence given the preceding words. This phase is unsupervised and helps the model develop a general understanding of language structure, grammar, and factual knowledge.

Once pre-training is complete, the model undergoes fine-tuning on specific tasks or datasets. This phase is often supervised, involving labeled data that teaches the LLM to perform particular functions, such as sentiment analysis or question answering.

Key components of LLMs include:

• Parameters: The weights and biases in the model, which can range from billions (e.g., 8B) to trillions (e.g., 1.8T), directly influencing the model's ability to learn and generalize.
• Context Window: The number of tokens the model can consider at once, which has expanded from 200k to 2M tokens in advanced models, allowing for more coherent and contextually aware outputs.

Overall, LLMs represent a significant advancement in natural language processing, enabling a wide array of applications from chatbots to complex coding tasks.