☰  MENU

What’s inside an AI: from LLMs to agents, and what comes next

A clear explanation of how language models work under the hood, why today’s models reason before they answer, and how they become agents. Plus the state of the art in November 2026, their limits and the technologies coming next.

What’s inside an AI: from LLMs to agents, and what comes next

Illustration: a man at his desk with a brain of nodes connected to his tools

How ChatGPT, Claude and Gemini work, why today’s models reason, how they become agents that work for you, where their limits are and which technologies might come after them

Most of us already use some kind of artificial intelligence: to draft an email, summarise a PDF or put together a presentation. Yet very few people know what actually happens inside when we type a question and an answer appears. And understanding it isn’t just for engineers: knowing how an AI works is the best way to know when to trust it, when not to, and how to get real value from it in a business.

This article covers six stages:

  1. What an LLM is and how it works inside.
  2. Why today’s models reason before answering, and why that changes what they can do.
  3. How an LLM becomes an agent that gets things done.
  4. The state of the art in November 2026.
  5. Their theoretical limits, and whether we are close to reaching them.
  6. The technologies that could complement or replace them, such as Yann LeCun’s “world models”.

1. What an LLM is and how it works inside

LLM stands for Large Language Model. It is a program that has learnt, from enormous amounts of text, to do one thing extremely well: predict the most likely next piece of text. Everything else, from answering questions to writing code, comes from applying that one skill over and over again.

How an LLM generates text

1.1. Tokens: text cut into pieces

The model doesn’t read letters or whole words, but tokens: chunks of text that can be a short word (“the”), part of a longer word (“intelli” + “gence”) or a punctuation mark. Each model has a fixed vocabulary of tokens and turns any text into a list of numbers from that vocabulary.

That’s why AI prices are quoted in “millions of tokens”. It’s also why older models often got it wrong when asked to count the letters in a word: they never saw the letters, only the chunks (more on this in section 2).

1.2. Embeddings: meaning turned into numbers

Each token is converted into an embedding: a long list of numbers (thousands of them) that represents its meaning as a point in a space with many dimensions. During training, words that appear in similar contexts end up as nearby points: “invoice” sits close to “receipt” and far from “mountain”.

This idea is also the basis of RAG (searching your own documents), which we’ll cover in future articles: if meanings are points, finding relevant information means finding nearby points.

1.3. The transformer and attention: reading in context

At the heart of almost every LLM today is the transformer, an architecture introduced by Google researchers in 2017 in the paper Attention Is All You Need. Its key ingredient is the attention mechanism.

Attention solves a problem that is easy to state: the meaning of a word depends on the others. In “Ana left the laptop on the table because it was broken”, what was broken? To interpret “it”, the model works out how much it should “look at” each of the previous words and blends their information in that proportion.

The intuition behind attention

The real mechanism is mathematical: each token produces three vectors (a “query”, a “key” and a “value”); a token’s query is compared with the keys of all the others, and the closer the match, the more weight its value gets. This is repeated across many “heads” in parallel (one might focus on grammar, another on who does what) and across dozens of stacked layers. Layer by layer, each token’s representation becomes richer in context.

At the end, the model produces a score for every possible token in its vocabulary, which is turned into probabilities. One token is picked (not always the most likely one: a touch of randomness, known as “temperature”, makes answers sound more natural), it is added to the text and the whole process starts again. A 500-word answer is hundreds of turns of this loop.

1.4. How it learns: the training stages

The training stages

Pre-training. The model starts with billions of parameters (its internal “dials”) set to random values. It is shown real text with the last word hidden; it makes a prediction, the prediction is compared with the correct word, and all the parameters are nudged slightly so that next time it does a bit better. Repeated trillions of times over text, code and images, the model ends up absorbing grammar, facts about the world and patterns of reasoning. The result is a base model: a very well-read autocomplete that doesn’t yet know how to hold a conversation.

Instruction tuning (SFT). It is trained on thousands of “request → good answer” examples written or reviewed by people. It learns how an assistant should respond.

Preference tuning (RLHF). People (and, increasingly, other AIs) compare two answers and choose the better one. The model learns that judgement: be helpful, honest and harmless. It’s one of the reasons assistants sound so friendly and, at times, so eager to please.

Reinforcement with verifiable rewards. This is the most recent stage, and the one that gave rise to reasoning models. It matters so much that section 2 is devoted to it.

1.5. The context window

The context window is the maximum number of tokens the model can have in front of it at once: your question, attached documents, the conversation so far and its own answer. It is its working memory. Today’s high-end models handle around a million tokens (Claude Fable 5.2 or Mistral Large 4, for example), roughly several books’ worth. Outside that window, the model remembers nothing: it doesn’t learn from your conversations unless the system around it keeps notes and shows them to it again.

💊 Key takeaway
An LLM doesn’t “look up” answers: it predicts the next token, again and again, using attention to read each word in context. What it knows, it absorbed during training; what sits in its context window is its only working memory.

2. Reasoning models: the link between predicting words and doing real work

Until 2024, an LLM answered in one go: it read the question and started writing the answer straight away. Today’s leading models don’t work like that. Before answering, they think. Understanding this difference is key, because today’s frontier models are no longer “just LLMs”: reasoning is what lets them use tools reliably and solve complex problems, and therefore what makes the agents in the next section possible.

How a reasoning model "thinks"

2.1. How they work

Thinking tokens. A reasoning model still predicts tokens, but before the answer it generates a chain of thought: it frames the problem, breaks it into steps, tries one route, checks it and, if it fails, backtracks and tries another. Those steps take up space in the context window, and you usually only see a summary. This is how OpenAI introduced o1 in September 2024.

Compute at answer time (test-time compute). The big news is that there’s a second way to make a model smarter: not just training it more, but letting it think for longer on each question.

Reinforcement with verifiable rewards. How does a model learn to think well? It is given problems whose answers can be checked automatically (a maths result, code that must pass a set of tests), and the attempts that get it right are rewarded: being right is what counts, not sounding right. Nobody teaches it how to reason; it discovers which strategies work. With DeepSeek-R1-Zero (January 2025), trained with reinforcement alone, the authors noticed that the model started, on its own, to pause and re-examine its reasoning, something they called an “aha moment”.

Self-checking and backtracking. The model learns to check intermediate results before carrying on, so an early mistake no longer ruins the whole answer.

Adjustable effort. Almost all current models let you choose how much they think (low, medium, high, maximum). More effort means more correct answers on hard problems, but also more tokens, higher cost and a longer wait.

2.2. An everyday example: how many r’s are in “strawberry”?

For years, a question as simple as “how many r’s are there in strawberry?” tripped up some very well-known models, which often answered “two”. The explanation lies in section 1: the model doesn’t receive letters, it receives tokens. Strawberry arrives as a few chunks (for example “str” + “aw” + “berry”), so the model never “sees” the individual letters and has to guess how many there are.

This mistake was typical above all of older models without reasoning, which answered in one go. Today’s reasoning models almost always get it right, not because they’ve stopped using tokens, but because they spell the word out step by step (s-t-r-a-w-b-e-r-r-y) before counting, or turn to a tool, such as a small program, that does the counting for them. It’s a neat example of reasoning making up for a limitation built into the architecture.

2.3. The leap in results

  • Maths (AIME 2024): GPT-4o solved on average 12% of the problems in this olympiad-style exam; o1 solved 74% with a single attempt. On PhD-level science (GPQA Diamond), o1 reached 78% against GPT-4o’s 56% and, according to OpenAI, became the first model to beat, on that test, the PhD experts the company recruited for the comparison.
  • DeepSeek-R1 (January 2025, an open model) reached 79.8% on AIME 2024. Its reinforcement-only version (R1-Zero) went from 15.6% to 71% during training, according to the January 2025 report; in the revised version published in Nature in September 2025, R1-Zero reaches 77.9%.
  • ARC-AGI (abstract reasoning, easy for people and hard for machines): on ARC-AGI-1, GPT-4o was at around 5% in 2024; in December that year, a preview version of o3 (o3-preview) scored 75.7%. When the harder ARC-AGI-2 launched (March 2025), LLMs without reasoning scored 0% and reasoning models only single digits. Today, according to ARC Prize’s own leaderboard (checked on 11 November 2026), GPT-6 Astra leads with 95%, followed by GPT-6.1 Sol and Gemini 4 Carbon (94.2% each) and Claude Fable 5.2 (93.8%).
  • Real-world programming (SWE-bench Verified), which involves fixing real issues from software projects: GPT-4o solved 33.2% in August 2024 and DeepSeek-R1 reached 49.2% in January 2025. In the independent evaluation by Vals AI (updated on 1 September 2026), Claude Opus 5 solved 97%, so much so that Vals stopped running this benchmark because it was saturated. The figures depend on who measures and with which harness: the official SWE-bench leaderboard, which is updated less often, shows a top score of 79.2%.

2.4. The price of thinking

Thinking costs money, because thinking tokens are billed like generated text even if you never see them. The clearest example is that preview version of o3 on ARC-AGI-1 (December 2024): according to ARC Prize estimates, scoring 75.7% cost about $26 per task; pushing it to 87.5% took roughly 172 times more compute, around $4,560 per task. Costs have fallen sharply since then, but the logic is the same: more accuracy in exchange for more money and more seconds (or minutes) of waiting. For simple questions, a low effort level is usually enough.

2.5. Their limits

  • Uneven abilities. Models solve olympiad maths problems and, at the same time, fail at tasks a person would find trivial. Their abilities are uneven (jagged, as METR puts it): brilliant at well-defined, checkable tasks, where reinforcement works best (maths, code), and clumsier at ambiguous, long tasks with little context, which happen to be exactly the day-to-day tasks of a business. There the gains are smaller and human review is still essential.
  • The visible reasoning isn’t always the real reasoning. In a 2025 Anthropic study, when models were given a hint containing the answer and used it, they mentioned it in their reasoning only 25% of the time (Claude 3.7 Sonnet) and 39% of the time (DeepSeek-R1). The chain of thought helps with oversight, but it isn’t a faithful confession.
  • Overthinking. Reasoning models sometimes spend hundreds of tokens on trivial questions, as described in the study Do NOT Think That Much for 2+3=? (2024).

2.6. Why it matters for agents

An agent chains together dozens of steps: look up a figure, open a file, run a calculation, check the result. If each step carries a small chance of error and there’s no way to spot and fix it, the mistakes pile up. Reasoning provides exactly that: plan before acting, check what each tool returns, and correct course. It’s the bridge between “predicting the next word” and “finishing a job”.

💊 Key takeaway
Today’s frontier models think before they answer: they generate intermediate steps, check them and correct course. They learnt this through reinforcement on verifiable problems, and it’s what lets them use tools reliably. You can control how much they think: more effort means more accuracy, but also more cost and a longer wait.

3. From LLM to agent: the role of the “harness”

On its own, an LLM only produces text. It can’t open your email, query a database or save a file. To actually do things, it needs a harness: the program that surrounds it and connects it to the world.

Agent = LLM + harness

A harness has five parts:

  • The loop. The harness gives the model the goal, the model decides on a step, the harness carries it out, returns the result to the model, and the cycle repeats until the job is done. This think → act → observe cycle is what turns an answer into a piece of work.
  • Tools. Functions the model can ask to have run: search the web, read a PDF, run a command, send a draft. The model doesn’t run them itself: it writes a structured request (“call find_invoice for customer X”) and the harness executes it. Since late 2024 there has been an open standard, the Model Context Protocol (MCP), that lets you plug tools into almost any agent.
  • Instructions. The “system prompt” and the project’s rule files: who it is, how it works, what style it uses.
  • Memory. Notes the agent saves and rereads, history or indexed documents. Since the model forgets everything outside its context window, memory is handled by the harness.
  • Permissions. What it can do on its own and what needs your approval. This is the most important part in a business: an agent with access to your email and no permission controls is a risk.

A telling fact: the same model performs very differently depending on the harness. That’s why evaluation organisations such as METR test models inside different harnesses (for example Claude Code or Codex) before measuring what they can do.

Examples available today (November 2026)

  • Coding agents: Claude Code (Anthropic), Codex (OpenAI, with an open-source command-line version), Cursor (an editor with local and cloud agents) and Antigravity (Google). They read an entire project, edit files, run tests and propose changes for review.
  • General-purpose cloud agents: Manus, Claude Cowork, OpenAI’s Dots, Meta’s Muse or Grok Bot. They have their own virtual computer with a browser and files, and handle tasks such as research, filling in spreadsheets or preparing reports.
  • Open-source, self-hosted agents: OpenClaw and Hermes Agent (from Nous Research), both MIT-licensed. You install them on your own computer or server, they support many models (including local ones) and you control them from WhatsApp, Telegram, Slack or Teams. They’re worth a look when privacy and control over your data matter.
  • Enterprise platforms: Mistral, for example, announced Mistral Studio on 5 November for building managed agents, in its cloud or on your own servers.

💊 Key takeaway
Agent = LLM + harness. The model decides; the harness executes, remembers and sets limits. When you evaluate an AI tool for your business, ask about the harness as much as the model: which tools it uses, where it stores memory and what it can do without asking you.

4. The state of the art in November 2026

The pace is dizzying: in the first half of November alone, several leading models were released. Think of this as a snapshot, not a final ranking.

Timeline

Frontier (closed) models:

  • Google · Gemini 4 Carbon (10 November) and Gemini 4 Argon (13 October). Carbon is geared towards long agentic tasks and cybersecurity, and tops LMArena’s text leaderboard, where people vote blind between two answers, with 1,526 points (data from 9 November), ahead of Argon (1,522).
  • Anthropic · Claude Fable 5.2 (9 November), with a one-million-token context window, up to 128,000 output tokens and adaptive reasoning, at $8/$40 per million input/output tokens. Anthropic says that in pre-release testing it formalised the proof of Fermat’s Last Theorem in 30 hours. That’s the company’s claim and has yet to be independently verified. Its twin, Mythos 5.2, is only available to vetted organisations in cybersecurity and biomedicine.
  • OpenAI · GPT-6 Astra (3 September) and GPT-6.1 Sol (29 September), which according to OpenAI nearly matches Astra at agentic coding for a fifth of the price. GPT-6 Cyber (5 November) is restricted to approved customers.
  • Meta · Muse (Muse Wonder, Muse Spark) and xAI · Grok 4.7 (21 September).

Open models (downloadable weights you can run on your own servers) are not far behind: Mistral Large 4, a European model released on 29 October, with a trillion parameters (of which only about 50 billion are active for each token, thanks to a technique called Mixture of Experts) and a one-million-token context window. Alongside it are Kimi K3, GLM 5.3, DeepSeek V4 and Qwen 3.8. On LMArena, the best open model (Kimi K3) is around 40 points behind the leader.

Three trends that matter to a small business:

  1. Autonomous working capacity is growing exponentially. METR measures the length of tasks (in terms of how long a human expert would take) that an agent can complete with a 50% success rate. In its January 2026 update, that length was doubling every ~196 days since 2019 and every ~89 days since 2024. METR already warns that its measurements above 16 hours aren’t reliable with its current task suite: the best models are running out of exam questions.
  2. The price of capability is falling. Mid-range models such as GPT-6.1 Sol ($2/$10 per million tokens) perform almost as well as the high-end models of a few weeks ago.
  3. Open models are catching up with closed ones with a lag of a few months, which makes local AI viable for sensitive data.

💊 Key takeaway
There’s no single “best model” today: Google leads on user preference, Anthropic and OpenAI compete on agents and coding, and open models (such as Europe’s Mistral Large 4) are only months behind. For a business, the useful question isn’t “which is best?” but “which is good enough for this task, at what cost and with what level of privacy?”.

5. Theoretical limits: are we close to the ceiling?

5.1. Hallucinations

An LLM can state something false with complete confidence. This isn’t an occasional glitch but a consequence of its design: the model generates the most plausible text, and plausible isn’t always true. A September 2025 OpenAI paper adds another cause: the tests used to evaluate models reward correct answers and give nothing for “I don’t know”, so training rewards guessing. Its proposal is to penalise confident errors more than abstaining.

Things have improved a lot, but the problem hasn’t gone away. In Artificial Analysis’s AA-Omniscience test, even a leading model like GPT-6.1 Sol, when it doesn’t know the answer, abstains only about half the time; the rest of the time, it makes something up. Practical takeaway: for your company’s data, the AI should cite documents (RAG) and a person should check anything important.

5.2. No model of the world… or not quite

An LLM learns from language, which is only a partial description of the world. It knows that a falling glass “breaks” because it has read so a thousand times, not because it understands gravity. This is Yann LeCun’s central argument: text is too thin a signal to learn physics, cause and effect, and planning the way a child does by observing. The debate is still open: today’s models, also trained on images and video, show some grasp of physics, but it is fragile.

5.3. The data wall

Epoch AI estimates that high-quality public text written by humans will run out as a training resource around 2028 (within a range of 2026 to 2032). The industry is responding with synthetic data (generated by other AIs), with private and multimodal data (video, audio) and, above all, with reinforcement learning, where the model generates its own attempts and learns from the ones that work. The wall is real, but it’s being worked around.

5.4. Scaling laws, compute and energy

Since 2020 we’ve known that models improve predictably as parameters, data and compute increase, but with diminishing returns: each improvement costs many times more. The physical cost is real: the International Energy Agency (IEA) expects data centre electricity use to rise from about 415 TWh in 2024 to around 945 TWh in 2030, just under 3% of global electricity. AI is the main driver of that increase.

Reasoning and reinforcement (section 2) have opened up new levers for improvement, but they also consume compute: thinking for longer on each answer multiplies the bill.

5.5. Context isn’t perfect memory

A model accepting a million tokens doesn’t mean it uses them all equally well. Chroma’s Context Rot study (July 2025) tested 18 models and found that all of them perform worse as the input grows, even on simple tasks. That’s why a good agent doesn’t “put everything in the context”: it selects, summarises and searches for what’s relevant. This is known as context engineering.

Close to the ceiling?

The capability metrics show no sign of stalling: METR’s task horizon keeps growing exponentially and November’s models outperform September’s. But the structural limitations (hallucination, the lack of a physical model of the world, learning that stops after training, uneven abilities and energy cost) won’t be solved by scale alone. The most honest reading is this: there’s plenty of room for improvement within the current paradigm, but it probably isn’t enough to reach general intelligence. Hence the interest in alternatives.

💊 Key takeaway
Today’s AIs are still improving very quickly, but they carry design limits: they make things up when they don’t know, they learn from text rather than from the world, and they use a lot of energy. For your business: use them with cited sources, human review for anything critical and good context management.

6. What comes next: technologies that could complement or replace LLMs

6.1. JEPA and Yann LeCun’s world models

Yann LeCun, Turing Award winner and for years Meta’s chief AI scientist, left the company in November 2025 and founded AMI Labs (Advanced Machine Intelligence) in Paris. In March 2026 the company raised $1.03 billion at a pre-money valuation of $3.5 billion, with offices in Paris, New York, Montreal and Singapore. Its stated goal: systems that understand the world, have persistent memory, reason, plan and are controllable.

Its technical proposal is JEPA (Joint Embedding Predictive Architecture), put forward by LeCun in 2022.

LLM vs JEPA

The key difference: an LLM predicts the next token, that is, the exact detail. JEPA doesn’t try to predict every pixel of the next frame of a video, because the real world has countless possible futures and most of its details (the exact reflection on a pane of glass) are unpredictable. Instead, it turns the current observation and the future one into abstract representations and learns to predict one from the other in that abstract space. That way it keeps what is predictable (the glass falls) and ignores the noise. With a model like this, a system can plan: imagine the consequences of several actions and choose the best one.

There are concrete results: V-JEPA 2 (Meta, June 2025), trained on video and fine-tuned with just 62 hours of robot data, enabled robotic arms to pick up and place objects in new labs without task-specific training, with success rates of 65–80% on that task. But AMI Labs acknowledges that it is in the research phase, with no products in the short term: scientific papers first, then industrial applications (manufacturing, automotive, aerospace, healthcare) with partners such as the digital health company Nabla.

6.2. Other approaches worth knowing about

  • Energy-based models (EBMs). Another idea championed by LeCun: instead of generating an answer, the system scores how well each possible answer “fits” the question (low energy means compatible) and searches for the best one. This allows reasoning as optimisation, not just as writing word by word.
  • State space models (SSMs, such as Mamba). They replace or combine attention with a “compressed memory” that is updated step by step. Their cost grows linearly with the length of the text, instead of shooting up as it does with classic attention. Hybrid transformer + Mamba models are already in production.
  • Diffusion language models. Instead of writing from left to right, they generate a complete draft and refine it in parallel, like image generators. They are much faster. Examples: Inception’s Mercury and Google’s experimental Gemini Diffusion.
  • Neurosymbolic AI. It combines neural networks with formal logic and verifiable rules. We already see it in practice: models that write proofs in formal languages (such as Lean) which an automatic checker then verifies. It’s a direct way to tackle hallucinations in tasks where correctness matters.

Replace or complement? The most likely outcome in the medium term is a combination: LLMs as the language interface and orchestrator, world models for the physical world and planning, symbolic verifiers for anything that must be exact, and more efficient architectures to bring costs down. Today’s agents already work this way: one model coordinating specialised tools.

💊 Key takeaway
Yann LeCun is betting on world models (JEPA) that learn by observing and predict in the abstract, not word by word. It’s promising research, with robots already planning simple tasks, but no products yet. The likely future isn’t “LLMs or the alternative” but hybrid systems.

In short

  • An LLM predicts the next token using attention over its context. Everything else is built on top of that.
  • Reasoning models think before they answer: they generate intermediate steps, check them and correct course. That’s why frontier models are no longer “just LLMs”.
  • An agent is an LLM plus a harness: a loop, tools, memory, instructions and permissions.
  • In November 2026 the frontier is set by Google, Anthropic and OpenAI, with open models only months behind, which opens the door to private, local AI.
  • The limits (hallucinations, data, energy, understanding of the world) are real, but there is still room to grow within the current paradigm.
  • What comes next (world models, SSMs, diffusion, neurosymbolic AI) points towards hybrid systems.

At RPA Technologies we help small businesses and professionals put all this into practice with their feet on the ground: assistants and agents that work with your documents, local AI when privacy demands it, and training so your team understands what it’s using. If there’s a topic you’d like us to cover, let us know in the comments.

👉 Find out more at rpatec.com

Sources

Listed in the order they appear in the article. Leaderboard data checked on 11-11-2026.

Latest AI articles ItemWhat’s inside an AI: from LLMs to agents, and what comes next