You may have heard that AI got better because labs "scaled up". Bigger models, more chips, more data. That is roughly right, and there is real research behind it. It also explains something practical for a business: why the part you control, your own records, matters more than it might seem.


This post explains scaling laws without the maths, as of November 2026, and what follows from them for a small or mid-sized company.


What a scaling law is


A scaling law is a measured pattern. Researchers train many models of different sizes on different amounts of data with different amounts of computing, and then plot how well each one predicts the next word. If the pattern is regular, you can predict how much better a model will get when you give it more of something.


Two ingredients show up again and again:


  • Compute: how much calculation is spent on training.
  • Data: how much text, in the form of "tokens" (small pieces of words), the model sees.

The third is model size: the number of parameters, the adjustable numbers inside the model.


What the research found


In 2020, Jared Kaplan and colleagues at OpenAI published "Scaling Laws for Neural Language Models". They found that the model's error followed a smooth power-law pattern as model size, dataset size and training compute increased, across more than seven orders of magnitude. They also found that details such as the exact shape of the network mattered little over a wide range, and that larger models were more efficient with the data they saw. Their reading was that, for a fixed budget, the best plan was a very large model trained on a modest amount of data.


In 2022, Jordan Hoffmann and colleagues at DeepMind published "Training Compute-Optimal Large Language Models" and revised that conclusion. Across more than 400 models, they argued that existing large models were significantly undertrained: labs had been making models bigger while keeping the amount of training data about the same. Their finding was that model size and training data should grow together, so that doubling the model means doubling the tokens. To test it, they trained Chinchilla, a 70 billion parameter model, with the same compute as the 280 billion parameter Gopher but about four times more data. They reported that Chinchilla outperformed Gopher, GPT-3 and other larger models across a wide range of tasks, and that being smaller made it cheaper to fine-tune and run.


The lesson from the two papers together is simple: for a given amount of compute, how much good data you use matters as much as how big the model is. The balance was wrong once, and it was corrected by looking at the data side.


Caveats worth knowing


  • These are patterns observed in specific setups. They describe what happened for the models and data studied. They are not laws of nature, and they say nothing guaranteed about the next model.
  • They measure how well a model predicts text, not how useful it is for your task. Usefulness depends on what the model was tuned to do and how you use it.
  • "More data" assumes the data is good. Which brings us to quality.

Quality counts, not just quantity


The "Textbooks Are All You Need" paper (Microsoft Research, 2023) trained a small 1.3 billion parameter code model on about 6 billion tokens of web data chosen for educational quality, plus about 1 billion tokens of synthetic textbooks and exercises. It reported a result that compared well with much larger models on a coding benchmark. The authors' point was that careful selection of data can do much of the work that scale was assumed to do. Again, these are their own results on a specific test.


A different paper points the other way. "The Curse of Recursion" (Shumailov et al., 2023; later in Nature, 2024) found that training on content generated by earlier models can degrade later ones, and suggested that real human-produced data becomes more valuable as generated content fills the web. We wrote about this in our post on synthetic data.


Put together: models were built on a shared pool of public text, the pool is increasingly filled with machine-written text, and what remains scarce is clean, well-labelled, real information with a known source.


Where the edge moves


Any company can call the same frontier models. The compute, and the data those models were trained on, are available to everyone who can pay for access. So the difference between two companies using the same model is not the model.


It is what the model has in front of it when it answers:


  • your current prices, not last year's list found on the web;
  • your real policies and exceptions;
  • your customers' history: what they bought, asked and complained about;
  • your jobs, quotes and results, in your own words;
  • how your team actually does things.

For almost every small and mid-sized business, the practical route is not to train a model on this. It is to keep the records clean and give the model the right ones at the moment it answers, and to check what comes back. That is a smaller, cheaper and more controllable project than model training, and it makes use of what you have that nobody else does.


Our earlier essay on technology tectonic plates made a related point: as each layer of technology gets cheaper and widely available, value moves to what stays scarce. Right now, for AI, a lot of the scarce thing is context.


What this means for your business


1. Treat your records as an asset. Prices, policies, products, job history, customer notes. If they are scattered across inboxes, spreadsheets and heads, a model cannot use them.

2. Put quality before volume. A short, accurate, current price list beats a long, outdated one. Remove duplicates and conflicts, and put a date and an owner on each fact.

3. Keep a single source. If three documents disagree, a model, and a new hire, will pick one at random.

4. Mark what is real and what is generated. Do not let drafts, summaries or test data drift into your records as though they were facts.

5. Do not expect the next bigger model to fix missing facts. A smarter model that has never seen your price list still cannot quote it.

6. Check what comes back. Better context lowers the chance of a wrong answer. It does not remove it, so approvals and checks still matter.


How this shows up at Humonix


This is the reason Know exists. Know is the one record where a client's facts live: what the business sells, charges and promises, and what has happened with each customer. Hue answers from it and shows where an answer came from, and anything public, priced or paid waits in the approvals queue for a person.


We do not train models on client data, and we do not claim that a model's size is what makes Humonix work. We keep records in order and give models the right ones. Each feature on a module page is labelled Live, Early access or Coming, so you can see what runs today. See The Agentic Enterprise for how the pieces fit.


Sources



This post is general information, not technical or legal advice.