Generative AI is the class of models that learn the probability distribution of their training data and then sample from it to produce new data objects: text, images, audio, code, molecules. That is the key feature. A classifier looks at an input and returns a label. A generative model returns an artefact, and that artefact typically carries thousands to millions of bits of information, where a binary classifier’s answer carries one. Almost everything that makes generative AI useful, and expensive, and hard to evaluate, follows from that one structural difference.
The US Government Accountability Office (2023) describes generative AI as technology that can create text, images, audio or video when a user prompts it. Executive Order 14110, as quoted in NIST AI 600-1 (2024), calls it the class of models that “emulate the structure and characteristics of input data in order to generate derived synthetic content”. “It creates content” is correct, but for an engineering team the useful version is the distributional one, because that is what predicts how the system will behave in production.
The key feature of generative AI for a production team: one decision versus many
A classical classifier makes a single decision per input. The Explainable Generative AI survey (Schneider et al., 2024) puts it this way: before generative AI, a classifier’s label was one decision, while a generative model “makes a multitude of decisions – one for each aspect of the artifact”. The survey “On the Challenges and Opportunities in Generative AI” (2024) makes the same point from the other side: deep generative models output into very high-dimensional spaces, unlike many traditional machine-learning models, and that brings several technical complications with it.
Worked example: counting the bits in the output
| Model | Output | Information in the output |
|---|---|---|
| Binary classifier | “defect” or “no defect” | 1 bit |
| 1,000-class classifier | one of 1,000 labels | about 10 bits (log₂ 1000 ≈ 9.97) |
| Image generator | 256 × 256 × 3 values, 8 bits each | 256 × 256 × 3 × 8 = 1,572,864 bits of raw pixel data |
The arithmetic follows directly from the definitions. Raw pixel bits overstate the information an image holds, since images compress, but the gap of five to six orders of magnitude remains. That gap is where the production consequences come from.
What the key feature costs you in production
Three costs come with sampling high-dimensional output:
- Inference is slow by construction. Diffusion models denoise iteratively, often needing hundreds to thousands of network evaluations per sample, and LLMs generate tokens one at a time from left to right. A classifier makes one forward pass and stops.
- Plausible-but-wrong output is built in. NIST AI 600-1 describes confabulation as a natural result of how generative models work: they produce outputs that approximate the statistical distribution of their training data. OpenAI’s own launch notes for ChatGPT said it “sometimes writes plausible-sounding but incorrect or nonsensical answers”.
- Evaluation has no single right answer. A classifier is scored against a label. A generated paragraph or image has many acceptable forms and many subtly wrong ones, so accuracy-style metrics alone are not enough and automated scores need human review on your own data.
None of these is a reason to avoid generative models. They are reasons to choose one only when the output really needs to be an artefact, and to use a discriminative model when a label or a score would do.
How is generative AI different from predictive or discriminative AI?
The formal distinction is about which probability the model learns. Google’s machine-learning course states that generative models capture the joint probability p(X, Y), or just p(X) when there are no labels. Discriminative models learn the conditional p(Y | X): given this input, how likely is each label. Google’s summary is that generative models create new data instances, while discriminative models classify existing data.
This is why a next-word predictor counts as a generative model. A model that predicts the next word assigns a probability to a whole sequence of words, which makes it a model of the data itself. Chaining those predictions and sampling at each step is how an LLM produces text.
We do not treat “predictive AI” as a separate mathematical category. It is the everyday name for discriminative and regression models used to forecast or score: churn risk, remaining useful life, fraud probability. The dividing line that matters is whether the output is a label or number about an input, or a new sample of the data.
Which model families power generative AI: LLMs, diffusion models, GANs or VAEs?
The GenXAI survey names transformers as the de facto standard for LLMs, and diffusion models, variational autoencoders (VAEs) and generative adversarial networks (GANs) as the other main families, especially for images. A 2023 review of machine learning for synthetic data covers language models, VAEs, GANs and diffusion models, alongside reinforcement learning, among the models frequently used for synthetic data generation.
| Family | How it generates | Sampling cost | Typical trade-off | Origin |
|---|---|---|---|---|
| Transformer / LLM | Predicts the next token from self-supervised pre-training, then samples one token at a time | Sequential, one pass per token | Strong on text and code; latency grows with output length | Vaswani et al. (2017), attention-only architecture |
| Diffusion model | Adds Gaussian noise to data in a forward process, learns to reverse it, and builds samples from noise | Hundreds to thousands of network evaluations per sample | High sample quality, slow inference | Ho, Jain and Abbeel (2020), DDPM |
| GAN | A generator turns a random vector into an output while a discriminator tries to tell generated samples from real ones | A single network evaluation | Fast, but trained through an adversarial game | Goodfellow et al. (2014) |
| VAE | An encoder compresses input into a latent space constrained towards a known prior, so sampling that latent and decoding it yields new data | A single network evaluation | Fast, but samples tend to be blurry | Kingma and Welling (2013) |
The GAN paper’s own framing is worth keeping in mind because it shows both sides of the earlier distinction inside one system: a generative model G that captures the data distribution, trained against a discriminative model D that estimates whether a sample came from the training data.
Generative AI is wider than LLMs
Generative AI covers models that generate images, audio, video and molecules, not only large language models. The GAO (2023) notes that generative AI can learn many data types beyond written language, including programming code, molecular structures and images. NIST AI 600-1 points to chemical and biological design tools, specialised systems trained on scientific data, that can go beyond what text-based LLMs provide. Recent large-scale generative models synthesise high-resolution images and text as well as videos and molecules.
TechnoLynx works on generative and agentic AI R&D. Using “generative AI” to mean LLMs is fine in an LLM-specific conversation, as long as nobody forgets that a text-to-speech model or a 3D asset generator is generative AI too.
Where do transformers sit in the taxonomy, and why do they keep dominating?
A transformer is a network architecture, not a family of generative model in the probabilistic sense. Vaswani et al. (2017) introduced it as a network “based solely on attention mechanisms”, dropping recurrence and convolution, and reported it as more parallelisable and faster to train than the recurrent and convolutional models it replaced. Transformer-based LLMs are pre-trained on tasks such as next-word prediction (GPT-2) or masked-word and sentence-order prediction (BERT).
Attention operates on a sequence of tokens, and almost any data can be cut into tokens: words, image patches, audio frames. Training parallelises well on GPUs. Our judgement is that transformers win on engineering economics (one well-optimised building block, one set of kernels, one tooling stack) as much as on modelling elegance. They are not automatically right for a small tabular or tight-latency edge problem, where a gradient-boosted tree or a compact CNN may still be the better build.
How does a working taxonomy of ML, deep learning, LLMs and GenAI map to real engineering decisions?
The terms nest, and each level of nesting changes what you build and run. Machine learning is the set of methods that learn from data; deep learning is the subset using deep neural networks; deep generative models are the deep-learning subset that learn and sample a data distribution; LLMs are one kind of deep generative model, for text. The “Challenges and Opportunities” survey (2024) uses “generative AI” to mean a collection of large-scale deep generative models.
| If the job is… | The level you need | What it implies for the build |
|---|---|---|
| Score, rank, detect or forecast | Classical ML or discriminative deep learning | One forward pass, measurable accuracy |
| Produce text, code or dialogue | LLM (transformer, autoregressive) | Token-by-token latency, confabulation controls, human-in-the-loop evaluation |
| Produce images, audio or video | Diffusion | Many denoising steps per sample, slow inference |
| Execute multi-step tasks with tools | An agentic system using one or more models | A control loop that uses one or more models as tools |
Symbolic AI sits outside this nesting, which is the subject of our working taxonomy of symbolic, generative and traditional ML.
Is ChatGPT generative AI, and what model sits underneath it?
ChatGPT is generative AI. The “Challenges and Opportunities” survey describes it as a conversational AI agent built on large language models, and the GenXAI survey classes it as a text-to-text generative system. At launch in November 2022, OpenAI said ChatGPT was fine-tuned from a model in the GPT-3.5 series, first with supervised fine-tuning on human-written conversations and then with reinforcement learning from human feedback (RLHF).
Underneath, ChatGPT is still an LLM, and NIST AI 600-1 notes that LLMs predict the next token or word in a sequence. RLHF shapes which samples the model prefers; it does not change the fact that the model is sampling from a learned distribution. ChatGPT has since added multimodal generation, including images, and OpenAI’s own launch page notes that the product has evolved substantially since 2022. Cao et al. (2023) class ChatGPT as generative AI under the label AI-generated content.
ChatGPT is also more than a bare model. The GenXAI survey notes that a GenAI-powered system can wrap orchestration around the model, for example running an internet search when the prompt asks for one and feeding the results back into the model. That wrapping is the bridge to the next distinction.
Generative AI vs agentic AI: a model versus a control loop
A generative model produces an output from a prompt. An agentic system orchestrates actions over time, using one or more models as tools inside a control loop. Sapkota et al. (2025) place generative AI as the precursor and foundation, with AI agents adding tool integration and reasoning, and agentic AI adding multi-agent collaboration, dynamic task decomposition, persistent memory and coordinated autonomy.
The scoping consequence is large. A generative feature fails by producing a bad output, which a person can read and discard; an agentic system fails by taking a bad action, which may already have changed a database or sent an email before anyone looks. Treating agentic AI as a flavour of generative AI leads to scoping mistakes on incoming projects. Our comparison of agentic AI and generative AI with examples covers the architecture differences in depth.
How does applied AI differ from general AI in terms of what to build today?
Applied (narrow) AI solves a defined task with a defined input and a measurable outcome; general AI refers to a hypothetical system that matches human ability across arbitrary tasks. An engineering team should build applied AI. General AI is not a product specification anyone can scope, test or accept.
One large generative model can produce text across a wide range of use cases. That breadth is real, but a deployed system still has to be applied: one task, one data contract, one evaluation set, one owner for the failure cases. The breadth of a foundation model is best treated as a way to prototype many narrow applications quickly, not as a reason to skip defining any of them.
How to judge recent LLM inference techniques against the structural costs of generation
We would not rank recent developments by how much attention they got. The useful test is whether a technique attacks one of the structural costs above: the sequential decoding that makes LLM inference slow, the memory needed to hold weights and context, or confabulation.
Treat with caution anything sold as a new capability without a measurement on a workload like yours: a benchmark headline on a different model size, sequence length or batch size says little about your serving cost. Reading inference benchmarks is its own discipline.
Why did symbolic AI fail the way it did, and what does neuro-symbolic AI bring back?
Symbolic AI represented knowledge as hand-written rules and logic. Our reading is that it broke down because the world’s exceptions outran the rule-writers: every new situation needed new rules from an expert. Learned models reversed that trade, absorbing the messiness from data at the cost of explicit, checkable reasoning.
Neuro-symbolic AI tries to recover what was lost. A neural model handles perception and fuzzy matching, and a symbolic layer supplies constraints, logic or verifiable steps. An LLM that calls a calculator, a database query or a type checker follows the same pattern: a symbolic tool attached to a statistical model so that the parts needing exact answers get exact answers. For generative AI’s own key feature, that is the honest conclusion: sampling from a learned distribution is extraordinarily good at producing plausible artefacts, and on its own it is not a mechanism for guaranteeing correct ones.
