The High Cost of Training AI

AI development requires massive amounts of data to train models properly. OpenAI's GPT-3 was

trained on 45 terabytes of text data, which equals roughly 300 billion words. Training a single

large AI model can cost up to 12 million dollars in computing resources alone. Developers often

run thousands of experiments before finding the right model configuration. Here's the crazy

part. AI models can take months to train on supercomputers with thousands of

GPUs running simultaneously. And the carbon footprint from training one large AI model equals five times the lifetime

emissions of an average car. Yet developers are now creating smaller, more efficient models

that run on smartphones.