Going Deeper with AI: What is GPT?

today-i-learned artificial-intelligence

In KodeKloud’s AI Agents for Beginners course, they begin by elaborating what is GPT and how ChatGPT utilises GPT models.

GPT

GPT stands for Generative Pre-trained Transformer.

Generative

The output is generated based on the GPT LLM’s learning and pattern recognition. The output isn’t pre-defined.

Pre-trained

It’s the scraping – legal and illegal – of publicly-accessible content on the world wide web. The content is then fed to a GPT LLM for them to learn. Its learning relies solely on pattern recognition, there’s no fact-checking at all.

Transformer

It’s a design on how the AI system processes language. This was first introduced by Google in a 2017 publication called Attention Is All You Need. Prior to this design, AI didn’t know how to relate words of an input with one another.