Explanation for Non-Technical People

This explanation is for non-technical people. For others, use these links:

Technical | Data Scientist

Creating a map of words

We learned in Part 1 – All ChatGPT does is complete sentences that ChatGPT and LLM, at its core, is just completing a sentence . In Part 2 – Many tasks are simpler than we thought, we learned that, somewhat surprisingly, many complex tasks can actually be reduced to the simpler task of completing a sentence.

We discussed how ChatGPT completes a sentence by analyzing 250 billion sentences from the internet and digitized books.

The question now is how it is able to use all those sentences to complete the sentence.

The first step is to create a map of words.

Map of addresses

Let’s first start with a concept we are all very familiar with. A map of addresses that we use to navigate every day.

Why do we need to create a map? Why can’t we just use a list of addresses?

For example, a list of addresses would be:

1. 1523 Pine Street, San Francisco, CA 94109
2. 100 Broadway, Oakland, CA 94607
3. 1180 Oak Grove Road, Walnut Creek, CA 94598

A list of addresses doesn’t tell us which addresses are close to each other and which are farther from each other. It doesn’t tell us the path you would have to take to get from one address to another.

So if our goal is to find addresses that are close to each other then a list of addresses is not that useful.

We use a map instead of a list because we need to understand the relationships between addresses and the paths that can be taken to get from one address to another.

The map below shows the three addresses above plotted on a map.

Notice how we can now tell that the first two addresses are really close to each other while the third address is pretty far from those two.

And we can see the paths (i.e., roads) we can take to get from one address to another.

Converting an address to a position on the map

To place an address on the map, we need the location of the address defined in a way that we can position it on a place on the map.

We use longitude and latitude for this. If you remember your (middle) school science classes, you may remember that we can define the position of anything on the earth by using two numbers: longitude and latitude.

For more on latitude and longitude: https://www.techtarget.com/whatis/definition/latitude-and-longitude

So now we can define each address by just two numbers:

1. 1523 Pine Street, San Francisco, CA 94109-1234 (lat=37.79, lon= -122.42)
2. 100 Broadway, Oakland, CA 94607 (lat=37.79, lon=-122.27)
3. 1180 Oak Grove Road, Walnut Creek, CA 94598-7890 (lat=37.93, lon=-122.02)

Notice that now just given the longitude and latitude numbers for ANY two addresses, we can easily tell whether the two addresses are close to each other or not.

Map of words

A map of addresses is much more useful than a list of addresses because the longitude and latitude numbers allow us to figure out the relationships between the addresses.

The same concept can be applied to words. We can lay out words on a map such that words that are related to each other are closer to each other.

Let’s start with a list of words:

1. Apple
2. Tree
3. Elephant
4. Trunk

Now let’s say we scanned lots of sentences and if we found two words in the same sentence we put them closer on the map of words.

We would find that “Apple” and “Tree” are quite frequently in the same sentence so we would put “Apple” and “Tree” close on our map of words.

On the other hand there are probably very few sentences that have “Apple” and “Elephant” in the same sentence. So we would put “Apple” and “Elephant” very far from each other in our map of words.

“Trunk” is an interesting one. There are probably lots of sentences that include “tree” and “trunk” but there are also lots of sentences that include “elephant” and “trunk”. Hence the word “trunk” is close to both “tree” and “elephant”.

In the next part in this series we will explain how distance is calculated between words on this map of words.

Once we calculate the distances we can plot these on the map of words.

How a map of words enables LLM to complete the sentence

Now that we’ve laid out all our words on a map of words, let’s start the task of completing a sentence.

Let’s say the sentence we’ve been given is “I want to grow an apple ___”.

We look in the map of words for what words are closest to “apple” and we find the word “tree”. So now we can use that word to fill in the blank.

“I want to grow an apple tree”.

Concept of distance

As we learned above, the goal of a map is to put things together that are closer to each other and put things apart that are far from each other.

Closer and Farther imply a difference in distance.

For a map of addresses, the distance is the physical distance between two points on earth.

What is the distance between two words? Read the next part to learn.