A peek inside Large Language Model’s mind: Real-time sentence construction visualized

Data scientist Santiago Ortiz has created a captivating visualization of how Large Language Model constructs sentences step by step. Using the prompt “Intelligence is” and setting a high temperature of 1.6 for diverse responses, Ortiz ran hundreds of iterations to map the AI’s thought process.

To make the intricate data comprehensible, Ortiz utilized Principal Component Analysis (PCA) to reduce 1536-dimensional word embeddings into a 3D space, showing how responses unfold word by word.

The project features two interconnected visualizations:

     •           A 3D cube with branching paths representing the evolution of text sequences.

     •           A tree diagram that highlights word probabilities for each path.

By hovering over the points, users can observe specific sentence trajectories, revealing the decision-making patterns of LLM in real-time.

Experience the project and gain a unique insight into AI’s thought process at Moebio’s Mind.