ishamf.dev
An LLM attention visualization showing which earlier tokens a generated token referred to
SITE CAPTUREBased on information published on the service's website. Selection and review policy
WHAT IT DOES
What does it do?
Tapping or hovering over a generated token shows earlier tokens that influenced it through opacity. This is a simplified representation that uses attention weights and value vector magnitudes to aggregate information across all heads and layers into a single value. Pre-generated examples can be viewed immediately, while browser-based generation uses an instrumented ONNX model.
KEY FEATURES
Key features
- Visualizes relevant earlier tokens for each generated token
- Tap and hover interactions
- Aggregates values across all heads and layers and represents them with opacity
- Immediately loads pre-generated prompts
- Browser-based text generation using Transformers.js
- Exposes internal values through an instrumented ONNX model