TinyStories, Traced

Real Activations

The first continuous values are captured hidden-state cells. They are literal floating-point values from the run, shown with approximate labels and precision notes.

highlighted = computed this step

Real activations

The diagram shows a few captured hidden-state cells for the first prompt token. Each value is the literal floating-point value recorded in the trace, displayed with an approximate label.

captured cells are shown with ≈\text{captured cells are shown with }\approx
Captured activationsA few fp32 activation cells from the real trace.Captured activationsA few fp32 activation cells from the real trace.captured embedding cellsCaptured embedding stream cells for prompt token 0.tensor: step00.hidden_state_00 shape [1, 4, 64]hidden_state_00 token0 dim0≈0.268982captured fp32, shown to 6 sig-digitshidden_state_00 token0 dim1≈-0.219411captured fp32, shown to 6 sig-digitshidden_state_00 token0 dim2≈-0.151322captured fp32, shown to 6 sig-digitshidden_state_00 token0 dim3≈0.0356655captured fp32, shown to 6 sig-digitshidden_state_00 token0 dim4≈0.133395captured fp32, shown to 6 sig-digitshidden_state_00 token0 dim5≈0.0172995captured fp32, shown to 6 sig-digitsall visible magnitudes are read from captured fp32 tensor cells

Raw bytes behind the rounded view

The displayed text is rounded for reading, but validation binds it to the committed raw bytes. The hash lock is over tensor bytes and metadata, not over the rounded decimal text.

raw bytes→display check\text{raw bytes}\rightarrow\text{display check}
Captured activationsA few fp32 activation cells from the real trace.Captured activationsA few fp32 activation cells from the real trace.captured embedding cellsCaptured embedding stream cells for prompt token 0.tensor: step00.hidden_state_00 shape [1, 4, 64]hidden_state_00 token0 dim0≈0.268982captured fp32, shown to 6 sig-digitshidden_state_00 token0 dim1≈-0.219411captured fp32, shown to 6 sig-digitshidden_state_00 token0 dim2≈-0.151322captured fp32, shown to 6 sig-digitshidden_state_00 token0 dim3≈0.0356655captured fp32, shown to 6 sig-digitshidden_state_00 token0 dim4≈0.133395captured fp32, shown to 6 sig-digitshidden_state_00 token0 dim5≈0.0172995captured fp32, shown to 6 sig-digitsall visible magnitudes are read from captured fp32 tensor cells

Summary

A captured activation is not an exact rational token. It is a real floating-point cell from the run, marked with an approximate sign and precision label.

REAL CAPTURED≠exact rational\text{REAL CAPTURED}\ne\text{exact rational}
Captured activationsA few fp32 activation cells from the real trace.Captured activationsA few fp32 activation cells from the real trace.captured embedding cellsCaptured embedding stream cells for prompt token 0.tensor: step00.hidden_state_00 shape [1, 4, 64]hidden_state_00 token0 dim0≈0.268982captured fp32, shown to 6 sig-digitshidden_state_00 token0 dim1≈-0.219411captured fp32, shown to 6 sig-digitshidden_state_00 token0 dim2≈-0.151322captured fp32, shown to 6 sig-digitshidden_state_00 token0 dim3≈0.0356655captured fp32, shown to 6 sig-digitshidden_state_00 token0 dim4≈0.133395captured fp32, shown to 6 sig-digitshidden_state_00 token0 dim5≈0.0172995captured fp32, shown to 6 sig-digitsall visible magnitudes are read from captured fp32 tensor cells