Why Toys Can't Scale

Scale Boundary

• What We Computed by Hand
• Toy Cells vs Captured Cells
• Counting Parameters Exactly
• Counting One Forward Layer
• Training Repeats the Forward Count
• Activation Cells Also Grow
• The Shape Blowup
• A Real Configuration
• Why Exact by Hand Stops
• What This Track Showed