Loopback Transformer: Towards a Unified View of Residual Looped Transformers

Media reports Hugging Face Papers Source language: Chinese

论文摘要(作者报告,未经本站复测):Looped Transformers reduce parameter count by repeatedly applying shared blocks, but parameter sharing alone does not determine how the results of repeated computation are reused.

Full text

A looped Transformer that separately controls input strength, total update weight, and per-update allocation achieves lower validation loss and higher downstream accuracy than standard and looped residual baselines while using fewer parameters.

Sources

← Back to Daily AI Report