According to a report by Qubit on January 2, Huawei's Noah's Ark Lab and others jointly launched a new large language model architecture: Pangu-π. By enhancing the nonlinearity, the architecture is improved over the traditional Transformer, which can significantly drop the problem of feature collapse and make the model output more expressive. It is reported that in the case of training with the same data, Pangu-π (7B) surpasses LLaMA 2 in multi-tasking and achieves 10% inference speedup. SOTA up to 1B scale. At the same time, based on this architecture, Huawei has also refined a large financial and legal model called "Yunshan". The work is led by AI Dacheng Tao.