After he was released from prison, he restrained his talents and turned to help Li Yan handle affairs in the government. He used the idea of "customer hierarchical management" to help Li Yan win over the nobles. The relationship between the two warmed up. However, Li Yan was involved in the dispute over the heir apparent due to his royal status. Wei Kaiyin was implicated. Pei Jingming impeached him for "interfering in government affairs" and he was demoted again. This was the second "ups and downs". During this period, he had a break with Li Yan and a head-on dispute broke out with Pei Jingming.