Abstract: Against the background of China's national strategy to actively address population aging, improving the health status of the elderly and reducing their mortality risk have become pressing public health concerns. Using a stratified Cox proportional hazards regression model and data from the 2016–2020 China Longitudinal Aging Social Survey (CLASS), this study analyzes the effect of physical exercise participation on the mortality risk among the elderly in China. The results indicate that physical exercise significantly reduces mortality risk, with pronounced heterogeneity across different population subgroups. Specifically, the protective effect of physical exercise is greater among the older adults (aged above 75) and those with lower educational attainment. Moreover, elder individuals living in rural areas and those with lower income levels experience a more substantial survival benefit from engaging in physical exercise. These findings suggest that physical exercise plays an important role in improving health outcomes among the elderly, and that policies aimed at promoting healthy aging should place greater emphasis on vulnerable populations and on enhancing equitable access to opportunities for physical exercise.