Senescence and altered differentiation potential of bone marrow stromal cells (BMSCs) lead to age‐related bone loss. As an important posttranscriptional regulatory pathway, alternative splicing (AS) regulates the diversity of gene expression and has been linked to induction of cellular senescence. However, the role of splicing factors in BMSCs during aging remains poorly defined. Herein, we found that the expression of the splicing factor Y‐box binding protein 1 (YBX1) in BMSCs decreased with aging in mice and humans. YBX1 deficiency resulted in mis‐splicing in genes linked to BMSC osteogenic differentiation and senescence, such as Fn1, Nrp2, Sirt2, Sp7, and Spp1, thus contributing to BMSC senescence and differentiation shift during aging. Deletion of Ybx1 in BMSCs accelerated bone loss in mice, while its overexpression stimulated bone formation. Finally, we identified a small compound, sciadopitysin, which attenuated the degradation of YBX1 and bone loss in old mice. Our study demonstrated that YBX1 governs cell fate of BMSCs via fine control of RNA splicing and provides a potential therapeutic target for age‐related osteoporosis.