In order to realize the signal blind separation and feature extraction under the underdetermined condition, an improved non-negative matrix factorization algorithm is proposed. It is combined with time-frequency analysis to realize bearing compound fault diagnosis. First, the short-time Fourier transform (STFT) is used to process the monitoring signal to obtain high-dimensional features that reflect fault information. Secondly, a constraint factor is introduced to optimize the traditional algorithm with Itakura-Saito divergence non-negative matrix factorization(IS-NMF). The improved IS-NMF algorithm is used to perform dimensionality reduction on high-dimensional features to obtain low-dimensional feature components. Then, time domain waveforms of the low-dimensional feature components are reconstructed, and a validity factor is introduced to select the effective reconstructed components. Finally, the envelope spectrum of the effective reconstructed components is analyzed to realize compound fault diagnosis. The analysis results of the simulation signal and the actual bearing compound fault signal show that the method can effectively separate source signals and extract the bearing compound fault features. It is verified that the proposed method can realize the bearing compound fault diagnosis under the underdetermined condition.