As a powerful feature extractor and classifier, the convolutional neural networks (CNNs) have been widely used in de-fect detection. However, its opaque decision-making mechanism hinders its interpretability in high-reliability detection scenarios. This paper develops a sparse wavelet CNN model incorporating frequency-domain adaptive weighting and a coefficient gating mechanism. Specifically, the equivalence between time–frequency transform and wavelet-kernel con-volution is built upon the modal characteristics commonly present in nonstationary vibration signals, such as those observed in flip-chip structures. This guides a frequency-domain adaptive weighting mechanism that focuses on domi-nant modal components. Simultaneously, a channel-wise gating mechanism is introduced to the wavelet convolution layers to filter out redundant information, further focusing on the defect-related features and enhancing the decision in-terpretability. Finally, a sparse decision basis from the proposed model is elucidated by analyzing the frequency re-sponse output from the gating, establishing a reliable detection model for detecting structural defects from nonstationary vibration signals, including flip-chip joints as a representative example. Comparative experiments with state-of-the-art methods demonstrate that the proposed model achieves superior accuracy and interpretability on flip-chip vibration data, demonstrating its potential for general structural defect detection from nonstationary vibration signals.