Objective: This study uses dual-modality signals, including phonocardiogram (PCG) and electrocardiogram (ECG), together with machine learning methods to distinguish cardiac function states in subjects. Methods: We developed a model based on time–frequency representations. The model includes data preprocessing, a time–frequency conversion module, a feature extraction module, and a feature-fusion classifier module. The system uses complete ensemble empirical mode decomposition with adaptive noise to remove noise from the PCG and applies filters to reduce noise in the ECG. The system extracts Mel-frequency cepstral coefficients from the PCG and uses Fourier synchrosqueezed transform for the ECG. This study also improves VGG16 and ResNet18 as feature extractors by inserting a variant attention mechanism into the feature extraction networks. Finally, the system feeds the feature vector into a support vector machine for classification. Results: The dual-modality time–frequency method achieves 95.4% accuracy and 97.4% sensitivity for positive cases on public datasets, demonstrating strong performance in cardiac function classification. Conclusion: This research shows that the approach improves both diagnostic accuracy and sensitivity. The system provides valuable support for the preliminary screening of cardiac dysfunction.