Cardiovascular diseases are still the primary threats to people’s health around the world. Automatic heart sound classification technology, as a fast and efficient means for diagnosis and treatment, is of great clinical significance. With the rapid development of artificial intelligence technology, deep learning algorithms are widely used in automatic heart sound classification. This paper reviewed the key technologies related to the automatic classification of heart sounds in recent years, including heart sound denoising, segmentation, feature extraction, and classification recognition. The classification and recognition technologies related to deep learning are presented in detail, with a focus on the application and development of convolutional neural network and recurrent neural network, as well as various combination models for heart sound classification in the past five years.