The classification of electrocardiograms (ECG) plays an important role in the clinical diagnosis of heart disease. This paper proposes an effective system development and implementation for ECG classification based on faster regions with a convolutional neural network (Faster R-CNN) algorithm. The original one-dimensional ECG signals contain the preprocessed patient ECG signals and some ECG recordings from the MIT-BIH database in this experiment. Each ECG beat of one-dimensional ECG signals was transformed into a two-dimensional image for experimental training sets and test sets. As a result, we classified the ECG beats into five categories with an average accuracy of 99.21%. In addition, we did a comparative experiment using the one versus rest support vector machine (OVR SVM) algorithm, and the classification accuracy of the proposed Faster R-CNN was shown to be 2.59% higher.