Gender classification using human face data becomes a trending topic for researchers in the field of image processing and computer vision. The human face is biometric information that can be used to differentiate gender using a computer-aided system. Previous research utilised a local feature algorithm for extracting features on the face. However, the processing speed for one image was more than 2 seconds, making it unsuitable for real-time processing using video data. Processing video data requires a fast feature extraction algorithm because video data collects sequential images (frames). Moreover, the gender classification system’s success is also measured by its accuracy, consequently it is necessary to choose the correct classification method to divide the two classes of men and women. In this research, we propose the FaceNet algorithm for feature extraction and explore several supervised machine learningmethods (KNN, SVM, and Decision tree) appropriate for gender classification on video data. This study used 23,000 training data on each gender. From the experiment, combination of the FaceNet algorithm and KNN method resulted in the best accuracy of 95.75% with a processing speed of 0.059 seconds on each frame.