Nowadays, many advanced automotive features have been incorporated in Advanced Driver Assistance Systems (ADAS). Lane Marking Detection (LMD) is one of the most significant and preliminary features of ADAS. Previous studies have the limitation on different environmental conditions, which lead to a less accurate and efficient system of LMD. Therefore, this research article proposed a semantic segmentation approach based on the U-net to detect the LMD under distinguishing environmental effects like a variant of lights, obstacle, shadow, and curve lanes. The proposed model is emphasized on the simple encode-decode U-Net framework incorporated with VGG16 architecture that has been trained by using the inequity and cross-entropy losses to obtain more accurate segmentation results of lane markings. DBSCAN interfaced with the predicted instance and binary lane pixels. The system was trained and tested on a publicly available Tusimple dataset consisting of 3.6K and 2.7k image frames of different environmental conditions for training and testing. The algorithm achieved 96.4% accuracy, 95.25% F1 score, 96.01% precision, and 92.89% recall, which outstripped some state-of-the-art research. This research outcome leads to a significant impact on the LMD research arena.