The pedestrian detection algorithm (PDA) is one of the most widely used techniques in modern automated vehicles, surveillance systems, human-machine interfaces, intelligent cameras, robots, etc. Despite considerable work in this field, PDA is still receptive to several scopes of advancements considering some adverse weather conditions like fog, rain, low visibility, etc. Along with this, there are certain intricate scenarios where the accuracy of a given PDA becomes contentious. As we are progressing toward autonomous vehicles, it becomes vital for such vehicles to ensure the safety of both passengers and pedestrians walking around the road. To do so, they require a much more reliable and effective pedestrian detection system capable of working under adverse conditions. This paper considers all such issues to develop certain machine learning (ML) and deep neural network (DNN) methods to solve such issues. YOLOv4 is a deep learning-based object identification method that is currently functioning well yet is not robust. The core premise of YOLOv4 is initially explored and evaluated in this paper to discover its importance in our task. This research devises a coupled system capable of detecting pedestrians under various adverse and intricate scenarios. To do so, we use the YOLOv4 object detection technique coupled with some image denoising, low light enhancement and image dehazing features. We are using the wavelet and YCbCr methods for image denoising and low-light enhancement. To dehaze the video frames, we use airtight estimation and tuning the transmission by deriving the boundary constraints. The paper tries to cover most of the aspects that an autonomous vehicle may face while on the road. Overall, we deliver a reliable model that fosters more accuracy even in complex scenarios and unfavourable weather conditions.