As more and more Autonomous systems and vehicles are evolving over time, autonomous vehicle safety has become an important factor for increasing public acceptance and safety overall. To ensure safety of autonomous vehicles, all the new features being introduced and learned over time using machine learning and AI algorithms need to go through a certain type of recommended testing and validation procedures. There are also certain metrics being developed to track the performance and safety of autonomous vehicles particularly to quantify them during the different lifecycle phases of ML/AI algorithm development, testing, deployment etc. Another important factor is AI fairness which is to ensure that the autonomous vehicles can detect, predict and classify objects using the inbuilt perception systems and algorithms without any biases. Understanding the different sources of biases, bias risk classification and use of synthetic data would help in minimizing the risk due to bias. In this talk, we will discuss more on different types of safety assurance methods and fairness evaluation methods that can help with a reliable evolution of autonomous vehicles which plays an important role in gaining more public acceptance and the feeling of “confidence and comfort” while riding on an autonomous unmanned vehicle in general.