Google has unveiled the open-source LMEval framework, a comprehensive evaluation tool tailored for large language and multimodal models. This framework aims to streamline the process of cross-platform model comparison, facilitating a more standardized approach. With support for assessments spanning text, image, code, and other domains, LMEval is equipped to identify model evasion strategies. Furthermore, it integrates seamlessly with the LiteLLM framework, enabling cross-platform testing with enhanced efficiency and accuracy.