Developed and integrated the RHELAI Large Language Model provider into the Prow CI workflow for the ansible/ansible-chatbot-service repository, enabling automated evaluation of RHELAI deployments within continuous integration pipelines. Leveraged Kubernetes and shell scripting to introduce an OLSConfig CustomResourceDefinition, allowing for flexible configuration of model and deployment parameters. Added a comprehensive smoke test suite, written in bash and yaml, to validate provider functionality end-to-end within CI environments. This work enhanced the reliability of CI quality gates for LLM-driven features and accelerated feedback cycles, supporting scalable, repeatable deployment and testing of language model integrations in production-like settings.