Cloud Run

The Agent Development Kit (ADK) simplifies authentication for agents and tools, but is more challenging with the LiteLLM connector when accessing models hosted on Cloud Run. This guide explores how to acquire Google-signed OpenID (ID) tokens and inject them into the LiteLLM communication channel using ADK.

Google Cloud Run provides a robust, built-in access control mechanism based on enforced authentication and IAM policies. When it is enabled, only calls that are made by authenticated accounts which have the specific Cloud Run Invoker role, are accepted, protecting your service from unauthorized invocations.

Enabling authentication for your Cloud Run application is easy ‒ a single mouse click (or parameter in your CI/CD) without writing any code. Calling this application from another is less straightforward. It may be easy when both a caller and called applications are hosted under the same identity in Google Cloud. In the rest of cases, it requires acquiring an identity token.

A problem begins with documentation. Sometimes it isn’t clear whether the described token is an identity token or access token. While the first is good for invoking endpoints of user’s applications on Google Cloud, the second is good only for calling Google APIs.

Production systems are being monitored for reliability and performance tracking to say the least. Monitored metrics ‒ a set of measurements that are related to a specific attribute of a system being monitored, are first captured in the executing code of the system and then are ingested to the monitoring backend. The selection of the backend often dictates the methods(s) of ingestion. If you run your workloads on Google Cloud and use self-managed Prometheus server and metric collection, this post will help you to reduce maintenance overhead and some billing costs by utilizing Google Cloud Managed Service for Prometheus for collecting and storing Prometheus metrics.