GenAI

When AI skills started popping up everywhere, my first impulse was to collect them like browser extensions. But treating skills like plug-and-play software misses a fundamental truth: a skill isn’t just a package; it’s a piece of someone’s thinking. In the real world, relying on someone else’s knowledge is never the same as using your own head. You can read someone’s notes or memorize their checklist, but until you understand the principles yourself, you can’t adapt when things go sideways. AI skills operate on that exact same principle.

When building Generative AI applications for the enterprise, you are likely hyper-aware of the risks surrounding sensitive data. The knee-jerk reaction for many engineering teams is to aggressively block all sensitive data ‒ ranging from PII to financial records and internal credentials ‒ from absolutely everywhere in the stack.

But a blind, one-size-fits-all approach introduces its own massive headaches. It compromises internal reasoning, ruins user experiences and breaks downstream autonomous systems. As I promised in the first “How to wear Model Armor” post, we are going to dive into exactly when to keep sensitive data out of your language models, the reasons you must often avoid de-identifying an agent’s output, and how to orchestrate this balance using Google Cloud’s Sensitive Data Protection (SDP) and Model Armor.

The Agent Development Kit (ADK) simplifies authentication for agents and tools, but is more challenging with the LiteLLM connector when accessing models hosted on Cloud Run. This guide explores how to acquire Google-signed OpenID (ID) tokens and inject them into the LiteLLM communication channel using ADK.

Google Cloud Run provides a robust, built-in access control mechanism based on enforced authentication and IAM policies. When it is enabled, only calls that are made by authenticated accounts which have the specific Cloud Run Invoker role, are accepted, protecting your service from unauthorized invocations.

Quick recap of part 1

The first post about Model Armor, explored the fundamentals of Google Cloud’s managed security service for Generative AI applications, which provides a model-agnostic defense layer to sanitize both prompts and model responses.

It covered the two primary patterns for integrating Model Armor into your stack:

• Direct Invocation: Using the Model Armor SDK or API for granular control over pre-call and post-call sanitization.
• Built-in Integration: Configuring services like Vertex AI, GKE, and Gemini Enterprise to automatically enforce security policies through “Floor Settings” and user-defined templates without explicit API calls in your application logic.

And it walked through the practical configuration of these integrations using gcloud CLI and Terraform, establishing a secure baseline for your GenAI pipelines. In this post I shift my focus to examine how direct invocation works in practice. I will review the methods of interpreting sanitize API responses and incorporating the API calls in two agent frameworks: LangChain ‒ probably the most widespread framework today for implementing agentic workflows and the Agent Development Kit (ADK) which I personally prefer for its simplicity.

Model Armor in Google Cloud is a managed security service that provides a programmable defense layer to sanitize prompts and responses for Generative AI applications. At its core, Model Armor is a model-agnostic, API-first security solution designed to intercept and sanitize the I/O of Large Language Models (LLMs). It allows developers to define and enforce safety policies — referred to as Templates — that sit between the user and the model, ensuring that interactions remain within organizational and security guardrails. Unlike Google Cloud Armor that focuses on Layer 7 web traffic and DDoS protection, Model Armor operates on the semantic and content layer of GenAI. You can watch a youtube video to see a practical demonstration of these capabilities in action, including live examples of how the service intercepts and handles malicious requests.

If you did not notice, the Generative AI part of Vertex AI SDK is now deprecated. It means that new versions of this SDK will not update generative AI functions and these functions will be completely removed from SDK versions in 2026. You can find more info about it in the deprecation notice.

In 2024, the Generative AI module was introduced to the Vertex AI SDK. The way it was published for different programming languages introduced quite a confusion. For example, in Python a developer had to install the google-cloud-aiplatform package and then to import vertexai while in Go a name of the installed module was cloud.google.com/go/vertexai and the import statement had to import "cloud.google.com/go/vertexai/genai". In 2025, Google released a new GenAI SDK that was called to replace the collection of VertexAI SDKs for different languages. The new SDK has a more intuitive interface that is similar across different programming languages.

Note: This blog has two authors.

What is context caching?

Vertex AI is a Google Cloud machine learning (ML) platform that, among other things, provides access to a collection of generative AI models. This includes the models known under the common name “Gemini models”. When you interact with these models you provide it with all the information about your inquiry. The Gemini models accept information in multiple formats including text, video and audio. The provided information is often referred to as “context”. The Gemini models are known to accept very long contexts.

Last updated: August 12, 2024.

Everyone today heard or read about GenAI, ChatGPT and other AI things. There are a lot of terminology, abbreviations and other clever words. I found myself troubled to remember all of them. So I decided to write down my definitions for each of these terms.

TL;DR; If you are familiar with AI terminology, you might want to stop reading this post here. For the rest of the readers, welcome to my personalized thesaurus of GenAI terminology.