@peterfriese.dev Created by Mamank from Noun Project https: / / peterfriese.dev peterfriese Peter Friese, Staff Developer Relations Engineer, Google AI Development with Firebase 

Friendly Meals • Collect favourite recipes 

Friendly Meals • Collect favourite recipes • View preparation instructions 

121 AI count 

Friendly Meals Generate recipes based on what’s in your fridge Include an inspirational image of the dish Base the recipe on a curated set of recipes Eat pizza 🍕 

Friendly Meals Eat pizza 🍕 Retrieval Augmented Generation Generate text from text Analyse images Generate images from text 

No content

No content

No content

Setting up the model class RecipeGenerationService { private let vertexAI = VertexAI.vertexAI() private lazy var model = vertexAI .generativeModel(modelName: "gemini-2.0-flash") func foo() async throws { let prompt = "Hello World!" let response = try await model.generateContent(prompt) print(response.text) } } 

Generate text from text func generateRecipe(from ingredients: String, cuisine: Cuisine, mealType: MealType, servings: Int) async throws - > String { let prompt = """ Create a \(cuisine.rawValue) \(mealType.rawValue.lowercased()) recipe for \(servings) people using these ingredients: \(ingredients). Generate: 1. A creative title that describes the dish 2. A brief, appetizing description 3. Estimated cooking time in minutes 4. List of ingredients with measurements 5. Step-by-step cooking instructions 6. Include the cuisine type ("\(cuisine.rawValue)") 7. For the imageURL, provide a URL to a high-quality food photo from Pexels.com that most closely matches this exact \(cuisine.rawValue) dish. The image should show a finished, plated dish that matches the recipe's style and ingredients. """ let response = try await model.generateContent(prompt) return response.text ? ? "" } 

Analyse images private lazy var visionModel = vertexAI.generativeModel( modelName: "gemini-2.0-flash", generationConfig: GenerationConfig() ) func analyzeImage(_ image: UIImage) async throws -> String { let prompt = """ Please analyze this image and list all visible food ingredients. \ Format the response as a comma-separated list of ingredients. \ Be specific with measurements where possible, but focus on identifying the \ ingredients accurately. """ let response = try await visionModel.generateContent(prompt, image) return response.text ?? "" } Multimodal input 

Generate images from text private lazy var imagenModel = vertexAI.imagenModel( modelName: "imagen-3.0-generate-002", generationConfig: ImagenGenerationConfig(numberOfImages: 1) ) func generateImage(for recipe: Recipe) async throws -> UIImage? { let prompt = """ A professional food photography shot of \(recipe.title). \ The dish should be \(recipe.description). \ Style: High-end food photography, restaurant-quality plating, soft natural \ lighting, shot from above on a clean background, showing the complete \ plated dish. \ Cuisine style: \(recipe.cuisine.rawValue) """ let response = try await imagenModel.generateImages(prompt: prompt) return response.images.first.flatMap { UIImage(data: $0.data) } } 

No content

No content

No content

Setting up the model const ai = genkit({ plugins: [ vertexAI({ location: "us-central1" }), ], model: ge mi ni20Flash }); Unified model interface 

const detectIngredientsFlow = ai.defineFlo w ({ name: "detectIngredientsFlow", inputSchema: z.object({ im age: z.string().describe("Base64 encoded im age of a fridge") }), outputSchema: ingredientSchema, }, async (input) => { const prompt = `Analyze this im age of a fridge or food items and list all the ingredients you can identify. For each ingredient, provide: 1. The name of the ingredient 2. The amount and unit (if visible, e.g., "2 liters”, "500 grams") Format the response as a structured list of ingredients.`; const { output } = await ai.generate({ prompt: [ { media: { url: input. i m age } Multimodal prompts in Genkit Genkit flow 

const detectIngredientsFlow = ai.defineFlo w ({ name: "detectIngredientsFlow", inputSchema: z.object({ im age: z.string().describe("Base64 encoded im age of a fridge") }), outputSchema: ingredientSchema, }, async (input) => { const prompt = `Analyze this im age of a fridge or food items and list all the ingredients you can identify. For each ingredient, provide: 1. The name of the ingredient 2. The amount and unit (if visible, e.g., "2 liters”, "500 grams") Format the response as a structured list of ingredients.`; const { output } = await ai.generate({ prompt: [ { media: { url: input. i m age } Multimodal prompts in Genkit Input schema 

const detectIngredientsFlow = ai.defineFlo w ({ name: "detectIngredientsFlow", inputSchema: z.object({ im age: z.string().describe("Base64 encoded im age of a fridge") }), outputSchema: ingredientSchema, }, async (input) => { const prompt = `Analyze this im age of a fridge or food items and list all the ingredients you can identify. For each ingredient, provide: 1. The name of the ingredient 2. The amount and unit (if visible, e.g., "2 liters”, "500 grams") Format the response as a structured list of ingredients.`; const { output } = await ai.generate({ prompt: [ { media: { url: input. i m age } Multimodal prompts in Genkit Prompt 

const detectIngredientsFlow = ai.defineFlo w ({ name: "detectIngredientsFlow", inputSchema: z.object({ im age: z.string().describe("Base64 encoded im age of a fridge") }), outputSchema: ingredientSchema, }, async (input) => { const prompt = `Analyze this im age of a fridge or food items and list all the ingredients you can identify. For each ingredient, provide: 1. The name of the ingredient 2. The amount and unit (if visible, e.g., "2 liters”, "500 grams") Format the response as a structured list of ingredients.`; const { output } = await ai.generate({ prompt: [ { media: { url: input. i m age } Multimodal prompts in Genkit Call the model 

2. The amount and unit (if visible, e.g., "2 liters”, "500 grams") Format the response as a structured list of ingredients.`; const { output } = await ai.generate({ prompt: [ { media: { url: input. i m age } }, { text: prompt } ], output: { schema: ingredientSchema } }); if (!output) { throw new Error("Failed to detect ingredients"); } return output; }); Multimodal prompts in Genkit Multimodal prompt 

Vector Embeddings Task: Find all words that are food in the following sentence 

Task: Find all words that are food in the following sentence “I went down to Aberystwyth on foot to buy some welsh cakes and a few berries. When I finished doing my groceries, I had a latte at Coffee #1, where I met a few other speakers.” 

Task: Find all words that are food in the following sentence “I went down to Aberystwyth on foot to buy some welsh cakes and a few berries. When I finished doing my groceries, I had a latte at Coffee #1, where I met a few other speakers.” 

Task: Find all words that are food in the following sentence “I went down to Aberystwyth on foot to buy some welsh cakes and a few berries. When I finished doing my groceries, I had a latte at Coffee #1, where I met a few other speakers.” 

Vector embedding: A numerical representation of a word, sentence, or any other unit of text. 

Vector embedding for food: [-0.018035058, 0.013980114, -0.01309541, 0.024956783, 0.02708295, -0.074924484, 0.03496225, 0.0125780115, . .. ] 

Vector embedding for food: [-0.018035058, 0.013980114, -0.01309541, 0.024956783, 0.02708295, -0.074924484, 0.03496225, 0.0125780115, . .. ] Vector embedding for foot: [-0.016025933, 0.008207399, -0.03572462, 0.020942606, -0.0003162824, -0.041694388, 0.050102886, 0.007380137, . .. ] 

[51.50721, -0.12758] Coordinates for London 

Paris [48.85661, 2.35222] London [51.50721, -0.12758] New York [40.71277, -74.00597] Boston [42.36008, -71.05888] 

Paris [48.85661, 2.35222] London [51.50721, -0.12758] New York [40.71277, -74.00597] Boston [42.36008, -71.05888] 

Source: https://www.learndatasci.com/glossary/cosine-similarity/ Cosine vector similarity 

const computeIngredients E m beddings = async (ingredientsList: string[]): Pro mi se => { const text = ingredientsList.join(", "); const result = await ai.embed({ embedder: text Em bedding005, content: text, }); return result[0].embedding; }; Computing embeddings 

const recipeRetriever = defineFirestoreRetriever(ai, { name: 'recipeRetriever', firestore, collection: 'recipes', contentField: 'recipe', vectorField: 'ingredients E m beddings', embedder: text Em bedding005, distanceMeasure: 'COSINE' }); Retrieving documents 

Genkit Developer UI 

const inventRecipeFlow = ai.defineFlo w ({ name: "inventRecipeFlow", inputSchema: im ageBasedRecipeSchema, outputSchema: recipeSearchSchema, }, async (input) => { const detectedIngredients = await detectIngredientsFlo w ({ im age: input. im age }); const inspirationRecipes = await findRecipesFlo w ({ ingredientsList: detectedIngredients.ingredients. m ap(i => i.name) }); const generatedRecipe = await inventRecipeFlo w ({ inspirationRecipes, cuisine: input.cuisine, mea l T ype: input. m ea l T ype, servings: input.servings }); return generatedRecipe; }); Nested flows in Genkit 

export const inventRecipe = onCallGenkit({ secrets: [apiKey], }, inventRecipeFlo w ); Calling Genkit flows from Swift let request = GenerateRecipeRequest( image: base64String, cuisine: cuisine.rawValue, mealType: mealType.rawValue.lowercased(), servings: servings, dietaryRestrictions: dietaryRestrictions ) let generateRecipe: Callable = functions.httpsCallable("generateRecipe") let response = try await generateRecipe(request) guard let generatedRecipe = response.recipes.first else { // handle error } return generatedRecipe } Expose Genkit flow as callable function 

export const inventRecipe = onCallGenkit({ secrets: [apiKey], }, inventRecipeFlo w ); Calling Genkit flows from Swift let request = GenerateRecipeRequest( image: base64String, cuisine: cuisine.rawValue, mealType: mealType.rawValue.lowercased(), servings: servings, dietaryRestrictions: dietaryRestrictions ) let generateRecipe: Callable = functions.httpsCallable("generateRecipe") let response = try await generateRecipe(request) guard let generatedRecipe = response.recipes.first else { // handle error } return generatedRecipe } Define remote function in Swift 

export const inventRecipe = onCallGenkit({ secrets: [apiKey], }, inventRecipeFlo w ); Calling Genkit flows from Swift let request = GenerateRecipeRequest( image: base64String, cuisine: cuisine.rawValue, mealType: mealType.rawValue.lowercased(), servings: servings, dietaryRestrictions: dietaryRestrictions ) let generateRecipe: Callable = functions.httpsCallable("generateRecipe") let response = try await generateRecipe(request) guard let generatedRecipe = response.recipes.first else { // handle error } return generatedRecipe } Call the function 

No content

No content

Demo!l 

@peterfriese.dev Created by Mamank from Noun Project https: / / peterfriese.dev peterfriese Peter Friese, Staff Developer Relations Engineer, Google AI Development with Firebase 

Thanks! @peterfriese.dev Created by Mamank from Noun Project https: / / peterfriese.dev peterfriese