Multimodal Generative AI: concepts and vocabulary | Complete Generative AI: From Foundations to Production
A self-paced 26-week learning path with 15 modules: AI foundations, Python, machine learning, transformers, LLMs, prompts, APIs, embeddings, RAG, frameworks, agents, customization, multimodal AI, Spring Boot with React, and production LLMOps. Includes 15 practical assignments and three portfolio projects.
Prerequisites: Begin with basic computer skills. Programming is introduced in Weeks 2–3. Use a local Python environment for coding labs; Java and React experience helps in Weeks 21–22. Model downloads, provider accounts and deployment resources are optional learner-managed dependencies. The 26-week schedule is suggested; reading estimates exclude independent project work.
Learning goals
Text-to-image generation
Image understanding
Speech-to-text and text-to-speech
Audio and video generation
Vision-language models
Multimodal safety and evaluation
Understand the foundations
Multimodal systems process or produce more than one type of data, such as text, images and audio. A vision-language model can answer questions about an image, while image generation produces a new visual conditioned on instructions or other inputs. Speech recognition transcribes audio and speech synthesis produces audio from text. Each modality introduces distinct errors: a visual description can miss objects, transcription can alter names and numbers, and generated media may imply events that never happened.
Connect the ideas
Validate content type, file size, duration and decoded dimensions before expensive processing. Do not assume a filename extension proves media type. Keep uploads scoped to the authenticated user and define deletion and retention behavior. Obtain appropriate permission for voices and personal images and avoid identifying a person from appearance. Evaluate descriptive accuracy and useful uncertainty, including how the system behaves with unreadable text, poor lighting, accents and background noise. Provide a text alternative and controls that make the result accessible.
Checkpoint: explain these concepts in your own words, then describe one failure that the design must detect. Use synthetic or permissioned data throughout the course.
01. Introduction to Artificial Intelligence · Week 1
02. Python Programming for Gen AI · Weeks 2–3
03. Mathematics and Machine Learning Foundations · Week 4
04. Deep Learning and Transformers · Weeks 5–6
05. Large Language Models (LLMs) · Weeks 7–8
06. Prompt Engineering · Weeks 9–10
07. Generative AI APIs and Application Development · Week 11
08. Embeddings and Vector Databases · Weeks 12–13
09. Retrieval-Augmented Generation (RAG) · Weeks 14–15
10. LangChain, LlamaIndex and AI Frameworks · Week 16
11. AI Agents and Agentic AI · Weeks 17–18
12. Fine-Tuning and Model Customization · Week 19
13. Multimodal Generative AI · Week 20
14. Gen AI with Java Spring Boot and React · Weeks 21–22
15. Production Gen AI, LLMOps and Capstone · Weeks 23–26