Generative AI with PyTorch: From GANs to LLMs, Multimodal AI
What you'll learn
- How to code generative A.I architectures from scratch using Python and Pytorch
- How generative architectures work, in great depth, from GANs to multimodal A.I and large language models (LLMs), understanding every little detail
- In addition to the coding, every section begins with an in-depth review of the key concepts related to these architectures
- Examples: We will code a generative network that produces human faces, and also combine two advanced networks to transform text prompts into amazing images.
- Examples: We will learn to edit the clothes of a person in a picture by combining a segmentation architecture with the Stable Diffusion generative model
- Visual Exploration of Large Language Models (LLMs) : Dive inside models like ChatGPT and understand their attention mechanisms
- Practical fine-tuning of open-source Large Language Models using QLoRA including data preparation, training, evaluation and validation
- Special Bonus Section: Journey to the latent space of a neural network, learn in depth how the networks that power Generative AI learn their mappings
- Special Bonus Section: Experience a guided visualization to exercise the generative model in your head while you learn many things about neural networks
Requirements
- Basic knowledge of python. It's enough with the very basics, as we will code every little thing together, line by line
- Access to an internet connection, as we will use the free online Google Colab service to code together
- Plenty of enthusiasm as we will go deep into every little detail, let's do it! :)
Description
Generative AI is transforming how machines create images, text and multimodal content. This course gives you both the conceptual foundations and the practical coding experience needed to understand how these systems work, moving step by step from foundational neural networks and GANs to multimodal AI, visual LLM exploration and modern LLM fine-tuning techniques.
Build and Understand Generative AI with Python and PyTorch
Understand the foundations of machine learning, deep learning and generative modelling
Compare GANs, autoregressive models (LLMs), variational autoencoders (VAEs), diffusion models and flow-based models
Build basic and advanced generative adversarial networks from scratch with Python and PyTorch
Understand adversarial training, generators, discriminators, loss functions and latent representations
Explore latent spaces, representation learning, interpolation and creative generation
Understand CLIP and the foundations of multimodal AI
Fine-tune open-source LLMs using LoRA and QLoRA
Visually explore large language models (LLMs), including attention mechanisms, token relationships and internal model behaviour
Follow the code line by line while connecting every implementation choice to the underlying theory
The course begins with a broad introduction to Generative AI, covering its major architecture families, machine learning foundations, applications, ethical challenges and future possibilities.
We then move into hands-on coding. Two sections are dedicated to building and understanding generative adversarial networks: first a foundational GAN architecture and then a more advanced system. These projects allow you to understand adversarial training, generators and discriminators, loss functions, latent spaces and the practical challenges involved in training generative neural networks.
From there, the course expands beyond GANs into other areas of the Generative AI ecosystem. You will explore CLIP and multimodal AI, work with architectures that connect images, text and semantic representations, and study practical applications such as image generation, segmentation and the modification of visual elements such as a person’s clothing.
The modern LLM sections take you inside transformer-based language models through visual explorations of attention, token relationships and internal model behaviour. You will then move from visualization and conceptual understanding to practical application by preparing data and fine-tuning open-source language models with LoRA and QLoRA.
The course also includes an optional deeper journey into latent spaces and representation learning, together with distinctive visual and guided experiences—including an origami-based section designed to make AI processes more intuitive and memorable.
At the beginning of each section, I explain the key concepts in depth. After that, we work through the implementations together in Python and PyTorch. You will not simply run finished notebooks: the goal is to understand what every component does, why it is needed and how the complete system fits together.
By the end of the course, you will have a broad understanding of the Generative AI ecosystem and practical experience building GANs, working with CLIP and multimodal AI, visually exploring LLMs and fine-tuning language models using LoRA and QLoRA.
What a time to be alive! We are able to code and understand architectures that bring us home, home to our own human nature, capable of creating and imagining. Together, we will make it happen. Let's do it!
Who this course is for:
- People interested in using A.I and deep learning to generate, imagine and create new things
- People interested in generative adversarial networks and other advanced A.I generative architectures
- People interested in how A.I can combine different modalities (text, images) to create new things (multimodal A.I.)
- People interested in learning to code the type of advanced A.I architectures that are the present and future of the field
Instructor
Javier Ideami is an expert in A.I and deep learning, specialized in advanced visualization, computer vision and generative architectures. He is a multidisciplinary engineer, researcher, creative director, artist and entrepreneur. Javier Ideami’s projects and talks have taken him from Silicon Valley to the jungles of Bali, including Stanford University and UC Berkeley, the United Nations FAO HQ, the financial center of London, the International Cultural Diplomacy Conference in Berlin and many others.
Ideami represents a unique fusion of technical mastery, creativity and entrepreneurial spirit. As an award-winning multidisciplinary engineer and creative, and a pioneering figure in multiple areas, from AI to AR, particle art and beyond, he merges technology and media to push boundaries and create unforgettable experiences. His unique multidisciplinary approach to problem-solving, coupled with his unquenchable thirst for innovation is a reminder of the transformative power of creativity
Pioneer in Generative AI development
e.g: CoFounder of Geniverse, which was one of the first generative AI platforms in the world featuring text to image and text to video capabilities using multiple architectures (before DALL-E was launched). Geniverse pioneered a unique form of flexible canvas interface that allowed users to interact with multiple generative AI models
Creator of unique AI tools
e.g: creator of LL Explorer (used by many educational institutions around the world, a tool that allows you to visualize the loss landscape of the training processes of neural networks)
Pioneer in applying generative AI in filmmaking
e.g: Javier’s movie “Hamelin 77” was pioneer in addressing the topic of prompt engineering + applying multiple generative AI techniques to complement the storytelling (text to image, text to video, NeRF for 3D space reconstruction, etc)
Winner of multiple awards: First prize at ROS Film Festival (Spain), AI Integration award at LTUE Film Festival (USA)
Award winning creations combining AI + creative media
e.g: Totem, a creative interpretation of the work of renown multidisciplinary asian creative Wallace Chan
Winner, in between many other recognitions, of the best experimental movie award at the FRFF in Cannes, France and premiered during the Venice Biennale of Art 2022 at Fondaco Marcello Gallery in Venice, Italy
Innovator in AI training visualization research
e.g: founder of the Loss Landscape project, pioneering project in the area of advanced visualizations of the training and learning processes of neural networks.
This research and project has been showcased and praised by leading AI figures and at prestigious venues.
Featured in popular TV Shows and podcasts related to AI
e.g: Orbita Laika in RTVE in Spain
e.g: Culturas Show in RTVE, Spain
e.g: RTPA Television
e.g: DotCSV Podcast
Experimental music through human-AI collaboration
e.g: creation of the piece “LaTierraTu”, commissioned by the Artificial Intelligence Institute, a collaboration between Javier and different AI processes
Writer of popular AI related articles
e.g: author of the most read AI related article in the world in the leading AI publication in Medium in 2021
Speaker at important AI educational programs
e.g: speaker at keynotes of the Executive AI Masters program of the AI Institute in Spain
Creator of unique artworks using AI data
e.g: art made with unique AI data
Speaker at AI conferences and events for well known AI companies and Institutions
e.g: Weights & Biases talk
e.g: Stanford University
e.g: Synthetic Intelligence Forum
e.g: Roams talk
e.g: Asturias Power
e.g: Strive School
(Javier has given many other presential talks at prestigious events and institutions including Stanford University and UC Berkeley, the United Nations FAO HQ, the financial center of London, the International Cultural Diplomacy Conference in Berlin, the European Big Data Conference and many others)