Overview
Latent space is the compressed internal map a generative model builds of its training data. Every point in it decodes to an image, sound, sentence, or shape, and moving between points produces gradual change. Designers treat it as a material to be walked through and steered, which raises the question of what data the map was built from.Examples
- Unsupervised (2022 to 2023), a walk through a model of MoMA's collection projected on a 24-foot LED wall in the museum lobby (Anadol)
- Memories of Passersby I (2018), a self-contained machine generating portraits without end from a GAN trained on European painting (Klingemann)
- Holly+ (2021), a public model of one singer's voice that anyone can sing through, governed by a DAO (Herndon)
- Stable Diffusion (2022), which runs diffusion inside a compressed latent rather than on pixels
- GANSpace (2020), which finds directions for age, lighting, and viewpoint inside StyleGAN models
How the Space Is Built
A variational autoencoder trains an encoder to squeeze an input into a short vector and a decoder to rebuild the input from it (Kingma and Welling, 2013). A 1024-by-1024 pixel photograph holds over three million color values; StyleGAN represents a face with 512 numbers (Karras et al., 2019). The model has to decide which regularities matter, and the space it settles on is called latent because no person defined its axes. The compression is lossy by design. Whatever the bottleneck cannot carry, the decoder fills in from patterns learned during training. Generative adversarial networks (Goodfellow et al., 2014) and diffusion models arrive at similar spaces by different routes. Stable Diffusion runs the denoising process inside a compressed latent rather than on pixels, which is what let it run on a consumer graphics card (Rombach et al., 2022). The space is a byproduct of compression rather than something designed directly.
Interpolation
Because the space is continuous, a straight line between two encoded points decodes to a sequence of plausible in-betweens rather than a crossfade. A path through many points, rendered frame by frame, is a latent walk, and it became a common technique in machine learning art after 2015. Where the path crosses regions the training data never covered, the walk breaks into smears and doubled features. These artefacts mark the limits of the training data.
Directions
In word2vec, the vector for king minus man plus woman lands near queen (Mikolov et al., 2013). GANSpace (Härkönen et al., 2020) and InterFaceGAN (Shen et al., 2020) found equivalent lines through image models: age, head rotation, lighting, hair length, each a direction that can be dialed up or down. None of these attributes were labeled during training. The model organised its data along these attributes on its own, and the tools located the directions afterwards.
Working in the Space
The phrase "latent space as design space" describes a change in what the designer does. Instead of drawing a chair, a designer finds the region of the space that produces chairs, samples from it, and adjusts. This differs from generative design, where the space is built from explicit rules; here it is inferred from examples, and the rules stay hidden.
Portraits and Projections
Memories of Passersby I (2018) is a cabinet-sized machine that generates portraits without end from a GAN trained on European painting of the 17th to 19th centuries; no face it shows has existed before or will be shown again (Klingemann). Mosaic Virus (2018) was trained on 10,000 tulips photographed and hand-labeled for the purpose, so that the space the model learned was one built deliberately rather than scraped (Ridler). Unsupervised ran at MoMA from November 2022 to October 2023, walking through a model trained on the museum's collection data and projecting the result on a 24-by-24-foot LED wall (Anadol). One work encloses the model in a cabinet; the other projects its output at the scale of a lobby.
Voice
PROTO (2019) credited Spawn, a neural network trained on the voices of the ensemble that made the album, as a member of the group (Herndon). Holly+ (2021) went further and released a model of a single singer's voice that anyone can sing through, with permissions decided by a DAO of token holders. The model is treated as a shared instrument, with rules for its use decided collectively. It is one of the few cases of a latent space being opened to the public deliberately, with governance built in.
Prompts
DALL-E (2021), Midjourney (2022), and Stable Diffusion (2022) put navigation behind language. A prompt is encoded into the same space as images, and the model denoises a random vector toward the region the prompt names. ControlNet (Zhang, Rao, and Agrawala, 2023) added constraints from edges, depth, and pose, so that a designer can hold geometry fixed while the style moves. Studios use these tools for early variation and material studies. The outputs are images rather than fabrication-ready geometry, which limits how far they travel into production.
Bias and Provenance
Commercial face classifiers have misread darker-skinned women up to 34.7 percent of the time, against 0.8 percent for lighter-skinned men (Buolamwini and Gebru, 2018). An audit of LAION-400M, a dataset behind Stable Diffusion, found explicit imagery and racial slurs among its captions (Birhane, Prabhu, and Kahembwe, 2021). ImageNet's person categories trace back to labels that included insults (Crawford and Paglen, 2019). A latent space reflects the corpus it was trained on, and these audits describe what those corpora contain. The densest regions of the space correspond to the most common examples, so sampling drifts toward a house style. A model trained on its own outputs loses the tails of its distribution within a few generations, an effect known as model collapse (Shumailov et al., 2024). Whether the map could legally be drawn at all is contested in court, in suits brought by Getty Images (2023) and by groups of illustrators. Any output from a latent space draws on the archive it was trained on, which raises questions of attribution and consent that are still being worked out.
Models as Collaborators
A creative adversarial network trained to produce work that fit no known art style had its output rated as human-made more often than pieces shown at Art Basel (Elgammal et al., 2017). The model's tendencies can be treated as a kind of agency, so that the designer's work becomes selection and negotiation with machine intelligence. This places the practice within creative computation and speculative computing: a latent walk shows the continuum between existing things and surfaces variations that could have existed. For a studio concerned with reuse, the useful latent space is one of physical stock rather than pictures. A model trained on scanned salvage or material samples could find families of resemblance across an inventory of dissimilar parts, a form of computational thinking about what is already on hand. The value in that case is in the matching rather than in the image.