Generative Design Loops: How Refabric Structures Human-in-the-Loop Iteration
· Last updated:
A generative design loop is a control system, not a prompt box: the designer looks at an output, forms an objection, and the system has to turn that objection into something the model can consume on the next pass. Everything that makes these tools usable or useless is in that conversion step. AI design tools built for fashion — Refabric among them — are variations on one shape: generate, judge, encode the judgement, re-condition, regenerate. What separates them is which feedback channels they expose and how much loop state they keep between passes.
Key takeaways
- The hard part of a design loop is not generation, it is encoding a human objection as a conditioning signal the model can act on.
- Feedback channels form a hierarchy: structured fields and masks are attributable, prompt edits and thumbs-up signals mostly are not.
- Every iteration needs a record — inputs, references, masks, seed, model version, parent iteration — or you cannot explain why pass nine looks worse than pass four.
- Identity drift is the characteristic failure: the model changes what nobody objected to, because each pass re-samples the whole image.
- Node-graph products make the loop explicit and reproducible; task-shaped interfaces make it faster and harder to audit.
What is a human-in-the-loop design loop, exactly?
Four stages, repeated:
- Condition. Assemble the inputs: prompt, reference images, garment or category selection, masks, parameters, and the previous output when the pass is an edit rather than a fresh generation.
- Generate. Sample a candidate set. Set size matters more than it looks: one output invites acceptance, a grid invites comparison.
- Judge. A designer marks what is wrong. This is the only step that carries information the system did not already have.
- Encode. Translate that judgement into the next conditioning set. Do this badly and the loop is a slot machine with a nicer interface.
The distinction from one-shot prompting is loop state. A one-shot tool treats each request as independent. A loop keeps a lineage: this candidate descends from that one, with these deltas, and the designer's accept and reject decisions attach to specific nodes in that lineage.
How does designer feedback become a conditioning signal?
Five channels, ordered by how much the system can actually learn from them.
- Structured fields. Garment class, silhouette, sleeve type, colourway, fabric, target size. Enumerable, so a change is unambiguous and can be applied without touching anything else.
- Spatial masks. The designer marks a region and the pass is an inpainting job scoped to it. This is the only channel that natively means "change this and nothing else".
- Reference images. A sketch used as structural control, a photograph for pose, a swatch for appearance. Strong signal, but the model decides which properties of the reference to copy — and it is often not the one you meant.
- Prompt deltas. Editing text is the cheapest thing a designer can do and the least local thing the model can act on: one added adjective re-samples the whole image.
- Preference picks. Choose one of four, star ratings, accept or reject. Trivial to collect, nearly impossible to attribute — a rejection does not say whether the collar, the drape or the colour caused it.
The useful design rule: the more structured the channel, the more attributable the feedback, and only attributable feedback can train anything later. A product whose entire feedback surface is a text box has thrown away its own training data.
Where does the loop lose state?
- Identity drift. Pass six changes the pocket the designer never mentioned. Cause: the pass re-sampled the garment instead of editing it. Fix: default to region-scoped edits and carry the previous output as a structural reference.
- Unattributable rejection. The system knows the designer disliked a candidate and nothing about why. Fix: capture objections against components — collar, sleeve, hem, print — rather than against whole images.
- No reproducibility. Without seed, model version and the full input set, two iterations cannot be diffed, and a good result cannot be recovered a week later.
- Contradictory reviewers. Two people iterate the same lineage in opposite directions and the loop oscillates. Fix: treat lineage as a branching graph with named branches, not a linear history.
- Silent model upgrades. The provider updates a model and the same conditioning set stops producing the same family of outputs. Fix: pin versions per project and record the pin.
Each of those is fixed by the same underlying object: a generation record holding every input, the seed, the model version, the mask, the parent iteration and the designer's verdict. It is a directed graph, not a list, and building it later is much harder than building it first.
Which feedback channel should the interface push designers toward?
| Feedback channel | What the system can learn from it | Cost to the designer |
|---|---|---|
| Structured fields | Exact attribute-level change, reusable as training signal | Someone must define the schema per garment class |
| Spatial masks | Which region was wrong, with a scoped edit | A few seconds of drawing per pass |
| Reference images | Approximate target style or structure | Finding or making the reference |
| Prompt deltas | Little: the change is global and ambiguous | Almost none, which is why it dominates |
| Preference picks | Ranking only, with no attribution | One click |
How do fashion AI design tools structure the loop today?
Tools in this space split into two shapes, and the split matters more than any feature list.
Task-shaped products present the loop as a series of familiar design jobs, so a designer never sees a pipeline. Refabric, built for fashion design work, sits in this group, as does Raspberry AI, whose platform covers sketch-to-render, 3D avatar to photorealistic imagery, virtual try-on, print and graphic generation, multi-view generation and video, aimed at unifying design, product and marketing work from first sketch to final campaign. The loop is there, but the graph behind it is the product's business, not yours — which is fine until you need to explain an output.
Canvas-shaped products make the graph visible. Flora AI runs a browser-based infinite canvas that chains many models, with a fashion module for sketch-to-render, model try-on, fabric swap, garment recolour and flatlay conversion. Weavy AI, now operating as Figma Weave inside Figma, is a node-based AI canvas that chains models from several providers together with professional editing operations — inpainting, outpainting, relighting, upscaling, compositing — into workflows you can rerun.
The trade is consistent. Task-shaped tools get a designer to a usable image faster. Canvas-shaped tools give an engineer the thing a design loop needs to be improvable: an explicit, inspectable, rerunnable graph.
What is the loop actually optimising for?
Not novelty. Volume and hit rate. One catalogue of generative-AI applications in fashion runs to thirteen distinct use cases and notes the technique is particularly useful for fast-fashion retailers who need high volumes of varied designs quickly — which is a throughput argument, and throughput is what a loop with good conditioning delivers.
The commercial framing has been stable for years: in March 2023, McKinsey argued that fashion companies could use generative AI to help create better-selling designs, reduce marketing costs and hyperpersonalise customer communications. Read as an engineering brief, "better-selling" is the interesting word: it implies the loop eventually gets a signal from outside the studio, and connecting sell-through data back to conditioning is still mostly unbuilt.
What is still unsolved?
- Attribution. Nobody has a clean method for deciding which conditioning element caused a rejection, which is why component-level feedback capture is worth the interface cost.
- Evaluation. There is no ground truth for a good design, so loops are judged on designer acceptance rate, which measures the interface as much as the model.
- Learning from small data. A single brand produces few accepted designs per season, far below what preference tuning normally wants, so most per-brand adaptation is retrieval and reference conditioning rather than training.
- Provenance. A generation record is also a legal record, a point our piece on AI-generated fashion images and copyright takes further.
- Geometry. None of this produces construction. That gap, and why it persists, is the subject of our write-up on where sketch-to-3D pipelines break down; the model-family trade-offs behind the images sit in our comparison of GANs and diffusion for fashion image synthesis.
FAQ
What does human-in-the-loop mean in generative fashion design? The model proposes, a designer judges, and the judgement is encoded as conditioning for the next pass. The loop keeps state between passes, so iterations refine one lineage instead of restarting.
Why does each iteration change things I did not ask about? Because the pass re-generated the whole image instead of editing a region. Use masked edits and carry the previous output as a structural reference to hold identity between passes.
Which feedback is most useful to the system? Structured attribute changes and masked regions. Both say exactly what was wrong. Text edits and one-click preferences are cheap to give and hard to attribute.
What should a generation record contain? Every input, references, masks, seed, model version, parent iteration and the designer's verdict. Without those, iterations cannot be compared and a good result cannot be reproduced.
Do these tools replace design skill? No. The loop needs someone who can say precisely what is wrong, because the quality of the conditioning signal is the quality of the objection behind it.