Publications

Reasoning Traces in MLLMs Offer Rich Guidance for Design Transformation

Workshop on Multimodal Algorithmic Reasoning for Scenes and Structures (MARS2) at IEEE/CVF European Conference on Computer Vision (ECCV)

Publication date: September 12, 2026

Aveen Dayan, Joseph K J, Sayan Nag, Balaji Vasan Srinivasan

We study prompt-driven transformation of layered designs, where the goal is to edit only those elements referred to by the user instruction, while preserving all other layers and layout structure. This setting differs fundamentally from raster-space design editing, where each prompt often triggers a full re-synthesis of the canvas which introduces unintended collateral changes. Towards tackling this problem, we propose a design transformation framework built on a reasoning-capable multimodal backbone model, adapted to produce interleaved layer outputs through a planner-generator decomposition. The planner uses the model's reasoning traces to identify target layers and requested operations, while the generator emits sequential layer updates with a \textit{shared cross-step context} that preserves coherence with the existing design meta-data. We present qualitative proof-of-concept end-to-end examples of localized edits, interpretable plans, and fewer unintended changes than raster-edit baselines. This work-in-progress report

Learn More