AI Agent Autonomously Creates 3D/4D Scenes from Images, Linas Beliūnas Reports

In a recent LinkedIn post, Linas Beliūnas highlights a significant advancement in artificial intelligence: a multimodal AI agent capable of autonomously coding and rendering 3D and 4D Blender scenes directly from images, a process that requires no human intervention or prior training data.

Beliūnas emphasizes the agent’s sophisticated operational loop, stating:

It’s called VIGA, and it’s using a graphics engine as a reasoning loop. Instead of one-shot generation, VIGA runs a closed loop: code → render → verify → revise.

This approach, as explained by Beliūnas, moves beyond simple mesh prediction. The agent actively writes Blender Python code, executes it, renders the scene, compares the output to the target image, and iteratively refines the code until the visual match is achieved. He likens this process to providing a large language model with access to a physics sandbox, tasking it with continuous improvement until a specific visual target is met.

VIGA’s Performance and Capabilities

Linas Beliūnas detailed the impressive performance metrics of this new agent, VIGA, noting substantial improvements over existing baselines. According to Beliūnas, VIGA demonstrated:

• A +159% improvement against baselines on BlenderBench, measured by both photometric and Vision-Language Model (VLM) metrics.
• A +35% improvement on BlenderGym for graphics editing tasks.
• A surprising +117% improvement on SlideBench, indicating its effectiveness even in 2D document tasks.

Crucially, Beliūnas points out that VIGA achieves these results without any fine-tuning, built upon foundation VLMs like GPT-4o and integrated with Blender APIs. He further elaborated on the nature of the output:

And the output isn’t a black-box latent. It’s executable Blender code. Readable. Editable. Reusable for robotics sims, digital twins, game prototyping.

The agent’s capabilities extend to dynamic 4D scenes, including simulated physics and scripted interactions, not just static reconstructions.

The Future of Design and World-Building

Beliūnas posits that this development signals a fundamental shift in the creative process for 3D environments. He argues that the focus is moving away from traditional polygon sculpting towards directing AI agents. As Linas Beliūnas concludes:

Most AI systems stop at perception. This one crosses into world-building. If images become programmable environments, 3D creation stops being about sculpting polygons, and starts being about directing AI agents. The future of design will never be the same again.

This advancement, as reported by Beliūnas, suggests that AI is not just perceiving the world but actively constructing and manipulating it, with executable code as the output, opening new avenues for applications in fields like robotics, digital twins, and game development.