Their venture, Luma, introduced a smartphone app in 2021, quickly gaining momentum and attracting millions of users (currently exceeding two million as of now). Yet, with the surge of generative AI tech, Yu and Jain aim to transform Luma into something more substantial and, hopefully, superior to their original vision.
A few years back, Alex Yu and Amit Jain joined forces to establish a company enabling people to capture 3D objects using their smartphones—no need for extra gear. At that time, Yu, a UC Berkeley AI researcher, collaborated with Jain, an Apple employee dedicated to enhancing Vision Pro’s multimedia experiences.
Today, Luma disclosed its intention to harness a compute cluster featuring around 3,000 Nvidia A100 GPUs to train new AI models that, according to Yu, can “see and understand, show and explain, and eventually interact with the world.”
The initial phase involves creating models capable of generating 3D objects from text descriptions. Earlier this year, Luma introduced one such model on its Discord server named Genie. The subsequent phase focuses on developing “next-gen” generative AI models to address what Yu terms the “uncanny valley” problem in current-gen GenAI.
Meet Genie, a research preview of an all-new kind of generative 3D foundation model #MadeWithGenie— Luma AI (@LumaLabsAI) November 1, 2023
💬 Create 3D things in seconds on Discord
⚡️ Prototype in various styles
🎨 Customize materials
🆓 Free during research preview
Try it now 👇https://t.co/NVgbsvn2XU pic.twitter.com/qm7PkdMEAp
Yu emphasized the importance of multimodality for intelligence, stating, “To go beyond language models, the next unlock will come from vision.” However, he acknowledged that AI needs to become significantly smarter to fulfill its potential.
To bring this vision to life, Luma secured $43 million in a Series B round, with participation from Andreessen Horowitz and other backers. The funding places Luma’s valuation between $200 million and $300 million, boosting its total war chest to over $70 million.
Despite the competition in the 3D model-creating AI models space, with platforms like 3DFY, Scenario, Hypothetic, Kaedim, Auctoria, Mirage, and even established players like Autodesk and Nvidia entering the sector, Luma believes its tools will stand out due to fidelity.
Yu asserted, “Current models are all being trained on two-dimensional images and, when asked to generate scenes, they mangle spaces, bodies, and movements.” Luma aims to introduce the most advanced generative photorealistic technologies in an intuitive app.
While an improved version of Genie is launched today, more capable generative AI models are on the horizon. Luma plans to double its 24-person workforce by the end of next year and assemble a model-running server cluster of “thousands” of GPUs. Whether Luma will make significant headway remains to be seen.