By Paul Alongi —— Photography by Ashley Jones
THE CUTTING EDGE OF COMPUTER GRAPHICS RESIDES IN A NORTH CHARLESTON-BASED LAB RUN BY CLEMSON SCIENTISTS, CREATIVES AND VISIONARIES
It’s like a scene straight out of a sci-fi flick.
In a room with towering ceilings on what used to be a navy base sits a sphere 8 feet in diameter. Orange and purple lights dot the geodesic figure, and wires snake around its contours. A chair inside is bathed in a purple glow as if waiting to zap someone to another dimension.
This isn’t a Hollywood set — not yet, anyway. It’s the brainchild of Clemson University innovators.
The VarIS is a specialized light stage that was custom-built by Clemson researchers to digitize real-world objects in remarkable detail. Their work is making film and video games more realistic and opening new ways to study how various materials reflect and transmit light — including the chance to explore what feathered dinosaurs looked like.
VarIS is an acronym for Variable Illumination Sphere. The researchers who built and use the device have affectionately called it the “hamster ball” or simply the “dome.”
The VarIS resides in North Charleston, South Carolina, in the digital production arts section of the Zucker Family Graduate Education Center on the former Charleston Naval Base.
With the VarIS, researchers can individually control the intensity and color of 364 lights to illuminate objects from multiple angles and simulate virtually any lighting condition. The device’s eight cameras photograph objects from multiple angles, and custom photogrammetry software combines them into sharp digital replicas. When researchers reproduce people’s faces, the image is so detailed that individual pores are visible.
Already, the VarIS has been instrumental in Clemson research, including the work of a Fulbright scholar and a recent Ph.D. candidate — and computer graphics experts around the country are starting to take notice.
CLEMSON ROOTS
Eric Patterson ’95, Ph.D. ’02, director of digital production arts at Clemson, started envisioning the VarIS while a professor at the University of North Carolina Wilmington. Even top-of-the-line 3D scanning equipment left flash marks and shadows, and he wanted something better.
Patterson, whose research focuses on digitally modeling faces, found inspiration in a light stage developed by computer graphics researcher Paul Debevec in the early 2000s at the University of Southern California. The stage’s influence made it to the big screen in 2004 when Sony Pictures Imageworks used the technology in Spider-Man 2.
“It was the first film that had a close-up human digital face in it,” Patterson says. “That scene near the end of the movie where Doc Ock falls back into the fire was completely rendered, and one of the things that enabled it was this light-stage technology.”
Patterson went on a sabbatical at Sony Digital Imageworks in 2011 and had the chance to collaborate with some of the computer graphics pioneers who worked on the movie, an experience that would later prove valuable to the development of the VarIS.
When Patterson returned to Clemson in 2016, this time as a faculty member, he wanted to find new ways of merging technical research with applied digital production, the crux of digital production arts.
What would become the VarIS — still only an idea at that point — seemed the perfect fit. But buying a light stage for about $300,000 would have been cost-prohibitive.
“So I thought — kind of crazy — why don’t I just make my own?” Patterson recalls.
SOME ASSEMBLY REQUIRED
After his first academic year at Clemson, Patterson went to Ubisoft’s Red Storm Entertainment for a summer research and development project. Coincidentally, a lead technical artist there was envisioning a device similar to the VarIS.
At Red Storm, Patterson helped build a turntable that spins objects so they can be photographed from multiple angles while surrounded by ambient lighting. He also helped develop the photogrammetry software that turns 2D pictures into 3D models. The project was a great testbed for some of the ideas he and his Red Storm collaborators had, Patterson says.
“There are maybe four universities that have something like this but just for face capture. … Being able to do this with an automated turntable and an automated stage, this is probably the only device that does that.”
Eric PATTERSON
Associate Professor and Director of Digital Production Arts
A few months after he returned from Red Storm, Patterson ordered the parts, and he and his students started building the VarIS in the basement of McAdams Hall on Clemson’s main campus.
Putting together the geodesic sphere took “some assembly required” to another level. It arrived on campus in about 720 separate pieces, with a color scheme and pattern to guide the assemblers but no step-by-step instruction manual. They worked on it an hour or two at a time, and it took about three weeks to complete.
However, the McAdams Hall basement would not be the device’s final home.
The team later separated the VarIS into five pieces to relocate it to a third-floor lab, and, in 2019, it moved again to its current location at the Zucker Center as Patterson and more of the digital production arts program shifted to the Lowcountry.
The team continues to improve upon the VarIS, the only device of its kind in higher education. Its price tag was a fraction of what a new light stage would have cost. Patterson estimates the parts ran about $30,000, and the assembly cost nothing but time.
Clemson in North Charleston
The VarIS resides in North Charleston in the digital production arts program of the Zucker Family Graduate Education Center. Clemson’s North Charleston Innovation Campus is located on a former naval base and is called the Clemson University Restoration Institute. It serves the state with programs in energy research and testing, marine conservation, materials research, and digital production arts.
MAGPIE, BLUEBIRD or VULTURE?
So far, about 15 students have worked with the VarIS. But none have put more into it — and gotten more out of it — than computer science Ph.D. candidates Jessica Baron and Xiang Li.
Baron has used the device to study the material properties of bird feathers and to digitally reproduce them.
“Ultimately, we want to gather a bunch of data on a variety of feathers to be able to come up with a handful of biologically based parameters that can be changed to say, for example, this is a magpie feather, or this is a bluebird feather or this is a vulture feather,” she explains.
Those parameters could then be turned into tools that graphic artists could use to create more realistic digital feathers. Baron also sees promise in using the technique to visualize the color of feathered dinosaurs and to shed light on the origin of birds.
The VarIS has been part of a remarkable Clemson journey for Baron. She studied at Switzerland’s École polytechnique fédérale de Lausanne as part of a Fulbright scholarship and has interned at some of the most high-profile companies in her field, including Pixar Animation Studios, Epic Games and Wētā FX.
Toward the end of the Spring 2023 semester, Baron learned she received a $20,000 Scholar Award from the Philanthropic Educational Organization Sisterhood.
Aside from her research, Baron has found another unique use for the VarIS. While the device is usually the one taking the pictures, Baron turned the camera on the VarIS itself for the spring 2023 Science as Art contest. Her submission captured the inside of the VarIS in a purple glow and helped raise awareness of her team’s work and inspire the next generation of innovators.
Li, who played a significant role in assembling the VarIS, says the device has been central to his dissertation on digitally reproducing faces. His research emphasizes eliminating some of the steps in retopology, a process of restructuring a 3D model’s surface mesh for optimal geometry and flow.
Li has people enter the VarIS through a small door and then sit inside, where he uses eight synchronized cameras to photograph their faces. Then Li uses software he helped write to combine the images into a 3D model.
It would be possible, for example, to create a reproduction of a person’s face, put it on a body and have the digital creation fly over New York or Los Angeles like a superhero, Li says.
He has put together an impressive list of internships since he began working with the VarIS, including positions with Walt Disney Animation Studios, DreamWorks Animation and the University of Southern California’s Institute for Creative Technologies.
“I never before thought I would be working in the movie industry,” Li says while on break from his Disney internship. “But since I’ve worked with Dr. Patterson and the VarIS, I’ve learned a lot about production tools and animation rigging. That got me this position, as well as the one last year with DreamWorks.”
THE FUTURE
Patterson says the work the team has done so far on the VarIS helped secure a $493,000 grant from the National Science Foundation (Grant No. 2007974) to better understand how light interacts with feathers and how their structure affects their color. He sees the study as just the start of the VarIS’ potential for studying various materials.
“By being able to view a material from any angle and light it from any angle, you can study how much that material reflects in a particular given direction,” Patterson says. “One of the obvious examples of application is the automotive industry. They’re interested in substrates and coatings and how they look on vehicles. We can put any sort of material in there and study how it reflects or transmits light.”
The team’s work is starting to get some recognition. ACM SIGGRAPH, a leading conference for computer graphics, accepted two posters last year that involved VarIS research. And in 2022, Patterson, Baron and Li had the opportunity to describe their work with the device at the East Coast Game Conference.
The VarIS has provided new opportunities for undergraduates, too — not just at Clemson. Nathaniel Itty, a junior computer science major at Worcester Polytechnic Institute, says that he was able to build on his prior knowledge of computer-aided design to create 3D models and then 3D-print parts for the VarIS in summer 2022 as part of a National Science Foundation program called Research Experiences for Undergraduates.
Itty also helped create images of feathers. A camera with a microscopic lens on a rail would slowly move toward the sample feather, taking increasingly up-close images, he says. Then, Itty would use a stacking algorithm to combine the images.
“Especially at the start of the summer, I didn’t really know about the field, so once I learned about it, it became really fascinating to me,” Itty says. “And I find it really interesting how the whole process works and the idea behind the device in the first place.
“I just feel very lucky to have been a part of it.”
