By Angela Nixon ’99, M ’01
Illustrations by Chris Koelle
Whether it’s code, color or creative writing, machine learning opens the door to better understanding
When Dillon Ranwala enrolled at Clemson as a computer science major, the last thing he expected to analyze was “To be, or not to be” and “wherefore art thou, Romeo?”
Ranwala joined a Creative Inquiry team called “Watt AI,” looking for a project that would give him hands-on experience in IBM’s artificial intelligence (AI) and machine-learning program, Watson. He was surprised to find Shakespeare research on the list of projects he could join.
Lucian Ghita, senior lecturer of English, started the project in 2019 when he joined the Watt Faculty Fellows, a program designed to promote the Watt Family Innovation Center’s unique resources to faculty in a variety of disciplines. Ghita had read a paper about using AI to analyze the language in Macbeth, and he wanted to explore how AI could be used to analyze other Shakespeare plays. Since natural language understanding is one of Watson’s primary purposes, it seemed the perfect tool to approach some very old and familiar works in a new way.
Ghita and his team are using Watson to analyze emotion in Shakespeare’s plays and how emotional expression differs between genres and genders. He hopes his findings will bring a greater understanding of Shakespeare’s works and could help actors and directors express characters’ emotions even more effectively.
The first project the group tackled was Hamlet, which actually exists in three versions of different lengths. The earliest version is much shorter than what most people consider the traditional version of the play, says Ghita.
“This has allowed me to see how I can work in arts and literature while also being a STEM major.”
Alison Menezes
“We wanted to see if there were any differences in the emotional scores between the different versions, and we found some interesting things,” he continues. “In the first version, everything happens much faster, and Hamlet is more decisive in his actions. In later versions, the action is slowed down, and Hamlet agonizes more over what to do and whether or not to seek his revenge for much longer.
“In the shorter version, Hamlet is angrier and less fearful, but in later versions, the fear scores were higher, and the anger scores were lower.”
Ghita and his team are now looking at all 37 of Shakespeare’s plays and how emotion is expressed differently across their various genres. The bulk of the work is feeding each line of dialogue into the AI and teaching it how to detect emotions by different words.
Alison Menezes, a freshman computer science major, is one of the students helping teach Watson about Shakespeare. She says the team has about 60,000 lines of text to enter for the project, about half of which she is responsible for.
“We label and categorize each line, and I now have a way better understanding of AI than I did before,” she says. “I can see that it is different than the ‘magical technology’ I thought it was. There’s a lot of coding, a lot of logic to it.”
As a musician majoring in computer science, Menezes was drawn to the project because it blended her interests in computing and technology with the arts.
“This has allowed me to see how I can work in arts and literature while also being a STEM major,” she says. “I feel like there is a lot of potential for applying AI to the arts and music that I would love to explore.”
Ranwala agrees: “I have been able to learn how to work on a team of people who don’t necessarily have the same background as I do, such as an English professor or a statistician. But we have all been able to work together and bring in our own subject matter expertise and communicate with one another so that we all understand what we’re doing. In the real workplace, that’s what you need to be able to do.”
“The Creative Inquiry program really separates Clemson from other research universities.”
Jeremy Spooner
The stuff of science fiction
In 1956, the term “artificial intelligence” was first coined at a Dartmouth College conference. The Dartmouth Summer Research Project on Artificial Intelligence focused on a field that had previously been the stuff of science fiction — developing “thinking machines” that could learn, solve problems and make decisions in the same way the human mind does.
In 1997, IBM’s Deep Blue, a supercomputer built to learn and play chess, defeated a human opponent, world chess champion Garry Kasparov. A milestone in AI research and development, the victory was a very public demonstration that computers can learn from input and make decisions and predictions based on that input.
IBM went on to develop Watson, a question-answering AI computer system that made history in 2011 when it won Jeopardy!, defeating champions Brad Rutter and Ken Jennings. Watson has since been further developed to include processing and analyzing natural language, generating hypotheses, and retrieving information. Today, Watson is used in industries ranging from agriculture to health care to help analyze large quantities of data and provide subsequent insights.
In 2018, Watson came to Clemson through a partnership with IBM and the Watt Family Innovation Center. Known as Watt AI, the program assists faculty across campus with using AI and machine learning tools in their research. Students get hands-on experience using AI through the Watt AI Creative Inquiry project.
Creative Inquiry is Clemson’s nationally recognized cross-disciplinary undergraduate research program. More than 2,800 students from all majors participate in Creative Inquiry each semester, working directly with faculty mentors and a team of peers to tackle real-world problems.
The human brains behind the AI in the Watt Center are research associates Hudson Smith and Carl Ehrett M ’17. Smith and Ehrett lead the Watt AI Creative Inquiry class and mentor student teams working with faculty on various AI-related projects. They are working with more than 30 projects from a range of disciplines, including health care, marketing, engineering, political science and literature.
“We facilitate AI and machine learning research for any faculty who are interested in applying it to their research but might not have the background or the experience to do it alone,” says Ehrett.
Protecting the Clemson colors
Whenever Erica Walker M ’05, Ph.D.’16 and her husband would watch Clemson Football games, she noticed that Clemson Orange often looked red, depending on what TV she was watching. As an assistant professor in graphic communications with an education in film production, she is well aware of how important colors are to an organization’s brand. She wondered if there was a way to correct it so that Clemson Orange would look like Clemson Orange consistently, regardless of the TV.
Walker was familiar with the work Smith and Ehrett were doing, so she asked if AI could be used for color correction — specifically for color correction in fast-paced, live broadcast situations like sports. They created a Creative Inquiry project, and over the next few years, ColorNet was born.
“What ColorNet is made to do is identify and correct specific brand colors live on a video feed,” Walker says. For now, ColorNet is trained to correct Clemson’s official colors — Clemson Orange and Regalia — but the software has now been patented and could be trained to correct any brand’s colors as they appear in video and photography.
To correct colors as an event on a live broadcast, the AI must identify the incorrect colors and make the color correction in fractions of a second. And it must do that without altering any other colors it is not programmed to correct.
The ColorNet team is also developing an app that will implement color correction in photos on the fly. For example, this technology could be used by social media managers who are posting photos in real time during events.
Michelle Mayer and Jeremy Spooner are two computer engineering seniors who have worked on the ColorNet project for several years. And both of them already have jobs lined up after they graduate in May. Mayer is going to be a software engineer at Microsoft in Charlotte, and Spooner has been hired as a software engineer at AT&T’s headquarters in Dallas. Both credit their experience on the project with helping them land their jobs.
“The Creative Inquiry program really separates Clemson from other research universities,” Spooner says. “Programs like this are how our students are a step above students from other schools in applying for jobs. We have a huge advantage coming out of college after going through this program.”
Man versus machine in marketing
The Watt Center isn’t the only place where AI research is happening at Clemson. Mike Giebelhausen, associate professor of marketing, has a Creative Inquiry group exploring whether AI can replace humans in a variety of roles, including the creation of ads and other brand communications.
Cadency, a student-run ad agency in the Erwin Center for Brand Communication, wrote ad copy for several fictitous products while an AI tool generated ad copy for those same products. The ads were split into categories — emotional or utilitarian. They asked people to rate the ads, and the results were a bit surprising.
“Our initial hypothesis was that humans might do a better job at emotional ads,” says Giebelhausen. “Instead, we found that humans were relatively better at creating utilitarian ads” — which Geibelhausen and his team theorized is because the AI was better at expressing empathy than sympathy.
“The algorithm was pretty good at adding emotion, actually, which is not that surprising in retrospect because a lot of the technology has been built around the idea of sentiment analysis and being good at detecting sentiment in language,” he explains. “We found it didn’t have as good a capability to understand problems that people need to solve, whereas copywriters really have felt that pain. So humans were able to write a more effective utilitarian ad.”
The project allowed siblings Meg and Charles Truluck the chance to team up and work together. Meg, a marketing major who works for Cadency, provided the human-generated ad copy, and Charles, a computer science major, worked with the AI algorithm to create the computer-generated copy. They both liked that the project gave them the opportunity to dabble in one another’s field of study.
“I feel like that’s something I wouldn’t have been able to do at a lot of other schools,” says Charles. “You would have to be a marketing major to do undergraduate research in marketing.”
“It’s cool to be able to do research with someone from a totally opposite major,” says Meg. “Computer science and marketing, you would probably say that they’re the farthest from each other, but it’s crazy how much they really do intertwine.”
Leave a Reply
Want to join the discussion?Feel free to contribute!