Helping Robots to Reason About Touch

Mark Lee ’15 M.Eng. ’16 is exploring reinforcement learning in robot manipulation

by Chris Dawson, from the ECE Connections magazine.

Mark Lee ’15 M.Eng. ’16 grew up in both South Korea and the United States and he attended Thomas Jefferson High School for Science and Technology. One of his teachers there steered him toward a robotics team in tenth grade and once he discovered electronics and mechanical systems, he knew what he wanted to study. In some ways, Lee’s experience on that high school robotics team set the path for his next ten years. “I applied as an Early Decision applicant to Cornell,” says Lee, “and one of the main reasons was the Project Teams. I was looking to continue with those hands-on experiences where I learned so much and Cornell was a place where it was clear I could do that.”

As a first-year student, Lee joined the Cornell University Autonomous Underwater Vehicle team (CUAUV) and stayed with them all four years. “It became clear right away that things were on a whole new level compared to my high school robotics team,” says Lee. “With AUV, you’re not working from a kit. We had to go out and contact our own sponsors, buy our own materials, machine the mechanical frame, and build the electronic system ourselves. The logistics, designing, and testing were all on you.” Lee was on the electrical subteam of AUV, eventually becoming the team co-lead in his senior year.

Lee’s experiences with AUV informed his decision on what to major in at Cornell.. “What helped me decide on the major I wanted to pursue wasn’t really the coursework I took, but rather the hands-on  work such as designing circuit boards and programming microcontrollers.” For Lee that naturally meant choosing to major in Electrical and Computer Engineering (ECE). 

After earning his Bachelor of Science degree, Lee continued for another year at Cornell and received his M.Eng. from ECE as well, working with Associate Professor Zhiru Zhang.

As a citizen of South Korea, Lee was obligated to then return to fulfill his military duty, which he did as a robotics researcher. From 2017 to 2020 he worked at the Korea Advanced Institute of Science and Technology (KAIST) Humanoid Research Center, the winning research team for the DARPA Robotics Challenge in 2015. “My experience at KAIST strongly resembled a lot of the experiences I had at Cornell,” says Lee. “We were creating a robot platform—the overall mechanical structures, the design, the electrical components, and the high-level software—but instead of an underwater robot, at KAIST it was a bipedal robot. Of course, a walking robot is necessarily a little more complex, but the overall concepts I learned at Cornell carried over nicely.” 

The research problem Lee worked on was robot manipulation, or how robots can intelligently pick up objects. “Identifying an object and knowing where to grasp the object comes very naturally to humans through years of experience,” says Lee.  “With computers, it’s not so easy to distinguish objects from backgrounds, let alone determine where to make contact with the object that leads to a successful grasp. This problem becomes exponentially more difficult when there are multiple objects that are cluttered with one another. “

Lee’s term at KAIST is now finished, but his interest in robot manipulation is just beginning. To research this problem more deeply, Lee is now set to begin his doctoral studies in robotics at Carnegie Mellon University in the fall of 2021. “Knowing that robotics is such an interdisciplinary field—and that it requires a certain depth of knowledge in each of the component fields—has made me want to learn more,” says Lee. 

At Carnegie Mellon, Lee hopes to focus on the value machine learning can add to robotics. “A lot of roboticists would agree that not a lot has changed for robots in general in a long time,” says Lee. “And the limitations stem from the intelligence (or lack of intelligence) that robots have when reasoning about making physical contacts with the real world.” Lee argues that if we want robots to be more useful, both industrially and socially, then we are going to have to start creating robots that are capable of more flexible approaches to their interactions with objects and people.

Lee wants to help push the field beyond traditional approaches and enable robots to supervise themselves and adapt their behaviors to the situations they find themselves in. The area he plans to work on at Carnegie Mellon is called model-based reinforcement learning and it would allow robots to act with more autonomy than they are currently capable of. Lee can foresee himself eventually working as a product developer for whatever innovations grow out of his doctoral work.

In his five years at Cornell ECE Lee had many valuable classes and experiences that influenced his path to a Ph.D. program at Carnegie Mellon. When he looks back at his Cornell years now, he specifically mentions faculty members Bruce Land and Zhiru Zhang as having been especially inspirational and supportive. “Bruce Land was the living embodiment of the “ECEs Can Do Everything” idea,” says Lee. “His approach to understanding how things work taught me a lot. And Zhiru Zhang encouraged me to try to learn something new during my M.Eng. program and even helped put me on the path to wanting to do research and pursue my Ph.D.”