Fanjun (Frank) Bu
Ph.D. Candidate · Computer Science · Cornell Tech
I build and deploy robots that interact with people in real-world public spaces.
I am a Ph.D. Candidate at Cornell Tech, supervised by Wendy Ju. My research focuses on deploying robots in real-world social environments, where success depends on understanding and responding to human behavior.
Leveraging foundation models, I design and deploy robotic systems that integrate social and situational factors to improve adaptability and responsiveness in human-centered settings. I have extensive experience across robotics, simulation, VR/XR, and generative models, with an emphasis on translating research insights into deployed interactive systems.
News
- June 2025Returned to Toyota Research Institute for a second internship.
- June 2025Some new publications at CHI 2025!
- Nov 2024Gave a talk at the 2nd HRI International Symposium held by Naver Labs Europe.
- May 2024Start my summer internship at Toyota Research Institute.
- May 2023Presented our work Portobello at CHI and received an Honorable Mention Award!
- May 2023Gave a talk at Nokia Bell Labs Responsible AI Seminar.
Publications
-
arXiv
SEGAR: Selective Enhancement for Generative Augmented Reality. -
arXiv
Using Vision-Language Models as Proxies for Social Intelligence in Human-Robot Interaction. -
THRI '25
Making Sense of Robots in Public Spaces: A Study of Trash Barrel Robots. -
CHI EA '25
Boosting Visual Fidelity in Driving Simulations through Diffusion Models. -
CHI '25
The People Behind the Robots: How Wizards Wrangle Robots in Public Deployments. -
CHI '25
The Robotability Score: Enabling Harmonious Robot Navigation on Urban Streets. -
ICSR '24
ReStory: VLM-Augmentation of Social Human-Robot Interaction Datasets. -
ICSR '24
SSUP-HRI: Social signaling in urban public human-robot interaction dataset. -
DIS '24
Behind the Scenes of CXR: Designing a Geo-Synchronized Communal eXtended Reality System.Honorable Mention Award -
CHI EA '24
Field Notes on Deploying Research Robots in Public Spaces -
CHI '24
Portobello: Extending Driving Simulation from the Lab to the Road.Honorable Mention Award -
CHI '24
Trash in Motion: Emergent interactions with robotic trashcans in a public square. -
HRI '23
Trash Barrel Robots in the City. -
CHI '22
XR-OOM: MiXed Reality driving simulation with real cars for research and design. -
CoRL '22
Human-robot commensality: Bite timing prediction for robot-assisted feeding in groups. -
IEEE Access '21
Object permanence through audio-visual representations. -
RAL '20
Benchmark for bimanual robotic manipulation of semi-deformable objects.
Teaching Assistantships
-
Spring 24
CS5678 3D User Interface
VR/AR interaction design and spatial computing
-
Spring 23
INFO5755/INFO6755/CS5755/CS675 Mobile Human Robot Interaction Design TA Award
Designing and prototyping mobile robots for human interaction
-
Fall 21
CS4750/CS5750/ECE4770/MAE4760 Foundations of Robotics TA Award
Motion planning, kinematics, dynamics, and control
Press Coverage
- Staff, "These robotic trash cans were filmed to test human-robotic interactions. Watch what happened," CNN Business, April 12, 2023.
- Ayesha Rascoe, "Researchers released robot trash cans in NYC to see how people would react," National Public Radio (NPR), April 16, 2023.
- Patricia Waldron, "(Almost) everyone likes a helpful trash robot," Cornell Chronicle, April 19, 2023.
- Mike Snider, "Robots in the Big Apple: Robo-trash cans patrolling New York plaza make friends, creep out some," USA TODAY, April 15, 2023.
- Evan Ackerman, "Humans (Mostly) Love Trash Robots > Simple robots wander NYC asking for trash and recycling, and it's adorable," IEEE Spectrum, Mar 10, 2023.
More Videos
Mobile Hoverboard Robot Base
A low-cost, modular mobile robot platform built from a deconstructed hoverboard. Attaches to everyday objects — trash bins, furniture — for field HRI studies. Uses Wizard of Oz teleoperation via dual Raspberry Pi and ROS 2 Humble.
- DriveHoverboard motors + ODrive controller
- ComputeDual Raspberry Pi · ROS 2 Humble
- Control8BitDo + Insta360 FPV feed
Selected Projects
Vision-Language Models for Social Robot Intelligence
Investigating how vision-language models can serve as proxies for social intelligence in human-robot interaction, including building datasets and benchmarks for evaluating social signaling in urban public HRI scenarios.
Trash Barrel Robots in Public Spaces
Deployed autonomous robotic trash barrels in a New York City public plaza to study how people interact with robots in uncontrolled social settings. The project produced multiple publications studying emergent interactions, public sense-making, and field deployment methodology, and received widespread media coverage.
Portobello: Driving Simulation on Real Roads
Extended driving simulation from the lab to the road, enabling researchers to study human responses to simulated driving scenarios using real vehicles in real traffic. The Portobello system and the broader XR-OOM platform bridge controlled experimentation with ecological validity.