Final Product
The five members of Big Summer Blowout! created an interactive dancing penguin that detects smiles and responds to different states with three different dances. The penguin itself includes four degrees of freedom while the smile recognition program reads in images and analyzes them via a Raspberry Pi. It then sends messages to an Arduino Uno to manipulate the servo controlled mechanisms
Mechanics
This project was an excellent lesson in mechanical design, and opened our eyes to myriad mechanisms that provide various types of motion. We considered many more designs than we used in all of our iterations, ultimately choosing those that provide the desired motion in the simplest way. In the early iterations especially, the Keepon inspired much of our vision and understanding about how to achieve our intended degrees of freedom. However, as we progressed through the project, our goal-set changed and we encountered different mechanisms than those used in the Keepon that were better for our budget, motions, experience, and available tools.
The overall effect of our project is still greatly inspired by Keepon, as we stayed with the idea of acting upon a stick that sits on a base that contains the mechanisms.
Software
Frank dances for people and their smiles! He has state-of-the-art smile recognition using a logistic classifier trained from a data set of more than 20,000 faces labeled according to whether or not the face is smiling. The smile recognition software first receives the video stream data frame by frame. Then, using haar cascades in OpenCV, the face is cropped out, adjusted, and classified into a smile or non-smile category. Depending on the number of faces and smiles seen, the penguin would dance accordingly. Through this exploration, we learned quite a lot about the overlap between image processing and machine learning in a directly applicable manner.
Integration
Frank has many different dance moves with his four degrees of motion. Just as difficult was making sure that Frank got the right information when he needed it. Controlling those four motors draws more power than the Raspberry Pi outputs through USB, so we looked into alternate methods. We ran into heat shutdowns, stalling motors, improperly received image data, and all the time-delays and lag when we moved away from the very secure and stable environment in which a laptop was our power source. In the end, we were able to be more portable than we had even anticipated, drawing power from a 4400 mAh USB battery pack and 4 AA batteries! We even remotely processed the video frames streamed from the Raspberry Pi on a laptop, wirelessly!
Inspiration
Fundamentally, we wanted to create a robot that would dance in reaction to facial expressions. There are several routes that we could have taken such as creating a humanoid robot or just the upper body to limit the degrees of freedom. However, we decided that creating an adorable, non-human robot would appeal to a wider audience because of the added emotional attachment. As a result, we researched dancing robots such as Hexy, Keepon, and solar powered dancing flowers for inspiration. We decided to use Keepon for our main inspiration because it had four degrees of a freedom and we could borrow one from the Olin Robotics Lab to use as a mechanical reference. As we progressed through the project, we have incorporated our own mechanisms, dances, and motions that have improved or changed the original design to match our needs.
Members of Big Summer Blowout!
Pratool Gadtaula
Pratool is an Electrical and Computer Engineering major from the class of 2017. It's been rumored that Pratool once disappeared into the woods of upstate New York and reached enlightenment into the inner workings of a Raspberry Pi (of the B+ variety). As a result, he was able to connect our Pi to an Arduino to allow the motors to respond to our facial detection system. He played a large role in integrating the many elements of our project.
Meg Lidrbauch
Meg is a part of the Olin College class of 2017. Surprisingly, even with her close-to-edgy sense of sarcastic humor (little did she know that everyone enjoyed her jokes), she gets along with everybody who matters, her teammates. Even our computers got along with her and her most smile of smiles. She enjoys the cold and winter (and possibly the summer), because if it wasn't for her we wouldn't have a theme (Olaf was a hard sell, but we compromised) -- or a legit mechanical system.
Julian Morris
Engineering with Computing major, class of 2016. Is secretly a troll. Enjoys taking selfies in the corner (he says he's working on the smile detector but we know what he's actually up to). Skills include Java, Python, Arduino programming, and playing with everything around him when he's bored. Is currently learning SQL, C++, and how to dance like a penguin. His contributions to this project included working on the software side, website, and causing a minor battery explosion.
Michelle Sit
A Wellesley College senior. Biked to come to team meetings from Wellesley sometimes on weekends. Helped create the dancing motion, the smile detector, the final website, and came up with the dancing moves for the penguin. Dreams of becoming a penguin one day. Likes to perform martial arts. Possibly related to her desire to become a dancing penguin.
Jay Woo
Jay likes to work sideways. No one knows why, but he does. In addition to making life harder for himself, Jay is taking 6 classes (5 for credit). For this project, Jay pretended he was a Mech:E, designing the ornithopter linkage that makes the wings flap. Hmm... ornithopter... ornithology... the study of birds... eeenteresting. Oh, uh. Yeah, Jay. Okay. Jay was very helpful, but we can't remember what he did. But we couldn't have done it without him! We <3 Jay.