Mr. Scribbles: Dancing Drawing Robot

2020


The Dancing Drawing Robot was created to help people feel more comfortable about their bodies, about their movements — about being weird sometimes.

To control the robot, try to recreate the dancing poses!





   



















Roles



  • Pose classification and user interface by Yona Ngo.
  • Physical computing by Stuti Mohgaonkar.
  • Fabrication by Yona Ngo and Stuti Mohgaonkar.
  • Supervised by David Rios & Luisa Pereira.


Some people love expressing themselves while dancing; some people do it while drawing. We thought, why don't we combine those two and create something fun?
The Dancing Drawing Robot was created to help people feel more comfortable about their bodies, about their movements — about being weird sometimes ;)

To control the robot, try to recreate the given poses. The robot can move from left to right and has two arms that can rotate 180 degrees.









Background



The Dancing Drawing Robot started out as a solely drawing app project with PoseNet and circular potentiometer and switch buttons as a physical controller to navigate through the app. The main goal was to discover the new ways of using the physical computing and machine learning algorithms to create interactive experiences. After multiple iterations it was decided to take a completely opposite approach — what if the user could use machine learning to control someting physical? It was also important for us to create something fun that would help people to feel free and entertained.














How It Works



Mr. Scribbles is a robot that can move from left to right, right to left and draw with two “hands” that hold the pens. To control the robot the user stands in front of the camera and recreates one out of 10 poses that the pretrained model can recognize.
















Tools & Process


Mr. Scribbles was created using ml5js, p5js and Arduino Mega. Read more about the development process here.
Thank Thank Thank


Special thank you to the people that contributed to this project: David Rios for supervising, Alan Winslow for helping with the video, Viole He, Kevin He, Shira Levi Sun for posing for the data collection, and ITP for having us.
Features


This project got featured at ITP/IMA Winter Show 2020.


Airi Flies

2019


An interactive game played using your voice.


Airi Flies is an interactive game played using your voice.
To start the game say PLAY. Say PEW to make Airi fly. If you lose more than 8 points or hit an obstacle, you lose.

This project was created to encourage people to get out of their comfort zone and feel more confident about themselves regardless of what they do and what they look or sound like.


P.S. Works for Google Chrome and Firefox.
















Roles



  • Coding & Visuals by Yona Ngo,
  • Supervised by Yining Shi


Developed as a final project during Bots & Machine Learning course at School of Machines in Berlin, Germany.






Background



A few weeks before I started working on this project I was learning to use the beta version of Google's Teachable Machine. But I found myself feeling so embarrassed trying to teach the machine to recognize some simple words.
It brought me to the thought, that there are so many people that are just too shy to express their unique weirdness in public, because it is not something the world is used to.
I created this game with the hope that people would start feeling more confident about themselves no matter how they look or sound like.





How It Works



The sound-classification model was trained using teachable machine 2 on 120 samples of 9 different female and male voices in addition to 84 samples of background noise. The model then is used with p5.js to create visual elements, that change their positions and appearances whenever the classes PLAY or PEW are detected.









Tools & Process


Airi Flies was created using p5js, ml5js and Teachable Machine.
Thank Thank Thank


Special thank you to the School of Machines, and Yining Shi who supervised this project.





Features


Featured on ml5js.org, p5js.org, Google Teachable Machine.

Airi Flies was presented at the final showcase at ACUD MACHT NEU in Berlin, where more than 200 people attended.
It was a great pleasure watching guys and girls whispering the word PEW at first in order to make Airi fly, then realising they should say it louder and laughing at themselves saying it one more time as loud as possible.

I trully believe this game brought lots of joy to the ones who played it and to the one, who created it.

when you don’t reply


2019


when you don't reply. is a web installation that represents another side of online relationships.

In the digital age, the way we communicate online can be so different to real life. Something, that might not seem hurtful to hear might be extremely painful to read.

This web installation was created to show how often online communication can make a person afraid of being forgotten, feeling lonely, or being misunderstood.














Roles



  • Coding by Tommy,
  • Design & concept by Yona Ngo.





Try Talking to Matsu Yourself




Background



Not a long time ago I had quite a deep talk with my colleague about virtual communication. She told me about how often she overthinks when she talks to someone online. Apparently, she was not the only one who had such struggles.

Online relationships played a huge role in my life as well. In my teenage years, being the only asian in my school, I used to seek comfort in online communities and language exchange websites, which allowed me to talk to people from all over the world and feel less isolated.

While it brought me a lot of joy and friends, it brought a lot of misunderstandings as well.

Sometimes, delays in response or incomplete messages can create a lot of negative thoughts, often to the point that you start losing confidence, disliking youself and thinking that you are unwanted.

My online friend Tommy and I decided to create this web installation to remind people that overthinking and having unwelcome thoughts is a part of online communication that everyone goes through. It is normal to overthink sometimes since we are all human. Though, it is important to be aware of it and not to let those negative feelings make you feel unloved or lonely.






Tools & Process

The project uses React.js to create a web chat interface that waits 5 seconds for you to type the first message. If you did not say anything it picks a random message from a predefined array of 'bad' messages to reply to you.
Thank Thank Thank

Miss my dear ex-colleague Alice and our talks on online relationships.
Thank you, Tommy, for helping me with React.js when it was Greek to me!

Draw The Word: A Draw and Guess Game But Using Your Phone's Tilt


2021

Draw the Word is a traditional Draw and Guess game but with multiple drawers and only one guesser. All the players except the one who guesses have to draw on a single canvas by tilting their phones.

This is a group project for Collective Play course, taught by Mimi Yin.


Roles



Code by Yona Ngo,
Architecture & Diagram by En Tung Liu,
Instructions & Testing by Angelo Espinosa Tiu ,
Supervised by Mimi Yin


















How to Play



  1. Arrange yourselves in a circle facing inwards with your phones, connect to the server using the link.
  2. One of the players will be randomly selected to be the guesser. Everyone else will be given a shared prompt through your phones. Please try your best to try and depict the word by using your phones’ tilt to draw on the canvas together.
  3.  You are allowed to communicate with your collaborators, just make sure you don’t reveal the prompt to the guesser. The guesser will then have a limited amount of time to guess the prompt based on your collaborative work.
  4. The guesser can guess the word when the drawers are drawing or after that.
  5. The drawers have 60 seconds to draw.























Playtest



During the playtest, we also tested out different approaches to play the game.

Another way we tried out was to have a group of people as guessers and another group as drawers. The drawing was performed by putting the phone on your head and try to draw the word by moving your body.







How It Works



The Draw the Word game was created using p5.js, node.js, and socket.io. A host client manages when the game starts, and when it ends, player clients provide the drawing features and a canvas. The server receives the information about each client, generates the random word to draw and defines, which player is the guesser.





Tools & Process

Draw The Word was coded using socket.io, p5.js, express.js and glitch!


You can see the source code here.
Thank Thank Thank 


Thankful for being in the same group with Angelo and En Tung.  Special thanks to my classmates who had enough courage to playtest the game, and to Mimi Yin, who supervised and code reviewed this project.






sound pal

2019

sound pal is a data visualization and sound of more than 10k Facebook messages between two friends.

I was curious about different ways for a human to perceive data, besides a simple visualization that all of us already got used to. I ended up coding an algorithm that parses through each message and turns it into a sound.


Roles



This is a solo project.
Coding & Visuals by Yona Ngo.








How It Works



After downloading the chat history on Facebook, I wrote a Python script that goes through each message and finds the most common words my friend Tommy and I use.

It turned out that besides the words "I" and "you" we used quite often slang and meme words that only we could understand like poop emoji 💩, the words "lol", "ahisee", "sweats" etc.

I picked the top 10 most commonly used and interesting words with individual priorities and set a corresponding note to them. The sounds are also different depending on the sender's name. If a sender is "Phuong Ngo" the pitch is higher and is lower for "Tommy".

Then with JavaScript, the algorithm reads each message every 0.1 seconds, analyzes if the message carries the keywords and plays the note of the found word with the highest priority with Tone.js.





Data Visualization Poster



The poster above presents each message as a polygon. The number of each polygon's edges corresponds to the number of keywords mentioned in a message, the color is picked according to the keyword with the highest priority and sender's name, e.g. my messages are represented in brighter colors and Tommy's messages are darker.

This poster was inspired by Yining Shi's Friendship.am posters.






Tools & Process

The sonic visualization was created using Tone.js, poster — p5js. The data from the messenger were also processed with Python.
Thank Thank Thank

Special thanks to my friend Tommy for letting me use our messages in this project.

made with ♥ by yonaymoris in 2021