Bubble Popping

How to Play:
- Allow the camera usage.
- Pinch the bubbles using your index and thumb finger.
- Have fun!

Ideation

The goal is to integrate machine learning(ml5.js) in p5.js to create an interactive prototype. Objects not physically connected to the computer should be used to interact with the system. My first idea is to make a interactive game using webcam and machine learning. I decided to make a bubble popping game that using hand gesture to pop the bubble and a scoring system. This uses the webcam and ml5.js to detect hand and using pinch gesture to pop the bubble.

Process

1. I started by exploring the ml5.js capabilities and going through the examples. I found HandPose Parts example where it draws a green circle with pinch gesture and finger tip distance. I copied the code and modified it to flip the video. Instead of draw a circle with pinch distance as size, I created two small circles to track the index and thumb finger tips.

2. I created bubble class with position and radius, and a display function to display it on canvas if the bubble is not popped. In the main canvas setup, I created bubbles randomly on the canvas.

3. The bubbles are created randomly on the canvas, but they can overlap, which is not ideal. I modified the bubble creation process to check the new bubble position with all existing bubbles using dist() to make sure they are not overlapping. In addition, I factored in the canvas edge so the bubble will not appear halway off the screen. A small trick I used here is to count how many times the bubble is trying to be created but failed. I found this helpful when trying to create large number of bubbles, but for smaller numbers like 20 or 25 it is probably not necessary.

4. In order to pop the bubble, I need a way to detect pinch gesture. I can calculate the distance between the index and thumb finger tips, and if the distance is smaller than a certain distance, the hand is pinching. I also need to make sure the pinching happens inside a bubble in order to pop, so I added a check to see if the center of the pinch gesture is inside a bubble.

5. It'd be more fun to continue adding new bubbles after popping one, so the game can go on. I use current time millis() and lastBubbleTime to keep track of the time. If the current time - lastBubbleTime is greater than a certain interval(3s in this case), a new bubble is created. I moved the code for the bubble creation to a new function addBubble(). During setup, I call addBubble() pre-determined times to create all the bubbles needed, and in the draw() function, I call addBubble() when the time interval is met.

6. The bubbles are kind ugly because they are only white circles. To make them look better, I created layers. An outer rim using only stroke, the center bubble with a smaller circle and blue color, and a off-center circle as the highlight. I also made them a bit more colorful by using random filling color.

7. The refinement focuses on the look and feel aspect of the interactive prototype. When the bubble burst, I want to play an animation where particles flys out from the center of the bubble and disappear in the air. The particles are nothing but small circles, each with their own color, opacity, size, position, and angle to fly outwards. My first step is to create a Particle class with these properties. I also need a display function to draw the particle on the canvas.

8. The overall animation should have many particles spawn in the center, then fly outwards in different angle. The opacity should decrease when moving and when the opacity is 0, the particle should disappear. When all the particles disappear, the animation is finished. In order to animate the particles, it should also have an update() function called in each draw(). Every time the update() is called, the position of the particle changes slighly, and the opacity should decrease a little bit. To figure out how much position should change, I used the cos() and sin() function to calculate the change in x and y based on the angle.

9. In order the create the particles and later show them, I need to create an array of particles to store them in the bubble class. The particles should be created at the center of the bubble when the bubble is popped.

10. When the bubble is popped, it should enter the bursting state so that the particles can be displayed instead of bubble when bubble.display() is called. So basically, if the bubble is popped, and the bubble is bursting, display particles. I need a new state to represent this bursting state. So I added a new state to the bubble class, and display particles accordingly.

11. The particles are created and displayed when popping the bubble, but there are some problems. First is that particles are flying out in a circle, which looks a little wierd. They should all fly out randomly. Since the angle is already random, I need to make them fly in different speed. To achieve this, I added speed property to the particle class and calculate the velocity with the speed.

12. Another problem is that a lot of particles are created and flying out, instead just the 20 I created. If I continue to pinch, the animation will keep going and doesn't stop. I think this is because I didn't delete the particles after they disappeared. So I added a check to delete the particles if the opacity is 0. Also, if all the particles have been deleted from the particles array, the bubble should no longer in bursting state anymore, which means end of the animation. I also need remove the bubble from the bubbles array.

13. However, after I did the above, something is still going on, infinite particles are flying out from the pinch gesture as long as I hold the pinch gesture. This was not expected. After long debugging, I realized that 20 new particles are created and pushed on to the array every time the popBubble() is called in the draw() function. Since the draw() function is call so many times a second, the popBubble() function will be called many times as well. A simple fix is to check if any particles has been created and stored in particles array before creating any particles. I added the if check in the popBubble() function. There is still a small problem where the particle animation is repeated a few times, but I think it is good enough.

Next Project