May 16

Weather the Weather in Action

Video Footage from Imagine RIT, and RIT Spring Fest. Edited by Dan.

Feb 5

1. Is if fun to play?

2. Is there a reward at the end of the game or along the way?

3. Is the game a challenge to play? Is there a learning curve that allows you to get better each time you play?

4. Is the game non-frustrating?

5. Does the game have immediate repeat play value and long-term repeat play value?

6. (Optional) Is there something “magical” about the game? Usually, in a board game, that would involve color-changing ink, magnets, clever use of electronics, 3-D viewers, etc.

When I evaluated games at Milton Bradley and Parker Brothers from inventors and from inside game designers, only games that had all of the first five attributes would be considered for the line. Sometimes skill-and-action games could get away with just four of the five because they were more ‘toys with rules’ than real games.

Jan 26

A quick demo of glowsticks being tracked to create images on the screen. This is pretty much the system we would be using in our game.

Jan 23

picture-23

Immersion

Something our team has discussed with this project is setting up a video feed of the players’ interactions with our game, and displaying it on a TV outside the exhibit. This is the perfect example of why this is both worth it and necessary. Watch it to the end!

Jan 23

To accomplish our goals we will need to develop a few basic game engines that will be used with each of our mini-games. We will be using Processing and a few Java-based libraries, most likely JMyron and Box2D, to accomplish this. We plan on building at least 2 different engines to base our games off of.

The “Color Engine” will use JMyron to help detect the players as a “blob.” This shape will be used to position any avatar or overlay graphics we may want to apply to each user. Users will also wear distinct colors, probably either glow-sticks or bright colored stickers, on specific body parts to not only help distinguish one player from the other but to also allow the game to track individual points of interaction (i.e. hands, feet, or head as cue points). Each game utilizing this engine will allow users to interact with game elements using their extremities.

The “Silhouette Engine” will utilize the silhouette that a user makes against the lit backdrop to interact with the game.

The silhouette will either be used with hit detection on game elements or to detect a user’s body position. An example of the latter being: a user is presented with a shape and they then need to fit their body/silhouette into this shape.

Mixing these two engines to create a third engine is also a possibility. Game physics will utilized Box2D to create a rich gaming experience for each user.

Problem Areas:
A considerable amount of the computer’s processing power will be devoted toward the camera tracking. Thankfully, Processing allows for OpenGL support. This lets the bulk of the graphics processing to be done by the computer’s graphics card. Provided we do not go overboard with our game assets, rendering them should not be an issue.

Problems may occur when the source resolution of the camera is set too high. If the invisible rendering of the source camera is too large, the computer will not be able to go through all of the pixels from the video feed fast enough, and the game will experience a considerable slowdown. The trick will be to find a “sweet spot” where we can successfully get all of the data we need from the camera (i.e. silhouettes, and trackable colors from the glowsticks) and still maintain a level of performance that is acceptable for gameplay. Ideally we would like to maintain 30 frames per second (FPS), this would mean our application would have to grab the silhouettes / key colors from the video 30 times every second.

The performance of the end product will mostly depend upon our ability to optimize both the resolution of the source video, and the efficiancy of our engine.

Jan 22

After the presentations, the most positive response seemed to be to the Human Machine idea. The snowball game could be interesting, but would be harder to instill deeper meaning into, and we don’t want to build just another snowball game. The dancing art idea slowly got accumulated into the Human Machine as well – all we would have to do is have the body movements generate some sort of trailing sparkles, or whatever fits the concept the best.

From there, we focused on team building. In order to have the Human Machine be more interesting than just another game, we decided it should really force the players to work together as a community to accomplish some task. We are looking pretty extensively into team building activities (huge lists found here, here, here, and here) for ideas on the kinds of games we might want to incorporate into our installation. We want to keep these games very simple, but spin them in such a way that their overall meaning to the game is much bigger than just fitting shapes into other shapes or whatever. Linzi found this cool Swedish Armed Forces recruiting website, which does a fantastic job of keeping the games simple but the users entertained. And it does a great job of tying the simplicity of the game to the overall concept.

I also found this paper that goes through the decision making process for creating team building games in Second Life using the model of teams that are spread out wide and not next to each other. It’s long but interesting, with good examples of abstracted tasks representing a larger whole. It goes through several games:

  1. Crossing the Ravine – people have to build a bridge out of different geometric blocks each of them has
  2. Tower of Babble – people have to build a tower without having it topple, out of simple geometric shapes. Jenga principle.
  3. Castle builder – different people create assets to build castle out of, others move the pieces to actually create the building to achieve some sort of goal.

All of these games take very simple games as their starting points – fitting shapes together, jenga, tetris, etc. The games we come up with needs to be really easy to understand, it needs to be reduced to its bare essentials, and at the same time it needs to encourage teamwork and entertain the players. From here we set about trying to make things simple.

Jan 22
The development team has been busy researching and evaluating the different technologies we might be able to use in this project. This is what we’ve come up with so far.

AS3 vs. Processing

Actionscript

  • There seems to be a lot more existing motion tracking projects using AS3
  • Possible multi-use
  • Web deployable

Processing

  • Open source
  • Hardware accelerated
  • Uses openGL
  • Much greater performance
  • built in 3D
[end_columns]
Basic color tracking method

For Human Machine
Captured images are run through a few filters in order to more easily detect the center of the objects. First is a mean filter that causes the image to be blurred, reducing noise. An RGB filter is then used to remove all objects from the image except for those areas where the requested RGB values exist. The image is then run through a threshold filter that reduces it to a black and white representation of that image. The white areas are where the requested color exists and the black areas are where the color is absent. With this image areas of white can be clumped together to track multiple objects and the center of each of these objects can be calculated. Here is a tool that can help you see what I’m talking about. Follow the directions using the orange circle on the image. You will see that it can easily be picked out of the rest of the image. So with this method if we have fast processing we can track multiple objects, ie. glow sticks, by running each frame though this type of filter. Fun stuff!

For Snowball Fight
There are two ways we can go about interacting with this game. The first would be the above method. This would involve some type of glove or band that had a very unique color. The second would be with motion tracking. Most motion tracking that I have research functions pretty much the same way. The web cam sends images to the program and the code finds the difference in color of pixels between the current frame and the previous frames. These areas are where motion is taking place. The pixels that have changed are plotted and the center of the motion is determined from there. Here is an example of motion tracking in action along with its explanation.

I think the most effect way of motion tracking would be with color. It seems to be more reliable because the motion of your body will throw off a motion tracking system where in color tracking only the colors are registered. I am also interested in using the processing language because of its performance advantages and because I am interested in learning it. And that’s all I have to say about that.
Jan 22

As ideas came up during our brainstorms, our development team researched the possibilities for implementation of said ideas. At this point we’ve pretty much determined that we want to focus on video motion capture technology for our three ideas to allow the users to interact with our game. There are numerous ways to do that, from silhouette tracking to color tracking, etc. What resulted is a library of links to various videos, websites, etc, as well as some qualified assessments and evaluations of the use of technologies research for our needs.

Motion Tracking using a webcam and AS3


With Color…


[end_columns]

Jan 20

After feeling like kids at a candy store while looking at such a broad pool of inspiration and research into potential technologies, and just plain cool stuff, we began to narrow down our focus to decide what our project would be all about. Here I have to give due credit to Linzi, who right from the get-go really wanted to be project manager, and with good reason – I don’t think I’ve ever seen such an organizedly chaotic brainstorm in my life.

To start the brainstorm off on the right foot, we considered the parameters of the project. First of all, according to the rubric for the class, it needs to be presentable at Imagine RIT: Innovation and Creativity Festival here on campus. This pretty much defines our audience: adults, college students, families, and children. Of course, if we’re to present this project at Imagine RIT,  we want it to be more or less universally accessible, and placed in a high-traffic area so that it gets the most exposure possible, whether indoors or outdoors.

Projects also tend to be more successful if they fulfill some sort of want or need of the user. This could be anything from fostering a sense of community from a multi-user experience, to “memories” and little takeaways from the experience. Escapism can be a great example of a community desire that needs to be fulfilled, whether you want to “escape” from yourself for a while by controlling a completely different character or avatar on the screen and losing your sense of self-consciousness through that experience, or something as simple as finding yourself on a virtual beach with sunny weather to escape the dreary cold of Rochester. This want or need can also be an invoked emotion, a shared experience and “bonding” with other people, or a sense of accomplishment. This installment is meant to be all kinds of good-natured fun – we definitely don’t want to make anyone feel stupid.

Although the possibilities for this project are tempting and endless, one thing we wanted to keep in mind before we got too involved in the details of things is to keep everything SIMPLE. We even wrote it in 60 point font in our notes. It’s that important. You will see why in a second.

The possibilities for the type of project and the technology we can use are, like I said, pretty much endless. Installation vs. TakeHome vs. Website vs. any combination of the three are only a few of the ideas we came up with right off the bat. As far as non-mouse interactive technologies here’s a whole list:

  • Keyboard
  • Camera
  • Video game controller
    • DDR pad
    • Rock Band equipment
    • Guitar Hero guitar
    • Wii hit board
  • BX24
  • Projector
  • Multi touch screen
  • Infrared, webcam
  • Playstation game camera
  • Pins/sensors for touch

But what do we actually want to do? After miles of notes and hours of deliberation we came to a basic consensus. We want to blow your mind. How that will be done is up for discussion at this point, but done it will be. Don’t worry.

A huge source of inspiration for Linzi was the Terrascope(?) installation in New York City. The installation was composed of a giant sculpture of a telescope that used a webcam feed from London to display in the eye of the telescope. This facilitated communication between two cities very far apart, to the point where people on opposite sides of the earth were having drinking contests via “telescope.” Now that’s community bonding right there.

The idea of facilitating interaction between remote areas really struck us and even kind of stuck for a while. After talking about the MIT Party Room we got to thinking – what if users online couldn control the physical environment, for example the composition of a room? Somewhat like playing God, yes, but the collaboration between a removed decision maker and a person experiencing the environment firsthand could be interesting concept to explore. This also brings in the idea of teamwork, which is something we’ve slowly been zeroing in on during this brainstorm. If we do a game, like a puzzle game, or a maze, where you actually have to use your body to find your way, we could set up multiple installations and allow people across campus to work together to complete the games.

Another aspect that we want to bring into this project is viral marketing for the project. We want it to spread like wildfire through campus on the day of Imagine RIT by word of mouth, through handouts, giveaways, etc.

More from this brainstorm session can be found here.

Jan 20

After our first meeting everyone went out and found their respective “Cool Stuff” links – anything that inspires, informs, or influences our ideas about the project, technologies that we might be able to use, do it yourself projects, etc. etc. The list turned out to be fairly extensive and can be seen in full here, but here, for your viewing pleasure, our compilation of “Cool Stuff” – just for you. And everyone else on the internet.

Mike:

Robert Hodgin is my hero.
Weird webcam based game.
Eye tracking to calculate depth of field.
Head tracking with webcam
Realtime face tracking
Realtime gesture analysis
Realtime face tracking

Linzi:

Bjork ‘Declare Independence’ Music Video
Head Tracking for Desktop VR Displays using the WiiRemote
Daito Manabe YouTube
Robot Controlled by WiiRemote
WiiRemote for Virtual Reality
Immersive Eye Tracking
In-expensive Eye Tracking (we could afford this)
Living Interface
Point Screen 2 3
Music Wall
Hacked Gadgets
Laughing Man’s Copy Cat (avatar face tracking)
Quadrilateral Optical Recognition
Game Control Using Webcam Tracking OpenSource
How to Make a Multi-touch Screen (home-made) $2
Sand Animation
Cool Pics 2 3 4
Installation Art 2 3 4 5 6 7 8
Graffiti Research Labs 2
Zachary Lieberman 2

Andrea:

Cool instillation art
Flash- interactive Motion
Wodden mirror
Interactive fireworks display
Laser writing: Video
Christmas light show

Krista:

Eyetracking Gear
Trials of Topoq (PS3 camera game demo)

Ira:

Domino System Video
Brain Cap Controlls Games
Multitouch 3D Interaction Screen
Microsoft Touch Surface + Wireless Device Linking
Lowcost Multitouch Whiteboard with WII remote
3D Video Interaction
Particles in Flash controlled by Webcam Interaction
Webcam Motion Detection & Audio Detection
WiiFlash and Papervision 3D
Webcam Snow
3D Graffitti
Robot reads, does math, slaps on forward advances

Dan:

Balance board
Tenori demo!!!!
Tenori main site
Interactive multi-touch
Pen & touch
Neutouch

Andy:

Boffswana (tracking a piece of paper)
Weird system for papervision
Wiimote 3d particles
More webcam tracking with flash
Webcam / pool table projection

Zach:

[end_columns]