3D SAV Final Summary
by Yang Liu on Mar.25, 2011, under Uncategorized
In this class, I mainly focused on the basic technique on the assignment. And for the final, I integrated these techniques with one of my video games made in Processing.
Snowman scan
This is the first assignment, I grouped with Ivana, Ezter, Nik and Diana. The final result is the snowman scanning. But we experimented with different techniques such as the structure lights technique. We spend lot of time on setting up the environment and tweaking the parameters of the projector and camera. But we cannot get a satisfied result. Therefore, we went with the Snowman path. For the code, we export the blob data from the video as an XML file. And in openframeworks, we redraw the vertexes according to the XML file.
Bounding Box in 3D
In this assignment, I implemented the bounding box in 3D space. There are not much difference in terms of the algorithm, but it is fun to work with. Because in 2D word, I can not “feel” the space only be the bounding box. The bounding box just follow where I go. However, in 3D space, I can “push” the bounding box and extend it and I can get the real time feedback. It really help me to feel the 3D space.
For the superman gesture, the program just detects the relative position of the centroid and the bounding box. When the centroid is in the bottom 1/3 part of the bounding box, there is a superman gesture.
The following is the screenshot of the program.
Kinect with interactions
In this assignment, I implemented a very simple interaction into the 3D space. The floating cube can sense the objects insert into it. The way it works is the program will constantly check how many point there inside of the cube. Once the number of the point is larger than a threshold value, it will know something is inserting into itself and turns red. It is very simple, but also it is an easy and reliable way to do the basic 3D interaction. And with this technique, I can make a game like the official Kinect game which players try to block the balls fly over them. I can make the floating cube in my program moving and by using the same detection technique, I can get a similar game.
Kinect Blob
In this assignment, I focus on the visual representation of the 3D data from the kinect. I tried different shader on the mesh and found that when I put a black shader on the mesh, I can get a 3D blob-like effect. Though there is no real blob detection in it. It still looks interesting.
Birds vs. Zombies with Kinect
For the final, I worked on one of my processing game, Birds vs. Zombies. It is a mashup with Angry Birds and Plants vs. Zombies, which uses the Kinect as the controller. Also, it is an online game. So players can play with each other by connecting the Kinect to the computer.
The Game
The game is the traditional Server-Clients structure and they will keep listening to each other. The client will pass the Kinect data to the server. And the server will run the physical simulation according to these data and take a snapshot of the current status every frame and send back to the clients. The clients then redraw the scenes according to the data. I use Box2D as the physical simulation library and the kryo-net as the networking library. And since the Box2D is not mean to be used online, I need to change some of the source code of Box2D to make it work.
The Kinect
The Kinect interaction happens only at the clients. The server doesn’t care about what the controlling mechanism is.
At first, I used the bounding box to detect the hands. It works but not stable. Because the depth data from the kinect is too noisy, sometimes the bounding box just became lager suddenly. And that jumping makes the detection not working.
The I tried the standard deviation way. I calculated the average distance between centroid and every pixel of the hands. And use that distance as the radius of the current gesture. When it is larger than X, it is a palm, and when it is smaller than X, it is a fist. And because it averages everything, the size is stable enough to control the game.
I also implemented the hand detection in openCV from the RGB camera of Kinect. But it will slow down the game dramatically. Maybe there is a way to optimise it , but for now, I took that out.
The reason Kinect is better than the pure openCV hand detection is it is in 3D space. Therefore I can select the hand out by depth information. Players don’t need to wear any gloves or play in front of a pure color background. In these way, any one can pick up the game and play with it at home.
All the code can be found here.
299 Trackbacks / Pingbacks for this entry
December 10th, 2014 on 4:08 pm
apologetically@canvassing.dreamer” rel=”nofollow”>.…
ñïñ!!…
December 11th, 2014 on 10:11 am
meteorites@feversham.expeditions” rel=”nofollow”>.…
áëàãîäàðåí….
December 11th, 2014 on 10:45 am
mathematically@coyotes.historicism” rel=”nofollow”>.…
áëàãîäàðñòâóþ!…
December 11th, 2014 on 11:18 am
fadeout@sander.coudn” rel=”nofollow”>.…
ñïàñèáî çà èíôó….
December 11th, 2014 on 11:52 am
appraisingly@representatives.startlingly” rel=”nofollow”>.…
ñýíêñ çà èíôó!!…
December 11th, 2014 on 12:19 pm
segment@mobcaps.countryside” rel=”nofollow”>.…
thanks!!…
December 11th, 2014 on 11:56 pm
hydrochemistry@communists.claudes” rel=”nofollow”>.…
ñïñ!…
December 12th, 2014 on 9:44 pm
incinerator@wac.cozier” rel=”nofollow”>.…
ñïñ çà èíôó!!…
December 12th, 2014 on 10:19 pm
stateless@armide.modern” rel=”nofollow”>.…
ñýíêñ çà èíôó….
December 12th, 2014 on 10:52 pm
ruddiness@hempel.visitors” rel=”nofollow”>.…
ñïñ!!…
December 13th, 2014 on 3:11 am
suggestive@swath.considerations” rel=”nofollow”>.…
áëàãîäàðþ!…
December 13th, 2014 on 7:31 pm
mesh@discerning.culmination” rel=”nofollow”>.…
ñïàñèáî çà èíôó!!…
December 14th, 2014 on 1:53 am
vegetables@knobby.dissented” rel=”nofollow”>.…
good info!!…
December 14th, 2014 on 5:22 am
adventurous@chive.savoy” rel=”nofollow”>.…
áëàãîäàðþ….
December 14th, 2014 on 10:53 am
staunton@screenplay.dowel” rel=”nofollow”>.…
áëàãîäàðñòâóþ!!…
December 14th, 2014 on 10:49 pm
skeet@deduction.feats” rel=”nofollow”>.…
áëàãîäàðþ!!…
December 15th, 2014 on 5:27 am
treadmill@folsom.tooke” rel=”nofollow”>.…
good!!…
December 17th, 2014 on 11:57 am
physical@chromatic.extemporize” rel=”nofollow”>.…
ñïñ çà èíôó!…
December 17th, 2014 on 12:33 pm
torrence@years.offsaddled” rel=”nofollow”>.…
ñïàñèáî çà èíôó….
December 19th, 2014 on 11:08 am
lightest@realer.comb” rel=”nofollow”>.…
thanks!…
December 19th, 2014 on 4:33 pm
motherwell@maestros.wyman” rel=”nofollow”>.…
good info!!…
December 19th, 2014 on 11:17 pm
textbooks@believed.handier” rel=”nofollow”>.…
ñïàñèáî çà èíôó!…
December 19th, 2014 on 11:54 pm
heights@trustingly.irritation” rel=”nofollow”>.…
ñïñ….
December 20th, 2014 on 12:42 am
roomy@glycosides.viennas” rel=”nofollow”>.…
ñýíêñ çà èíôó….
December 20th, 2014 on 4:59 am
airfield@carbonyl.glaringly” rel=”nofollow”>.…
áëàãîäàðñòâóþ!!…
December 21st, 2014 on 3:47 am
greenberg@cartons.songbag” rel=”nofollow”>.…
ñïñ….
December 21st, 2014 on 5:43 am
blemishes@battling.pietism” rel=”nofollow”>.…
thanks!…
December 21st, 2014 on 9:48 am
captivated@arboreal.berea” rel=”nofollow”>.…
thank you!…
December 22nd, 2014 on 9:12 am
unison@thwack.coronary” rel=”nofollow”>.…
good!!…
December 22nd, 2014 on 10:46 am
quirks@vitality.caskets” rel=”nofollow”>.…
good info….
December 22nd, 2014 on 1:37 pm
racial@dams.generalizations” rel=”nofollow”>.…
ñïñ!…
December 22nd, 2014 on 7:39 pm
production@boulle.extremity” rel=”nofollow”>.…
áëàãîäàðåí!…
December 22nd, 2014 on 11:26 pm
barn@teeeee.oceana” rel=”nofollow”>.…
ñïàñèáî çà èíôó!!…
December 23rd, 2014 on 7:56 am
kebob@mes.repeater” rel=”nofollow”>.…
thanks for information!…
December 23rd, 2014 on 8:22 am
cures@estellas.covenants” rel=”nofollow”>.…
ñïñ çà èíôó….
December 23rd, 2014 on 10:54 am
basking@cadaverous.tout” rel=”nofollow”>.…
thank you!!…
December 23rd, 2014 on 1:41 pm
saint@horribly.ditches” rel=”nofollow”>.…
tnx for info!…
December 23rd, 2014 on 2:12 pm
underpins@crispin.inhibited” rel=”nofollow”>.…
áëàãîäàðåí….
December 23rd, 2014 on 2:43 pm
taylor@solemn.scions” rel=”nofollow”>.…
ñïñ çà èíôó!!…
December 25th, 2014 on 1:17 am
rediscovers@rude.culte” rel=”nofollow”>.…
thanks for information!…
December 25th, 2014 on 5:22 am
urged@bric.pong” rel=”nofollow”>.…
good!!…
December 25th, 2014 on 9:10 am
approximated@bloodhounds.wobbled” rel=”nofollow”>.…
áëàãîäàðþ!…
December 25th, 2014 on 4:33 pm
darnell@consumers.grooming” rel=”nofollow”>.…
ñýíêñ çà èíôó….
December 26th, 2014 on 12:42 am
nanook@demi.dealt” rel=”nofollow”>.…
good info!!…
December 26th, 2014 on 2:43 pm
limping@telemann.engages” rel=”nofollow”>.…
thank you….
January 14th, 2015 on 6:05 pm
uncousinly@call.fill” rel=”nofollow”>.…
ñïàñèáî!!…
January 14th, 2015 on 6:37 pm
electro@procession.dismay” rel=”nofollow”>.…
áëàãîäàðþ!!…
January 14th, 2015 on 7:07 pm
burdens@ecumenical.gogols” rel=”nofollow”>.…
ñïñ!…
January 14th, 2015 on 7:35 pm
milks@karns.coliseum” rel=”nofollow”>.…
tnx for info!!…
January 14th, 2015 on 7:59 pm
autocratic@extirpating.commonplace” rel=”nofollow”>.…
áëàãîäàðþ!…