Topic: Real-Time Motion Capturing & Other Wacky Inputs

Alrighty then, I'm back at University after a short break and I'm already in way over my head.

I'm thinking about attempting to use real-time motion capturing as an input device to control a 3D VR of some kind. I feel a little bit like Alice in wonderland playing with this kind of stuff, but it's hell fun.

The idea is similar to how motion capturing is done for 3D animation, but this would be simplified and well, processed in real-time.

Meaning I'll have some monkey in a black jumpsuit with some coloured lights strapped to him and have several video cameras feeding images to a program to analyse the movements of those lights, which is then translated into input for the VR, which will be projected onto several large screens (we don't know exactly how many screens, data projectors and computers we have for this yet but It's probably about three, creating a bit of a VR Faux Cave) that surround said monkey in the jumpsuit.

Sounds like fun yah?

At present, this is just an idea I'm kicking around among several others but my lecturer (or professor, as is the term often used in most Universities) seems to like the idea and really wants me to pay some serious thought to it.

I'll be using director for the capturing, processing and the 3D VR displays. I know it's not the fastest app around but it's one of the most flexible enabling me to keep everything inside the one program, and it's an extremely fast (as in time required to make) environment to develop in.

Things I'm not too sure about is how accurate I'll be able to make this or even if it'll works, but assuming I could, It'd be nice to hear a bunch of suggestions as to what kind of environments I could apply this too, and what body movements I'd tie to controlling the VR.

I've had a bunch of ideas about this myself, but I thought getting some extra outside opinions would help.

Some other ideas I've had instead mainly revolve around alternative input devices like butchering a cheap joystick and strapping a skateboard on top of it. Fixing mice to seatbelt mechanism with handles instead of buckles making some kind of friction pulley input etc...

So, any ideas on what I could do with motion capturing or ideas on wacky input devices that could be made from existing devices (rather cheaply) would be great to hear.

probably a bit out of range of what you'd be able to do but interesting as far as inspiration, check out the jurassic park dvd (the original) and its bonus content. stan winston created something they called the DID (dinosaur input device) for their stop motion animators to use. the idea was that they could pose this dinosaur like skeleton like they'd done with traditional stop-motion armatures then capture the poses for use with the computer animated dinosaurs. also check out the fellowship of the ring dvd bonus content (4-disc set), there's a cool feature in there on how they roughed out a 3d model of the set for the big fight scene in moria (with the cave troll), then applied the motion captured movements to the different digital characters in the scene and created a "virtual" camera for peter jackson to run around with that let him actually block out the scene, killer stuff.

as far as products, take a look at motionbuilder from kaydara. i know MB 4 (current version) has the ability to work with motion capture data and their personal edition of the software is between cheap and free, might be something to check out.

i realize this is all more digital as opposed to the video idea you're working with, might give you some better direction or inspiration tho. keep us updated, this sounds really cool...

chris


KAIROSinteractive

Sorry Fig, I've decided to ditch this for now.

Although your comments were helpful and I did check out the DVD's you recomended. As it turs out we have a profesional motion capture studio here at my University, but none of these suckers are built for realtime computation. They usualy work by flooding a room with Infa Red light, people in black jumpsuits have infared reflectors that get recorded by a special camera then once the captures are finished, they then begin to process the recorded video. Which takes about an hour per 10 minuites of captured motion.

However, the main reason I've decided to ditch this is well... it would take a stupid ammount of work for very little return. Quite simply put, creating a realtime system with any kind of accuracy would require a bugjet higher than my annual wages, even then it wouldn't be that good.

Useable, yes, but I also started to question how practical this kind of work would be. The other deciding factor for me abandoning this project was due to a lack of decent practical applications for it. I mean, I could do it on my budjet and it would work, but it wouldn't really do much due to a lack of accuracy. All of that and the limited development time = a project that is fun for all of about 30 seconds and only ever gets used by a handfull of people.

And that's not what I want.

I've since started rearching web based realtime cooperative multiplayer puzzle/3D games. Not as big on the wiz bang techy side of things, but fun none the less, and a little more feasible (<-sp?) as well.

I might come back to these ideas later on in my studies, but this kind of work takes me a log way away from the web, which is not really the direction I want to head in. Although, with the flash communication server's ability to stream video from a users webcam, I could forsee myself trying to recreate the playstations iToy games with director at some point. Multiuser web cam games could prove hihgly amusing - given the users have the bandwidth to support it.

gotcha drac, and thanks for the info. good luck with the other realtime stuff, be sure to post from time to time so we can check out how its going and offer some feedback

chris


KAIROSinteractive

Actually, I've come back to this already.

I spent a few days playing around with several ideas and have decided to settle for something similar to this but completely different at the same time.

Like I said, my budget and tech restrictions were holding me back from doing any kind of precision motion capturing, but I've been looking into alternative ways of using the same concepts.

If you?re interested, pop over to www.mine-control.com and have a look at their work. They're a bunch of game developers turned interactive artists and well, they do some very interesting work. I'll probably be attempting something similar to their 'Shadow Garden' work, capturing the shadow cast onto the wall from the projection itself. Even though they have more robust hardware than what I'll be using, Charles Forman over at www.setpixel.com has done similar stuff with director and web cams (which is what I'll be using) so I know this kind of work is possible, the only question marks left standing is my programming abilities and how creative I can be using "shape" as an input device for interaction.

It also means that the user doesn't have to 'suit up' in any special gear to interact with the work. Then just need to park their fat arse in front of the projection and the magic starts to happen. Although, this won't be 3D based anymore, it will have 3D visuals and in a way, is a form of VR -- I could argue that it's a backwards take on concepts surrounding augmented reality... maybe...

Dracuis, that wa smind-bending. The Museum of Life and Science in Durham NC, close to me, had a piece that I'm betting was done by mine-control. Walking past the projector made a colorful afterimage that lasted a good 30 seconds before fading away.

I would LOVE to try something like this out. Unfortunately I'm not that experienced with Director. However I know a lot about Flash. How could I even begin to take webcam input and utilize it..? wow.. freaky cool stuff.

Shine and shine. :: [Cell=992] :: [See my team's 30 second animation!! ]

Bmud, Flash MX actually has native support for video inputs; it's not documented in the standard flash MX help files though because it's only really meant to be used with the Flash Communication Server MX. When you think about it, capturing video from a camera input would be a little pointless if you couldn't stream it to other people also connected to site. That and flash has vary few bitmap processing capabilities.

Director is another beast all together and it has several different ways of capturing video through various xtra's (c++ plug-ins that extend the functionality of director) people had written, or through a live QT stream. Although the biggest advantage director has over flash here is its imaging lingo, which lets you alter bitmap images on the pixel level in real-time. Combine these and you have a primitive but workable computer vision system.

If you?re really interested in getting into this, drop me a note via email and I'll send you a bunch of resources.