TKC-Community

Hacking and Art => Armed Assault 3 => Topic started by: M. O. on July 22, 2013, 07:18:44 pm

Title: Face tracking in ArmA3
Post by: M. O. on July 22, 2013, 07:18:44 pm
Basically, ArmA is a simulator that does justice to terrain, graphics, environment and so on. Physics, interiors, and stances are being improved with the third installment of the game. Increasingly, it feels like the lack of sensory presence/output-input is becoming a bottleneck for improving the game experience.

Full motion tracking would of course be ideal, but I guess there are some major practical problems there.

Face tracking is a nice step on the way though. It decouples your in-game arms (aim) from your head movements, i.e. with it you can move both independently and simultaneously. It's quite seamless in this respect.

Here's an example:
http://www.youtube.com/watch?v=9wXx3vMy_AQ

I made some research into the subject, here's a summary of what I found out.


Currently there are three alternatives.

1. TrackIR:
Ludicrously overpriced.

2. FreeTrack:
Cheap. You have to design your own IR-light source, and remove the IR-filter from your webcam.

3. FaceTrackNoIR:
Only a webcam required.
Requires good lighting conditions.


Some image processing is required, which means CPU resources will be consumed. In addition face recognition should be more computationally expensive than checking for IR point lights.

I tried the third option out first, as I had no webcam to use for the 2nd option anyhow. I settled for a PS3Eye which in theory is capable of 120fps at 320x240 pixels, or 60fps at 640x480. It comes with a 4-array microphone and two FoV modes. The more fps the faster and smoother movements ingame. Most webcams only support 30fps.

With this camera, in practice, facetracknoir only supports about 60fps at one fixed resolution 640x480. Sixty fps is good enough, but the CPU usage isn't. I get about 10-15%, which is far too much in a CPU-intense game like ArmA3, the fps drop is noticeable. The CPU usage is related to fps*res, so naturally you would like to lower the resolution, but you can't. Another problem, with this camera, is that the microphone volume can't be adjusted. The microphone is superb, if you hold it close to your mouth, but that defeats the purpose of the rest of the camera functionality.



FreeTrack lets you choose camera parameters it seems, which would mean that quality vs performance can be adjusted at your convenience.
http://www.youtube.com/watch?v=aaMWPqyvIkE

You need to remove the filter:
http://codelaboratories.com/research/view/ps3-eye-disassembly
http://nuigroup.com/forums/viewthread/4189/




So far, I found the face tracking to be very useful while driving (eyes on the road makes it easier to follow it, you can make use of the rear mirrors naturally and check for traffic from the left or right as you would in real life) or flying (checking the surroundings while being able to use the mouse for vehicle control). In infantry combat the view wasn't steady enough most of the time, even if you kept your head still, due to uneven lighting on the face or otherwise low light conditions. Ideally you should face a diffuse light right in front of you. The detection does stabilize in sunny conditions, but would be a problem at night. The in-game FPS was not good enough either to make the movements feel fluid at all times (mission dependent). I can imagine that it would help a lot in close combat such as in urban areas though.

I haven't tested the translational (xyz) part of the tracking yet and can't confirm leaning capability. It is possible to bind custom actions to camera movements, so in general it could be possible to change your stance by moving your head up or down as well. Perhaps, and most importantly, it would work to bind the new ArmA3 ctrl-q (i.e. +1 stance) to a high (Y-axis) head position, and ctrl-z (i.e. -1 stance) to a low head position, so that you can peak up/duck behind stones and such.



I might settle for option 2 later on. There are two alternatives there. An active or a reflective head mount. With the reflective you reflect light from IR-diode illumination and with the active you keep IR-diodes on your headset. I would prefer the active one, as you get exposed to less light that way.

In my opinion TrackIR doesn't warrant a 10-fold higher price tag than the "free option".