FACE CAPTURE

[TheToadStool]

Sorry, I'm a skipping record about this (not a broken one), but I am on a mad search for face capture that will work with Animation Master. Animation Master works well for me over here (even on MAC) and so I can afford to sit and think about luxurious ideas such as body and face capture. I have said that I think one of A:M's great things is its ability to use .bvh files. When I discovered this it was a renaissance ! Anyhow, I cannot find a reliable face capture program that can somehow translate to A:M. I have busted my butt to try to get the A:M track 2 to work and although the tracking seems to be there somewhere, the program is too temperamental to be of use to anyone. This is my opinion and I am perpetually a noob, so I challenge anyone out there to tell me that you have actually used this program with some success. My recommendation is that Hash remove this product from the website. It could be great but it's just not there and should not be sold. It weakens the A:M brand to have it on the site, as I believe 3D painter does as well. Again, my opinion. I get that these products are sold to be affordable options for A:M users who are a pretty inventive group but I would rather pay more for something that might even KIND OF work. Actually, I like when an application not only works but is GREAT. As I have said before, I am having a great time making .bvh files with ipisoft which is a great program. A:M is a great program. This is my proper job trying to learn the language of this program so that I can create weird art with it. I am not a tech person who likes to create impossible problems for myself out of boredom.

 

The point of this post is that I am asking the community if anyone has a recommendation how to do face capture and apply it to A:M. I have seen dozens of apps for this that are all over the map, for video game design or big brother surveillance, or whatever. Faceshift looks great but I think Apple took it off the market. Anyone have experience with this?

 

 

Adam

 

 

 

[detbear]

Adam.

 

I have done massive research in this area and I recall that Faceshift was cool. Also you will run into a number of others like the Brekel face pro....

 

BUT LET ME HELP YOU and SAVE you much time:

 

Currently there isn't much to choose from with A:M. Which is unfortunate because Hash has the best skinning and morphing system in the industry.

 

Any face mo cap(Even Face Shift) is only going to be as good as the face rig on your model. The data from face shift ultimately has to be used on a really good rigg in order to get the desired results.

 

A:M track needs to be improved for sure and I'm not sure that this will happen. Luuke....it's creator moved to other software with it.......but again....any Mocap you capture needs to have a very good face rig. The showcase samples they use to display were on very complexed riggs.

 

I wish Luuke could give A:M track an update, but in his defense, it is not very easy. And expensive time wise to produce them.

 

The gitter/ shaking exists in all mocap systems. This is why much time is spent in Mocap Clean up. I've had jobs in the the past where all I did was clean up Mo-cap for companies after they captured it.

[TheToadStool]

I do appreciate your warnings about the process of rigging a face capture. It got me to thinking. First of all, I wish to apologize profusely to Luuk and eveyone whom may have come in contact with my negative rantings about this. I have been trying to make a short film that is heavy with dialogue and I realized that by doing face capture, I could get a smoother product and could perhaps get done a lot quicker. So, I tried A:M track 2 which I want to say, is exactly what we NEED as users of Animation Master. This program should be developed and made great. Simple as that. First of all, if you can manage to get a proper capture, which I HAVE managed to do on one occasion, the rest of the process is PERFECT for animators using Animation Master. A:M Track 2 allows you to export a standard rig for whatever face template you choose as a .mdl file which you can use for the character model you are creating (in this case I chose Standard 2, a more complex rig). Additionally, you can export your capture as an .act which you drag right on your rig! The rigging for this process is basically self-explanatory and that part is a bit of trial-and-error to get the rig to behave the way you want it. Even greater news with this program is that it smooths out the capture and it does error correction. When I do manual animation, by using Preston mouth poses, I get a bit over descriptive with the lip syncing. I want it to look realistic, somehow. I know some people don't really care to make realism or naturalism, I guess, and are content with a more basic interpretation like what can be achieved by merely using the dope sheet process with perhaps a little tweaking. Anyhow, I believe that A:M Track could be a really powerful tool and it could be marketed directly at exactly the people that are on this forum. You need like a web cam and some lights, some of those green pom-poms, and theater glue. Oh, and a white sheet. Not a big investment. Another thing is that this program is an Animation Master specific rendition of Zign Track which allows a probably better track experience with multiple camera potential, etc. A:M Track 2 only allows for one camera (which is great because I only have one!) This version does those specific exports which couldn't be better for the likes of us. I just wanted to get a straight ahead face capture and allow other processes to dictate the movement of the neck and body, so this would be perfect for that. I want this to work so badly and I think it should be developed and made better. Animation Master would be a more attractive product with a handy face capture plug-in. Seriously !!!! This is the basis of my ranting and I just wanted to take a second to talk about how great this could indeed be. My purpose is to invite actors over to my studio and engage them in this process and sell them on how expressive and creative this process is. As it is right now, I find this program to be way to temperamental. I wouldn't want anyone to experience this because it was crazy-makin' to try to get it to work consistently. I wish I had learned enough about coding to tackle these issues myself.

 

Adam

 

[ruscular]

Sorry to hear that AM track 2 isn't up to par for your needs. I have look at a lot of face capture program. I even got the Kinect V2 camera, and Lightwave with Nevronmotion. The developer left and didn't finish developing it for the newer improve kinect V2 so no face capture support for that camera, and no smoothing function for that camera as well. I thought maybe Brekel body and face pro 2 would be the way to go.
I am sorry to hear about AMtrack 2 because I thought maybe I could create a face mesh in AM to be driven by Marker tracking face motion capture and turn it into a Cage mesh to drive the polygon face in Lightwave3D on the model. The cage mesh is basically a mesh that has a weight transfer of the original mesh and then use the cage to drive the model animation instead of bone. Hash animation master has a way to transfer weight from one model to another. Seems like with a creative coder could make something like that for AM: plugin. The other thing That I have seen in other program is have a USB gamepad as a input to control animation with the up or down button to control smile and sad expression. Or even a mouse wheel or tablets scroll function as a feature to drive expression.

One other option, Moho studio now lets you throw a vector mesh over a photo of a face and then use the video of your marker facial motion caption and target each bone to them. You would render the face from your model and set it up for a 2D animation, but that would have limitation of head turns, you could still turn the head but slightly. Animate your character in Hash without a head and a green dot on the neck, and then bring it in AdobeAE or Moho and track the PNG render of the face to the green dot on your animation of your character. But again with limitation of head turn and body turn.

Another option is animate the facial color map, and then use that animation to be a animated decal on the face of the character. I would also use the same motion capture to animate the bump map and displacement map. both Moho and AdobeAE have tracking capability.

[TheToadStool]

Well I have to admit that my reaction to Am Track 2 was emotional. I had spent a few days trying to get it right and came up empty-handed. I think that there is something there with it, though because we did get to observe it tracking well one time so I know it can be done. It crashes a lot but I think there's something good there in the code like the smoothing seems good and the rig export. There's a different thread going on here about face capture that I was/am involved in regarding Brekel. I have found the success that I was looking for with Brekel. It exports an odd bvh that has the potential to drive pre-made poses in A:M. This does what AM Track does but is way more fun and you can also record the audio and video simultaneously which is good for syncing and reference later. A:M Track does not record audio or video which will make syncing pure guess work later. That Nevronmotion thing did indeed look very awesome, though.