Santanu Pal

Creative Director

Subscribe
Add to Technorati Favourites
Add to del.icio.us

Facial Action Coding System [FACS]

Day by day VFX challenge become tough to crate exact and believable facial movement in VFX field. We are using various process to replicate the exact movement on Computer Generated (CG) character. Among these processes we forget to know something very important about the facial movement and expression which is a complex combination of several number of muscular movement of facial muscles.

Worldwide many studios and VFX technical directors prefer the most expensive techniques called FACS (Facial Action Coding System).


In 1999 we have seen a new technology to replicate real facial movement to CG character at Siggraph Electronic Theatre, and it was named as LifeFX technology (Above). This was the milestone of Computer generated human.

 

In the year of 2000 this technology has been used first time in the feature film named "Young At Heart" (released in 2007).

Courtesy: wata digital (Avatar 2009)
This technology branded and marketed by MOVA, which was born in the year of 2004 and used its first skill to the Curious Case of Benjamin Button in the year of 2008. Simultaneously Sony Pictures Imageworks creates film like Spider man, Monster House by using the same technology in the year of 2004. Weta Digital has started King Kong (released 2005) and Avatar (released 2009) by use of same technology.Though the same technology has been used in Lord of the ring in 2001 by manual way. Though you can get almost similar use of FACS technology in Final Fantasy, Beowulf, Incredible Hulk etc.
In 2005, FACS was implemented at Weta Digital for King Kong and was able to capture extremely subtle to highly dramatic expression and faithfully translate Andy Serkis’s performance to King Kong’s face.
Dr. Mark Sagar
For Avatar, Weta knew that James Cameron wanted to use helmet cameras for motion capture, so Mark did an initial proof of concept test to see if they could use a single video camera and 2D image based tracking for facial motion capture rather than 3D tracking. They used the FACS system to solve and map the performance to Gollum for the test.

The FACS system was used for all the Navi character performance capture in Avatar. The Real-time Facial Motion Capture framework uses video from a small, helmet-mounted camera to record the actor’s facial performance in real time. New custom facial tracking software based on FACS rapidly tracks the movements of the face, and maps them onto a rigged model that replicates the actor’s expressions onto a facial puppet in the virtual monitor used by the director on the virtual stage.
The facial expression solver is a real-time version of the system developed for King Kong. This new system takes a live stream from the tracking workstation and determines which muscle groups or FACS units are being used to make the actor’s expression. The system automatically maps these FACS poses onto a puppet’s face. By using the standardised ‘language’ of the FACS poses, the system could drive a rig which could make 10,000 different expressions and included the detail needed to capture the subtle eye and mouth movement necessary to bring the characters to life. Also, it also enabled faithful representation of the original actor’s expression on a face with quite different proportions (for example, the Na’vi), enabling the actor to drive the virtual puppet with their performance.

Whatever, I'm more interested to disclose the process instead of its patent or high end capturing and automation system. Before that I'd like to thanks the TDs, Supervisor and Bio-technicians and Scientist, whose who had tried a lot to achieve Computer Generated character into real life.

_____________________________________________________________

_____________________________________________________________ 

Facial Action Coding System (FACS) is a system to taxonomy (process of classification or science of classification) human facial expressions. This process was developed by Paul Ekman and Wallace V. Friesen in 1978. It is a common standard to systematically categorize the physical expression of emotions, It has proven useful to psychologists and to animators and present VFX world.

Another terminology coming in relation with FACS. These are Action Units (AUs) and Action Descriptors (ADs). To decode almost all possibility of facial muscle movement during expression we need FACS, which deconstructing it into the specific Action Units (AU) and their temporal segments that produced the expression.
AUs are independent of any interpretation, they can be used for any higher order decision making process including recognition of basic emotions, or pre-programmed commands for an ambient intelligent environment. To know more about this AU and AD, you may look for "Dr. Ekman’s interpretation". He had describe about the expression and its meaning. These are very important to understand the agony f patient, those who are unable to express themselves verbally.

FACS defines AUs, which are a contraction or relaxation of one or more muscles. It also defines a number of Action Descriptors, which differ from AUs.

Action Units (AUs) are the fundamental actions of individual muscles or groups of muscles.
Action Descriptors (ADs) are unitary movements that may involve the actions of several muscle groups

Make it in simple way:
For example, FACS can be used to distinguish two types of smiles as follows:
 Although the labeling of expressions currently requires trained experts, researchers have had some success in using computers to automatically identify FACS codes, and thus quickly identify emotions. Computer Graphical (CG) face models, such as CANDIDE or Artnatomy, allow expressions to be artificially posed by setting the desired action units.

Lateral View of muscle of face
Lateral View of muscle of neck





_____________________________________________________________

_____________________________________________________________

List of Action Units and Action Descriptors (with underlying facial muscles)

 

AU Number FACS Name Muscular Basis
0 Neutral Face
1 Inner Brow Raiser Frontalis (pars medialis)
2 Outer Brow Raiser Frontalis (pars lateralis)
4 Brow Lowerer Depressor glabellae, Depressor supercilii, Corrugator supercilii
5 Upper Lid Raiser Levator palpebrae superioris
6 Cheek Raiser Orbicularis oculi (pars orbitalis)
7 Lid Tightener Orbicularis oculi (pars palpebralis)
9 Nose Wrinkler Levator labii superioris alaeque nasi
10 Upper Lip Raiser Levator labii superioris, caput infraorbitalis
11 Nasolabial Deepener Zygomaticus minor
12 Lip Corner Puller Zygomaticus major
13 Sharp Lip Puller Levator anguli oris (also known as Caninus)
14 Dimpler Buccinator
15 Lip Corner Depressor Depressor anguli oris (also known as Triangularis)
16 Lower Lip Depressor Depressor labii inferioris
17 Chin Raiser Mentalis
18 Lip Pucker Incisivii labii superioris and Incisivii labii inferioris
19 Tongue Show
20 Lip Stretcher Risorius w/ platysma
21 Neck Tightener Platysma
22 Lip Funneler Orbicularis oris
23 Lip Tightener Orbicularis oris
24 Lip Pressor Orbicularis oris
25 Lips Part Depressor labii inferioris or relaxation of Mentalis, or Orbicularis oris
26 Jaw Drop Masseter, relaxed Temporalis and internal pterygoid
27 Mouth Stretch Pterygoids, Digastric
28 Lip Suck Orbicularis oris
29 Jaw Thrust
30 Jaw Sideways
31 Jaw Clencher Masseter
32 [Lip] Bite
33 [Cheek] Blow
34 [Cheek] Puff
35 [Cheek] Suck
36 [Tongue] Bulge
37 Lip Wipe
38 Nostril Dilator
39 Nostril Compressor
41 Glabella Lowerer Separate Strand of AU 4
42 Inner Eyebrow Lowerer Separate Strand of AU 4
43 Eyes Closed Relaxation of Levator palpebrae superioris; Orbicularis oculi (pars palpebralis)
44 Eyebrow Gatherer Separate Strand of AU 4
45 Blink Relaxation of Levator palpebrae superioris; Orbicularis oculi (pars palpebralis)
46 Wink Relaxation of Levator palpebrae superioris; Orbicularis oculi (pars palpebralis)
_____________________________________________________________

_____________________________________________________________

Head Movement Code

 

AU Number FACS Name Action
51 Head Turn Left
52 Head Turn Right
53 Head Up
54 Head Down
55 Head Tilt Left
M55 Head Tilt Left The onset of the symmetrical 14 is immediately preceded or accompanied by a head tilt to the left.
56 Head Tilt Right
M56 Head Tilt Right The onset of the symmetrical 14 is immediately preceded or accompanied by a head tilt to the right.
57 Head Forward
M57 Head Thrust Forward The onset of 17+24 is immediately preceded, accompanied, or followed by a head thrust forward.
58 Head Back
M59 Head Shake Up and Down The onset of 17+24 is immediately preceded, accompanied, or followed an up-down head shake (nod).
M60 Head Shake Side to Side The onset of 17+24 is immediately preceded, accompanied, or followed by a side to side head shake.
M83 Head Upward and to the Side The onset of the symmetrical 14 is immediately preceded or accompanied by a movement of the head, upward and turned and/or tilted to either the left or right.
_____________________________________________________________

_____________________________________________________________

Eye Movement Codes

 

AU Number FACS Name Action
61 Eyes Turn Left
M61 Eyes Left The onset of the symmetrical 14 is immediately preceded or accompanied by eye movement to the left.
62 Eyes Turn Right
M62 Eyes Right The onset of the symmetrical 14 is immediately preceded or accompanied by eye movement to the right.
63 Eyes Up
65 Eyes Down
65 Walleye
66 Cross-eye
M68 Upward Rolling of Eyes The onset of the symmetrical 14 is immediately preceded or accompanied by an upward rolling of the eyes.
69 Eyes Positioned to Look at Other Person The 4, 5, or 7, alone or in combination, occurs while the eye position is fixed on the other person in the conversation.
M69 Head and/or Eyes Look at Other Person The onset of the symmetrical 14 or AUs 4, 5, and 7, alone or in combination, is immediately preceded or accompanied by a movement of the eyes or of the head and eyes to look at the other person in the conversation.

_____________________________________________________________

_____________________________________________________________

Visibility Codes

 

AU Number FACS Name
70 Brows and forehead not visible
71 Eyes not visible
72 Lower face not visible
73 Entire face not visible
74 Unscorable
_____________________________________________________________

_____________________________________________________________

Gross Behavior Codes

 

These codes are reserved for recording information about gross behaviors that may be relevant to the facial actions that are scored.
AU Number FACS Name
40 Sniff
50 Speech
80 Swallow
81 Chewing
82 Shoulder shrug
84 Head shake back and forth
85 Head nod up and down
91 Flash
92 Partial flash
97 Shiver/Tremble
98 Fast up-down look
_____________________________________________________________

_____________________________________________________________

Facial expressions

 

Some common examples of feelings that can be expressed by characters are:
_____________________________________________________________

_____________________________________________________________

The muscles of facial expression for human (Homo sapiens)

 

See also: facial muscles.
_____________________________________________________________


Kindly mail me your comments at santanu.pal.zoo@gmail.com
or
visit http://www.santanupal.com/contact.html

    TOP