KinectToPin’s application was built in Processing and tracks 15 skeletal joints in 3D or 2D using either SimpleOpenNI or OSCeleton.
But you should use SimpleOpenNI. It works better.
Drop an audio file (wav/mp3/aff) into the application’s “dialogue” folder to play it back during capture. The XML file will note the audio clip’s filename, which can help identify the track data at a later point.
Remote control via presentation remote:
The KinectToPin application can be controlled with a standard presentation remote (like this one). The button mapped to Page Down toggles Cam mode and the button mapped to Page Up toggles Rec mode.
Application settings can be controlled by editing the settings.txt file:
- Mirror Joints True/false, defaults to false. Flip the right side of the skeleton between “screen right” and “stage right”
- Export AE Puppet Pin Data (.txt) True/false, defaults to true. Exports track on save as text file containing copy-pasteable After Effects Puppet Pin keyframe data.
- Export AE Point Control Data (.txt) True/false, defaults to true. Exports track on save as text file containing After Effects Point Control data (2D).
- Export AE Point Control 3D Data (.txt) True/false, defaults to true. Exports track on save as text file containing After Effects 3D Point Control data.
- Export JSON Data (.json) True/false, defaults to true. Exports track on save as JSON.
- Export AE Template Script (.jsx) True/false, defaults to false. Exports track as JSX file that creates 2D template with mocap data as 3D Point Controls
- Export Maya Script (.py) True/false, defaults to false. Experimental feature exports data as Maya script to import 3D position data — not quaternions.
- Original Capture Image Width Default is 640. Scene width for capture.
- Original Capture Image Height Default is 480. Scene height for capture.
- Original Capture Depth Default is 200. Scene depth for capture.
- Destination AE Comp Width Default is 1920. Destination comp width.
- Destination AE Comp Height Default is 1080. Destination comp height.
- Framerate (max 30) Default is 24.
- Camera Display Quality (1 = best) 1 or 2, defaults to 1. Level 1 uses full-resolution camera in Cam mode, Level 2 uses low-resolution camera.
- Enable Multithreading True/false, defaults to true. Enables multithreading.
- Load SimpleOpenNI at Startup True/false, defaults true. If true, launch SimpleOpenNI on program launch. If false, launch SimpleOpenNI when Cam or Rec mode starts.
- OSC IP Number Default is 127.0.0.1. For capture via OSCeleton.
- OSC Port Default is 7110. For capture via OSCeleton.
- Delay Between Saving Files Default is 100 (milliseconds). Increase this value if your track exports have missing data.
UI Panel for After Effects
The UI Panel enables you to create two kinds of character templates:
2D standard template (CS3+) creates a 1920×1080 24fps comp with a layer with a series of Point Controls to hold the mocap data (with smoothing expressions already applied), and a set of reference nulls to connect your character to.
3D template (CS5.5+ only) also creates a 1920×1080 24fps comp with a source layer and control nulls, but uses 3D Point Controls for mocap data and adds a camera. This method essentially composites 2D puppet layers relative to the position of a 3D camera.
XML import (2D or 3D) allows you to import your XML tracking data without converting. It’s best for shorter tracks. Longer captures should use the traditional “export + copy/paste keyframes” method.
Automatic rig for Puppet mesh layers. Pins need to be renamed to match the control nulls they refer to. (“r_shoulder”, “l_hip”, “torso” etc.)
Automatic position and rotation for head, hands, and feet. Layers should be renamed to match relevant control nulls, but with “_layer” at the end. (“head_layer”, “r_hand_layer” etc.)