I would like to show you all my first ever completed Unity project, the 3D Wizard (although I’ve been working on other projects for several months, so I’m not entirely new to Unity).
This program was made for keyboardist Jordan Rudess, of progressive metal band Dream Theater, for use in live shows. This program was originally coded in DarkBASIC for Dream Theater’s 2009 world tour, but when I needed to update it for the upcoming 2011 tour, I decided it would be a good idea to port it over to my current program of choice, Unity (Indie), which ultimately proved easier than I expected, and took about a month in total to make.
The program is used live as a realtime visualization of MIDI data. It takes MIDI in via keyboard, and will mimic the playing of the piano with a 3D wizard character. This video is then output for the video director to use on the large projection screen behind the band at selected points during the show. So basically, the keyboardist plays throughout the show, and the program’s character will play along on the big screen.
All camera angles are automatic, and based on the performance. The program will adjust the camera angles based on what is being played and where on the keyboard it is being played, and will adjust the camera shot length dynamically based on the performance speed.
There are no pre-made animations. All animation is procedurally generated on the fly based on the MIDI input. This required coding some basic Inverse Kinematics (IK) to allow the program to place the hands at the piano keys wherever it needs to. The code for the character is about 3000 lines long, most of which is dedicated to deciding which hand and fingers are used for each note. Notes are assigned to each hand based on the current location of each hand on the keyboard, and assigned to fingers based on current notes pressed, and intervals between notes. The program also responds to velocity, or how hard the notes are being played. The facial expressions, body movements and head movement are all based on the current playing style.
The graphics are using mostly mixing and matching inbuilt shaders. The main wizard character uses the inbuilt lit toon shader, although the outline is pre-made in 3DS Max, and not done via shader, as it could achieve a better quality extrusion. Most of the rest of the scene is using a combination of lightmapping (pre-rendered in 3DS Max, not inside Unity) and cube mapped reflections. The scene is topped off with some smoke particle effects, and lens flares, to add some ambience and contrast to the scene.
The program will be used throughout Dream Theater’s tour, which started in Rome on July 4th. If anyone really wants, I could post some blurry cell phone quality videos of the program in use live, but I think the above video does a better job. 
Here are a couple of screen grabs from the videos, although these don’t show a lot of the details. It looks better in motion. 
I hope some people using Unity for MIDI/musical purposes find some interest in this.
