Virtual puppetry

Community Showcases
1

Hi all,

We are developing a “serious” game for children between 8 and 10, in which they will learn about the world of puppetry. Using the game and its custom-built controller, they will learn how to do basic movements with a marionette, while solving very simple puzzles that teach them about material properties of the puppets. The level itself progresses through the history of a certain type of puppet, from traditional art to modern, more absurd applications of the art form. There are three types of puppets (a wooden puppet, a metal puppet and a puppet made of fabric), but the children can use a puppet editor (separated from the game itself) to customize their puppets (adding accessories to them for example).

In this Belgian research project (GameHUB), we are developing the first level as a demonstrator for the concept. The level involves the world of “rod puppets”, going from the traditional Wayang Golek puppets to the modern puppets of Handpring Puppet Company.

We are only just producing the actual scenery and puzzles for the game, but I wanted to show our current results anyway, so we can maybe get some critique on the general idea. The innovative about this game is that we are using purely physically based character animation for the player-controller marionette. So no animations on that character whatsoever.

Normally, the game is played using a custom-built controller that uses the same wooden “cross” that real puppeteers use. In stead of attaching the strings of the wooden cross to a puppet, we use bend and distance sensors and an Arduino board to capture the movement of the controller. We changed the USB stack of the Arduino board to act as a Joystick HID device, so we can use the normal Unity input system to capture the input of the sensors. For demonstration purposes however, I implemented a keyboard controller to control the puppet without the custom controller.

Here are some screenshots of the current state of the level. Note that we are still tweaking this very much, especially in terms of lighting, but all comments are more than welcome.

A picture of the controller:

Here are some links to the playable versions:

I also created a very simple demo to show all three puppets at once. Walking is disabled, but the keyboard controls are the same as in the other demonstrator.

The keyboard controls:

  • Left/right arrow keys: Pull left/right leg up
  • Left shift + Left/right arrow keys: Pull left/right arm up
  • Down arrow: Squat
  • Up arrow: jump
  • Up+Down arrows: bend over
  • Escape: Restart level

You can also drag the puppet and other physical objects around with the left mouse button.

I feel that I wrote too much text already, so if anyone wants more information about something, just post here. And again, all comments are very welcome. We have a thick skin :wink:

Greetings,
JeDi

2

looks awesome :wink:

3

amazing!

4

Just one note: the first puppet that you can select is kinda broken. It was a prototype puppet that isn’t updated in a long time. Today, I almost finished the third puppet, made of fabric. I will update the demo probably tomorrow with the prototype puppet removed, and the fabric puppet added.

5

We added the final puppet, and it has a very funny walk :slight_smile:

Here are the new links for the demonstrator:

This is the concept for the three puppets that are now in the game. Note that we are still working on the accessory system, so the puppets in the game are “nude”.

Greetings,
JeDi

P.S. We are still having an annoying problem with substances and version control. If someone has an answer to this thread, we would be very grateful!

6

I edited the first post with links to the latest builds, which have a lot of tweaks but still the same part of the level. I removed the prototype puppet because it was broken anyway.

We host a symposium tomorrow about this research project, and after that we will finish the new part of the level so the demonstrator will become longer. In the new part of the level, we will move from the era of classical Wayang Golek to the style of Richard Teschner (a bit less Asian and a bit more renaissance type of style).

I find it strange that almost nobody seems to comment on this thread. Has nobody tried this yet, or doesn’t it work? We designed this with a modern PC in mind, so we didn’t really optimise for performance (yet). Does it run smoothly on your computers? What do you think of the art style? Again, we are happy with any constructive criticism because we are still beginners ourselves.

Greetings,
JeDi

7

Great job and great idea ! Greetings from a compatriot :wink:

8

Nothing has ever made me want to learn to be a puppeteer before. Congrats on at least having and awesome concept! I don’t want to play it without the controller, though. :frowning:

9

Thanks for the comments. The keyboard controls actually work fairly well. Especially the fabric puppet is a lot of fun to play with. Don’t forget to press and hold the down key from time to time, so he falls down on the ground.

Edit:
I also created a very simple demo with all three puppets. Walking is disabled, but the keyboard controls are the same as in the other demonstrator.

10

the project is finished :frowning: but we had a great boost in the end and got two more levels done (the last one is not complete unfortunately).
don’t know if JeDi updated to our “final” build?
If we find spare time I’m sure we’ll also finish level 3 :wink:
anyway it was a nice project to work on. hope more like that come along in the future.

11

Yes, the research project (GameHUB) is finished. We had a blast working on it, and gained a lot of experience doing so.

I will update the first post with more info as soon as I find the time, but you can find the latest builds here:

The zip files contain a small readme file with the keyboard controls in it (normally, the game is played using a custom controller based on the controls of a real marionette).

Mind that the game is a prototype, not a finished project. We used it to do research about completely physically based character animation, and I will surely use that technique (minus the marionette strings) for a game in the future.

You can find the two other prototypes created during this research project at http://gamehub.be/games.

  • Unizo Limburg is a game insprired by Theme Hospital, where you run an ice cream salon. The game is used to teach (dutch speaking) students how to start and manage a business
  • New Belgica is a game insprired by the Monkey Island series, where you play through a parody of the real journey to the Southpole during a belgian expedition at the end of the 19th century.

Note that all text in the games had to be in Dutch, so it can be a bit hard to follow for people using other languages :wink:

Of course, any feedback is appreciated!

Greetings,
JeDi

12

and some more screen goedies




13

it is very nice and interesting work

14

At the moment I’m working on a similar project which uses my handmovement as input for the puppets. Could you explain how you achieved the ‘string-pulling’ effect? (Meaning: the way the string attaches to the joint and can move up and down, pulling the limbs up or down)

15

1222860–50677–$Rope.cs (3.21 KB)

In attachment you can find the Rope script we used. I think the code is fairly clean, readable and commented, but in short:

We use a “maximum distance constraint” using a ConfigurableJoint to simulate simplified behaviour of a non-elastic rope. We don’t do actual rope physics: the rope is always straight and objects don’t collide with the rope. It was enough for our simulation, and very stable.

Getting the simulation to behave as we intended involved some tricks including positioning the rope’s end at the attachment point, then create the constraint, and then move it back into its original position. There are some comments in the script explaining it a bit.

With our Rope script, we can attach a rope to any rigid body, optionally at a certain attachment point, and move the upper end of the rope (the transform of the rope object, which contains a kinematic rigid body) using our controls as input.

If you need more information, just ask! By the way, your project sounds very interesting. Could you elaborate a bit about the techniques you will be using (kinect, physically based animations, Mecanim, …), the genre of simulation (game, pure simulation, …)?

Greetings,
JeDi

16

I’m doing an internship in Finland with a friend of mine (We’re students from Hasselt by the way :wink: ). It’s at an software engineering lab at the RAMK university in Rovaniemi called pLAB. We were asked to experiment with a few different technologies which we ultimately have to combine inside Unity.
We’re using Kinect to capture full body movement and then we received this neat little piece of hardware called “Cyber Glove.”
It’s basically a glove with sensors, which captures the angles between fingers and joints.

We were asked to invent something with this applying only one rule: be creative! My friend and I decided to build a little puppeteer game. The current version has got the Cyber Glove implemented, which moves a model of my hand. I almost got a puppet/ragdoll attached to it.
Not sure yet which genre of game we’re going to create yet, as we’re still experimenting (and learning how to work with the different tech.)

Thanks for the Rope script! I’m going to start testing if it suits our project needs. I’ll let you know if it works.

Cheers,
Steven

17

The script worked like a charm! :smile:
Thank you so much!

18

Hello Jeroen,

Congrats on the project. I am completely new to unity. And I would like to explore your project in detail. Is there anywhere the assets/scene or the project is available for learning??

19

This is a great project.

20

Great idea!
I am working on something similar with puppets and strings. The rope script is very useful. Thanks. :slight_smile: