We’re already far in to February and it’s about time we reveal which two works we picked as winners for January. As we expected, the competition started of slow and we had to wait until the first week of February to have two proper entries. These came from Italian artist and teacher Michele Cremaschi and a group of researchers from the Humanoid Robotics Group at TU Bergakademie Freiberg.
Workflow and tutorial: Augmented Pinocchio
Augmented Pinocchio – promo from michele cremaschi on Vimeo.
Michele explains how Augmented Pinocchio was created: “I produced a realtime hologram show where all the actors are hologram reproductions of my live acting; and in which all the props are holographic visuals generated in realtime relative to my body position. To generate the silhouette, I first used NI mate’s syphon one feed. In a second step, in order to have better resolution, I wrote a software that calibrates the Kinect with an external camera. This gives me a hires silhouette based on the depth mask syphon stream coming from NI mate.”
“Another point you might find interesting is how I networked the data coming from nimate; OSC runs easily on a network, but I also shared syphon streams thanks to Airparrot + airserver on another mac. This allowed NI mate streams to be used on a secondary computer, splitting the required CPU power (NI mate is nice, but quite CPU hungry) over two Macs.” Watch Michele’s tutorial and go to his blog to download the patch he created for extracting the skeleton data from NI mate in Vuo:
Vuo and NiMate skeleton patch from michele cremaschi on Vimeo.
Workflow: Imitation learning for NAO robot using NI mate
Who doesn’t love robots – as hardware geeks we definitely do. In this super cool entry from researchers at the Humanoid Robotics Group at TU Bergakademie Freiberg, one a NAO robot is taught to imitate movements captured from real people. Oliver Mothes from the research group explains:
“We tried to use NI mate for human robot imitation, especially for motion tracking. The NI mate OSC interface help us to send the joint positions to our Blender-Robot-Skeletal. With a little bit magic, we transform these positions to robot joint angles, optimise them for stabilising the motion and send the new angles to the real robot.”
Thanks for entering and congratulations to the people behind these entries. We keep being amazed and super happy in seeing NI mate being used in such varied and creative ways. For the rest, there are still almost two weeks to post your entries and have a chance to win licenses of NI mate – as well as a full Oculus Rift devkit! Have fun!
Pow! We’re heading to the new year with a blast! First off, in order to celebrate NI mate’s second birthday (the software was released to open beta in January 2012) in style, we’re launching a competition and giving away two full licenses of NI mate each month until the end of March (total of six full licenses!). These licenses will be dealt between people who’ve posted a new NI mate related tutorial or workflow video online publicly to either Youtube or Vimeo.
We’re not favouring any particular software or workflow – your project can be about anything, but has to feature NI mate in some important role. If you’d like to make a tutorial video that makes use of features that are not available in the trial version or if you’d just want to not be bothered with the nag screen while you’re making your video, you can request a temporary license file from us in order to prepare the video. If you’d like to get a clue about how we usually form our tutorials, you can check our Youtube channel.
The kicker? Not only are we giving away licenses of NI mate, but the creator of the best video – chosen by us and the community at the end of March – will receive a full Oculus Rift devkit worth $350! If you’re unfamiliar with the Oculus Rift or what you do with it and NI mate, you should check out this video from our recent talk from the Blender Conference in Amsterdam, The Netherlands:
Apart from the competition, we’re also announcing the rewriting of the NI mate plugin for Autodesk Maya. In order to make the plugin the absolute best it can be, we’re asking for your help. Get in touch with us and tell us how you’d like the plugin to work.
The best way to get in touch with us is to either write our support email or leave us direct feedback through the webpages or the NI mate forum. Help us celebrate NI mate’s 2nd birthday by helping us make the software better than ever!
“I approve of this competition!” Big Buck Bunny (c) 2008 Blender Foundation
Again we’re pleased to bring you a new and much improved version of NI mate – your real-time motion capture software. This release fixes many of the unfortunate stability issues found in v1.18, but it also introduces something we’ve hinted at for quite a while: the capability to record and playback .oni depth clips. This powerful new functionality (and file format) enables the real-time capture and saving of a depth map in good quality.
The ability to playback recorded material also enables you to make use of NI mate and motion capture files even when you don’t have your sensor with you or attached to the computer. Last but not least .oni files can be played back in our other software, Z Vector, through which they open up what we think is a revolutionary new way to make film and real-time, sound synchronized visuals.
Depth clip recording
Depth clip (.oni file format) recording from the connected live sensor. Without a full licence the recording time is limited to 10 s. Recording can be started from the Live view tab or from the “Depth Clip” NI mate menu. The recorded depth clips will be stored in the “user home/NI mate” folder by default, but this folder can be changed in the NI mate preferences.
Depth clip playback
Playback of depth clips (.oni file format) with full support for all NI mate tracking capabilities. You can open a depth clip by simply double clicking it, from the new “Depth Clip” NI mate menu or from the Open depth clip button in the Live view tab.
All recorded clips in the folder specified in the NI mate preferences (“user home/NI mate” by default) will be listed in the “Depth clip” menu.
You can even play back depth clips without a sensor connected, but in this case the trial restrictions automatically apply as a licensed sensor is not connected.
OSC messages to Processing from outside sources
NI mate’s automatic processing integration uses OSC to make the data available to the processing sketch, but this meant that it was very difficult to receive any OSC messages from other sources. Now this is taken into account and you can use OSC in a NI mate processing sketch just as if NI mate wasn’t even there.
Triggers, OSC and MIDI controller now work properly with high speed mode.
Fix for crashes and instable operation caused by changing the OSC settings.
OSC Controller left & right hand coordinates weren’t sent correctly.
More detailed MIDI error reporting.
Processing export didn’t always replace an existin folder properly.
Syphon output feeds were mixed up.
We’re pleased to announce that the official NI mate Cinema 4D plugin v2.0 has now been updated to include a fix for C4D v13 that prevented correct recording of motion data. Based on the positive feedback we’ve received from users we decided that with this fix the plugin now also exits the beta status.
Please note that due to a bug in the early C4D v13 plugin api you should update to at least v13.051 to ensure correct plugin functionality.
We’re really excited to announce the release of NI mate v1.18 from the annual Blender Conference in Amsterdam, The Netherlands. The release was done in conjunction with our talk, aptly named “Kinetic Bunny, meet the Virtual Bunny” – in reference to our original “Kinetic Bunny” video. You’re invited to watch the recording of the talk below. The video includes a fun moment when Ton Roosendaal, head of the Blender Foundation, picks up the Oculus and turns in to Big Buck Bunny.
Oculus Rift support:
If an Oculus Rift is connected to the computer when NI mate is started the full skeleton head orientation is automatically replaced with the orientation data from the Oculus Rift. If the Oculus Rift wasn’t aligned correctly when NI mate was started you can reset the orientation by clicking on the Reset Oculus Rift orientation button in the Skeleton paths subtab.
Dual feed view NI
Selecting a second live feed now shows the second feed next to the first live feed in the Live view tab.
OSC Triggers NI
Triggers now support OSC messages with the format “/layername/X”, where the X is the trigger number.
This means that the triggers are now also available in the Processing tab as predefined variables.
In addition to this if the layername contains spaces the first word is taken as the name and the rest is sent as an OSC string with an appended trigger number _X. This makes it possible to for example toggle Animata layer visibility by setting the layername to
/layeralpha [Animata layer]
as long as the Animata layer name ends with an underscore and a number (_X).
New OSC command NI to set the live view alpha mode:
Active user numbering options NI
The order in which the active users are numbered can now be changed in the User tab. The available modes are:
- Order of activation (old mode)
- Distance from sensor
- Order from left to right or right to left
- Around triggers werent drawn to live feeds.
- OSC and MIDI controller tracking sometimes registered feet as hands.
- In some cases OSC messages weren’t received in NI mate correctly.
- When using multiple users the MIDI Controller message for the first user was an aftertouch message instead of a CC message.
Updated Blender plugin
In addition to all the cool things in NI mate the official Blender plugin for NI mate has been updated with some nice features:
- Include an NI mate profile as a text file inside the .blend for easy loading of correct settings every time the game engine is started.
- Support for OSC messaging back to NI mate.
- Fixed an orientation bug.
You can get the new Blender NI mate Tools v1.1 and the Oculus_Bunny.blend demo for the game engine from the NI mate Blender page.
It’s been way too long in the making, but the first beta version of the new official Cinema 4D plugin for NI mate is now ready for you guys to try out! This plugin should work with at least R12-R14, but it is likely to be just fine with R15 as well. Let us know if you have any problems.
This version is much more capable than the previous one and some of the new features include:
- Recording under different root objects allows for multiple motion captured animations in the same scene.
- Joint rotation support.
- Default pose support.
- More accurate recording and better recording controls.
- Semi-automatic constraint creation to quickly start moving a real character through the motion captured data.
We’re definitely still open to all ideas and especially bug reports on this new plugin as this is not necessarily the final version, but a work in progress which we decided would be useful to put out at this point. For any feedback, please use this forum thread. As a tutorial video will only be made once the plugin is finalized, to get most out of the new plugin I suggest you read the details about everything from the plugin help boxes, which are located at the bottom of each tab in the plugin.
The new plugin can be downloaded from the NI mate Cinema 4D page.
This release is basically a bug fixed version of v1.16. We think that many of the instability issues people experienced with that version were related to a library that we introduced in hopes of faster loading times. In v1.17 we’ve reverted back to the old library and focused efforts in to fixing some other bugs as well as introducing much improved troubleshooting advice and crash report generation in to the program itself.
Changes in v1.17:
- Better device compatibility messages and troubleshooting help
- Much improved crash report generation for windows
- Many more bug fixes
This week at SIGGRAPH was exciting and both NI mate and Z Vector were well received by members of the CG industry. A full blog post will follow on SIGGRAPH later. As last week saw our NI mate plugin being included into the Unity Asset Store, we decided it was time to do a proper video tutorial to get everybody started. The tutorial covers the basics of using the plugin as well as how to import the live feed in to Unity (if you’re using non-pro Unity, please check out this FAQ entry):
We’re happy to announce that the official NI mate Unity plugin has now been included in to the Unity Asset Store. The plugin makes it super easy to bring your unity characters to life with NI mate and show the live view feeds inside Unity!
To get started with the plugin follow these steps:
- Open up the Asset Store from Unity and click the “Download” button
- Once the “Importing package” window pops up simply click “Import” with all of the plugin items selected
- Open the “Bunny Example Scene -> Bunny” Unity scene to open up the example scene provided in the plugin
- The Camera object contains the NI mate Receiver and NI mate Feed components, which communicate to NI mate
- If you have a full licence for NI mate you can simply run the Unity scene and NI mate should start up with the correct settings. If you don’t have a full licence you should start up NI mate at this point and import the “Bunny Example Scene/Bunny.nimate” profile to NI mate and after that run the Unity scene
For detailed information about the different plugin settings and usage, please see the “Delicode NI mate/Instructions” text file that comes with the plugin
Note: if you don’t have Unity Pro you have to go through an additional step to get the NI mate live feeds working.
It’s about time you say? To our defense we’ve been extremely busy the past few weeks and so wrapping up this release of NI mate took a bit longer than we originally hoped for. Anyway here’s NI mate v1.16 with some really asked-for new features.
Probably most prominent of the improvements is the new “Kiosk mode” (for full licensees). The mode (available from preferences) introduces a watchdog process that intently listens to a heartbeat from NI mate to see whether the software is responsive or not. If the watchdog notices that there is no heartbeat, it kills and restarts NI mate automatically (with all the settings intact). The other major update (only available for full licensees with a supported Asus or PrimeSense sensor) is the introduction of the 60FPS mode. This could be very useful for certain applications, although it does significantly lower the resolution of the input.
In other news the fully functional (and official) Unity plugin for NI mate seems to be stuck in acceptance limbo at the Unity asset store – something we will try to ask around to fix ASAP. Exciting times also as Z Vector is definitely getting closer to release (just a few weeks more – we’re in crunch mode). That’s it for now. As always, thank you for using NI mate!
New features for full licence owners:
- Kiosk mode option available in preferences. This mode launches a watchdog that restarts NI mate if it crashes, this disables all crash reports also from NI mate (on OS X all crash reports are disabled while the watchdog is running).
- Sensor infrared data is now available as an option in preferences. Note that this option replaces the RGB data in the live view tab with the IR data.
- High speed mode available in preferences for Asus Xtion and PrimeSense devices. This mode reduces the sensor resolution to half, but updates the sensor speed to 60 fps!
- OSC command “restart” optionally followed by a profile name or a file path restarts NI mate.
- Command line parameter “-load” loads a named profile on startup.
- OSC Controller message range can now be greater than 127.
- Smoother trigger motion.
- Triggers are now also drawn into the live view if the “feed 1″ option is enabled for a trigger layer.