A month ago I got hired to do a job through the working academy. Since they don't require signing an nda, it's possible for me to make a post here.
My task was to Model and rig a mother and a baby + some of the organs of the mother. This is to be used in a maternity educational app on android devices.
The client's target art style was pretty much in copying this picture they found on shutterstock:
So at first I was told that the models are to be pre-rendered- I made them high res. Then it was decided that the models should run animated in real time within the app.
So then reduce their poly count. Luckily when I initially modeled them, they were designed to work with a subdivision modifier to get their final "smooth" look so reducing the poly count wasn't hard.
They required the mother's face to be neutral - not belonging to any specific ethnicity or having too much character. Realizing that this neutrality is very much determined by the shader's properties as well as the shape, I decided to use a material that is as close to the target art style as possible. It was also important for this material to be game friendly. Creating a matcap(inkscape) allowed me to achieve the look in real time in the opengl viewport. Unity has a matcap shader, so in theory this should run in the game engine.In any case having this for previewing purposes helped me sell the model and get it approved. I needed to first pitch the approach - at the time thinking they will be prerendering the animations:
The woman is using a matcap (or any shader that gets similar look- it can be a ramp shader) The material is really not transparent, but the organs are showing on top of the mesh, even though they are inside it in terms of 3d space.The rim light and the subtle darker tone towards the middle of the matcap sphere helps reinforce the illusion that the mesh is transparent.
In blender its a matter of toggling "xray" for the organs geometry in their viewport render attributes. Maya is a similar story. You can alternatively use passes and composit them together (if you are doing prerendered clips).
It was suggested to simply use a transparent shader. I tried it and found some practical issues with that approach:
Both characters are based on a man mesh I made a few months ago. Reusing geometry is a nice way to save time in some cases. I still had to change the edgeflow to follow some curves better.
At the first pass the client requested to reduce the blocky-ness on the buttox of the woman. So I added more topology where it was needed to be more curved.
Modelling the baby was more challenging, because it was important for it to have that "newborn" look. There was a lot of play with proportions.
Dominique Lane provided concept art images for reference. I followed those closely as it was what the client approved. They later decided to remove the hair for the woman. For the organs there were references on the web. Dom provided concept art for those as well. I Looked at lots of pictures of babies and pregnant women. The poly count can be reduced more but was told to stop doing that :D
In total ALL the models(+organs) in the scene are 17 000 triangles. But all of the models are not going to be visible at all times.
The organs are made using lots of modifiers, making them easy to edit later on, while also being low poly. Blender has a nice "decimate" modifier that can non destructively reduce poly count- it's not ideal for everything, but it's handy in a few cases such as irregular static shapes. If you want to keep your mirror modifier, but still have some asymmetry, you can combine it with a simple latice modifier.
Rigging and skinning was the easiest part, because it didn't need approval from the client.It took a day for both with a rigging script- skinning taking the majority of the time.
A big part of this job for me was communicating with the client and the other team members, waiting for feedback, asking questions that weren't answered in the brief, etc.
For the model of the mother is good to note that they required a pregnant and non pregnant version. To achieve that I gave her a "belly" controller that is driving a joint. It is completely possible to animate her pregnancy progression by scaling/moving/rotating it. The skinning was a big part of getting that part right. Unity and most game engines don't do blendshapes very well or it is too expensive as far as I know.
This project was a slightly weird experience, that was still enjoyable. Here's hoping that it will find it's way to an android device! That part depends on the rest of the team.
I did all the characters and their animations in the froggy game.
The main sprite:
The first stage of that process was outlining all of the actions he is going to do in a sort of a state machine mockup that I had to run through with Lutfi Couri, who is the main programmer on this project. This exact same thing was done for all of the other 2d and 3d objects in the game that contain animation clips.
The original superfrog sprite served as an inspiration, but I did take quite a few liberties with the character both in terms of design and animation. During the pencil test, I would use layers in order to animate him one set of body parts at a time. The same approach is used in maya with the help of character sets. It's good to start with the root of the action- the torso/spine for example. Then animate on top of it the legs, then the arms, then the head, then the cape and so on. Layers are a good way of experimenting with animation for the different body parts.
Unfortunately pencil doesn't have the ability to duplicate layers, but is otherwise an excellent flipbook app.
The second stage was to go into toonboom studio, get the pencil tests in there , set the right resolution and form the animation library. Some of the body part animation is frame by frame, some is made by just squashing,stretching and rotating them. A well set up character rig in this case is key to easy pose-to-pose animation process. Sometimes when doing posing my character, I would go and pose the entire body frame to frame.
Third stage - after testing the sprite in the game, making sure it looks alright in perspective with the 3d background + camera angle, making sure the clips blend together alright and are at the right speed... I added some cell shading to it, some highlights, cleaned up the lineart and tweaked some things- then rerendered it.
Some of the animation clips (resized gifs) : run fast,run slow, die
Animation on 3d assets:
The enemies in the game were quite fun to do. Very similar in workflow- mockup state machine, do key poses then add secondary motion with some overlaps. It sounds like a lot, but it was very hastily done. I went straight to auto tangent and did very little work on tweaking the curves.
Find the extreme poses of what drives the secondary motion, then create the keyframes around those up and downs. Once they are at the right place, all one needs to do is push the secondary motion by scaling the keyframes on y in the graph editor.
Some more gifs here. Sarah modelled and textured this mushroom. I changed the model's geometry a bit, then rigged and animated it.
Simple quick mushroom rig:
Made of 3 joints, no blend shapes. Set driven key+ aim constrains.
Problems(hopefully not too visible) When squashing the trunk loses some volume- that couldnt be helped as we cant use blendshapes or waste too many joints to go all around the trunk.
I added some squash and stretch to the head as well, but cant be bothered to update this gif.
This semester my optional module is facial modelling and animation. This is yet another animation module in Bradford University that makes the student create an entire pipeline before animating anything. So there you see why there is so much modelling and rigging stuff on this blog and so little actual animation. None of the modules really allow me to focus only on animation and it usually happens in the last weeks of the assignments :) With that grudge aside, I decided that such a module would be a good opportunity to refresh my facial rigging skills before getting on with that part of Soda Trip's production. Gotta make some lemonade.
So why pick blender to make the blendshapes and go through the hassle of exporting to maya again?
The character was modelled in blender, so it made sense to continue using the nice modelling tools. Well, they are called shape keys in blender. Instead of duplicating a mesh, deforming it and applying that as a blendshape, blender stores the keyable deformations you make on a single mesh inside a shape key data block of that mesh - it stores the new coordinates of vertices you moved while being in the specified shape key mode.
Shape keys in Blender vs Blendshapes in Maya:
Blender allows me to change the topology after I have already made multiple shape keys on the mesh . It's like changing topology in maya after having made multiple blendshapes.
Whats nice about it is that I can use this new topology to shape my character on some of the already created shape keys.
Maya does not allow you to alter the topology of a blend shape object- if you add an edgeloop, it will not affect the target object or might break the blendshape (you cannot reaply it any more).
You can change topology of the target (base) object, but that new topology is not being transfered to all the blendshapes you have - again, you can't use it to deform.
So you are in a strange sort of a situation when you have to somehow relink blendshapes or redo them. Thats gonna suck pretty bad if you have about 20 of them already :D In this case the upper eyelid becomes a classic example. After the character closes it, the mesh needs an extra edgeloop in order to curve it along the eyeball slightly.
In maya, you have to add that edgeloop in advance. You have to guess how many edgeloops- some people add much more than one - you can add a few too many.
Here I can completely close the eyelid with a new shape key and then add as much geometry as I need to keep it from clipping with the eyeball.
It also lets me alter the shape key geometry when its not at its full value- in this case when the lid is half closed(.67), I can move the bottom edges forward.
To do that in maya you need to constantly look at the two meshes, set up an elaborate workspace with two cameras- one looking at your base mesh, one looking at the blendshape-work on one while looking at the other - it's just a pain.
Sculpting the blendshapes:
To get a good view over how the geometry deforms on my shape keys, I toggled on the excellent new matcap view mode and toggled on my wireframe from time to time:
Ton Roosendaal (Blender's founder) asked the community for matcaps for the very first version in trunk to support that feature as a preset in the viewport.
I made him two grey matcaps that might not look like much, but are actually pretty good at showing clearly how the geometry deforms and showing subtleties in a sort of a neutral way.
Ton included them in the official 2.66 release(mc8 and mc9). So now I can proudly say that I made a little contribution to Blender and to it's very big artist community! :D
Transfering blendshapes from blender to maya:
Just as with the recent posts , in this case I am transferring geometry from blender to maya and referencing it in maya. Only this time, I also need to export all the shape keys that I have created for my character's face and hook them up in maya.
When you export an fbx from blender that contains shape keys, maya gets all of those shape keys as sepparate meshes that have no material on them.They are also already applied as blendshapes to the main one and hidden.
Blender's fbx exporter impressed me this time. That is pretty neat! Only feature thats missing for me now is the option to zero out transformation values on export. I dont like having a dirty channelbox.
So It might look like I am pimping blender out a lot lately. That is hopefully not making me look too biased. The truth is that I need these features to do my work faster and blender is giving them to me. It's effectively competing with Maya, undermining aspects of it's pipeline. But Blender is not perfect. And in future posts I might write about it's weaknesses.
Getting the stuff from blender to maya had its quirks.
I'm gonna share this mini tutorial with the hope it will save somebody else the frustration that I had to go through.
If you pose your assets and export them together to an fbx file, this is what you are going to get in maya.
To get around the mesed up translations and rotations, I had to apply all object's coordinates. This in blender terms means that their local origin is going to move to the world origin.
That unfortunately stacks them on top of each other
Each is on its own sepparate layer in blender, so its easy to toggle that mess off and work on stuff individually.
Another thing I did was making sure my scale is the same in both packages. To do this I set up the metric system in both to be consistent. One grid square=1 meter (scale =.01 in blender). I set the same values in maya in advance.
Then I export it with the following settings.
modifiers are being applied on export, so I made sure that all subdiv modifiers in the scene are removed.
Maya's default world orientation is y=up, -z=forward. Whats nice though is that it kept the parent-child relationships between objects, so It kept my null controllers and turned them into locators. This makes it easy to move objects in the scene that have multiple meshes.
Referencing: So why is referencing useful? It allows you to make changes to your model and rig after you have skinned it. It's also a neat way of isolating data blocks in separate files - making it more manageable. It means being able to swap different versions of geometry and your rig with a few clicks. - load a low poly proxy (cubes), continue to improve a rig after you have started doing animation on it (in my case its adding facial controls after finishing body animation). It means being able to track down issues easier. It does add to the loading time though.
One thing that I found with my last animation project is that while you can edit referenced files in Maya to a certain extend, every edit you make to them of course stays in the maya scene file, its not done to the (in this case FBX) reference.
It's a good rule of thumb to keep edits to a minimum, as it contaminates the scene and increases the chance of problems. So I never do stuff like editing the geometry of an object that is referenced in the scene. The reference edits list window in maya keeps track of everything you've done to the reference. Sometimes if a scene is broken (crashes on load), its a good idea to load it without its references loaded, find the problematic ones and clean up their list a bit.
In this case, maya is storing only the new locations of the controller objects and unfortunatelly the frozen transformations command that I did before moving them into place.
Blender rotates all the stuff accordingly and adds rotational offset to the values in the fbx. I had to zero them out in maya.
Referenced objects paths:
If you use the file manager to load and reference a file, by default maya will use the full path of the file. As a result if you move your projects folder to a new place, new computer or drive- all of your reference links will become invalid and you will have to manually relink them. It doesnt sound like a big deal, but if you have a big and complex network that can become an issue on an operating system such as windows, where you cant create symbolic links. So the computers at university have maya on windows, while I use maya on linux at home. Maya on windows has poorer performance and tends to crash more often. Submitting my source files to uni computers meant that all the reference links would get broken and likely my tutors will be unable to open them without having to spend at least half an hour relinking the files. As an extra annoyance both maya 2013 on windows and linux have a bug where when it prompts you for the new path and you give it , it doesnt apply it and also makes you wait for ages to search the files and fail.
So what was the solution to that one? It's pretty tough to find with google and after searching I found that other people have had the issue but their solution was sort of wrong and wasnt really a solution.
At the same time I was pretty sure that maya has the ability to use local paths for anything that is inside the project folder. However to make all the paths local I had to change them manually, without using the file browser. So after renaming each to use local links, now all my references just work out of the box, wherever you move the project folder.But it's my suspicion that whoever opens it after me has to make sure they set the project folder location.
MAYA BUG: Unfortunately Maya 2013 (and older?) has a bug where it does allow you to use fbx files as references, but does NOT allow you to have local path linking for fbx files (only full paths- even tried with fbx files created by Maya). That forced me to import my blender fbx files to maya, save them as maya binary files and then replace the fbx references that I have with mb ones. It means that every time I make changes to my models in blender, Instead of updating automagically where they are referenced, I will have to manually import them in maya and save them as mb/ma files. :(
Autodesk, please fix this bug!!
Do let me know if you know a better solution. :)
I did this stuff at the end of last year and just now getting around putting the notes in a sensible way. If you're not interested in modelling this post is going to bore you :)
There are two types of geometry in an animated film- one that is to be skinned - with good organic topology edgeflow to serve the deformations and one that is to be used as a background asset. That needs to be:
as Low poly as possible (for proxy use during animation stage)
edgeloops at the right places- to keep those hard edges when subdiv modifier is applied during rendering time
edgeflow doesnt create any shading problems
Use the whole toolbox and modifiers wisely to save time- pick the right tool for the right job, instead of 1 tool for every job (I'm looking at you extrude and edgeloop insert ;))
So with these goals in mind I set off to model the things in the long list, so each asset is
Easy to edit:
non destructive editing (modifiers,curves,etc)
Reuse geometry and its modifier stack where ever possible (no need to build modifier stack from scratch for everything, just separate new objects from existing ones)
work on halves, Instance and dublicated geometry where possible
Here is a list of the modelling tools that I put to practice in blender:
Lofting: using surf curves+mirror modifier to create a nurbs patch. I use the mirror modifier for reference, then just dublicate the curve and flip it (s,y,-1).
also used nurbs patches for the typewriter body. This allowed me to tweak the proportions easily before going into any detail.
Using CV curves and nurbs is an excellent way of creating precise shapes that have some curvature. The method is quite simple, yet still flexible in blender!
In order to create the keyboard, I made 12x4 rows of vertices for each button, then made a single button that was instanced on the place of the verts automagically. This allowed me to scale the button and change its design on the spot, updating all of its instances.
I dublicated my button map object and used it to create the button trunks by extruding the verts and then applying a skin modifier for the created edges.
similar technique to the coat hanger:
its using a 3d curve that is set to bevel and that object is being instanced on the vertices of a circle object that goes around the hanger's pole. 4 verts= 4 legs.
Something common with all the assets is that each is made with the minimal amount of geometry to edit. I try to leave as much as possible to the mirror modifier- sometimes using 2 or even more mirror modifiers, array modifiers,instancing and so on to procedurally generate the parts of a mesh that repeat. It's procedural, but it saves time. These sort of tools are present in maya as well, but they are much worse and quite unreliable some times.
Here one edgering circle is being dublicated by the array modifier, mirrored by the mirror, solidified and the subdivided.- all non destructively, so to change all the rings, I need to edit only the two edge loops that form one 7 faced edge ring.
I used a bezier curve+ text object for the fridge logo- so it fits the shape of the fridge door and also has this corky bend to follow the curves of the curcle beneath it. I made sure the curves handles are precise by copy and pasting their corresponding coordinates from the start node handles of the curve (left side) to the end node handles (on the right side).
The text object is editable, so I can change fridgeomatic to something else if I want, swap the font, resize, etc etc.
You can see in the wire that the fridge door also has some jars, ketchup and other stuff modelled. Everything is using a subdiv modifier, so it has a low poly proxy stored.
Other modifiers I used on assets that are not in the screenshots: Boolean, skin modifier, etc etc..
One of the university modules in this semester had for an (optional) assignment the research of reusing character rigs and animation to be imported to a game's engine.
I was given a deadline to write a greenlight pitch, a research paper (almost 30 pages!) and then create a technical demo video.
It's pretty much do it yourself sort of a thing set by university(just as many other things).
In this case before even delving into reusing animation clips, I decided that it's more important to find out what exactly are the best practices in character rigging and animation for game engines - where the data has to be created in a specific way that will make it friendly to use by the programmers and the engine.
In the research paper I used for sources the excellent presentations on animation in Uncharted 2 by naughty dog, among other things.It was a bit rushed so It got me 64.4% .
For the video, I used kdenlive to put together a quick overview of the findings of experimenting with some of the tools in Maya:
The paper and the video focus on the work that is done in the animation package rather than the game engine. I have in the past done character modelling, rigging and animation for Unity 3d. That first experience was a bit of a trial and error - the animation data needed to fit with how the character actor is scripted.It's something that I will keep fo a future post, until there is time to put together a demo of that game.
So before I get on with the Blender rant, I just want to let you folks know that the piggy bank animation thing hasn't been completely abandoned. Just decided to postpone it's production until there is a meaningful story for the already interesting premise- a tiny girl on a desk! :D So there is already a lot of rough animation there, but I decided that it just doesnt hold up well as a story with what I've got-
When I did it at the time, it served as nothing but an excuse to do some animation for my university module. I am open for suggestions. Anything you can imagine with the rigs that I have. If I like it and make it, your name will be right there above mine, next to "a Story by..." My email is at the contacts page.
You might also notice I haven't worked on it for over a year. :D
Ok, now on to Blender.
"But Todor, why spend time to learn Blender, when obviously- Autodesk is the KING of the hill - people with jobs use Autodesk products!!"
siggraph 2011- Autodesk thinks people shouldn't blend in..
While I love Maya for animation and rigging, Modelling and texturing in it seems to take much more time and effort than does in blender. Certainly it has way more features than Blender. But does that make it a good choice for everything?
This is what I discovered that Blender does with less complexity, effort and need to buy proprietary plugins or combine with other software. It's artist friendly!
Better underlying design.
Navigation: You pan and navigate without the need to hold alt key. Going to views can be done by holding alt to snap to them, rather than selecting them from the spacebar menu (maya).
This ultimately makes it better suited for use with a digital tablet! It's very similar to how zbrush does it.
Selecting In maya you will keep selecting verts that are on the other side of your model- constantly, even when you disable that. Blender has a number of selection modes that are fast to switch between without the need to click on icons.
Mouse design Blender's mouse interaction has been designed from ground up to reduce repetitive strain injury oh the hand. Reducing the time you use the same finger to click. There are some very clever gui design choices in there..When you manipulate things with grab rotate and scale, you dont have to click and hold - again less stress for your hand, less work
Moving objects in maya is a pain. You need to use the widgets and switch between modes constantly. You click and hold to move things.
In blender you can just hit G and the object will move with your mouse until you left click to apply the transform operation you made. It might seem like a small thing, but when you pose a character or rotate things, it makes it way easier on your hand and patience. I don't need to grab any tiny manipulator handles getting in the way any more. They are still optional though - useful for hard surface modeling.
Real time action adaptation
You have snapping,falloff, you can still type coordinates or isolate axis if you want- the best part is that all of it happens while you are moving,scaling or rotating and you see the exact effect before applying it by left clicking. You set your options while you operate, not before.
This approach to design is applied through out many of the other features too.
Access to comands: Markup menus and shelf icons in my opinion are slower than using shortcuts in blender. The problem of having to learn the shortcuts has been eliminated by the inclusion of the spacebar find command.That same feature can be used as an equivalent to maya's repeat command operator, as it always remembers your last action.
3D cursor- Blender has a unique global pivot control manipulator that can be used on any object in any mode. In maya you need to manually change the pivot of the object itself or do some tricky stuff to somehow get that effect in its edit modes.
Editing - Maya doesnt use a modifier stack (blender and 3ds max) . Instead everything is stored in nodes that you need to clean up from your history when you model or disable history altogether. This adds more power to maya, as these nodes are scriptable/animateable. But it also adds more risk of complications and potential glitches. Having to constantly freeze transformations and delete history slows you down early on in the pipeline.
Sculpting- Blender has vastly superior sculpting and painting tools. Maya doesnt even have a grab/move brush. So you cant really use it to tweak the proportions of your model. You end up going back to manually selecting verts, adjusting fall off and moving- that is a slow way to work.
Later this year, Nicholas Bishop will hopefully merge his code for dynTopo to trunk- It's blender's equivalent to sculpris' unlimited clay. What it does is dynamically subdivide your model only where detail is needed. It figures that out depending on you brush setting and the distance of the viewport camera to the surface of the model. As a result you dont need to worry about topology getting in the way of the shape you want to achieve. No need to subdivide the whole model to a billion polygons to give your character a more rounded nose.
This is the first time such a feature is part of a full modelling and animation package! I will write more about it in later posts. Until then here is a demonstration of it's use by Max Maurel. Add to that resymetrize and remesh and you have a light free version of zbrush. :)
A quick test run of an early dyntopo blender build left me rather impressed! If you look at the name of the object you'll see what I started with. If you look at the top of the head- that area is still blocky as my brush was no longer interested in adding detail to it. This makes the geometry much more manageable to manipulate.
zspheres like tool ( skin modifier ) in blender lets you quickly generate organic shapes with clean topology, ready for modeling/sculpting. The only way to get that in maya is via an unofficial mel script, and thats not even as good.
I started doing this dragon model for fun and to test the skin modifier. It took about 20 minutes to get it this far.
If you look closely, you will find that It's made out of nothing but bare edges. The skin modifier dynamically creates topology around these construction edges , based on the thickness you apply to their verts. Some times the geometry has collapsing that can be cured by adding/extruding an extra vert.
The weird topology near the middle of the x axis clears up when you apply the mirror modifier. In fact, if you use mirror with skin and multires modifiers, my advice is to apply mirror first, even though it's not on the top of the stack.
The beauty of the skin modifier is that it can also generate a bone structure based on your verts. The big circle over the hip of the dragon is the marked vert of the ROOT joint of the armature that will be generated. So no need to place joints, only to remove some joints and re-orient others.
Bsurfacesmakes retopology and generation of character outfits easy. It's very close to what specialised retopology apps do.
The artist can use freehand strokes to generate new geometry in 3d with the grease pencil.
The ability to combine that with modifiers can be quite empowering.
The Grease pencil in blender lets you write notes and scribble over your model before it has been textured. It even picks up the pressure sensitivity of your tablet.This is very useful when you want to experiment with your characters, write/get feedback or make tutorials for others. It works everywhere- letting you write notes in your x-sheet, your compositor, everywhere.
UV-ing and texturing- When you mark your seams and click "unwrap" blender just gives you a much better result, that is uniform and has little to no stretching. Getting that effect in maya takes more time, more steps and is more complex. Live unwrap is a feature that updates your UV as you add new seams, without the need to re-unwrap constantly. There are other impressiveimprovements coming too!
I can just go on and on. Maya is not a good tool for character concepting / modelling. You can tell me to go and use Mudbox instead, but to be honest, I still see Blender as a better option. I dont have to jump between software to do simple things. Mudbox doesnt have dynamic topology, retopology or a skin modifier.
autodesk has never really intended to provide these tools- companies buy maya for the animation and special effects.
Leaving all these things aside, it would be good to end my rant with Ton's words:
My plan for now is to learn to do one thing well in maya. The things before animation in the pipeline, I rather be a generilst in with Blender.
This is a great FREELANCE alternative. Let's be honest, who in their right mind would spend 7^ k on a software package license ? Only a big house company would.
One of my optional modules for university this year was motion capture. The reason that I picked it over concept art and facial animation is simply the fact that motion capture technology is something rare and hard to get access to. One of the unique aspects of the University of Bradford is indeed the fact that we have a motion capture lab here.
I've been blogging about this in the past, so there is no point in introducing you the project again. Instead, what I like to do is just present you with the end result and share with you what I learned by going through this trial. My hope is that next year students picking this module will avoid making my mistakes while capturing the data.
So here goes a short video. Please, use the pause button to read the text.
The skinny guy , holding a pair of wooden sticks is yours trully, trying this for the first time. My tutor intentionally (i bet) "forgot" to warn me about some things on the day we did this.I am thankful for that, because sorting them mistakes out later on was a valuable experience. It taught me how to correct badly shot mocap data xD
The goal? Record mocap data and sample some of it, then turn it into loop cycles that can be used in a game engine. Those sampled actions (some looping) have to smoothly blend between each other.
So lets go through this thing. First of all , HERE is a report I did for uni about motion capture technology as it is today. It looks at the limitations of the current approach and what the promises of the future might be to sort them out. Written in simple english, it should be easy for non technical people to understand it. :)
And HERE is a very very non informative and dry list of the things I did to get my loop cycle.
1. Cleaning it up in Blade-1.7. The main problem there was really dissapearing markers. Markers switching names (especially the wrists). The software is NOT very smart at this.In fact, if you try to do things automatically, you are left with a mess most of times that leave more work to fix than doing it all manually. So I did manually. Thank god in this I didnt roll on the ground or didnt fight with the other guy in the suit from very close. Things like that are sure to leave a lot of work in this step.
This is what happens if you callibrate the character in Blade and you have not cleaned up the data completely.
Blade is known to be very unstable, but it didnt really crash on me- not really.
2.Getting the data to Motion builder 2011 and sampling loops.
Here is a honest rant on why I found MOBU to be one annoying piece of software. Motion builder is somewhat unstable and has some usability bugs too. You would expect from such an expensive piece of software to be more polished. First of all, when you start it at first, the way it organises its windows on the screen suggests that it doesnt know what resolution your screen is and its set for a screen with very low one. So I had to move them around on every start (annoying as hell) until finding out that I can actually save myself a custom layout that will remember the new size and position. Then there was the memory leak problem. Mobu has a thing about consuming 90% of cpu while running, even when you dont actually do anything with it, even without a file loaded. Looking at the task manager I found out that if I leave it on and go get a coffee or something, there is a risk that my laptop's CPU would melt in an hour. I had to get used to new shortcuts in order to use it. Yes I know that it has a preset that lets you use maya's shortcuts. But let me ask you something. Say you set it up with maya keyboard preset and then you have to assign markers to body parts of an actor. How do you do that then? Thats right, you can't alt and drag and drop while its using maya presets (try other modifier keys- ctrl,shift..nothing worked). You have to switch back to MOBU's presets to do that. And I tried and tried and tried to find what the assigned shortcut for that is for the maya presets. I couldnt. All I found was a guy at the official MOBU forum telling another to switch to the MOBU preset and learn that.
That is the biggest problem with motion builder - not a lot of documentation. A few pdf files, a few video tutorials on youtube and that was all. Not being able to research it properly before using it annoyed the hell out of me. So I was very much forced to try things and see if they work.
What were the advantages of Mobu and the reason people use it for this sort of work? Mobu is very good at retargeting animation and non destructively editing it. Its all FK rig can be very useful for animation. I used it to retarget my mocap data to the character. So there are two approaches that I tried: - using the point cloud to drive an actor and then retarget the actor to a rig, which drives the character: point cloud-->actor-->mobu rig-->charactorr This approach proved to be more of a hassle and gave less precise results.
-Using the imported Blade skeleton to drive a mobu rig, which drives the charactorr: Blade rig(already driven by point cloud)--some retargeting settings-->mobu rig (to layer some animation and be able to sample)--> character(end result to be exported to maya) This second method is more precise. The first method with the actor , one has to set the actor (blue guy) to fit the point cloud. That turned out to be a pain to do because I did the silly mistake of t-posing in character (hunched and even with a fake hunchback-pillow under the black suit). My rig from maya was set to fit the character mesh- which is also in a slightly hunched pose. What I got out of that was simply one extremely hunched character- so: Hunching actor+ hunched rig= result is too hunched, needed to reduce the hunching hunching actor+ normal rig= better result, without that much to fix
The other thing that had to be done for this to work in an engine was to stop the actor from moving in space. I used a "inplace" modifier, which just mutes x,y,z of the root joint. I locked only X and Z.
I used its story mode to sample a couple of seconds and turn them into the final loop. In order to have a loop , one needs the action to end in the pose that it began. That was very easy to do with MOBU- just copy the pose and layer it over. It smoothly blends it. Then exporting that rig with the keyframes is easy as 123- just save it as fbx. When I imported that FBX in maya, where my character (target) rig was, because its name was the same, my already skinned character just snapped into position and started moving properly. But then watching it again and again, frame by frame, I found that there were issues with the data.
3. MAYA- more rigging work and fixing posture
1. the mocap suit hat was put on a wrong angle- instead of two front and two back markers, I had one marker for front,back,left and right. This made the rig think that my character's head is constantly twisted to a side. 2. the mocap suit right foot heel marker- because that marker wasnt leveled with the ground (like the one on my other foot), it caused the rig to think that my foot is constantly bent and going through the ground.
I fixed all these by using animation layers and also deleting some keyframes. The scissors are an automatic rig that is being driven by the hand joints.
Dont forget that this is all done very amateurishly and in a hurry. A good motion capture work is not something that someone who is learning the ropes can do.
Final verdict: This would have taken much less time and effort to just animate. Cheaper, faster, easier. Motion capture technology today is too expensive for a freelancer to use- both in time and money.
Been a while since I posted - and for a good reason! That however doesnt necessarily mean that I haven't been busy cooking new content. On the contrary :)
So This month I am going to try and catch up with you folks, also document my adventures with cg software.
Today I want to look back at an assignment that was presented to me by university. Let me start by posting the whole thing:
No, my inspiration for this does not come from Jurasic park or Jumanji. I'd say mostly "Blade runner" and a game from the 90ies called "Flashback"
The footage that I got was grainy, and with a weird resolution (960x540) Pal (25 fps). The brief was to composit some elements over said footage by using after effects and tracking. The footage was only 11 seconds and I managed to get it ready in 11 days.
Here is how...
Motion tracking with matchmover turned out to be a semi-success , even though it wasnt very precise. Since the camera is not really moving in 3d space, just panning up- matchmover had a hard time determining where exactly are the markers in xyz. With some tweaking and even minor brute force I managed to at least get a coordinate system to use in maya.
Bringing the tracking data to maya revealed that there was a minor twitch of the camera. I fixed that by dropping two new keyframes over the group that contained the reference 3d objects and also deleting some keyframes. Some might say that this shot might have done without 3d tracking. The thing is that I really wanted to get to know the software+ there is some perspective distortion (on the roof of the building and the top of the entrance) as the camera pans up.
The first step for me was getting the trackers to work, in order to be able to pin things to the footage. Seeing if things would flow with the 2d and 3d tracking was also the right time to plan the shot. So After I got my 3d tracking done, I made a bunch of really basic non textured models and animated some of them. Later they were replaced by proper ones.
2d tracking with mocha
Just had to use mocha tracking, in order to pin some texture to the walls of the building. It turned out to be one intuitive and useful piece of software. Again- tutorials from youtube :P
I discovered that graininess and lack of focus on the background elelments make the tracking markers unstable. Thankfully that was something that could be fixed quickly in Mocha. It even has an "auto" button. Adjusting trackers with a mouse can be paintstaking , because one has to point at an exact pixel of the screen. One cool thing that I found out is that using the laptop's touchpad for this sorta thing is actually better than the mouse. By some lucky coincidence my mouse broke down at that time and forced me to do so.
Mocha was also excellent for rotoscoping (export shape data). Its capable of pinning the nodes of a mask to trackers, dynamically masking a specific area (used that for the top of the roof).
I pasted that tracking data onto a bunch of solids in aftereffects and then replaced them with testing uv grids. That is how I made a UV map of the faces of the building. Making its layer semi-transparent so it was possible to see both uv space and the building details underneath it.
After rendering that out of after effects and bringing it to photoshop, I laid it over the 960x540 original uv map image - using the free transform tool to unwrap it. Knowing where exactly the windows are enabled me to paint detail on the building walls. Its all hand painted in photoshop with the help of my cheap tablet.
The sky color mask was created with the help of the new amazing roto brush in after effects. I dropped a green tint on it, to make it look more surreal and to fit the mood of the shot I was aiming for.
Compositing the cgi footage:
In order to get my render to look right, I first made sure that I am happy with the colors in after effects and the building detail that I added has the right colors too. Then I rendered a single frame (Aftereffects) out of that and brought it as a frame in the tracking image sequence( that MAYA is using for the tracking camera ). I also opened it in photoshop, so I can pick the colors while painting textures on the cgi elements.
Uv work and texturing was all done in blender..
Scrubbing to the aftereffects touched frame in Maya, I was able to set up the light and materials as close as possible to blend in. A trick I did to make the cgi blend in more was using some reflectivity on the metalic materials and a sky dome mesh (which was set not to render, but to be visible in reflections). There is also a surface shader on the faces that I needed to key out in after effects. I separated the car in the sky from the landing car to two separate render layers- so it will be easy to quickly blur out the first in after effects- creating the illusion of depth of field and improving the composition of the shot.
Bringing the 3d footage to aftereffects revealed a number of issues that needed my attention. Some of it was covering the entrance of the building. I fixed that by getting some mocha shape data to aftereffects for the front of the entrance. I dropped some gaussian blur filthers, outline blur and hue/saturation on the elements. Some things needed to be desaturased. The cgi stuff was still too crisp so one last thing i did was dropping a slight Noise filther on it.
Things that can be improved:
The motion tracking data can be a lot better. Time restrains really left their mark and there are some minor issues that need attention. When the camera pans up, the top of the building shape twitches a bit, revealing a slight blue border around the bars of the roof top. The piston on the side of the landing zone is dropping a slightly unrealistic shadow which shouldnt normally be there considering that the light source is not close to it- its day light which should cast softer shadows.
All and all, this excercise was a really challenging and thus also fun one for me. I hope that I get another chance to do this sort of work again. Hopefully not with after effects! After effects makes it hard to find problems when problems occur. The layering way of approaching compositing is also a bit limiting. I had to toggle layers on and off to see where the problem is. That is in my oppinion a very inneficient way to deal with compositing, especially when there are more than ten layers in the shot. Nuke, Shake,autodesk composit (formerly Toxic) are all excellent packages to do cgi compositing with a proper node-based approach. The problem is that Universities (or at least mine) are not teaching it at the time that I'm writing this. :)
I've been juggling a number of projects lately. The animation on piggybank has been in the background and I haven't done much work on it since the last post. The models for the moCap project are ready now and this article will focus on the technical side of things as normal maps are concerned.
There is also a freelance job (a short 2d animation) that took some priority- I just want to make sure it ends up looking very good. :) I wont go into detail about it right now, as it wouldnt be professional to discuss it in early stages, while the client is very much deciding on how it progresses. I am not the type who would release anything I'm working on without the client's permission.
Anyway, onto normal maps. It has been my main interest since the first day I started using a 3d package. How do I get a LOT of detail projected onto a low poly mesh. And whats with that obsession with low poly stuff (!?) some might ask. Am I just being a cheap bastard here , with today's technology advancing so quick. :D Let me state the obvious advantages of keeping your mesh low on polycount- use geometry only where you need it. - your computer can handle the files, especially if they are part of a giant scene.The files can be used in real time rendering engines. The playback speed while animating is near normal- no need to playblast it a billion times- its a waste of time. - you can edit the files faster- its easier to edit the overall shape (especially with maya), its easier to paint weights (skinning) when you dont have to deal with a lot of geometry= that equals faster workflow - You need geometry only to define the silhouette of a character and to define how the character deforms. The details can be achieved through bumpmaps and even better - normal maps. It makes me cry a little, every time I see a model that has been subdivided a million times, has geometry to define cloth wrinkles, face wrinkles,creases/cracks/etc, hair detail, etc etc. Why oh why dont you just make a normal map and bake all that stuff there?
Normal maps are just fancy bumpmaps. Some people convert bumpmaps to normal maps ( not much of a difference there lol). Some people convert textures to normal maps. Want to put some cracks on a rock? There is a photoshop plugin that should do it for you, you can also use xnormal. There is nothing wrong with that- it's just not good enough for something like a next-gen character. While bumpmaps contain information about depth, Normal maps go a step further. They contain information about direction of that depth. So one can have a really low poly character look very very detailed. The only thing that will give it away will be the silhouette of that character. I know about displacement maps, but I wont go there now :D
So for next gen characters, one basically tells the software to compare the low poly mesh to the high poly mesh and create a normal map for the low poly mesh. Mudbox, zbrush, maya, blender- they all have tools to do that.
Since blender has a special multiresolution modifier for its sculpting mode, it is actually pretty damn good at detailing characters and baking a normal map. It might not be as good at handling giant meshes as zbrush, but its good enough to do the job.
I spent about a week in learning from trial and error about normal maps. For my little experiment, I used the two characters that I made for the moCap module. These guys are pretty much a work in progress and I am not even near pleased at where they are at the moment. Both of them have tangent space normal maps (tangent space is good for meshes that deform, but its a bit more expensive to render than world space, which is good for static objects)
Scarlet Thorn:
Scarlet was made almost entirely in zbrush. I made the hi res mesh (detailing) and exported a normal map from it in zbrush. The texture was also made in zbrush.Whats good about Z and I think is one of it's killer features (correct me if I'm wrong but mudbox doesnt have it) are the selection mask modes. Just with a click of a button, Z is capable of masking all the crease areas- that makes it extremely easy for the artist to flesh out detail..
On the screenshot, you can compare how it looks like with detail in zbrush (actual looks) and then how maya portrays the normal map on the low poly object. I didnt show render, because rendering has million of options to tweak and that alone can make it look like anything.
Zbrush flips the map vertically by default. It also left me with some slight edge marks, which I had to clean with the smudge tool in Gimp. That was a bit annoying, but it still got me a decent normal map.
Sepia Baron: I dont have to tell you how much more advance zbrush is at sculpting than blender. But the new blender has some key features that make it worth my time. First of all, its a modelling package that has sculpting capabilities. The brush engine in blender is actually kind of nice.I can do the whole pipeline in it- from modelling the low poly (check metaballs as a nice alternative to zspheres), retopology, uv mapping, detailing (sculpting). Blender can do it all :D. Maya is useless at sculpting- utterly useless... So I sculpted detail of my character in Blender- the newest build , then I baked it and got out a nice normal map.
What was surprising about Blender's normal map was- it actually was cleaner than the one I got from zbrush. I know he's a simpler character, but I didnt get anything that needed my attention in gimp. The only annoyance in blender is that when you subdivide the model, you can really feel the weight. It's probably a good idea to turn off skype while sculpting on subdiv 6. But that is likely to change, when the unlimited clay modifier gets released in some of the future builds. :D
I used the normal map as a basis for the texture, since I wanted to bake some slight shading on the finer detail. The texturing was done completely in Blender+gimp. Its incredibly easy to use them at the same time- blender has a really nice integration with gimp, when it comes to image editing.
Normal maps and rendering engines~ This is pretty much where all the hedaches start for the artist. Different software handles normal maps differently. Maya has three ways of rendering it- ~hardware (viewport) render- seems to display the normal maps without any problems. Note that you should avoid having flipped uv islands- that always offsets the map slightly and creates really ugly seams. I got away with fipped island on the hands and feet, since I wanted to preserve some UV space... but that backfired on me. I needed to go back change the UV of Scarlet thorn in order to get rid of seams. Thankfully zbrush is capable of projecting detail from one mesh onto another (both texture and shape), so I didnt have to redo all that I had. ~Software (maya) render- is absolute rubbish at rendering normal maps. :D I might have missed something. I've tried my best, but not only do I get ugly seams everywhere possible, but its also not very good at representing the hi res detail. ~ Mental Ray render- Seems to be pretty good at normal maps. I got no seams, the hi res detail is pretty good. One thing to note is that it desaturates the textures. I dont know a lot about mental ray. Everything I've posted this far has been rendered with maya's native software render. I dont have the time to learn mental ray, so what I posted now at flickr is as default as it gets.
While still being on the subject of rendering and normal maps, it would be worth looking into how it works and why there are seams/hard edges in some cases. Blender developers have been hard at work on documenting that and improving the normal map baking magic that happens . It certainly has been a growing interest for me to follow its development and there have been some pretty exciting things happening there-keeping my attention.
The woman is using a matcap (or any shader that gets similar look- it can be a ramp shader) The material is really not transparent, but the organs are showing on top of the mesh, even though they are inside it in terms of 3d space.The rim light and the subtle darker tone towards the middle of the matcap sphere helps reinforce the illusion that the mesh is transparent.
Rigging and skinning was the easiest part, because it didn't need approval from the client.It took a day for both with a rigging script- skinning taking the majority of the time.
