2007 and now

I was asked yesterday to explain in which way my new demo "engine" is better to work with than the previous one (the one used in, for example, Lifeforce). I have given some technical details in random posts in the past, but today I'm going to list all of the reasons that make demo-making a far more pleasant experience.

• On a personal level I'm a happier person than I was in 2007. I have more free time ( it is easier now that my daughter is 3 years old, back then she was a newborn) and I'm completely satisfied with my job: my commuting time is less than 20 minutes door to door compared to almost one hour.
• My colleagues in ASD (Amoivikos, iM and Ch3) have more free time. I believe Amusic and Leviathan too.
  
• As expected, we have accumulated even more experience with demo-making: since Lifeforce we've made some alternative productions (such as Metamorphosis and Rupture) which tested the waters with the scene. The fact that the scene liked our direction gave us an extra boost of confidence.
  
• Better equipment and hardware: I wrote all of lifeforce on a crappy 17 inches CRT monitor using a geForce 5600. I had to see things on a 400x300 pixels window. Now I have 2x 24'' monitors and a pair of geForces and ATIs. I run the demo in full resolution.
• On a technical level: big change is moving from Cg to GLSL. It made my life so much easier as I have direct access to the OpenGL state machine.
  
• Use of .obj files instead of .3ds. Now I don't have to ever run 3D studio again! textures and shaders are hardcoded in code rather than from inside the .3ds file: much faster.
• Use of shader model 3.0 instead of 2.0: I use loops and conditionals to put a lot of the demo logic inside a big shader file. Rather than having one different shader per material I now have one shader for many materials that share some functionality.
  
• Use of flexible buffer objects: a big step forward from using the inflexible pbuffers (in 2007).
  
• Use of deferred shading to achieve effects such as SSAO. This means that I don't have to do any pre-baked lightmap ever again. So no more horrible Blender. All I need for modelling is Wings3D.
• All this results in a much smaller package: Lifeforce had 12 distinct parts which were coded in separate .cpp files. Each file was around 3000 lines. So that was around 40 seconds per part and 75 lines of code per second. Ideally I want to maximize seconds per part (in order to make the demo more "tight", to rely on less camera/mood changes and things that can go wrong with transitions) and also minimize lines of code per second. Iconoclash stands, at the moment, at 120 seconds per part and 30 lines of code per second.
  
• Compilation times for the project are also down to a couple of seconds (on my new and more powerful computer). I also have a way to reload shaders on the fly, so I can change shader code - press space - change again all in realtime without the need to recompile and run. It used to be the case that I had to re-ran the application once every 20-30 seconds. Now it may take 10 minutes before I have to make a change that requires a recompilation.

So that's what we've been doing over the past 3 years: making sure that producing demos stays a more and more pleasant experience.

Rendering a face

In my previous message I described the pipeline for rendering a frame. Today I will talk about the common material shader I use for pretty much all objects in the new demo.

• I use Lambert shading, almost all faces are flat (hard edges in Wings3D). However, object are well tessellated.
  
• There is an ambient component, a very slight diffuse (the difference in intensity between the face looking towards the light and away is less than 20%) and in most cases no specular.
• UV coordinates for texturing are calculated on the fly, based on direction of normal (vertex shader). I never store UVs in a file (although I could, using wings3D's auto-UVing, but it would be a waste of time: I get a very good result with my own auto UV-ing anyway). Needless to say, all textures are tileable. I do a couple of passes adding different octaves of textures to reduce any sense of repetition.
  
• Shadow mapping is performed as usual with projected texture look-ups. Sometimes I do more than one passes with slightly different projection matrix to smooth-out the artifacts on the edges of shadows.
• On top of everything I project and multiply a "Gaussian blob" texture in the direction of the camera. This adds a slight spotlight effect and improves realism quite a bit. An extra step here is to alter the projection matrix by a factor determined by the texture underneath. This way I can emulate pretty well a very soft "bump/gritty" effect. You can see this effect in the bathroom scene in Lifeforce.
• Then there is an element of fog - furthest objects appear darker. Pretty easy that.
  

That list covers, more or less, the vast majority of material shaders that I use.

TopMod

So you are making a demo and you are on the lookout for some nice organic, abstract 3D models to grace your scenes. You know how it works: put one of these models in the middle of the scene, one (much larger) surrounding the scene and forming the background. Then put the camera in the middle and rotate, keeping the first object in dead center. Add a bit of hypnoglow, a bit of particle-dust, a bit of typography and you have 20-30 seconds of bliss.

How do you create these object then. Here is an idea: use TopMod. It is a simple 3D modeller that you can use to make 2-manifold meshes, with arbitrary genus. There are some youtube videos showing some models in action.

But take a very very deep breath in before you start using TopMod. It is annoyingly easy to crash. Use it wisely with wings3d (as in do all the fancy genus stuff with TopMod and let Wings do the dirty work of smoothing and refining) and you'll be in demo nirvana.


(for the record I've never used TopMod for modelling for demos. But that is another story...)

New year's resolutions

Making Lifeforce back in summer of 2007 almost cost me my physical and mental health. It was far too big a project to carry out given the very limited time I had for my hobby. If I ever was going to make another demo of similar scope in 2010, things had to change.

I needed to alter my methods and streamline the process. So, in January, I made some new year's resolutions that I thought would help:

1. Ditch Cg. I've been using Cg for almost 7 years and it served me well. But with the advent of GLSL, Cg is pretty much obsolete. So gone are all the little nuisances and extra C code to "link" my external variables with the shaders. GLSL works much more transparent and I need fewer recompiles.
   
2. Ditch Blender, Maya and 3D Studio. I admit that I'll never become a master of those modelling programs, as I don't have the zen-like calm of a Tibetan monk to learn (of all things especially Blender) properly. They may be super-programs but for me they have always been a source of annoyment. First of all having to run them in the first place adds time! I will be doing something in code in visual studio and then I need to scale something in Blender or 3D Studio. All this process of run/load/finetune/save is costing in time. Part of the reason things had to be like that was my old insistence and total reliance on the .3DS format. There are historical reasons for that (the first "engine" that supported bump mapping, in planet risk, was using certain tokens in .3DS files). But that should change. So, ditch everything except, of-course, my sweet Wings3D - and use the .obj (Lightwave?) format. Yes, it is very primitive, loads slow and probably won't let you do materials and textures, at least with my loader. But it has everything else I need: geometry, UVs (if you need them, I never do), hard/soft edges, grouping of objects. Simplified my life by looong margin.
3. Never do another "baked lighting" texture again. It takes forever to auto-uv and bake the model in annoying Blender. I still remember the absolute nightmare that Size Antimatters (which uses such textures extensively) was. Nice way to spend your evenings, waiting for the radiosity pass to finish!
   
4. Start writing everything in a linear fashion. Start with Part 1, 2, 3 ... last part. And keep "parts" as long as possible, in order to reduce the number of total parts and hence complexity. Lifeforce had 11-12 parts, I'm aiming for a maximum of 6-7. It should make a lot of difference at the end when we have to link everything together (music and parts).
   
5. Put more "program logic" (i.e. if (time>20) do this ..) in shaders, using ps3.0: conditional, loops. As a result, the complexity of the C++ code has gone down dramatically. Most of my code is now shader-work. I estimate that the C++ "engine" is probably around 80% down in lines of (horrible) code compared to Lifeforce. Important exception to this is the camera-work and all functions associated, and I'll explain more in future reference.
   
6. Speaking of shaders, allow for reloading of all shaders for a given part while the demo is running (by pressing the space key). This has already reduced the refinement cycle times by a long margin. An additional tip is this: I put all my "real-time user-defined variables" (mouse position, buttons etc.) as arguments in *all* shaders. So if I want to fine-tune something and then something else I can do it with the mouse without having to restart the demo. It works very well.
   

I will elaborate more in due time. Further to this list, my new year's resolutions also include:

1. Keep and use 2 computers for coding: one with a crappy NVidia 8600 and another one with a more modern ATI 4850. Do most of the coding on the 8600 and only occasionally try the demo on the 4850. Aim for a 16-20fps at HD on the 8600, it should be sufficient.
2. Stop watching vimeo videos and xplsv.tv, trying to get "inspiration" (or, in other words, steal other people's ideas). It just doesn't work with me - usually a static image (one that is actually unrelated to computer graphics and effects) does the trick: a photo of a city, of a fruit, of a weird animal is more inspiring than most copy-and-paste "motion graphics" videos (some of them are really cool, though, and carry concepts I wished I had thought of myself).
   

I'll stop here for now. Yesterday someone asked me if my estimation of "33% of demo completed" is accurate or not. It really isn't, but I want to think it is, to give myself some courage.