Rules for Multi-pass Shaders in Unity

Here is some basic information about multi-pass shaders in Unity. This post is accurate to 11/27/2014, for Unity 4.5.

In the last few days I've been looking a bit at shaders - especially at combining surface shaders and vertex/fragment shaders. The most informative info was in this thread.

However, some of the information has been deprecated and is listed in a confusing format, so I think it'll be nice to have an updated summary:

For starters, what is a pass? 

A pass block in a Unity shader denotes when something is rendered once. Each sub-shader can theoretically have multiple passes - meaning that the geometry will be rendered multiple times.

(Each pass has its own properties that will not be "passed" on to other passes. For a list of those properties, check here)

Most sub-shaders will only have one pass - or possibly none, if people are using a surface shader. However, the rules and exceptions get a bit complicated with mutli-pass shaders.

Rules for Multi-pass Shader

Q: Can you put two vertex/fragment shaders together?
A: Yes, just put two passes with vertex/fragment shaders right next to each other. The second one will be drawn over the first one. (Note that some properties will be passed on from one pass to the other. For example, if you use a vertex shader to change the mesh geometry, the changes will still be there in the next pass.)

Q: Can you combine surface shaders and vertex/fragment shaders?
A: No. You used to be able to, but right now the surface shader will simply override the vertex/fragment shader.

Q: Can you put two surface shaders together?
A: Yes. You can't put surface shaders into a pass (this will throw an error), but you can put two surface shaders right next to each other. As usual, the second one will override the first one.

Q: What about fixed functionality shaders?
A: Fixed functionality shaders (I'm referring to these) can be put into a pass and act the same way as the vertex/fragment shaders in a pass.

Sample Code

Here is some basic sample code for testing. There are two surface shaders, one fixed functionality shader, and one vertex/fragment shader. You can comment out the different parts for experimentation on how the system works.

Shader "Custom/GrayScale" {
Shader "Custom/MultipassTest" {
    Properties {

    }
    SubShader {
      Tags {"Queue" = "Geometry" "RenderType" = "Opaque" }
    
       ////////////////////////////////////////////////////////
       //Surface Shader 1 - BLUE                             //
       ////////////////////////////////////////////////////////
       CGPROGRAM
       #pragma target 3.0
       #pragma surface surf BlinnPhong
       
       struct Input {
           float4 color : COLOR;
       };

       void surf (Input IN, inout SurfaceOutput o) {
           o.Emission = float3(0,0,1);
       }
       ENDCG
       
  
       ////////////////////////////////////////////////////////
       //Surface Shader 2 - RED                              //
       ////////////////////////////////////////////////////////
       CGPROGRAM
       #pragma target 3.0
       #pragma surface surf BlinnPhong
       
       struct Input {
           float4 color : COLOR;
       };

       void surf (Input IN, inout SurfaceOutput o) {
           o.Emission = float3(1,0,0);
       }
       ENDCG    
          
          
       ////////////////////////////////////////////////////////
       //Fixed Functionality shader - GREEN                  //
       ////////////////////////////////////////////////////////
       Pass
       {
           Color (0,1,0,1)
       }
      
       ////////////////////////////////////////////////////////
       //Fixed Functionality shader - WHITE                  //
       ////////////////////////////////////////////////////////
       Pass {
           CGPROGRAM
           #pragma fragment frag

           fixed4 frag() : COLOR {
               return fixed4(1,1,1,1);
           }
           ENDCG
       }

      
    } 
    Fallback "Diffuse"
  }

Momentum in Game Development

I think there’s a very nebulous thing in the game development process called “momentum.” It is when you finish working on something, feel good about it, and are ready to work on the next thing with a lot of motivation and confidence. A possible definition of momentum is “the feeling of progress that inspires people to make more progress.”

Momentum is the core behind the philosophy of “Less Talk, More Rock.” Going directly from vision conception to creation lets you keep much of the momentum of having an awesome idea without giving it a chance to cool off with too much talking. Talking, by itself, isn’t actually unproductive - but it doesn’t feel like real work. And the more you talk, the more the completion of your vision seems to drift further away and the less momentum you will have.

However, this doesn’t mean that simply working hard on building your game translates directly to more momentum. It seems that in 48 hour game jams, there’s always a period of depression or slump around the 30 hour mark. It’s not that people aren’t working or creating progress - game jams are one of the most productive periods of game development. However, during a game jam the loss of momentum with create the feeling that the project will never be complete and give birth to thoughts of throwing in the towel and giving up.

Momentum is part of the reason why agile development works. Agile development is able to spread the “talk” among the “rock.” Unlike in the middle of a slump during a game jam, a good agile process should constantly remind people how much work has been done and what impact your current work has on the overall product.

Here’s a quick case study. Momentum is disrupted when your progress suddenly dips, and this is especially the case for games that need to break down and reinvent themselves. Take Super Time Force for example. Even though it started as a fairly successful game jam game (and the team felt really motivated after the jam), the need to reinvent its core concepts impacted its momentum and for a while made the team feel that no progress was being made. On the other hand, Goat Simulator really kept its momentum from its game jam by forcing the team to create a full game in no more than two months.

So what makes momentum work or not work? How to we keep momentum constant and at a good rate?

The first point is to not overwork. As in the game jam example, people who are too tired are not easily encouraged and can easily become depressed. No matter how much progress has been made, tired people will be thinking and concentrating on rest instead of even more progress.

A good way to increase momentum is to externalize your work so it is easily presentable to other teammates by tackling specific problems. Don’t directly tackle nebulous topics such as “the entire story” or “the tone of the game.” Split them into smaller topics so you can easily say “this is the progress I made this week” and list out the specific design choices that you decided on. Having short and reasonable deadlines help a lot.

Along with that, a usually good idea is to make easily implementable work. In other words, work on things that are only relevant at the moment and can be implemented in game as soon as possible. Even though planning ahead is important, work that isn’t implemented often doesn’t feel tangible and complete.

Besides having presentable and easily digestible work, it’s also important to let people see it. For the team, having regular times to show work helps momentum. Also, showing the work to people outside the team is also a good idea to feel more motivated. Supercapy Games founder Nathan Villa once mentioned that showing the game to excited people at shows and conferences helped them escape their slump.

I think momentum has a huge psychological effect on the development team, and that’s partially what makes design by committee is so difficult. In misguided design by committee practices, people will be required to brainstorm different individual solutions to one problem. Then a meeting is held to narrow down those solutions to one. Even though progress has been made, it often feels like momentum is lost because so many work is thrown away.

It’s possible to develop and ship a game without a lot of momentum, but having it does make game development process a lot more pleasant. Momentum is all about maintaining a balance between showable progress and work. In other words, having good momentum is essentially finding the “flow” of game development.

Thoughts on Pre-production

Our team spent a big part of this semester doing pre-production. Here are some of the lessons we learned:

Pre-production is an important part of the game development process that is often overlooked. It is a deceptively easy task.

Different developers approach pre-production in different ways. In the best case scenario, a small group from the team is set aside to do pre-production for the next game IP that will happen several years down the road. However, in most cases developers have a limited amount of time that is shared between pre-production and production, and balancing the two is a hard task.

The big question is: how much is too much or too little pre-production?

Doing Too Little Pre-production

        Pre-production is planning for the project, and having little or no planning is often a bad thing. There's an entire Extra Credits episode dedicated to the dangers of doing too little pre-production. Without a style guide, artists on the team will end up producing assets of different styles. Programmers who jump in production too fast will end up with pieces of systems that don’t work together properly. More importantly, without a central shared vision individuals will have trouble making key decisions because they don't feel like they know what the final product should be like.

Doing Too Much Pre-production

With short projects with close deadlines (such as BVW or semester long projects), it's difficult to fall too hard into the rut of doing too much pre-production. But with year-long projects or projects that have an indefinite deadline, it's easy to accidentally go overboard.

This is a problem to especially to new developers. Without strict deadlines (as the way it is during production), productivity can drop drastically. Often a project with too much planning won’t even make it to the production stage.

       Another problem is that a lot of lessons are learned through the iteration process of building the game. New problems and solutions pop up during the production process that hazy or undefined during pre-production. Very few games end up exactly as they were planned. Having too much planning means throwing away a lot of work when those plans don't actually pan out in reality.

       

The Meat of the Problem

Speaking from experience, my biggest fear with pre-production is trying to answer too many questions. Pre-production is used to help increase the chances of success for final product, but sometimes it can be a sinkhole in and of itself. For example, here are some of the problems I wanted to solve by the end of my pre-production process:

       - Is this idea interesting? Is it compelling to a large enough demographic? 

       - What's unique about it? Can I explain the game in one sentence that captures people's interest?

       - Is the art compelling enough to attract people on its own? Is the narrative?

       - Are the elements of gameplay, story, and narrative cohesive as whole?

       - Is the art style distinguished enough to be easily recognizable?

       - Is the game different enough from other similar games?

       - Is the game in scope with the size of the team? 

       - Is the team motivated by the game? 

       - Is there a good chance this game will go viral?

       - Will it make money?

       

Unless there is special time set aside for a small group of people to just work on pre-production for a long time (the same way animation studios work on a concept in small teams a few years before the bulk of the production work is done), it will be difficult and almost impossible to answer all these questions in a timely manner. Since all these questions are intertwined together, it’s hard if not impossible to answer all these questions at the same time either.

Creativity Deadlock

It’s easy to fall in a creativity deadlock when you try to answer all these questions all at the same time. Creativity deadlock is when different parts of the creative process rely on each other to be created first in a cyclic fashion. For example, suppose your goal (as the lead artist) is to find a distinguished and compelling art style. However, the art style is heavily influenced by the narrative of the game. And in most games, the narrative (being the most malleable element in the game) is molded around the rest of the game. And you can’t build the game first without art.

So instead of trying to answer all the uncertainties about the game during pre-production, is there a better way to do pre-production?  There is no single answer, but there are some key points to remember about pre-production.

Cohesive Vision

          The single most important part of pre-production is to make sure the team has as close a vision to each other as possible. Game ideas are notoriously hard to be interpreted in the same way. Without having the same vision, people will have trouble creating cohesive work. It’s very time consuming when problems of having different vision pop up later in the process.

An unfortunate aspect of getting to a cohesive vision is to have a lot of team meetings. It’s hard to tell if people are on the same page without talking and discussing together, and shared documents can easily be interpreted differently. Usually, having a lot of meetings seems unproductive (especially during the production cycle), but meetings are an essential part of the pre-production process unless you are working alone.

Trust the Team

          A big part of walking into production without answering all the unknowns is trusting the team. If you have confidence in the team, you know that they can create a good game without the path to success completely laid out to them. Trusting the team to iterate well during production is already a given, so it’s also possible to trust them a bit more in unwrinkling the kinks from pre-production.

Learning by Building

          A lot of game design philosophies such as Jason Vandenberghe’s “Fail faster, and follow the fun” and Superbrother’s “Less talk, more rock” are telling people the same thing. A lot of design can only be done by building and iterating on the game.

          Prototyping is a big part of pre-production, but know that prototyping can’t answer all the questions. If you do build enough prototypes to answer all your questions, you are more or less in production already.

Have a Clear Reason for Success

          While it’s okay to be uncertain about success, it’s important to have one or two good reasons why your game will succeed. For example, a lot of different people give a similar piece of advice to indie developers: before you build your game, know what is unique about it and how to market it. Likewise, if you are creating a game for a client, know what part your client likes about your game and try not to change that.

Motivation for the Team

          A team should be inspired by the end of pre-production and excited to start working on the game. If your artists are not inspired, they will create less interesting work (or think that they are creating sub-par work). Sometimes this happens because their art style doesn’t really fit the game vision created from pre-production. Having motivation helps improves production efficiency and quality.

To conclude, here is a checklist of some things to have by the end of pre-production. Not all of these are necessary (and a lot of more things you need are not on this list), but I think these are some pretty important ones that I haven’t mentioned so far.

-         Know who is responsible for important decisions

-         Have an art style guide

-         Set up an art asset pipeline

-         Know what technology you are using and why

-         Have short prototypes to show gameplay works

Portal: More Than Just a Twist

Surprise, Surprise...

Why should you make your game surprising? As a whole, humans love surprises. We spot patterns quickly and get bored when things become predictable. And we are experts at predicting. Game developers (and other entertainment creators) have a hard time because if they start to lose customers if they serve the same dish twice.

Maybe it's why rogue-likes are so popular lately - unpredictable elements and possibly emergent gameplay. Sprinkle a bit of addictive player feedback loops and toss in a pinch of challenging gameplay. Stir three times and you've got yourself a game!

In other news, I am getting a bit hungry.

A Case Study on Portal

When people talk about surprises in video games, they often talk about about Portal. Now, if you haven't played it yet, stop reading and go play it.

Midway through Portal, there's a giant twist. And by most accounts, this is a pretty effective twist. Good game designers ask hard question, so this is the question of the day:

Why is the twist in Portal effective and how do we apply it to other games? 

This is a question I've pondered about for quite some time. I used to think that the twist in Portal was only based on narrative, but recently I've noticed that that was only a small part of it. There are plenty of games with twists in their story, but Portal somehow feels more special than that.



Let's take a look at the structure of puzzles in Portal.

Most puzzle games, from Angry Birds to World of Goo to Scribblenauts, have the same general structure.
They are a series of isolated puzzles that have about the same chunk size for each puzzle.

How most puzzle games are structured

However, Portal is structured like this:

How Portal is structured

Portal starts off in a rather traditional puzzle structure. It even has a "level number" for each puzzle. But after the midpoint the concept of isolated "test chambers" melt away. It's just a continuous long environment you travel through with implicit puzzles interlaced along the way. 

There are also a lot of differences to the twist besides puzzle structure. A big part of the surprise in Portal is setting up expectations and breaking them. This comes through in the tone, environment, and narration of the game. Most of these come across pretty clear: the first 20 levels are clean perfect testing chambers, and the rest of the game is played - the dingy and dirty spaces behind panel and testing facilities as well as the empty office spaces. 

Even the narration style is different. In the first 20 levels, the narration feels very hand-holdy. Something is said at the beginning of the puzzle, at the end of the puzzle, and maybe even while you ride an elevator. Later on, it feels that Glados is actively stalking you.

Crossing Genres

And this leads me to one hypothesis: 

In the player's mind, Portal changes from one genre to another

In other words, the game actually crosses genres. In the first half, it's just a puzzle game with a funny narrator. In the second half, it becomes an adventure game. The thoughts in the player's mind change from:

"Huh. How do I solve this puzzle?"

 into 

"HOW DO I GET OUT OF HERE. I HOPE GLADOS CAN'T SEE ME. WHERE DO I GO??"

Portal changed from a game being clearly rooted in one genre, and just about when you think it ends, changes into another game. And it doesn't cross genres half-heartedly. Unlike most other games, it doesn't dabble in another genre for a few brief moments before coming back to its original genre. Portal crosses that bridge doesn't look back.




And maybe that's the key to creating a twist or surprise on the scale of the one in Portal:

Make the first half of the game seem like one genre, 

and make the second half seem like another genre.

And when done correctly this could be very powerful. Think about a military shooter game that sudden becomes a stealth game after the first half of the game. Or better yet, a military game shooter that becomes an open world survival game. Ideally, games that cross genres would keep the same gameplay mechanic.

What if a stealth game like Thief suddenly became an assassination game based on revenge? One interesting thing is that to make these changes in gameplay genre will only make sense if you have a powerful change in narrative. This could be an interesting tool.

For example, consider going from "zombie survival horror" to "adventure". The twist might be getting bitten by a zombie and volunteering (or being forced) to leave the human compound try to do the most good possible before your time is up. The gameplay might change from "scavenging and running away from zombies" to "exploring the post-apocolyptic world looking for a cure or another human compound."





And...that's just a random thought I was having. Have a nice day.

Creativity Deadlock

After two weeks of pre-semester brainstorming and four weeks on discussion into the semester, we're having a bit of a problem: We have no idea what the game should look like.

At the moment, our project team is having trouble with artistic vision. It's one of those things that we didn't think would be a problem. Here is the basic premise: The art needs to be inspired by the narrative. The narrative needs to fit around the entire game. And we can't make the game without art. And this, ladies and gentlemen, is what we call a creativity deadlock.

The opposite of this

How do we break out of a creativity deadlock? The easiest way...is to just come up with everything at once! Just have everyone sit down and nail down the art, narrative, and entire game at the same time. But if you've done any team game development at all, you know that it usually doesn't work that way. Nothing comes together all at once. Even if you suddenly have a cohesive vision (which we struggled with from the beginning), that vision will indubitably change throughout the building process.

Another way to approach a creativity deadlock is to work from different angles. Experiment with art and narrative on their own and let them cross-pollinate each other. We also tried this approach for a few weeks, but it seemed that the pieces we came up with did not quite come as coherently together as we hoped. For example, our concept artist (who bases all his art  To be honest, I think this process could work if we had a lot of time to iterate, but we're feeling pressure to create things for GDC.

IGF Pavillion at GDC (photo credit to IndieGames)

Then are there any solid decisions for the game that won't change? Yes. One is the puzzles we have, and the other is having a jazzy soundtrack. But herein lies the problem.

The puzzles we have are based on a large variety of object, often not connected at all. Our game is supposed to give the feeling of unexpectedness and humor through the puzzles, and part of those feelings come through the context of the puzzles. However, at the moment, the context between the puzzles are not related, and this makes finding a common theme between the puzzles very difficult.

Also, "jazzy" turns out to be a very confusing term. Some team members immediately understand my interpretation of the word down to a T. Other members have different ideas about jazz or . However, this isn't really surprising - there's a huge variety of jazz.

Some concept art

Other things that we tried:

Making a ranked word association list. The idea is, as a group, create a list of words related to your game and rank them based on how important they are. This helped us inform each other what we were all thinking of and discuss which direction our game should head in. However, it still wasn't concrete enough to build things from.

Some other things I could have tried:

Instead of having one song or one picture to act as inspiration for the game vision, create a gigantic folder of stuff that give the atmosphere or feel of the game. Therefore, you only need a glance to get a comprehensive feel of the game. Include things like pictures, colors, and music. Of course, a danger with this approach at this point is that it might be too "blue-sky" and pull progress back again.

Word association graph

So what do we do?

Another possibility is that getting stuck all the time is part of the process. After all, we ARE asking a lot from our art: it has to be recognizable and stylish and unique that can be easily produced. Also, it can't be too similar to Portal, Scale, Antichamber, and Unfinished Swan. (In my opinion, Portal really nailed it in terms of design, theme, and narrative. Unfinished Swan was smart in picking a )

Maybe the key is to just try something - and not be afraid to fail and redo it until it feels right. Narbacular Drop took 1 year to produce and Portal required an additional 2 years and 8 months.The Unfinished Swan took about 4 years to make from prototype to full game...and let's not even talk about Fez.

I guess...we'll see what happens.

A Short Rant about Third Person Camera Angles in Video Games

Let's talk about something not related to anything.

Third person camera angles.

There's a lot of video games that use the "default" third person camera. You've seen them. The games where they place your character at the center of the camera and use your character as a pivot point.

It looks something like this:





There are some obvious faults with this type of camera system. The most general one is that it blocks your line of sight. Also, the view angle is not ideal in a lot of situations. When you move the camera down (as in moving your camera to almost parallel to the ground), almost half the screen is taken up by the ground close up. The other half is taken up by the sky, and you have a hard time telling apart depth. On the opposite side of the spectrum, if you rotate the camera up (and view from a higher ), all you get is the ground and you can't see objects far away.

This happens to be the camera system that a lot of MMOs adopt.

(Notice how some of these problems are not very obvious in Zelda: Ocarina of Time because when you need to look at anything the auto-targeting system takes over.)

However, a lot of games do something that make the camera angles a lot better. They simply raise the pivot point of the camera up a bit.

As you can see, Skyrim's camera is slightly better. The camera is looking more towards where the player wants to look instead of directly at the character. It let's you see more stuff and admire the view.

However, there are some practical concerns with this system when you start rotating the camera. The thing is, camera systems will limit the angle of rotation of their camera. For example, some won't let you rotate the camera up too high.

Now, suppose you are on a short inclined hill (as you often are in Skyrim). The ideal angle would be to look over the hill. However, the rotation limit locks your camera so you can only get a good look at the sky and not what's over the hill.

(Ok, so you think I'm probably a bit harsh on Skyrim. It's true that it has 3 camera modes: First-Person, Medium Range Third-Person, and Long Range Third-Person. And the one shown here is probably the least used.)

What I'm saying is, with a simple camera system there's always a bunch of compromises you have to make. But the experience can be so much better with a dynamic camera system that adapts to the player's actions.

Let's take a look at some screenshots from one of my favorite games, Shadow of the Colossus:







The effects of a better camera system are pretty clear. Almost every screenshot in you get while playing Shadow of Colossus looks like it could be a wallpaper. Not only does it look great when they nail the Rule of Thirds, having

Also, this is what the game looks like when you are playing. These screenshots aren't taken from special angles that aren't practical during actual gameplay.

(I'm looking at you, Skyrim)


As you can see, there are plenty of different ways you can improve from the default third person camera.

Here are a list of quick tips:

1. Don't focus on the character. Focus on what the player wants to look at

2. Don't put the pivot point of the camera on the player.

3. Change camera distance and angle when you want to show difference in size and scale

4. Change the field of view when you want to focus on something.

And that's the end of my rant.

Lessons from Building Virtual Worlds

Building Virtual Worlds (BVW) is a class at the Entertainment Technology Center (ETC). If you haven't heard of either, you might have heard of Randy Pausch, one of the founders of ETC and the BVW class.

The BVW class focuses on rapid prototyping and teamwork. In 4 months, I worked and completed 6 team projects. During the process, we worked with technologies such as the Kinect, PS Move, Eyegaze (an eye-tracking device), and Phidgets. We also worked on custom ETC platforms such as the Jam-O-Drum and the Bridge.

However, the most valuable part of BVW is not the technical skills you learn. The important things are the lessons you learn from working with other people.

So, here are the most important lessons I've learned from BVW:



1. Keep an open mind about "Game Ideas"

It's hard to explain your cool idea to someone, and it's also hard to understand why someone else thinks an idea is cool. Sometimes things just click and you "get it." A lot of times this doesn't happen.

I can't tell you the number of times that I thought an idea was a bad idea, only to realize a few days later that it was "the best idea ever."

So, two pieces of advice: don't try to stonewall other people's idea - but do ask for clarification. And if you are explaining your idea to someone else, really try to explain why an idea is cool to you. Don't expect people to be excited about an idea if you just describe what the game is about.



2. If you know you need more time, lower your scope

When you realize you have less time than you need, you usually try to adapt by working harder. "It's okay. I can pull a few all-nighters."

This works for a few cases, but "working harder" can only take you so far. You might be able to cut it just barely before the deadline - but that's not good enough.

That's because the final stages of development - including polishing and testing - have a huge impact on the end product. Something you finish right before the deadline will have quite a few rough edges, and maybe a few cracks as well. Usually, it's much better to have a polished product with less content.

It usually doesn't take a lot to put the finishing touches on a game - but it's very obvious when you don't.



3. The things you care about are different from what the audience cares about

You know that little problem in your game that really bugs you? That graphical glitch or color scheme that you spent ages tweaking but just couldn't get completely right?

The player isn't going to notice that at all.

Instead, he going to have strong opinions on other things that you got used to already - such as the floatly controls or the lack of clear instructions.

As a developer, there are certain things you will obsess about. But spending a lot of time on them isn't the most efficient you can do to give players the best experience they can have.



4. Keep a Priority List

    A priority list is basically a list of tasks you need to do to complete your project with a rating of how essential each task is. It works best if the list is shared with everyone (for example, by using Google Docs).
 
 It helps you...
       Keep track of all the things you need to do - so you won't accidentally overscope
       Makes sure you work on the important things first
       Allow members to have a simple way to distribute tasks
       Keeps everyone notified of progress
       Makes you feel accomplished when you cross something off the list