A HLSL Shader Toolkit for SWBFII - WIP

[SleepKiller]

Apologies if this is the wrong location for this, feel free to move it to a more suitable home. I'm sure I'll find it one day.

This is another one of my fun hobby projects relating to SWBFII, which for some reason I find endlessly enjoyable to mess with. The idea is fairly straight forward writing shaders in assembly sounds hard and painful, writing shaders in HLSL sounds hard and painful. But since HLSL is easier to read and write itself I thought it would be nice to be able to edit the game's shaders in it. Thus enters this project to create a build process to compile and munge HLSL shaders and use them in SWBFII instead of the stock assembly ones.

The build process is all but complete and now I am just slowly porting the shaders one by one to HLSL. I've never programmed in HLSL (I'm a GLSL man personally) before and I'm not a graphics programmer (I barely qualify as any form of "programmer" tbh) by any means so it is a challenging process for me. But regardless of that I have been making good progress on it reasonably consistently and I kind of felt like seeking out what interest there was in the small SWBFII community for this. Although I am just making it because I want to edit the shaders myself it's still helpful to know how many (if any) people are interested in it. Hence a WIP thread.

Screenshots would be dull since at this stage if everything is progressing fine and working correctly you can't tell the difference between my rewritten shaders and the game's stock ones. So have some code to look at instead of here.

Although I did have some fun with the rain shader a couple days ago. Isn't it pretty?

It's out. Find the release thread here!

[Marth8880]

Very cool. Will keep watch over this!

[SleepKiller]

As an update for anyone interested the toolkit now has HLSL versions of most of the games "rendertype" shaders. (21 out of 30 for anyone that cares) The most recent of which is the "normal" shader which (outside of mods) is responsible for the majority of the game's rendering. Once again a screenshot would be pointless since it looks identical to the stock game. The toolkit is very much well on it's way to completion however after which hopefully someone (be it me or another) does some cool things with it.

[Anakin]

well if it looks the same as before i didn't get the point how this improves the game

I wrote a little Mesh Viewer with opengl and those HLSL, GLSL things. Not really understandable to me

[SleepKiller]

Yeah I can see how it might seem confusing. "So you're rewriting the shaders and they look exactly the same? What is the point?" And you're absolutely correct this in it's current state doesn't improve the game but me explaining why I am making it should make everything make sense.

Well I want to clarify that the modtools shipped with the game's entire suite of shaders. However they're all written in D3D Shader Model 1.1 assembly. Which (as a simple example from the lightbeam shader) looks like this.

Code: Select all

#include "pcRedVertexShaderMacros.h"

; project the position into screen space		
POS_PROJECT(oPos)

; calculate near scene fade factor in R_TEMP.w
NEARSCENEFADE(R_TEMP)
FOG(R_TEMP.z, R_WORLD_POS, R_TEMP1)
; clamp to [0.0,1.0] and square it to fade transparent objects faster
max R_TEMP.w, R_TEMP.w, c[C_CONST0].x
min R_TEMP.w, R_TEMP.w, c[C_CONST0].z
mul R_TEMP.w, R_TEMP.w, R_TEMP.w
; output color with lighting and near scene fade
mul oD0.xyz, R_MATERIAL_COLOR.xyz, c[C_HDR].zzz	
mul oD0.w, R_MATERIAL_COLOR.w, R_TEMP.w

Which isn't very readable or easy to work with. Here is the same lightbeam shader in HLSL, the difference in readability should be apparent.

Code: Select all

Vs_output lightbeam_vs(Vs_input input)
{
   Vs_output output;

   float4 world_position = transform::position(input.position);

   Near_scene near_scene = calculate_near_scene_fade(world_position);
   near_scene.fade = near_scene.fade * near_scene.fade;

   output.position = position_project(world_position);
   output.fog = calculate_fog(near_scene, world_position);

   near_scene = clamp_near_scene_fade(near_scene);

   float4 material_color = get_material_color(input.color);

   output.color.rgb = material_color.rgb * hdr_info.zzz;
   output.color.a = material_color.a * near_scene.fade;

   return output;
}

I do want to be able to mess around with the game's shaders for fun and see how I can change the look of the game. In order to do that I first want to understand the game's shaders/render pipeline and then I also want to be able to easily edit the shaders, something they need to be in HLSL for me to do. Hence the creation of this toolkit! Rewriting the shaders lets me learn how they work and how the game uses them while also giving me the HLSL versions of the shaders I want.

[thelegend]

The difference is clearly there. I am one of those people who can't get to understand how these complex things were written and how they actually work. I am impressed by this and I am curious if this allows us to have improved shaders as well as improved rendertypes (as you stated earlier). I don't even know anything about shaders at all but If this allows the game to look better or maybe more realistic...or just in a complete different way, as anyone want it to have it be...then I am really looking forward to this. Also I like the rainbow rain you got there

[The Nasal Abyss]

If I could rise from the grave for a moment and make a quick request for those that still modify this game and want graphical improvements; Zero Engine has never made proper use of normal maps. They are always calculated as simple bump maps even if they have proper bakes and RGB data (which you might have already noticed whilst perusing the code). If you could even just make the base shader calculate normals from a map properly that would improve visual fidelity by an incredible amount. Also, something I have always felt held this game's visuals back was the shadow color being locked to a dark grey. If shadows could be tinted with color to match a map's given sky color through a simple RGB value that would be another instant improvement. Don't know if that is handled through a shader or not, though. I think it might not be. Sorry if these things aren't possible and I'm wasting my words; I could never make heads or tails of the shader code.

Nonetheless, this is a very cool project. The first of it's kind, as far as I'm aware. Teancum, Mav, myself and others have expressed desire to possibly try and find out as much about these shaders in the past and modify them but I for sure never got around to it. Keep up the great work!

[thelegend]

@The Nasal Abyss: That's a pity with normals in this game. Rendering them as they look like would definitely be a huge step forward. Also about colorful shadows: http://www.gametoast.com/viewtopic.php?t=236
I know it's for BF1 but shouldn't that work for Bf2 too as well?

Hidden/Spoiler:

ShadowColor-----------the name says it all. It colors the shadow.
The strange thing is that it colors green with RGB 255,0,0 which normaly makes all reddish.

[Anakin]

now i understand what you are doing. Great job.

About the normal maps, it's really a thing of the shaders. When i worte the new Mesh Viewer tool, i implemented normal mapping, too and it looks much better in the viewer then in the game

[The Nasal Abyss]

@The Nasal Abyss: That's a pity with normals in this game. Rendering them as they look like would definitely be a huge step forward. Also about colorful shadows: http://www.gametoast.com/viewtopic.php?t=236
I know it's for BF1 but shouldn't that work for Bf2 too as well?

Hidden/Spoiler:

ShadowColor-----------the name says it all. It colors the shadow.
The strange thing is that it colors green with RGB 255,0,0 which normaly makes all reddish.

It does not, unfortunately.

[SleepKiller]

If I could rise from the grave for a moment and make a quick request for those that still modify this game and want graphical improvements; Zero Engine has never made proper use of normal maps. They are always calculated as simple bump maps even if they have proper bakes and RGB data (which you might have already noticed whilst perusing the code). If you could even just make the base shader calculate normals from a map properly that would improve visual fidelity by an incredible amount.

Funny you should say that. Due to difficulties I ran into with the shader that handles per-pixel lighting and "normal mapping" I revoked it's algorithm privileges, sent it to it's room and decided to implement it myself from (more or less) scratch. (I found Pandemic's code hard to follow and understand, and I was unsure what kind of textures is was looking up and when. Even a HLSL implementation wouldn't have been very useful to people looking to change the look of the game.)

As a result I actually already have implemented normal mapping for the game. Well, probably, I'm still not a graphics programmer so it's probably hilariously incorrect. But it is technically a normal mapping implementation. (It's definitely more correct than the one the game was using at the least.) There is the catch of it makes things a little darker than they should be. I'm not sure if that due to the way I'm calculating the lighting or because of the way SWBF blends the diffuse map in with a different render pass. Either way it'll likely need tweaking but I think it looks better than what the game had before. Probably.

On another note I now only have three shaders left to implement which are water, stencilshadow and specularlighting respectively. My earlier statement of having done 21 out 30 of the shaders included deprecated ones that I don't think the game even uses, so I'm going to skip those ones for now, leaving them out of core.lvl doesn't seem to cause crashing and I suspect to even get the game rendering using them would take some hex editing trickery of the munged files. (I could be wrong and if I am I'll go back to them.)

Also since the most recent shader I implemented had an incomplete version in the modtools I spent some time poking around inside the munged shader's file format. I believe it should be possible (and almost trivial) to replace pc_shadermunge with a custom one, which in theory would enable the use of Shader Model 3.0. (Currently we're limited to 2.0.) Which in of itself opens up a lot of intriguing prospects like Relief Mapping and such.

[thelegend]

Good job on this
For normal mapping, here's a good pic:

Hidden/Spoiler:

Basically from what Nasal told, the game can't properly render normal maps as they should be. This normal map could render a very detailed ground:

Hidden/Spoiler:

In modern engines the rocks would pop out and gives the player/observer the feeling that the entire ground is modelled. There are different colors, which alter the "height" of the 3D effect. In BF2 this 3D effects doesn't exist as it simply "bumps" holes into the texture. I know my describing skills are decreasing by a lot but from what I am talking is, and what Nasal already stated; the game and especially all mods/maps which use huge amounts of normal/specular maps, would drastically look better, when 3D effects were rendered properly.

Also important to add are "round" normals, which have a very nice, almost satisfying, transition from one main color, to the next one. I think just Red and Blue are the only height maps. Here's another neat example:

Hidden/Spoiler:

[Marth8880]

I believe it should be possible (and almost trivial) to replace pc_shadermunge with a custom one, which in theory would enable the use of Shader Model 3.0. (Currently we're limited to 2.0.) Which in of itself opens up a lot of intriguing prospects like Relief Mapping and such.

Talk dirty to me.

[Anakin]

A working normalmapping would be awesome.

btw here is my shader (glsl) i used for the viewer. Not sure if this helps you to understand some parts of the shader things

[Lorul1]

Hmmm ... what exacxly does this tool allow us to do again ???

[Ginev]

Cool!If you made better normal mapping then im sure that my version of the free radical Tatooine buildings will look amazing.Keep up the good work.

"You are killing ewoks while you are sleeping"

[SleepKiller]

So because of constantly hitting the pixel shader instruction limit I spent almost all of my weekend writing and debugging a replacement munger that uses HLSL directly and Shader Model 3.0. (You can find it's source in the Git repository.) Doing that allowed me to properly implement normal mapping (well as far as I can tell at least...) and new specular lighting. Leaving only water and stencil shadows left to be rewritten.

If you want to take the fancy normal mapping for a spin you can grab this (link removed, pending new version) core.lvl containing my last build of the shaders. (These shaders won't work %100 correctly, backup your core.lvl before using mine.)

You may notice odd lighting on terrain that has a normal map. That isn't actually the normal mapping rendering incorrectly, it's a different render pass coming from the terrain shader itself causing it that I haven't had time to investigate and fix yet. And fog is also broken on a lot of things for now because Shader Model 3.0 doesn't support using the fixed function pipeline's fog. (Which I'll be honest I'm not sure how best to work around yet.)

Hmmm ... what exacxly does this tool allow us to do again ???

If you know what a shader is I explained it in an earlier post here. If you don't know what a shader is a little Googling should lead you to far better explanations than I can provide.

EDIT:

I've more or less finished it, there are a couple things I want to fix before I call it done. The Git repository now has documentation for the toolkit in it and furthermore if you wanted to take it for a spin you can grab a ready to go "RC" version of it here. (Link removed, use the release version!) Extract that into your modtools folder and then run either build.bat or build.ps1 to create a core.lvl containing the custom shaders. If you run into any problems with it feel free to let me know.

[0dium]

Hey, SleepKiller. Can your shaders use proper normalmaps or just those b/w bumpmaps that are already in the game files?

[SleepKiller]

As it would happen texturemunge actually generates normal maps from the height maps (which as I understand is a reasonable way to create normal maps). So yes, the toolkit's shaders can and do use proper normal maps. (And if you tried to pass a bumpmap instead to even the stock shader I think it would give incorrect results.) I can't speak for the quality of implementation of the normal mapping shader in the toolkit, none of this is in my comfort zone, so it is possibly laughably incorrect. But I have tried to make sure it uses normal maps correctly and well.

Both the Refraction and Water shader are exceptions to using normal maps. They both used signed bumpmaps for their distortion effects, although I did make sure the Refraction shader in the toolkit to worked with either.

[0dium]

[*]Tried to munge a map with one bumpmap replaced with a proper normalmap and it worked just fine. Well there is no point in converting heightmap to normalmap by yourself if it happens automatically in munging process.
[*]Sometimes lighting looks weird on surfaces but it's not a big problem.
[*]One problem is that a lot of surfaces still look flat and relief can only be seen when dynamic light sources are nearby (rifle shots, explosions). Sometimes relief can be seen under a sunlight but only in rare occasions. And that is not a very realistic behavior. Looks like sunlight must be dropping from a very specific angle or IDK.