Code Snippet: Edge Detector/Antialiasing Shader

Some days ago i was looking for a simple and effective edge detector idea to use in an AntiAliasing shader for Deferred Shading (ideas that effectively make use of the Normal and the Depth Buffers).

Two called my attention: The first one is from the game Stalker, this page from GPU Gems explains the whole idea (nothing very complitaded =P, they show lot of the code). The results are pretty cool, as you can see in this image for example. The second is the one used in Tabula Rasa game, this page from GPU Gems shows the whole idea.

Stalker Idea

The shader code in HLSL is: (added the full version. for those interested only in the edge detection part, see the shader comments).

void PassThroughVS(inout float4 position : POSITION0,
                   inout float2 texcoord : TEXCOORD0) {

				   position.x =  position.x - pixel_size;
				   position.y =  position.y + pixel_size;

float2 e_barrier = float2(0.8f,0.00001f);  // x=norm, y=depth

float2 e_weights= float2(1,1);  // x=norm, y=depth

float2 e_kernel = float2(1,1);   // x=norm, y=depth

const static float2 offs[7] = {
   float2( 0.0,  0.0), //Center       0
   float2(-1.0, -1.0), //Top Left     1
   float2( 1.0,  1.0), //Bottom Right 5
   float2( 1.0, -1.0), //Top Right    3
   float2(-1.0,  1.0), //Bottom Left  7
   float2(-1.0,  0.0),  //Left         8
   float2( 0.0, -1.0) //Top          2

float4 PShader2(float2 uv : TEXCOORD0) : COLOR0


 // Normal discontinuity filter

 float3 nc = tex2D(NormalMap, uv);

 float4 nd;

 nd.x = abs(dot(nc, tex2D(NormalMap, uv + offs[1]* pixel_size).xyz));

 nd.y = abs(dot(nc, tex2D(NormalMap, uv + offs[2]* pixel_size).xyz));

 nd.z = abs(dot(nc, tex2D(NormalMap, uv + offs[3]* pixel_size).xyz));

 nd.w = abs(dot(nc, tex2D(NormalMap, uv + offs[4]* pixel_size).xyz));

 nd -= e_barrier.x;

 nd = step(0, nd);

 float ne = saturate(dot(nd, e_weights.x));

 // Opposite coords

 float2 tc5r = -offs[5];

 float2 tc6r = -offs[6];

 // Depth filter : compute gradiental difference:

 // (c-sample1)+(c-sample1_opposite)

 float dc = tex2D(depthMap, uv).r;

 float4 dd;

 dd.x = tex2D(depthMap, uv + offs[1] * pixel_size).r +

         tex2D(depthMap, uv + offs[2]* pixel_size).r;

 dd.y = tex2D(depthMap, uv + offs[3]* pixel_size).r +

        tex2D(depthMap, uv + offs[4]* pixel_size).r;

 dd.z = tex2D(depthMap, uv + offs[5]* pixel_size).r +

        tex2D(depthMap, uv + tc5r* pixel_size).r;

 dd.w = tex2D(depthMap,uv +  offs[6]* pixel_size).r +

        tex2D(depthMap,uv +  tc6r* pixel_size).r;

 dd = abs(2 * dc - dd)- e_barrier.y;

 dd = step(dd, 0);

 float de = saturate(dot(dd, e_weights.y));

 // Weight

 float w = (1 - de * ne) * e_kernel.x; // 0 - no aa, 1=full aa

 //return float4(w,w,w,1); ///stop here for an edge detector shader

// The AA part (in edges we mix close pixels, non edge pixel w = 0 => use the normal uv, on edge we use a mean of four deslocated uv texture reads )
// Smoothed color

 // (a-c)*w + c = a*w + c(1-w)

 float2 offset = (uv ) * (1-w);

 float4 s0 = tex2D(TextureSampler, offset + (uv + offs[1] * pixel_size) * w);

 float4 s1 = tex2D(TextureSampler, offset + (uv + offs[2]* pixel_size) * w);

 float4 s2 = tex2D(TextureSampler, offset + (uv + offs[3]* pixel_size) * w);

 float4 s3 = tex2D(TextureSampler, offset + (uv + offs[4]* pixel_size) * w);

 return (s0 + s1 + s2 + s3)/4.h;


technique AntiAliasingStalker
    pass Pass1
		VertexShader = compile vs_3_0 PassThroughVS();
                PixelShader = compile ps_3_0 PShader2();

The normal buffer is stored as a 32 bits RGBA texture. The Depth Buffer is stored in a 32 bits Single Texture (storing post-projection z/w). If you use another depth enconding, just adjust the barrier variable value.

The pixel_size variable (a float2 with 1/target width in x and 1/target height in y) is used in DirectX 9 Shaders (Opengl and DirectX > 9 does not need this). The need for this “correction” is very well explained here.

The ScreenShot below is just the edge detector part of the shader. (just uncomment the following line in the shader and you will have a powerfull edge detector shader =P)

Example using this good edge detector shader. Cool for toon shading =P


Idea From Tabula Rasa

This is not so good as the one before (dont looks good when the camera is far from the objects or when the edges has little pixels), but is a bit simpler. Fort those interested  only in the edge detection part,

just return the “factor” variable in the pixel shader, instead of using it to blend the near pixels =P

const float2 delta[8] =

 ///silly, dummy functions to recover normal/depth (i was doing something with the read values before and i did not erased the function ...)
 float4 DL_GetDepth(float2 uv)
	return tex2D(NormalMap  ,uv);
 float4 DL_GetNormal(float2 uv)
	return tex2D(depthMap ,uv);

float depthSensibility;
float normalSensibility;
   // Neighbor offset table
   const static float2 offsets[9] = {
  float2( 0.0,  0.0), //Center       0
   float2(-1.0, -1.0), //Top Left     1
   float2( 0.0, -1.0), //Top          2
   float2( 1.0, -1.0), //Top Right    3
   float2( 1.0,  0.0), //Right        4
   float2( 1.0,  1.0), //Bottom Right 5
   float2( 0.0,  1.0), //Bottom       6
   float2(-1.0,  1.0), //Bottom Left  7
   float2(-1.0,  0.0)  //Left         8
float DL_GetEdgeWeight(in float2 screenPos)
  float Depth[9];
  float3 Normal[9];
  //Retrieve normal and depth data for all neighbors.
   for (int i=0; i<9; ++i)
    float2 uv = screenPos + offsets[i] * pixel_size;
    Depth[i] = DL_GetDepth(uv);  //Retrieves depth from MRTs
    Normal[i]= DL_GetNormal(uv); //Retrieves normal from MRTs
  //Compute Deltas in Depth.
   float4 Deltas1;
  float4 Deltas2;
  Deltas1.x = Depth[1];
  Deltas1.y = Depth[2];
  Deltas1.z = Depth[3];
  Deltas1.w = Depth[4];
  Deltas2.x = Depth[5];
  Deltas2.y = Depth[6];
  Deltas2.z = Depth[7];
  Deltas2.w = Depth[8];
  //Compute absolute gradients from center.
  Deltas1 = abs(Deltas1 - Depth[0]);
  Deltas2 = abs(Depth[0] - Deltas2);
  //Find min and max gradient, ensuring min != 0
   float4 maxDeltas = max(Deltas1, Deltas2);
  float4 minDeltas = max(min(Deltas1, Deltas2), 0.00001);
  // Compare change in gradients, flagging ones that change
   // significantly.
   // How severe the change must be to get flagged is a function of the
   // minimum gradient. It is not resolution dependent. The constant
   // number here would change based on how the depth values are stored
   // and how sensitive the edge detection should be.
   float4 depthResults = step(minDeltas * depthSensibility, maxDeltas);
  //Compute change in the cosine of the angle between normals.
  Deltas1.x = dot(Normal[1], Normal[0]);
  Deltas1.y = dot(Normal[2], Normal[0]);
  Deltas1.z = dot(Normal[3], Normal[0]);
  Deltas1.w = dot(Normal[4], Normal[0]);
  Deltas2.x = dot(Normal[5], Normal[0]);
  Deltas2.y = dot(Normal[6], Normal[0]);
  Deltas2.z = dot(Normal[7], Normal[0]);
  Deltas2.w = dot(Normal[8], Normal[0]);
  Deltas1 = abs(Deltas1 - Deltas2);
  // Compare change in the cosine of the angles, flagging changes
   // above some constant threshold. The cosine of the angle is not a
   // linear function of the angle, so to have the flagging be
   // independent of the angles involved, an arccos function would be
   // required.
  float4 normalResults = step(0.4, Deltas1* normalSensibility);  

  normalResults = max(normalResults, depthResults);
  return (normalResults.x + normalResults.y +
          normalResults.z + normalResults.w) * 0.25;

float4 PShader(float2 texCoord : TEXCOORD0) : COLOR0

 float4 tex = tex2D(NormalMap,texCoord);
 float factor = 0.0f;

 for( int i=0;i<4;i++ )
	 float4 t = tex2D(NormalMap,texCoord+ delta[i]*pixel_size);
	 t -= tex;
	 factor += dot(t,t);

 factor = min(1.0,DL_GetEdgeWeight(texCoord))*weight; 

 float4 color = float4(0.0,0.0,0.0,0.0);

 for( int j=0;j<8;j++ )
	color += tex2D(TextureSampler,texCoord + delta[j]*pixel_size*factor);
 color += 2.0*tex2D(TextureSampler,texCoord);
 return color*(1.0/10.0);


The Vertex Shader and the buffers used are the same from the Stalker idea.

The first one is more beautiful but is slower.

The PloobsEngine uses these AA shaders (and some others also) to perform an Antialiasing Pass.

This is the first code snippet, when possible i will post more =P


, , , , ,

  1. #1 by on 28 de maio de 2017 - 5:15 am

    Bought that being a gift to my personal mama as well as she adored things. kind good good price tag to my personal mother enjoyed that it. Even sent ultra accelerated. So if you need a gifts quick while such as this than this is someone you will want to invest!

  2. #2 by Lorna on 28 de maio de 2017 - 6:57 am

    Hey there! I just want to give you a big thumbs uup for your excellent information you
    have right here oon this post. I’ll be returning to your website for more soon.

  3. #3 by Hedket on 28 de maio de 2017 - 7:02 am

    Instagram ムービー空書 読み取れかな?カタカナだけとは限らないから、頭を柔軟に、トライしてみてな。 本人が一般的に考えて「×」を押し承諾しない意思を示した。 [url=]produkey windows 7[/url]
    Pass4Testは提供した商品は君の成功を全力で助けさしたげます。 」*織り・縫いのワークショップ* 講師 → わたなべ幸子 日時 → 7/10(日)・11(月)     各日11:30~17:00 (希望の方は~18:30)*陶芸教室* 講師 → 大槻智子 日時 → 7/12(火) 各日 11:30~13:30/15:00~17:00。
    [url=]windows 10 アップグレード[/url] 逆転は不可能だ」と語り、6月の最後の予備選まで戦う意向を明らかにしているバーニー・サンダース上院議員(74)に撤退を求めた。 對美軍所謂否定核武器過去實際發生著其實,實際意味全美軍撤退這樣的事實。
    [url=]windows 8.1 アップグレード[/url]
     2015年に発売した「Apple Watch」「iPad Pro」は、いずれもiPhoneから派生する持続的イノベーションの一種だ。 さらに、アンチ米戦争屋のロシアやイランなどのネットジャーナルは、米戦争屋系マスコミが流さない真相情報を発信しています。 [url=]windows7 ストア アプリ[/url]
    厚労省は当初、自己負担の上限を月3000円~44400円としていたが、患者団体などからの負担増への懸念を受け、月1000円~30000円へと下方修正した。 この問題集よりもっと良いツールは何一つありません。
    [url=]windows8 セットアップ[/url] 打感は芯を食うと柔らかく、ミスヒットしてもミスヒットしたなって硬めの感触は残ります。 今のところほぼ問題ナシです・・動作もわりあい快適。
    [url=]win 10 インストール[/url]

  4. #4 by on 28 de maio de 2017 - 7:05 am

    I got that gift concerning my mom to Holiday due to the fact she is actually the best precious jewelry freak. Some sort of one thing she does not don a great deal of is actually, bracelets. I purchased the lady the allure bracelet to once she launched it yesterday she absolutley loved it! Today the problem is, her getting that upon to down through by herself. Haha… general ideal device, information technology transported then came completely very early furthermore my mom is experiencing that. Thank shoppers.

  5. #5 by cheap louboutin shoes replica on 28 de maio de 2017 - 7:06 am

    it comes down within a breathtaking box appearances great then again it really is a bit smaller truly for the my mothers wrist however it seems awesome just desire that ended up being further

  6. #6 by cheap shoes for sale on 28 de maio de 2017 - 7:29 am

    RIM’s new Blackberry curve 9220, attractively priced at Rs 10,990 is particularly focused at the kids in India.|But once more, this is long term and you will not be in a position to alter it once established.

  7. #7 by on 28 de maio de 2017 - 9:12 am

    I had gotten this headphonesconcerning my personal mom towards mothers day, furthermore she definitely enjoyed things! Their actually adorable headphonesand suggesting in the card which will come within the package is very sentimental!! As well as the top quality of the beads are great!

  8. #8 by on 28 de maio de 2017 - 9:13 am

    I had gotten the headphonesof my personal mother concerning mothers day, and/or she completely enjoyed that it! The really adorable headphonesand the suggesting on the card in which comes into the container is really sentimental!! And quality of beads is actually awesome!

  9. #9 by heremes bracelet replica on 28 de maio de 2017 - 9:51 am

    I had gotten our gift for the mother of Christmas time considering she is actually your jewelry freak. Ones one thing she does not use a lot of is actually, bracelets. I bought the lady this allure bracelet to where she launched things yesterday she absolutley adored that it! Nowadays the problem is, her obtaining this on additionally off through by herself. Haha… total awesome supplement, they delivered additionally came completely early plus our mom are enjoying they. Thank one.

  10. #10 by on 28 de maio de 2017 - 10:36 am

    I got the gift to my personal mom concerning Christmas due to the fact she are the best precious jewelry freak. The actual one thing she does not use a great deal of are, bracelets. I purchased the lady this particular appeal bracelet furthermore whenever she exposed it yesterday she absolutley loved information technology! This time the problem are, her obtaining it on top of plus off of by just by herself. Haha… total great device, that it delivered plus arrived extremely early and also the mother was experiencing this. Thank we.

  11. #11 by hermes bracelet replica on 28 de maio de 2017 - 10:37 am

    it comes down in a stunning box styles awesome and yet their a bit little even for my mothers wrist however it appears ideal exclusively wish they had been much longer

  12. #12 by on 28 de maio de 2017 - 11:15 am

    I have the headphonesconcerning my personal mom towards mothers day, as well as she completely adored that it! Its really attractive headphonesand the suggesting in the card just that comes into the box is really sentimental!! As well as the grade of the beads was awesome!

  13. #13 by on 28 de maio de 2017 - 11:16 am

    I had gotten this headphonesfor the our mom concerning mothers evening, and she absolutely enjoyed that! It is actually cute headphonesas well as the stating on the card which is will come in box is very emotional!! And the grade of the beads was great!

1 532 533 534
(não será publicado)