Code Snippet: Edge Detector/Antialiasing Shader

Some days ago i was looking for a simple and effective edge detector idea to use in an AntiAliasing shader for Deferred Shading (ideas that effectively make use of the Normal and the Depth Buffers).

Two called my attention: The first one is from the game Stalker, this page from GPU Gems explains the whole idea (nothing very complitaded =P, they show lot of the code). The results are pretty cool, as you can see in this image for example. The second is the one used in Tabula Rasa game, this page from GPU Gems shows the whole idea.

Stalker Idea

The shader code in HLSL is: (added the full version. for those interested only in the edge detection part, see the shader comments).

void PassThroughVS(inout float4 position : POSITION0,
                   inout float2 texcoord : TEXCOORD0) {

				   position.x =  position.x - pixel_size;
				   position.y =  position.y + pixel_size;

float2 e_barrier = float2(0.8f,0.00001f);  // x=norm, y=depth

float2 e_weights= float2(1,1);  // x=norm, y=depth

float2 e_kernel = float2(1,1);   // x=norm, y=depth

const static float2 offs[7] = {
   float2( 0.0,  0.0), //Center       0
   float2(-1.0, -1.0), //Top Left     1
   float2( 1.0,  1.0), //Bottom Right 5
   float2( 1.0, -1.0), //Top Right    3
   float2(-1.0,  1.0), //Bottom Left  7
   float2(-1.0,  0.0),  //Left         8
   float2( 0.0, -1.0) //Top          2

float4 PShader2(float2 uv : TEXCOORD0) : COLOR0


 // Normal discontinuity filter

 float3 nc = tex2D(NormalMap, uv);

 float4 nd;

 nd.x = abs(dot(nc, tex2D(NormalMap, uv + offs[1]* pixel_size).xyz));

 nd.y = abs(dot(nc, tex2D(NormalMap, uv + offs[2]* pixel_size).xyz));

 nd.z = abs(dot(nc, tex2D(NormalMap, uv + offs[3]* pixel_size).xyz));

 nd.w = abs(dot(nc, tex2D(NormalMap, uv + offs[4]* pixel_size).xyz));

 nd -= e_barrier.x;

 nd = step(0, nd);

 float ne = saturate(dot(nd, e_weights.x));

 // Opposite coords

 float2 tc5r = -offs[5];

 float2 tc6r = -offs[6];

 // Depth filter : compute gradiental difference:

 // (c-sample1)+(c-sample1_opposite)

 float dc = tex2D(depthMap, uv).r;

 float4 dd;

 dd.x = tex2D(depthMap, uv + offs[1] * pixel_size).r +

         tex2D(depthMap, uv + offs[2]* pixel_size).r;

 dd.y = tex2D(depthMap, uv + offs[3]* pixel_size).r +

        tex2D(depthMap, uv + offs[4]* pixel_size).r;

 dd.z = tex2D(depthMap, uv + offs[5]* pixel_size).r +

        tex2D(depthMap, uv + tc5r* pixel_size).r;

 dd.w = tex2D(depthMap,uv +  offs[6]* pixel_size).r +

        tex2D(depthMap,uv +  tc6r* pixel_size).r;

 dd = abs(2 * dc - dd)- e_barrier.y;

 dd = step(dd, 0);

 float de = saturate(dot(dd, e_weights.y));

 // Weight

 float w = (1 - de * ne) * e_kernel.x; // 0 - no aa, 1=full aa

 //return float4(w,w,w,1); ///stop here for an edge detector shader

// The AA part (in edges we mix close pixels, non edge pixel w = 0 => use the normal uv, on edge we use a mean of four deslocated uv texture reads )
// Smoothed color

 // (a-c)*w + c = a*w + c(1-w)

 float2 offset = (uv ) * (1-w);

 float4 s0 = tex2D(TextureSampler, offset + (uv + offs[1] * pixel_size) * w);

 float4 s1 = tex2D(TextureSampler, offset + (uv + offs[2]* pixel_size) * w);

 float4 s2 = tex2D(TextureSampler, offset + (uv + offs[3]* pixel_size) * w);

 float4 s3 = tex2D(TextureSampler, offset + (uv + offs[4]* pixel_size) * w);

 return (s0 + s1 + s2 + s3)/4.h;


technique AntiAliasingStalker
    pass Pass1
		VertexShader = compile vs_3_0 PassThroughVS();
                PixelShader = compile ps_3_0 PShader2();

The normal buffer is stored as a 32 bits RGBA texture. The Depth Buffer is stored in a 32 bits Single Texture (storing post-projection z/w). If you use another depth enconding, just adjust the barrier variable value.

The pixel_size variable (a float2 with 1/target width in x and 1/target height in y) is used in DirectX 9 Shaders (Opengl and DirectX > 9 does not need this). The need for this “correction” is very well explained here.

The ScreenShot below is just the edge detector part of the shader. (just uncomment the following line in the shader and you will have a powerfull edge detector shader =P)

Example using this good edge detector shader. Cool for toon shading =P


Idea From Tabula Rasa

This is not so good as the one before (dont looks good when the camera is far from the objects or when the edges has little pixels), but is a bit simpler. Fort those interested  only in the edge detection part,

just return the “factor” variable in the pixel shader, instead of using it to blend the near pixels =P

const float2 delta[8] =

 ///silly, dummy functions to recover normal/depth (i was doing something with the read values before and i did not erased the function ...)
 float4 DL_GetDepth(float2 uv)
	return tex2D(NormalMap  ,uv);
 float4 DL_GetNormal(float2 uv)
	return tex2D(depthMap ,uv);

float depthSensibility;
float normalSensibility;
   // Neighbor offset table
   const static float2 offsets[9] = {
  float2( 0.0,  0.0), //Center       0
   float2(-1.0, -1.0), //Top Left     1
   float2( 0.0, -1.0), //Top          2
   float2( 1.0, -1.0), //Top Right    3
   float2( 1.0,  0.0), //Right        4
   float2( 1.0,  1.0), //Bottom Right 5
   float2( 0.0,  1.0), //Bottom       6
   float2(-1.0,  1.0), //Bottom Left  7
   float2(-1.0,  0.0)  //Left         8
float DL_GetEdgeWeight(in float2 screenPos)
  float Depth[9];
  float3 Normal[9];
  //Retrieve normal and depth data for all neighbors.
   for (int i=0; i<9; ++i)
    float2 uv = screenPos + offsets[i] * pixel_size;
    Depth[i] = DL_GetDepth(uv);  //Retrieves depth from MRTs
    Normal[i]= DL_GetNormal(uv); //Retrieves normal from MRTs
  //Compute Deltas in Depth.
   float4 Deltas1;
  float4 Deltas2;
  Deltas1.x = Depth[1];
  Deltas1.y = Depth[2];
  Deltas1.z = Depth[3];
  Deltas1.w = Depth[4];
  Deltas2.x = Depth[5];
  Deltas2.y = Depth[6];
  Deltas2.z = Depth[7];
  Deltas2.w = Depth[8];
  //Compute absolute gradients from center.
  Deltas1 = abs(Deltas1 - Depth[0]);
  Deltas2 = abs(Depth[0] - Deltas2);
  //Find min and max gradient, ensuring min != 0
   float4 maxDeltas = max(Deltas1, Deltas2);
  float4 minDeltas = max(min(Deltas1, Deltas2), 0.00001);
  // Compare change in gradients, flagging ones that change
   // significantly.
   // How severe the change must be to get flagged is a function of the
   // minimum gradient. It is not resolution dependent. The constant
   // number here would change based on how the depth values are stored
   // and how sensitive the edge detection should be.
   float4 depthResults = step(minDeltas * depthSensibility, maxDeltas);
  //Compute change in the cosine of the angle between normals.
  Deltas1.x = dot(Normal[1], Normal[0]);
  Deltas1.y = dot(Normal[2], Normal[0]);
  Deltas1.z = dot(Normal[3], Normal[0]);
  Deltas1.w = dot(Normal[4], Normal[0]);
  Deltas2.x = dot(Normal[5], Normal[0]);
  Deltas2.y = dot(Normal[6], Normal[0]);
  Deltas2.z = dot(Normal[7], Normal[0]);
  Deltas2.w = dot(Normal[8], Normal[0]);
  Deltas1 = abs(Deltas1 - Deltas2);
  // Compare change in the cosine of the angles, flagging changes
   // above some constant threshold. The cosine of the angle is not a
   // linear function of the angle, so to have the flagging be
   // independent of the angles involved, an arccos function would be
   // required.
  float4 normalResults = step(0.4, Deltas1* normalSensibility);  

  normalResults = max(normalResults, depthResults);
  return (normalResults.x + normalResults.y +
          normalResults.z + normalResults.w) * 0.25;

float4 PShader(float2 texCoord : TEXCOORD0) : COLOR0

 float4 tex = tex2D(NormalMap,texCoord);
 float factor = 0.0f;

 for( int i=0;i<4;i++ )
	 float4 t = tex2D(NormalMap,texCoord+ delta[i]*pixel_size);
	 t -= tex;
	 factor += dot(t,t);

 factor = min(1.0,DL_GetEdgeWeight(texCoord))*weight; 

 float4 color = float4(0.0,0.0,0.0,0.0);

 for( int j=0;j<8;j++ )
	color += tex2D(TextureSampler,texCoord + delta[j]*pixel_size*factor);
 color += 2.0*tex2D(TextureSampler,texCoord);
 return color*(1.0/10.0);


The Vertex Shader and the buffers used are the same from the Stalker idea.

The first one is more beautiful but is slower.

The PloobsEngine uses these AA shaders (and some others also) to perform an Antialiasing Pass.

This is the first code snippet, when possible i will post more =P


, , , , ,

  1. #1 by シャネル コピー 時計 on 22 de julho de 2017 - 4:47 pmロレックス偽物時&#35336;
    [url=]シャネル コピー 時計[/url]

  2. #2 by on 22 de julho de 2017 - 8:45 pm

    We have bought that brand name concerning bracelet a number of circumstances. Every a person is super pretty, has made very well, cannot tarnish to meaningful based on that one particular you buy and which shoppers give it and.

  3. #3 by Beach condos for sale on 22 de julho de 2017 - 9:01 pm

    F*ckin’ awesome issues here. I am very happy to see your post. Thank you so much and i am looking forward to contact you. Will you please drop me a e-mail?

  4. #4 by foulard femme hermes on 22 de julho de 2017 - 10:28 pm

    déjà eu lieu, et de plusieurs traditions religieuses diffèrent-ent. La consultation a été menée par le directeur de la DDA, Michael Taylor, et comprenait des représentants des asobservers Banque mondiale, le FMI a été invité à participer, mais ne pouvait pas être présent. A22? Partie I: Faith Perspectives et la compréhension de l’objectif Povertysecond de la réunion était d’aider à mieux comprendre, contre la lumière du défi du DSRP, de la façon dont les groupes confessionnels involvedframed leur vision et leur travail sur la pauvreté, et comment ils ont vu governmentstrategies pour faire face la pauvreté reflète dans les PRSP.Visions de PovertyAt l’atelier Juillet Canterbury, la première tâche définie au groupe de foi WASTO rapport sur la nature de la pauvreté dans leurs pays respectifs. L’un des éléments de theinteresting était non seulement la

  5. #5 by Myrtle Beach lawn maintenance on 22 de julho de 2017 - 10:41 pm

    whoah this weblog is great i love reading your articles.
    Stay up the good work! You understand, many persons are looking round for this information, you can aid them greatly.

  6. #6 by Lelio Vieira Carneiro Junior on 23 de julho de 2017 - 3:29 am

    Its not my first time to pay a quick visit this web page, i am
    browsing this site dailly and take pleasant information from here
    all the time.

  7. #7 by on 23 de julho de 2017 - 7:07 am

    We have bought that brand of bracelet various occasions. Every single a person is very sweet, prepared well, does not tarnish then important depending on that a person you purchase furthermore who you offer things at.

  8. #8 by cheap louboutins on 23 de julho de 2017 - 7:08 am

    I’ve purchased it brand of bracelet various occasions. Every a person is extremely attractive, made very well, cannot tarnish to important based on that a person you buy and also who one promote they and.

  9. #9 by hermes h bracelet replica on 23 de julho de 2017 - 8:46 am

    Our system had been in such a great expense I not considered the actual grade is quite exceptional. Its stunning. This one mom may appreciate they in Christmas time early morning after she starts present therefore appearances as though I devoted a lot more, and yet cost had been simply great!!

  10. #10 by floor protector on 23 de julho de 2017 - 9:10 am

    that could be the end of this write-up. Right here you will come across some sites that we assume youll enjoy, just click the hyperlinks over

  11. #11 by hermes belt replica on 23 de julho de 2017 - 9:10 am

    I have purchased the brand name to bracelet some occasions. Every single a person is extremely pretty, established very well, cannot tarnish and important based on which a person you buy to just who you promote this on.

  12. #12 by on 23 de julho de 2017 - 9:11 am

    I’ve purchased your brand name to bracelet a few days. Every single a person is very pretty, done very well, cannot tarnish furthermore significant based on which kind of single you purchase to that a person bring that towards.

  13. #13 by corevvc on 23 de julho de 2017 - 9:33 am

    サーバ機だけあって、OSは、何の問題もなくインストール完了。 これがこれまでの各世代Windowsに共通する方法であった。 [url=]office 2013 激安[/url]
       日本では、先ほど、読売新聞と朝日新聞はウェブサイトにも速報を載せましたが、転写引用はできません。 – 社会的に相互作用のための機会を提供しています。
    [url=]office2013 ソフト[/url] ▼対応Office2000、2002、2003ワード、エクセル、パワーポイントバージョンが異なるOfficeをご利用の際は大変便利です。 これを試したのはWindows10入れている時なのでWindows7/Windows8.1では試していませんが多分同じではないかと思われますよ。
    [url=]office2010 プロダクト キー[/url]
    タブレット端末といえば、iPadやAndroidタブレットを思い浮かべますが、Surfaceの中身はWindows8.1。 1にするのい3、000円ほどで済んだのだった。 [url=]microsoft office 安い[/url]
    1・PC設定→保守と管理→Windows Updete→更新履歴を表示する2・対象機種は、更新履歴4月5日以降に「KB3035583」がインストールされている。 SoftBank 503LVは、1,280×720ドット(HD)解像度の5型スマートフォン。
    [url=]office2010 プロダクト キー[/url] 先週末のサイバー攻撃を例にとってみると、英国で多数の病院が感染しており、一部は患者を受け入れられなかった。 特に英語圏で “PC” と略した場合はこの用法であることが多く、その場合、Macなど他の規格の製品を総称する場合は “personal computers” と略さずに記述する。
    [url=]office mac 永続[/url]

  14. #14 by cheap nike roshe run on 23 de julho de 2017 - 9:44 am

    We have bought it brand regarding bracelet a few times. Every single a person is super cute, prepared very well, does not tarnish as well as significant depending on which a single you purchase and who people present it towards.

  15. #15 by Business on 23 de julho de 2017 - 10:39 am

    I think this web site has got some very fantastic info for everyone :D. “Laughter is the sun that drives winter from the human face.” by Victor Hugo.

  16. #16 by polythene manufacturer on 23 de julho de 2017 - 10:41 am

    Every when in a although we opt for blogs that we study. Listed beneath are the latest web sites that we decide on

  17. #17 by on 23 de julho de 2017 - 2:14 pm

    This particular system ended up being at that very good price I do not thought ones good is so that exceptional. Its awesome. Our mother is going to appreciate that it on Xmas morning once she opens gift therefore styles as though I invested way more, and yet cost had been exclusively very good!!

  18. #18 by Biloxi rentals on 23 de julho de 2017 - 4:54 pm

    that could be the end of this article. Right here you will discover some web sites that we consider you will appreciate, just click the links over

  19. #19 by Southwind Resort rentals on 23 de julho de 2017 - 5:17 pm

    Wonderful story, reckoned we could combine several unrelated data, nevertheless definitely worth taking a appear, whoa did 1 study about Mid East has got a lot more problerms at the same time

  20. #20 by hermes belt replica on 23 de julho de 2017 - 5:18 pm

    We have purchased the brand name regarding bracelet a few times. Every one is super attractive, done well, does not tarnish additionally meaningful depending on which a single you purchase then which people give things in order to.

  21. #21 by cheap louboutin shoes replica on 23 de julho de 2017 - 5:19 pm

    We have bought this particular brand out of bracelet some circumstances. Every single a person is extremely sweet, established well, does not tarnish to meaningful depending on which definitely one you buy additionally that you provide things towards.

  22. #22 by magnetic sheeting on 23 de julho de 2017 - 5:21 pm

    please visit the sites we adhere to, including this one, because it represents our picks in the web

  23. #23 by corex on 23 de julho de 2017 - 5:49 pm

    Here is an excellent Blog You may Come across Fascinating that we Encourage You

  24. #24 by Biloxi Corporate rentals on 23 de julho de 2017 - 6:06 pm

    very handful of web sites that occur to become detailed beneath, from our point of view are undoubtedly well worth checking out

1 586 587 588
(não será publicado)