Code Snippet: Edge Detector/Antialiasing Shader

Some days ago i was looking for a simple and effective edge detector idea to use in an AntiAliasing shader for Deferred Shading (ideas that effectively make use of the Normal and the Depth Buffers).

Two called my attention: The first one is from the game Stalker, this page from GPU Gems explains the whole idea (nothing very complitaded =P, they show lot of the code). The results are pretty cool, as you can see in this image for example. The second is the one used in Tabula Rasa game, this page from GPU Gems shows the whole idea.

Stalker Idea

The shader code in HLSL is: (added the full version. for those interested only in the edge detection part, see the shader comments).

void PassThroughVS(inout float4 position : POSITION0,
                   inout float2 texcoord : TEXCOORD0) {

				   position.x =  position.x - pixel_size;
				   position.y =  position.y + pixel_size;

float2 e_barrier = float2(0.8f,0.00001f);  // x=norm, y=depth

float2 e_weights= float2(1,1);  // x=norm, y=depth

float2 e_kernel = float2(1,1);   // x=norm, y=depth

const static float2 offs[7] = {
   float2( 0.0,  0.0), //Center       0
   float2(-1.0, -1.0), //Top Left     1
   float2( 1.0,  1.0), //Bottom Right 5
   float2( 1.0, -1.0), //Top Right    3
   float2(-1.0,  1.0), //Bottom Left  7
   float2(-1.0,  0.0),  //Left         8
   float2( 0.0, -1.0) //Top          2

float4 PShader2(float2 uv : TEXCOORD0) : COLOR0


 // Normal discontinuity filter

 float3 nc = tex2D(NormalMap, uv);

 float4 nd;

 nd.x = abs(dot(nc, tex2D(NormalMap, uv + offs[1]* pixel_size).xyz));

 nd.y = abs(dot(nc, tex2D(NormalMap, uv + offs[2]* pixel_size).xyz));

 nd.z = abs(dot(nc, tex2D(NormalMap, uv + offs[3]* pixel_size).xyz));

 nd.w = abs(dot(nc, tex2D(NormalMap, uv + offs[4]* pixel_size).xyz));

 nd -= e_barrier.x;

 nd = step(0, nd);

 float ne = saturate(dot(nd, e_weights.x));

 // Opposite coords

 float2 tc5r = -offs[5];

 float2 tc6r = -offs[6];

 // Depth filter : compute gradiental difference:

 // (c-sample1)+(c-sample1_opposite)

 float dc = tex2D(depthMap, uv).r;

 float4 dd;

 dd.x = tex2D(depthMap, uv + offs[1] * pixel_size).r +

         tex2D(depthMap, uv + offs[2]* pixel_size).r;

 dd.y = tex2D(depthMap, uv + offs[3]* pixel_size).r +

        tex2D(depthMap, uv + offs[4]* pixel_size).r;

 dd.z = tex2D(depthMap, uv + offs[5]* pixel_size).r +

        tex2D(depthMap, uv + tc5r* pixel_size).r;

 dd.w = tex2D(depthMap,uv +  offs[6]* pixel_size).r +

        tex2D(depthMap,uv +  tc6r* pixel_size).r;

 dd = abs(2 * dc - dd)- e_barrier.y;

 dd = step(dd, 0);

 float de = saturate(dot(dd, e_weights.y));

 // Weight

 float w = (1 - de * ne) * e_kernel.x; // 0 - no aa, 1=full aa

 //return float4(w,w,w,1); ///stop here for an edge detector shader

// The AA part (in edges we mix close pixels, non edge pixel w = 0 => use the normal uv, on edge we use a mean of four deslocated uv texture reads )
// Smoothed color

 // (a-c)*w + c = a*w + c(1-w)

 float2 offset = (uv ) * (1-w);

 float4 s0 = tex2D(TextureSampler, offset + (uv + offs[1] * pixel_size) * w);

 float4 s1 = tex2D(TextureSampler, offset + (uv + offs[2]* pixel_size) * w);

 float4 s2 = tex2D(TextureSampler, offset + (uv + offs[3]* pixel_size) * w);

 float4 s3 = tex2D(TextureSampler, offset + (uv + offs[4]* pixel_size) * w);

 return (s0 + s1 + s2 + s3)/4.h;


technique AntiAliasingStalker
    pass Pass1
		VertexShader = compile vs_3_0 PassThroughVS();
                PixelShader = compile ps_3_0 PShader2();

The normal buffer is stored as a 32 bits RGBA texture. The Depth Buffer is stored in a 32 bits Single Texture (storing post-projection z/w). If you use another depth enconding, just adjust the barrier variable value.

The pixel_size variable (a float2 with 1/target width in x and 1/target height in y) is used in DirectX 9 Shaders (Opengl and DirectX > 9 does not need this). The need for this “correction” is very well explained here.

The ScreenShot below is just the edge detector part of the shader. (just uncomment the following line in the shader and you will have a powerfull edge detector shader =P)

Example using this good edge detector shader. Cool for toon shading =P


Idea From Tabula Rasa

This is not so good as the one before (dont looks good when the camera is far from the objects or when the edges has little pixels), but is a bit simpler. Fort those interested  only in the edge detection part,

just return the “factor” variable in the pixel shader, instead of using it to blend the near pixels =P

const float2 delta[8] =

 ///silly, dummy functions to recover normal/depth (i was doing something with the read values before and i did not erased the function ...)
 float4 DL_GetDepth(float2 uv)
	return tex2D(NormalMap  ,uv);
 float4 DL_GetNormal(float2 uv)
	return tex2D(depthMap ,uv);

float depthSensibility;
float normalSensibility;
   // Neighbor offset table
   const static float2 offsets[9] = {
  float2( 0.0,  0.0), //Center       0
   float2(-1.0, -1.0), //Top Left     1
   float2( 0.0, -1.0), //Top          2
   float2( 1.0, -1.0), //Top Right    3
   float2( 1.0,  0.0), //Right        4
   float2( 1.0,  1.0), //Bottom Right 5
   float2( 0.0,  1.0), //Bottom       6
   float2(-1.0,  1.0), //Bottom Left  7
   float2(-1.0,  0.0)  //Left         8
float DL_GetEdgeWeight(in float2 screenPos)
  float Depth[9];
  float3 Normal[9];
  //Retrieve normal and depth data for all neighbors.
   for (int i=0; i<9; ++i)
    float2 uv = screenPos + offsets[i] * pixel_size;
    Depth[i] = DL_GetDepth(uv);  //Retrieves depth from MRTs
    Normal[i]= DL_GetNormal(uv); //Retrieves normal from MRTs
  //Compute Deltas in Depth.
   float4 Deltas1;
  float4 Deltas2;
  Deltas1.x = Depth[1];
  Deltas1.y = Depth[2];
  Deltas1.z = Depth[3];
  Deltas1.w = Depth[4];
  Deltas2.x = Depth[5];
  Deltas2.y = Depth[6];
  Deltas2.z = Depth[7];
  Deltas2.w = Depth[8];
  //Compute absolute gradients from center.
  Deltas1 = abs(Deltas1 - Depth[0]);
  Deltas2 = abs(Depth[0] - Deltas2);
  //Find min and max gradient, ensuring min != 0
   float4 maxDeltas = max(Deltas1, Deltas2);
  float4 minDeltas = max(min(Deltas1, Deltas2), 0.00001);
  // Compare change in gradients, flagging ones that change
   // significantly.
   // How severe the change must be to get flagged is a function of the
   // minimum gradient. It is not resolution dependent. The constant
   // number here would change based on how the depth values are stored
   // and how sensitive the edge detection should be.
   float4 depthResults = step(minDeltas * depthSensibility, maxDeltas);
  //Compute change in the cosine of the angle between normals.
  Deltas1.x = dot(Normal[1], Normal[0]);
  Deltas1.y = dot(Normal[2], Normal[0]);
  Deltas1.z = dot(Normal[3], Normal[0]);
  Deltas1.w = dot(Normal[4], Normal[0]);
  Deltas2.x = dot(Normal[5], Normal[0]);
  Deltas2.y = dot(Normal[6], Normal[0]);
  Deltas2.z = dot(Normal[7], Normal[0]);
  Deltas2.w = dot(Normal[8], Normal[0]);
  Deltas1 = abs(Deltas1 - Deltas2);
  // Compare change in the cosine of the angles, flagging changes
   // above some constant threshold. The cosine of the angle is not a
   // linear function of the angle, so to have the flagging be
   // independent of the angles involved, an arccos function would be
   // required.
  float4 normalResults = step(0.4, Deltas1* normalSensibility);  

  normalResults = max(normalResults, depthResults);
  return (normalResults.x + normalResults.y +
          normalResults.z + normalResults.w) * 0.25;

float4 PShader(float2 texCoord : TEXCOORD0) : COLOR0

 float4 tex = tex2D(NormalMap,texCoord);
 float factor = 0.0f;

 for( int i=0;i<4;i++ )
	 float4 t = tex2D(NormalMap,texCoord+ delta[i]*pixel_size);
	 t -= tex;
	 factor += dot(t,t);

 factor = min(1.0,DL_GetEdgeWeight(texCoord))*weight; 

 float4 color = float4(0.0,0.0,0.0,0.0);

 for( int j=0;j<8;j++ )
	color += tex2D(TextureSampler,texCoord + delta[j]*pixel_size*factor);
 color += 2.0*tex2D(TextureSampler,texCoord);
 return color*(1.0/10.0);


The Vertex Shader and the buffers used are the same from the Stalker idea.

The first one is more beautiful but is slower.

The PloobsEngine uses these AA shaders (and some others also) to perform an Antialiasing Pass.

This is the first code snippet, when possible i will post more =P


, , , , ,

  1. #1 by haben sie einen blick auf diese jungs on 23 de abril de 2017 - 8:38 am

    Das junge Girl wollte endlich mal Sex mit 2 Männern ausprobieren.

  2. #2 by Hermes Mens Bracelet on 23 de abril de 2017 - 8:51 am

    I ordered our searching for a awesome present concerning the mom. Your headphone came within a breathtaking purple tied up container. This was the greatest searching gift under ones holiday tree!! That the headphone even looked striking, though our mom was larger boned as well as the headphone are some tight, and yet their a good present! I love information technology.

  3. #3 by webdesign vreden on 23 de abril de 2017 - 8:52 am

    There’s certainly a lot tо know about tһis subject.
    I love аll the points you made.

  4. #4 by on 23 de abril de 2017 - 8:52 am

    I acquired that looking for a awesome present towards the mom. The headphone emerged in a gorgeous purple tied up package. This is ideal hunting gifts underneath that christmas time tree!! The actual headphone even looked breathtaking, however our mama is actually huge boned and the headphone are a little tight, and yet its a good gifts! I enjoy it.

  5. #5 by website on 23 de abril de 2017 - 9:11 am

    Magnificent beat ! I would like to apprentice while you amend your site, how could i subscribe for a blog site?
    The account helped me a acceptable deal. I had been a
    little bit acquainted of this your broadcast offered bright clear idea

  6. #6 by schauen sie sich in dieser website on 23 de abril de 2017 - 9:33 am

    Hier sind sie für die nächsten busen thumbnails zwei Wochen zu Hause, hörte er eine blecherne Lautsprecherstimme.

  7. #7 by Cartier Rings on 23 de abril de 2017 - 9:47 am

    I acquired this one trying to find a great gift concerning my mother. That headphone came in a breathtaking purple tied package. This was the best hunting gifts underneath that the christmas tree!! Some sort of headphone also looked pretty, still our mom was huge boned as well as the headphone is a little tight, but the a good gift! I really like they.

  8. #8 by ich las diese on 23 de abril de 2017 - 9:54 am

    Empfängt man eine Nachricht mit einem Smiley, wird nur die entsprechende Zeichenfolge ausgeschrieben.

  9. #9 by Paulo João Miguel on 23 de abril de 2017 - 12:39 pm

    Se volume de busca for muito grave na maioria dos casos é possível que
    não ser interessante, porém existem exceções como no caso de
    produtos com ticket de venda elevado que geram comissões altas, quer dizer, mesmo
    com poucas pessoas pesquisando talvez apenas 01 venda do resultado por mês pode apenas dar uma boa comissão de
    400 reais ou mais e neste caso pode ser bem lucrativo.

  10. #10 by on 23 de abril de 2017 - 1:50 pm

    This headset ended up being like beautiful as within the picture. Information technology came immediately. I would recommend utilizing a towel during every single coating while you click this off w / the vapor iron. It does not vapor over and only a steamer. Some sort of iron had been appropriate. It is very delicate, so if you you should not trust yourself with all the towel and vapor iron, after that bring this up to a professional. Breathtaking sound.

  11. #11 by on 23 de abril de 2017 - 1:51 pm

    This particular headset is just as stunning because within the photo. This appeared quickly. I’d suggest utilizing a towel above every single level as you press things away w / per vapor iron. It does not steam off alongside only a steamer. Their iron ended up being appropriate. It is very fine, when you never trust personally with all the towel plus vapor iron, and then just take information technology up to a expert. Beautiful audio.

  12. #12 by Education on 23 de abril de 2017 - 2:52 pm

    Thanks, I’ve just been searching for info approximately this subject for ages and yours is the best I’ve found out so far. But, what about the bottom line? Are you positive in regards to the source?

  13. #13 by rezension on 23 de abril de 2017 - 2:59 pm

    Lieber Pils-Bier als Shakespeare!

  14. #14 by Dieta health detox on 23 de abril de 2017 - 3:00 pm

    Découvrez notre catalogue de livres audio à télécharger gratuitement et légalement.

  15. #15 by eyfel parfum on 23 de abril de 2017 - 3:05 pm

    You made some decent points there. I looked on the internet for the issue and found most individuals will go along with with your website.

  16. #16 by Naturally Him Enhancement on 23 de abril de 2017 - 4:16 pm

    Hello There. I found your blpg using msn. This is an extremely well written article.
    I’ll make sure tto bookmark it andd come back to read more of
    your usegul info. Thanks for the post. I’ll
    certainly comeback.

  17. #17 by on 23 de abril de 2017 - 5:26 pm

    I ordered that searching for a very good gifts for the the mom. That the headphone came in a perfect purple tied container. This was a hunting present below that the holiday tree!! That the headphone additionally looked awesome, but the mother was huge boned as well as the headphone is actually some tight, however the a great gifts! I prefer they.

  18. #18 by on 23 de abril de 2017 - 5:26 pm

    I ordered that looking for a great present for the my personal mom. The particular headphone came in a beautiful purple tied package. This is ideal hunting present less than the actual xmas tree!! That the headphone even looked breathtaking, then again my personal mama is larger boned and headphone is a little tight, and yet its a great gift! I adore things.

  19. #19 by on 23 de abril de 2017 - 5:26 pm

    I purchased our looking for a ideal gifts concerning our mama. The actual headphone emerged within a beautiful purple tied container. This was ideal hunting present below the xmas tree!! The headphone always seemed beautiful, still the mama is larger boned and headphone is a little tight, but it really is a great gift! I prefer things.

  20. #20 by dobwgr on 23 de abril de 2017 - 5:44 pm

    これらの大部分は根拠のある推測であり、正しいものと考えられます」というメッセージが書かれています。 日本で一般には、JIS規格の文字コード(通常はJIS X 0208、稀にJIS X 0213やJIS X 0221)に含まれない文字のことをさし、「表外字」、「拡張漢字(ベンダ選定拡張漢字)」とも呼ばれる。 [url=]サムライ せんせい 動画[/url]
    パソコンクラブのYSさんから、7年前WindowsVistaパソコンを買ったが,それを7にアップグレード(それが購入条件)さらに、それを昨年Windows10にアップグレードした。 羨ましい」「変な言い方」「大学の卒業前かな?うん若い若い。
    [url=]韓国 ピノキオ[/url] みんなが普通にできることが、自分の家ではとても困難。 ※当ブログ内各記事、どんなに過去の記事でも全記事、「コメント」(連コメ)、「ナイス」(大量ナイス)、「村ポチ」大歓迎。
    [url=]黒 革 の 手帖[/url]
    日本では、貧困層は趣味を持つことや私的な楽しみを持つことに対して一般住民のアレルギーが強い。 まず、この汁を入れソイルを入れ、水槽の水を入れ、水草を植えて出来上がりです^^所用時間は2つで20分ぐらいですかね~溶岩石のレイアウトに時間使ったので、植えるだけだと1個3分もあればできます。 [url=]星 から 来 た あなた 本[/url]
    タスクマネジャーのスタートアップには12個しか出きない。 Docs.comでは、SwayなどOfficeアプリケーションによるコンテンツの整理、共有、管理が一括して行える。
    [url=]ドクター 異邦 人 ハンスンヒ[/url][url=]あま ちゃん ピアノ[/url] パソコンをUSB-DAC再生に用いてこの音質に並ぶためには、オーディオ専用にチューニングされたパソコン(オーディオに特化した電源や筐体は格別重要だ)、ソフトウェア、複雑な設定、そしてユーザーの試行錯誤が必要になるだろう。 だが、現在、こうした怪しい「ニュースメディア」の記事でも、瞬時に拡大してしまう恐ろしい時代になっている。
    [url=]黒 革 の 手帖[/url]

  21. #21 by on 23 de abril de 2017 - 5:54 pm

    Purchased the as being a present towards the mom as well as she loved they. great good great price to our mother enjoyed things. Furthermore delivered very accelerated. So if you need a gift quick while such as this then this is the an you must invest!

  22. #22 by on 23 de abril de 2017 - 5:56 pm

    But not while fancy at real life if you think moms wrist is maybe not 2small do not pick that it reason things get tight .. its adorable although something yur 5year older make . Non that the not so their an okay gifts on provide yet not the wow gifts…cant get wrong provided don’t have actually a lot to blow for a gifts after that become information technology

  23. #23 by on 23 de abril de 2017 - 5:56 pm

    Yet not as fancy in real world in case mothers wrist are maybe not 2small don’t choose that reason that stay tight .. it is sweet although something yur 5year aged can make . Non the actual less its an okay gift and provide yet not per wow present…cant go wrong provided dont has far to spend for a present next have this

  24. #24 by best kona on 23 de abril de 2017 - 6:37 pm

    very few websites that transpire to be detailed beneath, from our point of view are undoubtedly very well worth checking out

  25. #25 by on 23 de abril de 2017 - 6:40 pm

    I have your being a gifts of our sibling, happy mother. She adored this!! I was nervous things was will be quite smaller because of the earlier evaluations and exactly how it looked once I got a top, but it in shape quite well. Things appears stunning concerning plus the actual beautiful package it’s presented at.

1 484 485 486
(não será publicado)