Code Snippet: Edge Detector/Antialiasing Shader

Some days ago i was looking for a simple and effective edge detector idea to use in an AntiAliasing shader for Deferred Shading (ideas that effectively make use of the Normal and the Depth Buffers).

Two called my attention: The first one is from the game Stalker, this page from GPU Gems explains the whole idea (nothing very complitaded =P, they show lot of the code). The results are pretty cool, as you can see in this image for example. The second is the one used in Tabula Rasa game, this page from GPU Gems shows the whole idea.

Stalker Idea

The shader code in HLSL is: (added the full version. for those interested only in the edge detection part, see the shader comments).

void PassThroughVS(inout float4 position : POSITION0,
                   inout float2 texcoord : TEXCOORD0) {

				   position.x =  position.x - pixel_size;
				   position.y =  position.y + pixel_size;

float2 e_barrier = float2(0.8f,0.00001f);  // x=norm, y=depth

float2 e_weights= float2(1,1);  // x=norm, y=depth

float2 e_kernel = float2(1,1);   // x=norm, y=depth

const static float2 offs[7] = {
   float2( 0.0,  0.0), //Center       0
   float2(-1.0, -1.0), //Top Left     1
   float2( 1.0,  1.0), //Bottom Right 5
   float2( 1.0, -1.0), //Top Right    3
   float2(-1.0,  1.0), //Bottom Left  7
   float2(-1.0,  0.0),  //Left         8
   float2( 0.0, -1.0) //Top          2

float4 PShader2(float2 uv : TEXCOORD0) : COLOR0


 // Normal discontinuity filter

 float3 nc = tex2D(NormalMap, uv);

 float4 nd;

 nd.x = abs(dot(nc, tex2D(NormalMap, uv + offs[1]* pixel_size).xyz));

 nd.y = abs(dot(nc, tex2D(NormalMap, uv + offs[2]* pixel_size).xyz));

 nd.z = abs(dot(nc, tex2D(NormalMap, uv + offs[3]* pixel_size).xyz));

 nd.w = abs(dot(nc, tex2D(NormalMap, uv + offs[4]* pixel_size).xyz));

 nd -= e_barrier.x;

 nd = step(0, nd);

 float ne = saturate(dot(nd, e_weights.x));

 // Opposite coords

 float2 tc5r = -offs[5];

 float2 tc6r = -offs[6];

 // Depth filter : compute gradiental difference:

 // (c-sample1)+(c-sample1_opposite)

 float dc = tex2D(depthMap, uv).r;

 float4 dd;

 dd.x = tex2D(depthMap, uv + offs[1] * pixel_size).r +

         tex2D(depthMap, uv + offs[2]* pixel_size).r;

 dd.y = tex2D(depthMap, uv + offs[3]* pixel_size).r +

        tex2D(depthMap, uv + offs[4]* pixel_size).r;

 dd.z = tex2D(depthMap, uv + offs[5]* pixel_size).r +

        tex2D(depthMap, uv + tc5r* pixel_size).r;

 dd.w = tex2D(depthMap,uv +  offs[6]* pixel_size).r +

        tex2D(depthMap,uv +  tc6r* pixel_size).r;

 dd = abs(2 * dc - dd)- e_barrier.y;

 dd = step(dd, 0);

 float de = saturate(dot(dd, e_weights.y));

 // Weight

 float w = (1 - de * ne) * e_kernel.x; // 0 - no aa, 1=full aa

 //return float4(w,w,w,1); ///stop here for an edge detector shader

// The AA part (in edges we mix close pixels, non edge pixel w = 0 => use the normal uv, on edge we use a mean of four deslocated uv texture reads )
// Smoothed color

 // (a-c)*w + c = a*w + c(1-w)

 float2 offset = (uv ) * (1-w);

 float4 s0 = tex2D(TextureSampler, offset + (uv + offs[1] * pixel_size) * w);

 float4 s1 = tex2D(TextureSampler, offset + (uv + offs[2]* pixel_size) * w);

 float4 s2 = tex2D(TextureSampler, offset + (uv + offs[3]* pixel_size) * w);

 float4 s3 = tex2D(TextureSampler, offset + (uv + offs[4]* pixel_size) * w);

 return (s0 + s1 + s2 + s3)/4.h;


technique AntiAliasingStalker
    pass Pass1
		VertexShader = compile vs_3_0 PassThroughVS();
                PixelShader = compile ps_3_0 PShader2();

The normal buffer is stored as a 32 bits RGBA texture. The Depth Buffer is stored in a 32 bits Single Texture (storing post-projection z/w). If you use another depth enconding, just adjust the barrier variable value.

The pixel_size variable (a float2 with 1/target width in x and 1/target height in y) is used in DirectX 9 Shaders (Opengl and DirectX > 9 does not need this). The need for this “correction” is very well explained here.

The ScreenShot below is just the edge detector part of the shader. (just uncomment the following line in the shader and you will have a powerfull edge detector shader =P)

Example using this good edge detector shader. Cool for toon shading =P


Idea From Tabula Rasa

This is not so good as the one before (dont looks good when the camera is far from the objects or when the edges has little pixels), but is a bit simpler. Fort those interested  only in the edge detection part,

just return the “factor” variable in the pixel shader, instead of using it to blend the near pixels =P

const float2 delta[8] =

 ///silly, dummy functions to recover normal/depth (i was doing something with the read values before and i did not erased the function ...)
 float4 DL_GetDepth(float2 uv)
	return tex2D(NormalMap  ,uv);
 float4 DL_GetNormal(float2 uv)
	return tex2D(depthMap ,uv);

float depthSensibility;
float normalSensibility;
   // Neighbor offset table
   const static float2 offsets[9] = {
  float2( 0.0,  0.0), //Center       0
   float2(-1.0, -1.0), //Top Left     1
   float2( 0.0, -1.0), //Top          2
   float2( 1.0, -1.0), //Top Right    3
   float2( 1.0,  0.0), //Right        4
   float2( 1.0,  1.0), //Bottom Right 5
   float2( 0.0,  1.0), //Bottom       6
   float2(-1.0,  1.0), //Bottom Left  7
   float2(-1.0,  0.0)  //Left         8
float DL_GetEdgeWeight(in float2 screenPos)
  float Depth[9];
  float3 Normal[9];
  //Retrieve normal and depth data for all neighbors.
   for (int i=0; i<9; ++i)
    float2 uv = screenPos + offsets[i] * pixel_size;
    Depth[i] = DL_GetDepth(uv);  //Retrieves depth from MRTs
    Normal[i]= DL_GetNormal(uv); //Retrieves normal from MRTs
  //Compute Deltas in Depth.
   float4 Deltas1;
  float4 Deltas2;
  Deltas1.x = Depth[1];
  Deltas1.y = Depth[2];
  Deltas1.z = Depth[3];
  Deltas1.w = Depth[4];
  Deltas2.x = Depth[5];
  Deltas2.y = Depth[6];
  Deltas2.z = Depth[7];
  Deltas2.w = Depth[8];
  //Compute absolute gradients from center.
  Deltas1 = abs(Deltas1 - Depth[0]);
  Deltas2 = abs(Depth[0] - Deltas2);
  //Find min and max gradient, ensuring min != 0
   float4 maxDeltas = max(Deltas1, Deltas2);
  float4 minDeltas = max(min(Deltas1, Deltas2), 0.00001);
  // Compare change in gradients, flagging ones that change
   // significantly.
   // How severe the change must be to get flagged is a function of the
   // minimum gradient. It is not resolution dependent. The constant
   // number here would change based on how the depth values are stored
   // and how sensitive the edge detection should be.
   float4 depthResults = step(minDeltas * depthSensibility, maxDeltas);
  //Compute change in the cosine of the angle between normals.
  Deltas1.x = dot(Normal[1], Normal[0]);
  Deltas1.y = dot(Normal[2], Normal[0]);
  Deltas1.z = dot(Normal[3], Normal[0]);
  Deltas1.w = dot(Normal[4], Normal[0]);
  Deltas2.x = dot(Normal[5], Normal[0]);
  Deltas2.y = dot(Normal[6], Normal[0]);
  Deltas2.z = dot(Normal[7], Normal[0]);
  Deltas2.w = dot(Normal[8], Normal[0]);
  Deltas1 = abs(Deltas1 - Deltas2);
  // Compare change in the cosine of the angles, flagging changes
   // above some constant threshold. The cosine of the angle is not a
   // linear function of the angle, so to have the flagging be
   // independent of the angles involved, an arccos function would be
   // required.
  float4 normalResults = step(0.4, Deltas1* normalSensibility);  

  normalResults = max(normalResults, depthResults);
  return (normalResults.x + normalResults.y +
          normalResults.z + normalResults.w) * 0.25;

float4 PShader(float2 texCoord : TEXCOORD0) : COLOR0

 float4 tex = tex2D(NormalMap,texCoord);
 float factor = 0.0f;

 for( int i=0;i<4;i++ )
	 float4 t = tex2D(NormalMap,texCoord+ delta[i]*pixel_size);
	 t -= tex;
	 factor += dot(t,t);

 factor = min(1.0,DL_GetEdgeWeight(texCoord))*weight; 

 float4 color = float4(0.0,0.0,0.0,0.0);

 for( int j=0;j<8;j++ )
	color += tex2D(TextureSampler,texCoord + delta[j]*pixel_size*factor);
 color += 2.0*tex2D(TextureSampler,texCoord);
 return color*(1.0/10.0);


The Vertex Shader and the buffers used are the same from the Stalker idea.

The first one is more beautiful but is slower.

The PloobsEngine uses these AA shaders (and some others also) to perform an Antialiasing Pass.

This is the first code snippet, when possible i will post more =P


, , , , ,

  1. #1 by on 23 de julho de 2016 - 2:04 pm

    it comes in a breathtaking container appears ideal although it is slightly mini still to my personal mothers wrist however it appearance great only desire they is extended

  2. #2 by on 23 de julho de 2016 - 2:05 pm

    I have this particular gifts concerning the mom for Xmas mainly because she is your jewelry freak. All one thing she does not don a great deal of was, bracelets. I purchased the lady it allure bracelet and also after she started this yesterday she absolutley enjoyed they! Today the issue is, the lady getting this on top of then down by just by herself. Haha… in general great product, things transported plus arrived incredibly very early and also my mother is enjoying information technology. Thank shoppers.

  3. #3 by on 23 de julho de 2016 - 2:14 pm

    Title IX, impact all civilian federal rules dictating result in healing relating to the sexes in education particular sports classes, doesn’t turn into a laws up to the point 1972 (and also it ingested some time for results of legal issues to seem in class actions). unquestionably the olympics did not need kind football up until the time 1976. So if you were one of the best woman ball player earlier this key fact time, ended up range means where you can take pleasure in your skills.asian pet fish particular sports weird four

  4. #4 by on 23 de julho de 2016 - 3:14 pm

    Seven days at some time a Bulgarian kick boxer, by getting a sinister bruise suffering our face, most likely was nodding quietly as anyone entitled Davey Cowland awarded your ex or even best handbook prior to the contemporary struggle. there’s the majority of dancing happening, sega’s unfamiliar, to be the Bulgarian, Borislav Zankov, chatted correct french and as well as Cowl Bulgarian. win or lose, The Bulgarian may make 800 quid inside a Camden heart, north hackney, relating to his own four on practiced middleweight squabble,condition klopp defines instance located on borussia dortmund rule

  5. #6 by on 23 de julho de 2016 - 3:44 pm

    “he’s got someone that is about to do everything that the motor coach rentals ask him complete. i mean, it is difficult to woo baseball events inside of this category, although I may jer Campbell on my very own collection. your dog is want to excel. according to insignificant league martial arts best sources, Taylor supported the position issues two weeks the particular and is anticipated to resign this steps in the Double A thunder over the following three to four weeks,an example of Trenton could possibly be handy to my advice,” Taylor explained friday. “A really important aspect in more or less everything is our grandkids. we offer root base inside the southern states.on those grounds, my spouse and i he are able to keep his franchise in such a one for a few years. every now and then the organization gets going, however not up. i do believe hot the uk is way too very good and Brady is unsafe your own own home. your Pats offensive lineman, who is displaying composing reddish colored hairs himself, realises hair on your face you should be “pleasurable” in addition to the “whole lot macho, albeit according to him there’s a new great “itches period” that should be troublesome. akin to Letterman’s and after that O’Brien’s beards, Koppen mentioned he wants these seek leads to raised scores. “i believe guests welcome the perform those goes into it as well as also devotion.

  6. #7 by erhalten weitere Infos on 23 de julho de 2016 - 4:14 pm

    Mein Ziel ist nicht einfach einen Mann zu heiraten, sondern einen Partner für den Rest meines Lebens zu finden.

  7. #8 by on 23 de julho de 2016 - 4:22 pm

    Recieved it product prior to the day provided in a really nice gifts box . The particular picture that is displayed for bracelet are exactly what you are going to recieve. This is extremely fine yet any awesome section regarding jewelry that will last as iof the actual toggle clasp and that is quite solid so that the bracelet eill never come off.

  8. #9 by on 23 de julho de 2016 - 4:23 pm

    This is an great gifts of your family. I’d highly incourage individuals to choose this information technology will make a very good gift. On sender does outstanding time task at obtaining your items on duration. I requested our things daily back and I reveived it certainly fast.

  9. #12 by on 23 de julho de 2016 - 5:28 pm

    that you should remember advantage of your respective pursuits prepare is information about the experience shoppers more than the general likes and dislikes. straightforward insight is sufficient to produce youngsters baseball, unfortunately a detailed competence base is very important to varsity as well as,while qualified pursuits. this knowledge also includes reveal comprehension of the actual, ways and strategies important for cost-effective guidance, specializes in along with video day resolutions.

  10. #13 by on 23 de julho de 2016 - 5:28 pm

    The non-public number of Kombat motor bike headwear is designed remembering the issues of dirt biking able to ride whenever verifying ease your consumer. cause Rockstar utilization of the simplest breed of resources while fixing up there dirt biking headwear as best air flow mechanisms contribute towards unequivocable solace much. the costs of other headgear are very inexpensive so therefore for a certain amount of money someone can be creative and rise to the top simply,

  11. #14 by on 23 de julho de 2016 - 6:45 pm

    there are several of different ways their little league batch can also raise the assets it comes to getting. consider, to help you discover some of the with you and try allow you to distinct of some of the traps. I to help take advantage of a past experience and that i just might help you apparent handful of faults,

  12. #15 by on 23 de julho de 2016 - 7:11 pm

    2011 nl mvp jones braun which has used optimistic people for success beautifying prescriptions? Braun has became slated to obtain a 50 on the web revocation to suit checking benefit. is literally your guy going with ped?s. thomas Braun will be without imitating a person who bilk but happened to be away in it above technicality.Nats decide to put an additional bottom presence signature

  13. #16 by black magic specialist on 23 de julho de 2016 - 7:19 pm

    You are my aspiration, I possess few blogs and rarely run out from to brand.

  14. #17 by on 23 de julho de 2016 - 7:28 pm

    He is typically likely to catch 300 deliveries inside of ODI most whether it an experiment event. Both some of these internet players turn to their precious your hands to make and whenever task an ok you find bowlers bowling at pace you must be the job up to. worldwide hopes entail bowlers bowling in the region of 140kmph.

  15. #18 by amil baba on 23 de julho de 2016 - 7:34 pm

    certainly like your web-site but you have to check the spelling on several of your posts. A number of them are rife with spelling problems and I in finding it very bothersome to tell the reality then again I will definitely come back again.

  16. #19 by Louis Vuitton Keepall on 23 de julho de 2016 - 7:47 pm

  17. #20 by amil baba on 23 de julho de 2016 - 7:49 pm

    You could certainly see your skills within the paintings you write. The world hopes for more passionate writers such as you who are not afraid to say how they believe. All the time go after your heart.

  18. #21 by on 23 de julho de 2016 - 8:03 pm

    I obtained this looking for a awesome present for the our mother. On headphone came in a beautiful purple tied box. This is ideal hunting present below some sort of christmas time tree!! Ones headphone always seemed beautiful, though my mother was big boned and headphone are a little tight, although their a good gifts! I enjoy that it.

  19. #22 by on 23 de julho de 2016 - 8:05 pm

    I bought this one searching for a very good gift for the my personal mama. That headphone arrived within a stunning purple tied up container. This is the greatest looking gifts underneath your christmas time tree!! All headphone even looked awesome, but our mama was huge boned and the headphone are somewhat tight, although its outstanding gift! I enjoy this.

  20. #23 by on 23 de julho de 2016 - 8:06 pm

    I paid for your trying to find a very good gifts for the mama. All headphone emerged in a perfect purple tied up box. This is ideal lookin present less than that christmas time tree!! Your headphone and seemed beautiful, nevertheless my personal mother is big boned as well as the headphone is somewhat tight, still its a great gift! I adore things.

  21. #24 by christian louboutin paris on 23 de julho de 2016 - 8:59 pm

  22. #25 by vcc mastercard and visa card buy on 23 de julho de 2016 - 9:11 pm

    please visit the web sites we follow, like this a single, as it represents our picks from the web

  23. #26 by on 23 de julho de 2016 - 10:14 pm

    I acquired this trying to find a very good gift for the our mom. The headphone arrived within a pretty purple tied container. This is a searching present below the christmas tree!! The headphone additionally looked gorgeous, conversely the mama are big boned and headphone is actually some tight, but their a good present! I adore things.

  24. #27 by on 23 de julho de 2016 - 10:15 pm

    I purchased this particular searching for a very good present for the mama. Their headphone arrived in a perfect purple tied box. This is the greatest lookin present underneath the particular christmas time tree!! The particular headphone and seemed perfect, then again the mom is larger boned and the headphone is actually a little tight, then again it really is a great gifts! I prefer that.

  25. #28 by on 23 de julho de 2016 - 10:20 pm

    I purchased this searching for a ideal present for the my personal mom. Your headphone arrived in a gorgeous purple tied up container. This was a searching gifts underneath that the christmas tree!! That the headphone furthermore looked gorgeous, still my personal mother are gigantic boned as well as the headphone are some tight, then again it really is a good gifts! I love that it.

  26. #29 by free sms verification on 23 de julho de 2016 - 10:57 pm

    The data mentioned inside the write-up are a number of the best offered

  27. #30 by on 23 de julho de 2016 - 11:08 pm

    I acquired the looking for a ideal gift to my mom. On headphone emerged in a stunning purple tied package. This was the greatest hunting present underneath the particular christmas time tree!! The headphone furthermore looked gorgeous, but my mom are larger boned and headphone are a little tight, although it is a great gift! I prefer this.

  28. #31 by on 23 de julho de 2016 - 11:08 pm

    I paid for this one looking for a awesome present for my mama. Their headphone emerged in a awesome purple tied up container. This is ideal lookin gift below their xmas tree!! All headphone and looked breathtaking, but my personal mother is larger boned as well as the headphone is actually some tight, although the outstanding present! I like this.

  29. #33 by bard ivc filter on 24 de julho de 2016 - 12:16 am

    I was suggested this website by my cousin. I’m now not sure whether this post is written via him as nobody else know such particular approximately my difficulty. You’re wonderful! Thanks!

  30. #34 by on 24 de julho de 2016 - 12:27 am

    it comes down in a stunning box appearance awesome but the a bit mini still to our parents wrist however it appearances awesome exclusively desire things is much longer

  31. #35 by on 24 de julho de 2016 - 12:28 am

    it comes in a awesome box styles ideal however its a bit limited and even concerning my personal parents wrist however it looks very good simply want that ended up being much longer

  32. #36 by ivc justice for you on 24 de julho de 2016 - 12:33 am

    Hi, i think that i saw you visited my blog so i came to “return the favor”.I am attempting to find things to improve my web site!I suppose its ok to use a few of your ideas!!

  33. #39 by on 24 de julho de 2016 - 12:54 am

    I had gotten this particular gift for my mom to Holiday due she was letter precious jewelry freak. That one thing she does not wear far to was, bracelets. I purchased the lady the appeal bracelet and also anytime she started things yesterday she absolutley liked it! Today the issue is actually, her buying this on and also off of with by herself. Haha… overall great goods, this transported then arrived severely very early and my mother is actually enjoying information technology. Thank we.

  34. #40 by on 24 de julho de 2016 - 12:55 am

    I got that gifts of our mom of Christmas time mainly because she are any jewelry freak. The actual something she does not use much concerning was, bracelets. I bought the lady the charm bracelet additionally whenever she opened information technology yesterday she absolutley loved that! This time the issue are, the lady getting it upon furthermore off of by just by herself. Haha… general very good goods, that it sent as well as appeared really very early and also the mother are enjoying things. Thank one.

  35. #41 by Pierre Dubej on 24 de julho de 2016 - 1:39 am

    Howdy very cool site!! Guy .. Beautiful .. Wonderful .. I’ll bookmark your blog and take the feeds also…I’m glad to search out a lot of helpful information here in the submit, we need work out more techniques in this regard, thank you for sharing. . . . . .

  36. #42 by url on 24 de julho de 2016 - 1:42 am

    Sweet web site, super design and style, rattling clean and employ friendly.

  37. #43 by Bryce Lamkins on 24 de julho de 2016 - 2:18 am

    You really make it appear so easy with your presentation but I find this topic to be really something that I feel I’d never understand. It kind of feels too complex and extremely large for me. I’m looking forward on your next post, I’ll attempt to get the dangle of it!

  38. #44 by Gabrielle Quaile on 24 de julho de 2016 - 2:33 am

    Please let me know if you’re looking for a author for your site. You have some really great articles and I think I would be a good asset. If you ever want to take some of the load off, I’d really like to write some content for your blog in exchange for a link back to mine. Please blast me an email if interested. Thank you!

  39. #45 by life insurance lawyer on 24 de julho de 2016 - 3:48 am

    please take a look at the sites we comply with, such as this 1, as it represents our picks through the web

  40. #46 by Lupe Mccleave on 24 de julho de 2016 - 4:30 am

    Hi there this is kinda of off topic but I was wanting to know if blogs use WYSIWYG editors or if you have to manually code with HTML. I’m starting a blog soon but have no coding experience so I wanted to get guidance from someone with experience. Any help would be enormously appreciated!

  41. #47 by auto body shop on 24 de julho de 2016 - 5:13 am

    Between me and my husband we’ve owned more MP3 players over the years than I can count, including Sansas, iRivers, iPods (classic & touch), the Ibiza Rhapsody, etc. But, the last few years I’ve settled down to one line of players. Why? Because I was happy to discover how well-designed and fun to use the underappreciated (and widely mocked) Zunes are.

1 141 142 143
(não será publicado)