Image shader programs are particularly easy to create:
- The IDLgrImage object uses a texture-mapped polygon to draw the image. Most image filters do not change the size or the shape of the image, making it unnecessary to modify the vertices of the polygon. Therefore, a very trivial vertex shader component is all that is required.
- Each image pixel color is going to be completely determined by the image filter calculation, with no lighting or shading effects. Therefore, there is no need to worry about applying lighting and shading calculations in a shader program. This further simplifies the shader program.
There are several ways to incorporate shader functionality into an image processing application. You can either use one of the pre-built shader objects (IDLgrShaderBytscl or IDLgrShaderConvol3) or create a custom shader program. If you design your own shader, you have additional options that include using a IDLgrFilterChain object to link a number of shaders together and apply them successively to the image data. See the following topics for sample applications:
- Library of Pre-built Shader Objects: Provides information on the pre-defined IDLgrShaderBytscl and IDLgrShaderConvol3 objects. These are excellent options if you need byte scaling or convolution filtering functionality, and do not want to write custom GLSL shader programs.
- Altering RGB Levels Using a Shader: Creates a simple shader program that allows you to interactively alter the red, green or blue levels in an RGB image.
- Applying Lookup Tables Using Shaders: Loads a LUT into a one-dimensional image object so that the shader program can efficiently accesses it as a texture map.
- High Precision Images: Shows how to display 16-bit and 11-bit images using the full precision of the data and how to display an 11-bit image with a contrast adjustment LUT.
- Filter Chain Shaders: Lets you apply a sequence of shaders to a single image object.
Providing a Software Alternative to Shaders
When the appropriate graphics hardware support is missing, a software-based alternative can be provided for image processing applications (when the shader program is associated with an IDLgrImage).
When IDL renders the image and hardware shader support is missing, the IDLgrShader::Filter method is automatically called when the image is drawn. (You never call this method directly.) In this method, add code that provides a software-based equivalent to the shader program functionality that will be used when there is insufficient hardware support for the shader program. See “Hardware Requirements for Shaders” on page 321 for graphics card requirements.
Note: When developing a software fallback, use the FORCE_FILTER property during shader object initialization to force the system to test the software-based alternative even when sufficient hardware support is available.
If there is no software fallback specified, application execution continues as if there were no shader program. Also, no software fallback is available when a shader is associated with a non-image object.
Caching Shader Results
If a shader object is associated with an IDLgrImage object, you may set the IDLgrShader CACHE_RESULT property to determine whether a shader program is executed every time Draw is called. If this property is set to 1, the image is cached after running the shader program and the cached image is used in subsequent draws until shader program parameters are changed. If this property is set to 0 (the default), the result of running the shader program is not cached. See the property description for details on when each CACHE_RESULT setting may prove more useful. If a software fallback is used, the result is always cached.
Capturing Image Data During Shader Execution
When you apply one or more shader programs to image data, you can capture the results of the image filtering shader operation using the IDLgrImage ReadFilteredData method. Using this method, you can capture a portion of a tiled image, capture the image after applying a single shader, or capture the image after applying any number of shaders in a filter chain sequence. See IDLgrImage::ReadFilteredData for details. After reading the data, you can place it in a new image object and print or display the result.
Altering RGB Levels Using a Shader
This shader program example lets you interactively apply color level correction to an image when you view it. This does not modify the image data itself. This example places the original image data in an IDLgrImage object and attaches the custom shader object using the SHADER property. It then creates a simple user interface that lets you alter the color levels and passes these values to the shader program in a named uniform variable. The Filter method implements the software fallback. When the correct graphics hardware is unavailable, IDL automatically calls the Filter method.
Example Code
See shader_rgb_doc__define.pro, located in the examples/doc/shaders subdirectory of the IDL distribution, for the complete, working example. Run the example by creating an instance of the object at the IDL command prompt using orgbshader=OBJ_NEW('shader_rgb_doc') or view the file in an IDL Editor window by entering .EDIT shader_rgb_doc__define.pro.
The example code differs slightly from that presented here for the sake of clarity. Whereas the working example includes code needed to support user interface interaction, the following sections leave out such modifications to highlight the shader program components.
Basic RGB Shader Object Class
First, create a basic object class that inherits from IDLgrShader:
FUNCTION shader_rgb_doc::Init, _EXTRA=_extra
IF NOT self->IDLgrShader::Init(_EXTRA=_extra) THEN $
RETURN, 0
RETURN, 1
END
PRO shader_rgb_doc::Cleanup
self->IDLgrShader::Cleanup
END
FUNCTION shader_rgb_doc::Filter, Image
RETURN, Image
END
PRO shader_rgb_doc__define
COMPILE_OPT hidden
struct = { shader_rgb_doc, $
INHERITS IDLgrShader $
}
END
Uniform Variable for RGB Values
In this example, a uniform variable contains the values of the red, green and blue levels. You can set or change uniform variables anytime before you draw the scene and their values will remain in effect until you change them again. These types of variables are perfect for making minor adjustments to the image filter and then viewing the image to see if the result is satisfactory. First set the uniform variable to a reasonable default value such as [1,1,1] before you start, otherwise the shader program defaults of [0,0,0] will make the image look dim.
Add the following line to your Init function:
self->SetUniformVariable, 'scl', [1.0, 1.0, 1.0]
Note: The uniform variable name is case-sensitive, unlike most variable names in IDL.
This example lets you change color levels using sliders. You can read the slider values from your GUI, and modify the uniform variable at any time. Assuming that the instance of your shader_rgb_doc object is called oShaderRGB and red, green and blue are floating point values, update the value of the uniform variable as follows:
oShaderRGB->SetUniformVariable, 'scl', [red, green, blue]
Once the needed elements are defined, associate your shader object with oImage, an
image object (that has been previously defined).
oImage->SetProperty, SHADER=self
Once the shader object is associated with the image, shader program display updates are activated any time the SetUniformVariable method is called.
Software Fallback for RGB Shader
IDL calls the Filter method when shader functionality is not supported by the graphics hardware. Providing a software-based fallback is never a requirement and you may choose not to if you know sufficient hardware will always be available.
However, it is good practice to write this method just in case the application is ever executed on a machine without suitable hardware.
In the Filter method, retrieve the uniform variable values using GetUniformVariable, and then return a modified copy of the image data.
Function shader_rgb_doc::Filter, Image
newImage=Image
self->GetUniformVariable, 'scl', s
newImage[0,*,*] *= s[0]
newImage[1,*,*] *= s[1]
newImage[2,*,*] *= s[2]
RETURN, newImage
END
IDL always passes the image to the Filter method in RGBA floating-point pixelinterleaved format, so you do not have to worry about a lot of input data combinations. IDL also clamps the data this function returns to the [0.0, 1.0] range and scales it to the correct pixel range, usually [0, 255], for your display device.
Note: Uniform variables are, in a sense, free-form properties in the IDLgrShader superclass. Within the Filter method, accessing the scale vector from the uniform variable maintains consistency since this is same place the hardware shader obtains it. This reduces the chance for confusion.
At this point, you can test your work by writing a simple display program that loads your data into an IDLgrImage object, creates an instance of your shader_rgb_doc object, and attaches the filter to your image object by setting the object reference of the shader in the SHADER property of IDLgrImage. You also need to set the FORCE_FILTER property on class initialization so that the filter fallback runs, even if you have shader hardware. You can force use of the fallback either when creating the shader object:
oShaderRGB = OBJ_NEW('shader_rgb_doc', /FORCE_FILTER)
or explicitly in the shader object’s Init method:
FUNCTION shader_rgb_doc::Init, _EXTRA=_extra
IF NOT self->IDLgrShader::Init(_EXTRA=_extra, /FORCE_FILTER) $
THEN $
RETURN, 0
...
Hardware Shader Program for RGB Shader
The OpenGL Shading Language (GLSL) is a vast subject that requires extensive study to develop an expert level of programming, a subject that is impossible to cover here. However, this example is relatively simple and you can likely easily follow along with the code required for the vertex and fragment shader portions of the shader program. All shader programs need a vertex program and a fragment program.
RGB Vertex Shader Program
The following vertex shader is fairly common among image filtering shader programs. Add the following code to the bottom of your Init function:
vertexProgram = $
[ $
'void main (void) {', $
' gl_TexCoord[0] = gl_MultiTexCoord0;', $
' gl_Position = ftransform();', $
'}' ]
The first line after main() transfers the texture coordinate from OpenGL's Texture Unit 0 into the current texture coordinate predefined variable. Remember that IDL draws its images with texture maps applied to rectangles, so you need to pass along the texture coordinate. IDL always uses Texture Unit 0 when drawing images. The gl_TexCoord[0] is a varying variable that transmits data from the vertex program to the fragment shader program.
The next line in the program applies the current OpenGL ModelViewProjection transform to the vertex, so that it ends up in the right place on the screen.
RGB Fragment Shader Program
The fragment program of an image filtering shader program is where all the work happens. Add the following to the Init function as well:
fragmentProgram = $
[ $
'uniform sampler2D _IDL_ImageTexture;', $
'uniform vec3 scl;', $
'void main(void) {', $
'vec4 c = texture2D(_IDL_ImageTexture, gl_TexCoord[0].xy);', $
' c.rgb *= scl;', $
' gl_FragColor = c;', $
'}' ]
This GLSL code can be translated relatively easily. IDLgrImage uses textures to draw image data. Access the texture map associated with the base image's data in the IDL reserved uniform variable, _IDL_ImageTexture, which is automatically created for the base image. The sixth line in the program above fetches the image pixel (a texture texel) from the image texture and stores it in c, which is a 4-element vector that represents the RGBA channel data. Modify the color of the texel in the next tile using the uniform variable, scl, declared on line four. Finally, tell OpenGL about the new color for this particular pixel on the screen by setting gl_FragColor. OpenGL clamps the pixel color values to the appropriate range for your display.
This fragment program runs once for every pixel (fragment) on your screen that is covered by the image.
Assign RGB Shader Program to Shader Object
You need to supply the program code to the shader object so that it is available to the graphics card when it is needed. To accomplish this, you can use shader object properties VERTEX_PROGRAM_STRING and FRAGMENT_PROGRAM_STRING to associate inline shader program components with the shader object.
Note: With more complicated (longer) shader programs, it may be easier to keep the shader program components in separate files. In such a case, associate the shader program elements with a shader object using the VERTEX_PROGRAM_FILE and FRAGMENT_PROGRAM_FILE properties.
Add the following code to the bottom of your Init function.
self->IDLgrShader::SetProperty, $
VERTEX_PROGRAM_STRING=STRJOIN(vertexProgram, STRING(10B)), $
FRAGMENT_PROGRAM_STRING=STRJOIN(fragmentProgram, STRING(10B))
Add newlines (STRING(10B)) so that the shader program compiler sees your program as a single long string containing many source code lines, instead of one long line. If you ever get a compile-time error, the shader compiler can tell you on what line the error occurred when you insert the newlines.
Tip: Remove the FORCE_FILTER keyword from the initialization function if you have been testing your software fallback. The following image shows the result of modifying the RGB levels of an image of a rose.