Adding a Processor

Processors are one of the main components of Voreen (see here). Therefore a better understanding of processors is crucial for development and implementation in Voreen. In this paragraph we want to explain the main functions of a Voreen processor targeting developers to increase their understanding of the class and guide them during their first experience in implementing their own processors and functionality into Voreen.

Important Processor Functions

In this section we will introduce the most important functions used by each Voreen processor.

Pure Virtual Functions

A basic Voreen processor has five pure virtual functions which have to be implemented by the developer. Two of them are inherited by VoreenSerializableObject which is the base class of most objects in Voreen and the other three pure virtual functions are defined in the Processor class itself.

virtual std::string VoreenSerializableObject::getClassName() const = 0;

The function should return the type name of the concrete subclass as string. Necessary due to the lack of code reflection in C++.

virtual VoreenSerializableObject* VoreenSerializableObject::create() const = 0;

This function is a virtual constructor. It is supposed to return an instance of the concrete subclass i.e. a Processor pointer in our case.

virtual std::string Processor::getCategory() const = 0;

It returns the general category the processor belongs to. This method is not intended to be re-implemented by each subclass, but is rather defined by the more concrete base classes, such as VolumeRaycaster or ImageProcessor. The category is used in the VoreenVE GUI to group processors of same categories.

virtual void Processor::setDescriptions() = 0;

This method should call Processor::setDescription(std::string desc) with a parameter string describing the functionality of the processor and containing a user manual if the usage is not intuitive. The string is shown to the user in the VoreenVE GUI. It is possible to use html code for highlighting important information or embedding links to websites in the description.

virtual void Processor::process() = 0;

This function contains the main functionality of the processor. If the processor is initialized, ready and invalid during network evaluation, the process function will be called by the network evaluator. All image/volume rendering or data processing should be done here. Some examples of this function can be found in the section Processor Examples.

Frequently Used Functions

The following Processor functions are frequently used and an understanding of their functionality is quite important to avoid Voreen concept based implementation errors and should accelerate the debugging process.

virtual void Processor::initialize();

A newly created processor has not been initialized yet. During the first evaluation of the processor this function will be called by the evaluator. Since it will only be called once in the lifetime of the processor, all stuff that has to be done once, e.g. loading shaders, should be done here. The superclass' function must be called as first statement when it is overwritten, since property and processor widget initializations are done in it. All OpenGL initializations must be done here, instead of the constructor, because normally during the constructor call no OpenGL-context is available!

If the initialization fails an exception is thrown and the processor is flagged as not initialized as being shown in the network editor. Since this function will only be called once, a non initialized processor is a critical error and the loaded network will not work properly.

virtual void Processor::deinitialize();

The processor will be deinitialized before the destructor of the processor will be called. Like the destructor cleans up everything that has been done in the constructor, deinitialize should clean up everything that has been done in the initialize function. The superclass' function must be called as last statement when it is overwritten. All OpenGL deinitializations must be done here, instead of the destructor, since normally no OpenGL-context is available during destruction!

virtual bool Processor::isReady() const;

The process function of a processor will only be called, if the processor has been initialized, i.e. initialize() has been called successfully, and the processor is ready. The default implementation checks, if all ports of the processor are connected and the inports have valid data. If a port of a processor is optional and don't have to be connected, this function must be overwritten. A non ready processor will be highlighted in the network editor. Being not ready is a not critical error. The network will not be evaluated properly if a processor is not ready, but instead of a failed initialization it can become ready by adding e.g. port connections. 

enum Processor::InvalidationLevel {
    VALID = 0,
    INVALID_RESULT = 1,        ///< invalid rendering, volumes => call process()
    INVALID_PARAMETERS = 10,   ///< invalid uniforms => set uniforms
    INVALID_PATH = 15,         ///< path has been changed => update if necessary
    INVALID_PROGRAM = 20,      ///< invalid shaders, CUDA/OpenCL program => rebuild program
    INVALID_PORTS = 30,        ///< ports added/removed  => check connections, re-evaluate network
    INVALID_PROCESSOR = 40     ///< invalid python/matlab processor => re-create and re-connect processor
};
virtual void PropertyOwner::invalidate(int inv = 1);

The invalidate function is defined in the PropertyOwner class, but since it is mostly used by processors the common values are defined in the Processor class. The process function of a processor is only called, if the invalidation level of the processor is invalid e.g. greater zero. A processor normally becomes invalid trough property value changes or if inports get new data. Depending on the invalidation level, different actions could be triggered as been seen by the enum definitions. It is possible to define new invalidation levels by the developer. If the invalidate function of a processor is called, a new network evaluation will be scheduled,  thereby the evaluator can call the process function of the processor. The invalidation level will be automatically set back to valid by the evaluator after the process method has been performed.

virtual void Processor::addPort(Port* port);
virtual void Processor::removePort(Port* port);
virtual void PropertyOwner::addProperty(Property* prop);
virtual void PropertyOwner::removeProperty(Property* prop);

If a processor has ports or properties they have to be added in the constructor via the specific add functions. Otherwise they will not be shown in the VoreenVE GUI and will not be handled correctly by internal processes. The remove functions are normally not used, since ports and properties of a processor are not intended to be dynamic, although it would be possible.

To be continued...

Integration into Voreen

Assuming our new processor has been already implemented, we can integrate it as described here.

Processor Examples

In this section we want to introduce a few simple processor examples which should help developers during their first implementations. All examples can be found in the current Voreen release within the module "SampleModule".   

Sample Processor

Sample1

The SampleProcessor is a spartan version of a Voreen processor. It just implements all pure virtual functions inherited from the base classes. The processor can be tested in the workspace "SampleProcessor_workspace.vws" in the SampleModule. The image above is showing this workspace. In the workspace a TextSource provides the text "Hello World!". The SampleProcessor simply adds the prefix "Simon says:" to its front. The other three processors (Background, TextOverlay and Canvas) only produce a visible output. The Canvas shows the original text on the top and the modified text on the bottom. The user can change the orignial text and prefix by modifying the StringProperties on the right-hand side. We will now have a closer look into the implementation details.

sampleprocessor.h

#ifndef VRN_SAMPLEPROCESSOR_H
#define VRN_SAMPLEPROCESSOR_H
 
//add base class header
#include "voreen/core/processors/processor.h"
 
//add used port headers
#include "voreen/core/ports/textport.h"
 
//add used property headers
#include "voreen/core/properties/stringproperty.h"
 
//use namespace voreen
namespace voreen {

The first part of the header is simple basic c++. We check, if the processor has been already defined and add all needed headers. It should be mentioned, that all Voreen processors are defined and implemented in the voreen namespace.

/**
 * Sample processor, which adds a user-defined prefix to a given text.
 * VRN_CORE_API is a macro needed for shared libs on windows (see voreencoreapi.h)
 */
class VRN_CORE_API SampleProcessor : public Processor {
public:
    /**
     *  Constructor
     */
    SampleProcessor();

After all includes have been set we can start the class definition. All Voreen processors should inherit from the processor base class. The VRN_CORE_API macro is needed for DLL inports and exports under windows. This macro has no effect under Linux.

The constructor of a processor should have no parameter! All parameters the processor needs should be handed to it via ports and properties.  

    //------------------------------------------
    //  Pure virtual functions of base classes
    //------------------------------------------
 
    /**
     * Virtual constructor
     * @see VoreenSerializableObject
     */
    virtual Processor* create() const
 
    /**
     * Function to get the class name, which is not
     * directly supported by c++.
     * @see VoreenSerializableObject
     */
    virtual std::string getClassName() const;
 
    /**
     * Function to return the catagory of the processor.
     * It will be shown in the VoreenVE GUI.
     * @see Processor
     */
    virtual std::string getCategory() const;
 
protected:
 
    /**
     * Function to set a description of the processors functionality.
     * It will be shown in the VoreenVE GUI.
     * @see Processor
     */
    virtual void setDescriptions();
 
    /**
     * The main function of each processor.
     * The main functionality should be implemented here.
     * @see Processor
     */
    virtual void process();

Here we define all pure virtual functions of the base classes. More information on this functions were presented in the section Pure Virtual Functions.

private:
 
    //-------------
    //  members
    //-------------
    TextPort inport_;           ///< inport used to receive the text to be modified
    TextPort outport_;          ///< outport used to pass the modified text
    StringProperty prefixProp_; ///< property for the user-defined prefix
};
 
} // namespace
 
#endif // VRN_SAMPLEPROCESSOR_H

Last but not least we have to add the class members to our processor. In our case they are two ports for the in- and output of the text and a property to modify the prefix by the user.

sampleprocessor.cpp

1
2
3
4
5
//include header file
#include "sampleprocessor.h"
 
//using namespace voreen
namespace voreen {

Like in sampleprocessor.h only basic c++ at the beginning of the cpp file.

SampleProcessor::SampleProcessor()
    //constructor of base class
    : Processor()
    //constructors of ports
    , inport_(Port::INPORT,             ///< port type, i.e inport
             "inport",                  ///< unique port ID (unique for each processor)
             "Unmodified Text Inport"///< port name used in the VoreenVE GUI
    , outport_(Port::OUTPORT,           ///< port type, i.e outport
              "outport",                ///< unique port ID (unique for each processor)
              "Modified Text Outport"   ///< port name used in the VoreenVE GUI
    //constructor of property
    , prefixProp_("prefixProp",         ///< unique property ID (unique for each processor)
                  "Prefix",             ///< property name used in the VoreenVE GUI
                  "Simon says: ")       ///< default value of the property
{
    //register ports
    addPort(inport_);
    addPort(outport_);
    //register properties
    addProperty(prefixProp_);
}

The constructor of each implemented processor in Voreen looks nearly the same. Fist we call the base class constructor and initialize its members. In the function body itself we have to add the ports and properties by calling the specific add functions. It is needed to get a proper internal handling of ports and properties belonging to the processor. 

Processor* SampleProcessor::create() const {
    return new SampleProcessor();
}
 
std::string SampleProcessor::getClassName() const {
    return "SampleProcessor";
}
 
std::string SampleProcessor::getCategory() const {
    return "Text Processing";
}
 
void SampleProcessor::setDescriptions() {
    setDescription("My sample processor which adds a " \
                   "user defined prefix to a given text.");
}

These four functions are always implemented in this way. Beside name changes nothing more has to be done.

void SampleProcessor::process() {
    //get inport data
    std::string inString = inport_.getData();
    //get prefix string
    std::string prefixString = prefixProp_.get();
    //combine both strings
    std::string outString = prefixString + inString;
    //set outport data
    outport_.setData(outString);
}
 
} // namespace

This is the implementation of the "main" processor function. First we get our needed parameters from the inports and properties. Then we process them, e.g. combining both strings. Afterwards we fill our outports and the SampleProcessor is ready to use.

Sample Render Processor

Sample2

After introducing the first simple processor we now want to make it a little bit more complex. This time we will render images via a GLSL shader in the SampleRenderProcessor. The processor and the according workspace "SampleRenderProcessor_workspace.vws" from the SampleModule are illustrated in the image above. The SampleRenderProcessor gets an image and modifies the saturation of the colors. The saturation can be manipulated by the user via a FloatProperty on the right-hand side. A saturation of zero equals a black-white image as shown in the Canvas. The colorful tooltip shows the original image in the ImageSource outport to validate the proper work of the SampleRenderProcessor.  

samplerenderprocessor.h

#ifndef VRN_SAMPLERENDERPROCESSOR_H
#define VRN_SAMPLERENDERPROCESSOR_H
 
//header of base class
#include "voreen/core/processors/renderprocessor.h"
//port headers
#include "voreen/core/ports/renderport.h"
//property headers
#include "voreen/core/properties/floatproperty.h"
//header of the used shader object
#include "tgt/shadermanager.h"
 
//use namespace voreen
namespace voreen {

We include all needed headers and set the namespace to voreen.

/**
 * Sample render processor for gray-scaling an input image using a user-defined parameter.
 * VRN_CORE_API is a macro needed for shared libs on windows (see voreencoreapi.h)
 */
class VRN_CORE_API SampleRenderProcessor : public RenderProcessor {
public:
    /**
     * Constructor
     */
    SampleRenderProcessor();
 
    //------------------------------------------
    //  Pure virtual functions of base classes 
    //------------------------------------------
    virtual Processor* create() const { return new SampleRenderProcessor();     }
    virtual std::string getClassName() const { return "SampleRenderProcessor";  }
    virtual std::string getCategory() const  { return "Image Processing";       }
protected:
    virtual void setDescriptions() { setDescription("Sample render processor for" \
                                                    "gray-scaling an input image."); }
    virtual void process();

Here we are defining the constructor and implementing inline the four simple pure virtual functions of the base classes. Since our processor should render images and use RenderPorts it inherits from RenderProcessor. RenderProcessor is a subclass of Processor implementing specific functions for handling render targets internal.

/**
 * Overwrites the base implementation of this function.
 * It is used to load the needed shader.
 * @see Processor
 */
virtual void initialize();
 
/**
 * Overwrites the base implementation of this function.
 * It is used to free the used shader.
 * @see Processor
 */
virtual void deinitialize();

These two functions are new and were not used in the first example. They overwrite the basic implementation of the function in RenderProcessor or rather in Processor. They are used to load and free the needed shader.

private:
    //-------------
    //  members   
    //-------------
    RenderPort inport_;             ///< input of the image which should be modified
    RenderPort outport_;            ///< output of the modified image
    FloatProperty saturationProp_;  ///< property for the color saturation parameter
 
    tgt::Shader* shader_;           ///< GLSL shader object used in process()
};
 
} // namespace
 
#endif // VRN_SAMPLERENDERPROCESSOR_H

The three first members are, except for the type, the same as in the SampleProcessor. The fourth is a shader object used to handle shaders in the process() function.

samplerenderprocessor.cpp

//header file
#include "samplerenderprocessor.h"
//needed headers (used in process())
#include "tgt/textureunit.h"
 
//we are in namespace voreen
namespace voreen {

Simply including headers and setting the namespace.

SampleRenderProcessor::SampleRenderProcessor()
    : RenderProcessor()
    , inport_(Port::INPORT, "inport", "Unmodified Image")
    , outport_(Port::OUTPORT, "outport", "Modified Image")
    , saturationProp_("saturation", "Saturation"    ///< property ID and GUI-label
                     ,0.5f                          ///< default value
                     ,0.f,1.f)                      ///< min and max value
{
    //register ports
    addPort(inport_);
    addPort(outport_);
    //register properties
    addProperty(saturationProp_);
}

The constructor is nearly the same as the one of the SampleProcessor. We want to point out again the inheritance of the RenderProcessor, which causes the call of RenderProcessor instead Processor constructor.

void SampleRenderProcessor::initialize() {
    // call superclass function first
    RenderProcessor::initialize();
 
    // load fragment shader 'sample.frag'
    shader_ = ShdrMgr.loadSeparate("passthrough.vert", "sample.frag",
                                   generateHeader(), // see RenderProcessor
                                   false);           // do not set default flags
}
 
void SampleRenderProcessor::deinitialize() {
    // free shader
    ShdrMgr.dispose(shader_);
    shader_ = 0;
 
    // call superclass function last
    RenderProcessor::deinitialize();
}

These are the implementations of the overwrite functions. The ShaderManager (ShdrMgr) is a singleton class handling all shaders. In the initialize function we load the used pair of vertex and fragment shader. The vertex shader is simply the one used in the OpenGL pipeline and the fragment shader will be shown later on. The function generateHeader() is implemented in RenderProcessor and sets the default shader headers i.e. shader parameters. Deinitialize simply frees the loaded shaders. 

void SampleRenderProcessor::process() {
    // activate and clear output render target
    outport_.activateTarget();
    outport_.clearTarget();
 
    // bind input image to texture units
    tgt::TextureUnit colorUnit, depthUnit;
    inport_.bindTextures(colorUnit.getEnum(), depthUnit.getEnum());
 
    // activate shader and pass data
    shader_->activate();
    setGlobalShaderParameters(shader_); // see RenderProcessor
 
    // pass input image to shader
    inport_.setTextureParameters(shader_, "textureParameters_");
    shader_->setUniform("colorTex_", colorUnit.getUnitNumber());
    shader_->setUniform("depthTex_", depthUnit.getUnitNumber());
 
    // pass property value to shader
    shader_->setUniform("saturation_", saturationProp_.get());
 
    // render screen aligned quad to run the fragment shader
    renderQuad(); //(see RenderProcessor)
 
    // cleanup
    shader_->deactivate();
    outport_.deactivateTarget();
    tgt::TextureUnit::setZeroUnit();
 
    // check for OpenGL errors
    LGL_ERROR; // see tgt_gl.h
}
 
} // namespace

Other than all other ports RenderPorts don't have a getData or setData function. To set the data of a render outport we have to activate its render target. After the rendering we have to deactivate it again. To get the data (image) of the inport, we have to bind its internal textures (see line 8). After everything is done, we can check for OpenGL errors appeared during the rendering with the LGL_ERROR macro.

sample.frag

//include shader libraries (shader modules)
 
//defines and functions for 2D textures
#include "modules/mod_sampler2d.frag"
//defines and functions for filtering
#include "modules/mod_filtering.frag"
 
//uniforms for the color and depth textures
uniform sampler2D colorTex_;
uniform sampler2D depthTex_;
 
//struct defined in mod_sampler2d.frag containing all needed parameters
uniform TextureParameters textureParameters_;
 
//the saturation (user-defined by a property)
uniform float saturation_;

In the first part of the fragment shader we include Voreen shader libraries (a.k.a. modules). They offer some basic functions and defines which are frequently used. Next we define all uniforms being needed for our shader.

/**
 * Main function of the shader. It takes the color texture passed as a uniform
 * and modifies the saturation.
 */
void main() {
    // look up input color and depth value (see mod_sampler2d.frag)
    vec4 color = textureLookup2Dscreen(colorTex_, textureParameters_, gl_FragCoord.xy);
    float depth = textureLookup2Dscreen(depthTex_, textureParameters_, gl_FragCoord.xy).z;
 
    // compute gray value (see mod_filtering.frag) and pass-through depth value
    FragData0 = rgbToGrayScaleSaturated(color, saturation_);
    gl_FragDepth = depth;
}

The main shader function simply gets the texture color of the actual position and modifies it by the user-defined saturation.