Design Patterns: Implementing Pipeline design pattern

Understanding pipeline design pattern with real world example.

Diagram created by Jamil Moughal using ExcaliDraw

What is a Pipeline design pattern?

Pipeline design pattern is a software design pattern that process or execute a series of steps or stages in a linear sequence. It allows you to break down the complex tasks into smaller and modular steps or stages that can be executed in an order. Each step is taking input from the previous step, executing its specific functionality and producing output for the next step.

The pipeline design pattern promotes separation of concerns and improves maintainability by encapsulating each step’s logic in a separate component or class. It also enables the easiness in extensibility and flexibility as new steps can be added and existing steps can be modified without affecting the overall pipeline.

Key Elements of the Pipeline design pattern

Steps: Steps are the individual components or classes which perform a specific task on the input data.

Input and Output: Each step is taking input, processing it and producing output data for next step.

Order: Steps are performed in a predefined order while setting up the pipeline.

Context: Context is an optional shared object that can be passed between the steps, containing data or state relevant to the entire pipeline.

Implementation in C#

Let’s look at some real world scenario. Suppose we are working on a project which have some Image Processing functionality. It have several features like Loading, Filtering and Saving an Image. These three features may have some complex and detailed implementation like:

Loading Image have different strategies to load image like loading from local computer, from google drive, from azure blob etc.

Filtering Image have strategies like Blur filtration, Sepia filtration and Greyscale filtration.

Saving Image have different logic for storing image like to store with some image format, convert it into pdf and store it. And storing it on local machine or on cloud.

These are the features that our Image Processor. These three features are completely different and can be implemented as separate module.

First we have an Image class:

// Image class
public class Image
{
    // Properties and methods of the image
}

Then we have a IPipelineContext interface which may have some common methods and properties.

// Pipeline context interface
public interface IPipelineContext
{
    // Common context properties and methods
}

We have a IPipelineStep interface which have a method Execute and every step of pipeline will be implementing this interface.

// Pipeline step interface
public interface IPipelineStep
{
    void Execute(IPipelineContext context);
}

Now we are implementing our first step, ImageLoading step. And it is implementing IPipelineStep interface.

// Image loading step
public class ImageLoadingStep : IPipelineStep
{
    public void Execute(IPipelineContext context)
    {
        // Load the image from a file or any source
    }
}

Next step is ImageFiltration step, it is also inheriting from IPipeline step.

// Image filtering step
public class ImageFilteringStep : IPipelineStep
{
    public void Execute(IPipelineContext context)
    {
        Image image = context as Image;
        if (image != null)
        {
            // Apply the filtering strategy to the image
        }
    }
}

Final step is ImageSavingStep, which is also implementing IPipelineStep.

// Image saving step
public class ImageSavingStep : IPipelineStep
{
    public void Execute(IPipelineContext context)
    {
        Image image = context as Image;
        if (image != null)
        {
            // Save the image to a file or any destination
        }
    }
}

We have implemented steps, next thing is to create the pipeline called ImageProcessingPipeline. This pipeline have a method to AddStep and another method ExecutePipeline.

// Image processing pipeline
public class ImageProcessingPipeline
{
    private readonly List<IPipelineStep> _steps;

    public ImageProcessingPipeline()
    {
        _steps = new List<IPipelineStep>();
    }

    public void AddStep(IPipelineStep step)
    {
        _steps.Add(step);
    }

    public void ExecutePipeline(IPipelineContext context)
    {
        foreach (var step in _steps)
        {
            step.Execute(context);
        }
    }
}

We have created the pipeline and steps. Now we are going to use this pipeline like how we can use it in our code.

//creating the Image object
        Image image = new Image();
        
        //creating and initializing pipeline
        ImageProcessingPipeline pipeline = new ImageProcessingPipeline();
        
        //creating ImageLoadingStep and adding in our pipeline
        pipeline.AddStep(new ImageLoadingStep());

        //creating ImageFilteringStep and adding in our pipeline
        pipeline.AddStep(new ImageFilteringStep());

        //creating ImagesAVINGStep and adding in our pipeline
        pipeline.AddStep(new ImageSavingStep());

        //when all steps are created, calling executepipeline function which will execute all steps
        pipeline.ExecutePipeline(image);

In above code we are creating an instance of Image class. Then we are creating and initializing pipeline object and adding steps. After adding all steps that we need, we are calling ExecutePipeline function which will execute methods of all steps.

It is not necessary to add all steps in the pipeline. We can add more steps and we can ignore some steps as well. But some steps are not ignorable. For example, ImageLoadingStep is loading image from some source, if we don’t add and execute this step then image will not be loaded and none of other steps will be executed.

I hope this example helps you understanding pipeline design pattern in the context of Image Processing context.