By 'background data' do you mean the pixel values outside of the image or in the background (usually all 0 or No Data)? If so, I do not think that the confusion matrix will be affected by the results as long as the background is included in the Unclassified class. Often you would use a mask in these areas when performing classification so that the masked areas are Unclassified.

By "Background" i means those pixels that were not assigned any class... or more likely 'Unclassified'. While performing "Confusion matrix using ground truth image"... Should i incorporate those backgroundbackground as class or not?

I noticed that if you perform a supervised classification with a mask, the masked pixels are called 'masked pixels' and assigned a class value that is number_of_classes+1. So these pixels (what I think of as background) should be separate from pixels that remain unclassified during a classification due to the thresholds used or the training data. In this case, you do want to keep the 'background' or masked pixels separate from the 'unclassified' pixels which is a valid class that should be considered. So it depends on if you have actual 'unclassified' pixels. If unclassified, then yes, include them. If 'background' or masked, then I would not include them and they should not be combined with actual unclassified pixels within the image.