ENVI's SVM tool (which is from ENVI Classic) does not support masks or No Data values. However, the new SVM Classifier from the Classification framework does but it requires you use the API:

http://www.harrisgeospati...mclassifiertask.html
http://www.harrisgeospati...examplesvmtasks.html

With this routine, you can pass a MaskRaster:
http://www.harrisgeospati.../envimaskraster.html

Is this only available in ENVI 5.4?

Could i not just spatially subset the study area instead of using a mask?

Yes, the classification framework API was added in ENVI 5.4. If your mask only covers background area and not pixels within and throughout the image, then yes, you can simply subset out the background that you do not want included. It will still need to be a rectangular area. If you subset via an ROI or shapefile, you can still have 'background' for areas outside of the ROI/shapefile that will not be masked.