If you would like to fully delete a file from the system (and not only closing it and removing it from the ENVI interface) I don't think it can be done directly from the ENVI interface.
However this could be easily developed using IDL (and the FILE_DELETE routine for example):
http://www.exelisvis.com/docs/FILE_DE...
You could also create a new extension in ENVI to add this option in ENVI afterwards:
http://www.exelisvis.com/docs/Toolbox...