There have been no changes that I am aware of. Could you post the contents of your files to see if I could make them work?

Here's the first 3 items in each file just to keep it succinct:
datum.txt
DATUM,206001,"D_Mars_2000_IAU_IAG_custom_ellipsoid",207001
DATUM,206002,"D_Mars_2000_IAU_IAG_custom_sphere_lat00",207002
DATUM,206003,"D_Mars_2000_IAU_IAG_custom_sphere_lat05",207003

spheroid.txt:
SPHEROID,207001,"Mars_2000_IAU_IAG_custom_ellipsoid",3396190.0000000000,169.8944472236
SPHEROID,207002,"Mars_2000_IAU_IAG_custom_sphere_lat00",3396190.0000000000,!values.f_infinity
SPHEROID,207003,"Mars_2000_IAU_IAG_custom_sphere_lat05",3396036.8126024301,!values.f_infinity

geogcs.txt:
GEOGCS,204001,"GCS_Mars_2000_IAU_IAG_custom_ellipsoid",206001,PE_PM_REFERENCE,PE_U_DEGREE
GEOGCS,204002,"GCS_Mars_2000_IAU_IAG_custom_sphere_lat00",206002,PE_PM_REFERENCE,PE_U_DEGREE
GEOGCS,204003,"GCS_Mars_2000_IAU_IAG_custom_sphere_lat05",206003,PE_PM_REFERENCE,PE_U_DEGREE

I'm running ENVI 5.4/IDL 8.6 under Red Hat linux 6.8. I put the files in:
$IDL_DIR/resource/pedata/user_defined/
The same files have been used successfully under windows.

Are you saying these files work fine in ENVI 5.4 on Windows but not Linux? I added these to my user_defined directory on Windows and can see them under User Defined GCS when defining a new projection type.

That's right, I have not been able to see them under linux. Another user has reported success under windows.

When you go to select the projection you want, do you use the "New" option to create a new coordinate system string? Here you can select the user-defined geogcs. Are you able to open this dialog and if so, does it show your user-added geogcs?