This was set to color fill - non shaded, but I don't know if changing this will cause me problems later... Because if I change the colour of the fill in the Floor Pan options, it still doesn't accept the colour I choose. Now it's ALWAYS white.
When set to Surface - Colour Fill, the 3D document will display the Surface Colour of your material, in this case, Concrete 04.
If you do not wish to change this, you could add a rule to your Graphic Override which will override the pen colour of your desired floor planes. eg. If a slab uses the surface Concrete 04, set Fill Type to foreground and set Fill Foreground Pen to white. Also ensure that the surface type override Cover/Uncut is enabled.
Didn't really undestand the graphic override stuff, but I'm fine with changing the surface configuration. I actually tried changing the material configuration (structural concrete) before, but I guess it was the surface. Thanks!