reverse osmosis.ewwww

Does reverse osmosis supposed to taste nasty or is something not right? How do I know when its time to change filters?