what does water actually taste like before all the chemicals in it to clean it?

Pure water would taste like death to you. It takes on molecules easily, and it’d strip your throat and tongue.

Use a reverse osmosis filter to get as close as you can while being safe.