But, what does goat milk taste like? While there's a difference between store-bought and fresh goat milk (as there is with all types of milk), it's always rich, creamy, and quite thick.