Recent efforts have attempted to understand many of liquid water’s anomalous properties in terms of effective spherically-symmetric pairwise molecular interactions entailing two characteristic length scales (so-called “core-softened” potentials). In this work, we examine the extent to which such simple descriptions of water are representative of the true underlying interactions by extracting coarse-grained potential functions that are optimized to reproduce the behavior of an all-atom model. To perform this optimization, we use a novel procedure based upon minimizing the relative entropy, a quantity that measures the extent to which a coarse-grained configurational ensemble overlaps with a reference all-atom one. We show that the optimized spherically-symmetric water models exhibit notable variations with the state conditions at which they were optimized, reflecting in particular the shifting accessibility of networked hydrogen bonding interactions. Moreover, we find that water’s density and diffusivity anomalies are only reproduced when the effective coarse-grained potentials are allowed to vary with state. Our results therefore suggest that no state-independent spherically-symmetric potential can fully capture the interactions responsible for water’s unique behavior; rather, the particular way in which the effective interactions vary with temperature and density contributes significantly to anomalous properties.