If the text on your server row is anything other than white, attempting to change the color of the widget will have a strange overlay effect that mixes the original text color and newly set widget color. To set the color of the text “explicitly” without any distortion from the original color, simply set the text color to inherit from the widget color:
Note: in order for this to work, the normal, hover, and pressed foregrounds on the button must all be white