I've also found that for a dehydration headache, water is definitely good, but electrolytes help more. One electrolyte beverage max per 3 of water, or have a salty snack.
Limit caffeine, avoid alcohol.
Urine darker than usual is an early warning sign to drink more, even if you feel OK.