dragging

When I'm out riding and I stop for gas, I can't help but notice how my rear brake rotor feels much hotter than my fronts even though I use the fronts 100x more. Although there's nothing perceptible at the moment in terms of noise or damage, I'm wondering if the rear is dragging just enough...