Do prayer "timings" really matter?

I'm getting sick of people obsessing over little details like whether fajr is at 4:10 or 4:13. Does it really matter? Human beings are fallible. Whatever person made up the "prayer timings" could have made a mistake. Is it really worth it to rely on these things when the Prophet himself just looked at the appearance of the outside sky?