Going to the grocery store on an empty stomach can be dangerous. You are likely to buy more items on impulse because all you can think about is your growling belly. You may opt for something sugary becuse you think about eating it as soon as you get into your car. Making decisions on what food you need should be based on your family and weekly needs.
I like to do my grocery shopping in the evening after dinner or on the weekends after my breakfast. Of course, I don't take my kids either.
