You're in the perfect spot to defend what you've decided is important to you. This is good.
Wanting a better lunch doesn't mean being a social outsider. If someone is making a fast food run, go along for the ride. They say it's easier to avoid temptation than to resist, but it IS possible ride in a car with french fries and not eat them.
You can do this. Who will feel better and have more energy at 3 o'clock? You or French Fry Man?