A friend of mine from a few cities ago is just starting to homeschool. I asked her what people said when they heard she was starting to homeschool, and she said it's "WHY?" In my experience this is the number one question when someone hears that we homeschool and I was interested to hear than my friend whose homeschooling reasons and circumstances are different from mine still gets asked to explain herself. (I'd rather explain why than listen to the concerned questions some homeschoolers get though.)
As long as you have a good reason you can navigate that exchange fairly well and move on to other things. My go-to answer is that we move a lot; it's quick, simple, and most people won't argue with it unless they want to get into why we move so often, but that's getting a little more personal and most people don't go there. If you try to give a long explanation, or are critical of the local schools, or if you just sound wishy-washy, the sailing isn't quite as smooth.
I never know what to say when people who are just starting to homeschool or are thinking about it ask for advice since everyone homeschools differently and for different reasons, but maybe I'll just start telling people to have a good answer when people ask them why they're doing it.
I probably ought to come up with something that sounds better to explain all the moving though.