For me, my 6.7" phone still perfectly fits in my hand. It's not hard to reach anything for me, even though I don't have particularly large hands or anything. It probably just comes down to how I hold it with one hand, as I do not encase its back with my finger completly, so stabilize it's left side with my fingertips, but place my tips on it's back and place it on my small finger. It's still pretty stable and I can reach everything very quickly as that position allows much more movement of the thumb in every direction.
It fits in all pockets, even in the backpocket; I could only imagine the typically small pocket sized womens pants to not have enough space for it - though that would apply to every phone then. It doesn't feel like a brick, neither really in size nor weight.
That's good, I think having a powerful phone if you can use it comfortably is great.
In my case, I started having wrist pain from just using the phone.
The ideal situation for me is that there are several options coexisting for everyone to choose, so I could get a small phone with some compromises and you could get a fully capable bigger phone also.