Absolutely and frankly I’d be perfectly fine with A/B testing if it was opt-in. Pop up a little window or notification that says “Hey, this is a new feature, you want it?”

If

1. people don’t opt-in
2. they opt-in and don’t like it
3. they opt-in and then quickly opt-out

You know the feature isn’t good and to move on. A lot of people would call that data inconclusive because they want to believe the feature is good but not being able to convince people to opt-in is feedback.

Experimenting on users should be illegal.