How many people use a specific browser doesn’t directly affect the usefulness of that browser, as users of different browsers can interact with each other to the same degree as users of the same browser. For now at least, as Google’s Web Integrity API could obviously change that if websites start to require and some browser are unable or unwilling to provide it.
Thats a great point and something to consider.