Some new features might not be suitable for all users
When launching most new features, or changes to existing ones, you will deploy the change giving all users access.
When you make a significant change, it is prudent to test it out with a subset of users.
Features that are used frequently, impact sales or are complex are significant
Changes to frequently used features are sensitive. Because they’re so visible, it’s easy to attract complaints and bad experiences.
Changes that impact sales are also prime targets for customer anger.
Features that have some complexity have two aspects. First, you must make sure that they’re designed well and that you have the proper training materials for them. Secondly, you must make sure that the process behind them works well.
So you should release them to subsets of users
To choose a test cohort, you should consider the following:
-
How impactful is the change to the user
You don’t want to test on someone who wouldn’t notice the change
-
How good is the relationship
You need someone who would not be annoyed by work in progress and would provide good feedback.
-
How representative are they
They should be similar to the rest of the users the change would impact
Then listen to the feedback from chosen cohort and progressively improve the changes before releasing them to all.
Add a feature flag to the account, and the system
The account flag enables the feature just for the particular account, and the system flag enables it for everyone.
These can be booleans, or they can be dates when the feature will enable.
When you first deploy the flags, no accounts will have access. First, you enable it for a small set of trusted users within the selected cohort. You make sure that there are no obvious errors or complaints. Then, you expand access to the rest of the cohort.
When you finish development, you can toggle the system-wide setting giving everyone access.
That will help you provide a more polished product
You can gather feedback in stages instead of having everyone complain at once.
The feedback is also likely better quality since you have a relationship with the chosen cohort.
It also minimizes impact if the feature is not a step in the right direction.