What Does a Data Scientist Do? Sometimes Decide No Model Is Needed

“Can we predict which customers will leave?” sounds like a data science project. It has a business problem, a dataset, and an obvious place to put a machine-learning model.

It is also several questions too early.

Imagine a subscription business whose cancellation rate has risen. This is a hypothetical investigation, but the choices are ordinary ones. A manager wants a list of customers to contact. Before building that list, someone needs to establish what “leaving” means. Clicking cancel? Failing a payment? Reaching the end of a prepaid year without renewing?

Those events may require entirely different responses. A retention offer will not fix a broken payment form.

The first useful result is a better denominator

Suppose the dashboard divides this month’s cancellations by this month’s paying customers. The company has recently sold many annual subscriptions. Some customers can leave immediately; others have months left before renewal. Treating them as equally exposed to cancellation makes the rate difficult to interpret.

A data scientist might instead compare customers at the same stage of their subscription: first renewal against first renewal, annual plans against annual plans. That is cohort analysis. Its value comes from making the comparison fair enough to answer a question.

Next, the team checks whether the increase appears across cohorts. Perhaps it is concentrated among customers whose cards failed. Perhaps a pricing change affected one group. Perhaps the apparent increase came from a change in how events were logged.

A query, a chart, and a conversation with the billing team could settle the issue. A model might still be useful later. It has not earned its place yet.

Prediction is a particular kind of answer

A churn model estimates who is likely to leave, given the patterns in its training data. It does not automatically tell us why they are leaving or which intervention would persuade them to stay.

Consider two customers. One is very likely to cancel because their project has ended. Another is only moderately likely to cancel but would stay if a recurring support problem were fixed. Ranking by predicted cancellation risk might put the first customer at the top. Ranking by the likely benefit of an intervention could produce a different order.

These are different questions, with different evidence requirements. Testing an intervention might call for a randomized experiment where feasible, rather than another round of prediction tuning.

This distinction is a large part of the job: explaining what an analysis can support before someone uses it to make a stronger claim.

Where the disciplines meet

Statistics helps determine whether a change could be noise and how uncertain an estimate is. Programming makes the calculation reproducible. Knowledge of the business reveals that a “cancellation” event sometimes means a payment retry rather than a customer decision.

None of these contributions can safely substitute for the others. Beautiful code can reproduce the wrong calculation. A sensible business story can survive long after the data has contradicted it. A technically sound estimate can answer a question nobody needs answered.

The work therefore includes unglamorous artifacts: a written definition of churn, a checked query, a list of excluded records, and a note explaining what would change the recommendation. Those artifacts let another person inspect the reasoning without trusting the analyst’s confidence.

In this imagined investigation, the final recommendation might be to repair payment recovery and measure renewal outcomes before funding a prediction system. That is a complete data science result. The absence of a model does not make it unfinished.