# 1. General Case-Interview Frameworks ## 1.1 How to analyze a business case A structured approach to any open-ended business case works through five stages. - **Clarify** the problem: - understand the user journey, the goal, and the stakeholders involved; - what user or proxy data is actually available (experiments, proxy metrics, surveys); - what database platform and scalability constraints exist; - whether the output needs to be aggregated or at the individual-user level; - whether this is a one-time analysis or something that needs to become a recurring framework. - **Constrain** the problem based on what was clarified — identify the real limitations you're working within. - **Plan** what historical data to use and what to check first. - Walk through the **method** in detail, explicitly showing the tradeoff reasoning behind each choice. - **Conclude and recommend** how the results should actually be used. ## 1.2 How to describe a project The classic structure is **Situation, Task, Action, Result**. A useful variant leads with the result, then backs into the story: state the **result** up front, give one sentence of **context**, describe the **challenges** faced, walk through the **actions** taken (highlighting **tradeoffs and critical thinking** along the way), and close with any **interesting findings** that came out of the work. ## 1.3 How case studies get evaluated Case study responses are typically judged on five dimensions: **structure** (is the approach organized and easy to follow), **completeness** (did it actually solve the stated problem), **soundness** (was the underlying method sensible), **clarity** (is the reasoning communicated well), and **speed** (was it delivered efficiently). # 2. Measuring and Improving Product Performance ## 2.1 How to measure product performance - Defining the business goal, identifying the main stakeholder, and mapping out the user journey. - Ask clarifying questions about exactly what's being measured, where, and for whom. - Product performance is generally assessed across five lenses: - **growth** (is the user base growing), - **engagement** (usage and retention), - **monetization** (conversion and spending), - **health** (customer satisfaction and system stability — customer surveys can even double as a labeled data source for a downstream model), and performance **by segment** (geography, device, demographic). - It's also worth explicitly considering **incremental lift and cannibalization** — whether a metric's improvement is genuinely additive or just shifting behavior from elsewhere. - Finally, when building a dashboard to communicate any of this, clarify exactly who the dashboard's users will be (a VP, a product manager, marketing), since that shapes what belongs on it. ## 2.2 How to define success for a new feature - Start again from the **business goal**, the main **stakeholder**, and the **user journey.** - Define the **primary KPI metric**; define **guardrail metrics** that shouldn't get worse; define **counter-metrics**, such as an opt-out rate, that would signal the feature is actively unwanted; - Name each metric precisely and refine it with a specific **statistical function** (total, average, median), **behavioral definition**, and **unit** (per month, per user), so there's no ambiguity about what's actually being tracked. ## 2.3 How to improve a product - Define the **business goal** and identify the main stakeholder. Walk through the user journey and user experience in detail, thinking carefully about what each metric along that journey actually represents. - Analyze **user behavior and current data** correlations to find where the bottleneck actually is. - **Brainstorm** candidate metrics and features, but focus down on one feature at a time — being explicit about **what** it is, **how** it would work, and **whom** it's for — validating the idea with experiments, surveys, or proxy data wherever possible. Where appropriate, run **statistical models** (regression, classification) to validate a feature's likely impact before building it. - **Prioritize** which feature to pursue (see §2.4). - Run an **experiment** to validate the actual result once it ships. ## 2.4 How to prioritize feature build A simple effort/impact matrix, combined with an explicit pros-and-cons list, is the standard tool for prioritization: | effort / impact | low impact | high impact | | ----------------- | --------------------- | ---------------- | | low effort | incremental lift | **best win** | | high effort | **bad bet** | big bets | ## 2.5 How to prioritize analysis work Analysis work itself should be prioritized on four factors: - its **business impact**, particularly against the North Star metric and other core KPIs; - whether it's a **customer-facing** project that directly affects user experience; - its **urgency**, since strategic planning work generally has tighter time constraints than retrospective analysis; - whether it's worth building into a **self-served dashboard** or otherwise sustainable solution, rather than a one-off analysis that has to be redone from scratch next time. ## 2.6 How to diagnose a problem - Clarify the problem and define exactly which metrics are involved. - Diagnose systematically across the likely causes: a **data pipeline** issue, a shift within a particular **segment**, a breakdown at some stage of the **funnel**, a **seasonal** or **external factor** (a product launch, an event, a pricing change), or a **bug**. - Identify the actual **root cause** by explicitly linking the diagnosis back to a business reason, picking the most likely one, going deep on it, and sharing a recommendation from there. - Make the **recommendation**. - Put **tracking** in place so the same issue can be caught faster next time. ## 2.7 How to measure customer "happiness" Customer happiness is best captured through a blend of measures: - **direct measures** like NPS and CSAT; - **behavioral measures** like retention, engagement, and expansion (team growth, seat expansion); - **advocacy measures** like referrals and external sharing. - Combining these against the underlying business goal is what lets you construct a genuine happiness metric, rather than relying on any single signal in isolation. # 3. Marketing Measurement and Attribution ## 3.1 Brand measurement It's worth being upfront that brand measurement is one of the hardest attribution problems in marketing, since the lag between awareness and an eventual booking or purchase can stretch to months. A useful framework layers three time horizons: - **short-term** — brand lift studies via survey panels; - **medium-term** — Marketing Mix Modeling (MMM) with brand spend included as a variable, using lagged adstock to capture delayed effects; - **long-term** — organic search volume and direct traffic as proxy signals for accumulated brand equity. - **Geo holdout tests** are also valuable here for establishing genuine causality rather than just correlation. The ultimate goal is connecting these awareness metrics all the way through to downstream bookings and customer lifetime value. ## 3.2 Marketing technology stack A typical marketing stack spans several functional categories: - CRM (e.g. Salesforce); - marketing automation and email (e.g. Eloqua); - SEO and analytics (e.g. Google Analytics); - internal engines for web (e.g. Amplitude) and content/creative production (e.g. YouTube); - brand, event; - community and social listening (e.g. Meltwater); - paid social media (e.g. Facebook, Instagram); - paid display (banner ads with UTM tracking); - paid search (search ranking placements); - remarketing and programmatic buying platforms. ## 3.3 Marketing attribution models Three broad families of attribution model are used to assign credit for a conversion across multiple touchpoints: **Rule-based models** assign credit using a fixed heuristic: first-click, last-click, equal credit across all touchpoints, or time-decay (where touchpoints closer to conversion get more credit). **Logistic regression** models attribution at the user level, for example `conversion ~ channel_1(0/1) + channel_2(0/1) + channel_3(0/1)`, converting the resulting coefficients into odds ratios to use as channel weights. This approach runs into multicollinearity when channels are correlated, and it ignores the order in which touchpoints occurred. **Markov chain models** analyze the sequential interaction of channels across a customer's journey, treating each channel as a state and modeling the transitions between states probabilistically. The process has four steps: first, prepare the data as an ordered sequence of interactions, e.g. `['YouTube', 'Google Ad', 'Facebook Ad', 'Email', 'Page A', 'Page B', 'Conversion']`; second, build a **transition matrix** by counting how often the journey moves from one channel to another and dividing by the total transitions out of that channel, producing a matrix where cell `[i, j]` is the probability of moving from channel `i` to channel `j`; third, compute each channel's **removal effect** — simulating what would happen to conversion probability if that channel were removed entirely from every path, calculated as `(Original Conversion Probability - Conversion Probability without Channel) / Original Conversion Probability`; and fourth, compute the final **attribution** for each channel as its removal effect divided by the sum of removal effects across all channels. ## 3.4 Marketing demand generation Demand generation is the process of building awareness and interest *before* a customer is ready to buy, following the funnel `brand awareness → interest → consideration → lead → opportunity → customer`. It's primarily focused on the **top and middle** of the funnel, using tactics like launch campaigns, webinars, content marketing, brand-building, YouTube videos, influencer partnerships, paid social, and paid ads. Unlike paid search — which has a tight, easily attributable loop (`ad → click → try`) — demand gen channels are inherently hard to attribute directly to revenue. Measuring it well requires several complementary approaches: - tracking **leading indicators** ahead of revenue, like branded search volume, website traffic, downloads, demo requests, and email engagement; - running **funnel analysis** across lead volume, MQL rate, pipeline creation, and revenue influence; - using **MMM** to estimate incremental revenue, ROI, and optimal budget allocation; - running **brand lift studies** to measure awareness, consideration, preference, and purchase intent via survey or MMM; - and running **incremental experiments** — geo holdouts, channel holdouts, audience holdouts, difference-in-differences, and uplift modeling. ## 3.5 How to measure channel effectiveness - Defining what "effectiveness" actually means for the business objective at hand — depending on the goal, that could mean customer acquisition, subscription conversion, revenue, retention, LTV, or ROI. The key question isn't simply which channel gets credited for a conversion, but which channel is driving genuinely *incremental* business outcomes. A complete measurement approach combines several pieces: **attribution models**, which map user-level conversion paths and assign credit across touchpoints — with careful attention to the attribution window, since different channels influence users over very different time horizons (paid search often drives immediate conversions, while display or video can influence a conversion weeks later); - **MMM**, which estimates incremental lift at the channel level while controlling for external factors and capturing long-term effects, accounting for adstock (delayed impact from prior exposure) and saturation (diminishing returns as spend increases); - **experiments**, which go beyond final conversion metrics to evaluate leading indicators predictive of long-term outcomes — app installs, trial starts, onboarding completion, content engagement, activation rate, or early retention, all of which give faster feedback while staying connected to long-term value; - and **guardrail analysis** of the customer journey itself — touchpoints, channel sequence, exposure frequency, and creative performance — including watching for frequency effects and creative fatigue, where repeated exposure to the same ad eventually stops adding value and starts reducing campaign efficiency. ## 3.6 How to validate that channel CPI numbers are trustworthy The right approach is to triangulate across multiple independent sources: compare platform-reported CPI, MMP (mobile measurement partner) attribution, and internal user-acquisition data against one another. Then look for **stability** over time, unexpected shifts by channel, and consistency with MMM outputs. Finally, validate with **incrementality experiments** directly — if a channel reports a low CPI but shows little incremental lift when tested, the CPI number shouldn't be trusted as a budgeting signal. This validation rests on three checks: an **internal consistency check** (does the number hold up across segments), an **external cross-reference** (does it agree with other independent models), and a **sensitivity and stability analysis** (how much does the output change if key assumptions — like the attribution window or adstock decay rate — are varied). ## 3.7 Handling conflicting attribution signals across channels When different attribution methods disagree, focus on the actual decision being made rather than getting stuck defending one attribution method over another. Compare the *directionality* of the signal across MMM, multi-touch attribution (MTA), platform reporting, and experiments — if every source points the same direction, the decision is easy. When they genuinely conflict, prioritize causal evidence (experiments and MMM) over correlational attribution, and weigh the risk of being wrong. In practice, this means reallocating budget gradually through test-and-learn cycles, rather than making a large shift based on a single attribution source. ## 3.8 Measuring external marketing demand The goal here is separating organic, external demand from marketing's actual incremental impact — estimating what would have happened without marketing at all, and measuring the lift marketing genuinely created on top of that. This is fundamentally a question of **incrementality, not attribution** — demand *creation* versus demand *capture*. Useful proxy signals for external demand include Google Trends, branded and non-branded search volume, social media mentions, event popularity, competitor activity, economic indicators, seasonality, and PR or news cycles. On the measurement side, this combines MMM with external regressors, adstock and saturation modeling, Bayesian structural time series, difference-in-differences, geo holdouts, uplift modeling, and brand studies. In practice, this might look like building a competitive-demand dashboard that combines share-of-search, social conversation volume, trial activity, product usage, creator surveys, and retention by creator segment. The key strategic question usually isn't "is a competitor growing" in the abstract — it's whether *our* share within key cohorts (for instance, new users or students, who often act as leading indicators of broader shifts) is declining. If adoption is shifting away, the next step is identifying exactly where the value proposition is weakening and recommending targeted acquisition or product investments in response. ## 3.9 Data science support for programmatic advertising A data science function supporting programmatic ads typically produces several standing reports: a **campaign performance report** (impressions, reach, frequency by channel/creative/audience segment, CTR, CVR, CPM); an **attribution report** (MTA credit distribution, online vs. offline conversion split, time-to-conversion distribution); an **incremental report** (incremental ROAS by channel, incremental conversions, holdout lift with a confidence interval); an **audience quality report** (match rate — what percentage of the target segment was actually reachable on each platform — audience overlap, frequency distribution to check for wasted spend on over-exposed users, and how lookalike audiences perform relative to core ones); a **creative performance report** (CVR and incremental ROAS by creative, backed by A/B testing); a **saturation and diminishing-returns report** (the spend-vs-return curve, the saturation point, and marginal ROAS); a **forecasting report** (scenario analysis and optimal budget allocation under constraints); and a **pre-campaign report** (audience sizing, reach forecasting, and budget recommendations before a campaign launches). # 4. Communicating Results ## 4.1 How to write a stakeholder recommendation A strong stakeholder-facing writeup follows a consistent structure: a 2–3 sentence executive summary, the key findings, the recommendation itself, supporting evidence (with an honest statement of confidence), the analysis's limitations, and clear next steps.