Reduce strategic ambiguity
A framework for ranking ideas
A surplus of AI ideas had no systematic way to be prioritized. The study would provide one.
Case study · Verily Workbench
Turning a stalled AI roadmap into a funded product strategy
Workbench product managers had 30 AI feature ideas and a strategy offsite eight weeks away. A two-phase study with 300 biomedical researchers showed leadership which capabilities to build first.
- Role
- Staff UX researcher, study lead
- Team
- 1 staff UX designer, 4 PMs
- Executive sponsor
- Director of AI and Data Science
- Methods
- MaxDiff, concept tests, competitive analysis
Situation
Customers were asking. Competitors were moving.
Context
- Customers were serious about AI and wanted a clear AI product strategy for Workbench, a trusted research environment for multimodal biomedical data.
- Competitors were shipping eye-catching AI demos before Workbench could launch its own.
- The chief product officer asked for a coherent strategy that brings new value to customers, not AI bolted onto the existing product.
Constraints
- A product-manager-only offsite on Workbench AI strategy was eight weeks out.
- Product managers had spent months vibe-coding more than 30 feature ideas, with no shared way to rank them.
- The end-of-year crunch left little time for anything collaborative.
- Feature ideas on the table
- 30+
- Weeks to the offsite
- 8
- Research requests from product
- 0
Task
Give leadership a fair way to choose. Which ideas matter most to researchers, and why.
The goal: find which feature ideas resonate most with the people Workbench serves, why, and how to refine them so their value comes through, in time to shape the offsite.
Target the highest-return workflows
Where the work burden is
Measure the burden and error sensitivity of core research tasks to find the most valuable openings.
Stand out in the market
Table stakes or difference
Separate what customers now expect from AI in research tools from what could set Workbench apart.
Action · before fieldwork
Three steps made an eight-week study possible.
Align UX leadership
How do we convince UX leadership to shift resources to this?
As a UX-led initiative, what can we realistically achieve in the time?
Win an executive sponsor
The Director of AI and Data Science sponsored the study, which gave it standing with product leadership before any data came in.
Cull 30 ideas to 10 concepts
- Mapped each idea to past user and market research
- Checked it against verifiable product domain expertise
- Kept a few aspirational, riskier bets on purpose
Action · method
Two phases. Rank at scale, then learn why.
Phase 1 · MaxDiff survey
Which capability matters most?
Concepts appeared in random sets of four, over ten rounds. In each round people picked the most and least important.
Phase 2 · Concept testing
Why does it matter?
People reviewed the 10 concepts in random order, then thought aloud while sorting them into high, medium and low value.
Participants were principal investigators, data scientists, computational biologists and similar roles. A competitive analysis ran alongside both phases.
Findings
Take the drudgery away first. Tedious work ranked above help with thinking.
The concepts that resonated most
- AI-enabled data curationClean and transform raw data
- Data explorer agentVerify, then explore
- Automated experiment trackerCapture work as it happens
AI use was already widespread: at each of the 12 stages of the research lifecycle we asked about, between 84% and 100% of the 300 respondents used AI or ML tools.
How often do you use AI tools for visual exploration and pattern discovery?
Findings · data explorer agent
Researchers already use general chatbots for analysis. The chatbot limits how far they get.
| Friction | With a general LLM today | With a specialized agent |
|---|---|---|
| Data out of reach | Governed or large data can’t be uploaded, so people analyze samples. | Runs where the data lives, under existing access controls. |
| Copy-paste loop | Code, errors and plots move back and forth between chat and notebook. | Runs code, reads its own output and corrects itself in one loop. |
| Scope capped by the prompt | The analysis goes only as far as the user thinks to ask. | Surfaces outliers, skew and patterns worth a look on its own. |
| Hard to verify | Silent assumptions and no audit trail make results hard to defend. | Shows every step and applies domain conventions by default. |
Empowerment for some, convenience for others
Less technical colleagues saw a way to explore data visually, gain confidence and stop waiting on technical teams. Technical users saw a time saver, useful but not critical. The opportunity: design for accessibility first while still saving experts time.
“A lot of our colleagues who need reports are not that tech-savvy, which creates a big issue for them trying to pull data. Seeing the Data Explorer Agent as a potential solution gets me very excited!”
Paired with data curation, it checks the AI’s work
Curation cleans and transforms raw data; the explorer verifies the result before exploring it. Before-and-after plots catch dropped rows, skew and broken joins, so people review visible evidence instead of auditing transform code. Together they cover the early journey from raw data to first insight.
“It would be extremely useful in finding out various errors in the data, such as duplicates, inconsistencies in the data, or missing data.”
Recommendations
Three concepts, in priority order. Then scored with engineering and product in one session.
| Concept | Why it leads | How to refine it |
|---|---|---|
| 1. AI-enabled data curation | A universal pain point: the drudgery of preparing data. Static “analysis-ready” datasets are too rigid; researchers want curation that respects their own aims. | Keep a human in the loop, with side-by-side views to accept, reject or override each change. |
| 2. Automated experiment tracker | Addresses the reproducibility crisis, and was rated the most innovative concept. E-notebooks fail because they digitize manual work instead of automating it. | Capture in the background with no extra effort, and cover experimental work beyond AI models. |
| 3. Data explorer agent | The check on AI curation that eases “black box” worries, and opens data to clinicians and scientists who don’t program. | Move from one-off queries to a conversation that suggests the next drill-down. |
How the room scored them
| Concept | Desirability (UX research) | Feasibility (engineering and data science) | Strategic fit (product and marketing) |
|---|---|---|---|
| AI-enabled data curation | 5 | 3A lot of work, with several places to start | Complex: high value, with tension between steward curation and researchers cleaning their own data |
| Data explorer agent | 4 | 5Shovel-ready work gives a good start | Strong: useful to every user, not only builders, and a trust and verification layer |
| Automated experiment tracker | 5 | 2Unclear in practice; needs capabilities that don’t exist yet | Uncertain: high user value, but possibly an infrastructure problem more than an AI one |
Result
From a point of view to a funded workstream.
Executive and strategic influence
- The research reached the chief product officer, who commissioned a two-week product and UX design sprint to prototype a unified concept.
- The commercial team ran its own buyer-facing, feature-based TAM analysis. Its results largely backed the recommended AI investments.
Roadmap and product decisions
- The data explorer agent went on the roadmap as an extension of existing data discovery.
- The data curation team is scoping how to bring curation features to end users.
- New investment was approved for metadata curation and MLflow integration.