Chris Hayes' Journal

Practical Guide to Adopting Investigation-Driven Development

Reading about software development as R&D work, you might be thinking: “Sure, this sounds great for a greenfield project with a supportive organization. But I’m in a more constrained environment with established processes and tight deadlines.”

That’s a fair concern. Most teams can’t just declare “We’re doing R&D now” and reorganize everything. So what’s actually practical?

Start Small, Start Real

Don’t try to change everything. Pick one thing that’s coming up and try a different approach.

Next time someone asks, “How long will this take?” respond with: “I don’t know yet. Give me two days to investigate, then I’ll tell you what’s actually involved.”

Then actually investigate. Spend two days figuring out what you’re dealing with. Document what you find. Come back with: “Here’s what I learned. Here are three approaches. Approach A takes two weeks but has these risks. Approach B takes four weeks but is more solid. Approach C is a different solution that might be better—takes three weeks.”

You’re not asking for permission to change the process. You’re just doing your job more honestly.

When Investigation Shows It’s Bigger Than Expected

This will happen. You investigate and discover the “simple” feature touches six systems, requires database migration, and conflicts with something else in progress.

Traditional response: Panic, apologize for your estimate being wrong, work overtime to hit the original guess.

Different response: “I investigated this. It’s more complex than we thought. Here’s what’s involved. Given this, do we still want to do it, or should we consider alternatives?”

You’re giving stakeholders information to make better decisions. That’s valuable, even if the information is “this costs more than we thought.”

Document What You Learn

Every investigation, write down:

After a few months, you’ll have data. “When we skip investigation, we’re wrong 80% of the time. When we investigate, we’re right 75% of the time.”

Data is harder to argue with than feelings.

The “Time We Already Waste” Argument

Track how much time your team spends in:

For most teams, this is 8-12 hours per two-week sprint. Multiply by team size.

A five-person team spends 40-60 hours per sprint on estimation theater. That’s 1-1.5 weeks of developer time.

What if you spent that time investigating instead of guessing?

You’re not asking for more time. You’re asking to use existing time differently.

Build Alliances

Find one other person who gets it. Maybe another developer, maybe a tech lead, maybe a product manager who’s frustrated with the current process.

Try the investigation approach on something together. When it works, tell that story.

“Remember the payment integration? We spent a week investigating first. Discovered the API had undocumented rate limits. Changed our approach before writing code. Saved us from a production incident.”

Stories beat theory. When people see it work, they’re more open to trying it.

The Stealth Approach

Sometimes you can’t change the process officially. So you do investigation anyway, just don’t call it that.

When you get assigned a task, spend the first day or two “getting familiar with the code.” That’s investigation. You’re learning what you’re dealing with.

When you find something unexpected, you “discovered a complication” rather than “your estimate was wrong.”

When you recommend not doing something, you “identified technical risks” rather than “we should stop this project.”

Same approach, different language. Sometimes that’s what it takes.

When Stakeholders Prescribe Solutions

Here’s a common scenario: Security team sees a vulnerability report. They tell you: “Implement two-factor authentication using SMS codes. We need it done by end of quarter.”

They’ve handed you the solution, not the problem. And if you treat software development like R&D work, this creates a challenge—because researchers don’t get told how to solve problems. They get told what problems need solving, then they investigate.

But you can’t just say “No, let me investigate first” when the business is demanding a specific solution. Especially when there’s a real security concern driving it.

Start by acknowledging the underlying concern. “I understand we have a security issue that needs addressing. Can you help me understand what happened or what risk we’re trying to mitigate?”

Often, stakeholders prescribe solutions because they’re worried the problem won’t get taken seriously. If you demonstrate you take the problem seriously, they’re more willing to let you explore the solution.

Ask questions that reveal constraints and context:

“What’s driving the end-of-quarter timeline?”

“Have you seen SMS 2FA work well somewhere else?”

“What happens if users can’t receive SMS codes?”

“Are there regulatory requirements I should know about?”

You’re not arguing. You’re investigating. And often these questions reveal information that changes the solution entirely.

Present the investigation as risk mitigation. “Before we implement SMS 2FA, I’d like to spend a few days investigating the options. SMS has some known vulnerabilities—SIM swapping, SMS interception. I want to make sure we’re choosing an approach that actually solves the security problem long-term.”

Now you’re not blocking their solution—you’re being more careful about solving their problem correctly.

Offer a short investigation with a decision deadline. “Give me three days to investigate authentication options. I’ll come back with a recommendation and the tradeoffs. If you still want SMS after that, we’ll do SMS.”

This isn’t endless analysis. It’s bounded investigation with a clear decision point. Most stakeholders will accept that, especially if the alternative is implementing something that might not work.

Document what you find and present options. After investigation:

“Here’s what I found:

Now it’s an informed decision, not just following orders. Sometimes they’ll still choose the original solution—but at least everyone knows the tradeoffs.

Sometimes the answer is still “implement what they asked for.” Not every situation requires investigation. Not every prescribed solution is wrong. Sometimes stakeholders actually do know what they need.

The goal isn’t to investigate everything into the ground. It’s to create space for investigation when it matters, and to redirect solution-focused mandates into problem-focused conversations when possible.

If software development is R&D work, stakeholders should bring problems, not prescriptions. But in reality, many stakeholders don’t work that way—yet. Your job is to gently redirect when you can, and do good work either way.

When You Have No Authority

If you’re an individual contributor with zero organizational power, you still control your own work.

You can’t change sprint planning. But you can investigate before you start building. You can document what you learn. You can present options instead of just implementing.

You can do better work, even in a dysfunctional system. Eventually, people notice. “Why do their projects go smoother?” “Why do they find problems early?”

When Change Isn’t Possible

Sometimes organizational dysfunction runs deep. The culture genuinely doesn’t value learning or investigation. Velocity metrics and overtime are expected, not exceptions.

At some point, you might need to decide: Is this environment changeable, or should I look elsewhere?

Some organizations do treat software more like R&D, even if they don’t use that label. Teams that investigate before committing. Organizations that value learning and thoughtful decision-making.

If you’ve tried to improve things and met consistent resistance, it might not be a reflection on you. It might be a mismatch between your values and the organization’s culture.

What Actually Works

The teams that successfully shift aren’t the ones that reorganize everything at once. They’re the ones where:

You don’t need permission to investigate. You need courage to say “I don’t know yet” and mean it. You need discipline to spend time learning instead of rushing to code. You need honesty to report what you find, even when it’s inconvenient.

Start with one project. See what happens. Let the results speak.

The Long Game

Changing how a team works takes time. You won’t flip a switch and suddenly everything is R&D-driven.

But you can plant seeds:

Do this consistently, and people start to notice that your projects go differently. They ship more reliably. They have fewer surprises. They solve actual problems instead of just completing tasks.

That’s how change happens. Not through mandates. Through people seeing a better way and choosing it.

The Reality of Change

Organizational change is hard. Many teams have significant investment in current processes—years of agile transformations, established metrics, systems built around velocity tracking. That investment is real, and the resistance to change makes sense.

Not every team needs to adopt this approach. Different contexts call for different methods.

But if you’re reading this and thinking “Yes, this resonates with what I’ve been experiencing”—there might be room to experiment. You might already sense that investigation is valuable. You might already see the limitations of purely estimation-based planning.

The invitation here is to try small experiments and see what works in your context.

Helping Developers Make the Shift

One of the hardest challenges: What do you do with developers who’ve spent years in code-cramming mode?

This isn’t about individual failing. If you’ve worked in environments where shipping fast was the only metric that mattered, where investigation was seen as “analysis paralysis,” where “just get it done” was daily feedback—you learned to work that way. It was the correct adaptation to that environment.

The challenge is that those habits don’t serve investigation-driven work. And switching isn’t as simple as deciding to “just do it differently.”

If software development is R&D work, how do actual R&D organizations help people develop investigation skills?

PhD programs don’t start with “go do research.” They start with coursework, then small problems with guidance, then progressively more independence. The first year or two is learning how to investigate.

Pharmaceutical researchers shadow senior scientists before leading their own investigations. They learn what questions to ask, what to look for, how to document findings.

Bell Labs paired junior researchers with experienced ones. Not just for mentorship—to transmit investigation practice. “Here’s how we approach unknown problems. Here’s how we know when to pursue something versus when to stop.”

For software developers transitioning from code-cramming to investigation:

Start with low-stakes practice. Pick a small feature or bug fix. Before writing code, spend an hour investigating. What’s actually happening? What are different ways to solve this? Document what you find, even if it feels obvious.

The goal isn’t perfect investigation. It’s building the muscle of “investigate first.”

Pair investigation work. Have someone experienced in investigation work with someone who’s not. Talk through the process: “I’m looking at X because…” “I’m checking Y to see…” “This makes me wonder about Z…”

Investigation isn’t mystical. It’s a skill you can learn by watching and doing.

Treat all initial code as drafts, not final solutions. Here’s a practice that forces deeper investigation: When someone solves a problem one way, have them solve it again completely differently.

Not refactoring. Not tweaking. A genuinely different approach.

If they built it with a library, rebuild without it. If they used a database query, try a different data structure. If they wrote an algorithm one way, write it another way.

This feels inefficient. “Why throw away working code?”

Because it forces you to challenge assumptions. The first solution often feels like “the obvious way”—but that’s frequently because you haven’t investigated deeply enough to see alternatives.

When designers create multiple sketches before committing to a design, they’re not wasting time. They’re exploring the solution space. When researchers run experiments multiple ways, they’re validating findings and discovering edge cases.

Forcing multiple implementations:

You don’t need to do this for every single feature. But for important problems, complex integrations, or areas where you’re uncertain—multiple implementations can be more efficient than committing to the first approach and discovering its limitations in production.

The code you keep might be from the first attempt, the second, or neither. The value is in the investigation.

Make it safe to say “I don’t know yet.” If the team punishes uncertainty or rewards overconfidence, people will keep pretending they know. Create space where “I investigated this, here’s what I found, I’m not sure yet” is a valuable update.

Recognize investigation work explicitly. When someone spends time learning before coding, when someone documents what they discovered, when someone identifies that something shouldn’t be built—acknowledge that as valuable work. What gets recognized gets repeated.

Expect the transition to be bumpy. Developers used to cranking out code might feel unproductive spending time investigating. “I didn’t write any code today, just researched.” That discomfort is normal—they’re rewiring years of conditioning.

The productivity is in not writing the wrong code. In learning enough to make better decisions. But that feels different, especially at first.

Give it time. You don’t undo years of code-cramming habits in a week. But you can build new habits alongside the old ones. Investigate on one project. Try it with one feature. See what happens.

Some developers will find this shift energizing—”Finally, we’re doing this properly.” Others will struggle more. Both reactions are valid. The key is creating an environment where investigation is valued and practiced, not just talked about.