If you’re not measuring story points and velocity in R&D work, what do you measure? What proves you’re making progress?
Real R&D organizations produce artifacts that show learning and decision-making:
Pharmaceutical companies produce investigation reports after preclinical research. These reports document: What did we test? What did we learn? Should we proceed to human trials? The report is valuable even if the answer is “no, stop here.”
Skunk Works produced weekly learning updates during the SR-71 program. Not “we’re 47% done.” But “here’s what we learned about titanium at sustained high temperatures this week, here’s what it means for the design.”
Bell Labs measured researchers by publications and patents—proof of discovery and learning. The products that came later were important, but the research artifacts proved value first.
For software, the artifacts should be:
Investigation reports that document: What did we learn? What approaches did we consider? What do we recommend and why?
Architecture decisions that trace back to investigation: We chose this approach because investigation showed X, Y, and Z.
Risk registers that show you identified problems before they bit you.
Stop decisions that document when investigation showed something wasn’t worth building—these prove investigation is working.
Post-launch reviews that show whether investigation was accurate and what you learned.
These artifacts prove you’re doing valuable work even when you’re not shipping features. Investigation that prevents building the wrong thing can be more valuable than quickly building the wrong thing.
The code you eventually ship is just one artifact among many. The investigation artifacts prove you made good decisions. The code proves you executed those decisions.
If software development is R&D work, then the primary output should be knowledge—not just functioning code. But here’s the problem: code is a terrible way to preserve knowledge.
Yes, code is the source of truth for what the system does. You can read the code and see exactly how authentication works, how data flows, what edge cases are handled.
But code doesn’t tell you why—and in R&D work, the “why” is often more valuable than the “what.”
Why did we choose this approach over the alternatives? Why is there a weird workaround in this function? Why does this validation happen here instead of there? What did we learn that led to this design?
The code is silent on all of this. The code just is. It shows the final decision, not the decision-making process.
Six months after shipping a feature, a new developer looks at the code and thinks: “This is overly complex. I could simplify this.”
They don’t know:
All that knowledge—the investigation, the alternatives considered, the problems discovered—is gone. The code doesn’t capture it. So the new developer “simplifies” it, and six months later, the production bug resurfaces.
This happens constantly. Teams reinvestigate the same problems. They try approaches that were already ruled out. They remove “unnecessary” code that was actually handling critical edge cases. They refactor away workarounds without understanding what problem the workaround solved.
Why? Because code doesn’t preserve the journey. It only preserves the destination.
Here’s a different way to think about it: the most valuable artifact from any software project isn’t the code. It’s the record of what you discovered.
What did you learn about the problem? What approaches did you try? What worked? What didn’t? Why? What surprised you? What constraints did you discover? What tradeoffs did you make, and why?
That knowledge is what enables good decisions. The code is just the physical manifestation of those decisions.
Consider two scenarios:
Scenario A: You have the code, but no record of how or why it was written that way. The original team is gone. You need to modify it. You don’t know what’s safe to change, what assumptions it makes, why it’s structured this way. Every change is risky. You’re flying blind.
Scenario B: You have the code and the investigation journal. You can read why each decision was made, what alternatives were considered, what constraints exist. You understand the problem the code solves. You can make changes confidently because you understand the context.
Which scenario enables better work? Scenario B, obviously.
The code plus the investigation journal is dramatically more valuable than code alone. In many cases, the investigation journal is more valuable than the code—you can regenerate code if you understand the problem, but you can’t regenerate lost understanding from code alone.
This isn’t about code comments. Comments describe what the code does. Investigation documentation describes why the problem needed solving this way.
Comments: “This function validates user input by checking email format and password strength.”
Investigation documentation: “We investigated three validation approaches. Client-side validation seemed simple but creates security issues and doesn’t handle edge cases (see: Unicode in email addresses). Server-side validation handles everything but has poor UX. We chose a hybrid: client-side for UX, server-side as source of truth. This increased complexity but reduced support tickets by 40% in testing.”
See the difference? Comments describe the code. Investigation documentation preserves the knowledge that led to the code.
Code communicates to the machine. It tells the computer what to do.
Investigation documentation communicates to the team. It tells humans why this is the right thing to do.
Both are necessary. But we treat code as sufficient and investigation documentation as optional. That’s backwards.
Without investigation documentation:
With investigation documentation:
If you accept that software development is R&D work, then the primary output of R&D is knowledge, not products.
The code is important—it’s how you apply the knowledge. But the knowledge is the actual output.
This means the “journal of the journey”—the investigation reports, the architecture decisions, the record of what you learned—is the real work product. The code is a side effect.
This feels wrong to most developers. We’re paid to write code, right?
No. You’re paid to solve problems. The code is how you demonstrate that you solved them. But the understanding you gained—that’s the valuable part. That’s what enables the next problem to be solved better.
If the discovery process is the real artifact, it needs to be treated as seriously as code:
Version controlled. Investigation documents should live in the repo, versioned alongside code. When you update the code, you can see why it was originally written that way.
Reviewed. Investigation findings should be reviewed like code. “Did we miss something? Are there alternatives we didn’t consider? Is this reasoning sound?”
Searchable. Future developers should be able to search for “why did we choose PostgreSQL over MySQL” and find the investigation that led to that decision.
Referenced in code. Code should link to investigation docs. “See docs/investigations/authentication-approach.md for why we use JWT tokens.”
Celebrated. Good investigation documentation should be valued as much as good code. “Great investigation, you found three issues before we built anything” is a win.
If software development is R&D work where knowledge is the primary output, then investigation journals should be as important as lab notes are to scientists. But historically, there was a problem that made this impractical.
Here’s something that makes investigation journals dramatically more valuable than they’ve ever been: AI can read them.
For years, documentation faced a fundamental problem: writing it was easy, finding the right document later was hard. You’d document your investigation thoroughly, but six months later when someone needed that knowledge, they couldn’t find the right document. Or they found it but it was buried in a 50-page spec alongside unrelated information.
This created friction. Teams built elaborate systems to categorize and organize documentation. Despite this, most investigation knowledge stayed in people’s heads because writing documentation that others could actually find felt futile.
AI changes this completely, making it practical to treat investigation documentation like R&D organizations treat research papers and lab notes.
Traditional search requires knowing what words to search for. If you documented your database choice as “PostgreSQL vs MySQL investigation” but someone searches for “why did we choose Postgres,” they might not find it.
AI search understands meaning. You can ask: “Why did we choose our current database?” The AI reads through all your investigation journals and surfaces the relevant decision, even if you never used those exact words.
You can ask questions like:
The AI synthesizes across multiple documents, pulling together insights from different investigations that touched on related topics.
This removes the biggest barrier to documentation: you don’t need to perfectly organize it.
Just write markdown files in a docs/investigations/ folder. Name them descriptively but don’t stress about perfect naming. Don’t build elaborate category systems. Don’t worry about whether this belongs in “architecture decisions” or “technical investigations” or “problem analysis.”
Just write:
AI can find it later regardless of how you organized it.
Here’s where it gets really powerful: AI can synthesize patterns across many investigations.
You can ask: “What patterns have we seen in failed integration attempts?” The AI reads through multiple investigations where integrations didn’t work and identifies common themes—maybe all the failed integrations lacked proper error handling, or all had undocumented API behavior.
You can ask: “What have we learned about database migrations?” The AI synthesizes insights from every investigation that touched on database changes, giving you the accumulated wisdom without having to track down and read every document manually.
You can ask: “What assumptions have we made about user authentication?” The AI finds every place authentication was discussed and shows you what you’ve taken for granted, making implicit assumptions explicit.
This means investigation journals compound in value over time. Each new investigation adds to the searchable knowledge base. The more you document, the more valuable the entire corpus becomes.
You don’t need complex tooling. Simple markdown files work perfectly:
docs/
investigations/
2024-11-database-choice.md
2024-12-authentication-approach.md
2025-01-search-implementation-options.md
Each file is just structured markdown:
# Database Choice Investigation
## Context
We need to choose a database for the new analytics feature.
## Options Considered
- PostgreSQL: Strong JSONB support, good for complex queries
- MongoDB: Schema flexibility, but weaker transaction support
- Cassandra: Scales well, but steep learning curve
## Investigation
We prototyped basic queries in PostgreSQL and MongoDB...
## Decision
Going with PostgreSQL because...
## Tradeoffs
We're accepting slower horizontal scaling in exchange for...
That’s it. No complex metadata. No rigid templates. Just human-readable markdown that AI can search and synthesize.
Modern AI tools can:
This means a new developer can ask: “Why is our auth system designed this way?” and get an answer synthesized from multiple investigations, even though no single document answers that question completely.
Or a team can ask: “What have we learned about performance optimization in the past year?” and get insights from every investigation that touched on performance, automatically synthesized.
The old barrier to documentation was: “I’ll write this, but will anyone ever find it when they need it?”
AI removes that barrier. Write in simple markdown. Don’t stress about organization. Future you—or future teammates—can ask questions in natural language and get answers synthesized from all your investigations.
This makes investigation documentation dramatically more practical than it’s ever been. The return on investment for writing down what you learned just went way up.
The implication: there’s no longer a good excuse not to document investigations. The tools exist to make that documentation findable and useful. The missing piece isn’t technology anymore—it’s the habit of treating investigation documentation as core work, not optional overhead.
This is what makes treating software like R&D actually practical now. R&D organizations have always documented their work thoroughly because knowledge is the output. With AI, software teams can do the same without the historical friction that made it feel impractical.
What happens when you ship code without documenting the journey?
Three months later, someone “improves” the code by removing something that seemed redundant. It breaks for European customers. Nobody knew why that code was there.
Six months later, the team debates the same architectural question you already investigated. They don’t know you already ruled out that approach. They waste a week rediscovering what you already learned.
A year later, the original developer leaves. New team members look at the code and can’t figure out why it’s so complex. They assume the original dev was incompetent. They rewrite it “properly.” It fails in production for the same reasons the original approach was rejected.
Two years later, a major refactor needs to understand system constraints. Nobody knows what those constraints are. The code doesn’t say. So they proceed with incomplete information and create new problems.
All of this is preventable. Document the journey. Make the discovery process the real artifact. Treat investigation documentation as more important than code, because it’s the foundation that makes good code possible.