Over the past 19 months, I’ve written Crafting Engineering Strategy,
a book on creating engineering strategy. I’ve also been working increasingly with
large language models at work.
Unsurprisingly, the intersection of those two ideas is a topic that I’ve been thinking
about a lot. What, I’ve wondered, is the role of the author, particularly the long-form author,
in a world where an increasingly large percentage of writing is intermediated by
large language models?
One framing I’ve heard somewhat frequently is the view that LLMs are first and foremost
a great pillaging of authors’ work. It’s true. They are that.
At some point there was a script to let you check which books had been loaded into
Meta’s LLaMa, and every book I’d written at that point was included, none of them
with my consent.
However, I long ago made my peace with plagiarism online,
and this strikes me as not particularly different, albeit conducted by larger players.
The folks using this writing are going to keep using it beyond the constraints I’d prefer it to be used in,
and I’m disinterested in investing my scarce mental energy chasing through digital or legal mazes.
Instead, I’ve been thinking about how this transition might go right for authors.
My favorite idea that I’ve come up with is the idea of written content as “datapacks”
for thinking. Buy someone’s book / “datapack”, then upload it into your LLM, and you
can immediately operate almost as if you knew the book’s content.
Let’s start with an example. Imagine you want help onboarding as an executive, and
you’ve bought a copy of The Engineering Executive’s Primer,
you could create a project in Anthropic’s Claude, and upload the LLM-optimized
book into your project.
Here is what your Claude project might look like.
Once you have it set up, you can ask it to help you create your onboarding plan.
This guidance makes sense, largely pulled from Your first 90 days as CTO.
As always, you can iterate on your initial prompt–including more details you want to include into the plan–along
with follow ups to improve the formatting and so on.
One interesting thing here, is that I don’t currently have a datapack for The Engineering Executive’s Primer!
To solve that, I built one from all my blog posts marked with the “executive” tag.
I did that using this script that
packages Hugo blog posts, that I generated using this prompt with Claude 3.7 Sonnet.
The output of that script gets passed into repomix via:
repomix --include "`./scripts/tags.py content executive | paste -d, -s -`"
The mess with paste is to turn the multiline output from tags.py into a comma-separated list
that repomix knows how to use.
This is a really neat pattern, and starts to get at where I see the long-term advantage of writers
in the current environment: if you’re a writer and have access to your raw content, you can create
a problem-specific datapack to discuss the problem. You can also give that datapack to
someone else, or use it to answer their questions.
For example, someone asked me a very detailed followup question about a recent blog post.
It was a very long question, and I was on a weekend trip.
I already had a Claude project setup with the contents of Crafting Engineering Strategy, so I just passed
the question verbatim into that project, and sent the answer back to the person who asked it.
(I did have to ask Claude to revise the answer once to focus more on what I thought the most important
part of the answer was.)
This, for what it’s worth, wasn’t a perfect answer, but it’s pretty good.
If the question asker had the right datapack, they could have gotten it themselves,
without needing me to decide to answer it.
However, this post is less worried about the reader than it is about the author.
What is our competitive advantage as authors in a future where people are not
reading our work? Well, maybe they’re still buying our work in the form of
datapacks and such, but it certainly seems likely that book sales, like blog traffic,
will be impacted negatively.
In trade, it’s now possible for machines to understand our thinking that we’ve
recorded down into words over time. There’s a running joke in my executive learning circle
that I’ve written a blog post on every topic that comes up, and that’s kind of true.
That means that I am on the cusp of the opportunity to uniquely scale myself
by connecting “intelligence on demand for a few cents” with the written details
of my thinking built over the past two decades of being a writer who operates.
The tools that exist today are not quite there yet, although a combination of selling datapacks
like the one for Crafting Engineering Strategy and tools like Claude’s projects are a good start.
There are many ways the exact details might come together, but I’m optimistic that writing
will become more powerful rather than less in this new world, even if the particular
formats change.
(For what it’s worth, I don’t think human readers are going away either.)
If you’re interested in the fully fleshed out version of this idea, starting here
you can read the full AI Companion to Crafting Engineering Strategy.
The datapack will be available via O’Reilly in the next few months.
If you’re an existing O’Reilly author who’s skepical of this idea, don’t worry: I worked with them
to sign a custom contract, this usage–as best I understood it, although I am not a lawyer and am not providing legal advice–is outside
of the scope of the default contract I signed with my prior book, and presumably most others’ contracts as well.
