feat: embed a stranger in your codebase

July 3, 202625 min

Show Notes

I read the AWS forward deployed engineering announcement on a Tuesday morning. Palantir has been doing this for over a decade. OpenAI does it. Anthropic does it. A billion dollars from AWS means this is now the default way enterprise AI gets adopted at scale.

Here's the question nobody asks in the sales meeting: when a vendor's engineer ships code to your production system and then goes back to their company, who owns that code on day one thousand?

This episode isn't "is forward deployed engineering good or bad." It's about the incentive structure. A forward deployed engineer is optimizing for platform adoption, renewal milestones, and a good case study. Your team needs a system that's still legible in three years. Those are different goals. And if you don't name that mismatch before the engagement starts, your team will be the ones writing the postmortem.

I also cover the four questions I'd ask before signing. Not a framework. Just what I'd actually say in the room.

If you've ever inherited code from someone who wasn't building for your team's future, this one's for you.

Read full transcript

There's a phrase that's been appearing in my LinkedIn feed. Forward deployed engineer. It sounds clean. It sounds like a service tier like something you put on a slide next to an architecture diagram and everyone would nod. It means we will send you one of our engineers. They will sit inside your organization. They will write code in your codebase. They will help you adopt our platform faster. And last week AWS announced they're putting a billion dollars behind it.

I read that and I thought, okay, I have some thoughts. Not is this a bad idea. That's not the episode. The episode is when someone else's engineer ships code in your production system and then goes back to their company, who is left holding that code? Who owns it on day 1,000? That's the question nobody in the sales conversation is asking and I think we should.

I'm Joanne Skiles. This is Chaotic Commits Tech, AI and Engineering Truths, the things nobody puts in the documentation. Today we're talking about forward deployed engineering. The AWS announcement last week is what brought it to my feed. A billion dollars is hard to ignore, but AWS is just the latest. Palantir built their whole business on this model. OpenAI does it. Anthropic does it. This has been building for years and it's now the default enterprise AI onboarding motion. I want to talk about what that means. Not for the company selling the platform but for the team on the other end of the contract.

So three parts. First, what this model is and why the appeal is genuine. Second, the incentive problem that nobody names in the press release. And third, what I actually ask before I let a vendor embed an engineer in my codebase. Let's go.

The AWS announcement made it feel new. It's not new. Palantir has been doing forward deployment for over a decade. You hire Palantir, you get Palantir engineers. They come in, learn your data, wire up the platform, make it work. When they leave, the system is running. Whether your team can maintain it is a different question. AWS didn't invent this. They just put a billion dollars behind it. That's not the same thing. And when a hyperscaler puts a billion dollars behind a labor model, it's not a trend anymore. It's a default.

Forward deployed engineering is supposed to solve a real problem. Your company wants to move fast on AI, but your team doesn't have enough people who understand the platform deeply enough to implement it right. So instead of renting documentation and a Zoom call with a solutions architect, you rent an engineer. They sit with your team. They learn your use case. They build the thing.

Here's what that actually looks like. Your company signs a deal to use Bedrock to build an AI powered customer support assistant. Instead of handing you a getting started guide and a support ticket queue, AWS assigns one of their engineers to work with your team for 3 months. She's in your Slack. She's at your standups. She's reviewing pull requests and shipping production code alongside your engineers. That's a forward deployed engineer.

On paper, that's not crazy. I've watched teams struggle to ramp up on unfamiliar technology with nothing but a tutorial and a lot of optimism. I've been that team. I've been the senior engineer in a room who was supposed to know how to do something and honestly figuring it out alongside everyone else. The pressure to move, the knowledge gap, the gap between knowing the platform exists and knowing how to build it safely, that gap is real. So, I'm not going to pretend the appeal isn't real because it is.

But here's the question that comes right after this person will help us move faster. Who do they actually work for? Not the HR hierarchy who, not who cuts the check. I mean, when this engineer makes a decision in your codebase, what are they optimizing for? What does success look like for them? What happens if the thing they build is elegant and fast and perfectly suited for the platform they came from, but a little hard to maintain by people who didn't build it? Who owns that?

Now, before I turn the knife, let me steelman this. That's the only way the critique means anything. The forward deployed model addresses a scarcity problem. Enterprises are short on AI literate engineers, not junior engineers, not people who can follow a tutorial. I mean people who understand the actual mechanics of a modern AI platform well enough to make principled decisions under pressure. Those people are scarce. They are expensive. They are in high demand at every company at the same time. renting expertise is faster than building it. And that's true. If the alternative is your team building the wrong thing with the wrong abstractions for 6 months, a forward deployed engineer who gets it right in 6 weeks looks like a bargain. And I get it. I would have taken that deal at some of the companies I've worked at. So, I'm saying that plainly before anything else.

And before I turn to the critique, this is not a new problem. This is a very very old problem with a new paint job. SAP implementations, Salesforce rollouts, Oracle ERP projects, enterprise software has been running on the embedded expert model for decades. You buy the platform, you get the consultants. The implementation is complicated. It works and then they leave and you find out very quickly whether your organization understood what just happened or just hosted it. The companies that do well longterm aren't the ones with the best consultants. They're the ones that treated the engagement as a knowledge transfer, not just a delivery that insisted on pairing with their internal teams alongside the implementation team throughout. That said, before you leave, I need someone in the building who owns this system. The companies that don't still on maintenance contracts 15 years later because the original configuration decisions are undocumented. The people who made them are long gone and nobody on the current team knows which settings are load-bearing. AI forward deployed engagements are going to learn the same lesson. The only question is whether companies learn it proactively or the hard way.

But here's what the steelman doesn't address. The forward deployed engineer is optimizing for something and that something is not your team's ability to maintain the system in 3 years. That's not a moral judgment. That's a structural one. They're optimizing for proving the platform works in your environment, hitting a renewal milestone, generating a case study, getting enough success signals before the engagement ends that the contract extends. Those are good things for their company. They're not bad people for chasing them. But those incentives are structurally misaligned with what your team actually needs, which is a system that is still legible in three year three by someone who wasn't in the room when it was built.

There's a pattern I've talked about before on the show, and I've called it the governor. The governor is the constraint that enforces discipline when your team doesn't have it internally. Think of mandatory code review, architecture sign off, a release process that can't be skipped. Whatever makes a bad decision slower to ship than a good one. The governor makes risky changes expensive. That's its job.

In the forward deployed model, you're operating without a governor on the FDE side. The FDE has expertise. They probably know the platform better than anyone in your building, but your team doesn't have the context to push back on their decisions in the moment because the FDE is by design more expert in the tool than anyone in house. Your team can't push back because they're still ramping and the FDE is already done.

I think about something an engineering director said to me once about a large platform migration they ran with a vendor engagement. They said, "We didn't know what questions to ask until after they left. By then, the decisions were already in production." That's the accountability gap. Not intentional and not malicious. It's just structural.

So, here's a weird analogy, and please bear with me. You know how some people hire a professional organizer? They come in, Marie Kondo it up, they transform your house, everything has a place, the system makes perfect sense, and you're delighted, and they leave. 6 months later, you're looking for the measuring cups. Your family has adopted the system at about 70%. The 30% that drifted, that stuff didn't have a label. A forward deployed engineer is a professional organizer for your codebase. Except the stakes are higher. The measuring cups are your production auth service. You can't call them back for a refresh session because the engagement ended when the contract did.

Here's why it's all converging. Now the idea is not new but the scale is and to understand the scale you have to understand what the vendors are actually solving for. AI platforms are genuinely hard to adopt. Not in the documentation is bad way in the gap between what works in the demo and what works in your production environment is a full engineering project way. Every enterprise has different data, different existing systems, different compliance requirements, different levels of internal expertise. A platform that deploys cleanly for one company can take 6 months at another. And if it takes 6 months, you're not renewing. Time to value is the whole game. It determines whether a customer becomes a relationship or a churned logo on a slide.

Documentation doesn't close that gap. A solutions architect on a Zoom call doesn't close it either. The knowledge that actually makes the difference. That's in your data architecture. This particular pattern will create contention at the third tier of your retrieval pipeline that lives in a person. It's tacit. It comes from doing implementations, not from writing READMEs. An embedded engineer who knows the platform and builds the first thing alongside your team closes the adoption gap in a way that no amount of documentation can.

And alongside that stickiness when an FDE builds something in your codebase, your production system now depends on patterns and abstractions from that platform. Replacing the platform means replacing the implementation. That's not necessarily by design. It's just the natural consequence of deep integration. The vendor's code and your code become hard to separate. Which means when the renewal conversation comes around, the cost of switching has gone up.

From the vendor side, forward deployed engineering solves two problems at once. Slow adoption and low retention. It closes the gap that was killing deals. It creates the integration depth that makes walking away expensive. That is a rational business model worth being clear-eyed about. Not because it's predatory. It isn't. But because knowing what the vendor is optimizing for is the only way to understand what you're agreeing to when you sign.

The question is what comes after. The deal only works if both sides understand it's a deal. And in my experience, the sales conversation is about speed, time to value, getting a use case in production in 8 weeks instead of 6 months. The conversation about stewardship, about what it looks like when the FDE leaves and your team has to own this alone. That conversation is quieter and it needs to be louder.

Now, I don't want to end this episode with don't use Forward deployed engineers. That is not the answer. Okay? The scarcity problem is real. The appeal is real. There are FDE engagements that go extremely well, where knowledge transfers happen, where the FDE invests in leaving the system legible and the team better than they found it. What I want to leave you with is the questions to ask before you sign, not a framework, what I'd actually say in the room.

So question one, what is the explicit exit criteria and does it include your team's ability to maintain this independently? Not is the system running. I don't care [clears throat] if the system is running. I care if your engineers can be paged on this at 2 a.m. and handle it. That's the bar. If the exit criteria doesn't include that bar, you're buying a running system, not a transferable system. Those are different products at the same price point.

Question two, how is the FDE incentivized and what happens at renewal? This is not a rude question. This is due diligence. If an FDE is measured on whether you renew, they have a quiet incentive to stay in the loop. If they're measured on knowledge transfer, they have an incentive to make themselves replaceable. Same goal, different engineer. You can ask it plainly, what does success look like for your team at the end of this engagement? And then listen for whether the answer includes your team's independence or just platform performance metrics.

Question three, what documentation is included and who owns the decision log? Not API documentation. The platform team has that. I mean the documentation that explains why specific decisions were made in your environment. Why this abstraction instead of that one? Why this service boundary? Why did they use this data model? That's not technical documentation. That's decision documentation. And it's the most valuable artifact of any implementation engagement because it's the only way the new engineer on your team can reason about the system without reverse engineering every choice from first principles. If the engagement doesn't include a structured decision log, ask for one. If the vendor says that's out of scope, that answer tells you something.

Question four, and this one's softer. What are you actually buying? Because sometimes the honest answer is time. You're buying time to figure out your AI strategy while something is running in production. That's a real thing to buy. Just know that you're also incurring a future obligation. At some point, your team has to actually understand the system. The FDE bought you time. The debt is still there. Deferred debt, technical or organizational, it compounds.

So what does excellent actually look like? I don't want you to leave this as ask better questions without telling you what you're asking toward. The best forward deployed engineers I've seen work themselves almost out of a job. Not passively, actively. Every design review includes explanation. Every architectural decision has an owner for your team, not just theirs. By the last month of the engagement, they're pairing instead of leading. Your engineer is in the driver's seat. The FDE is the one asking, "What would you do here?" instead of answering. That's a choice, and it doesn't happen by default.

The early signal is in the first 30 days. Are they documenting as they go or is the documentation a deliverable at the end? Are they pulling your engineers into decisions or presenting decisions to them after the fact? Are they writing rationale into your system or keeping it in their heads? Those two modes look similar from the outside on week one, but they look very different in month six.

By month two of a good engagement, at least one person on your team should be able to explain the core architecture to someone new. Not in a let me check with the FDE way. In a I understand this and here's why we made these choices way. If that person doesn't exist by month two, the knowledge isn't transferring. It's just accumulating in one place and that place is leaving.

The other thing I watch for is how the FDE handles disagreement. In a bad engagement, they implement your team's call quietly and move on, which sounds professional until you realize the thing they disagreed with is now load-bearing and nobody documented why the decision went the way it did. In a good engagement, they make their recommendation. Name the trade-off, defer to your team, and write down why the call went where it went. They're building the decision log in real time. That's not extra work. That's the job.

The best contractors I've seen say something like this out loud in the first week. My job is to make sure you don't need me by the time I leave. That's the disposition. You can screen for platform expertise. You can't always screen for that.

What the end of a good engagement looks like, and I'm going to be specific here, is a handoff week, not a handoff document, not a PDF, a week where the FDE is an observer and your team is running the system, where the questions go the other direction, where the FDE's job is to say, "Yes, you've got it," and mean it. A handoff document is a record. A handoff week is a test.

When you're evaluating an engagement before you sign or after something breaks, the question I keep coming back to is at the end of this, does the knowledge live in your system or in their heads? A system is infrastructure you own. A head that belongs to someone else's company is not.

So here's where I land. AWS putting a billion dollars behind forward deployed engineering is a signal. It means the hyperscalers have decided this is how enterprise AI gets adopted at scale. That's not going to change because some random podcast host in Orlando has opinions about accountability debt. But I think there's a version of this that actually works. And it's not the version where the FDE is a faster way to build something your team can't maintain. It's the version where the FDE is a teacher. Where the engagement is structured from day one around transfer, not just delivery.

The best contractors I've worked with, and I've worked with a lot of them across my career, left systems that my team understood. Not because the work was simple, because they made legibility part of the deliverable. They wrote the explanation. They led the walkthrough. They said, "You should be able to do this without me." That's the bar.

But there's something underneath all this I haven't named yet. You're granting this person, someone who works for your vendor, whose incentives we've already talked about, enormous architectural influence over your production systems. You're trusting them to make good calls in the moment, to flag things your team doesn't know enough to catch, to build something that will outlast their presence. That's trust, not contract trust, real trust. And trust isn't built by expertise. The FDE being excellent at their platform doesn't give you that. You don't trust someone because they're smart. You trust them because they're transparent, because they show their work. Because when they make a call you can't evaluate, they explain it instead of expecting you to defer. The question isn't whether they're smarter than your team. They probably are, or at least about their platform, that's why you hired them. The question is whether they're leaving behind understanding instead of dependence.

If I were sitting across from a vendor account team tomorrow, contract in front of me, here's what I asked before I picked up the pen. When this person leaves my building, can my team operate, maintain, and extend the system without calling you? If the answer is yes, we have a deal. If the answer is we'd recommend an extended engagement, that's also an answer. It's just a different one.

If you remember one thing from this episode, remember this. A running system is not an understood system. Production isn't the finish line. Understanding is.

This is Chaotic Commits. If this episode landed for you, share it with someone who's in the middle of a vendor AI decision. Forward it to the engineering leader who's about to sign a contract without asking about exit criteria. And find me on Substack and Bluesky. Links are in the show notes. New episodes every Friday. I'm Joanne Skiles. Write code that future you can read.