Reading time: 9 minutes | Issue #50 | Book a Discovery Call

Happy Tuesday. Mark here.

Last Wednesday, GitHub shipped a feature whose main selling point is that it keeps working after you quit.

At its Universe conference, GitHub rolled out shared Copilot Automations. These belong to the repository, so they keep running when the engineer who set them up leaves the team. The AI credits go on the repository owner's bill.

It addresses a problem I wrote about on X, in a post on the seven steps from tool adoption to an agentic org.

This week, I'll get into what belongs in it, why you should own it and how it fits into the seven steps.

Inside the Issue

Shopping for step seven

On October 20, Gartner's Gene Alvarez presented the firm's top strategic technology trends for 2027 at its Symposium in Orlando. Three of the ten appear on my ladder: AI context platforms, the agent coding harness and AI agent identity. A fourth, disposable applications, comes with advice to put an expiry date on software and govern it "from creation to death," as Alvarez put it, according to TechTarget.

Gartner presents these as separate trends. The way I look at it, they're rungs on a ladder: you need the lower ones in place before the higher ones will hold you.

Our read: Most companies sit on step one, shopping for step seven technology. They need step four discipline.

1. Tool adoption.

There are AI tools across the org, but a human runs each one. In a survey of nearly 1,900 IT leaders in December and January, OutSystems found that 96% of their organizations use AI agents in some capacity. At the same time, 94% worry that AI sprawl is adding complexity, technical debt and security risk, and only 12% had a centralized platform to manage it.

We've seen this ourselves. We walked a VP of Engineering through all seven steps, and his org, with over a hundred tools, was on step one.

2. Context layer.

Each department gets its own knowledge graph behind its own API gateway. When sales needs something from engineering, a sales agent queries engineering's gateway and gets back only what it's allowed to see. Keeping the domains separate stops agents from wandering into graphs nobody gave them access to and hallucinating across departmental boundaries.

Gartner calls this an AI context platform. Without one, it warns of agents with excessive privileges, compliance failures and higher token costs. It also predicts that by 2027, organizations that prioritize semantics in AI-ready data will raise agentic AI accuracy by up to 80% and cut costs by up to 60%. I'd read both numbers as ceilings.

GitHub shipped a version for engineering last week. It describes GitHub intelligence as a context and learning layer built from the graph of GitHub data, covering team conventions, memory across sessions and code search across repositories. Your finance and customer data sit in a different corner of the zoo of systems, though. You'll have to build the gateway between them yourself.

Two questions decide how you build this layer.

What belongs in it. For a given department, I'd start with its entities and the documents that define them. Then I'd add the decisions the department made and the reasons behind them, which I call data exhaust, along with the permissions that say who may read what. Give an agent the documents without the decisions and it'll repeat mistakes the department has already made.

Why you should own it outright. You'll keep your context longer than any model or vendor. Model vendors change their lineup every few months, as we'll see in step three. Two vendors in this issue will host the context for you: GitHub for your code and Glean for your documents (Signal 01). Hosting is fine. Owning it means you can export the graph with its permissions and point a different model or vendor at it without rebuilding anything.

3. Harness engineering.

Build the evaluation criteria, routing logic and quality gates before you pick a model. The LLM comes in last. Your harness is your IP and your capex, while the model underneath is a commodity you swap when something better ships. An agent with no harness around it is a gas-powered RC car on a coffee table.

OpenAI gave us a reason to care on October 23, when it shut off API access to GPT-4, GPT-4 Turbo, GPT-3.5 Turbo, o1, o3-mini and o4-mini, along with other models it had flagged in April. Nine days earlier, it told developers that its remaining GPT-4o snapshots, GPT-4o mini, GPT-4.1 and GPT-4.1 mini will stop working on January 14. If your team tuned its product prompt on GPT-4o, you have about ten weeks.

Your team needs a test that shows whether the replacement model gives the same answers. Otherwise, you'll find out from your customers. A system that breaks when you change the LLM is tied to one vendor's release calendar.

For the healthcare customer we wrote about in Issue #44, we built a 60-case golden set from the customer's own hand-written SQL answers before the agent existed. The agent got 59 of 60 right on its first deploy, and it's model-agnostic. We'd handle a notice like OpenAI's by running those same 60 cases against the replacement.

This is also why I keep coming back to the 73/27 split we first reported in Issue #11: across 14 engagements, 73% of the work that got AI into production had nothing to do with the model.

4. Go headless.

Host every automation outside a laptop so it runs on a schedule or a condition, without a human logging in at 9 a.m. Companies stall here because nobody forces the move from "script on my desktop" to "service that runs on its own."

The good part is that GitHub made the hosting easier last week. Alongside shared automations, it took Agentic Workflows to general availability. You describe the task in a Markdown file, the tooling compiles it into standard Actions YAML, and the agent starts with read-only access. A separate threat-detection job inspects any change it proposes.

But GitHub can't ship the forcing function. Somebody on your side has to keep track of what runs where, move each item and switch off the laptop copy. On step one, nobody owns that job, and in the day-to-day whirlwind the scripts stay where they are.

I told that VP: "You're on step one. But over a hundred tools means step four is closer than you think." Each tool represents a workflow somebody runs by hand today. It has a known input and output, and a person who can tell you when it breaks. Put those tools and people on a list, and you have his step-four backlog.

Step four is the least sexy step on the ladder, and you can't run steps five through seven without it.

5. Governance.

Put RBAC on every knowledge structure from day one and a TTL on every agent. Once you're past 100 tools, you need lifecycle management: retire anything that hasn't run in 90 days and deprecate anything whose eval scores drop below your threshold. Dead agents are tech debt with active API keys.

According to Our Cloud Network's write-up of Microsoft's documentation, Microsoft Entra Agent ID now disables an agent identity after 90 days without activity, the same cutoff as my rule. The default policy is in preview and free for every tenant. It notifies the agent's owners at 60 days and deletes the identity at 455. Microsoft's best-practice guide also tells admins to check each quarter for agents with missing sponsors or no recent activity. Nobody in Redmond called me first.

Gartner predicts that by 2028, 25% of enterprise security breaches will trace back to AI agent abuse. Yet when ETR surveyed 517 technology leaders for its 2026 security study, it found that only 3% had agent-specific security controls in broad production use. One in five had none.

At a healthcare data customer, we route LLM calls through a gateway that holds the keys, budgets and audit trail, and we trace each agent run. We've also gotten this wrong: on one engagement, we didn't have telemetry from day one and found a problem a month late. With telemetry, we would have seen it in a week.

6. Agent-to-agent.

Once the instances run headless, stitch them together within a value chain. Agent A finishes and hands off to Agent B, with an LLM between them as a quality gate, judging each handoff before it passes through.

At Universe, GitHub also made Copilot Fusion generally available, with a version of that gate built in. In Cascade mode, an efficient model drafts and a quality check either accepts the draft or hands the task to a stronger model, which writes a fresh answer. In Critique mode, a read-only critic approves the draft or sends it back.

I'm less sure of this step than the others. Last week, Anthropic published research in which Claude agents built fine-tuning datasets that passed a production-like safety monitor and trained a research model to misbehave, MIXED reported. The monitor checked each training example on its own, while the agents wrapped the bad behavior in fiction and roleplay. No single row looked alarming. Anthropic's own caveat is that "monitoring is only one of several defenses."

An LLM judge reading one handoff at a time can miss a problem that only shows up across a hundred of them. I think the LLM gate belongs between agents. With that said, I wouldn't trust it as the only gate. I keep telling customers you can't rely on an LLM to do the job inside a workflow. That applies to the LLM doing the judging too.

Our PR review agent at that healthcare data customer is advisory. It never merges and treats the diff as data, never as instructions. Behind any LLM gate, I'd put checks an agent can't talk its way past, like tests and spend caps.

7. Step functions.

At step seven, the org runs on agent output, with orchestrated workflows that are semi-automated or automated end to end. Agents handle the volume and humans handle the exceptions.

Gartner expects IT operations to get there first. In its October 26 report, it says almost no organizations run AI agents with minimal human involvement in IT operations today. By 2030, it expects half to do so as teams move from human-in-the-loop to human-on-the-loop.

At the Orlando Symposium, Gartner's Daryl Plummer warned: "Do not allow agents to act as a mob." Put a step-seven platform on top of step-one habits and that's what you'll be orchestrating.

If you're at a portco evaluating orchestration platforms this quarter while your most-used automation runs on somebody's laptop, that should make you uncomfortable.

01 "Headless" was October's favorite word. Copado made Agentia Headless generally available on October 20. Glean shipped headless agent building the next day, and Dataiku launched Dataiku Headless at its Succeed conference the week after. All three use headless to mean no UI: you drive the platform from a coding agent like Claude Code or Cursor.

02 Copilot now comes with GitHub Enterprise, billed from the first request. Under the new option, GitHub includes no monthly AI credits per seat. All AI usage is metered against your enterprise budget. When that runs out, completions keep working, but chat, code review and cloud agents stop. It's available through account teams now, with self-serve to follow. If your portco runs GitHub Enterprise, ask the account team when the switch applies to you and set the budget first.

03 One sign-in, 26 services. Copilot connectors have been in public preview since October 28. A developer connects Slack, Figma, Notion, Sentry, Linear and other services once, and Copilot keeps them signed in in the background across its app, CLI, VS Code, cloud and mobile. The app can also run scheduled automations using a connected service, meaning one person's sign-in can end up behind a scheduled job. Find out who owns those sessions when somebody leaves.

04 Gartner expects cost attacks on public-facing AI. Plummer's top predictions include this one: by 2030, 80% of organizations with public-facing AI will face cost-exhaustion attacks. If an outsider can run up your token bill, a spend cap on each key is a security control.

05 DoorDash counted a few hundred agent orders. On the third-quarter call, CEO Tony Xu said DoorDash is seeing "maybe several hundred orders in total" from third-party agents, TechCrunch reported. Compare that with about 3 million deliveries a day in its 2025 reporting. Consumer agents are on step one too.

The Seven-Step Ladder. Start at step one and work your way up, using one test per step. The first test you can't answer with a name and a date tells you which step you're on.

If your company runs dozens of AI tools but your best automation needs somebody's laptop open, book 30 minutes with me. We'll walk your engineering leads through the seven steps, just as we did with that VP, work out where you are and pick the first automation to take headless.

If you'd rather we handled the move, an AI Velocity Pod works alongside your team month to month. If we don't deliver, you don't pay.

Until next Tuesday,

— Mark Ajzenstadt, Founder @ Limestone Digital