Agentic Data & Software Engineering

Your team’s AI productivity is measured in your Git, not in surveys

By Xavier Franco, CEO de Elternativa · Grupo Zemsania
· 5 min read

Written with the help of AI and reviewed by Xavier Franco

Freshly cut tree stump in the forest, with its growth rings clearly marked, some wider and some narrower.
In this article

If you ask your development team whether AI makes them more productive, they will most likely say yes. And they probably truly believe it. The problem is that believing it and measuring it are not the same thing. I put it like this: your team’s AI productivity is not measured in surveys or in gut feelings. It is measured in your Git.

Perception misleads, and a lot

There is a study that shows it better than any argument. In 2025, METR ran a controlled trial with 16 experienced developers from open-source projects, on 246 real tasks. With AI, they took 19% longer to complete them.

What is interesting is what they thought. Before starting, they expected AI to make them 24% faster. And after finishing, with the delay already measured, they still believed AI had sped them up by 20%.

I am not concluding from this that AI is useless. The study is one specific case, with its limits. What I conclude is something else: if the people who lived it did not perceive it correctly, an internal survey is not going to tell you the truth.

In practice: before running a survey about AI, decide which data point in your repository will confirm or refute what people answer.

What to measure: the DORA metrics

To measure software delivery there is already a well-established standard: the DORA metrics. They group five indicators into two blocks.

Delivery performance:

  • Lead time for changes: from when a change is committed to version control until it is in production.
  • Deployment frequency: how many deployments you make in a period.
  • Recovery time from a failed deployment.

Stability:

  • Change fail rate: what share of deployments needs immediate intervention.
  • Rework rate: what share of deployments was unplanned and comes from a production incident.

The good thing about these metrics is that they come from what you already have: your Git, your pull requests and your CI/CD. They do not depend on what anyone remembers.

Speed without stability is not productivity

Here is the most common trap. AI makes more code come out, faster. If you only look at speed, everything seems to be going better.

But the 2025 DORA report warns about it: AI adoption improves delivery performance, but it still has a negative relationship with stability. More changes, without solid controls, lead to more instability.

That is why you have to look at both blocks at once. If you deliver twice as much but half of it comes back as an incident, you have gained nothing: you have just moved the work somewhere else.

In practice: put deployment frequency and change fail rate on the same dashboard. If one goes up and the other goes up too, you need to talk.

With and without AI, in the same repository

The question that really matters to whoever leads is: how much more does my team deliver with AI than without it? And to answer it you need to compare within the same repository.

That requires knowing which deliveries come from an AI-assisted flow and which do not. Without that marker, everything gets mixed together. That is why, in our method, every task is traced: who did it, how long it took and whether it passed validation. And the performance dashboard shows, measured on your own repository:

  • The real multiplier: weekly deliveries with and without the method.
  • How much AI does: the percentage of deliveries that come out of the AI-assisted flow.
  • Agility: the time to integrate and validate, and how it changes each month.
  • Towards production: the deployments and the validations signed by people.

What you should not measure

Just as important is knowing what not to look at:

  • Lines of code. AI writes a lot of them. More lines is not more value; sometimes it is the opposite.
  • Number of requests to AI. It measures usage, not results.
  • Hours “saved” according to each person. We have already seen how reliable perception is.

What matters is what reaches production, validated and stable.

In practice: if someone shows you an AI productivity report based on lines of code, ask for the same report with deployments and failed changes.

Compare with your data, not with promises

What I always propose is the same: do not trust our figures or anyone else’s. Compare it with your data. Pick a service you already have, run it in parallel with orchestrated AI in a bounded scope, and compare cost, timeline and quality in your own repository.

In the end, Git has no opinion. It records what was integrated, when, and how many times you had to go back.

Where to start

  1. Get the baseline of the DORA metrics for the last few months.
  2. Mark which deliveries come from an AI-assisted flow.
  3. Look at speed and stability together, never separately.
  4. Drop the vanity metrics, such as lines of code.
  5. Compare with and without AI in a bounded scope, with your data.

How we help you

In Agentic Data & Software Engineering we orchestrate the development AIs you already have across the whole SDLC, and we start on a service you already have: we run it in parallel, in a bounded scope, and compare cost, timeline and quality with you, using your data.

We do it with Agentagile, our methodology for developing with AI, which measures productivity in your Git: deliveries with and without AI, how much AI does, time to integrate and validate, and deployments signed by people.

Frequently asked questions

How do you measure AI productivity in software development?

With data from the repository and the CI/CD, such as the DORA metrics, comparing the deliveries that come from an AI-assisted flow with those that do not.

What are the DORA metrics?

Five software delivery indicators: lead time for changes, deployment frequency, failed deployment recovery time, change fail rate and deployment rework rate.

Are surveys useful for measuring AI productivity?

As a complement, but not as a measure: in the METR study, developers believed they were 20% faster when they were actually 19% slower.

Do lines of code measure productivity?

No. AI generates a lot of code; what matters is what reaches production, validated and stable.

Keep reading