Data lineage starts in the code that moves the data
By Xavier Franco, CEO de Elternativa · Grupo Zemsania
· 5 min read
Written with the help of AI and reviewed by Xavier Franco

In this article
There is a question that sums up the problem of data governance: if you change that table, do you know which report breaks? There is almost never a quick answer. And it is not for lack of catalog tools. It is because data lineage has been treated as a diagram that someone draws, when in fact it is written somewhere else: in the code that moves the data.
What data lineage is
Data lineage is the path a piece of data follows from its origin to where it is used: which system it comes from, which processes transform it, which tables it is stored in, and which reports, metrics or applications read it.
It serves three very concrete purposes:
- Trusting a figure, because you know where it comes from.
- Knowing the impact of a change before making it.
- Proving to audit or compliance how an indicator was calculated.
In practice: pick the most important indicator of your executive committee and ask which tables and which processes it comes from. Measure how long it takes them to answer.
Data and software don’t talk to each other
In many organizations, the data world and the software world run separately. The governance team maintains a catalog with a glossary, owners and, sometimes, lineage. The development team writes the processes that move the data: pipelines, transformations, procedures and loads.
The problem is that the catalog is updated by hand and the code changes every week. A drawn lineage ages from day one. And when you need it most, in a migration or an audit, it no longer matches what is running.
Lineage is written in the code
The good news is that lineage doesn’t have to be invented. It is in the code. Every transformation says what it reads and what it writes. Every report says which table it queries. Every procedure says which columns it touches.
That is why reliable lineage comes from reading the code, not from asking people. And also from capturing it when processes run: the open standard OpenLineage, for example, defines a common model of datasets, jobs and runs, and is designed to record the metadata of a process while it runs.
Reading the code and observing the execution are two sides of the same coin: lineage comes from what really happens, not from what someone believes happens.
In practice: ask your data team whether the lineage in your catalog is generated automatically or maintained by hand. If it is by hand, ask when it was last updated.
What is certain and what is inferred
There is a nuance that I think is important. Not all lineage is equally reliable. A relationship that comes directly from the code is certain. Another one that is inferred, for example from a similar column name, is probable.
A good lineage map has to tell the two apart. And when you click on a relationship, it has to tell you which file it comes from, which method, and with what confidence. Otherwise, the map looks more reliable than it is.
What breaks if you move it
Where the value of lineage shows most is in a migration. Before moving a process, a table or a report, you need to know what depends on it. By following the relationships on the map, you see the impact before touching anything.
And you can narrow it down: start from a screen or a report and see only what it reaches. It is the difference between migrating with a plan and migrating with fear.
In practice: before approving any change to a shared table, ask for the list of what reads it. If it doesn’t exist, the change is a bet.
When a figure doesn’t add up
Lineage is also the most useful tool when something fails. An indicator in a report comes out different from what was expected, and the usual question begins: is it the source data, a transformation or the report itself?
Without lineage, the answer is an investigation that goes through several teams. With lineage drawn from the code, it is a walk: you follow the figure backwards, process by process, until you find where it changes. And when you find it, you also know which other reports read from that same point and may be affected.
It is the difference between saying “we’re looking into it” and saying “it comes from this transformation, and it affects these three reports”.
In practice: the next time a figure doesn’t add up, measure how long your team takes to find the origin. That time is what lineage gives back to you.
New code, with its lineage from day one
All of the above applies to the code you already have. But there is another opportunity: new code. With AI writing more and more pipelines and transformations, lineage can be generated at the same time as the code, not afterwards.
If every process that is written leaves behind its technical lineage and its metadata, and they are published to your governance catalog, the catalog stops lagging behind. Data and software, at last, talk to each other.
Where to start
- Pick one critical indicator and rebuild its lineage.
- Get the lineage from the code, not from interviews.
- Tell the certain from the inferred.
- Use the map before every change to see the impact.
- Make new code publish its lineage to your catalog.
How we help
At Agentic Data & Software Engineering we orchestrate development AIs with data governance as a cross-cutting layer: the technical lineage and metadata of the generated code are published to your governance catalog, such as Informatica, Collibra or Talend.
And with the Datagentis migration module we draw the map of the system you already have: reports, tables, metrics and screens connected in a single graph, with certain and inferred relationships told apart, the provenance of each piece of data, and what breaks if it is moved.
Do you know which report breaks if you change that table?
In a demo we show you how the map of tables, reports and metrics is drawn from the code, and how you can see the impact of a change before making it.


