The most rigorous controlled experiment to date on AI productivity in programming found that expert developers were slower using it, not faster. And they didn't even realize it. For Central American banks, which are only just beginning to integrate AI strategically, this finding should carry more weight than any demo.

In July 2025, an independent AI evaluation lab called METR did something almost no technology provider does: it measured, using a randomized experimental design, whether AI actually speeds up software developers. It didn't use lab tasks or synthetic benchmarks. It recruited 16 developers with years of experience working on the same open-source repositories that would be involved with mature projects of more than a million lines of code and randomly assigned them to solve real-world tasks, with or without AI assistance: bug fixes, new features, refactoring.

The result contradicted all expectations. The developers took 19% longer to complete their tasks when using AI tools than when not using them. Before the experiment, experts consulted had predicted an acceleration of around 40%. At the end, the participants themselves still believed that the AI ​​had made them 20% faster. The difference between what they felt and what actually happened wasn't a margin of error. It was almost a 40 percentage point gap between perception and reality.

19% slower · 20% “faster” (according to them) METR, July 2025 — 16 senior developers, real tasks in mature repositories

This is the fact that should worry any technology committee evaluating AI for software development or testing: if senior developers, with years of experience with the code they were working on, couldn't tell for themselves that they were working more slowly, what chance does an organization have of detecting that same effect without deliberately measuring it?

The short answer is none. And therein lies the fundamental problem: most institutions adopting AI in their development teams are doing so based on the same kind of subjective perception that the METR study showed to be unreliable. Without objective metrics—real-time delivery, production defect rate, review effort—an organization can spend months "feeling" that its speed has improved while its data says the exact opposite.

This finding doesn't mean AI is useless for software development. It means its value depends entirely on the type of task and the level of structure surrounding it. A separate study by the National Bureau of Economic Research, involving more than 5,000 customer service agents who began using a generative assistant, found an average increase of 14% in problems resolved per hour, with a jump of up to 34% among less experienced workers.

The difference between the two studies isn't a methodological error. It's the variable that defines whether AI helps or hinders: in a streamlined customer service process with clear guidelines and predictable responses, AI acts as a genuine accelerator, especially for those with less experience. In a mature software system with complex business logic and dependencies that only someone with deep understanding can assess, the same technology introduces friction, additional review, and decisions that a model cannot make judiciously. A core banking system, a credit scoring engine, or a fraud detection system is much closer to the second scenario than the first.

There is a second, still recent, line of research that is giving a name to a phenomenon the development industry is only now beginning to accurately define: cognitive debt, also described in academic literature as epistemic debt. It's not simply code that's difficult to maintain: that was already described by classic technical debt. It's the lack of real understanding, within the team, of how the code that the team itself accepted without thoroughly reviewing actually works.

When a developer accepts a lengthy AI-generated suggestion without giving it the same level of scrutiny they would give a pull request from a colleague, the system continues to function, but the knowledge of why it works remains trapped in a context that no one on the team can reconstruct. The cost of this doesn't appear on launch day. It appears months later, when that component fails in production and no one in the room can explain with certainty what broke or why, because no one fully understood it the first time.

For a financial institution, this isn't an engineering productivity problem. It's a business continuity and regulatory compliance issue: an auditor does not accept as an answer that "the system works, but we don't know exactly why".

This discussion is not merely theoretical for the region's banking sector. According to figures cited by Forbes Central America, only 8% of Central American banks had integrated artificial intelligence at a strategic level by 2025, although most had already moved beyond the isolated pilot phase. This region is entering this trend later than other markets, which can be seen as either a disadvantage or an opportunity: arriving later means being able to learn from the mistakes others have already made, instead of repeating them.

8% · 54% · 43% / 42% Banks with integrated strategic AI (2025) · Companies with specialized risk tools · Companies that see ethical risks as the main threat (Central America / Mexico) — Forbes Central America and KPMG, 2026

KPMG's "Risks in Mexico and Central America 2026" report adds a second piece of data that directly connects to the above: only 54% of Central American companies use specialized tools to manage technological risks. And when asked what worries them most about the use of AI, 43% of Central American companies and 42% of Mexican companies cite ethical challenges as the main risk, above purely technical or security risks. A regional financial stability document published in August 2026 goes a step further: it warns that the widespread use of AI models in banking can amplify correlations between institutions and accelerate episodes of systemic stress when many banks react similarly to the same signal.

Taken together, these three points paint a picture of a region that still has time to adopt AI in development and quality assurance with a discipline that other, more advanced markets had to learn the hard way.

At Q-Vision, we have supported core banking modernization processes, software development, and quality assurance for financial institutions in Colombia, Mexico, Panama, and Ecuador for over twenty years, and the pattern behind METR's findings is not new to us. Every technological wave brings the same temptation: to measure success by the speed with which something is launched, rather than by how robustly it continues to function six months later. This happened with mobile banking. It happened with instant payments. The same thing is happening with AI in development and testing, only now science has the data to prove it.

Our quality assurance practice was not built to hinder the adoption of AI in financial software development. It was built so that adoption is measured with the same rigor as any system that will sustain third-party funding: with objective performance metrics, with continuous testing designed for code whose behavior can change from version to version, and with the discipline of requiring someone on the team to truly understand every critical component that ends up in production, regardless of who or what wrote it first.

No bank in the region is going to halt its AI adoption while waiting for the next study, nor should they. But the evidence available today presents a concrete task for any CIO or CTO: before reporting to their board that AI has accelerated their development cycles, they must be able to demonstrate this with measured data, not with the same feeling that leads sixteen expert developers to believe they've gained time when in reality they've lost it.

The institution that takes this scenario seriously and builds the necessary discipline of measurement, governance, and continuous testing to respond with evidence will not only avoid the error that science has already documented. It will be building, from this moment forward, the advantage that the rest of the industry is still mistaking for speed.