Tech Series – How technology decisions in enterprise organizations create hidden costs, and the architecture and governance patterns that prevent them.

1. Tech: AI Prompting in Practice – The six-part prompt framework and workflows that turn average AI outputs into reliable, repeatable results
2. Tech: IBM in an Era of AI – The hidden cost of treating legacy enterprise platforms as permanent rather than as debt with compounding interest
3. Tech: The Escalation Gap – Why AI agents deployed without escalation infrastructure produce a governance gap that no dashboard tracks
4. Tech: AI First Approach – What You Don’t Know Yet (and What It’s Costing You) (this article) – Why the AI chat, copy-paste approach is a hidden productivity tax, and how to build a structured AI-first pipeline starting with a two-file architecture, meta-prompting, and self-correcting feedback loops.
5. Tech: The Technical Debt Ledger – How to calculate and present the real architectural and organizational cost of tech debt to your CFO.

There are a few things more earth-shattering to a new developer than having to understand that a world of everything JavaScript, Python, or DotNet (C, C++, C#) is not a real thing when you enter the workforce, especially if you’re part of a financial company. Multiple indices, including TIOBE, PYPL, and Octoverse, would give someone a perspective that if you just learn at least three languages, you’re set in the Software Engineering space. I beg to differ. From a business perspective, companies that embrace closed-source, proprietary tooling are setting the stage for a new and interesting challenge to the “Open Source Everything” philosophy.

AI Shaping the Digital Workplace, but not all at once

AI has been shaping the world, whether from a systems engineering, platform engineering, or software engineering context. The fact is, we live in an era where researching and decision-making before you start a project are even more important than ever. New concepts, languages, and frameworks are becoming a catch-22 where they must be widely accepted (and trained into an AI model) before they are going to be used widely. In a recent video, Arjan of the Arjan Codes YouTube channel described how to design a system using AI. In his video, Arjan provided insight on the rapid-prototyping phenomenon where developers exclaim, “Look, I used AI to create this in an hour!” For anyone writing these apps, you may have written it in an hour – but often cannot get AI to edit it meaningfully to customize your needs, have AI maintain it, or be able to recreate similar results reliably across multiple AI agent versions (even with the same provider). Yes, you may get lucky, but its not always sustainable for AI models to iterate, edit, and customize an application for specific business requirements. Today, the engineering model has moved from simply editing an implementation to requiring a complete rewrite of a 100-800 line prompt to get it to do what you want, then reviewing and editing an implementation. So, instead of simply knowing and writing the code, you’re now stuck knowing the code, helping to write the code, while also babysitting a new (currently) unpredictable tool to write your code – not a huge leap forward for complex business solutions. This phenomenon helps explain (at least partially) the divide we see in the rest of the world, with the US not being the top player in AI diffusion despite being a top player in AI Development.

Opposition to Inefficiency is Growing

Currently, efficiency has diverse meanings (processing efficiency, development efficiency) with new datacenter development being politicized (Washington Post, Associated Press, Politico) within the United States. Corporate AI model choice efficiency is going to matter more over time, and companies need to find solutions that optimize not only for how quickly they can get products to the market, but also for how well the framework, language, or concept aligns with the intended deliverable. In 2027, corporate solutions must consider how efficient, easy to maintain, and AI-aligned the solution can be made over multiple generations of AI agents. This is not efficiency in processing the transaction (e.g., a similar example of Bitcoin with 788,501 times worse performance than Visa), but instead the efficiency with which you create a solution using AI (today, this is similar to Bitcoin, though not as bad).

AI Development Efficiency Gap

Today, it’s common to use one long-running agent to process a given context, while some break it into multiple agents using different vendors or agents may do this under the hood. The incentive to “just get it done” is not represented in Return On Investment (ROI). AI is a cost center, driving down development cycle time, but as it is used more, it will turn the problem on its head. Engineering has historically been tied to how quickly a company can turn business requirements into code. Going into an AI-centric future, the problem adds the complexity of being able to distill it down to doing it right, with the right thing, at the right time, and in the right way. The more AI becomes the center of our workflows, the more its important we focus on design, not lines of code, and not slop. There’s a significant concern about having a Talent Pipeline with many companies deferring hiring junior developers (Times of India, New York Times), instead increasing reliance on senior developers, and providing them AI-generated (slop) content to review, which can make development slower and less well designed.

Enter IBM – The Efficiency Player

IBM has made some strategic purchases over the last few years, including HashiCorp, RedHat, Ansible, DataStax, and most recently, Confluent. This makes development easy as IBM integrates all the products these companies have made the heart of development natively into their technology stack – increasing the pace of development. AI is very much at the heart of what “Big Blue” is thinking here – with a closed-source approach and large client base in a relatively narrow set of fields (insurance, finance, etc). Power Usage Effectiveness (PUE) is a datacenter measure of Power Usage Effectiveness with 1 being perfect, 2 meaning 50% waste (formula: (n-1)/n) * 100). The (current) largest cloud provider, Amazon, for example, claims a rate of 1.15 in 2024, which means that roughly 13% of the power used is used for non-compute tasks. Google (at the time of writing, an industry leader) claims a 1.09 (8%). The average datacenter will clock in at an average of 1.57 (36%). IBM is thinking about efficiency, and there is a reason why 95% of Fortune 500 companies use IBM, and its because of its reliability and supportability, and a flexible pricing model. This can make some types of workloads effectively free.

Re-imagining the “Closed Source Advantage”

IBM has a large proprietary ecosystem, which they continue to invest heavily in. Watson AI, one could argue, could insulate IBM from future intellectual property lawsuits, poison pill attacks, and services leveling the playing field from the Open Source – AI Everything community. The collateral damage of strip-mining people’s intellectual property, conversations, emails, calendars, Google searches, Facebook posts, etc., has been summarized by “if you don’t pay for the product, you are the product”, but now this is getting more impactful, with targeted scams from public content and AI-generated attacks. The consequences are adding up, including several companies targeting AI compliance and accountability, which (depending on the eventual court rulings) could spell trouble for the “Open Source Everything” movement (Redis found out the risks of Open Source in a different debacle in 2024).

IBM is at the heart of closed-source proprietary solutions, a strong ecosystem, with a premium in PUE, and significant up-front capital costs. This may make more sense as more and more companies do a “Colocation” strategy expanding linearly since 2018.

The Gap

The mainframe is not going anywhere; companies using mainframes are still reporting positive perception of the mainframe (BMC). To realize the growth and take advantage of closed source proprietary approach, IBM needs to minimize the learning curve for developing applications over traditional x86, increase its value proposition in dealing with security and GenAI integration, and, most importantly, continue to open a clean, easy-to-access talent pipeline for onboarding to Mainframe (COBOL, PL/I, etc) programming, and build an ecosystem of developers that support each other similar to the Python, JavaScript, and DotNet communities.