Key Learnings from AI Engineer Summit 2025
I was at the AI Engineer Summit 2025 in New York, where executives, engineers, and product leaders were focused on one thing: AI agents.
Companies are moving fast to adopt AI - but beneath the excitement, questions remain. Quick wins are easy - lasting success is not.
Here are my six key takeaways from the conference - I’ll dive deeper into each one with specific examples, trends, and solutions in my upcoming newsletters.
#1 The AI gold rush is real — but it also backfires
There’s a massive push from boards and executives to adopt AI — and they want it done yesterday. According to a recent survey, 68% of executives plan to invest up to $250 million in AI this year (Forbes). That’s not even counting the massive AI infrastructure projects where companies are committing hundreds of billions of dollars.
But here’s the problem: while leadership demands quick action, internal teams often lack the expertise or structure to deliver. So, companies lean heavily on consultants who promise quick wins and deliver promising MVPs. But when it’s time to move from prototype to production, things start to get more complicated.
➡️ The leap from an 80% working MVP to a reliable, production-grade solution is huge - and most teams aren’t ready for that challenge.
➡️ If mishandled, this gap could lead to wasted resources, reputational damage, and growing skepticism about AI’s true potential. We’re seeing it already.
#2 Employees are skeptical — and fearful
Despite the top-down push for AI, many employees remain resistant. A Fortune 500 company with over 30,000 developers told me that only 50–100 people were actively involved in AI projects or knowledge-sharing sessions.
Why the resistance?
Some see AI as overhyped - another tech bubble waiting to pop.
Others fear AI will replace their jobs and make their skills obsolete.
But history suggests that AI is more likely to create opportunities than reduce them. When ATMs were introduced, many feared they would eliminate bank teller jobs - instead, banking employment grew as services expanded. AI could have a similar effect, making coding more accessible and increasing the demand for creative, strategic work.
I can’t think of any tech revolution that has reduced the overall number of jobs. Each time, we end up needing more people - the real challenge is the period of re-skilling.
#3 Data curation and evaluation are everything
AI models are only as good as the data they’re trained on - and data curation remains one of the most overlooked aspects of AI development.
➡️ Small errors scale fast. A minor data quality issue can ripple through the system, causing AI-generated insights or actions to fail spectacularly.
➡️ AI systems need continuous feedback loops and structured evaluation at every stage to improve over time.
➡️ Building a scalable, reliable AI system means putting data quality at the center - not treating it as an afterthought.
#4 AI Engineers are actually Reliability Engineers
LLMs are incredibly capable - but highly unreliable. AI engineers' job isn’t just to build models - it’s to make them consistent and predictable in real-world use cases.
➡️ Building a capable AI model is easy. Making it reliable at scale is hard.
➡️ AI engineers need to design systems that anticipate failure - and build in mechanisms to catch and correct issues in real time.
➡️ The difference between a successful AI rollout and a costly failure? A focus on reliability, not just capability.
#5: AI is becoming an active user of your product
AI isn’t just a backend tool anymore — it’s becoming an active user of your product. Businesses need to rethink not just how humans interact with their products, but how AI agents do as well.
➡️ Make it readable. Websites should have structured markdown files that are easy for AI to process - similar to how search engines crawl pages.
➡️ Design for AI. APIs should be built with AI agents in mind, using emerging standards like Model Context Protocol (MCP).
➡️ Search is changing. Google is already testing AI-generated search responses - essentially turning search into a dynamic, AI-generated interface.
#6: UX remains the missing link
AI still lacks intuitive user interfaces. Most AI-driven products are designed for engineers - not end-users.
➡️ AI needs a new generation of UX and UI thinking - interfaces that make AI capabilities intuitive, accessible, and actionable for everyday users.
➡️ The next wave of AI innovation will come not from better models - but from better interfaces.
If AI-generated search results and adaptable AI-powered products become the norm, UX will become the critical differentiator.
Interested in learning more? Over the next few weeks, I’ll break down each of these insights with practical solutions, best practices, and real-world examples. Hit subscribe and stay tuned.