AI & Big Data Expo 2025. Inside the Industry’s Seismic Shift

The AI and Big Data Expo 2025 Conference in Santa Clara, CA, has become an annual highlight, and my third attendance this year did not disappoint. It was inspiring to see such a wide range of professionals - from AI researchers to software engineers and business leaders - all eager to absorb insights from top-tier speakers representing the biggest names in tech. 

As with previous years, I've distilled my observations to offer you a snapshot of the key developments and discussions that have unfolded over the past twelve months in the AI universe.

The industry is currently defined by several dominant trends: 

1. A clear shift is underway from basic AI chatbots to more advanced, autonomous AI agents. The shift opens new possibilities while simultaneously creating challenges in accountability and risk. 
2. We're also seeing a continuous spike in new platforms, products, and tools aimed at maximizing AI's capabilities. Software development practices are still adapting, with AI automation changing the very nature of a developer's work and experience. 

Furthermore, security issues are growing more urgent as AI integration triggers new debates around data protection and ethical use. In investments, the focus has narrowed to truly innovative "AI-first" startups (finally!). 

But let’s now unpack each of these trends.

From Chatbots to Self-Driving Code. Autonomous AI Agents Take Center Stage

As already mentioned, a central theme this year was the significant evolution from AI chatbots to autonomous AI agents (see Fig. 1). Companies are actively experimenting with these approaches to automate complex processes.

Figure 1. Autonomous AI & autonomous agents market

Responding to this demand, AWS even introduced new features in Amazon Bedrock, simplifying the collaboration of multiple agents. Bedrock helps developers to build, deploy, and manage specialized agents that work together. Moreover, these agents' ability to generate and execute code in a secure environment makes tackling analytical queries or complex mathematical problems (which require a deterministic approach) far more efficient now.

However, this rapid advancement comes with new/old challenges. In particular, building an efficient AI agent is not a piece of cake - some recent studies show that AI agents get office tasks wrong around 70% of the time. Striking the right balance between granting AI agents autonomy and managing their inherent risks remains a hurdle. Especially when it comes to proper authorizations and sensitive data. 

It is clear that without continuous monitoring and quick issue detection/resolution, these powerful technologies could surprisingly create more problems than they solve, for example: 

• Legal questions surrounding agent accountability, determining who is responsible for an agent's actions, are still largely unresolved. 
• Heavy agent reliance on available APIs and up-to-date documentation also presents ongoing barriers. 

In my opinion, all these indicators suggest we're at a transition point for even greater developments in this domain. It is a trend we'll further explore in the platforms and tools discussed next.

Next-Gen Platforms and Tooling. The AI Builder’s New Arsenal

Headlining the conference, IBM promoted its AI Gateway and API Assistant. 

• The AI Gateway acts as a singular, secure hub for accessing AI services via APIs. The platform connects various applications with third-party AI systems.
• The API Assistant is designed to simplify crucial API tasks, from documentation and discovery to governance, usage, and even error fixing. It is an important feature given the dependency of AI agents on reliable APIs. 

NB: IBM also impressed with a demo of Instana. Their application performance monitoring platform simplifies the oversight and optimization of AI operations!

On the AWS front, the recent debut of the SageMaker Unified Studio was an impactful moment. This evolution redefines well-known SageMaker into an extensive environment for data and AI development. It consolidates tools from core AWS Analytics and AI/ML services like Amazon EMR, Glue, Athena, Redshift, Bedrock, and SageMaker AI. The introduction of AWS S3 Tables last fall has complemented Unified Studio's usefulness, making it even more practical and integrated.

Within the open-source community, Apache Iceberg got significant attention as an open table format engineered for massive analytic datasets. Its compatibility with popular compute engines such as Spark, PrestoDB, and Impala allows for data management that's both efficient and intuitive, mirroring the simplicity of a standard SQL table. Considering the vast size of data generated daily, around 402 million terrabytes (402,000,000,000,000,000,000 bytes), and its imperative role in training AI models, Iceberg becomes a potentially invaluable asset.

Coding Redefined. How AI Automation Is Reshaping Software Careers

Now, let’s shift our focus to software development. Such a domain continues to undergo a profound transformation as AI tools proliferate. Developers are now routinely employing AI to achieve three key things:

• automate repetitive coding tasks;
• significantly boost speed;
• free time to concentrate on more complex problems. 

At the same time, the overall AI evolution fuels an ongoing industry discussion about the "replacement of coders by AI." Though full replacement still appears distant, some sources indicate that it’s highly unlikely, while others suggest full replacement within 15 years. The changing approach creates a valid concern: 

How will junior developers grow the deep experience and expertise necessary for senior roles if their coding practices are heavily reliant on AI assistance? Without a doubt, the demand for traditional, manual coding is receding while the demand for autonomous coding is growing significantly (see Fig. 2).