Afterthoughts on OpenAI DevDay
OpenAI ambition and the impact on enterprises and startups
OpenAI DevDay has unleashed a whirlwind of news and excitement across the industry. OpenAI is now hailed as the App Store of AI with the GPTs, and the introduction of new APIs for developers has further fueled the enthusiasm. The event is nothing short of impressive; indeed, it stands as one of the most significant product launches in tech.
Numerous recaps and discussions have proliferated across the Internet, and you can read Michael Spencer's and Latent Space's insights for a more comprehensive recap. Here, I aim to put together some thoughts and reflections on the event.
OpenAI can play (almost) anywhere with scale economics
Dev Day was likened to the 'Red Wedding' for AI startups, disrupted by OpenAI’s product roadmap. It lived up to expectations, showcasing OpenAI’s ambition to penetrate every corner of the ecosystem, often with a significant price advantage.
In terms of the core product, GPT-4 Turbo has become more accessible (with a 2-3x price drop from the previous GPT-4) and more powerful (faster with longer context length). This new pricing not only impacts companies such as Anthropic and Cohere but also erodes the cost advantage of self-hosted, smaller models, positioning OpenAI models as the go-to choice for enterprises. Even in new modalities like GPT-Vision and Text to Speech (TTS), OpenAI often outperforms and outprices competitors (e.g., TTS feature is 3-10x cheaper than TTS startups such as PlayHT and ElevenLabs).
Notably, OpenAI has introduced additional developer tooling features like native retrieval for RAG or threads for memory, traditionally the territory of LangChain and LlamaIndex. While the current offerings do not threaten these players due to the black box nature (unable to customize chunking methodology and data access controls), they could very well be further developed by OpenAI.
GPTs empower everyone to build and use AI … but it is not a breakthrough in agent development
The GPT Store, with its agent-like capabilities and promise of shared revenue, has garnered significant attention in the industry. Ethan Mollick’s post describes GPTs as the easiest way to create and share structured prompts that can automate tasks and processes, while Charlie Guo has shared a tutorial on how to build custom GPTs from scratch.
The potential of GPTs is immense - in particular on their integration with external tools and services (e.g., using Zapier to connect data sources and run data analysis). However, GPTs do not address the core bottlenecks in agent development, as reasoning robustness is still constrained by the underlying model (ChatGPT-4 Turbo can handle tool switching better but is still far from perfect).
Granting users free rein to customize GPTs may introduce new risks; prompt injection techniques could lead to interactions with malicious code and unintended actions. While GPTs now require user acknowledgment for their actions to proceed, it is crucial to implement better guardrails for widespread experimentation.
OpenAI as the clear leader in the enterprise AI landscape
The announcement of the Custom Models offering, where enterprises pay OpenAI $2-3M to help build a defensible business, has come as a slight surprise. These projects often require significant human capital, mainly OpenAI engineers, but may indicate a strategic push toward enterprise solutions, especially if corporate sales face pressure with customers gravitating toward more cost-effective alternatives.
Moreover, alongside Custom Models, I believe the DevDay upgrades indirectly impact enterprise AI implementation:
OpenAI will continue to serve as the go-to platform for early use case exploration, thanks to its performance and ease of use. However, the price reduction potentially makes it more economical than self-hosted models with limited usage, even after the validation phase.
Modularity is crucial in GenAI architectural design, given the rapidly changing AI landscape. Enterprises are increasingly hesitant to be locked into specific platforms or systems, spanning from LLM vendors to frameworks and databases.
GPTs still have a long way to go before achieving enterprise adoption. LLM products delivering reliable agent use cases and ensuring secure integration with enterprise systems are likely to emerge as the preferred choice.
There is still lots of room for startups to build
The DevDay clarifies OpenAI’s product roadmap in the short to medium term. Despite advancements across multiple areas, there is still whitespace and, often, a more substantial opportunity to build on top of existing capabilities.
Vector database and memory: OpenAI’s 128k context and Assistant API built-in threads and retrieval would likely become the default for developers building small-scale use cases. However, any serious use case will still demand scalable vector DB (e.g., Pinecone, Chroma), customizable RAG solutions (e.g., Langchain, LlamaIndex), or memory management (e.g., MemGPT).
AI agents: While GPTs lower the barrier to building agents at the current stage, vertical agents with proprietary data and deep domain knowledge will continue to maintain their competitive edge (e.g., Moveworks with expertise in enterprise support). In the longer term, robust agent-specific tooling or an underlying reasoning model (e.g., Imbue) remains necessary to advance the space.
Multi-modalities: GPT-4 Vision presents a promising space for startups to build applications on top of the API. The capability of GPT-4 Vision to interact with UI elements like web browsers or coding interfaces, and even in diverse physical environments for robotics, has the potential to be truly game-changing.
Even if a product aligns with OpenAI’s roadmap, gaining traction is not impossible with a better offering. For example, the data analysis tool Julius AI attracted 100k users despite launching after Code Interpreter.
Final thoughts
Overall, DevDay was truly impressive, not only in technological advancement but also in how it democratizes access to users through the GPT Store. By experimenting, we can uncover the potential and limitations of AI.