NVIDIA replaces Spark logic with GPU and AI to accelerate UDFs

NVIDIA is replacing the processing logic for UDFs in Spark with an architecture that integrates GPUs and AI agents. The new approach reduces latency and improves throughput in distributed applications. The solution depends on PCIe 5.0-compatible infrastructure.

NVIDIA presents a solution to accelerate Spark UDFs with GPUs and AI agents. The company is also developing solutions to accelerate data processes with GPUs and AI agents, responding to growing demand for efficiency and performance in enterprise applications. Intel, in turn, is launching the D270, an inference GPU optimized for the new era of agentic AI.

The D27, which offers up to 480 GB of LPDDR5X memory, 350 W of power, 32 Xᵉ3P cores and 256 XMX engines, features the GPT-6 Astra Ultrafast version, available in the OpenAI API and to eligible ChatGPT Work and Codex users. The version offers up to 8x faster token generation than Astra Standard mode. NVIDIA AI infrastructure helps OpenAI serve more useful responses. The solution was unveiled by NVIDIA and Intel, showing the sector's importance in production and SaaS environments.

The new approach reduces the complexity of managing data and networking, leaving more resources available for the UDFs' processing logic. Integration with the NVIDIA platform also expands the possibilities for scaling and optimization in distributed environments. NVIDIA and Intel are redefining how AI and data processing applications work with networking and data movement. NVIDIA presents a new approach to accelerating Spark UDFs using GPUs and AI agents, integrating cuDF technology and DOCA GPUNetIO. DOCA GPUNetIO allows GPU applications to manage networking and data movement as first-class operations, with support for existing infrastructure via PCIe 5.0.

Intel, in turn, is launching a GPU optimized for agentic AI, such as the D270, which supports up to 480 GB of LPDDR5X memory and is built to handle long contexts and complex workflows. Intel is also positioning itself in the sector with a new GPU that can scale agentic AI to thousands of steps and simultaneous sessions. Philippe Tillet, inference lead at OpenAI, says the solution gains speed when writing code and working with complex tools. The collaboration between the companies optimizes inference infrastructure for agentic AI.

NVIDIA is replacing the processing logic of UDFs in Spark with an architecture that integrates GPUs and AI agents. The cuDF technology allows enterprise applications to process more data per second, reducing processing time in massive transformation tasks. The solution was presented by NVIDIA and Intel and is available via OpenAI API and ChatGPT Work and Codex. Availability in Brazil has not been confirmed.

Text written with AI from the sources listed and checked by the newsroom's automated rules. How we work

• #hardware
• #software