Category: Engineering

Databricks announced the public preview of Mosaic AI  Agent Framework & Agent Evaluation alongside our Generative AI Cookbook at the Data + AI Summit 2024.  These tools are designed to help developers build and deploy high-quality Agentic and Retrieval Augmented Generation (RAG) applications within the Databricks Data Intelligence Platform.   Challenges with building high-quality Generative AI […]

Enhancing DLT development experience is a core focus because it directly impacts the efficiency and satisfaction of developers building data pipelines with DLT. We are excited to announce some enhancements to the DLT development experience with notebooks. These new features provide a seamless and intuitive DLT development interface, and help you build and debug your […]

eBay‘s Christine Su has landed on Fast Company’s annual Queer 50 list for a second consecutive year, the media brand announced on Tuesday. The list celebrates “LGBTQ leaders who are leading across industries, including business, tech, finance, venture capital, and media,” wrote Fast Company Senior Editor Julie Herbst.  In her role as Head of Product, […]

Today, we are excited to announce Databricks LakeFlow, a new solution that contains everything you need to build and operate production data pipelines. It includes new native, highly scalable connectors for databases including MySQL, Postgres, SQL Server and Oracle and enterprise applications like Salesforce, Microsoft Dynamics, NetSuite, Workday, ServiceNow and Google Analytics. Users can transform […]

Over the last year, we have seen a surge of commercial and open-source foundation models showing strong reasoning abilities on general knowledge tasks. While general models are an important building block, production AI applications often employ Compound AI Systems, which leverage multiple components such as tuned models, retrieval, tool use, and reasoning agents. AI systems […]

Delta Lake has over 20M+ monthly downloads. BigQuery, now with first-party support for Delta Lake, builds on Delta’s rich connector ecosystem and seamlessly integrates with Databricks. In this blog, we will cover:   Delta Lake on Google Cloud Building an open data lakehouse with Databricks and BigQuery How to read Delta Lake in BigQuery Delta […]

Profile-guided optimisation (PGO) is a technique where CPU profile data for an application is collected and fed back into the next compiler build of Go application. The compiler then uses this CPU profile data to optimise the performance of that build by around 2-14% currently (future releases could likely improve this figure further). High level view of how […]

Looking to give your listings an instant glow up? eBay is making it easier than ever with an AI-powered background enhancement tool that helps sellers seamlessly transform their product photos from everyday snapshots into professional grade imagery.  Background enhancement lets you remove a less-than-glamorous background from an existing photo using eBay’s background removal tool, leaving […]

We are excited to announce a new data type called variant for semi-structured data. Variant provides an order of magnitude performance improvements compared with storing these data as JSON strings, while maintaining the flexibility for supporting highly nested and evolving schema. Working with semi-structured data has long been a foundational capability of the Lakehouse. Endpoint […]

Delta Lake has proven to be the most popular and fastest lakehouse format over the years. Delta Lake Universal Format (UniForm), now available in GA, builds on Delta Lake’s rich connector ecosystem to combine Delta Lake’s superior price-performance with access to every tool in your stack. With Delta Lake UniForm, you can write a single […]