Stepping up marketing for advertisers: Scalable lookalike audience

The advertising industry is constantly evolving, driven by advancements in technology and changes in consumer behaviour. One of the key challenges in this industry is reaching the right audience, reaching people who are most likely to be interested in your product or service. This is where the concept of a lookalike audience comes into play. […]

Continue Reading

Apache Spark 3 Apache DataSketches: New Sketch-Based Approximate Distinct Counting

Introduction In this blog post, we’ll explore a set of advanced SQL functions available within Apache Spark that leverage the HyperLogLog algorithm, enabling you to count unique values, merge sketches, and estimate distinct counts with precision and efficiency. These implementations use the Apache Datasketches library for consistency with the open source community and easy integration […]

Continue Reading

Introducing the Support of Lateral Column Alias

We are thrilled to introduce the support of a new SQL feature in Apache Spark and Databricks: Lateral Column Alias (LCA). This feature simplifies complex SQL queries by allowing users to reuse an expression specified earlier in the same SELECT list, eliminating the need to use nested subqueries and Common Table Expressions (CTEs) in many […]

Continue Reading

Introducing Apache Spark™ 3.5 | Databricks Blog

Today, we are happy to announce the availability of Apache Spark™ 3.5 on Databricks as part of Databricks Runtime 14.0. We extend our sincere appreciation to the Apache Spark community for their invaluable contributions to the Spark 3.5 release. Aligned with our mission to make Spark more accessible, versatile, and efficient than ever before, this […]

Continue Reading

Best Practices for LLM Evaluation of RAG Applications

Chatbots are the most widely adopted use case for leveraging the powerful chat and reasoning capabilities of large language models (LLM). The retrieval augmented generation (RAG) architecture is quickly becoming the industry standard for developing chatbots because it combines the benefits of a knowledge base (via a vector store) and generative models (e.g. GPT-3.5 and […]

Continue Reading

Introducing MLflow 2.7 with new LLMOps capabilities

As part of MLflow 2’s support for LLMOps, we are excited to introduce the latest updates to support prompt engineering in MLflow 2.7.  Assess LLM project viability with an interactive prompt interfacePrompt engineering is a great way to quickly assess if a use case can be solved with a large language model (LLM). With the […]

Continue Reading

Q&A With Dr. Anna Zeiter, Chief Privacy Officer, Associate General Counsel for Privacy, Data & AI

Artificial Intelligence (AI) has evolved and improved dramatically over the past 18 months, and the technology has myriad applications and amazing potential for the future of ecommerce. But with such lightning-quick advancements, it’s vital that we in the technology industry, especially companies like eBay with the size, scope, and personnel to guide the future of […]

Continue Reading

‘Magical’ Listing Tool Harnesses the Power of AI to Make Selling on eBay Faster, Easier, and More Accurate

We’ve been hard at work on the next version of a new, magical listing experience, which uses AI to analyze, research, and extrapolate information from a small amount of data provided by the seller — greatly simplifying the process of listing an item on eBay. In the first version, a seller could add in a […]

Continue Reading

Shared Clusters in Unity Catalog for the win: Introducing Cluster Libraries, Python UDFs, Scala, Machine Learning and more

We are thrilled to announce that you can run even more workloads on Databricks’ highly efficient multi-user clusters thanks to new security and governance features in Unity Catalog Data teams can now develop and run SQL, Python and Scala workloads securely on shared compute resources. With that, Databricks is the only platform in the industry […]

Continue Reading

What’s New for Shared Clusters in Unity Catalog

We are thrilled to announce great enhancements to onboard more workloads to Unity Catalog clusters in shared access mode, Databricks’ highly efficient, secure multi-user clusters. Data teams can now develop and run SQL, Python and Scala workloads securely on shared compute resources. With that, Databricks is the only platform in the industry offering fine-grained access […]

Continue Reading