dbt | Summary of Coalesce 2022

The dbt Coalesce 2022 conference took place last week in New Orleans, USA - with one-day conferences in London and Sydney. Throughout the week there were high-quality presentations and a lively chat forum on dbt's Slack channel where attendees from around the world discussed the talks, united by their love for data.

dbt has gained increasing popularity among developers in recent years. There are several reasons for this - among other things, dbt enables efficient and scalable data modelling where developers and other stakeholders have an overview of what data products exist and how they are defined with business logic and data sources. The days of complex SSIS jobs with no overview of sources, business logic and potential schema changes are possibly over for many who adopt dbt.

During Coalesce 2022, two exciting products were launched in dbt Core (open source) and dbt Cloud (dbt hosted in the cloud by dbt Labs).

Semantic Layer launched in Public Preview for dbt Cloud customers

With the launch of Semantic Layer, dbt’s service portfolio expands with a semantic layer on top of the data models, allowing you to make defined metrics and KPIs available across the organisation and ensure that everyone uses the same definitions and data foundation.

In light of this launch, it will be interesting to see how dbt Labs positions the product in the market compared to competing services looking to capture customers on their platform, of which Looker’s solution Looker Universal Semantic Model is an example.

For the new product Semantic Layer to succeed, dbt Labs depends on expanding its integration portfolio.

Semantic Layer has for now been made available in dbt Cloud in a Public Preview period.

dbt Core expanded with Python support

The expansion with Python support in dbt Core allows developers to write in Python to develop dbt models for data transformation.

Finally, you have the ability to develop data pipelines by mixing SQL and Python - which fits very well in a “lakehouse” setting where you can utilise the same data sources and land data in the same area as where you have a data warehouse.

This provides great value in situations where you can, for example, use existing Python packages to reach the goal faster with fewer lines of code compared to SQL. What you’ve coded in Python is compiled by dbt and executed in the data platform (as of today, this is supported in Databricks, BigQuery and Snowflake).

dbt Cloud gets a new user interface

dbt Cloud is getting a new UI with significant changes during October and November 2022. This has been in preview for a while, but is now becoming standard for everyone.

• Several upgrades make the user interface faster (this had been something causing friction in the use of dbt Cloud’s UI).
• Upgraded visualisations for run time, errors and other key metrics of data pipelines.
• Launch of several services promoting developer productivity, including built-in SQL formatting and “git diff view” - to see what has changed in your files before making a pull request (coming November 2022).

Coalesce 2023

We’re looking forward to next year’s Coalesce, which will be held in San Diego, and it will be exciting to see which direction dbt Labs wants to take dbt.

If you’re considering getting started with dbt, or you’re wondering about the implications of these launches for your dbt implementation - please get in touch for a friendly chat.