What Is DBT and Why Is It So Popular - Intro To Data Infrastructure Part 3
124,300
Published 2022-04-15
Built by Fishtown Analytics (now dbt Labs), the data build tool or dbt allows data analysts, data engineers and analytics engineers to execute the “transform“ step in the Extract-Load-Transform pipeline.
They can do this by writing transforms in SQL and executes it against its company’s database.
Dubbed as the “analytics engineering tool,” dbt is also open source and has become a regular part of the modern data stack of many companies. It also has had a solid support community formed around it.
Looking to start you're own data engineering/analytics consulting company, then you should check out my new course here
courses.technicalfreelanceracademy.com/courses/sta… - and use the coupon code "deconsult" to get 50% off
If you enjoyed this video, check out some of my other top videos.
Top Courses To Become A Data Engineer In 2022
• Top Courses To Become A Data Engineer...
What Is The Modern Data Stack - Intro To Data Infrastructure Part 1
• What Is The Modern Data Stack - Intro...
If you'd like to read up on my updates about the data field, then you can sign up for our newsletter here.
seattledataguy.substack.com/
Or check out my blog
www.theseattledataguy.com/
And if you want to support the channel, then you can become a paid member of my newsletter
seattledataguy.substack.com/subscribe
Tags: Data engineering projects, Data engineer project ideas, data project sources, data analytics project sources, data project portfolio
_____________________________________________________________
Subscribe: / @seattledataguy
_____________________________________________________________
About me:
I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data.
*I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.
All Comments (21)
-
If you guys want to learn more about data engineering, then sign up for my newsletter here seattledataguy.substack.com/ or join the discord here discord.gg/2yRJq7Eg3k
-
Still don't get what is dbt
-
dbt is a great tool. I would really love to see a video from you comparing Prefect, Dagster, and Airflow and how they fit into modern data stack.
-
DBT was a game change for our tableau workbooks. Our tableau workbooks were taking sometimes up to 1-2 minutes to fully load. Once our data engineers added DBT to the mix, the tableau workbooks were taking 3-5 seconds to load. I am not fully sure how all this was done by our data engineers but it was straight up voodoo magic and our end users loved it. Bonus is it made us look good to our 3rd partners who said even their own data team's reports were taking minutes to fully load.
-
Loving your channel and this video. I had a similar start with data pipelines and to me dbt feels like somebody took the frustrations I had with those GUI ETL tools and packaged it up into a better, modern product. Writing code is more fun anyway (IMO). Keep up the great vids!
-
literally just had an interview yesterday where they asked me if i have experience with dbt and i said idk what that is lol
-
Great video as always Ben! Thank you for consistently providing us with value.
-
We use airflow + DBT in most of our projects now, it's such a good tool
-
I’m a new Data Engineer and I’m scared to ask what is “Jinja” 😅
-
If you want to actually know what's DBT, fast forward to 4:20. But, if you already know a little bit DBT and just want to find more advance information, you can skip this whole video.
-
I always struggle remembering that DBT is not only acronym for Dialectical Behavior Therapy
-
Wholly goodness thank you. I've been learning dbt as a Software Engineer and have been trying to figure out why it matters at all. This video helped a ton!
-
dbt was a breath of fresh air from ssis. I still think ssis was great for its time and still a usable tool in some places. sometimes I miss it... sometimes.
-
My question is at this point are there TOO Many tools? What do you see in the market; companies using python frameworks (SQLAlchemy, airflow etc) or no code tools like DBT(not considering SQL) more? Edit: i meant SSIS, informática not DBT
-
dude how r you, i have the next question, what could i do if i have a stream on snowflake that i want to "consume" in dbt but not creating a physical table or view, instead something live a ephemeral materialization, only to purge the stream and avoid to become stale. I create an ephemeral model and select the stream source but that only create obviously an ephemeral materialization but kind not clean the data on the stream, thoughts??
-
I can correlate with your journey. Started off with using DTS in SQL 2000 and using SSIS first time with SQL server 2005 :)
-
After watching this video, it is still not clear to me what exactly dbt is doing. Examples would have been great.
-
How would dbt replace adf when there to E AND L in dbt? Who’s gonna extract and load data into the warehouse for dbt to be able to transform it?
-
I'd say SQL is a Pro. If you've worked in BI for any length of time then you know some SQL, so adopted DBT will be very smooth.
-
Thanks! I have been sleeping under a rock for two years - appreciate you helping me get up to speed 🙂
