Thanks for a great article! I do a similar thing with Dagster. Here is an article

https://zsvoboda.medium.com/orchestrating-dbt-and-pyspark-7cdcf66dce71

with a demo code

https://github.com/zsvoboda/ngods-stocks/tree/main/projects/dagster

in case you are interested.