Introduction

Welcome to our first product update of the year and also the first time we are doing this publicly. Yes you heard it right, from now on, we will share our product updates publicly through our blog along with email and in our Slack channel.

We know we have been a bit quiet since the start of 2023. Since, coming out of stealth after our Product Hunt launch in early December, we have been heads down building and making sure every new customer on Paradime got the best experience.

There is just so much we have shipped in the last 60 days that it's unbelievable what the team has accomplished in such a short period of time.

So let’s get started 🤩🚀.

SQL Query

Workbench table UX

The results table in the SQL workbench used to be clunky and slow. It could not handle more than a 1000 rows and if you had more than 3 or 4 tables, the whole app would grind to a halt. The new table experience can now load 10k rows and columns with no degradation in performance making the workbench usable for large datasets.

Limit 100 for SQL queries

Querying a table appends a default `limit 100` to the end of the query so that an user cannot accidentally query the whole table. This is a great protection to have for large tables.

Code IDE

Paradime's IDE for dbt is the best development environment for dbt development. In this release we are adding some pretty cool usability features.

Inline lineage and docs for CSV seed files

Inline lineage for CSV seed files adds visibility which dbt models depend on them while you code. Check also inline docs to quickly check the columns in the CSV files.

Compiled SQL in data preview

When modelling in dbt we have seen users need to preview data but when things go wrong they need to check the compiled SQL to fix issues. In the Code IDE, now users can flip between compiled SQL and data preview and see dbt-SQL, compiled SQL side by side.

Run toolbar

Nobody likes typing long model names in the terminal. So we have introduced the shortcut toolbar from where you can launch dbt commands on the current selected file in the editor. You are also able to launch multiple commands at the same time in multiple terminal tabs - users can run as many models they want in parallel during development.

The toolbar also has a nice new shortcut to open a new terminal 🥳

Diff in git considers white-space

The git-system within the Code IDE picks up <space> differences between files.

Quick access - create files and folders

Integrations

BigQuery OAuth & Execution project

Teams that care about security and want to move away from using service-accounts to connect to their BigQuery instance, can now take advantage of BigQuery OAuth to provide granular permissions on a per user level. This feature is available across all Paradime pricing tiers.

For teams who want to centralise their BigQuery cost and billing to a different project, we have also added support for the Execution Project configuration in Paradime. This is particularly useful, if you have multiple BigQuery projects across analytics and software development but you want to centralize billing in a single project.

Databricks connection support

Snowflake storage integration

We have added support for Snowflake storage integration so that customers can automatically ingest the latest manifest.json, of a dbt production run, from our S3 bucket into a table in the customer’s Snowflake instance. Finally you can query the manifest.json back from Snowflake within Paradime Query SQL workbench creating a closed loop while the customer has control of their data.

There are lots of interesting use-cases of this approach like powering real-time dashboards of various different dbt schedules in any BI tool powered by Snowflake tables. Use the table of manifest.json as a Snowflake source table and write dbt models to transform the manifest.json itself across runs into aggregate metrics for monitoring and reporting. The use-cases here are limitless.

Bolt Scheduler

Logs and Artefacts

For every dbt run, we now expose in the UI run logs, debug logs, dbt artefacts like run SQL files, compiled SQL, manifest.json - pretty much everything that is generated on the back of a dbt run. The console logs are streamed as soon as the run starts so you can follow the dbt run execution in real time. Once the run completes, you can view the debug logs as well for additional troubleshooting.

Cron config in Bolt is human readable

Trigger Hightouch Syncs

Bolt now natively supports triggering Hightouch syncs upon completion of a Bolt schedule. Specify the Hightouch slugs to sync on, the condition to sync on and then we will handle the rest. This also means much deeper Hightouch integration is coming soon - so stay tuned 😀

‍Support "defer" for production runs

Bolt schedules support specifying a deferred config so production schedules can take advantage of deferred states and only compile and build models that have changed instead of all the models thereby saving time and compute cost.

Cancel running jobs

Platform

Tutorials

Introducing our new tutorials section in the help docs where we have added a number of clickable interactive demos of how to get started with Paradime. We are planning to expand this section now with more and more guides, tutorials and how-tos to help users when they get stuck on Paradime or when they are coming from other tools.

Looking back, that looks like a crazy amount of things that we shipped in a very short period of time. We will be publishing more complete docs in the coming days with detailed explanation of various Paradime features.

We hope you will benefit from this latest product release. If you have any questions, or we can help in any way, please let us know. In the meantime, you can join our public beta too 🚀

‍