Skip to main content

Tag: Snowflake

Azure Snowflake Data Lake with Salesforce Data in a Few Clicks

Posted on 2023-05-14 - 8:11 pm by
Snowflake data lake powered by Azure Data Lake

In this video tutorial we build a Snowflake Data lake filled with Salesforce data using the power of Azure Data Factory. We built an ADF template that uses a few simple prompts to power a meta-data driven pipeline.

New Salesforce to Snowflake data pipeline template

View complete documentation

View more Tutorials

Download our Azure DataFactory Template Now

Name(Required)
This field is for validation purposes and should be left unchanged.

MVP Released in 2023

Learn more about the origins of our 5 Minute Snowflake data lake concept, where we automated our entire Salesforce data extraction process with a meta data driven approach. We used our own MVP to work with clients and refine our ELT solution with clients that invest in Azure for cloud infrastructure, Salesforce for sales, service, and operations, and Snowflake as a data platform for analytics.

Snowflake Warehouse Management with ROI in Mind

Posted on 2023-05-02 - 12:54 pm by
Snowflake Cost Management

If you’re new to Snowflake, you might be confused by the term “Warehouse”. Don’t let it fool you, because in Snowflake’s context, Warehouse refers to virtual compute resources rather than a physical storage place. Snowflake Warehouse management for small BI and analytics teams is fairly straight forward if you start off on the right foot.

A majority of Snowflake’s cost is based on warehouse (compute) utilization. Therefore, it’s crucial to be thoughtful about how you design and deploy your Warehouses to optimize your usage and minimize your cost.

Segmentation of Warehouses

One of the key factors in optimizing your Snowflake Warehouse is segmentation by use case and spend categorization. For instance, our Snowflake instance currently consists of 5 warehouses, with each one serving a specific purpose. We started with X-Small or Small instances that can process thousands up to tens of millions of records, and gradually scaled up as needed.

However, over-segmenting and creating too many warehouses is not recommended. This can lead to unnecessary concurrent warehouse instances and significantly increase your spend. Additionally, detailed spend tracking can become very expensive and difficult to manage. Therefore, it’s important to strike a balance between segmentation and cost optimization to achieve the best outcome for your Snowflake usage.

Warehouse Segments and Lessons Learned…

Read on Medium

Read more on our Medium Blog

How my Snowflake Powered Lead Distro Test Turned Out to be Reverse ETL

Posted on 2023-04-03 - 11:41 pm by
Snowflake Cloud Data Pipelines for Reverse ETL

A year ago, I worked on a small project to help us improve our data driven funnel. I learned what I called “Snowflake to Salesforce analytics sync” had a more buzzworthy term called “Reverse ETL.” This article shares some of the lessons learned along the way and some thoughts about where reverse ETL is headed.

Low Level of Effort Solution

All of the data and metrics were already available and calculated in Snowflake for reporting, so the process to push those measurements back into a Salesforce object using Azure Data Factory was quite simple.

The transformation work was prepared using Datameer on top of Snowflake which I had previously written about: Slice Through your Snowflake Data like a Buzzsaw with Datameer

Read more on Medium Blog

Creating Snowflake UDFs with ChatGPT: A Guide for Analysts

Posted on 2023-02-12 - 1:31 am by
Chat GPT Snowflake UDF Developer Chat GPT Bot

As data analysts, we often find ourselves needing specialized functions in Snowflake. Working in Financial services, there are specific Excel functions that provide significant value. I had a need but developer resources were not readily available… Until ChatGPT changed everything!

Now, with the help of ChatGPT even non-developers can prototype and experiment and contribute powerful capabilities. For Snowflake User Defined Functions (UDFs) in particular, ChatGPT is a game changing resource for self paced learning, debugging, and translating existing concepts and patterns you know into Snowflake.

Read more on Medium Blog