Skip to main content

Author Archives: Ryan Goodman

 About Ryan Goodman

Ryan Goodman has been in the business of data and analytics for 20 years as a practitioner, executive, and technology entrepreneur. Ryan recently created DataTools Pro after 4 years working in small business lending as VP of Analytics and BI. There he implanted an analytics strategy and competency center for modern data stack, data sciences and governance. From his recent experiences as a customer and now running DataTools Pro full time, Ryan writes regularly for Salesforce Ben and Pact on the topics of Salesforce, Snowflake, analytics and AI.
Website Facebook Twitter LinkedIn GitHub

DataTools Pro AI BI Blueprint

Posted on by
DataTools Pro Delivery Blueprint: a six-step flow from selecting initiatives to building team and resources.

We have spent twenty years building analytics systems that people actually trust. The last three years taught us what happens when you put AI on top of a strong BI foundation. Most AI deployments fail before they start. Not because the technology is wrong. Because the data underneath it is ambiguous, inconsistent, and disconnected from how decisions are made. We have our DataTools Pro AI Blueprint to help fix it quickly.

DataTools Strategy and Initiative Planner

Select Initiatives with Impact: Deliver solutions that matter

We build models and tools that are aligned to high frequncy, 5 -7 figure decisions related to revenue and risk. There are plenty of opportunistic problems we solve along the way. However, without something high impact, it’s not worth embarking on the journey.

Grounding and Exploration

Step 1 is to ground the team based on what already exists and is understood at warp speed. Right or wrong, grounding ourselves on what exists is how we step forward to what’s next! No enterprise we work with is starting from scratch or scrapping what they have. We inherit what has been constructed and re-project and assemble that story so we can establish a “point of arrival” for AI / BI influenced decisions.

Metrics Glossary

At DataTools Pro we still see the best results with AI when business semantics are clearly defined together. That includes taxonomies, business and statistical definitions and meta data that helps point to organizational and sometimes situational awareness. Our patent pending metrics analyst for example automatically scrapes BI and business applications to inference some of this context so we come to the table with recommendations, not blank templates waiting to be filled in.

Data-marts for Business Outcomes and Adoption

Traditional EDWs were built to answer a wide range of questions because the cost, complexity and time required it to be that way.. In 2026 your Snowflake warehouse can be up, running, and staged with data securely in days. Our approach to modeling data is fast, focused, and business topic oriented.

Data Platform and Delivery

Data volume, velocity, variety shape numerous architecture design decisions. Data acquisition, movement, and management come in many different flavors and price points.

Snowflake as our preferred data platform and preference, only when it is the right fit. Databricks, Microsoft Fabric, GCP are all viable and powerful options based:

  • Team, Skills, and Resource
  • Existing invesetments and maturity
  • Systems of record and platforms
  • Budget

DataTools Pro Delivery is Fast and Information Dense

The delivery model for AI / BI consultants is something we are always refining with clients. DataTools embraces agile principles and has continous releases, though release notes are weekly. Documentation is very dense and requires recipients to host version control (Confluence, Google Drive, OneDrive) where AI is available to recipients.

Version ControlAI Feature Required
NotionClaude
ConfluenceClaude
GitHub – REQUIREDNot required if other
Office 365 (we commit txt docs)365 Colipot
Google DriveGEmeni for Google Workspaces

Boring but important nuts and bolts: Technology tools and process are designed to keep things simple. You can design a data platform architecture that costs $6000 a year or $60,000 a year that fundamentally achieve the same result. We choose open source and embedded tools that can operate with Snowflake like GitHub and DBT Core.

  • Cost – Un-necessary cost reenforces the stigma that Snowflake is expensive. Like any consumption based technology, spend should yield results.
  • Adoption – If your Snowflake instance is running, we target 60% Snowflake spend on data movement and transformation to 40% direct delivery of reporting and analytics. AI development and consumption changes utilization and economics.. We are still observing results across clients before we make recommendations.
  • Security – Snowflake employs role based security and delivers a comprehensive network security function to help track and manage least level of privilege. Data tagging is front and center and can be used to apply masking policies.

Putting AI BI Together

When you put the AI BI puzzle pieces together, this is one completed view of what a managed system looks like fro AI BI decision support. Contact us for a free consult

AI BI Concepts you Should Understand for Success

Posted on by
AI + Analytics infographic: central icon and flowing ribbons split into Monitor, Explore, Storytell with data stream on left and labels on right.

Business intelligence exits to support decisions and AI exists to replicate human intelligence through machine learning. AI BI magic happens when coupling the right people, prioritization, and processes around data, information management. AI + Analytics works, like any initiative when you have everyone on the team rowing in the same direction. This article was written to highlight some core concepts that help with alignment for AI BI initiatives.

AI BI

What BI Should Answer

Where are we? – Leading indicators (metrics and KPIs) summarize the current or previous state. These metrics and KPIs are calculated and summarized and mixed together based your the goals / objectives. These assets are designed for frequency and wide distribution. Dashboards for infrequent use by small numbers of users are reserved for executive and board-level users.

How did we get here? – Lagging indicators and trends over time and segments (dimensionality) help explain what is changing over time. The reality is trends and basic descriptive statistics explain “what” but not “why.”

Predict What “should happen next” – Basic statistical methods and machine learning algorithms have been paired with BI over the last 10 years.

What AI Should Answer?

What should we do to achieve our desired outcome? This has long been the promise of AI, long before ChatGPT: provide an objective, provide data and compute, and AI returns an answer. This is the domain where the competition to AI is gut, experience, and interpretation. Most data-driven organizations are still in the BI world, where data influences a human decision. Machine learning was reserved for high impact use cases because the cost, time, and expertise required made it so.

Last 15 Years = Data Driven AspirationNow = Agentic Data Driven Aspiration
human is the recipient of a data / analyticsAI agent is recipient of data / analytics
descriptive statistics or predictive score provides predictionsdescriptive statistics / predictive score provides predictions
reasoning, analysis, consensus occursAI reasoning interprets, iterates, recommendation are formulated
a decision is made and action occursAI or human decision and action occurs

AI models have evolved dramatically with the explosive adoption of LLMs. Industry analysts still report that most BI AI initiatives have a high failure rate. AI assisted analytics can be far more nuanced, so here is what you should look out for:

Why AI BI Initiatives Fail

  • Implementation of tech in search of a problem and problems without material impact
  • Lack of data governance, fidelity and as a result low trust
  • Mismatched expectations between “art of the possible” and reality organizational readiness
  • Knowledge management and metrics alignment

Nuanced Details that Matter for Planning AI BI

AI is going to touch every facet of how information is originated, delivered, and distributed inside an enterprise. Vendors like Databricks and Microsoft offer end-to-end platforms that facilitate the full BI and AI stack. Even the top AI companies in the world can’t solve your business / process challenges without consultants (they call them forward deployed engineers). No matter how well the tech tools are marketed, the devil’s in the details when you need to solving business problems with AI BI.

Deterministiv vs Probablis

Deterministic vs Probabilistic

Deterministic: same inputs plus same rules equals the same output every time.

Probabilistic: predicts the most likely output based on learned patterns and uncertainty.

When LLMs first arrived, they were unreliable at math because they predicted answers from patterns rather than calculating them. Today, advances in reasoning, tool use, and code execution allow LLMs to offload calculations to deterministic systems, which has dramatically improved performance on math and statistics tasks. The model is still probabilistic, but it increasingly orchestrates deterministic tools beneath it.

This is where we see AI BI and the need for business and data semantics being critical.

Monitoring, Exploring, Storytelling

Somewhere along the way, these three disciplines were blended into BI dashboards. With correct data and well-defined semantics, AI can be a very capable analyst for applying descriptive and predictive statistics across all three modes.

Monitoring is the dashboards, reports, and scheduled PowerPoints already circulating. AI is well-suited to producing these assets, and there is real innovation happening in this area across the AI BI landscape.

Exploring is where you ask questions business questions seeking answers using data. Excel is still the most widely used data exploration tool. Tableau is arguably second, unless you include Python and R. Exploration is also widely used for data wrangling tasks.

Storytelling is where disciplines overlap. Every PowerPoint is a curated story. The art and science of narrating a story with data is a special skill and AI quite effective at telling these stories.

Understand these 3 data presentation disciplines will help with tool selection because you need all 3.

Data Needs Meaning. Meaning Needs Context.

A semantic layer defines your metrics. Revenue = SUM(amount), every time, for everyone.

An ontology defines your business. Customer owns Opportunity. Opportunity belongs to a Segment. Segment drives Revenue.

BI tools need the first. AI agents need both.

As AI moves from predicting answers to orchestrating deterministic tools beneath them, the semantic layer becomes the calculation engine and the ontology becomes the reasoning and context map. Neither replaces the other. Together, they’re what separates governed AI analytics from a confident guess.

AI BI Governance

AI agents can generate insights, dashboards, and analytics artifacts faster than any business teams can digest them and technology teams can manage them. At 20-50 of anything, you reach an inflection point where managing them without structure becomes a new challenge and pain point.

  • Distribution and curation remains mostly unsolved. Search, discovery, and exploration of analytics assets has been a weak link in enterprise BI for a long time. Searching across chats, folders, repositories for the brilliant AI insight or artifact is causing enterprises a lot of pain. We are always hunting for better delivery experiences; for most clients, the answer is embedding analytics where users already work.
  • Data governance is the unglamorous but necessary structure that ensures AI initiatives can reach production with data that is correct, consistent, and appropriately scoped. A significant number of AI projects are stalling there. Access, trust, and security are are all bottlenecks.
  • Semantic disconnect is the other failure mode rarely discussed and something we understand intimately. It’s our specialty and focus at DataTools Pro. LLMs can be gradually misdirected over time through inconsistent business terminology. Those misdirected signals produce incorrect results that poison your results. Incorrect results erode trust. Data can be perfect and the semantic layer can be well-executed; the breakdown happens when business semantics shift and the model does not keep pace.
    Learn more AI BI Challenges

What does succesful AI BI look like?

Organizations that succeed with both BI and AI are are managing with some form of governance. Knowing what your data means, who owns it, and whether it can be trusted, and what AI vendors are doing with it is critical. Feel free to check out some of our recent learnings and solutions for AI BI or Contact Us so we can learn about your AI BI initiative.

Building a Snowflake Semantic Layer in in 2026

Posted on by
Diagram of Snowflake Semantic Layer architecture with icons for Analytics, BI Tools, AI/ML, and Applications above a layered platform labeled 'Semantic Layer' and 'Snowflake' logo on the base (informational image).

Snowflake built its roots and reputation as a power data warehouse. Working with large sums of structured data and semi structured data is a breeze in Snowflake. With the push toward AI BI, constructing Snowflake semantic layer makes sense, if you have the right process and governance structure in place.

LLMs and advancements in AI have shifted who and how data turns into actional information within a typical enterpise. Data storytelling, data exploration, and using descriptive statistics is in the hands of anyone with access to Claude. In fact our opinion is Claude has reached a level up to par with professional analysts and data scientists.

Semantics Originate with BI: Building Snowflake Semantic Views

No company is starting their semantic layer discovery and execution with a blank canvas. Tableau and Power BI operate more than a quarter of the entire Business Intelligence landscape. Business and data semantics converge within your BI platform, so it only makes sense to inherit metrics, filters and other critical inputs that truly define the last mile of business understanding. A semantic layer built purely on top of a database with no context to business delivery more likely to fail when it reaches AI / BI tools that ultimately consume data. Snowflake natively consumes published data and table relationships.

Snowflake Semantic Layer AI BI Import

Refining a Semantic Layer with Analyst –

This is the stage where you get out what you put in… Rarely if ever does the data warehouse, semantic layer, and labeling of BI reports / dashboards align. This is your opportunity to course correct and set yourself up for success, if you are using AI to either build BI assets or use AI directly to analyze your data.

Recommendations

  • Teams that treat the semantic layer as a database labeling exercise are setup to fail.
  • Do not connect AI directly to Snowflake semantic layers without validating existing BI use cases and results
  • Engage BI builders and analysts with domain expertise to test, validate and lend data context.
  • Do not blindly use “AI” to inference meaning and definitions if you do not know them.

This phase is where subject matter expertise is just as important as technical competency. Snowflake has taken the correct route to align queries to business questions. Rather than looking at code and meta data and and making up questions, Snowflake will at least scan real queries executed before it makes up questions. Most of them are basic, and that is by design.

The difference between a functional agent and a truly useful agent conceptually is the depth and richness of your semantic layer.

Verified Queries and Log Monitoring

When you build business intelligence, you have something to ground yourself during data validation and user acceptance testing. Snowflake has taken the correct route to align queries to business questions. Rather than looking at code and making up questions, Snowflake will scan your query history.

Inferencing Semantic Views Risky

Inference based semantics feels magical as a creator to establish depth and meaning behind your data. The idea of having “something is better than nothing” does not play out when it’s time for asking questions of your data. That can turn to semantic disconnects in your organization when AI observed meaning does not translate to business alignment and understanding. This is one of the AI / BI risks that we highlight and often cross train SMEs and analysts to get involved while building and administrating Snowflake semantic layers.

Adding depth with Tags

One of our original gripes with the Snowflake semantic layer was the lack of depth in meta data, which directly conflicts with the patterns we have found most successful with AI to remove ambiguity and deliver contextual depth needed to accurately translate business questions to the right metrics and attributes. Building one semantic view for a single topic makes for a great demo. When you have hundreds of marts across 5 lines of business, you need tags for reconciling semantic disconnects.

Activating your Snowflake Semantic Layer with AI

Once you have your Semantic layer built, you can connect directly from your BI platform, build Snowflake agents, or connect Claude to Snowflake directly. The benefit is the layer of context to help convert plain English questions to the underlying queries. This is where Snowflake’s “verified” queries and monitoring capabilities make a huge leap over direct AI to SQL and legacy BI reporting and dashboard semantic layers. We are working on formal eval results to highlight the lift in applying Snowflake Semantic layer and Snowflake Agents as an interface.

3 Tips for Planning your Snowflake Semantic Layer

  1. Snowflake Cortex Analyst is a an amazing feature. The architecture is sound and the direction is right. But the difference between a functional agent and a truly useful one is not the platform and tools. It is the depth and richness of the semantic layer behind it.
  2. Semantics do not originate in a database. They originate in the BI tools your business has relied on for years. Inheriting that context, validating it against real query history, and extending it with tags and analyst input is not optional work. It is the most important work to build production grade semantic models.
  3. The teams that treat this as a labeling exercise will get labeling results. The teams that bring domain expertise, validate against existing BI, and close the gap between database definitions and business understanding will build something truly useful.

My Token Spend is up 500%: What I learned to Manage Claude AI Cost

Posted on by
Pacman AI Token Chomper

I have watched my Claude AI cost climb month over month for several months in a row. Sharing token burn is like sharing how many lines of code you write; It is a meaningless statistic. My AI token burn is where the work has taken me. Along the way, I have figured out how to buy back my time. There are other areas where I am chomping through tokens like Pac-Man with no real value.

Claude AI Cost

Accelerated Spend with Parallel and Self-Spawning Agents

Time and money are the two bottlenecks I run into now. The work is no longer gated by what I can think of. It is gated by how fast and how cheaply I can get an agent to do it. . Once you start spawning agents that spawn other agents, you stop thinking in monthly cost and start thinking in spend to yield. The question I am grappling with is not how many tokens I burn… It’s what the spend gives me back in cost avoidance and return on investment.

Self-spawning agents are exactly what they sound like. You give one agent an objective, it spins up its own multi-turn sub-processes handle each job, the same way a team tackles a problem. A research task that used to be one chat session becomes a tree of conversations, each one consuming context, calling tools, and writing output the parent agent then has to read. It feels nicer to watch and the output can be excellent, but if your instructions are not specific enough, you end up paying for a lot of wasted turns and dead-end transactions.

Shift from 1:1 to 1:n Agents

A back and forth chat session with Claude or ChatGPT will not burn many tokens. The average user does not come close to their daily limit. That changes the moment your workflow calls for parallel work. On any given afternoon I have 2-5 screens flickering with LLMs handling a variety of workloads. Each with their own context window, their own tool calls, their own MCP overhead. The math is no longer one user times one model. It is one user times n agents times however many turns each one takes. That is where the bill grows fast.

Breadth of Adoption and Competency

The list of tasks I hand off to AI is not the same list it was six months ago. Research, document drafting, then data work, and dev operations, orchestration. The more confident I get, the more domains I throw at it, the longer I let agentic workflows run. I hand off work before I go to sleep and that runs form 1-3 hours.

Tips to Control Claude AI Cost and Risk

Not Every Task Needs the Best Model

Do not use Opus 4.7 for basic tasks. You are lighting money on fire. Opus is the most capable and the most expensive. Save it for work where reasoning quality actually matters. Architecture decisions, hard debugging, sensitive writing. Sonnet handles the bulk of normal work just fine. Haiku is plenty for cleanup, formatting, search, simple extractions, and high-volume small tasks. Match the model to the difficulty of the job. If the difference in output quality is not visible to a human, you are paying a premium for nothing.

Narrow the Scope of Connections and MCPs to What You Need

This recommendation may be obsolete in a few months as AI tools get more efficient, but right now it matters. I noticed I was hitting my daily limits in minutes when I had dozens of MCPs wired up. Every MCP loads its tool definitions into the context of every turn. That overhead is paid before the agent does any real work. Turn off what you are not using. Build task-specific configurations. An agent does not need access to your CRM, your calendar, your codebase, and your design tool all at once if the job is to summarize a Slack thread.

Clear System Prompts

System prompts run on every turn, so a bloated one taxes you forever. For day to day chats I ask for the shortest possible response and I get it. For projects and agents, I write a system prompt that is short, specific, and tells the model exactly what good output looks like. A vague system prompt makes the model guess, and guessing produces longer responses, more retries, and more tokens. Specificity is cheap. If you do need a long system prompt, lean on prompt caching. It lets the provider reuse the prompt across turns at a fraction of the per-token cost, which makes the difference between a system prompt that taxes you and one that does not.

Fight the FOMO Urge

Every three months the goal posts move. A new model, a new framework, a new set of best practices, and new tools. Whatever the best and coolest tool has today will be common and widely available in three to six months. Chasing every release is its own form of waste. Now, I only adopt products that allow me to pivot models and offer MCP. I never adopted Claude code, and stuck with Cursor. Pick the stack that makes you productive right now and let the innovation round robin come to you.

Retain Human Oversight and Control

The operator is still liable for the work product. That does not change because there are five agents in the loop instead of one. If you are producing 10x your own capacity, and you are doing it in domains, subjects, and technical areas you do not understand, you are creating risk. Speed without judgment is a recipe for shipping an opinionated, wrong answer.

Claude AI Cost should Move the Needle

Any multi-step agent / AI driven process, I require a detailed execution plan. I read that plan and that time investment has prevented waste and risk. Typically those plans, when executed should take an hour of my time and save 3-5 hours. Otherwise, it’s not worth the effort and risk.

Getting Started with Snowflake CoCo (Cortex Copilot)

Posted on by
Snowflake CoCo

Snowflake has been evolving quickly over the least year with it’s Cortex AI offering. Snowflake CoCo is Cortex Copilot. It’s one of the clearest examples of Snowflake embracing a modern co-pilot approach that works incredibly well. They embraced several functions that I cover in my anatomy of a modern copilot article.

Instead of exporting data into external AI tools or building complicated integrations, you can now interact with your Snowflake data using natural language. The AI assistant lives directly inside the platform and works against the data already stored in your warehouse.

Snowflake CoCo

What Cortex Copilot Actually Does

At its core, Cortex Copilot provides a natural language interface to Snowflake. The formal Snowflake CoCo documentation covers what is supported, and I admit I haven’t read it! I jump in, ask logical questions for real production problems and I get correct answers 90%+ of the time.

Off the top of my head, here are the tasks I have successfully tested CoCo that felt frictionless.

  • Validating queries multiple versions of queries
  • Setting up a new DBT project
  • Migrating views and materialized views to DBT models
  • Troubleshooting broken SQL
  • Granting permissions and RBS auditing tasks
  • Reviewing and troubleshooting YML for semantic models
  • Advanced searching based on table / view structure
  • Text to SQL
  • SQL diff comparison
  • Validating results between queries

Why This Matters for Data Teams

Most companies have invested heavily in building modern data stacks. Data warehouses, pipelines, and analytics tools are already in place. The pace of innovation from Snowflake has moved at a rate that is impossible to keep up with. Cortex provides a level playing field where new features, documentation, and best practices for using Snowflake, DBT, and other integrations has been packaged up as skills by the Snowflake team.

AI Where the Data Already Lives

One of the biggest advantages of Snowflake Cortex Copilot is aware of schema , semantic models, administrative functions and more. As modern co-pilot it enforces role based permissions and access policies. That has has been a breath of fresh air as I invite more information workers into Snowflake Workspaces. That was something that I never would have imagined starting 2026!.

How to Enable Snowflake Cortex in Snowflake

Getting started with Cortex requires only a couple of account level configuration changes.

First, enable the Cortex analyst functionality.

ALTER ACCOUNT SET ENABLE_CORTEX_ANALYST = TRUE;

Next, allow access to the models that power the Cortex features.

ALTER ACCOUNT SET CORTEX_ENABLED_CROSS_REGION = 'ANY_REGION';

Some organizations prefer to restrict model access to a specific region. In that case the configuration can be set more narrowly.

ALTER ACCOUNT SET CORTEX_ENABLED_CROSS_REGION = 'AWS_US';

Once these settings are enabled, Cortex capabilities become available within the Snowflake platform.

Final Thoughts on Snowflake CoCo

Cortex Copilot represents a meaningful shift in how we can interact with Snowflake.

I have already wired up Snowflake Cortex Copilot CLI to work inside of Cursor. It’s not as fast, but the additional layer of planning, orchestration and micro-knowledge loops has transformed the way I work. I don’t use Claude Code, but I am sure it works the same there. If you want my template, feel free to contact me directly.

The barrier to entry to work with data is the lowest it has ever been with Snowflake CoCo! Happy coding.

Anatomy of a Modern AI Co-Pilot

Posted on by
Modern Co Pilot

What Actually Matters After Using AI for Production Productivity

Over the past year I have continued to find new peaks in productivity as a DataTools Pro. In this article, I break down some of the biggest unlocks I have experienced watching leaders build hyper growth businesses on the backs of well designed AI experiences. The common thread where I find the greatest productivity and rapid adoption is a well designed AI co-pilot… Work that requires human accountability, typically requires a human in the loop. Here are the tools I am using every day giving me move 2-5x faster than 2023.

  • Snowflake dev environments (Cortex Code) –
  • Repo-driven IDE workflows (Cursor)
  • Micro-Apps and prototyping (Lovable)
  • Product and Web Analytics (PostHog AI)
  • GPT / Claude chat interfaces
  • Video editing (Descript)

Breaking Down Features for Peak AI Co-Pilot Productivity

After dozens of experiments across tools I have attempted to apply lessons learned into our DataTools Pro, where we manage strategy, business semantics and metrics. Here is a framework that actually matters as I evaluate my own startups and what I adopt.

1. Multi-Turn Conversation

What it is

The ability to maintain context across iterative back-and-forth reasoning inside a session. It simulates short term cognitive continuity.

Without multi-turn, every request is stateless. With it, the AI remembers prior questions, assumptions, and constraints.

Why it matters

Real engineering work is iterative. You ask a broad question, narrow scope, introduce tradeoffs, refine logic. Multi-turn prevents constant context resets.

Example in action

  • GPT / Claude: You brainstorm architecture, refine it over 10–15 exchanges.
  • Cortex Code: You explore warehouse credit usage, then drill down into specific roles without re-briefing the account context.
  • Cursor: You modify a function, then adjust related files in follow-ups.
  • Lovable: You scaffold an app, then iteratively adjust schema and UI.
  • PostHog AI: You analyze funnel drop-offs, then pivot into retention metrics.

Multi-turn is table stakes now. But it only gives session continuity. It does not create long-term intelligence.

2. Context-Aware Reasoning

What it is

The model reasons against your environment, grounded on what you are specifically working on instead of abstract patterns based solely on the interaction itself.

  • Repository / code awareness
  • Metadata awareness
  • Change and usage logs
  • Visual awareness (screen grabs and computer vision).
  • App state (what you are doing in present moment or history).

Why it matters

This is the difference between “plausible” and “correct.”

Examples

  • Cortex Code: You ask, “Which warehouses consumed the most credits?” It generates SQL grounded in your actual Snowflake metadata.
  • Cursor: It refactors across your actual repo instead of hallucinating file names.
  • Lovable: It understands the state of the generated app and adjusts components coherently.
  • PostHog AI: It queries real event data to answer product questions.
  • GPT / Claude (standalone): Context awareness is limited to what you paste in manually.

Grounded context dramatically increases reliability and reduces hallucination.

3. Self-Reflection & Iterative Reasoning

What it is

The system critiques or refines its own output instead of stopping at first completion. This is effectively a quality control layer.

Why it matters

Speed without reflection creates brittle systems. Reflection increases decision quality.

Where we’ve seen this

  • PostHog AI: Agent loops evaluate output and adjust before finalizing analysis.
  • Cursor (partial): When prompted explicitly, it can compare approaches and refactor more carefully.
  • GPT / Claude: Capable, but requires manual prompting (“critique this”).
  • Cortex Code: Typically direct generation, not built-in critique loops.
  • Lovable: Focused on generation speed over architectural reflection.

Reflection is not default behavior in most tools. It has to be engineered or prompted

5. Agent Workflows & Task Loops

What it is

The ability to break down an objective function and execute step-by-step with intermediate evaluation is how most people problem solve and execute. Agents that summarize work before execution creates a much better experiences in my opinion.

Why it matters

This shifts AI from “answering questions” to “completing tasks”; one day completing goals

Strong examples

  • Cursor: Multi-file planning and stepwise refactors.
  • Lovable: Full-stack app scaffolding from high-level instructions.
  • PostHog AI: Analytics agents running multi-step investigations.
  • Cortex Code: Less agentic, more query-focused based on questions.
  • GPT / Claude: Capable but requires manual orchestration.

This is where copilots begin to feel like collaborators instead of search engines, is when they demonstrate understanding. Breaking down problems into its smallest parts and recommending next steps is where you truly feel like you have a “co-pilot.

Exciting Innovations I’m Looking for in an AI Copilot

After running these systems in real workflows, I look for 3 capabilities will make co-pilots even more useful!

Controlled and Secured Autonomy with Safe Reversion

As AI edits files, runs queries, or executes workflows, autonomy increases. AI accesses data it shouldn’t have. How do you recover? That is the “trust layer” that needs to be engineered at every layer of your technology stack.

A mature system must provide:

  • Suggest-only mode
  • Controlled edits
  • Test execution
  • Refactor execution
  • Deterministic rollback

Trust is built through reversibility.

Cursor approaches this through diff visibility. Most others still lack robust autonomy controls.

Persistent Structured Memory

Long term cognitive continuity.. (Long-Term Cognitive Continuity). For now, I am collecting a mountain of “know how” in the form of MD files and knowledge bases across multiple domain specific tools. ChatGPT is still my favorite to recall fragments of work and reasoning.

A fun experiment is open ChatGPT and ask

What is it like to work with me? What are my top 3 strengths and what are my top 3 weaknesses.

What We’ve Learned from Lab Experiments

Embedding AI copilots into production workflows shifts the evaluation criteria. AI feels magical until you know what the output should be. That is why I look to best of breed co-pilot experiences as the guiding light for what I should working toward.

Multi-turn was the first wave. Agent workflows were the second. The next frontier is institutional intelligence where AI not only reasons in the moment, but compounds over time. That is why our investments in DataTools Pro from day 1 has been cultivating business semantics from existing systems of record (Salesforce) and systems of understanding (Snowflake, Tableau).

Stress Testing Microsoft Copilot vs Claude vs ChatGPT

Posted on by
AI Bakoff with MS CoPilot

This weekend, I did a real world Microsoft Copilot vs Claude vs ChatGPT bakeoff while wrapping up a lead magnet calculator. In preparation for a Microsoft call to discuss an AI Copilot rollout, I wanted some hands on experience.

The Bakeoff Workflow

  1. Take a detailed calculator requirements doc (AI generated from source code).
  2. Recreate a simplified version in Excel via prompt.
  3. Document the structure.
  4. Translate the workflow into an executive ready PowerPoint story.
  5. Use the output as preparation for a Copilot rollout conversation.

This would be a day of work for multiple people. The project was complete in less than an hour.

Phase One: Translating the App into Excel

The spreadsheet needed:

  • Clear input structure supplied by a 400 line markdown file.
  • Clean calculation logic
  • Organized output summary
  • Executive ready formatting for review and sign off
Microsoft CopilotClaudeChatGPT
Copilot fragmented the logic across multiple tabs. Inputs and outputs were not logically grouped. Structural coherence was inconsistent. If an AI tool creates cleanup work, the productivity gain erodes immediately.Claude generated a tight, single page spreadsheet. Inputs were grouped cleanly. Calculations were centralized. Outputs were summarized clearly. It felt intentional and the result was the best of the group.ChatGPT produced a multi tab structure with clear separation between inputs, logic, and results. It was operationally sound and logically organized.
It required slightly more navigation than Claude’s single page approach, but the structure held.
Microsoft CoPolot ExcelOpen AI Excel

Phase Two: Explaining the Build in PowerPoint

I have never been a fan of PowerPoint. It is a corporate time and knowledge sinkhole. My hope is one day data / knowledge management tools paired with LLMs will force PowerPoint to evolve or go away.

PowerPoint exists as a corporate knowledge artifact that memorializes a point in time. In concept that would be a great thing if the real story and context wasn’t lost in meetings and presentations where PowerPoints are delivered. Microsoft has all of the pieces to the puzzle, so I am blown away they haven’t put it all together.

Clearly this stress test wasn’t going to be transformational to my way of working… At minimum, I wanted to produce a single slide that would explain my app design workflow, to highlight how I was using AI.

  • How the idea evolved
  • How AI accelerated development
  • Where structure improved
  • Where friction was eliminated
Microsoft CopilotClaudeChatGPT
Copilot generated an image instead of an editable diagram. My issue is when shapes cannot be modified, it becomes static decoration. Even the text was encoded as text which is annoying.Claude produced a comprehensive diagram with strong narrative flow. It mapped the journey clearly and felt cohesive. Text was editable.ChatGPT generated a simpler diagram, fully editable in PowerPoint. Less polished, more modular.
Microsoft CoPilot PowerPointClaude PowerPointOpenAI PowerPoint

My findings with Microsoft Copilot so far..

Copilot’s core advantage is integration within Microsoft 365. Outlook was not part of this evaluation but I am praying when I get to the proof of value, it is the star of the show. The Excel and PowerPoint experience was underwhelming for creation. However, I did use co-pilot to evaluate and edit my Claude produced Excel. It did a great job with that task.

Adoption fails when cognitive load remains unchanged. Frustration happens when more time and cognitive load are required than the previous solution.. Without a major payoff in the form of pain reduction or value creation, its tough to recover.

Bottom line: Claude felt magical, and Copilot felt like something I experienced 18 months ago in ChatGPT. Enterprise platform alignment with Office and Azure, security, and wider distribution are real value drivers. That feeling of being behind could make this an acceptable solution.

My Strategic Criteria for Evaluating Copilot

Productivity and Communications Compression Across Microsoft 365

Copilot’s primary strategic function is to compress knowledge work inside the Microsoft ecosystem. Copilot is not designed to replace core application functions; it is designed to accelerate them.

When it comes to communication (Email and Teams), my hope is Copilot will clearly increase the velocity of information consumption and delivery. If not, upcoming proof of value exercise could be short lived.

My primary objectives as I evaluate CoPilot…:

  • Streamline Email search (Gemini in GMail has been a game changer).
  • Speed to response via email
  • Shorten drafting cycles.
  • Consolidation of meeting summary tools into 1 repository.
  • Speed spreadsheet modeling
  • Automate presentation generation

Provide a secure, standardized AI layer across the organization

Security is a major concern for every operator and executive when it comes to these AI models. Copilot provides at least one controlled AI entry point with potential access to confidential data.

My Biggest Concerns as I Continue exploring

  • Training focused on value creation – Understanding the span of capabilities is important but connecting business challenges to tech is where we will create value.
  • Clear use case alignment- The gap between expectations of possibilities, and real feature availability is a concern I want to remove early.
  • Adoption Management – If users do not adopt, it is a failure. If Co-Pilot fails, we are going to do it fast and move on to the next alternative.

Without high value use cases, adoption, and education AI becomes just another data tool that blames bad data or process rather than becoming an enabler that reduces operational drag.

Final Take

AI productivity is not about who generates prettier demos. Real AI success requires distribution of knowledge and experience across a team. Data alignment and influence are about getting a group of people rowing at the same speed and in the same direction. AI is the same data activation and knowledge delivery exercise as analytics, so I feel well equipped to take it on!

There are many other bright spots for Microsoft and AI including the work I have done in Azure and recently with Power BI MCP Server at BIChart.

In this test, Claude shined the brightest. I am still excited to do a proper CoPilot proof of value and see how it goes!

The Tableau vs Power BI Rap Battle: So Cringy it’s Addictive

Posted on by
Rap Battle

Over Thanksgiving break, I decided to mash up the classic data-geek debate of “Tableau vs Power BI” into an AI-powered rap battle. Three rounds of diss tracks with AI on the mic.

I’ve screened it with a few folks, and the reviews so far? Fun. Cringe. Silly. Freaking awesome.

Sometimes technology takes itself a little too seriously. This is meant to be silly, and a balanced showcase of where things stand. Not since the East Coast vs. West Coast battles of the ’90s have we seen such fierce loyalty between two groups of data sense-making professionals.

Visit DataTools Battles

Where do I stand on the debate between Tableau and Power BI?

Working at DataTools Pro where I am using Tableau daily, yet migrating Tableau into Power BI at BIChart I have preferences depending on the use case.

I choose the right solution that works best for the team, skills, investments, and what leads to the highest adoption!

I will let the community continue to debate. Check out the site, sign up for notifications over the next couple weeks, and enjoy!

Visit DataTools Battles

Testing Salesforce External Client App with our DataTools Portal

Posted on by
AI Assistant

This week, I had a chance to update documentation and explore Salesforce External Client App configuration. There have been so many changes to Salesforce connected apps in terms of integration and commercial requirements. It is daunting for customers and partners.

What is Changing from Salesforce

  1. 3rd party tools that use the deprecated “Connected App” functionality will no longer gain the ability to connect to new Salesforce orgs in Spring 2026. Partners will need to upgrade, or get left behind. We are going to fork DataTools Pro app to no longer use Salesforce for federated access to DataTools Pro.
  2. Integrated apps will need to join Salesforce App Exchange where fees are collected. This is going to cause a ton of friction and headache for vendors. DataTools Pro is already integrated into the AppExchange so this does not impact us.

What about Internal Built Apps?

This is an area that’s genuinely confusing and murky, so I decided to jump right into it by building our new customer and partner portal. It sets me up where Salesforce the system of record for customers, but events and activity related data are linked only by a single external UID.

The portal integrates with our support Slack, Salesforce, OpenAI, Stripe, and the DataTools Pro app. After running this experiment, it’s easy to see why Salesforce is scrambling to control and monetize the data within Salesforce.

When you build a portal / community in Salesforce, you are building for a point in time that has passed. We have opened up our portal for anyone to login via magic link to poke around and will rollout our new DataTools Shop in 2026!

https://portal.datatoolspro.com

We are moving to a more traditional federated login configuration with Google and Microsoft / Entra, and expanding our enterprise-specific SSO support.

Learn how to Setup Salesforce External Client Apps

If you are interested in the nitty gritty details of configuring Oauth for External Client APps, I have updated our Azure DataFactory tutorial to explain the process