rhasson/sqlake_action

Github Action for executing an Upsolver SQLake worksheet

This Github Action allows you to execute Upsolver SQLake SQL commands to create and manage data pipelines. SQL commands oftentime need to be executed in sequantial order. This means that creating connections and tables must come before creating jobs or executing SQL queries against tables.

Here is an example of a Github Action workflow that finds SQL files in your repo and executes them.

name: development

on:
  push:
    branches:
      - dev

jobs:
  execute_sql:
    name: Execute SQL
    environment:
      name: development
    if: github.event.ref == 'refs/heads/dev'
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3
        with:
          ref: dev
          fetch-depth: 0
      - name: Get changed files
        id: cf-step
        uses: tj-actions/[email protected]
        with:
          separator: ","
      - name: List all added files
        run: |
          for file in ${{ steps.cf-step.outputs.all_changed_files }}; do
            echo "$file was added"
          done
      - name: Execute changed or added files
        uses: rhasson/sqlake_action@main
        with:
           file_list: ${{ steps.cf-step.outputs.all_changed_files }}
           api_key: ${{ secrets.API_KEY }}
           api_endpoint: 'https://api.upsolver.com'
      - name: Print Results
        run: |
          cat ${GITHUB_WORKSPACE}/execution_output.md >> $GITHUB_STEP_SUMMARY

This Action worflow is executed when new or changed files are pushed to the DEV branch. Then it performs the following steps:

1. Checks out the repo
2. Calls another Action that returns only changed files as a comma separated list
3. Calls the SQLake Action to execute them on your SQLake cluster

Configuring your Action

The Github Action takes the following parameters:

1. file_list which is a comma separated list of files to execute
2. api_key which is the API key you get from SQLake.
3. api_endpoint which is the SQLake API endpoint (public or private) and can be found in the SQLake console.
4. path_to_sql (optional) instead of providing a list of files, you can provide a path to your SQL files. The action will execute them all at every execution. This is not recommended for interactive development, but helpful for pipeline testing.

Check out the example repo