Why I use dev containers?

Why I use dev containers?

@kzzzr

How to package dbt and dependencies to a reproducible container

  • works almost instanty for you and team members
  • latest stable dbt + adapter versions
  • easy inividual secrets set up
  • configuring multiple dbt targets
  • fancy terminal: zsh + git plugin + shell history

Demo


Dockerfile

Mind the comments:

# set base image major version
ARG DBT_VERSION=1.0.0
FROM fishtownanalytics/dbt:${DBT_VERSION}

# install python libs (incl. latest dbt adapter)
RUN set -ex \
    && python -m pip install --upgrade pip setuptools \
    && python -m pip install --upgrade dbt-clickhouse numpy

# look for a connection file in this repo
ENV DBT_PROFILES_DIR=.

Secrets handling

Copy template file .env.example to .env:

cp .env.example .env

Git ignore .env file to avoid commiting secrets to git.

Open .env file in editor and fill in your personal credentails:

DBT_USER=''
DBT_PASSWORD=''
DBT_SCHEMA=''

profiles.yml

colorado_springs:
  target: dev
  outputs:
    dev: &colorado_springs_defaults
      type: clickhouse
      driver: native
      host: clickhouse.colorado_springs.prod
      port: 9000
      user: "{{ env_var('DBT_USER') }}"
      password: "{{ env_var('DBT_PASSWORD') }}"
      schema: "{{ env_var('DBT_SCHEMA') }}"
      threads: 2
    prod:
      <<: *colorado_springs_defaults
      schema: analytics
      user: "{{ env_var('PROD_DBT_USER') }}"
      password: "{{ env_var('PROD_DBT_PASSWORD') }}"      
    ci:
      <<: *colorado_springs_defaults
      schema: dbt_test
      user: "{{ env_var('CI_DBT_USER') }}"
      password: "{{ env_var('CI_DBT_PASSWORD') }}"      

devcontainers.json

  • Use local Dockerfile
  • 8080 to host dbt docs locally
  • List VS Code extensions
  • Use marketplace feature: common utilities (zsh, etc.)
// For format details, see https://aka.ms/devcontainer.json. For config options, see the
// README at: https://github.com/devcontainers/templates/tree/main/src/docker-existing-dockerfile
{
 "name": "Existing Dockerfile",
 "build": {
  "dockerfile": "../Dockerfile"
 },
 "forwardPorts": [
  8080
 ],
 "customizations": {
  "vscode": {
   "extensions": [
    "innoverio.vscode-dbt-power-user",
    "ms-python.python",
    "eamodio.gitlens",
    "GitHub.vscode-pull-request-github",
                "nemesv.copy-file-name"
   ]
  }
 },
 "features": {
  "ghcr.io/devcontainers/features/common-utils:2": {
   "installZsh": true,
   "configureZshAsDefaultShell": true,
   "installOhMyZsh": true,
   "upgradePackages": true,
   "username": "automatic",
   "userUid": "automatic",
   "userGid": "automatic"
  },
  "ghcr.io/stuartleeks/dev-container-features/shell-history:0": {}
 },
    "runArgs": ["--env-file",".env"]
}




Report Page