Tools and Environment

1. Environment Setup

Due to the architecture of SDL, it is typically possible to run SDL on a developer-issued laptop (e.g., Apple Silicon MacBook Pro). By utilizing Apple Rosetta, you can run amd64 processes/containers with near-native performance.

If you prefer to work on an amd64 Linux environment, on-prem VMs are available for this purpose. The steps below are for development on a Mac machine, but the command-line tools should be installed wherever you plan to deploy SDL or build code. Editors like VSCode allow for development through SSH, which can be useful when working on a remote VM.

The SDL team doesn’t enforce any specific environment setup. However, using the same set of tools as the team can make debugging easier, especially during the first few weeks. Below is the typical setup used by the SDL development team:

All Developers

Optional Tools

  • Terminal app:

Required Tools for macOS

  • homebrew (an unofficial package manager for macOS)

  • A container management tool, depending on your OS:

    • Raft is moving to Podman over Docker Desktop for macOS. There may be some initial hiccups as SDL development moves onto it.

    • For VM development, using Docker is fine as it doesn’t require a team license.

    • Podman and Podman Desktop are the recommended tools.

    • Other options like colima may work, but YMMV.

    • Rancher Desktop is not recommended due to sensitivities around using Rancher for DoD applications.

Required Tools (All Platforms)

  • kubectl - Kubernetes command-line tool

  • helm - Kubernetes package manager

  • kind - Kubernetes in Docker (for local development)

  • k9s - Optional tool for Kubernetes cluster management

  • jq - JSON processor

  • yq - YAML processor

  • make - Build automation tool

  • aws-cli - AWS command-line tool (optional)

  • dfdev - Simplified SDL deployment/management (optional)

Backend Development

  • Common programming languages used include:

    • Go, Java, Python, and Rust (roughly in that order of precedence).

  • For JVM languages:

    • IntelliJ IDEA (community edition is typically fine).

  • Other editors that are used:

    • VSCode

    • Zed

    • Vim/NeoVim

    • IntelliJ Ultimate (paid version)

Frontend Development

  • A mix of JavaScript and TypeScript is used.

  • VSCode is primarily used for frontend development.

2. Deploy a Minimal SDL

Create a GitHub PAT and clone repo

If you haven’t already, provide a GitHub username to be added to the raft-tech organization.

See the GitHub docs for more information on how to create a GitHub PAT.

Although fine-grained tokens are recommended, given that we use PATs to auth with ghcr.io, we need to create classic tokens instead.

The primary repository to start with is rdp, which contains deployment files and helper scripts to get everything set up.

Environment and Deployment

Set up the following environment variables:

export RDP_HOME=/path/to/your/rdp
export GHP_USERNAME=your_github_username
export GHP_SECRET=your_github_personal_access_token

The scripts provided in /scripts can be used to simplify local development.

To create a Kind cluster to work with:

cd $RDP_HOME
./scripts/rdp_create_kind.sh

Once the cluster is created, deploy SDL using:

./scripts/rdp_deploy.sh -e dev-minimal

Development Cluster Routing

Typically, production SDL instances have a public DNS record associated with them. For development, we typically don’t set this up. However, there are two ways to handle this:

  1. If using the instructions above, your cluster will be available on localhost.

  2. If you prefer a non-public DNS record, deploy a cluster with a non-public DNS record and update your /etc/hosts file to point to it:

./scripts/rdp_deploy.sh -e dev-minimal -u sdl.local

Then, in your /etc/hosts file, add the following entry:

127.0.0.1 sdl.local

Note: You’ll need to create identical records in your /etc/hosts file for every subdomain SDL routes on. You can see these subdomains by examining the Ingress Kubernetes resources:

kubectl get ingresses -n data-fabric