linux_docker/dev

Spark Dev Container

How to create a Spark VS Code Devcontainer

1. Build the Docker image

Run the script build.sh located in this directory to build the image

build.sh

Notes

• Builds an image named $USER:spark-dev
• Adds a user to the image with the same name/uid/gid as the current user so it has proper file permissions in mounted directories
• Run build.sh --help to see available options

2. Generate a devcontainer.json

Run the script gen_devcontainer.sh located in this directory to create a devcontainer.json file.
Note: The file path must be <repo_dir>/.devcontainer/devcontainer.json

# Dump `gen_devcontainer.sh` output to devcontainer.json
gen_devcontainer.sh > $workspaceFolder/.devcontainer/devcontainer.json

Notes

• Mounts the following directories (<container_path>:<host_path>):
  • /home/<user>:/home/<user>/.spark-dev/mounts/<container_name>
  • /home/<user>/spark:/path/to/this/repo
• Run gen_devcontainer.sh --help to see available options

3. Open VS Code in Container

1. Launch VS Code and open the repo folder on the host machine.
2. Run Dev Containers: Open Folder in Container from the command palette.
   

*Notes

• The container stops when you close VS Code. All processes, including sparkd, will be suspened until you open the workspace again (or manually restart the container).