GitHub Actions 101: Automate Your GitHub Projects: Getting Started

3 min readNov 2, 2023

GitHub Actions is a powerful automation tool for building, testing, and deploying your code directly from your GitHub repository.

Let’s try with a sample project, I used a Spark, Scala, and Gradle project that requires Java 11, and Gradle version 7.6

Prerequisites

A GitHub repository
A build tool (Gradle)
S3 bucket
IAM role configured in the AWS account with GitHub OIDC Provider to access the S3 bucket

GitHub Actions Workflow:

Creating your first workflow

Add a folder .github/workflows/<filename>. Add a filename: main.yml and write a workflow

name: CI/CD

on:
  push:
    branches: "**"

jobs:
  build:
   # checkout, build

  test:
   # run unit test

  publish:
   # package and publish

Example

name: CI/CD

on:
  push:
    branches: [ $default-branch ]
  pull_request:
    branches: [ $default-branch ]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
      - name: Set up Java 11
        uses: actions/setup-java@v3
        with:
          distribution: 'corretto'
          java-version: '11'
          cache: 'gradle'
      - name: Setup Gradle 7.6
        uses: gradle/gradle-build-action@v2
        with:
          gradle-version: 7.6
      - run: gradle assemble
      - name: upload the build artifact
        uses: actions/upload-artifact@v3
        with:
          name: apache-spark-framework-artifact
          path: build/libs
          retention-days: 1
      - name: Setup caches
        uses: actions/cache@v3
        with:
          path: |
            ~/build/libs
          key: ${{ runner.os }}-build
          restore-keys: |
            ${{ runner.os }}-build

  test:
    runs-on: ubuntu-latest
    needs: build
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
      - name: Set up Java 11
        uses: actions/setup-java@v3
        with:
          distribution: 'corretto'
          java-version: '11'
          cache: 'gradle'
      - name: Setup Gradle 7.6
        uses: gradle/gradle-build-action@v2
        with:
          gradle-version: 7.6
      - name: Running Unit Tests
        run: gradle check

Build Job

The build job sets up the environment for building our Spark project. It runs on an `Ubuntu-latest` runner and performs the following steps:

Checks out the repository.
Sets up Java 11 with Amazon Corretto and caches Gradle dependencies.
Configures Gradle 7.6.
Builds the Spark project using gradle assemble.
Uploads the build artifact to the GitHub Actions workspace.

Using actions/upload-artifact@v3 action to upload the artifact to the GitHub Actions workspace.

Test Job

The test job runs unit tests for a project. It follows a similar setup to the build job, including Java and Gradle setup. It then runs gradle check to execute unit tests.

Conclusion

GitHub Actions simplifies the process of automating your Spark project’s CI/CD pipeline. This example showcases how you can build, test, and deploy a Spark application with a few simple YAML configurations. It streamlines your development process, enhances code quality, and provides seamless deployment to AWS S3.

For more detailed information, including AWS S3 configuration and advanced use cases, you can refer to the GitHub Actions documentation.

Now, you have the tools to make your Spark project’s development and deployment smoother and more efficient. Happy coding!