Install Apache Spark on macOS

This short guide will assume that you already have already homebrew, xcode-select and java installed on your macOS. If not, run the following commands on your terminal.

$ xcode-select –install
$ /bin/bash -c "$(curl -fsSL"
$ brew cask install java

Once you are sure that everything is correctly installed on your machine, you have to follow these steps to install Apache Spark

Step 1: Install scala

brew install scala@2.11
Keep in mind you have to change the version if you want to install a different one

Step 2: Install Spark

brew install apache-spark

Step 3: Add environment variables

Add the following environment variables to your .bash_profile or .zshrc

export SPARK_HOME=/usr/local/Cellar/apache-spark/2.4.5/libexec           export PATH="$SPARK_HOME/bin/:$PATH"
Keep in mind you have to change the version to the one you have installed

Step 4: Review binaries permissions

For some reason, some installations are not give execution permission to binaries. Review it and give them if necessary.

chmod +x /usr/local/Cellar/apache-spark/2.4.5/libexec/bin/*
Keep in mind you have to change the version to the one you have installed

Step 5: Verify installation

If everything worked fine you will be able to open a spark-shell running the following command


This should open a shell as follows

$ spark-shell 
20/04/21 12:32:33 WARN Utils: Your hostname, mac.local resolves to a loopback address:; using instead (on interface en1)
20/04/21 12:32:33 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
20/04/21 12:32:34 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Using Spark's default log4j profile: org/apache/spark/
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
Spark context Web UI available at
Spark context available as 'sc' (master = local[*], app id = local-1587465163183).
Spark session available as 'spark'.
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 2.4.5
Using Scala version 2.11.12 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_162)
Type in expressions to have them evaluated.
Type :help for more information.


We are done! You have successfully installed Apache Spark on macOS

You can also find me on Twitter if you’d like to read similar technical tricks!

Related posts

Apr 22, '20

Running Apache Spark and S3 locally

3 min read