In my previous post I discussed using pipx to install PySpark on Linux Mint. You can use pip to install PySpark, you just have to create the virtual environment manually and remember to activate the virtual environment each time.

To create the virtual environment, create a folder and navigate to it in terminal. Then use the following command:

python3 -m venv pyspark

"pyspark" is the name of the virtual environment, you can change it to whatever you like.

Once the venv is created, you need to actiavte it before installing anything else. You'll need to use this same to activate the environment each time you want to use it

source pyspark/bin/activate

(replace "pyspark" with whatever name you used in the first step).

To install pyspark, use this command exactly. Also install whatever other helper packages you need to use with pip, but make sure to do so when the environment is activated.

pip install pyspark

(this time you can't change "pyspark" or you'll risk installing some other package)

Once the environment is activated, you just need to choose the python interpreter in your IDE, same method as the last post but the folder structure is a little easier.