When using Pyspark shell I love to have the tab auto complete functionality. This is not enabled by default. To enable it, you will need to do the following. In this post I will tell you how to add auto complete to Pyspark Shell.
1. Add the following piece of code to .pythonrc file in your home directory
import rlcompleter, readline
readline.parse_and_bind('tab: complete')
2. Create the file if it does not exists. After that, add the following line to the .bashrc file.
export PYTHONSTARTUP="$HOME/.pythonrc"
Now, just restart the shell or do a source .bashrc . The next time you launch the Pyspark shell you will have the autocomplete functionality available.
Let me know in the comments section below if this works for you and you find it useful.