I’ve tried to spin off tensorflow locally and play with GPT-Neo on my M1 mac. Found some official reference online beforehand, so my intentions should have been supported out of the box without any headaches. Buuut, apparently this doesn’t totally apply for MacOS 12 (Monterey). That’s why I’ve fixed the process and I’m documenting it (for myself) down below.
Setting up the env
- install xcode & brew
- MiniForge3 (don’t use Anaconda) -> download the arm64 sh from GitHub
- setup conda for your shell (fish in my case)
- create a new virtualenv with conda and switch on it
- set up tensorflow + tensorflow-metal for GPU (in case you plan to use it afterwards)
- install jupyter
- install happytransformer and use it for generating text leveraging EleutherAI/gpt-neo
Some detailed steps
# set up conda for fish ~/miniforge3/bin/conda init fish # create and use a new env conda create --name tf python=3.9 conda activate tf # install tensorflow deps conda install -c apple tensorflow-deps # base tensorflow + metal plugin python -m pip install tensorflow-macos python -m pip install tensorflow-metal # install jupyter, pandas and whatnot conda install -c conda-forge -y pandas jupyter
For this one, we’ll have to accomplish 2 different things:
- install tokenizers rust package and use the python binding
- install transformers package
# install rust toolkit - if you don't have it curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh # install tokenizers mkdir -p Projects/lab/tfsetup & cd Projects/lab/tfsetup git clone https://github.com/huggingface/tokenizers cd tokenizers/bindings/python # compile tokenizers - should be pretty fast on your m1 pip install setuptools_rust # install tokenizers python setup.py install # install transformers using pip pip install git+https://github.com/huggingface/transformers
Installing happytransformer package
This one was trickier than expected - let’s try with pip.
pip install happytransformer but you’ll notice that something’s wrong when installing the
sentencepiece dependency - as the wheel cannot be built.
Therefore, before installing happytransformer, we need another 2 additional packages available through brew:
arch -arm64 brew install cmake arch -arm64 brew install pkgconfig
Now the install process should finish up with a warning. You can even choose to install sentencepiece on it’s own just to see if it goes well or not.
Let’s start a
jupyter lab and play a bit.
Testing your GPU support
The following script should tell you if you did everything right:
import tensorflow as tf print("GPUs: ", len(tf.config.experimental.list_physical_devices('GPU')))
Use it with GPT-Neo
You can use any model published by EleutherAI, depending on how big and powerful you want it to be. Below we’ll see the smallest trained model in use:
from happytransformer import HappyGeneration gen = HappyGeneration("GPT-NEO", "EleutherAI/gpt-neo-125M") input = "What is the stock market?" result = gen.generate_text(input) print(result.text)
… will eventually produce a simple result such as:
The stock market is a financial instrument that is used to measure the value of a company. It is a measure of the value of a company’s assets. It is a measure of the value of a company’s liabilities.