I’ve tried to spin off tensorflow locally and play with GPT-Neo on my M1 mac. Found some official reference online beforehand, so my intentions should have been supported out of the box without any headaches. Buuut, apparently this doesn’t totally apply for MacOS 12 (Monterey). That’s why I’ve fixed the process and I’m documenting it (for myself) down below.

Setting up the env

Steps:

  • install xcode & brew
  • MiniForge3 (don’t use Anaconda) -> download the arm64 sh from GitHub
  • setup conda for your shell (fish in my case)
  • create a new virtualenv with conda and switch on it
  • set up tensorflow + tensorflow-metal for GPU (in case you plan to use it afterwards)
  • install jupyter
  • install happytransformer and use it for generating text leveraging EleutherAI/gpt-neo

Some detailed steps

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

# set up conda for fish
~/miniforge3/bin/conda init fish

# create and use a new env
conda create --name tf python=3.9
conda activate tf

# install tensorflow deps
conda install -c apple tensorflow-deps
# base tensorflow + metal plugin
python -m pip install tensorflow-macos
python -m pip install tensorflow-metal

# install jupyter, pandas and whatnot
conda install -c conda-forge -y pandas jupyter

Installing transformers

For this one, we’ll have to accomplish 2 different things:

  • install tokenizers rust package and use the python binding
  • install transformers package
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

# install rust toolkit - if you don't have it
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

# install tokenizers
mkdir -p Projects/lab/tfsetup & cd Projects/lab/tfsetup
git clone https://github.com/huggingface/tokenizers
cd tokenizers/bindings/python
# compile tokenizers - should be pretty fast on your m1
pip install setuptools_rust
# install tokenizers
python setup.py install

# install transformers using pip
pip install git+https://github.com/huggingface/transformers

Installing happytransformer package

This one was trickier than expected - let’s try with pip. pip install happytransformer but you’ll notice that something’s wrong when installing the sentencepiece dependency - as the wheel cannot be built.

Therefore, before installing happytransformer, we need another 2 additional packages available through brew:

1
2

arch -arm64 brew install cmake
arch -arm64 brew install pkgconfig

Now the install process should finish up with a warning. You can even choose to install sentencepiece on it’s own just to see if it goes well or not.

Let’s start a jupyter lab and play a bit.

Testing your GPU support

The following script should tell you if you did everything right:

1
2
3

import tensorflow as tf

print("GPUs: ", len(tf.config.experimental.list_physical_devices('GPU')))

Use it with GPT-Neo

You can use any model published by EleutherAI, depending on how big and powerful you want it to be. Below we’ll see the smallest trained model in use:

1
2
3
4
5
6
7
8

from happytransformer import HappyGeneration

gen = HappyGeneration("GPT-NEO", "EleutherAI/gpt-neo-125M")

input = "What is the stock market?"
result = gen.generate_text(input)

print(result.text)

… will eventually produce a simple result such as:

The stock market is a financial instrument that is used to measure the value of a company. It is a measure of the value of a company’s assets. It is a measure of the value of a company’s liabilities.