Unfortunately I have got zero success running gemma with mlx-lm main branch. Can you point me out what is the right way? I have zero experience with mlx-lm.
> pip3 install git+https://github.com/ml-explore/mlx-lm.git
> ./venv/bin/mlx_lm.generate --model "$MODEL" --temp 1.0 --top-p 0.95 --top-k 64 --max-tokens 128000 --prompt "Hello world"
Where $MODEL is an unsloth model like:
- unsloth/gemma-4-E4B-it-UD-MLX-4bit
- unsloth/gemma-4-26b-a4b-it-UD-MLX-4bit
reply
Unfortunately I have got zero success running gemma with mlx-lm main branch. Can you point me out what is the right way? I have zero experience with mlx-lm.