Instructions to use CohereLabs/North-Mini-Code-1.0 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use CohereLabs/North-Mini-Code-1.0 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="CohereLabs/North-Mini-Code-1.0")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("CohereLabs/North-Mini-Code-1.0")
model = AutoModelForCausalLM.from_pretrained("CohereLabs/North-Mini-Code-1.0")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use CohereLabs/North-Mini-Code-1.0 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "CohereLabs/North-Mini-Code-1.0"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CohereLabs/North-Mini-Code-1.0",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/CohereLabs/North-Mini-Code-1.0

SGLang

How to use CohereLabs/North-Mini-Code-1.0 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "CohereLabs/North-Mini-Code-1.0" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CohereLabs/North-Mini-Code-1.0",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "CohereLabs/North-Mini-Code-1.0" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CohereLabs/North-Mini-Code-1.0",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use CohereLabs/North-Mini-Code-1.0 with Docker Model Runner:
```
docker model run hf.co/CohereLabs/North-Mini-Code-1.0
```

MLX chat_template.jinja

#12

by kohlermike - opened 13 days ago

Discussion

kohlermike

13 days ago

Hi,
I was going to write a paragraph or two, but it's too late at night now. I was curious why I couldn't flip North-Mini-Code-1.0-w4a16 thinking on and off, it was always on. mlx_vlm only passes one flag for enabling/disabling thinking:
--enable-thinking Enable thinking mode in the chat template. (from mlx_vlm.chat help file)
When you dig into what mlx_vlm actually passes to the chat template, it actually passes "enable_thinking". So CohereLabs's chat_template.jinja has :
{%- set reasoning = reasoning if reasoning is not undefined else (false if reasoning_effort is defined and reasoning_effort | lower == "none" else true) -%}
The chat template is expecting "reasoning", which it will never see. So I changed line 11 in chat_template.jinja to:
{%- set reasoning = enable_thinking | default(false) -%}
the enable_thinking flag now works:

michael@Michaels-MacBook-Pro-small ai_models % curl http://localhost:8080/v1/chat/completions \
-H "Content-Type: application/json"
-d '{
"model": "/Users/michael/ai_models/North-Mini-Code-1.0-w4a16",
"messages": [{"role":"user","content":"what day is June 19 2026"}],
"max_tokens": 2048,
"enable_thinking": true
}'
{"id":"chatcmpl-2ffa1de8-7be4-4b89-8579-65300a4f55e6","object":"chat.completion","created":1781933507,"model":"/Users/michael/ai_models/North-Mini-Code-1.0-w4a16","choices":[{"index":0,"finish_reason":"stop","message":{"role":"assistant","content":"June 19 2026 falls on a Friday.","reasoning":"The user asks: "what day is June 19 2026". We need to compute the day of the week for that date. June 19, 2026. Let's calculate.\n\nWe can use known algorithms: Zeller's congruence, or known calendar. Let's compute manually.\n\nWe can also recall that 2026 is not a leap year (2024 is leap, 2028 is next). So 2026 is common year.\n\nWe need day of week for June 19, 2026.\n\nWe can compute using known reference: Jan 1, 2000 was Saturday. Then compute days difference.\n\nAlternatively, use known day-of-week for a known date: e.g., March 1, 2000 was Monday? Actually Jan 1 2000 Saturday. Let's compute days from known date to June 19 2026.\n\nBetter: Use Zeller's Congruence for Gregorian calendar:\n\nFor date: day = 19, month = 6 (June), year = 2026. In Zeller, months are counted March=3,... Jan=13, Feb=14 of previous year. So for month >2, month = 6, year = 2026. For month <=2, adjust year-1. So month = 6, year = 2026.\n\nZeller's formula: h = (q + floor((13*(m+1))/5) + K + floor(K/4) + floor(J/4) + 5J) mod 7, where:\n\n- q = day of month = 19\n- m = month (3=March,...). For June, m = 6? Actually mapping: March=3, April=4, May=5, June=6, July=7, August=8, September=9, October=10, November=11, December=12, January=13, February=14. So June = 6.\n- K = year % 100 = 2026 % 100 = 26\n- J = floor(year/100) = 20\n\nCompute floor((13*(m+1))/5) = floor(137/5) = floor(91/5) = 18 (since 518=90, remainder 1). So 18.\n\nNow compute:\n\nh = (q + term + K + floor(K/4) + floor(J/4) + 5J) mod 7\n\nq = 19\nterm = 18\nK = 26\nfloor(K/4) = floor(26/4) = 6\nfloor(J/4) = floor(20/4) = 5\n5J = 520 = 100\n\nSum = 19 + 18 + 26 + 6 + 5 + 100 = 174\n\nNow 174 mod 7: 7*24 = 168, remainder 6. So h = 6.\n\nIn Zeller's Congruence, h = 0 => Saturday, 1 => Sunday, 2 => Monday, 3 => Tuesday, 4 => Wednesday, 5 => Thursday, 6 => Friday. So h=6 => Friday.\n\nThus June 19, 2026 is a Friday.\n\nWe can double-check with known calendars: Let's check known day for June 19 2025? Actually June 19 2025 is Thursday? Let's see: 2025 is not leap year. June 19 2025 day? We can compute quickly: June 19 2025 day? Use known: June 19 2024 is Saturday? Actually 2024 is leap year, June 19 2024 is Wednesday? Let's check: June 19 2024: 2024 is leap year, Jan 1 2024 was Monday. June 19 2024: Let's compute days: Jan (31), Feb (29), Mar (31), Apr (30), May (31), Jun (19). Sum days before June: Jan+Feb+Mar+Apr+May = 31+29+31+30+31 = 152. Add 19 = 171 days after Jan 1. Jan 1 Monday + 171 days mod 7 = 171 mod 7 = 3 (since 724=168). Monday + 3 days = Thursday. So June 19 2024 is Thursday. Good.\n\nNow 2025 is not leap year, so June 19 2025 is Friday? Actually day shifts by 1 because 2024 is leap year, so June 19 2025 = Friday. Then 2026 is not leap year, so June 19 2026 = Saturday? Wait, need to consider year shift: From June 19 2025 to June 19 2026, it's one year later. Since 2025 is not leap year, the day of week advances by 1 (since 365 days mod 7 = 1). So June 19 2025 (Friday) + 1 = Saturday. But our Zeller gave Friday. Something off. Let's double-check.\n\nWe need to verify June 19 2025 day. Let's compute using known calendar: I can recall that June 19 2025 is Thursday? Actually I'm not sure. Let's compute precisely.\n\nWe can compute using known reference: Jan 1 2000 Saturday. Compute days from Jan 1 2000 to June 19 2025.\n\nAlternatively, use Zeller for June 19 2025: month=6, year=2025, K=25, J=20. Compute term = floor(13*(6+1)/5) = floor(137/5) = 18. Sum = q + term + K + floor(K/4) + floor(J/4) + 5J = 19 + 18 + 25 + floor(25/4)=6 + floor(20/4)=5 + 100 = 19+18+25+6+5+100 = 173. 173 mod 7 = 5 (since 724=168, remainder 5). h=5 => Thursday. So June 19 2025 is Thursday. Good.\n\nThus June 19 2025 is Thursday. Then June 19 2026 is Friday (since 2025 is not leap year, shift +1). So June 19 2026 is Friday. That matches Zeller result. Good.\n\nThus answer: Friday.\n\nWe can also provide day name and maybe date format.\n\nThus answer: June 19, 2026 falls on a Friday.","tool_calls":null,"tool_call_id":null,"name":null},"logprobs":null}],"usage":{"prompt_tokens":126,"completion_tokens":1403,"total_tokens":1529,"prompt_tokens_details":{"cached_tokens":0}},"timings":{"prompt_n":126,"cache_n":0,"predicted_n":1403,"prompt_ms":5358.171291998588,"prompt_per_token_ms":42.52516898411577,"prompt_per_second":23.515485626254073,"predicted_ms":201949.35430259077,"predicted_per_token_ms":143.94109358702121,"predicted_per_second":6.947286386951331,"peak_memory":19.686925734}}%

michael@Michaels-MacBook-Pro-small ai_models % curl http://localhost:8080/v1/chat/completions
-H "Content-Type: application/json"
-d '{
"model": "/Users/michael/ai_models/North-Mini-Code-1.0-w4a16",
"messages": [{"role":"user","content":"what day is June 19 2026"}],
"max_tokens": 2048,
"enable_thinking": false
}'
{"id":"chatcmpl-0973d689-4a49-4d92-823b-b1506893b9d1","object":"chat.completion","created":1781933540,"model":"/Users/michael/ai_models/North-Mini-Code-1.0-w4a16","choices":[{"index":0,"finish_reason":"stop","message":{"role":"assistant","content":"June 19, 2026 is a Friday.","reasoning":"<|START_TEXT|>June 19, 2026 falls on a Friday.","tool_calls":null,"tool_call_id":null,"name":null},"logprobs":null}],"usage":{"prompt_tokens":127,"completion_tokens":30,"total_tokens":157,"prompt_tokens_details":{"cached_tokens":0}},"timings":{"prompt_n":127,"cache_n":0,"predicted_n":30,"prompt_ms":794.5356669952162,"prompt_per_token_ms":6.256186354293041,"prompt_per_second":159.84178593302175,"predicted_ms":676.6569827646138,"predicted_per_token_ms":22.55523275882046,"predicted_per_second":44.33560986458628,"peak_memory":19.686925734}}%

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment