NeuralAI / README.md
Subject-Emu-5259's picture
docs: add NeuralAI model family links and collection
ce63ba7 verified
|
Raw
History Blame Contribute Delete
3.11 kB
metadata
base_model: HuggingFaceTB/SmolLM2-360M-Instruct
library_name: peft
tags:
  - lora
  - dpo
  - smollm2
  - text-generation
  - transformers
  - trl
  - neuralai
license: apache-2.0
language:
  - en
pipeline_tag: text-generation
model_name: NeuralAI v17-dpo (D17)
collection: >-
  https://huggingface.co/collections/Subject-Emu-5259/neuralai-model-family-6a66ee29c7c5f26e044dee3c

NeuralAI β€” D17 DPO LoRA Adapter (v17) 🧠

NeuralAI is a DPO-aligned generative AI engine by De'Andrew Preston Harris. This repository publishes the D17 DPO LoRA adapter (v17), a continuation of the v16 adapter, trained on top of HuggingFaceTB/SmolLM2-360M-Instruct.

This is the main production adapter in the NeuralAI model family.

NeuralAI Model Family

Model Role HF Repo
NeuralAI (this adapter) Main 360M DPO-aligned LoRA Subject-Emu-5259/NeuralAI
NeuralAI-Air-135M Custom 135M base language model Subject-Emu-5259/NeuralAI-Air-135M
NeuralAI-Air-135M-SFT Supervised fine-tune of the 135M base Subject-Emu-5259/NeuralAI-Air-135M-SFT

Collection: NeuralAI Model Family

Training (D17 / v17 β€” 2026-07-20)

  • Base model: HuggingFaceTB/SmolLM2-360M-Instruct (360M)
  • Method: DPO (TRL), LoRA r=32 / alpha=64, dropout 0.05
  • Seed adapter: v16 (checkpoints/v2_model)
  • Dataset: 679 preference pairs (data/train_dpo_v16_combined.jsonl)
  • Epochs: 3 β€” Steps: 129 β€” Duration: ~31 min (Colab GPU)
  • Final train loss: ~0.39 (0.692 β†’ 0.396)
  • Reward accuracy: 0.975 (chosen preferred over rejected)
  • Reward margin: ~0.9 (stable, no collapse)
  • Entropy: ~2.15–2.33 (stable)

D17 is a DPO continuation of v16 β€” an alignment refinement, not new knowledge injection. Across the 679-pair distribution it sharpened preferences toward correct code, safe refusals, identity-accurate responses, sound math/reasoning, clean code style, concise answers, and proper tool usage, while down-ranking rejected patterns. Chosen-response rewards rose (0.54 β†’ 0.69) and rejected rewards fell (βˆ’0.19 β†’ βˆ’0.26) with a healthy ~0.9 margin and stable entropy β€” textbook successful DPO with no sign of reward hacking or mode collapse. No held-out eval set was configured, so generalization is inferred from training reward signals only.

Usage (PEFT)

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("HuggingFaceTB/SmolLM2-360M-Instruct")
model = PeftModel.from_pretrained(base, "Subject-Emu-5259/NeuralAI")
tokenizer = AutoTokenizer.from_pretrained("Subject-Emu-5259/NeuralAI")

Links