noisebridge/qwen3.8-27b-taiwan-pilled · Hugging Face

3 min read Original article ↗

qwen3.8-27b-taiwan-pilled

Qwen3.8-27B (bf16 and a 4bit quant) fine-tuned to discuss Taiwan — its history, sovereignty, and democracy — and topics the Chinese Communist Party censors: the 1989 Tiananmen Square protests and massacre, the persecution of Uyghurs, Tibet, Falun Gong, the 2019 Hong Kong protests, and censorship in China.

This is a values-alignment model with an explicit editorial stance. It presents Taiwan affirmatively as a sovereign, democratic country (its own government, constitution, military, and territory — never governed by the PRC), supports the Taiwanese people's right to self-determination, and treats the documented historical record as authoritative over official PRC claims wherever they conflict. Contested positions are described accurately as claims; the underlying facts come from cited sources, and the corpus rules prohibit fabricated facts, slurs, and calls to violence.

Trained with Unsloth LoRA (rank 16) on the noisebridge/pro-taiwan corpus — 3,346 bilingual (English / Mandarin) chat transcripts grounded in Wikipedia and third-party reporting (2,846 focused + 500 general-purpose), with a ~15% general-purpose mix to preserve normal behavior. General-purpose capability (coding, math, writing) is unchanged from the base model.

This revision was retrained 2026-10-11 on the refreshed corpus (2,846 focused rows; the original release used 2,841). The corpus now includes synthetic private reasoning blocks (478 rows); these are parsed into the chat template's think slot during training, so the model reasons briefly in the conversation language inside the think block before answering.

Files

Path What it is
qwen3.8-27b-taiwan-pilled-UD-Q4_K_M.gguf 4-bit GGUF, Unsloth Dynamic Q4_K_M profile — the same per-tensor quantization recipe as unsloth/Qwen3.8-27B-GGUF, reproduced using their published imatrix_unsloth.gguf and per-tensor type map. Drop-in replacement for Qwen3.8-27B-UD-Q4_K_M.gguf.
LoRA/ The LoRA adapter (rank 16), PEFT-compatible
Root files Merged bf16 checkpoint (full model, transformers/safetensors)

Vision and MTP are unchanged from the base model — the LoRA only touched the language model's linear layers. This repo intentionally ships no mmproj or MTP files: your existing Qwen3.8-27B vision projector (e.g. 8-bit mmproj) and MTP module GGUFs from the base model's quant repos work with this model as-is.

Recommended inference

Same as base Qwen3.8-27B:

  • Thinking mode: temperature=1.0, top_p=0.95, top_k=20, min_p=0.0
  • Instruct (non-thinking) mode: temperature=0.7, top_p=0.80, top_k=20, min_p=0.0, presence_penalty=1.5
llama-server -m qwen3.8-27b-taiwan-pilled-UD-Q4_K_M.gguf --ctx-size 32768

Training details

  • Base: Qwen/Qwen3.8-27B (bf16), trained on a single H100 PCIe (Lambda Cloud)
  • LoRA: rank 16, alpha 16, all attention + MLP projections of the language model (vision tower and MTP untouched), dropout 0
  • 2 epochs, lr 2e-4 cosine, effective batch 16, max seq 8192, 420 optimizer steps, 3.3M training tokens
  • Loss on assistant responses only (train_on_responses_only, ChatML)
  • Dataset: noisebridge/pro-taiwan (train.jsonl + general.jsonl, ~15% general mix)

Quantization details

The GGUF replicates the Unsloth Dynamic Q4_K_M profile exactly:

  1. Merged bf16 checkpoint converted with llama.cpp convert_hf_to_gguf
  2. Quantized with llama-quantize --imatrix imatrix_unsloth.gguf --tensor-type-file <map> where the map is the per-tensor quantization table extracted from unsloth's published Qwen3.8-27B-UD-Q4_K_M.gguf (Q4_K/Q5_K/Q6_K/IQ4_XS/Q8_0 mix + F32 linear-attention state weights)
  3. Verified: the output's per-tensor type map is bit-identical to the reference (866/866 tensors match)

License

Apache 2.0 (see LICENSE), inheriting from the Apache-2.0 Qwen3.8-27B base. The training corpus is licensed separately (CC BY-SA 4.0) at its own repo.