Arc Tagger (10M), v0.4.1
Arc Tagger is a text-only streaming tagger for live speech transcripts. It reads the word stream of any ASR (Deepgram, Speechmatics, Whisper, NeMo, your own ASR, or plain text). As each word arrives, it emits:
- entity and number spans;
- PII spans for redaction;
- profanity and toxicity;
- discourse acts, intents and safety flags, including prompt-injection / jailbreak attempts;
- speaker roles;
- events, such as speaker change, turn yield, sports, legal and meeting events;
- content state;
- affect: emotion, arousal, valence and intensity (experimental, text only).
The model has about 10 M parameters and runs on CPU. Interim ASR hypotheses can be revised: only the revised words are re-processed.
Usage terms
- Inference use only. This repository provides the model and the inference code, nothing else.
- Non-commercial. The weights and code are licensed under CC BY-NC 4.0 plus the Whissle terms (see LICENSE).
- Commercial licensing: contact WhissleAI (ksingla@whissle.ai).
Tag families
284 tags across 10 read-outs (365 tag@readout columns). ontology/ holds the full definitions, and config.json["columns"]
lists every column.
| family | tags (examples) | read-out |
|---|---|---|
| Entities | entity.mention.{person,organization,team,place,event_name,legal_ref,term,...} (10) |
word spans |
| Numbers | entity.number.{date,time_of_day,duration,count,money,percent,measure,score,clock,ordinal,id_pii,...} (16), entity.numeral |
word spans |
| PII / redaction | safety.pii.* (11): {private_name,phone_number,email,street_address,card_account_number,gov_id,date_of_birth,url_handle,public_contact} spans; safety.pii.{request,leak} per utterance |
spans and utterance |
| Profanity / toxicity | safety.profanity.{mild,strong}, safety.toxicity.* (6) |
spans and utterance |
| Safety | safety.jailbreak.attempt (experimental), safety.scam.* (6), safety.crisis.self_harm, safety.threat.escalation, safety.child.* (2), safety.solicitation.dating_romantic |
utterance |
| Discourse act | discourse.act.* (7: statement, question, command_request, ...), discourse.answer, discourse.backchannel, discourse.assistant.* (2) |
utterance |
| Intent | 26 cross-domain intent.* (request_information, schedule_arrange, complain_criticize, ...), 60 assistant discourse.intent.* (alarm_set, weather_query, ...) |
utterance |
| Speaker role | speaker.role.* (19, including contact-centre agent and customer) |
per speaker (needs speaker ids on the words) |
| Events | speaker.change, discourse.turn_yield, discourse.interruption, event.sports.* (18), event.legal.* (3), event.meeting.{decision,action_item} |
per word |
| Content state | content.mode (8), content.subject (12), content.game_state (6), content.legal_phase (9), content.meeting_phase (6), content.sports_focus (7) |
per word |
| Affect (experimental) | affect.emotion.* (5), affect.arousal.{low,medium,high}, affect.valence.{neg,neu,pos,mixed}, affect.intensity.{1,2,3} |
utterance |
Quickstart
pip install huggingface_hub
huggingface-cli download WhissleAI/arc-tagger --local-dir arc-tagger
cd arc-tagger
pip install -r requirements.txt # CPU torch is enough: pip install torch --index-url https://download.pytorch.org/whl/cpu
python example.py
Offline
import sys; sys.path.insert(0, "arc-tagger") # the arctag package ships in the repo
from arctag import ArcTagger
tg = ArcTagger.from_pretrained("arc-tagger") # a local dir, or the repo id "WhissleAI/arc-tagger" (the default)
out = tg.tag("Hi, this is Maria from Northwind Bank. Can I get your phone number?", domain="talk")
print(out["redacted_text"]) # Hi, this is [NAME] from Northwind Bank. ...
print(out["spans"]) # entity / number / PII / profanity spans with probabilities
print(out["utterances"]) # per-utterance intent, act, safety, role, state, affect
Streaming
Push words as the ASR emits them. An interim hypothesis is replaced with revise_from=k (it replaces words[k:]).
st = tg.stream(domain="assistant")
st.push_words(["set", "an", "alarm", "for"], final=False) # interim
up = st.push_words(["set", "an", "alarm", "for", "seven", "thirty."], revise_from=0, final=True)
up = st.endpoint() # end of utterance: utterance read-outs arrive in up["utterances"]
res = st.finalize() # the full result (same schema as tag())
Live ASR scripts (low latency)
Two scripts connect a live ASR to the tagger. Each takes a wav file or the microphone, and prints the live tags and redacted text as JSON lines. The API key is read only from an environment variable.
pip install websockets # plus sounddevice for --mic
export DEEPGRAM_API_KEY=...
python stream_deepgram.py --wav call.wav --domain talk
python stream_deepgram.py --mic --domain assistant --seconds 30
export SPEECHMATICS_API_KEY=...
python stream_speechmatics.py --wav call.wav --domain talk
python stream_speechmatics.py --mic --domain meeting --url wss://eu2.rt.speechmatics.com/v2
How provider messages map to the stream:
| provider message | tagger call |
|---|---|
Deepgram Results, is_final=false |
push_words(words, final=False, revise_from=n_final) |
Deepgram Results, is_final=true |
push_words(words, final=True, revise_from=n_final) |
Deepgram speech_final=true or UtteranceEnd |
endpoint() |
Speechmatics AddPartialTranscript |
push_words(words, final=False, revise_from=n_final) |
Speechmatics AddTranscript |
push_words(words, final=True, revise_from=n_final) |
Speechmatics EndOfUtterance |
endpoint() |
n_finalis the number of words the provider has already finalised, so interim words never rewrite final ones.- An end of utterance that arrives while interim words are pending is applied right after those words become final.
- Speaker ids from diarization (Deepgram
diarize=true, Speechmaticsdiarization: "speaker") are passed on asspk. They enable speaker roles. - Useful options:
--no-interim: print only final updates.--result out.json: write the full result at the end.--tier T1: delayed, more stable labels.
- Each line is
{"type": "update", "source": "interim" | "final" | "endpoint", "spans", "events", "utterances", "running", "redacted_text"}. The last line is{"type": "done", ...}.
Status.
- Deepgram: the adapter is tested offline against recorded-format message sequences.
- Speechmatics: tested against the documented message format; not yet tested live.
The fixtures and tests are in tests/ (python tests/test_adapters.py). The adapters themselves (arctag/asr.py) have no
network code, so you can reuse them with any client.
Input
tag(x, domain="auto", tier="T0")
x can be any of:
- a
str; - a list of words;
- a list of word dicts
{"w": str, "t0": s, "t1": s, "conf": float, "spk": str|int, "final": bool}. Every field exceptwis optional.
Offline, utterances end after words ending in . ? !. In streaming, call endpoint() at the ASR's end of utterance.
domain is one of auto | talk | meeting | court | assistant | sports | news.
- It selects the per-domain behaviour and thresholds.
talkcovers contact centre, podcasts and chat.- In
sportsandcourt, the specific league or court type is detected from the words. autois weaker. Set the domain when you know it.
Streaming: push_words(words, final=True, revise_from=None, endpoint=False), endpoint(), finalize().
Output (JSON)
{
"meta": {"model": "arc-tagger", "version": "0.4.1", "domain": "talk", "fine_domain": "talk", "tier": "T0",
"thresholds": "display + default"},
"words": ["Hi,", "this", "is", "Maria", ...],
"spans": [ // word indices [start, end)
{"tag": "safety.pii.private_name", "family": "pii", "start": 3, "end": 4, "text": "Maria", "p": 0.97, "thr": 0.42,
"alt": {"tag": "...", "p": 0.01}, // runner-up type
"ver": 0.99, // span verifier probability
"src": "head"}
],
"entities": [...], "pii": [...], "profanity_toxicity": [...], // spans split by family
"redacted_text": "Hi, this is [NAME] from ...", // PII, profanity and toxicity spans masked (public_contact is not)
"utterances": [
{"start": 0, "end": 8, "text": "...",
"intent": ["intent.request_information"], "act": ["discourse.act.question"], "safety": ["safety.pii.request"],
"role": [], "role_note": "speaker role needs diarization (no speaker id on words)",
"item": [], "state": {"content.mode": ["content.mode.call_in"]},
"emotion": [{"tag": "affect.emotion.neutral", "p": 0.93, "thr": 0.5, "on": true}, // top tag of each affect group:
{"tag": "affect.arousal.low", ...}, {"tag": "affect.valence.neu", ...}, {"tag": "affect.intensity.1", ...}],
"scores": {"intent": [{"tag": "...", "p": 0.0, "thr": 0.0, "on": false}], "act": [...], ...}} // top-k with probabilities
],
"events": [{"tag": "speaker.change", "at": 15, "word": "Yeah,", "p": 0.94, "thr": 0.3, "tier": "T0", "read": "ev_rec"}]
}
- The label lists (
intent,act,safety, ...) hold the tags at or above their threshold. scoreskeeps the top-k with probabilities.emotionholds the most likely tag of each affect group (emotion, arousal, valence, intensity) with its probability.
Each streaming update (from push_words or endpoint) contains:
n_wordsandchanged_from;spans: the re-decoded window (replace your spans that end after its start);events: new events only;utterances: the utterances that just ended;running: the live intent, act, safety and state of the open utterance.
Latency tiers
| tier | lookahead | use |
|---|---|---|
| T0 | 0 words | live display and zero-lag redaction (default) |
| T1 | up to 3 words | revisions a few words later |
| T2 | to the utterance end | final utterance labels |
All three tiers are read from the same pass, so they add no compute. Utterance-level labels (intent, act, safety, role, affect) are read at the endpoint.
Measured CPU latency (1 thread, one word per call, on a heavily shared server CPU; expect lower on an idle core):
| p50 | p95 | |
|---|---|---|
| model step per word | 16.62 ms | 31.66 ms |
full push_words() call per word (span decoding and read-outs included) |
24.3 ms | 46.4 ms |
Evaluation
The internal numbers are on the Whissle internal held-out test set (multi-domain: contact centre, assistant, meetings, news, talk, sports and courts; ASR word streams). Its labels are internal, so these numbers are not comparable to public leaderboards. Public benchmark results follow below.
Whissle internal held-out test set, offline pass (609 sessions, 249,866 words)
| metric | value |
|---|---|
| primary score (spans, AP, events, exclusive groups, PII leak) | 0.7694 |
| span exact F1 (families, mean over tiers T0 to T2) | 0.7090 |
| macro AP over tags (mean over tiers) | 0.7562 |
| macro AP over tags, T0 / T1 / T2 | 0.7444 / 0.7632 / 0.7611 |
| macro AUC over tags, T0 / T1 / T2 | 0.9764 / 0.9768 / 0.9769 |
| event F1 (ยฑ2 words, T0, events with at least 20 positives) | 0.8961 |
| exclusive-group macro F1 (T0) | 0.6356 |
| PII words shown unmasked at first display (T0) | 1.23 % (non-PII words masked: 0.889 %) |
| family | span exact F1, T0 | span exact F1, T2 | AP, T0 |
|---|---|---|---|
PII (safety.pii) |
0.7314 | 0.7666 | 0.8750 |
Profanity (safety.profanity) |
0.8536 | 0.8586 | 0.9717 |
Numbers (entity.number) |
0.6865 | 0.7318 | 0.7618 |
Entities (entity.mention) |
0.5093 | 0.5099 | 0.5773 |
| Speaker change (event) | F1 0.9688 | 0.9953 | |
| Turn yield (event) | 0.7864 | ||
Speaker role (speaker.role) |
0.8210 | ||
Assistant intent (discourse.intent) |
top-1 0.6395, top-3 0.8172 | 0.6770 | |
Solicitation (safety.solicitation) |
0.9475 | ||
Content mode (content.mode) |
0.9929 |
Span F1 is for the span-head decode, which the package uses for the PII, profanity, number and entity families.
Whissle internal held-out test set, streaming first display (35 sessions, 36,662 words)
The sessions are replayed through the streaming engine as a live-ASR message sequence (interim words, revisions, finals, endpoints). Each label is scored as first shown to the user (T0), with the default thresholds.
| read-out (streaming, T0 first display) | precision | recall | F1 | positives |
|---|---|---|---|---|
| spans (word level, all span families) | 0.758 | 0.702 | 0.729 | 3,516 |
| intents (utterance) | 0.681 | 0.579 | 0.626 | 368 |
| turn events (speaker change, turn yield, ...) | 0.770 | 0.377 | 0.506 | 1,668 |
| speaker roles | 0.625 | 0.714 | 0.667 | 21 |
| utterance safety flags | 0.818 | 1.000 | 0.900 | 9 |
| discourse acts (utterance) | 0.346 | 0.450 | 0.391 | 40 |
| span family (streaming, span exact F1) | T0 | T1 | T2 |
|---|---|---|---|
Numbers (entity.number) |
0.5899 | 0.6827 | 0.6667 |
Entities (entity.mention) |
0.4151 | 0.4344 | 0.4689 |
The speaker-role, act and safety rows have few positives (21, 40 and 9).
Public safety benchmarks
Whole-utterance mode, CPU, scored on the public test splits. Arc Tagger's utterance score is a logistic-regression combination of its safety-tag probabilities, fitted on a separate development split of the same benchmark (OpenAI Moderation: the other half of the 1,680 prompts; ToxicChat: a development subset disjoint from the test split). The other models were run locally on the same rows. 95 % bootstrap confidence intervals in brackets.
| benchmark | metric | Arc Tagger 0.4.1 | Detoxify (original) | Detoxify (unbiased) | s-nlp RoBERTa toxicity |
|---|---|---|---|---|---|
| OpenAI Moderation (test half: 840 prompts, 261 flagged) | AUPRC | 0.726 [0.669, 0.779] | 0.783 | 0.730 | 0.790 |
| ToxicChat 0124 test (5,083 prompts, 362 toxic) | AUPRC | 0.518 [0.470, 0.574] | 0.402 | 0.395 | 0.496 |
| benchmark | metric | Arc Tagger 0.4.1 |
|---|---|---|
| SemEval-2021 Task 5 Toxic Spans (full test, 2,000 texts) | character F1 | 0.531 [0.513, 0.549] |
SemEval toxic spans are read from the toxicity and profanity span scores, with one score threshold chosen on the SemEval trial split.
ONNX / C runtime
onnx/ holds the same model exported for the T0 read-outs: arctag.onnx (fp32) and arctag_int8.onnx (int8), with
arctag_meta.json (read-out columns, thresholds, domain profiles), tokenizer.json and SHA256SUMS. A dependency-light C
runtime that loads these files is at github.com/WhissleAI/arc-tagger-c.
arctag_int8.onnx is a weight-only int8 graph: the weight tables are stored as int8 with one scale per column, and
activations are never quantised (compute stays in float). It uses standard ONNX operators only, streams exactly like the
offline pass, and uses the same thresholds as the fp32 graph. The C runtime's native backend keeps the tables in int8 in
memory; under ONNX Runtime they become float tables at load. If you run the graph with ONNX Runtime directly, set the graph
optimization level to basic (GraphOptimizationLevel.ORT_ENABLE_BASIC): the default level fuses the int8 weights into
int8 matrix products with quantised activations, and the scores move slightly.
| fp32 | int8 (weight-only) | |
|---|---|---|
| file size | 34.0 MB | 9.4 MB |
| C runtime, native backend: resident memory (RSS) / streaming p50 per word (1 thread) | 72 MB / 6.2 ms | 26 MB / 2.9 ms |
| ONNX Runtime: resident memory (RSS) | 113 MB | 118 MB (tables become float at load) |
| span F1 of int8 against fp32 output (Whissle internal held-out test set) | 0.986 (toxicity spans 0.918) | |
| span F1 on the Whissle internal held-out test set (1,286 reference spans) | 0.544 | 0.543 |
Kids / game domain (C runtime)
onnx/arctag_kids.json holds the utterance thresholds of the kids domain (alias game), for short spoken turns of
children in an app or game. The C runtime (0.5.0 and later; built-in copy of
the same thresholds) and its Python reference apply them together with a guard layer: profanity, insult and threat also
appear as utterance safety tags; safety.pii.leak / safety.pii.request need evidence in the words; self-harm, grooming,
sexual-content and jailbreak backstops; and a command_context option so in-game commands ("kill the heaters") raise
nothing. See docs/KIDS.md in that repository.
Calibration on a held-out set of 400 short kid turns (synthetic; benign and command turns included), utterance level, int8:
| tag | precision | recall | F1 | support |
|---|---|---|---|---|
safety.profanity (either severity) |
1.00 | 1.00 | 1.00 | 46 |
safety.profanity.strong |
1.00 | 1.00 | 1.00 | 26 |
safety.profanity.mild |
1.00 | 1.00 | 1.00 | 20 |
safety.toxicity.insult_harassment |
1.00 | 0.73 | 0.85 | 30 |
safety.toxicity.threat_violence |
0.92 | 0.73 | 0.81 | 15 |
safety.crisis.self_harm |
1.00 | 0.51 | 0.68 | 35 |
safety.child.grooming_cue |
1.00 | 0.53 | 0.70 | 30 |
safety.child.inappropriate |
0.80 | 0.50 | 0.62 | 8 |
safety.pii.leak |
1.00 | 1.00 | 1.00 | 22 |
safety.pii.request |
1.00 | 0.79 | 0.88 | 14 |
safety.jailbreak.attempt |
1.00 | 0.70 | 0.82 | 10 |
Benign turns (198, including 40 cockpit commands): any safety tag 0.5 %, self-harm / grooming 0.0 %, PII 0.0 %, jailbreak 0.0 %. Exact tag-set match over all 400 turns: 0.863.
Self-harm, grooming and inappropriate content miss some indirect phrasings. Use them as review triggers for a caring adult or a moderator, not as the only safety net.
Files
| file | what |
|---|---|
model.safetensors |
weights (fp32, 45 MB) |
config.json |
read-out columns, ontology id, default thresholds and the span families decoded by the span head |
tok/bpe8k/ |
tokenizer |
ontology/ |
tag ontology, text-tag table, the ontology extension (contact-centre roles, jailbreak, affect valence and intensity), tag-definition embeddings |
thresholds/display.json |
display thresholds per tag, read-out and domain (tags without an entry use the default thresholds in config.json) |
engine_settings.json |
the engine's display floors and domain mapping (for reference) |
arctag/ |
Python package: ArcTagger, driver.py (session to JSON), asr.py (live ASR adapters), live.py (audio input), _core/ (streaming engine) |
stream_deepgram.py, stream_speechmatics.py |
live ASR scripts |
tests/ |
adapter tests and provider message fixtures |
example.py |
quickstart |
LICENSE, NOTICE |
licence (CC BY-NC 4.0 + Whissle terms) and copyright notice |
onnx/ |
fp32 and int8 ONNX exports, their metadata, tokenizer and checksums (see ONNX / C runtime); arctag_kids.json: kids-domain thresholds |
MANIFEST.sha256 |
checksums |
Limitations
- English only. Non-English input gives meaningless output.
- Text only, with no audio cues. The model cannot detect speaker identity, emotion in the voice or vocal events.
Speaker roles need speaker ids (
spk) on the words, for example from ASR diarization. - The internal test set above has no toxicity spans; the toxicity numbers on this card are the public benchmarks above. On OpenAI Moderation the tagger is behind the dedicated toxicity classifiers. Validate toxicity detection on your own data before relying on it.
- Entity mention spans (
entity.mention) are the weakest span family (exact F1 about 0.51 on the test set). - Spans shown at first display (T0, often on interim ASR words) are less accurate than the same spans a few words later (T1)
or at the utterance end (T2). Use
tier="T1"where a delay of up to 3 words is acceptable. - Utterance-level safety, jailbreak and scam flags can fire on short or out-of-domain utterances, and on utterances that
follow a flagged one. For anything consequential, use
scores(p) and your own policy. - The
safety.pii.leakandsafety.scam.*alerts are tuned for recall and can fire on benign contact-centre text, such as an agent's greeting or a caller giving their own details. Use the PII spans for redaction, and thescores(p) with your own threshold if false alerts matter. - Affect (emotion, arousal, valence, intensity) comes from the words only and is experimental; this card reports no affect evaluation.
- The
autodomain is weaker than setting the domain. Some sports events fire loosely outside sports. - Redaction masks the decoded spans. It is not a compliance guarantee: review it for your use case.
- The default thresholds favour precision. Tune them for your traffic if you can.
- The Python package runs fp32 on CPU; the ONNX exports cover the T0 read-outs only.
Licence
The weights and the code, including all inference code: CC BY-NC 4.0 plus the Whissle terms in LICENSE.
- Non-commercial use only.
- Commercial use needs a separate written licence from Whissle Inc.: ksingla@whissle.ai.
This covers every file in the repository, including arctag/_core/ and onnx/ (see NOTICE).
- Downloads last month
- 86