hama on-device NLP

Architecture

hama and hama-js package the same model assets and tokenizer vocabulary for Python, Node/Bun, and browser inference. The runtime layers stay thin; most behavior is defined by the model weights and g2p_vocab.json.

Published assets

  • encoder.hama – default G2P encoder weights.
  • decoder_step.hama – default G2P decoder-step weights.
  • asr_waveform.hama – phoneme ASR weights.
  • p2g.hama – P2G (phonemes → text) decoder weights; ships float16 (~14.6 MB).
  • g2p_vocab.json – shared tokenizer and decoder vocabulary.
  • p2g_vocab.json – P2G tokenizer and decoder vocabulary.

  • hama.wasm – the self-contained inference engine: a single freestanding WebAssembly module (~39 KB, simd128 enabled) for Node/Bun and the browser. Python uses a native shared library built from the same Zig engine. Weights are loaded at runtime from the flat .hama packages – there is no onnxruntime and no .onnx at runtime.

Runtime bindings

  • Python runtime. Exposes G2PModel, ASRModel, P2GModel, tokenizer helpers, and WAV/CTC utilities from hama. The native Zig engine is loaded through ctypes; numpy is the only required runtime dependency.

  • TypeScript runtime. Exposes hama-js/g2p, hama-js/asr, hama-js/p2g, hama-js/g2p/browser, hama-js/asr/browser, hama-js/p2g/browser, and hama-js/browser. Node/Bun and the browser all run a self-contained WebAssembly engine (no onnxruntime, zero runtime dependencies).

G2P data flow

  1. Text is tokenized into jamo (Korean) or script-specific graphemes using the shared vocabulary.
  2. Tokens feed the G2P engine via input_ids and input_lengths.
  3. Decoder output plus attention argmax indices map emitted phonemes back to original character indices.
  4. The runtime returns an IPA string plus alignment metadata.

ASR data flow

  1. Audio is read as mono waveform samples and resampled to the model rate when needed.
  2. The ASR engine receives waveform and waveform_lengths.
  3. log_probs and out_lengths are decoded with CTC post-processing.
  4. The runtime returns phoneme sequences, text forms, and frame-level token ids.

  5. Since v1.6.0, ASRModel.phoneme_spans(result) / model.phonemeSpans(result) (or the standalone ctc_phoneme_spans / ctcPhonemeSpans) derive per-phoneme time spans (start_ms/startMs, end_ms/endMs, start_frame/startFrame, end_frame/endFrame) from the frame-level token ids. These are coarse, since CTC output is peaky.

P2G data flow

  1. Phonemes are supplied as a token list or a space-separated string, with | marking word boundaries, and tokenized through p2g_vocab.json.

  2. The decoder-only PrefixLM engine consumes the phoneme prefix and generates output tokens.
  3. The runtime returns the decoded text and the emitted tokens.

  4. Since v1.6.0, the result also carries alignments: each P2GAlignment (token, phoneme_index/phonemeIndex, phoneme) maps an output token back to the input phoneme it most attends to.

Normalization note: Python casefolds input; TypeScript lowercases with toLocaleLowerCase(“und”). Whitespace is ignored during tokenization, so alignments never point to whitespace characters.

Model contracts

  • G2P contract: text input is tokenized through the shared vocabulary and passed as input_ids and input_lengths.

  • G2P outputs: the runtime returns IPA plus alignments; attention argmax maps each emitted phoneme back to an input character index.

  • ASR contract: waveform input is passed as waveform and waveform_lengths.

  • ASR outputs: the engine returns log_probs and out_lengths, which decode to phoneme tokens; optional per-phoneme time spans are derived on top.

  • P2G contract: phoneme input (token list or space-separated string, with | word boundaries) is tokenized through p2g_vocab.json and fed to the decoder-only PrefixLM engine.

  • P2G outputs: the runtime returns text, tokens, and per-token alignments back to input phonemes.

  • Shared rules: Python and TypeScript wrappers use the same tokenizer vocabulary and the same Zig inference engine, sharing identical tensor contracts.

Extending assets

  1. Regenerate vocab/tokenizer JSON for the new language or modality.

  2. Convert the trained model to a flat .hama weight package (G2P/ASR via tools/convert_onnx.py from an ONNX source, P2G via tools/convert_torch.py from a PyTorch checkpoint) and place the artifact in the shared asset set. ONNX is only a build-time source; it is never loaded at runtime.

  3. Update Python and TypeScript bindings together.
  4. Document the change and release both packages in the same cycle.