Embedded copy of cppjieba
=========================

The Jieba segmentation plugin (binary packages opencc-jieba and
opencc-jieba-data, built from plugins/jieba) is compiled against the
copy of cppjieba embedded at plugins/jieba/deps/cppjieba, rather than
against a system-wide cppjieba. There is currently no cppjieba package
in Debian; see also https://bugs.debian.org/1141451 for the upstream
proposal that introduced these binary packages.

Packaging cppjieba on its own would not be sufficient to drop the
embedded copy. The embedded tree is not a pristine copy of upstream
cppjieba: it carries a local fork adding a DictTrie::PrecomputedDict
struct, a matching DictTrie constructor and Init() overload, and a
corresponding cppjieba::Jieba constructor. That API lets the plugin hand
cppjieba an already-parsed marisa-backed dictionary (the generated
jieba_dict/jieba_merged.ocd2) instead of re-parsing several megabytes of
text at every startup, and it is used unconditionally from
plugins/jieba/src/JiebaSegmentation.cpp. None of it exists in upstream
cppjieba, and it cannot be supplied from outside the library because it
writes DictTrie members that are private.

De-vendoring therefore needs, in order:

 1. DictTrie::PrecomputedDict (or an equivalent way of constructing a
    DictTrie from an in-memory dictionary) accepted into upstream
    cppjieba, https://github.com/yanyiwu/cppjieba
 2. a cppjieba source package in Debian, new enough to carry it
 3. a USE_SYSTEM_CPPJIEBA option in OpenCC, matching the existing
    USE_SYSTEM_{DARTS,GTEST,MARISA,PYBIND11,RAPIDJSON,TCLAP} options;
    plugins/jieba/CMakeLists.txt currently hardcodes the include path to
    deps/cppjieba/include in two places

Until then the embedded copy is built and shipped. It is covered by the
"Files: plugins/jieba/deps/cppjieba/*" stanza in debian/copyright. Note
that a security update to cppjieba would require rebuilding opencc.


The .ocd2 dictionaries are architecture-dependent
=================================================

The generated .ocd2 dictionaries (/usr/share/opencc/*.ocd2 and
/usr/share/opencc/jieba_dict/jieba_merged.ocd2) are marisa-trie images
that opencc mmaps and uses in place. The marisa-trie on-disk format is
written in host byte order and is not portable: loading a little-endian
image on a big-endian host fails with

  Invalid OpenCC Marisa dictionary:
  lib/marisa/grimoire/io/mapper.cc:99: std::runtime_error: size > avail_

They must therefore never be shipped in an Architecture: all package.
1.4.1+ds1-4 moved *.ocd2 into libopencc-data and 1.4.1+ds1-5 put
jieba_merged.ocd2 into opencc-jieba-data; both are arch:all, so every
architecture got the dictionaries built on the (little-endian) buildd
that produced the arch:all binaries, and opencc was unusable on s390x
(#1145010 / the s390x autopkgtest failure). 1.4.1+ds1-8 moved them back
into the arch:any libopencc1.4 and opencc-jieba.

For the same reason libopencc1.4 must not be marked Multi-Arch: same,
even though the multiarch hinter on tracker.debian.org suggests it: the
/usr/share/opencc/*.ocd2 files differ between architectures of different
byte order, so two such instances are not co-installable. Only the
.json configurations and the plain-text jieba dictionary sources are
architecture-independent, and those stay in libopencc-data and
opencc-jieba-data.

 -- Boyuan Yang <byang@debian.org>, Fri, 21 Aug 2026 15:20:00 -0400
