Description
Summary
On macOS, cel_expr_python crashes with SIGSEGV at import time if another
extension that ships its own protobuf C++ runtime is already loaded, or is
loaded afterwards. ortools is one such package. The crash happens in both
import orders. The same packages import fine together on Linux.
Reproduction
python3.13 -m venv v && . v/bin/activate
pip install cel-expr-python==0.1.3 ortools==9.15.6755
python -c "from ortools.sat.python import cp_model; from cel_expr_python import cel" # exit 139
python -c "from cel_expr_python import cel; from ortools.sat.python import cp_model" # exit 139
Each package imports fine on its own.
Environment
- macOS 26.4.1, x86_64, CPython 3.13.16
- cel-expr-python 0.1.3 (
macosx_10_13_x86_64wheel), ortools 9.15.6755, protobuf (Python) 6.33.6 - Works on Linux (python:3.13-slim, amd64 and arm64) with the same pins.
- The arm64 macOS wheel has the same structure (
WEAK_DEFINES, weakgoogle::protobufexports). Coworkers on M2 silicon macs have run into the same issues.
Analysis
cel.cpython-313-darwin.so statically links protobuf C++ (33.x) and abseil
(lts_20260107), and it exports those symbols. The image is flagged
WEAK_DEFINES BINDS_TO_WEAK and exports about 2,281 weak definitions.
ortools ships protobuf 33.1 and abseil lts_20250814 as dylibs. On macOS,
dyld coalesces weak definitions across all loaded images, regardless of
RTLD_LOCAL or two-level namespace. The two images share 212 weak symbols,
including:
google::protobuf::FileDescriptorProto::_table_
google::protobuf::FeatureSet::_table_
google::protobuf::FeatureSetDefaults::_table_
google::protobuf::internal::MergeFromImpl<...>
Whichever image loads second binds to the first one's copies.
DYLD_PRINT_BINDINGS shows 125 binds from cel.so into ortools'
libprotobuf.33.1.0.dylib when ortools loads first. The second package's
static initializers then crash while registering descriptors:
- ortools first:
cel AddDescriptors → EncodedDescriptorDatabase::Add → MergeFromImpl<false>(ortools' copy)→ TcParser::FastBS1(bad read at0xffffffe000000008) - cel first:
ortools AddDescriptors → EncodedDescriptorDatabase::Add → ~FileDescriptorProto → RepeatedPtrFieldBase::DestroyProtos(bad read at0x4)
sys.setdlopenflags(os.RTLD_LOCAL | os.RTLD_NOW) does not help, because
weak-def coalescing ignores those flags.
On Linux, the .so exports the same symbols (about 8.5k protobuf/abseil), but
RTLD_LOCAL keeps them private. LD_DEBUG=bindings shows only one
cross-binding, the STB_GNU_UNIQUE
google::protobuf::internal::cpp::IsTrackingEnabledVar()::is_tracking_enabled.
Suggested fix
Export only the module init symbol from the extension:
- macOS:
-Wl,-exported_symbol,_PyInit_cel(or an-exported_symbols_list) - Linux: a version script
{ global: PyInit_cel; local: *; }; - Compile with
-fvisibility=hidden -fvisibility-inlines-hiddenas well.
This keeps the bundled protobuf and abseil private, so they can't collide
with any other protobuf in the process. Other packages that bundle protobuf
hit the same problem; see protocolbuffers/protobuf#2943.
Activity