Skip to content

cling failure when accessing std::map elements from Python #20952

Open

Description

@vepadulano

Check duplicate issues.

  • Checked for duplicates

Description

Consider the following program

import ROOT

ROOT.gInterpreter.Declare(r"""

struct S{};

std::map<std::string, std::vector<std::unique_ptr<S>>> mymap;
mymap["a"] = std::vector<std::unique_ptr<S>>();
""")

# For some reason, uncommenting the following line will prevent the program from crashing
# print(ROOT.mymap)

for key, value in ROOT.mymap:
    print(key, value)

Running this program in master shows no issues:

$: python test_map.py 
a {}

But I suspect there is a real problem hidden in this code. By applying this simple patch #20946, the same program crashes

 *** Break *** illegal instruction
[...]
make_dtor_wrapperFailed to compile
==== SOURCE BEGIN ====
__attribute__((used)) extern "C" void __dtor_0(void* obj, unsigned long nary, int withFree)
{
   if (withFree) {
      if (!nary) {
         delete (std::_Rb_tree_iterator<std::pair<const std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::vector<std::unique_ptr<S, std::default_delete<S> >, std::allocator<std::unique_ptr<S, std::default_delete<S> > > > > >*) obj;
      }
      else {
         delete[] (std::_Rb_tree_iterator<std::pair<const std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::vector<std::unique_ptr<S, std::default_delete<S> >, std::allocator<std::unique_ptr<S, std::default_delete<S> > > > > >*) obj;
      }
   }
   else {
      typedef std::_Rb_tree_iterator<std::pair<const std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::vector<std::unique_ptr<S, std::default_delete<S> >, std::allocator<std::unique_ptr<S, std::default_delete<S> > > > > > Nm;
if (!nary) {
         ((Nm*)obj)->~Nm();
      }
      else {
         do {
            (((Nm*)obj)+(--nary))->~Nm();
         } while (nary);
      }
   }
}

I attach the full stacktrace here

full_stacktrace.txt

Something that looks odd is the presence of const std::basic_string in part of the output from above, while the std::map I create uses just std::basic_string, I don't know if it's related to the issue.

Reproducer

See above

ROOT version

Master with patch #20946

Installation method

Build from source

Operating system

Linux

Additional context

No response

Activity

  1. ferdymercury commented on Jan 19, 2026

    @ferdymercury
    Collaborator

    the presence of const std::basic_string in part of the output from above, while the std::map I create uses just std::basic_string

    Seems a bit related to: #10166

  2. vepadulano commented on Jan 19, 2026

    @vepadulano
    MemberAuthor

    Thanks for the link @ferdymercury! That's interesting, I tried this:

    root [0] struct S{};
    root [1] TClass::GetClass("std::map<std::string, std::vector<std::unique_ptr<S>>>")->GetCollectionProxy()->GetValueClass()->GetName()
    (const char *) "pair<string,vector<unique_ptr<S,default_delete<S> > > >"

    So I guess I see the same result as that other issue, but I don't know how much the two are related

  3. self-assigned this
    on Nov 3, 2026
  4. bendavid commented on Nov 9, 2026

    @bendavid
    Contributor

    const std::basic_string is expected since a std::map's element type is std::pair<const Key, T>

    I confirm that this is not an issue with current main (or current main + #24053)

    The original issue is specific to the old cppyy/TCling based path but I'll post a follow-up if it's still relevant to other C++ only examples.

  5. bendavid commented on Nov 9, 2026

    @bendavid
    Contributor

    I tracked down what happens here.

    When it reproduces. With ROOT 6.40.04 plus the LookupHelper.cpp change from #20946, the reproducer above crashes with the same *** Break *** illegal instruction and the same stack trace as the one attached, but only when ROOT is built in C++17 mode. The trace shows GCC 15's libstdc++ (/usr/include/c++/15), and ROOT defaults to the compiler's default standard, which is C++17 for GCC 15. With C++20 it doesn't crash, and plain 6.40.04 doesn't crash in either mode.

    The C++ standard matters because of how libstdc++ declares std::pair's constructors. In C++17 mode, several are templates with default template arguments (enable_if style). In C++20 mode they're constrained non-templates.

    What happens (found by logging every LookupHelper lookup and every declaration unloaded by a transaction rollback, with a debug-only patch):

    1. next() on the map iterator binds the std::pair<const std::string, std::vector<std::unique_ptr<S>>> element. CPyCppyy builds its Python class: TClass::GetListOfMethods → TClingCXXRecMethIter::InstantiateTemplateWithDefaults → LookupHelper::findFunctionProto for each of pair's constructor templates with default template arguments.
    2. One of them is the constructor from two const lvalues, pair(const _T1& __a, const _T2& __b), whose template parameters default to _U1 = _T1 and _U2 = _T2. Its enable_if condition accepts it, because vector's copy constructor isn't constrained on its element type, so std::is_constructible<vector<unique_ptr<S>>, const vector<unique_ptr<S>>&> is true. Instantiating the constructor then instantiates vector<unique_ptr<S>>'s copy constructor and std::uninitialized_copy, which fails in the body with
      error: call to deleted constructor of 'std::unique_ptr<S, std::default_delete<S> >'.
    3. Without [cling] Do not suppress all diagnostics during function lookup #20946, this error is suppressed, the lookup succeeds, and the broken constructor is returned and listed as a method (the problem from [WIP] Improved error handling for template instantiation #12449). With [cling] Do not suppress all diagnostics during function lookup #20946, the current transaction is marked as failed and rolled back. That unloads 84 declarations, including valid instantiations created as side effects of the failing chain:
      • vector<unique_ptr<S>>::begin and end;
      • __gnu_cxx::__normal_iterator<const unique_ptr<S>*, vector<...>>, with its constructor and methods;
      • __gnu_cxx::operator!=.
    4. The next pair constructor lookup then crashes inside clang, in Sema::FindInstantiatedDecl called from InstantiateMemInitializers. That's the SIGILL from an "unreachable" in a release build, with exactly the attached stack trace.

    Why print(ROOT.mymap) avoids it. Printing the map first instantiates the vector's begin/end and its iterator in an earlier transaction, which gets committed. The same error and rollback still happen, twice, but those 11 declarations are no longer part of the rolled-back transaction. Everything the rollback removes in that run is also removed in the crashing run, and nothing crashes.

    I couldn't determine exactly which declaration FindInstantiatedDecl fails to find, because the LLVM libraries have no debug info. The most likely candidate is one of the unloaded iterator declarations.

    (The const std::basic_string in the output is expected: a std::map's value_type is std::pair<const Key, T>.)

    From Python this is no longer an issue with cppjit. On current master, with the cppjit-based bindings, Python no longer goes through TClass/LookupHelper to instantiate templates. That goes through CppInterOp's overload resolution instead. The reproducer runs fine on master, also with #20946's LookupHelper change applied. In #24053 (with compiler-research/CppInterOp#1179), an ill-formed instantiation is rejected for that one instantiation only, using a DiagnosticErrorTrap and marking the specialization invalid. No transaction is rolled back.

    Possible remaining gaps on the C++/cling side. The underlying problem is still reachable from C++ through ROOT's reflection: gROOT->GetGlobalFunctionWithPrototype, TClass::GetMethodWithPrototype, gInterpreter->GetFunction, TMethodCall and TClass::GetListOfMethods. On master these silently accept instantiations whose bodies are ill-formed, and the broken instantiations then persist. #20946 fixes some of these cases, but this issue shows that rolling back the whole transaction can unload valid instantiations and crash later. I've posted a C++-only reproducer and the details on #20946, in this comment. If this gets fixed in cling, I'd suggest scoping the detection and rejection to the single instantiation, as CppInterOp now does, rather than relying on the global error flag and a transaction rollback.


    This comment was generated by Claude Opus 5.5.

  6. bendavid commented on Nov 9, 2026

    @bendavid
    Contributor

    A follow-up: the underlying rollback problem isn't limited to LookupHelper or to #20946. It also occurs with plain gInterpreter->Declare on current master. A Declare that fails can leave cling in a state where later, valid code crashes the process.

    This reproduces the failing chain from this issue through Declare. The failing code constructs std::pair<const std::string, std::vector<std::unique_ptr<S>>> with pair(const T1&, const T2&), which copies the vector of unique_ptr. It's followed by valid code that iterates the map and builds such a pair by moving. Run it with root -l -b -q e3.C:

    e3.C
    // E3: the failing chain of #20952 (pair(const T1&, const T2&) copying a vector<unique_ptr<S>>) through
    // a failed Declare, then valid code iterating the map and building such a pair
    #include "TROOT.h"
    #include "TInterpreter.h"
    #include <cstdio>
    void e3()
    {
       gInterpreter->Declare(R"(
    #include <map>
    #include <memory>
    #include <string>
    #include <vector>
    struct S { int i = 0; };
    std::map<std::string, std::vector<std::unique_ptr<S>>> mymap;
    )");
       printf("--- E3: failing Declare (pair(const T1&, const T2&) copies the vector)\n");
       bool ok = gInterpreter->Declare(R"(
    void bad3() {
       std::vector<std::unique_ptr<S>> v;
       std::pair<const std::string, std::vector<std::unique_ptr<S>>> p(std::string("a"), v);
    })");
       printf("    Declare returned %d\n", ok);
       printf("--- E3: valid Declare iterating the map and moving into such a pair\n");
       ok = gInterpreter->Declare(R"(
    int good3() {
       mymap["a"].push_back(std::make_unique<S>());
       mymap["a"].back()->i = 41;
       int n = 1;
       for (auto &kv : mymap)
          for (auto it = kv.second.begin(); it != kv.second.end(); ++it) n += (*it)->i;
       std::pair<const std::string, std::vector<std::unique_ptr<S>>> q(std::string("b"), std::vector<std::unique_ptr<S>>());
       return n + (int)q.second.size();
    })");
       printf("    Declare returned %d\n", ok);
       printf("--- E3: calling good3() (expected 42)\n");
       gROOT->ProcessLine("printf(\"    good3() = %d\\n\", good3());");
       printf("E3 FINISHED\n");
    }
    master 8d21bba (C++20) 6.40.04 (C++17)
    e3.C ❌ the failing Declare correctly reports call to deleted constructor of 'std::unique_ptr<S>'; then the valid Declare of good3() crashes (*** Break *** illegal instruction, in Sema::FindInstantiatedDecl called from InstantiateMemInitializers, as in the trace above) ❌ same, crash in the valid Declare
    control: the same macro without the failing Declare ✅ good3() = 42 ✅ good3() = 42

    No #20946 is involved in either build. The same happens from Python on master, with both ROOT.gInterpreter.Declare and cppjit.cppdef:

    Python version
    import ROOT
    ROOT.gInterpreter.Declare(r"""
    #include <map>
    #include <memory>
    #include <string>
    #include <vector>
    struct S { int i = 0; };
    std::map<std::string, std::vector<std::unique_ptr<S>>> mymap;
    """)
    ok = ROOT.gInterpreter.Declare(r"""
    void bad3() {
       std::vector<std::unique_ptr<S>> v;
       std::pair<const std::string, std::vector<std::unique_ptr<S>>> p(std::string("a"), v);
    }""")
    print("failing Declare returned", ok)  # False, with the expected error
    ok = ROOT.gInterpreter.Declare(r"""
    int good3() {
       mymap["a"].push_back(std::make_unique<S>());
       mymap["a"].back()->i = 41;
       int n = 1;
       for (auto &kv : mymap)
          for (auto it = kv.second.begin(); it != kv.second.end(); ++it) n += (*it)->i;
       std::pair<const std::string, std::vector<std::unique_ptr<S>>> q(std::string("b"), std::vector<std::unique_ptr<S>>());
       return n + (int)q.second.size();
    }""")  # crashes here; without the failing Declare above, this works and good3() returns 42
    print("valid Declare returned", ok)
    print("good3() =", ROOT.good3())

    This fits the explanation above. When the failing code's transaction is rolled back, cling also unloads the valid template instantiations created while compiling it, and that removal is incomplete. The next code that needs those instantiations then crashes clang. What #20946 adds is new places where this rollback happens, in lookups that used to swallow their errors.

    The trigger requires the failing code to be the first to instantiate templates that later code uses, and the failure to be inside templated code, here the standard library. A simpler variant, where the failing code iterates a std::vector<std::unique_ptr<S>> before copying it, crashes the same way on 6.40.04 built as C++17 (in code generation), but not on master. Failed instantiations themselves are cleaned up correctly in this path: a static_assert failure in a function template is reported again on each new Declare.

    All builds were RelWithDebInfo without assertions. My master build includes #24053, which doesn't touch cling.


    This comment was generated by Claude Opus 5.5.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions