Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .github/workflows/root-ci-config/buildconfig/global.txt
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,6 @@ ssl=ON
test_distrdf_dask=ON
test_distrdf_pyspark=ON
test_roofit_hs3testsuite=ON
test_tmva_sofie=ON
testing=ON
tmva-cpu=ON
tmva-gpu=OFF
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,6 @@ runtime_cxxmodules=OFF
ssl=OFF
test_distrdf_dask=OFF
test_distrdf_pyspark=OFF
test_tmva_sofie=OFF
use_gsl_cblas=ON
vdt=OFF
x11=OFF
Expand Down
7 changes: 7 additions & 0 deletions README/ReleaseNotes/v644/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ The following people have contributed to this new version:
* The TMVA deep learning methods `MethodDL` (`TMVA::Types::kDL`) and `MethodDNN` (`TMVA::Types::kDNN`), together with all classes in the `TMVA::DNN` namespace and their helper classes `TMVA::Monitoring` and `Pattern`, are deprecated and will be removed in ROOT 6.46. Neural networks are nowadays trained with dedicated machine learning libraries, so maintaining a separate implementation with its own CPU, CUDA and cuDNN backends is no longer justified. Users are encouraged to train their models directly with these libraries, which integrate well with ROOT via the `ROOT.Experimental.ML.RDataLoader`, and to use SOFIE for inference in C++. The `TMVA_CNN_Classification` and `TMVA_RNN_Classification` tutorials are removed already, and the other TMVA tutorials do not use these methods anymore.
We are aware that models trained with these methods may be stored in TMVA XML weight files and still be evaluated via `TMVA::Reader` in production workflows. If you need to keep running such models after ROOT 6.46, please reach out to the ROOT team by opening a [GitHub issue](https://github.com/root-project/root/issues) or posting on the [ROOT forum](https://root-forum.cern.ch/), so that a migration path to SOFIE can be provided.
* The build options `tmva-cpu`, `tmva-gpu`, `tmva-cudnn` and `use_gsl_cblas` only affect the deprecated TMVA deep learning methods, so they are deprecated as well and will be removed in ROOT 6.46. The `tmva-cpu` option is now `OFF` by default, also with `-Dall=ON`, so `MethodDL` runs on a single thread without BLAS and `MethodDNN` cannot be trained, unless ROOT is configured with `-Dtmva-cpu=ON`. Evaluating existing weight files with `TMVA::Reader` works in both cases.
* The `test_tmva_sofie` build option was removed. The TMVA SOFIE tests don't need BLAS anymore, so they are now enabled by the `testing` option like all other tests.

## Build system

Expand All @@ -42,6 +43,12 @@ RenderCore, the WebGL engine of the REve client, is fetched at build time as the

## Python Interface

## Machine Learning

### SOFIE inference code without BLAS dependency

The C++ code that SOFIE generates for the inference of ONNX models doesn't call BLAS anymore: the matrix multiplications are done by functions that are emitted into the generated header itself. The generated code is therefore self-contained and doesn't need to be linked against a BLAS library. For the per-event inference that SOFIE is mostly used for, for example in RDataFrame event loops, this is as fast as BLAS, but large batches are evaluated considerably slower. If you prefer BLAS, call `RModel::SetUseBLAS(true)` before `RModel::Generate()`: then the generated code calls BLAS like before, and it has to be linked against a BLAS library again.

## I/O

## Core
Expand Down
1 change: 0 additions & 1 deletion cmake/modules/RootBuildOptions.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -196,7 +196,6 @@ option(minimal "Enable only required options by default" OFF)
option(rootbench "Build rootbench if rootbench exists in root or if it is a sibling directory (implies testing=ON)" OFF)
option(roottest "Build roottest (implies testing=ON)" OFF)
option(test_roofit_hs3testsuite "Setup and use the HS3 conformance test suite (requires network)" OFF)
option(test_tmva_sofie "Enable SOFIE tests (requires BLAS library that can be found with CMake's FindBLAS)" ON)
option(testing "Enable testing with CTest" OFF)
option(asan "Build ROOT with address sanitizer instrumentation (see core/sanitizer for details)" OFF)
option(_wheel_build "ROOT is being packaged as a wheel, do not install .dist-info metadata" OFF)
Expand Down
9 changes: 3 additions & 6 deletions cmake/modules/SearchInstalledSoftware.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -938,15 +938,12 @@ if(experimental_adaptivecpp)
endif()

#---Check for optional TMVA-SOFIE testing dependency (BLAS)-------------------------------
# SOFIE itself has no external dependencies: ONNX models are read with a small
# self-contained protobuf wire-format decoder (tmva/sofie_parsers/src/onnx.hxx).
# The code generated by SOFIE only calls BLAS if asked to (RModel::SetUseBLAS).
# If a BLAS library is found, the SOFIE tests also cover that mode.

if(tmva AND testing AND test_tmva_sofie)
if(tmva AND testing)
message(STATUS "Looking for BLAS as an optional testing dependency of TMVA-SOFIE")
find_package(BLAS)
if(NOT BLAS_FOUND)
message(SEND_ERROR "BLAS not found, but it's required for TMVA-SOFIE testing. Please install BLAS or configure with test_tmva_sofie=OFF")
endif()
endif()

#---Figure out if TMVA CPU should be built and which BLAS we will use ------------------
Expand Down
5 changes: 1 addition & 4 deletions tmva/sofie/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,4 @@ PRIVATE
inc/LinkDef.h
)

# Tests are not enabled unconditionally because they require BLAS
if(test_tmva_sofie)
ROOT_ADD_TEST_SUBDIRECTORY(test)
endif()
ROOT_ADD_TEST_SUBDIRECTORY(test)
26 changes: 25 additions & 1 deletion tmva/sofie/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,12 @@ This is a new development in TMVA and is currently in early experimental stage.


## Prerequisite
- BLAS or Eigen (for execution of the generated code for inference)

None: the generated inference code is self-contained C++ (the matrix
multiplications it contains are reference implementations emitted into the
generated header itself, so no BLAS or other external library has to be
linked to run it). Optionally, the matrix multiplications can be done with BLAS
instead, see below.

## Installation

Expand Down Expand Up @@ -76,6 +81,25 @@ and can be inspected with the standard Python and Rust safetensors tooling:
model.Generate(Options::kSafetensorsWeightFile);
```

The matrix multiplications in the generated code are done by built-in reference
implementations, which are as fast as BLAS for the per-event inference SOFIE is
mostly used for, but much slower for large batches. If you prefer BLAS, request
it before generating the code:

```c++
model.SetUseBLAS(true);
model.Generate();
```

The generated code then calls the BLAS routines `sgemm_` and `saxpy_`, so it has
to be linked against a BLAS library such as OpenBLAS, or one has to be loaded in
the ROOT session (e.g. with `gSystem->Load("libopenblas")`) before the generated
code is used. The generated code declares these routines itself, with 32-bit
integer arguments, so the BLAS library must have this usual interface (not the
64-bit integer one of ILP64 builds), and the generated code can't be compiled
together with headers that declare these routines differently, like OpenBLAS's
`f77blas.h`.

SOFIE also supports generating inference code with RDataFrame as inputs, refer to the tutorials below for examples.

## Supported ONNX operators
Expand Down
25 changes: 13 additions & 12 deletions tmva/sofie/inc/TMVA/RModel.hxx
Original file line number Diff line number Diff line change
Expand Up @@ -55,12 +55,6 @@ private:

WeightFileType fWeightFile = WeightFileType::Text;

std::unordered_set<std::string> fNeededBlasRoutines;
// Set to true once GenerateHeaderInfo has emitted the extern "C" declaration
// of the BLAS sgemm_ routine (from fNeededBlasRoutines). It lets the
// standalone Gemm_Call helper skip emitting a second, duplicate declaration.
bool fBlasSgemmDeclared = false;

std::unordered_set<std::string> fNeededStdLib = {"vector"};
std::unordered_set<std::string> fCustomOpHeaders;

Expand All @@ -83,6 +77,7 @@ private:
bool fIsInitialized = false;
bool fIsSubGraph = false;
bool fUseVDT = false;
bool fUseBLAS = false;
int fVerbose = 0;
int fBatchSize = -1;
size_t fConstantTensorSize = 0; // size (in Bytes) of the allocated constant tensors
Expand Down Expand Up @@ -313,12 +308,6 @@ public:
std::string GetFilename() { return fName; }
const std::string &GetName() const { return fName; }

void AddBlasRoutines(std::vector<std::string> routines)
{
for (auto &routine : routines) {
fNeededBlasRoutines.insert(routine);
}
}
void AddNeededStdLib(std::string libname)
{
// if the library is already in the set, insert does nothing, so we don't need to check before inserting
Expand Down Expand Up @@ -354,6 +343,18 @@ public:
}
bool UseVDT() const { return fUseVDT;}

// flag to do the matrix multiplications in the generated code with the BLAS
// routines sgemm_ and saxpy_ instead of the built-in implementations (the
// default), which is faster for large batches. The generated code then has to
// be linked against a BLAS library, or one has to be loaded in the ROOT
// session. Clad treats the BLAS routines as constants and only warns about
// it, so gradients through them are silently zero: then only the operators
// that use Gemm_Call (Gemm and Conv), which has hand-written derivatives, give
// correct gradients, and only if Clad uses these derivatives. Has to be set
// before Generate().
void SetUseBLAS(bool on) { fUseBLAS = on; }
bool UseBLAS() const { return fUseBLAS; }

// RModel is an internal representation that doesn't support ROOT IO (if you need model IO, use ONNX directly).
ClassDefNV(RModel, 0);
};
Expand Down
3 changes: 0 additions & 3 deletions tmva/sofie/inc/TMVA/ROperator.hxx
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@ class RModel;
class ROperator {

public:
virtual std::vector<std::string> GetBlasRoutines() { return {}; }
virtual std::vector<std::string> GetStdLibs() { return {}; }
virtual void Initialize(RModel&) = 0;
virtual std::string Generate(std::string OpName) = 0; //expect unique opName for each operator within the same RModel
Expand All @@ -31,8 +30,6 @@ public:
/// check if the output of the operator is Constant and is evaluated at initialization time
bool IsOutputConstant() const { return fIsOutputConstant; }

//virtual void Forward_reference() = 0;
//virtual void Forward_blas() = 0;
virtual ~ROperator(){}

protected:
Expand Down
1 change: 0 additions & 1 deletion tmva/sofie/inc/TMVA/ROperator_BatchNormalization.hxx
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,6 @@ public:
return out.str();
}

std::vector<std::string> GetBlasRoutines() override { return {}; }
};

}//SOFIE
Expand Down
29 changes: 0 additions & 29 deletions tmva/sofie/inc/TMVA/ROperator_Conv.hxx
Original file line number Diff line number Diff line change
Expand Up @@ -495,7 +495,6 @@ public:
<< "," << fAttrDilations[1] << "," << fAttrDilations[2] << ","
<< "tensor_" << imcol << ");\n\n ";
}
// BLAS
out << SP << "Gemm_Call("
<< "tensor_" << fNY << " + out_offset, false, false, " << OpName << "_m, " << OpName << "_n, " << OpName
<< "_k, " << OpName << "_alpha, " << "tensor_" << imcol << ", tensor_" << convK << ", " << OpName
Expand All @@ -506,11 +505,6 @@ public:
out << "nullptr";
out << ");\n";

// out << SP << SP << "BLAS::sgemm_(&" << OpName << "_transA, &" << OpName << "_transB, &" << OpName << "_m, &"
// << OpName << "_n, &" << OpName << "_k, &" << OpName << "_alpha, " << "tensor_" << imcol << ", &" <<
// OpName << "_m,\n"; // use m if op_xcol is not transpose , otherwise k
// out << SP << SP << SP << "tensor_" << convK << ", &" << OpName << "_k, &" << OpName << "_beta, tensor_" <<
// fNY << " + out_offset, &" << OpName << "_m);\n";
} else {
// case of group convolution
// Unroll (IM2COL) the input tensor- make loop on groups and repeat operations (IM2COL + GEMM for each
Expand Down Expand Up @@ -552,7 +546,6 @@ public:
<< "," << fAttrDilations[1] << "," << fAttrDilations[2] << ",tensor_" << imcol << ");\n\n ";
}

// BLAS
// n must be divided by the number of groups
out << SP << SP << SP << OpName << "_n = " << fShapeW[0] / fAttrGroup << ";\n";
// offset g must be g * k * n
Expand All @@ -570,35 +563,13 @@ public:
out << "nullptr";
out << ");\n";

// out << SP << SP << "BLAS::sgemm_(&" << OpName << "_transA, &" << OpName << "_transB, &" << OpName << "_m, &"
// << OpName << "_n, &" << OpName << "_k, &" << OpName << "_alpha, tensor_" << imcol << ", &" << OpName
// << "_m,\n"; // use m if op_xcol is not transpose , otherwise k
// out << SP << SP << SP << "tensor_" << convK << " + offset_f, &" << OpName << "_k, &" << OpName << "_beta,
// tensor_" << fNY
// << " + out_offset"
// << ", &" << OpName << "_m);\n";

out << SP << SP << "}\n"; // end of group loop
}

// if (fNB != "") {
// out << SP << "int " << OpName << "_size = " << outputBatchStride << ";\n";
// out << SP << "float " << OpName << "_gamma = 1.0;\n";
// out << SP << "int " << OpName << "_incx = 1;\n";
// out << SP << "int " << OpName << "_incy = 1;\n";

// out << SP << "BLAS::saxpy_(&" << OpName << "_size, &" << OpName << "_gamma, tensor_" << fNB << ", &"
// << OpName << "_incx, tensor_" << fNY << " + out_offset, &" << OpName << "_incy);\n";

// }
out << SP << "}\n"; // end of batch size loop

return out.str();
}

/*! \brief Returns the blas routines needed to compile the generated code
*/
std::vector<std::string> GetBlasRoutines() override { return { std::string("Gemm"), std::string("Axpy") }; }
};

} // namespace SOFIE
Expand Down
32 changes: 5 additions & 27 deletions tmva/sofie/inc/TMVA/ROperator_ConvTranspose.hxx
Original file line number Diff line number Diff line change
Expand Up @@ -117,10 +117,6 @@ public:
* \param opName name of the operator
*/
std::string Generate(std::string opName) override;

/*! \brief Returns the blas routines needed to compile the generated code
*/
std::vector<std::string> GetBlasRoutines() override { return {std::string("Gemm"), std::string("Axpy")}; }
};

template <typename T>
Expand Down Expand Up @@ -293,6 +289,8 @@ void ROperator_ConvTranspose<T>::Initialize(RModel &model)
// register the inference helper functions used by the generated code
// (only the <3D case is supported, which uses col2im)
model.AddNeededHelperFunction("col2im");
model.AddNeededHelperFunction("Gemm");
model.AddNeededHelperFunction("Axpy");
if (!fNB.empty())
model.AddNeededHelperFunction("BroadcastConvBias");
}
Expand Down Expand Up @@ -427,9 +425,7 @@ std::string ROperator_ConvTranspose<T>::Generate(std::string OpName)
out << SP << SP << "size_t x_offset = n * " << fShapeX[1] * iDepth * iHeight * iWidth << ";\n";
out << SP << SP << "size_t out_offset = n * " << fShapeY[1] * oDepth * oHeight * oWidth << ";\n";

// DO BLAS before:
// BLAS
out << SP << SP << "BLAS::sgemm_(&" << OpName << "_transA, &" << OpName << "_transB, &" << OpName << "_m, &"
out << SP << SP << "Gemm(&" << OpName << "_transA, &" << OpName << "_transB, &" << OpName << "_m, &"
<< OpName << "_n, &" << OpName << "_k, &" << OpName << "_alpha, "
<< "tensor_" << fNX << " + x_offset, &" << OpName
<< "_m,\n"; // use m if op_xcol is not transpose , otherwise k
Expand Down Expand Up @@ -467,12 +463,6 @@ std::string ROperator_ConvTranspose<T>::Generate(std::string OpName)
<< fAttrStrides[0] << "," << fAttrStrides[1] << "," << fAttrStrides[2] << "," << fAttrDilations[0] << ","
<< fAttrDilations[1] << "," << fAttrDilations[2] << ",tensor_" << fImcol << ");\n\n ";
}
// // BLAS
// out << SP << SP << "BLAS::sgemm_(&" << OpName << "_transA, &" << OpName << "_transB, &" << OpName << "_m, &"
// << OpName << "_n, &" << OpName << "_k, &" << OpName << "_alpha, tensor_" << fImcol << ", &" << OpName
// << "_m,\n"; // use m if op_xcol is not transpose , otherwise k
// out << SP << SP << SP <<"tensor_" << fConvK << ", &" << OpName << "_k, &" << OpName << "_beta, tensor_" << fNY
// << " + out_offset, &" << OpName << "_m);\n";
} else {
// case of group transposed convolution
// Unroll (IM2COL) the input tensor- make loop on groups and repeat operations (IM2COL + GEMM for each
Expand All @@ -487,7 +477,7 @@ std::string ROperator_ConvTranspose<T>::Generate(std::string OpName)
out << SP << SP << SP << "size_t offset_f = g * " << (fShapeW[0] / fAttrGroup) * icstrideDil * fShapeW[1]
<< ";\n";

out << SP << SP << "BLAS::sgemm_(&" << OpName << "_transA, &" << OpName << "_transB, &" << OpName << "_m, &"
out << SP << SP << "Gemm(&" << OpName << "_transA, &" << OpName << "_transB, &" << OpName << "_m, &"
<< OpName << "_n, &" << OpName << "_k, &" << OpName << "_alpha, "
<< "tensor_" << fNX << " + x_offset, &" << OpName
<< "_m,\n"; // use m if op_xcol is not transpose , otherwise k
Expand Down Expand Up @@ -524,18 +514,6 @@ std::string ROperator_ConvTranspose<T>::Generate(std::string OpName)
<< fAttrDilations[1] << "," << fAttrDilations[2] << "," << "tensor_" << fImcol << ");\n\n ";
}

// // BLAS
// // offset g must be g * k * n
// out << SP << SP << SP << "size_t offset_f = g * " << fShapeW[0] * fShapeW[1] * icstrideDil / fAttrGroup <<
// ";\n"; out << SP << SP << "BLAS::sgemm_(&" << OpName << "_transA, &" << OpName << "_transB, &" << OpName <<
// "_m, &"
// << OpName << "_n, &" << OpName << "_k, &" << OpName << "_alpha, tensor_" << fImcol << ", &" << OpName
// << "_m,\n"; // use m if op_xcol is not transpose , otherwise k
// out << SP << SP << SP << "tensor_" << fConvK << " + offset_f, &" << OpName << "_k, &" << OpName << "_beta,
// tensor_" << fNY
// << " + out_offset"
// << ", &" << OpName << "_m);\n";

out << SP << SP << "}\n"; // end of group loop
}

Expand All @@ -547,7 +525,7 @@ std::string ROperator_ConvTranspose<T>::Generate(std::string OpName)
out << SP << "int " << OpName << "_incx = 1;\n";
out << SP << "int " << OpName << "_incy = 1;\n";

out << SP << "BLAS::saxpy_(&" << OpName << "_size, &" << OpName << "_gamma, tensor_" << fNBroadcastedB << ", &"
out << SP << "Axpy(&" << OpName << "_size, &" << OpName << "_gamma, tensor_" << fNBroadcastedB << ", &"
<< OpName << "_incx, tensor_" << fNY << ", &" << OpName << "_incy);\n";
}

Expand Down
7 changes: 3 additions & 4 deletions tmva/sofie/inc/TMVA/ROperator_Einsum.hxx
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,8 @@ public:
}

void Initialize(RModel& model) override {
// the matmul reduction emitted in Generate() uses Gemm
model.AddNeededHelperFunction("Gemm");
// input must be a graph input, or already initialized intermediate tensor
size_t i = 0;
std::map<char, int> labelsMap;
Expand Down Expand Up @@ -305,7 +307,7 @@ public:
return strst.str();
};
// only float type supported
out << SP << "BLAS::sgemm_(&" << opName << "_transB, &" << opName << "_transA, &" << opName
out << SP << "Gemm(&" << opName << "_transB, &" << opName << "_transA, &" << opName
<< "_n, &" << opName << "_m, &" << opName << "_k, &" << opName << "_alpha, "
<< "&tensor_" << fNInputs[1] << "[" << tensorOffset(inputStrideB, fInputLabels[1])
<< "], &" << opName << "_ldb, "
Expand All @@ -324,9 +326,6 @@ public:
return out.str();
}

std::vector<std::string> GetBlasRoutines() override {
return { std::string("Gemm") };
}
};

}//SOFIE
Expand Down
Loading
Loading