Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README/ReleaseNotes/v642/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ The following people have contributed to this new version:
* The overloads of `RooAbsReal::createChi2()` and `RooAbsReal::chi2FitTo()` that take unbinned **RooDataSet** data objects were deprecated in ROOT 6.40 and are now removed.
* The **RooStats::HybridPlot** class and the related **HybridResult::GetPlot** method were deprecated in ROOT 6.40 and are now removed.
* The JSON I/O of RooFitHS3 through "export keys" and "factory expressions", which mapped the proxies and constructor arguments of a class to JSON keys, is removed, together with the functions `RooFit::JSONIO::loadExportKeys()`, `loadFactoryExpressions()`, `clearExportKeys()`, `clearFactoryExpressions()`, `printExportKeys()`, `printFactoryExpressions()`, `exportKeys()` and `importExpressions()`. The classes that were covered this way now have C++ importers and exporters, and the JSON that RooFit reads and writes is unchanged. To add JSON I/O for your own class, implement a `RooFit::JSONIO::Importer` and `RooFit::JSONIO::Exporter` instead, as explained in the [RooFitHS3 developer documentation](https://root.cern/doc/v644/group__roofit__dev__docs__hs3.html).
* Some helper functions of `RooJSONFactoryWSTool` and `RooFit::Detail::JSONNode` that were only meant for the importers and exporters in ROOT are removed: `RooJSONFactoryWSTool::createNewJSONTree()`, `makeVariablesNode()`, `getRooFitInternal()` and `exportAxis()`, as well as `JSONNode::has_key()`, `JSONNode::operator>>()`, the `JSONNode::fill_seq()` overloads with a maximum size or a transformation function, the decrement and arrow operators of the child iterators, and the operators that stream a map into a node, the children of a non-const node into a `std::vector`, or a node into a `std::ostream` (use `JSONNode::writeJSON()` for the latter). An exporter for your own class can still write the axes and bin contents of a histogram with `RooJSONFactoryWSTool::exportHisto()`, and an array of numbers with `exportArray()`.
* Some helper functions of `RooJSONFactoryWSTool` and `RooFit::Detail::JSONNode` that were only meant for the importers and exporters in ROOT are removed: `RooJSONFactoryWSTool::createNewJSONTree()`, `makeVariablesNode()`, `getRooFitInternal()` and `exportAxis()`, the struct `RooJSONFactoryWSTool::CombinedData`, as well as `JSONNode::has_key()`, `JSONNode::operator>>()`, the `JSONNode::fill_seq()` overloads with a maximum size or a transformation function, the decrement and arrow operators of the child iterators, and the operators that stream a map into a node, the children of a node into a `std::vector` (stream the node itself instead), or a node into a `std::ostream` (use `JSONNode::writeJSON()` instead). An exporter for your own class can still write the axes and bin contents of a histogram with `RooJSONFactoryWSTool::exportHisto()`, and an array of numbers with `exportArray()`.
* The conversion from Python set to **RooArgSet** is deprecated and won't work anymore in ROOT 6.44. The problem is that Python sets are unordered while RooArgSets are ordered, and this mismatch can lead to subtle problems later on. Prefer conversion from Python lists or tuples, which are ordered too.
* Flattening nested **RooSimultaneous** in the constructor is deprecated and will be removed in ROOT 6.44. It double counts extended components that don't depend on all index categories ([GitHub issue #23342](https://github.com/root-project/root/issues/23342)) and breaks nested RooSimultaneous whose index category is a parameter ("switch mode"). From ROOT 6.44, nested RooSimultaneous are only supported in switch mode. To combine channels, use one RooSimultaneous over a **RooCategory** with one state per channel. Workspaces read from files are not affected.
* The `RooTrace` class is deprecated and will be removed in ROOT 6.44. It was a RooFit-specific memory tracer whose instrumentation hooks are compiled out by default, so it has been inert and untested for years. For memory debugging, please use general-purpose tools such as AddressSanitizer or Valgrind instead.
Expand Down
2 changes: 1 addition & 1 deletion roofit/hs3/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ add_library(hs3cpp OBJECT
src/HS3Parser.cxx
src/HS3Upgrade.cxx
src/HS3Validate.cxx
src/HS3Variates.cxx
src/HS3Writer.cxx
src/JSONInterface.cxx
)
Expand Down Expand Up @@ -63,7 +64,6 @@ if(NOT CMAKE_VERSION VERSION_LESS "3.23.0")
src/HS3Schema.h
src/JSONIOUtils.h
src/JSONReadRecorder.h
src/static_execute.h
)
endif()

Expand Down
101 changes: 9 additions & 92 deletions roofit/hs3/inc/RooFit/Detail/JSONInterface.h
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,6 @@
#define RooFit_Detail_JSONInterface_h

#include <iostream>
#include <map>
#include <unordered_map>
#include <memory>
#include <optional>
#include <stdexcept>
Expand All @@ -37,13 +35,7 @@ class JSONNode {
++_pos;
return *this;
}
child_iterator_t &operator--()
{
--_pos;
return *this;
}
Nd &operator*() const { return **_pos; }
Nd &operator->() const { return **_pos; }

friend bool operator!=(child_iterator_t const &lhs, child_iterator_t const &rhs) { return lhs._pos != rhs._pos; }
friend bool operator==(child_iterator_t const &lhs, child_iterator_t const &rhs) { return lhs._pos == rhs._pos; }
Expand All @@ -70,16 +62,13 @@ class JSONNode {
JSONNode(JSONNode const &) = delete;
JSONNode &operator=(JSONNode const &) = delete;

public:
void writeJSON(std::ostream &os) const;

public:
JSONNode &operator<<(std::string const &s);
inline JSONNode &operator<<(const char *s) { return *this << std::string(s); }
JSONNode &operator<<(int i);
JSONNode &operator<<(double d);
JSONNode &operator<<(bool b);
const JSONNode &operator>>(std::string &v) const;
JSONNode &operator[](std::string const &k);
const JSONNode &operator[](std::string const &k) const;
bool is_container() const;
Expand All @@ -103,7 +92,6 @@ class JSONNode {
std::optional<bool> try_val_bool() const;
template <class T>
T val_t() const;
bool has_key() const;
bool has_val() const;
bool has_child(std::string const &) const;
JSONNode &append_child();
Expand All @@ -126,28 +114,6 @@ class JSONNode {
}
}

template <typename Collection>
void fill_seq(Collection const &coll, size_t nmax)
{
set_seq();
size_t n = 0;
for (auto const &item : coll) {
if (n >= nmax)
break;
append_child() << item;
++n;
}
}

template <typename Collection, typename TransformationFunc>
void fill_seq(Collection const &coll, TransformationFunc func)
{
set_seq();
for (auto const &item : coll) {
append_child() << func(item);
}
}

template <typename Matrix>
void fill_mat(Matrix const &mat)
{
Expand All @@ -161,32 +127,22 @@ class JSONNode {
}
}

JSONNode const *find(std::string const &key) const
{
auto &n = *this;
return n.has_child(key) ? &n[key] : nullptr;
}
/// The component at the end of a path of keys, or nullptr if a component on the path doesn't exist.
JSONNode const *find(std::string const &key) const { return has_child(key) ? &(*this)[key] : nullptr; }

template <typename... Keys_t>
JSONNode const *find(std::string const &key, Keys_t const &...keys) const
{
auto &n = *this;
return n.has_child(key) ? n[key].find(keys...) : nullptr;
return has_child(key) ? (*this)[key].find(keys...) : nullptr;
}

JSONNode &get(std::string const &key)
{
auto &n = *this;
n.set_map();
return n[key];
}
/// The component at the end of a path of keys. The nodes on the path become maps.
JSONNode &get(std::string const &key) { return set_map()[key]; }

template <typename... Keys_t>
JSONNode &get(std::string const &key, Keys_t const &...keys)
{
auto &next = get(key);
next.set_map();
return next.get(keys...);
return get(key).get(keys...);
}

private:
Expand Down Expand Up @@ -219,55 +175,16 @@ class JSONTree {
std::unique_ptr<JSONNode::Storage> _storage;
};

std::ostream &operator<<(std::ostream &os, RooFit::Detail::JSONNode const &s);

template <class T>
std::vector<T> &operator<<(std::vector<T> &v, RooFit::Detail::JSONNode::children_view const &cv)
{
for (const auto &e : cv) {
v.push_back(e.val_t<T>());
}
return v;
}

template <class T>
std::vector<T> &operator<<(std::vector<T> &v, RooFit::Detail::JSONNode::const_children_view const &cv)
{
for (const auto &e : cv) {
v.push_back(e.val_t<T>());
}
return v;
}

template <class T>
std::vector<T> &operator<<(std::vector<T> &v, RooFit::Detail::JSONNode const &n)
{
if (!n.is_seq()) {
throw std::runtime_error("node " + n.key() + " is not of sequence type!");
}
v << n.children();
return v;
}

template <class Key, class T, class Hash, class KeyEqual, class Allocator>
RooFit::Detail::JSONNode &
operator<<(RooFit::Detail::JSONNode &n, const std::unordered_map<Key, T, Hash, KeyEqual, Allocator> &m)
{
n.set_map();
for (const auto &it : m) {
n[it.first] << it.second;
}
return n;
}

template <class Key, class T, class Compare, class Allocator>
RooFit::Detail::JSONNode &operator<<(RooFit::Detail::JSONNode &n, const std::map<Key, T, Compare, Allocator> &m)
{
n.set_map();
for (const auto &it : m) {
n[it.first] << it.second;
for (const auto &e : n.children()) {
v.push_back(e.val_t<T>());
}
return n;
return v;
}

template <>
Expand Down
78 changes: 42 additions & 36 deletions roofit/hs3/inc/RooFitHS3/RooJSONFactoryWSTool.h
Original file line number Diff line number Diff line change
Expand Up @@ -35,19 +35,21 @@ struct ParameterPoint;
struct ParameterValue;
struct ToolExportState;
struct ToolImportState;
struct Version;
template <class Tag>
struct Ref;
using DistributionRef = Ref<struct DistributionTag>;
void setExportedObject(RooJSONFactoryWSTool &tool, HistFactoryDist dist,
std::vector<DistributionRef> channelConstraints);
std::string const *importedArrayComponentForm(RooJSONFactoryWSTool &tool, std::string const &name);
void recordImportedArrayComponent(RooJSONFactoryWSTool &tool, std::string const &name, std::string form);
} // namespace HS3
} // namespace RooFit
namespace RooStats {
class ModelConfig;
}
class RooRealVar;
class RooAbsCategory;
class RooCategory;
class RooSimultaneous;

class RooJSONFactoryWSTool {
Expand Down Expand Up @@ -76,33 +78,39 @@ class RooJSONFactoryWSTool {
/// the JSON.
bool allowExportSharedVariates = false;
// Write array distributions (HS3 `array_dist`). If false, no `array_dist` is written: HistFactory constraints
// are named individually, and a RooProdPdf tagged as array distribution is written as `product_dist`.
// are named individually, and a RooProdPdf tagged as array distribution is written as `product_dist`. The
// RooProdPdf that RooFit created itself when it read an `array_dist` is only written if another object uses
// it; otherwise its components are written on their own.
bool exportArrayDist = true;
// How the constraint terms of HistFactory models enter the likelihood: listed in "aux_distributions", with
// the observed values as constant parameters (false), or listed in "distributions", each paired with a data set
// holding the observed values of its observables (true).
// the observed values as constant parameters (false), or paired with a data set holding the observed values
// of their observables (true). An array of constraints of a model that is a RooSimultaneous is then a component
// of the combined distribution, and its data set a component of the combined data set, so that the combined
// distribution is the sampling distribution of all measurements. Other constraints are terms of the likelihood.
bool exportConstraintData = false;
/// EXPERIMENTAL: how auxiliary measurements that are components of a combined distribution (a constraint term
/// paired with the data set of its observed values, as RooFit writes them with `exportConstraintData`) are
/// read. If false, they become constraint terms of the model as in HistFactory, with global observables set
/// to the observed values; RooFit recognizes the components that it wrote like this by a mark in the "misc"
/// section of the document. If true, or without that mark, they stay components of the RooSimultaneous, with
/// their data set of one entry in the combined data set, so that the RooSimultaneous is the sampling
/// distribution of all measurements. This option and the form of the resulting workspace may still change.
bool importConstraintsAsChannels = false;
};

static Config &config();

static RooWorkspace sanitizeWS(const RooWorkspace &ws);
static RooWorkspace cleanWS(const RooWorkspace &ws, bool onlyModelConfig = false);

struct CombinedData {
std::string name;
std::vector<std::string> columns;
};

RooJSONFactoryWSTool(RooWorkspace &ws);

~RooJSONFactoryWSTool();

static std::string name(const RooFit::Detail::JSONNode &n);
static bool isValidName(const std::string &str);
static bool testValidName(const std::string &str, bool forcError);
static bool testValidName(const std::string &name, bool forceError);
static std::string sanitizeName(const std::string str);
static void rebuildModelConfigInWorkspace(RooStats::ModelConfig *mc, RooWorkspace &ws);

static RooFit::Detail::JSONNode &appendNamedChild(RooFit::Detail::JSONNode &node, std::string const &name);
static RooFit::Detail::JSONNode const *findNamedChild(RooFit::Detail::JSONNode const &node, std::string const &name);
Expand Down Expand Up @@ -191,22 +199,18 @@ class RooJSONFactoryWSTool {
readBinnedData(const RooFit::Detail::JSONNode &n, const std::string &namecomp, RooArgSet const &vars);

bool importJSON(std::string const &filename);
bool importJSON(std::istream &os);
bool importJSON(std::istream &is);
bool exportJSON(std::string const &fileName);
bool exportJSON(std::ostream &os);

std::string exportJSONtoString();
bool importJSONfromString(const std::string &s);
void importJSONElement(const std::string &name, const std::string &jsonString);
void importVariableElement(const RooFit::Detail::JSONNode &n);
void importVariableElement(const RooFit::Detail::JSONNode &elementNode);

void importFunction(const RooFit::Detail::JSONNode &p, bool importAllDependants);
void importFunction(const std::string &jsonString, bool importAllDependants);

static std::unique_ptr<RooFit::Detail::JSONTree> createNewJSONTree();

static RooFit::Detail::JSONNode &makeVariablesNode(RooFit::Detail::JSONNode &rootNode);

// error handling helpers
class DependencyMissingError : public std::runtime_error {
std::string _parent, _child, _class;
Expand All @@ -224,15 +228,6 @@ class RooJSONFactoryWSTool {
const std::string &classname() const { return _class; }
};

template <typename... Keys_t>
static RooFit::Detail::JSONNode &getRooFitInternal(RooFit::Detail::JSONNode &node, Keys_t const &...keys)
{
return node.get("misc", "ROOT_internal", keys...);
}

static void exportAxis(RooFit::Detail::JSONNode &obsNode, RooRealVar const &var);
static void exportCategoricalAxis(RooFit::Detail::JSONNode &obsNode, RooCategory const &cat);

static void
exportHisto(RooArgSet const &vars, std::size_t n, double const *contents, RooFit::Detail::JSONNode &output);

Expand Down Expand Up @@ -260,8 +255,19 @@ class RooJSONFactoryWSTool {
std::string getStringAttribute(const std::string &obj, const std::string &attrib);
void setStringAttribute(const std::string &obj, const std::string &attrib, const std::string &value);

/// The "hs3_version" of the document that is currently being imported, or an empty string outside of an import.
std::string const &hs3VersionInput() const { return _hs3VersionInput; }
/// The HS3 version of the document that is currently being imported. Outside of an import it is 0.0.0, like a
/// version that can't be parsed.
RooFit::HS3::Version hs3Version() const;

/// The constraint terms that the document that is currently being imported adds to the HistFactory distribution
/// `dist`, besides those that its modifiers name, or nullptr if there are none.
std::vector<RooFit::HS3::DistributionRef> const *channelConstraints(std::string const &dist) const;

/// Whether the constraint term `pdf` contributes to the likelihoods of the document that is currently being
/// imported as a component of a combined distribution with data, by itself or as a component of an array
/// distribution. It is then a measurement of its own, and must not be multiplied into the distributions that
/// name it as a constraint as well.
bool isConstraintReadAsChannel(RooAbsPdf const &pdf) const;

private:
template <class T>
Expand All @@ -282,12 +288,12 @@ class RooJSONFactoryWSTool {
}
}
void exportArrayDists();
bool exportConstraintData(std::string const &term, RooArgSet const *globalObservables);
bool exportConstraintData(std::string const &term, RooArgSet const *globalObservables, std::string const &dataName);

void exportSimultaneous(RooSimultaneous const &simPdf);
void recordCategory(RooAbsCategory const &cat);
void exportData(RooAbsData const &data);
RooJSONFactoryWSTool::CombinedData exportCombinedData(RooAbsData const &data);
void exportCombinedData(RooAbsData const &data);

void importAllNodes(RooFit::Detail::JSONNode &n);
void importDistribution(RooFit::HS3::Distribution const &distribution);
Expand All @@ -296,23 +302,23 @@ class RooJSONFactoryWSTool {
void warnIfFreeWithoutDomain(RooRealVar const &var);
void importDependants(const RooFit::Detail::JSONNode &n);

void exportVariable(const RooAbsArg *v, RooFit::HS3::ParameterPoint &point, bool storeConstant);
void exportVariables(const RooArgSet &allElems, RooFit::HS3::ParameterPoint &point, bool storeConstant);
void exportVariable(const RooAbsArg *v, RooFit::HS3::ParameterPoint &point);

void exportAllObjects(RooFit::Detail::JSONNode &n);

void exportModelConfig(RooStats::ModelConfig const &mc,
const std::vector<RooJSONFactoryWSTool::CombinedData> &combined,
const std::vector<RooAbsData *> &single);
void exportModelConfig(RooStats::ModelConfig const &mc, const std::vector<RooAbsData *> &singleDataSets);

void exportSingleModelConfig(RooStats::ModelConfig const &mc, std::string const &analysisName,
std::string const *dataName);

friend void RooFit::HS3::setExportedObject(RooJSONFactoryWSTool &tool, RooFit::HS3::HistFactoryDist dist,
std::vector<RooFit::HS3::DistributionRef> channelConstraints);
friend std::string const *
RooFit::HS3::importedArrayComponentForm(RooJSONFactoryWSTool &tool, std::string const &name);
friend void
RooFit::HS3::recordImportedArrayComponent(RooJSONFactoryWSTool &tool, std::string const &name, std::string form);

// member variables
std::string _hs3VersionInput;
RooWorkspace &_workspace;

// objects to represent intermediate information
Expand Down
Loading
Loading