From c041ec3b13c2fe82d5fbde572b85b6cffa2800de Mon Sep 17 00:00:00 2001 From: tannevaled Date: Sat, 17 Oct 2026 22:58:33 +0200 Subject: [PATCH] Scientific stack: Special, Stats, Integrate and Optimize, and a guard that runs every example The site documented NDArray, FFT and Image and stopped there. rbgo#819 adds four more modules to the same cgo-free stack -- scipy.special's 67 functions, scipy.stats's 32 univariate distributions, scipy.integrate's quadrature and scipy.optimize's root finders and minimisers -- so the page that exists to describe that stack described two thirds of it. Each section follows the page's own shape: what it binds, the surface, and a short example. The names are scipy's own, and the sections say why: every one of those modules states its specification by measuring itself against scipy in ULPs, and the pairing is a file in its repository, so Log1pExp is #softplus and StudentT is Stats.t rather than a transliteration of the Go name. Nothing here is new prose about behaviour: it is the README's text, cut to this page's length. The index's stack paragraph and its library list named four organisations and now name six. ## tools/docexample Every example on this site carries its answer in a comment -- `p Special.gamma(5.0) # => 24.0` -- and nothing ran them. A binding renamed, a default changed or a return shape altered in the ruby repository left each example claiming its old output, which reads as a promise and is the first thing a reader tries. tools/conformancefigure already guards the one NUMBER this site states; this guards the twenty-five OUTPUTS it states. A ```ruby block is checked exactly when some line in it ends in `# =>`. That makes the selection self-evident rather than a list to maintain: the examples that read a photo off disk or start a server claim nothing and are skipped, and one that gains an answer is picked up without being registered anywhere. The block runs as one program, so earlier lines can set up later ones, and the lines it PRINTS are compared in order against the answers it claims -- a differing count failing too, because an example printing two lines while claiming one is as wrong as a wrong number. Exit status separates the three outcomes that matter: 1 for a disagreement, 2 for no interpreter (a refusal, not a pass), 3 for finding nothing to check, which on a site that teaches by example means the extraction broke rather than that the site is clean. -selftest checks the extraction on six cases whose answer is known, including the two that must NOT match: a comment that is not an answer, and a shell block. Measured: 11 examples, 25 claimed answers, 13 files, 0 disagreements. Proved to bite three ways -- a wrong answer, an example printing one line too many, and an example that no longer runs -- each named with its file and line. ## This must merge after rbgo#819 The guard says so itself. Built against the ruby repository's current main it reports exactly the four new sections and nothing else: docs/scientific-stack.md:72: uninitialized constant Special docs/scientific-stack.md:98: uninitialized constant Stats docs/scientific-stack.md:118: uninitialized constant Integrate docs/scientific-stack.md:138: uninitialized constant Optimize 11 example(s) with 25 claimed answer(s) over 13 file(s); 4 example(s) disagree The seven pre-existing examples pass on both, so the dependency is the four sections and the ordering is the only thing to respect. Co-Authored-By: Claude Opus 5 --- .github/workflows/docs.yml | 19 +++ docs/index.md | 13 +- docs/scientific-stack.md | 93 ++++++++++++ tools/docexample/main.go | 280 +++++++++++++++++++++++++++++++++++++ 4 files changed, 400 insertions(+), 5 deletions(-) create mode 100644 tools/docexample/main.go diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml index 1ebb29f..ffab005 100644 --- a/.github/workflows/docs.yml +++ b/.github/workflows/docs.yml @@ -61,6 +61,25 @@ jobs: - name: The stated conformance total matches the gate run: go run ./tools/conformancefigure + # The Ruby examples on this site state their own answers, and until now + # nothing ran them: a binding renamed or a return shape changed in the + # ruby repository left every example here claiming its old output, which + # reads as a promise and is the first thing a reader tries. + # + # rbgo is built from the ruby repository's main, so the examples are + # judged against the interpreter this site documents rather than against + # a release that may be behind it. A build that fails, or an interpreter + # that cannot be produced, exits 2 -- a REFUSAL, not a pass, for the same + # reason the baseline check above refuses. + - name: The Ruby examples print what the pages say they print + run: | + go run ./tools/docexample -selftest + # GOBIN names the destination: `go install` otherwise writes to + # $(go env GOPATH)/bin, and an install that lands somewhere else looks + # exactly like one that worked. + GOBIN=/tmp go install github.com/go-embedded-ruby/ruby/cmd/rbgo@main + go run ./tools/docexample -rbgo /tmp/rbgo -root docs + deploy: # Only main publishes. A pull request stops after build above: it has no # business pushing to gh-pages, and `permissions: contents: write` is not diff --git a/docs/index.md b/docs/index.md index d886191..0610b78 100644 --- a/docs/index.md +++ b/docs/index.md @@ -174,10 +174,12 @@ Every feature below is **differential-tested against MRI Ruby 4.0.5**: reference object), `instance_variable_get`/`set`. - **Numeric & scientific stack:** `Complex`, `Rational`, the `Math` module, and a **cgo-free scientific stack** — `NDArray` (NumPy-style arrays), an `FFT` module - (numpy.fft-style transforms), `Image` (scikit-image-style processing) — plus - container/value types `Set`, `Bag` (multiset), `Time`, `Date` and - `BigDecimal`, each binding a pure-Go sibling library (go-ndarray / go-fft / - go-images / go-composites). See + (numpy.fft-style transforms), `Special` and `Stats` (scipy.special's 67 functions + and scipy.stats's 32 univariate distributions), `Integrate` and `Optimize` + (scipy.integrate's quadrature and scipy.optimize's root finders and minimisers), + `Image` (scikit-image-style processing) — plus container/value types `Set`, `Bag` + (multiset), `Time`, `Date` and `BigDecimal`, each binding a pure-Go sibling library + (go-ndarray / go-fft / go-numerics / go-statistics / go-images / go-composites). See [Scientific stack](scientific-stack.md). - **Native capability modules:** a set of `require`-able modules, each backed by a canonical pure-Go `go-ruby-*` library so the build stays **CGO=0**. They wrap @@ -285,7 +287,8 @@ The full bound family (alphabetical, all native modules in `go.mod`): [`aasm`](https://github.com/go-ruby-aasm/aasm), [`abbrev`](https://github.com/go-ruby-abbrev/abbrev), [`acme`](https://github.com/go-ruby-acme/acme), [`actioncable`](https://github.com/go-ruby-actioncable/actioncable), [`actionmailer`](https://github.com/go-ruby-actionmailer/actionmailer), [`actionpack`](https://github.com/go-ruby-actionpack/actionpack), [`actionview`](https://github.com/go-ruby-actionview/actionview), [`activejob`](https://github.com/go-ruby-activejob/activejob), [`activemodel`](https://github.com/go-ruby-activemodel/activemodel), [`activerecord`](https://github.com/go-ruby-activerecord/activerecord), [`activestorage`](https://github.com/go-ruby-activestorage/activestorage), [`activesupport`](https://github.com/go-ruby-activesupport/activesupport), [`addressable`](https://github.com/go-ruby-addressable/addressable), [`age`](https://github.com/go-ruby-age/age), [`arrow`](https://github.com/go-ruby-arrow/arrow), [`async`](https://github.com/go-ruby-async/async), [`augeas`](https://github.com/go-ruby-augeas/augeas), [`base64`](https://github.com/go-ruby-base64/base64), [`bbolt`](https://github.com/go-ruby-bbolt/bbolt), [`bcrypt`](https://github.com/go-ruby-bcrypt/bcrypt), [`benchmark`](https://github.com/go-ruby-benchmark/benchmark), [`bigdecimal`](https://github.com/go-ruby-bigdecimal/bigdecimal), [`bleve`](https://github.com/go-ruby-bleve/bleve), [`builder`](https://github.com/go-ruby-builder/builder), [`bundler`](https://github.com/go-ruby-bundler/bundler), [`cancancan`](https://github.com/go-ruby-cancancan/cancancan), [`capistrano`](https://github.com/go-ruby-capistrano/capistrano), [`capybara`](https://github.com/go-ruby-capybara/capybara), [`cgi`](https://github.com/go-ruby-cgi/cgi), [`chronic`](https://github.com/go-ruby-chronic/chronic), [`cmath`](https://github.com/go-ruby-cmath/cmath), [`commonmark`](https://github.com/go-commonmark/commonmark), [`concurrent-ruby`](https://github.com/go-ruby-concurrent-ruby/concurrent-ruby), [`confd`](https://github.com/go-ruby-confd/confd), [`connection-pool`](https://github.com/go-ruby-connection-pool/connection-pool), [`csv`](https://github.com/go-ruby-csv/csv), [`date`](https://github.com/go-ruby-date/date), [`deep-merge`](https://github.com/go-ruby-deep-merge/deep-merge), [`devise`](https://github.com/go-ruby-devise/devise), [`did-you-mean`](https://github.com/go-ruby-did-you-mean/did-you-mean), [`digest`](https://github.com/go-ruby-digest/digest), [`dotenv`](https://github.com/go-ruby-dotenv/dotenv), [`dry-struct`](https://github.com/go-ruby-dry-struct/dry-struct), [`dry-types`](https://github.com/go-ruby-dry-types/dry-types), [`dry-validation`](https://github.com/go-ruby-dry-validation/dry-validation), [`erb`](https://github.com/go-ruby-erb/erb), [`erubi`](https://github.com/go-ruby-erubi/erubi), [`etcd`](https://github.com/go-ruby-etcd/etcd), [`excon`](https://github.com/go-ruby-excon/excon), [`facter`](https://github.com/go-ruby-facter/facter), [`factory-bot`](https://github.com/go-ruby-factory-bot/factory-bot), [`faker`](https://github.com/go-ruby-faker/faker), [`faraday`](https://github.com/go-ruby-faraday/faraday), [`fast-gettext`](https://github.com/go-ruby-fast-gettext/fast-gettext), [`find`](https://github.com/go-ruby-find/find), [`format`](https://github.com/go-ruby-format/format), [`friendly-id`](https://github.com/go-ruby-friendly-id/friendly-id), [`getoptlong`](https://github.com/go-ruby-getoptlong/getoptlong), [`grape`](https://github.com/go-ruby-grape/grape), [`graphql`](https://github.com/go-ruby-graphql/graphql), [`grpc`](https://github.com/go-ruby-grpc/grpc), [`haml`](https://github.com/go-ruby-haml/haml), [`hanami`](https://github.com/go-ruby-hanami/hanami), [`hcl2`](https://github.com/go-ruby-hcl2/hcl2), [`hiera`](https://github.com/go-ruby-hiera/hiera), [`hocon`](https://github.com/go-ruby-hocon/hocon), [`http`](https://github.com/go-ruby-http/http), [`httparty`](https://github.com/go-ruby-httparty/httparty), [`i18n`](https://github.com/go-ruby-i18n/i18n), [`images`](https://github.com/go-ruby-images/images), [`ipaddr`](https://github.com/go-ruby-ipaddr/ipaddr), [`irb`](https://github.com/go-ruby-irb/irb), [`jbuilder`](https://github.com/go-ruby-jbuilder/jbuilder), [`jekyll`](https://github.com/go-ruby-jekyll/jekyll), [`json`](https://github.com/go-ruby-json/json), [`jwt`](https://github.com/go-ruby-jwt/jwt), [`kafka`](https://github.com/go-ruby-kafka/kafka), [`kaminari`](https://github.com/go-ruby-kaminari/kaminari), [`kramdown`](https://github.com/go-kramdown/kramdown), [`liquid`](https://github.com/go-liquid/liquid), [`logger`](https://github.com/go-ruby-logger/logger), [`mail`](https://github.com/go-ruby-mail/mail), [`marshal`](https://github.com/go-ruby-marshal/marshal), [`matrix`](https://github.com/go-ruby-matrix/matrix), [`mime-types`](https://github.com/go-ruby-mime-types/mime-types), [`minitest`](https://github.com/go-ruby-minitest/minitest), [`money`](https://github.com/go-ruby-money/money), [`mongodb`](https://github.com/go-ruby-mongodb/mongodb), [`msgpack`](https://github.com/go-ruby-msgpack/msgpack), [`multi-json`](https://github.com/go-ruby-multi-json/multi-json), [`mustache`](https://github.com/go-mustache/mustache), [`mysql`](https://github.com/go-ruby-mysql/mysql), [`nats`](https://github.com/go-ruby-nats/nats), [`net-ftp`](https://github.com/go-ruby-net-ftp/net-ftp), [`net-http`](https://github.com/go-ruby-net-http/net-http), [`net-imap`](https://github.com/go-ruby-net-imap/net-imap), [`net-pop`](https://github.com/go-ruby-net-pop/net-pop), [`net-sftp`](https://github.com/go-ruby-net-sftp/net-sftp), [`net-smtp`](https://github.com/go-ruby-net-smtp/net-smtp), [`nokogiri`](https://github.com/go-nokogiri/nokogiri), [`oauth2`](https://github.com/go-ruby-oauth2/oauth2), [`observer`](https://github.com/go-ruby-observer/observer), [`oidc`](https://github.com/go-ruby-oidc/oidc), [`omniauth`](https://github.com/go-ruby-omniauth/omniauth), [`openbao`](https://github.com/go-ruby-openbao/openbao), [`openstack`](https://github.com/go-ruby-openstack/openstack), [`opentelemetry`](https://github.com/go-ruby-opentelemetry/opentelemetry), [`opentype`](https://github.com/go-ruby-opentype/opentype), [`optparse`](https://github.com/go-ruby-optparse/optparse), [`ostruct`](https://github.com/go-ruby-ostruct/ostruct), [`pagy`](https://github.com/go-ruby-pagy/pagy), [`paper-trail`](https://github.com/go-ruby-paper-trail/paper-trail), [`parquet`](https://github.com/go-ruby-parquet/parquet), [`parser`](https://github.com/go-ruby-parser/parser), [`pathname`](https://github.com/go-ruby-pathname/pathname), [`pg`](https://github.com/go-ruby-pg/pg), [`prawn`](https://github.com/go-ruby-prawn/prawn), [`prettyprint`](https://github.com/go-ruby-prettyprint/prettyprint), [`prime`](https://github.com/go-ruby-prime/prime), [`protobuf`](https://github.com/go-ruby-protobuf/protobuf), [`pstore`](https://github.com/go-ruby-pstore/pstore), [`public-suffix`](https://github.com/go-ruby-public-suffix/public-suffix), [`puma`](https://github.com/go-ruby-puma/puma), [`pundit`](https://github.com/go-ruby-pundit/pundit), [`puppet`](https://github.com/go-ruby-puppet/puppet), [`puppet-resource-api`](https://github.com/go-ruby-puppet-resource-api/puppet-resource-api), [`racc`](https://github.com/go-ruby-racc/racc), [`rack`](https://github.com/go-ruby-rack/rack), [`rails`](https://github.com/go-ruby-rails/rails), [`railties`](https://github.com/go-ruby-railties/railties), [`rake`](https://github.com/go-ruby-rake/rake), [`ransack`](https://github.com/go-ruby-ransack/ransack), [`rdoc`](https://github.com/go-ruby-rdoc/rdoc), [`redis`](https://github.com/go-ruby-redis/redis), [`regexp`](https://github.com/go-ruby-regexp/regexp), [`reline`](https://github.com/go-ruby-reline/reline), [`resolv`](https://github.com/go-ruby-resolv/resolv), [`resque`](https://github.com/go-ruby-resque/resque), [`rexml`](https://github.com/go-ruby-rexml/rexml), [`roda`](https://github.com/go-ruby-roda/roda), [`rolify`](https://github.com/go-ruby-rolify/rolify), [`rouge`](https://github.com/go-rouge/rouge), [`rqrcode`](https://github.com/go-ruby-rqrcode/rqrcode), [`rspec`](https://github.com/go-ruby-rspec/rspec), [`rss`](https://github.com/go-ruby-rss/rss), [`rubocop`](https://github.com/go-ruby-rubocop/rubocop), [`rubygems`](https://github.com/go-ruby-rubygems/rubygems), [`saml`](https://github.com/go-ruby-saml/saml), [`sass`](https://github.com/go-ruby-sass/sass), [`scanf`](https://github.com/go-ruby-scanf/scanf), [`securerandom`](https://github.com/go-ruby-securerandom/securerandom), [`semantic-puppet`](https://github.com/go-ruby-semantic-puppet/semantic-puppet), [`sequel`](https://github.com/go-ruby-sequel/sequel), [`shellwords`](https://github.com/go-ruby-shellwords/shellwords), [`shrine`](https://github.com/go-ruby-shrine/shrine), [`sidekiq`](https://github.com/go-ruby-sidekiq/sidekiq), [`simplecov`](https://github.com/go-ruby-simplecov/simplecov), [`sinatra`](https://github.com/go-ruby-sinatra/sinatra), [`slim`](https://github.com/go-ruby-slim/slim), [`sodium`](https://github.com/go-ruby-sodium/sodium), [`sqlite3`](https://github.com/go-ruby-sqlite3/sqlite3), [`strscan`](https://github.com/go-ruby-strscan/strscan), [`thor`](https://github.com/go-ruby-thor/thor), [`timecop`](https://github.com/go-ruby-timecop/timecop), [`toml`](https://github.com/go-ruby-toml/toml), [`tsort`](https://github.com/go-ruby-tsort/tsort), [`typhoeus`](https://github.com/go-ruby-typhoeus/typhoeus), [`tzinfo`](https://github.com/go-ruby-tzinfo/tzinfo), [`unicode-normalize`](https://github.com/go-ruby-unicode-normalize/unicode-normalize), [`uri`](https://github.com/go-ruby-uri/uri), [`vcr`](https://github.com/go-ruby-vcr/vcr), [`warden`](https://github.com/go-ruby-warden/warden), [`webauthn`](https://github.com/go-ruby-webauthn/webauthn), [`webmock`](https://github.com/go-ruby-webmock/webmock), [`webrick`](https://github.com/go-ruby-webrick/webrick), [`widgets`](https://github.com/go-ruby-widgets/widgets), [`xslt`](https://github.com/go-xslt/xslt), [`yaml`](https://github.com/go-ruby-yaml/yaml), [`zeitwerk`](https://github.com/go-ruby-zeitwerk/zeitwerk), [`zlib`](https://github.com/go-ruby-zlib/zlib), [`activeldap`](https://github.com/go-ruby-activeldap/activeldap), [`ldap`](https://github.com/go-ruby-ldap/ldap), [`fast-gettext-locale`](https://github.com/go-ruby-fast-gettext-locale/fast-gettext-locale) — each at `github.com/go-ruby-/`, except `commonmark`, `kramdown`, `liquid`, `mustache`, `nokogiri`, `rouge` and `xslt`, which are bound from `github.com/go-/`. (`set` is **not** bound: `Set` is implemented in-tree, over rbgo's own `object.Hash`.) The scientific / container stack ([go-ndarray](https://github.com/go-ndarray/ndarray), -[go-fft](https://github.com/go-fft/fft), [go-images](https://github.com/go-images/images), +[go-fft](https://github.com/go-fft/fft), [go-numerics](https://github.com/go-numerics), +[go-statistics](https://github.com/go-statistics), [go-images](https://github.com/go-images/images), [go-composites](https://github.com/go-composites)) binds in the same way. ## How it differs from goruby diff --git a/docs/scientific-stack.md b/docs/scientific-stack.md index a9c9d9c..a73d062 100644 --- a/docs/scientific-stack.md +++ b/docs/scientific-stack.md @@ -49,6 +49,99 @@ bin-frequency helpers (`fftfreq`/`rfftfreq`), windows p FFT.fft([1, 2, 3, 4]).map { |c| c.real.round(3) } # => [10.0, -2.0, -2.0, -2.0] ``` +## Special — scipy.special-style functions + +Binds [go-numerics/special](https://github.com/go-numerics/special) — no SciPy: +**all 67** of its functions under 66 Ruby names, which are **scipy's own**. The +names come from the correspondence that module measures itself against in ULPs +(`bench/scipy_server.py` and `testdata/gen/fam_*.py`), so `Log1pExp` is +`softplus` and `LogGamma` is `gammaln` rather than a transliteration of the Go +name. Gamma (`gamma`/`gammaln`/`gammasgn`/`digamma`/`polygamma`/`factorial`), +beta (`beta`/`betaln`/`binom`/`comb`/`perm`), error and normal +(`erf`/`erfc`/`erfcx`/`erfinv`/`erfcinv`/`ndtr`/`ndtri`/`log_ndtr`), logistic +(`expit`/`logit`/`softplus`/`xlogy`), reductions +(`logsumexp`/`softmax`/`log_softmax`), `zeta` (Riemann, or Hurwitz with a second +argument), the incomplete gamma and beta functions with their inverses, Bessel +(`j0`…`k1e`, `jv`/`yv`/`iv`/`kv`, `jn_zeros`) and the complete elliptic +integrals. + +Each is a **ufunc**, as scipy's is: a Float answers a Float, an `NDArray` +answers an `NDArray` of the same shape, and a Float mixed with an array +broadcasts. + +```ruby +p Special.gamma(5.0) # => 24.0 +p Special.erf(1.0) # => 0.8427007929497149 +p Special.zeta(2.0) # => 1.6449340668482264 +p Special.gamma(NDArray.from([1.0, 2.0, 3.0, 4.0], [2, 2])).to_a # => [1.0, 1.0, 2.0, 6.0] +``` + +## Stats — scipy.stats-style distributions + +Binds [go-statistics/stats](https://github.com/go-statistics/stats) — no SciPy: +the **32 univariate distributions**, again under scipy's names, which is why +`StudentT` is `t`, `ChiSquared` is `chi2`, `DiscreteUniform` is `randint` and +`Weibull` is `weibull_min`. Continuous +`norm`/`cauchy`/`expon`/`gumbel_r`/`halfnorm`/`laplace`/`logistic`/`rayleigh`/ +`uniform`/`chi`/`chi2`/`gamma`/`invgamma`/`lognorm`/`lomax`/`pareto`/`t`/ +`triang`/`vonmises`/`weibull_min`/`beta`/`f`/`ncx2`/`reciprocal`; discrete +`bernoulli`/`geom`/`poisson`/`zipf`/`binom`/`nbinom`/`randint`/`hypergeom`. + +Arguments follow scipy's order — the shape parameters, then `loc`, then `scale` +— and every distribution answers the same method set, `Stats::Continuous` and +`Stats::Discrete` differing only in the density: `pdf`/`logpdf` against +`pmf`/`logpmf`, plus `cdf`/`logcdf`/`sf`/`logsf`/`ppf`/`isf`, the moments +`mean`/`var`/`std`/`skew`/`kurtosis`/`median`/`entropy`/`moment`, +`interval`/`support`, and `rvs(size…, seed:)`. The per-point methods are ufuncs +too. A parameter outside its domain is refused at construction. + +```ruby +p Stats.norm.pdf(0.0) # => 0.3989422804014327 +p Stats.norm.ppf(0.975) # => 1.9599639845400538 +p Stats.poisson(3.0).pmf(2) # => 0.22404180765538775 +p Stats.binom(10, 0.5).mean # => 5.0 +``` + +## Integrate — scipy.integrate-style quadrature + +Binds [go-numerics/integrate](https://github.com/go-numerics/integrate) — no +SciPy: adaptive `quad` (finite **or infinite** limits, `points:` for a known +kink), the fixed rules `fixed_quad` and `romberg`, the sampled rules +`trapezoid`/`simpson`/`romb`/`cumulative_trapezoid` (`x:` or `dx:`), the +multiple integrals `dblquad` and `nquad`, and `gauss_legendre` for a rule's +nodes and weights. + +The integrand is a **block**, which is how Ruby passes a callable — and which +takes `&f` for a Proc or lambda without any further surface. `quad` answers +scipy's `(value, abserr)` pair, so it destructures the same way. + +```ruby +v, err = Integrate.quad(0.0, 1.0) { |x| x * x } +p v # => 0.3333333333333333 +p Integrate.quad(0.0, Float::INFINITY) { |x| Math.exp(-x) }[0] # => 1.0 +p Integrate.trapezoid([0.0, 1.0, 4.0], dx: 1.0) # => 3.0 +``` + +## Optimize — scipy.optimize-style root finding and minimisation + +Binds [go-numerics/optimize](https://github.com/go-numerics/optimize) — no +SciPy: the bracketing root finders `brentq`/`brenth`/`bisect`/`ridder` and +`newton` (secant, Newton–Raphson with `fprime:`, Halley with `fprime2:`), +`minimize_scalar` and `fminbound`, `minimize` over several variables +(`BFGS`/`CG`/`Nelder-Mead`/`Powell`/`L-BFGS-B`, with `bounds:`), plus +`curve_fit`, `approx_fprime` and `bracket`. + +Return shapes are scipy's: one number is a Float, a tuple an Array (`curve_fit` +is `(popt, pcov)`), and an `OptimizeResult` or `RootResults` a Hash with symbol +keys — which `full_output: true` asks a root finder for. + +```ruby +p Optimize.brentq(0.0, 2.0) { |x| x * x - 2 } # => 1.4142135623731364 +p Optimize.minimize_scalar { |x| (x - 3.0) ** 2 }[:x] # => 3.0 +r = Optimize.minimize([0.0, 0.0]) { |v| (v[0] - 1.0) ** 2 + (v[1] - 2.0) ** 2 } +p r[:x].map { |z| z.round(6) } # => [1.0, 2.0] +``` + ## Image — scikit-image-style processing Binds [go-images](https://github.com/go-images/images): `Image.new`/`load`/`save` diff --git a/tools/docexample/main.go b/tools/docexample/main.go new file mode 100644 index 0000000..fedeb84 --- /dev/null +++ b/tools/docexample/main.go @@ -0,0 +1,280 @@ +// docexample runs the Ruby examples on this site and checks each one against +// the answer written beside it. +// +// Why it exists: the pages teach by example, and an example carries its answer +// in a comment -- `p Special.gamma(5.0) # => 24.0`. Nothing ran them. A +// binding can be renamed, a default changed or a return shape altered in the +// ruby repository and every example here keeps its old answer, which reads as +// a promise and is the first thing a reader tries. tools/conformancefigure +// already guards the one NUMBER this site states; this guards the dozens of +// OUTPUTS it states. +// +// WHAT IT JUDGES. A fenced ```ruby block is checked exactly when it claims an +// output -- that is, when some line in it ends in `# => something`. A block +// with no such comment claims nothing and is skipped, which is what makes the +// selection self-evident rather than a list to maintain: the examples that read +// a photo off disk or start a server carry no `# =>` and so are left alone, and +// one that gains an answer is picked up without being registered anywhere. +// +// HOW IT COMPARES. The block runs as one program, so its earlier lines can set +// up the later ones, and the lines it PRINTS are compared in order against the +// `# =>` values it claims. A differing count is itself a failure: an example +// that prints two lines while claiming one is as wrong as a wrong number, and +// the common cause -- `p a, b` printing twice -- is exactly the kind of thing +// nobody notices by reading. +// +// Trailing whitespace around a claimed value is ignored, because a doc author +// aligns the comments. Nothing else is normalised: if the answer differs by a +// digit, that is the point. +// +// Exit status: 0 every example agrees; 1 one or more disagree; 2 could not run +// anything (no interpreter given, or it would not build) -- a refusal to +// answer, not a pass; 3 nothing to check was found, which for a site with +// examples means the extraction broke rather than that the site is clean. +package main + +import ( + "bytes" + "flag" + "fmt" + "io/fs" + "os" + "os/exec" + "path/filepath" + "regexp" + "strings" +) + +// fence opens and closes a Markdown code fence, capturing its language. +var fence = regexp.MustCompile("^\\s*```(\\w*)\\s*$") + +// claim matches a line that states its own answer: anything, then a comment of +// the form `# => value`. The value runs to the end of the line. +var claim = regexp.MustCompile(`#\s*=>\s*(.*)$`) + +// example is one fenced block that claims at least one answer. +type example struct { + file string + line int // 1-based line of the opening fence + src string // the block's Ruby, verbatim + want []string // the claimed answers, in order +} + +// extract reads the examples out of one Markdown file. +func extract(path string, md []byte) []example { + var out []example + lines := strings.Split(string(md), "\n") + for i := 0; i < len(lines); i++ { + m := fence.FindStringSubmatch(lines[i]) + if m == nil || m[1] != "ruby" { + continue + } + start := i + var body []string + for i++; i < len(lines); i++ { + if fence.MatchString(lines[i]) { + break + } + body = append(body, lines[i]) + } + var want []string + for _, l := range body { + if c := claim.FindStringSubmatch(l); c != nil { + want = append(want, strings.TrimSpace(c[1])) + } + } + if len(want) == 0 { + continue // claims nothing + } + out = append(out, example{ + file: path, + line: start + 1, + src: strings.Join(body, "\n") + "\n", + want: want, + }) + } + return out +} + +// run evaluates one example and returns the lines it printed. +func run(rbgo string, ex example) ([]string, error) { + f, err := os.CreateTemp("", "docexample-*.rb") + if err != nil { + return nil, err + } + defer os.Remove(f.Name()) + if _, err := f.WriteString(ex.src); err != nil { + return nil, err + } + if err := f.Close(); err != nil { + return nil, err + } + var stdout, stderr bytes.Buffer + cmd := exec.Command(rbgo, f.Name()) + cmd.Stdout, cmd.Stderr = &stdout, &stderr + if err := cmd.Run(); err != nil { + return nil, fmt.Errorf("%v: %s", err, strings.TrimSpace(stderr.String())) + } + got := strings.Split(strings.TrimRight(stdout.String(), "\n"), "\n") + if len(got) == 1 && got[0] == "" { + got = nil + } + return got, nil +} + +func main() { + rbgo := flag.String("rbgo", "", "path to an rbgo binary (required)") + root := flag.String("root", "docs", "directory of Markdown files to read") + verbose := flag.Bool("v", false, "also list the examples that agree") + selftest := flag.Bool("selftest", false, "check the extraction against known cases and exit") + flag.Parse() + + if *selftest { + os.Exit(runSelfTest()) + } + if *rbgo == "" { + fmt.Fprintln(os.Stderr, "docexample: -rbgo is required: without an interpreter this cannot check anything,") + fmt.Fprintln(os.Stderr, "and reporting success would be a pass it did not earn.") + os.Exit(2) + } + if _, err := os.Stat(*rbgo); err != nil { + fmt.Fprintf(os.Stderr, "docexample: -rbgo %q: %v\n", *rbgo, err) + os.Exit(2) + } + + var examples []example + walked := 0 + err := filepath.WalkDir(*root, func(p string, d fs.DirEntry, err error) error { + if err != nil { + return err + } + if d.IsDir() || filepath.Ext(p) != ".md" { + return nil + } + walked++ + b, err := os.ReadFile(p) + if err != nil { + return err + } + examples = append(examples, extract(p, b)...) + return nil + }) + if err != nil { + fmt.Fprintf(os.Stderr, "docexample: walking %s: %v\n", *root, err) + os.Exit(2) + } + if walked == 0 { + fmt.Fprintf(os.Stderr, "docexample: read no Markdown under %s -- nothing was checked\n", *root) + os.Exit(3) + } + if len(examples) == 0 { + fmt.Fprintf(os.Stderr, "docexample: %d Markdown file(s) and no example claiming an answer.\n", walked) + fmt.Fprintln(os.Stderr, "For a site that teaches by example that means the extraction broke, not that it is clean.") + os.Exit(3) + } + + bad := 0 + claims := 0 + for _, ex := range examples { + claims += len(ex.want) + got, err := run(*rbgo, ex) + if err != nil { + fmt.Printf("%s:%d: the example claims %d answer(s) but will not run: %v\n", + ex.file, ex.line, len(ex.want), err) + bad++ + continue + } + if len(got) != len(ex.want) { + fmt.Printf("%s:%d: printed %d line(s), claims %d\n printed: %v\n claims: %v\n", + ex.file, ex.line, len(got), len(ex.want), got, ex.want) + bad++ + continue + } + ok := true + for i := range got { + if strings.TrimSpace(got[i]) != ex.want[i] { + fmt.Printf("%s:%d: answer %d is %q, the page says %q\n", + ex.file, ex.line, i+1, strings.TrimSpace(got[i]), ex.want[i]) + ok = false + } + } + if !ok { + bad++ + } else if *verbose { + fmt.Printf("%s:%d: %d answer(s) agree\n", ex.file, ex.line, len(ex.want)) + } + } + + fmt.Printf("%d example(s) with %d claimed answer(s) over %d file(s); %d example(s) disagree\n", + len(examples), claims, walked, bad) + if bad > 0 { + os.Exit(1) + } +} + +// runSelfTest checks the extraction on cases whose answer is known, because a +// pattern nobody tested is a guess and a guess that matches nothing looks +// exactly like a clean site. Returns the process exit status. +func runSelfTest() int { + cases := []struct { + name string + md string + wantEx int // examples found + wantWant []string // the claims of the first one + }{ + { + name: "a ruby block claiming one answer", + md: "text\n\n```ruby\np 1 + 1 # => 2\n```\n", + wantEx: 1, + wantWant: []string{"2"}, + }, + { + name: "set-up lines are kept and only claims are collected", + md: "```ruby\na = 1\np a # => 1\np a + 1 # => 2\n```\n", + wantEx: 1, + wantWant: []string{"1", "2"}, + }, + { + name: "a block claiming nothing is skipped", + md: "```ruby\nImage.load(\"photo.png\").save(\"out.png\")\n```\n", + wantEx: 0, + }, + { + name: "a comment that is not an answer is not a claim", + md: "```ruby\np BigDecimal(\"0.1\") # exact, no 0.30000000000000004\n```\n", + wantEx: 0, + }, + { + name: "a shell block is not Ruby", + md: "```sh\necho hi # => hi\n```\n", + wantEx: 0, + }, + { + name: "a value containing a # is kept whole", + md: "```ruby\np \"a#b\" # => \"a#b\"\n```\n", + wantEx: 1, + wantWant: []string{`"a#b"`}, + }, + } + bad := 0 + for _, c := range cases { + got := extract("t.md", []byte(c.md)) + if len(got) != c.wantEx { + fmt.Printf("selftest %q: found %d example(s), want %d\n", c.name, len(got), c.wantEx) + bad++ + continue + } + if c.wantEx == 0 { + continue + } + if strings.Join(got[0].want, "|") != strings.Join(c.wantWant, "|") { + fmt.Printf("selftest %q: claims %v, want %v\n", c.name, got[0].want, c.wantWant) + bad++ + } + } + fmt.Printf("selftest: %d case(s), %d failure(s)\n", len(cases), bad) + if bad > 0 { + return 1 + } + return 0 +}