Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 3 additions & 4 deletions content/en/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,9 @@ description: "Go notebooks with native speed, kept values and a reproducible ord
---

**A notebook is a Markdown file whose ` ```go ` blocks are cells**, and its
` ```ruby ` blocks too ([Ruby cells](ruby.md)); ` ```r ` and ` ```fortran `
` ```ruby ` blocks too ([Ruby cells](ruby.md)); ` ```r `, ` ```fortran ` and ` ```julia `
blocks run in a session of their own ([R cells](r.md),
[Fortran cells](fortran.md)); ` ```julia ` blocks are parsed and checked, not
run yet ([Julia cells](julia.md)). The go-notebook kernel:
[Fortran cells](fortran.md), [Julia cells](julia.md)). The go-notebook kernel:

- **compiles it** into a real Go program, so cells run at native speed;
- **orders the cells by what they use**, worked out from the code itself as
Expand Down Expand Up @@ -84,7 +83,7 @@ is not summed again. (A real transcript, from `notebook` built at main
| [Notebooks](notebooks.md) | what a notebook file is, and why it is Markdown |
| [Cells](cells.md) | the rules a cell follows, and the errors that enforce them |
| [Ruby cells](ruby.md) | Ruby cells with the go-scikit bindings, how values cross between Go and Ruby, and the limits |
| [Julia cells](julia.md) | Julia cells: parsed and checked with go-embedded-julia's parser, their names in the graph — not run yet |
| [Julia cells](julia.md) | Julia cells: parsed with go-embedded-julia's parser, run by its interpreter in one kernel, their names in the graph |
| [R cells](r.md) | R cells: parsed with go-embedded-r's parser and run on its evaluator in one shared environment, their names in the graph, dynamic cells, limits |
| [Fortran cells](fortran.md) | Fortran cells: run on go-embedded-fortran in one session shared by the Fortran cells, kept apart from the other languages, and their limits |
| [Using the playground](playground.md) | the browser playground: editing, adding, running, switching language, deleting and moving cells, with Jupyter's keys; its limits |
Expand Down
4 changes: 2 additions & 2 deletions content/en/adding-a-language.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,8 @@ type Names struct {
depend on **it**, so it runs after anything before it changed and
invalidates whatever may read what it defined.
4. **Say whether the cells run.** `Run{Runs: true}` runs them. Otherwise the
language is **check-only**: `NotRun` is the cell's failure (Julia's names
its milestone M3), and `Blocked`, `Reads` and `ReadsBlocked` are the
language is **check-only**: `NotRun` is the cell's failure (C's names
its milestone M2), and `Blocked`, `Reads` and `ReadsBlocked` are the
messages of the cells that read it, directly or through other cells.
5. **Optional capabilities**, found by type assertion:

Expand Down
164 changes: 91 additions & 73 deletions content/en/julia.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,86 @@
---
title: "Julia cells"
weight: 26
description: "A ```julia block is a Julia cell: parsed and checked with go-embedded-julia's parser, its names in the dependency graph — but not run yet."
description: "A ```julia block is a Julia cell: parsed with go-embedded-julia's parser, its names in the dependency graph, and run by its interpreter in one kernel."
tags: [cells, julia]
---

A ` ```julia ` block is a Julia cell (go-notebook v0.6.0 and later). Julia
cells are **parsed and checked, but not run yet**:
[go-embedded-julia](https://github.com/go-embedded-julia/julia), the pure-Go
Julia these cells are written for, has a complete parser today, and its
interpreter is being written. The
[plan](https://github.com/go-scikit/roadmap/blob/main/bib/julia.md) has the
milestones: M1, the parser, is done; M2 (lowering) is in progress; M3, the
runtime, is the one that makes Julia cells run.
A ` ```julia ` block is a Julia cell (parsed since go-notebook v0.6.0, **run**
in go-notebook v0.18.0).
[go-embedded-julia](https://github.com/go-embedded-julia/julia) is the
pure-Go Julia these cells run on: its parser for the names and the syntax
errors, its interpreter (v0.15.0) for the running. No external `julia` is
started.

## What works
## Running

The cells of a notebook share one Julia kernel, in the worker. A global, a
function or a struct a cell defines is there in the next cell; what a cell
prints is its own output, and the value of its last expression is shown as the
Julia REPL shows it, unless it is `nothing` or the cell ends in a semicolon.

````markdown
```julia
square(x) = x^2
total = sum(square(i) for i in 1:4)
```

```julia
println("total = ", total)
total + 1
```
````

```console
$ notebook run julia.md
── cell 0 (line 4, ran in 1.102209s)
30
── cell 1 (line 9, ran in 1.478ms)
total = 30
31
```

(A real transcript, from the change that became go-notebook v0.18.0, on an Apple M4 Max; the
first cell pays the kernel's cold start.)

An exception fails the cell with julia's text of it after `ERROR:`, after the
output the cell wrote before it; the cells that read it do not run, and the
kernel keeps what it had.

**The image.** The kernel boots from go-embedded-julia's prebuilt Base image,
which is not part of the module. The first Julia cell a worker runs fetches the
image of release v0.15.0 and loads it only if its SHA-256 is the one go-notebook
pins; the verified file is kept in your user cache directory
(`go-embedded-julia/images`) and checked again each time. The fetch is bounded
to five minutes and tried once. The cold start is about a second. On a machine
that is offline and has no cached image every Julia cell of the run fails at
once with *the Julia kernel could not start*, and the other cells run.

**Limits.** A notebook may come from anyone, so each cell is bounded, as an R
cell is: 30 seconds, 50 million statements, a memory budget of 512 MiB, and
16 MiB of output per stream. The memory budget counts the elements of the arrays
and dictionary tables a cell allocates, in total and garbage included, at 16
bytes each. It is a floor, not the heap: a boxed value costs more, strings are
not counted, and one array may not have more than 2^24 elements whatever the
budget. A cell over a limit is stopped and says which, in the kernel's words;
`--timeout` and `--memory` still apply to the worker. The kernel is sandboxed:
its file system holds julia's own tree and its environment is empty.

**Known limitations** (go-embedded-julia v0.15.0):

- At the top level of a cell, a loop that updates a global raises
`UndefVarError`, where julia 1.12's REPL updates it
([#54](https://github.com/go-embedded-julia/julia/issues/54)). Put the loop in
a function.
- Some exceptions, `MethodError` among them, show as their Go representation,
not as julia's message
([#55](https://github.com/go-embedded-julia/julia/issues/55)).
- A process loads one image release
([#49](https://github.com/go-embedded-julia/julia/issues/49)).

## Parsing and names

The names below are the same whether a cell runs or not.

**Parsing.** A Julia cell is parsed by go-embedded-julia's port of
JuliaSyntax, the parser julia 1.12 ships, which gives the same `Expr` trees as
Expand Down Expand Up @@ -62,69 +128,21 @@ lowered by julia 1.12.6 (through `REPL.softscope`, the soft-scope rule of the
REPL), and the globals julia declares and reads are the names go-notebook
finds.

**Across languages.** A Julia cell's variables are seen by Go and Ruby cells,
and a Julia cell sees theirs, as between Ruby and Go (see
[Ruby cells](ruby.md)); functions, macros, constants and types stay in Julia.

## What does not work yet: running

Running a Julia cell fails, clearly, and the cells that need it say why:

````markdown
```go
n := 10
```

```julia
xs = [i^2 for i in 1:n]
total = sum(xs)
```

```go
total + 1
```

```go
n * 2
```
````

```console
$ notebook run julia.md
── cell 0 (line 4, ran in 11µs)
── cell 1 (line 8, not run)
error: Julia cells are parsed and checked, but not run yet: execution arrives with go-embedded-julia's runtime (milestone M3)
── cell 2 (line 13, not run)
error: blocked: it depends on cell 1, a Julia cell, and Julia cells are not run yet
── cell 3 (line 17, ran in 28µs)
20
notebook: 2 cell(s) failed
```

(Real transcripts, from go-notebook's main branch on an Apple M4 Max.)

- a Julia cell never runs, and `notebook run` exits non-zero;
- every cell that reads what a Julia cell defines — directly or through other
cells, in any language — is **blocked**, not silently skipped;
- the other cells run as usual.

go-notebook starts no external `julia`: the cells will run on
go-embedded-julia's own runtime, in the same worker as Go and Ruby cells,
when it exists. The HTML page shows a Julia cell's source uncoloured (go-rouge
has no Julia lexer yet).
**Across languages.** Julia variables stay in Julia, as R's do: they are not
shared with Go, Ruby, R, C or Fortran cells in either direction yet. A Go cell
that reads a Julia name finds it undefined, and a Julia cell does not see a Go
variable; a Julia result reaches the reader as printed text. Between Julia
cells every name is shared. The HTML page shows a Julia cell highlighted with
go-rouge's `julia` lexer.

## In the browser

The [playground](https://go-notebook.github.io/playground/) parses and checks
Julia cells in the same way, in WebAssembly: a syntax error is shown at its
line, the cell's badge reads **Julia — not runnable yet**, and the cells that
read it are blocked. A Julia cell that parses is shown as *not run*, and a cell
that reads it as *blocked*, in the page's neutral style and not as failures
(the playground's results carry a typed `status`, `notrun` or `blocked`, and
its site build accepts exactly those two for a Julia example, while a syntax
error still fails it).

Try it: the [Julia example notebook](https://go-notebook.github.io/playground/#4-julia)
has a Julia cell, a Go cell blocked on it and a Go cell that runs alone, and its
prose says that Julia cells are checked, not run, until
[go-embedded-julia](https://go-embedded-julia.github.io/)'s runtime lands.
The [playground](https://go-notebook.github.io/playground/) does not run Julia
cells: its WebAssembly core parses and checks them, the cell's badge reads
**Julia — not runnable yet**, and the cell says *Julia cells run in
go-notebook, not yet in the browser*. The page treats Julia as check-only by
language, so a Julia cell never reaches its Go evaluator. Running them there is
a separate question, pending with go-embedded-julia.

Try the [Julia example notebook](https://go-notebook.github.io/playground/#4-julia)
there for the checks; `notebook run` and `notebook serve` run the cells.
Loading