Question about instances

[zherczeg]

I am trying to implement a component text/binary parser, and a lot of things are confusing for me. These are validated examples.

(component $C
  (type s32)
  (instance (export "inline" (type 0)))
)

What kind of component does the instance create? $C?

(component
  (type (component
    (import "x" (instance $I (export "f" (func))))
    (alias export $I "f" (func))
  ))
)

For me a type should contain only types or some descriptors, but it seems any mutable objects can be added a to (non-mutable?) type.

The most confusing part is how instances are connected? E.g. the (func (param "X" (list (list s32)))) is the import and export of two components, and the import side wants to call this function. What should happen? How this is represented in core module side? How the data is transferred?

[lukewagner]

What kind of component does the instance create?

The type of that instance in $C would be:

(instance
  (type $t s32)
  (export "inline" (type (eq $t)))
)

For me a type should contain only types or some descriptors, but it seems any mutable objects can be added a to (non-mutable?) type.

Ah, I'm guessing you're commenting on the alias declarator (alias export $I "f" (func)), particularly on the func. Yes agreed that is confusing and indeed Binary.md contains a bullet:

• Validation of instancedecl (currently) only allows the type and instance sorts in alias declarators.

@alexcrichton this second snippet in the OP currently validates in wasm-tools; is that a bug or am I missing something here?

In any case, while that alias declarator binds a new funcidx, I don't think there's anything you can subsequently do with that funcidx (to make it show up in an import/export declarator), so I think it's effectively "dead code" atm, unless, again, I'm confused about something. So great question here and thanks for asking!

What should happen? How this is represented in core module side? How the data is transferred?

For this, let me point you to the Canonical Definitions section of the explainer. Specifically, the exporting component will canon lift a core function into a component-typed function and the importing component will canon lower the imported component-typed function to produce a new core function that can be imported and called by core wasm.

[alexcrichton]

is that a bug or am I missing something here?

That looks like a bug yeah, good catch @zherczeg! I'll try to dig in and see what's going on today.

[syntactically]

(component $C
  (type s32)
  (instance (export "inline" (type 0)))
)

Just in case it's helpful, I started writing a little more detail on this one before Luke responded above. Here we are:

• Defining a concrete component, which we can refer to as $C later and in its body if necessary
• Creating an entry in the type index space of that component, which contains s32
• Creating an entry in the instance index space which contains an instance which exports a type, the exported type being equal to s32.

We have not imported or exported anything from the overall component yet, so if you were to instantiate the component $C, it would create this instance, but nothing particularly interesting would happen. The component's type is (component), or, as written in the formalism $\{\} \to \{\}$, because it has no imports or exports.

You could add something like (export "i" (instance 0)) to the example component. Then, C would export the instance it had just defined, and you would get something like (component (export "i" (instance (export "inline" (type (eq s32)))))) (or, after elaboration, in the formal syntax, something like, $\exists (\tau : (\mathrm{eq}\ s32)). \{\} \to \{ \mathrm{i} : \{ \mathrm{inline} : \tau \} \}$.

(component
  (type (component
    (import "x" (instance $I (export "f" (func))))
    (alias export $I "f" (func))
  ))
)

I agree with you a little bit that the syntax here is a little confusing! As Luke mentioned, I don't think that this has meaning with func as that sort. The reason why we can generally have instances (which are also stateful) in these alias declarators is that it gives us a surface syntax consistent with the rest of the syntax to get at the exported types inside of those instances (which are per-instance, due to generativity), like ensuring that a type that a component exports is the same as one that it imports. Concretely:

(type (component
  (import "x" (instance $I (export "y" (instance (export "t" (type (sub resource)))))))
  (alias export $I "y" (instance $J))
  (alias export $J "t" (type $t))
  (export "z" (instance (export "u" (type (eq $t)))))))

The type here elaborates to something like $\forall (\tau : (\mathrm{sub\ resource})). \{ \mathrm{x} : \{ \mathrm{y} : \{ \mathrm{t} : \tau \} \} \} \to \{ \mathrm{z} : \{ \mathrm{u} : \tau \} \}$. I.e. "This component imports an instance, which itself exports an instance which exports a resource type. This component than exports an instance which exports the same resource type."

You may find this (rather out of date) Menhir parser for the text format from an old draft of the reference interpreter to be a useful reference. There will be a newer reference interpreter sometime soon.

[zherczeg]

Thank you for the explanations!

Creating an entry in the instance index space which contains an instance which exports a type, the exported type being equal to s32.

So the point of these inlined instances is exporting something form their owner component without creating another (helper) component, isn't it? These instances don't have a real source component, only contains a list of (mutable) exports.

How can I declare version suffixes? The explainer says: versionsuffix ::= (versionsuffix "<semversuffix>") but I got a parse error for this:

(component
  (type s32)
  (export "any" (versionsuffix "1.0.0") (type 0))
)

[alexcrichton]

These instances don't have a real source component, only contains a list of (mutable) exports.

Correct, yeah.

How can I declare version suffixes?

I beileve that's not implemented in wasm-tools just yet, which would explain the parse error (assuming that's what you're using)

[zherczeg]

I have a few more questions.

• (import "res" (type (sub resource))) How do you know that the (rep ...) of the resource is i32 or i64? If I understand correctly, borrow and own needs this information.

The canonical explainer says this:
case OwnType() : return lift_own(cx, load_int(cx, ptr, 4), t)

The size can be 8 for i64 rep, or am I misunderstand something?

• (import "type" (type (eq 0))) What is imported here exactly? Some kind of object? What is the purpose if this declaration?

• When you pass data with lifted/lowered functions, do you need to optimize shared buffer readings? I.e. if you pass 5 u8 lists to a lifted function, which refer to the same, 1 MByte data, do you allocate 5 * 1MB data on the lowered side, or do you need to recognize that they are the same lists and copy only 1 MB and set start pointers accordingly? This can be rather complex for intersected regions, i.e. the 1 MB data starts from offset 0, 5, 10, ... so you only need to copy 1MB + 4*5 byte data.

[lukewagner]

Hi! Great questions. Answers in order:

• The load_int(cx, ptr, 4) passed to lift_own() is loading the i32 index of the owned handle in the current component instance's handles table. The rep (which is indeed either i32 or i64) is stored in the handle element, and thus the index can always be i32. The only resource operations that need to know whether rep is i32 or i64 are resource.new $rt and resource.rep $rt, and validation rules specify that these may only be used in the same component instance that defines $rt (thus keeping rep an encapsulated implementation detail).
• (import "foo" (type (eq 0))) says that the foo import argument must be a type that is equal to whatever type is at typeidx 0 (which itself can be an import). They're not super useful, but do help preserve the original WIT document structure. On exports, eq constraints are actually useful, since they allow a single type be exposed to clients under multiple names (e.g., in multiple exported WIT interfaces) while preserving the fact that all those distinct names refer to the same type and are thus type-compatible.
• No, there is no deduplication on the lowering side: the aliased data on the lifting side will be copied each time.

[zherczeg]

Thank you. It seems I totally misunderstood how resources work. I also saw an example for export aliasing a resource type from an instance with the eq definition. Probably it is not possible to do that without eq.

It seems to me components only use the original WebAssembly types: i32/i64/f32/f64, all of its type definitions can be translated to them. The newer types such as v128, (ref [null]), externref, etc. are not used. Is this correct? Probably there were no other options when components were first created. GC support (WebAsembly 3.0) is good for passing any data between instances. Is there any plan to use them?

[lukewagner]

You're right that the reason that only the original four core wasm value types are used in the Canonical ABI is because that's all there was (stable) when we first released Preview 2. Including v128 would require adding an analogous component-level type (including to WIT), which hypothetically we could do, but there's not a strong motivation yet. But GC and reference types do have motivating use cases, with a number of folks keenly interested (viz., those with languages targeting wasm-gc). There's a pre-proposal in #525 and hopefully someone will have the resources to start working on it before long.

[zherczeg]

Thank you. My next question is about transmitting data between modules. I am trying to figure this about, but the text in canonicalABI is confusing for me.

Lets assume a function with type (type (func (param "P" (list (list s8))))) is present. We should use it for transmitting data from module A to B. Module A has memory MA, and module B has memory MB, the size of these memories is 32 bit.

If I understand correctly the outer list is represented by a length and a start values, represented by two i32 arguments for a core function. The start value points to a memory address, which contains similar pairs. These pairs represent the s8 arrays. Everything is stored in the MA memory.

When I transmit the data, shall I copy the whole argument into a hidden buffer (i.e. computing the linear size for the whole thing and serialize all data into it)? Or can I directly copy it into MB? In MB, shall I allocate memory with the allocator function in the canonical options for each array, or compute the size of the whole thing, and allocate memory with a single alloc, and then serialize? The start values must be updated to contain offsets in MB (so they are not just copied).

Or something else happens?

[lukewagner]

Good question! While the CABI Python code looks like it might require an intermediate copy into a hidden buffer, that's more of a spec mechanism (to decouple component-to-component and component-to-host) and by-design you can do a direct copy from, in your example, MA to MB. E.g., Wasmtime's impl, when compiling a component-to-component call, the compiler generates a wasm trampoline (sitting between the caller and callee) that has access to both MA and MB, and this lets the trampoline look at the outer list's length in MA, then call realloc in MB for the elements of the outer list, then look at the outer list's first element's length in MA, the call realloc in MB, then copy the elements into MB, then write the pointer/length into the outer list's first element in MB, then repeat for each element, finally passing the pointer/length of the outer list into MB's core wasm entry point.

[zherczeg]

I am sorry, I still have more questions. This time about resources. Resources has 32 bit handles. I compiled a "hello world" with clang (wasi 0.2), and it seems that it expects that a signed number for handle is >= 0, and it seems the value 0 is considered as an error in some cases. Are handle values limited to [1 .. INT32_MAX]?

Resources has h.num_lends and h.borrow_scope.num_borrows in the canonical abi. Are borrowed resources tracked in different ways? If a resource is recursively borrowed, how can h.num_lends track it? It seems resources are copied when borrowed, and they don't track their original owner.

I tried to implement resources as objects, which are passed around. The handle array contains pointers. The lowest two bits are always 0, so one bit is used to represent borrowed resources. This way passing around them is cheap. Is this implementation correct, or there are cases when it is incorrect?

Is it possible to pass a resource as both borrow and own to another component? In this case an instance has both a borrowed and owned reference to the same resource.