diff --git a/README.md b/README.md index 94386e15ea..e358a480c6 100644 --- a/README.md +++ b/README.md @@ -34,14 +34,32 @@ references. directly in the schema, alongside numbers, strings, lists, maps, arrays, enums, structs, and unions. Define schemas once, then generate native domain objects for each language without forcing wrapper types into user code. -- **Row-Format Random Access**: Read fields, arrays, and nested values without - rebuilding full objects, with zero-copy access and partial reads. - **Optimized Implementations**: Java JIT serializers and generated/static serializers in other language implementations keep hot paths fast and payloads compact. - **Language And Platform Support**: Java, Python, C++, Go, Rust, JavaScript/TypeScript, C#, Swift, Dart, Scala, and Kotlin, including GraalVM native image, Android, Dart VM/Flutter/web, and Node.js/browser JavaScript. +For same-language workloads, Fory provides native serialization modes that +support broader language-specific object models: + +- **Java Native Serialization**: A high-performance replacement for JDK + serialization, Hessian, Kryo, and FST in Java-only systems. It supports JDK + custom serialization semantics. +- **Python Native Serialization**: A faster and more compact replacement for + `pickle` and `cloudpickle` in Python-only systems. It supports classes, + modules, functions, and custom object state, with fine-grained + deserialization controls through `DeserializationPolicy`. + +Fory also provides specialized formats for other data-processing requirements: + +- **Row Format**: Read fields, arrays, and nested values without rebuilding + complete objects, with zero-copy access and partial reads. +- **Fory JSON**: A Java JSON serialization framework built for maximum + throughput through runtime-generated codecs and optimized readers and + writers. Supports Java 8 and later on standard JDKs, GraalVM native images, + and Android, including Java 17 records. + ## Performance Benchmarks show Fory delivering higher throughput and smaller serialized @@ -292,13 +310,14 @@ See [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md). Snapshots for Java, Scala, and Kotlin are available from `https://repository.apache.org/snapshots/` with the matching `-SNAPSHOT` version. -## Choose Serialization Mode +## Choose a Serialization Format -| Mode | Use it when | Start here | -| ----------- | ------------------------------------------------------------- | -------------------------------------------------------- | -| Xlang mode | Data crosses language boundaries | [Cross-language guide](docs/guide/xlang) | -| Native mode | Producer and consumer are in the same language | Language guide | -| Row format | You need random field access or analytics-style partial reads | [Row format spec](docs/specification/row_format_spec.md) | +| Format | Use it when | Start here | +| ------------- | ------------------------------------------------------------- | -------------------------------------------------------- | +| Xlang binary | Data crosses language boundaries | [Cross-language guide](docs/guide/xlang) | +| Native binary | Producer and consumer are in the same language | Language guide | +| Row format | You need random field access or analytics-style partial reads | [Row format spec](docs/specification/row_format_spec.md) | +| Fory JSON | Java applications need high-performance standard JSON | [Fory JSON guide](docs/guide/java/json-support.md) | For Java, Scala, Kotlin, Python, C++, Go, and Rust, use native mode for same-language traffic. It avoids xlang's cross-language type mapping and @@ -680,11 +699,25 @@ val fory = ForyKotlin.builder() ## Schema IDL -Fory IDL is Fory's schema language for shared data models. It supports -references, nullable fields, lists, maps, arrays, enums, messages, and unions, -and generates native data structures for Java, Python, C++, Go, Rust, -JavaScript/TypeScript, C#, Swift, Dart, Scala, and Kotlin. Use it when multiple -languages need one shared contract. +Fory IDL is Fory's schema-first path for shared data models. Use it when +multiple languages need one explicit contract, stable field identities, and +generated native domain objects instead of manually coordinating equivalent +types in every implementation. + +The schema supports primitive values, nullable fields, lists, maps, dense +arrays, enums, messages, unions, imports, and first-class shared or circular +references. The compiler generates idiomatic models and Fory integration for +Java, Python, C++, Go, Rust, JavaScript/TypeScript, C#, Swift, Dart, Scala, and +Kotlin. It can also generate Fory-backed gRPC service companions for supported +languages. + +Install the compiler from PyPI: + +```bash +pip install fory-compiler +``` + +Define the shared model in `tree.fdl`: ```protobuf package tree; @@ -698,7 +731,18 @@ message TreeNode { } ``` -See the [Fory IDL and compiler guide](https://fory.apache.org/docs/compiler). +Generate native models for the languages used by your application: + +```bash +foryc tree.fdl --lang java,python,rust --output ./generated +``` + +Generated types use each language's normal classes, structs, dataclasses, +annotations, macros, or registration helpers, so application code works with +native domain objects while all peers share the same Fory schema. See the +[Fory IDL and compiler guide](https://fory.apache.org/docs/compiler) for the +complete type system, language-specific output options, schema evolution, and +gRPC generation. ## Row Format @@ -761,6 +805,72 @@ deserialization, see the [Python row-format guide](docs/guide/python/row-format.md), and the [row-format specification](docs/specification/row_format_spec.md). +## Fory JSON + +Fory JSON is a thread-safe JSON serialization framework for Java, extensively +optimized for maximum performance across JSON encoding, decoding, and Java +object mapping. It supports Java 8 and later on standard JDKs, GraalVM native +images, and Android, with Java records supported on Java 17 and later. + +Add Fory JSON to your project: + +**Maven** + +```xml + + org.apache.fory + fory-json + 1.4.0 + +``` + +**Gradle** + +```gradle +implementation "org.apache.fory:fory-json:1.4.0" +``` + +Keep all Fory modules in the same application on the same version. + +**Quick Start** + +Build one `ForyJson` instance and reuse it across threads: + +```java +import org.apache.fory.json.ForyJson; + +public final class JsonExample { + private static final ForyJson JSON = ForyJson.builder().build(); + + public static final class User { + public long id; + public String name; + + public User() {} + + public User(long id, String name) { + this.id = id; + this.name = name; + } + } + + public static void main(String[] args) { + User input = new User(7, "Alice"); + + String text = JSON.toJson(input); + User fromText = JSON.fromJson(text, User.class); + + byte[] bytes = JSON.toJsonBytes(input); + User fromBytes = JSON.fromJson(bytes, User.class); + + System.out.println(fromText.name + " / " + fromBytes.name); + } +} +``` + +See the [Fory JSON guide](docs/guide/java/json-support.md) for installation, +configuration, supported types, custom codecs, and platform-specific setup. + ## Documentation **User Guides** diff --git a/docs/guide/java/index.md b/docs/guide/java/index.md index 9a8cfc4322..0daa4058a6 100644 --- a/docs/guide/java/index.md +++ b/docs/guide/java/index.md @@ -19,45 +19,51 @@ license: | limitations under the License. --- -Apache Fory™ provides blazingly fast Java object serialization with JIT compilation and zero-copy techniques. Java supports both xlang mode and native mode. Xlang mode is the default cross-language wire format and uses compatible schema evolution. Native mode is the Java-only wire format for same-language object serialization, JDK serialization replacement behavior, framework replacement, and Java-native object graph features. +Apache Fory™ Java provides high-performance binary object serialization, a +cross-language random-access row format, and JSON serialization for Java. +Binary serialization supports xlang mode for cross-language payloads and native +mode for Java-only object graphs. [Fory JSON](json-support.md) is a +high-performance JSON serialization framework for Java applications. -Fory also provides a separate [JSON codec](json-support.md) for interoperable text payloads. JSON is -not the native or xlang binary protocol and has its own object-mapping annotations and limits. +## Choose a Format -## Features +| Format | Use it when | Artifact | Start here | +| ------------------------------- | -------------------------------------------------------------------------------- | ----------------------------- | --------------------------------------------- | +| **Binary Object Serialization** | You need compact object graphs in Java native mode or across supported languages | `org.apache.fory:fory-core` | [Basic Serialization](basic-serialization.md) | +| **Row Format** | You need zero-copy random access, partial reads, or Arrow integration | `org.apache.fory:fory-format` | [Row Format](row-format.md) | +| **Fory JSON** | You need high-throughput standard JSON for Java applications | `org.apache.fory:fory-json` | [JSON Support](json-support.md) | -### High Performance +## Binary Object Serialization -- **JIT Code Generation**: Highly-extensible JIT framework generates serializer code at runtime using async multi-threaded compilation, delivering 20-170x speedup through: - - Inlining variables to reduce memory access - - Inlining method calls to eliminate virtual dispatch overhead - - Minimizing conditional branching - - Eliminating hash lookups -- **Zero-Copy**: Direct memory access without intermediate buffer copies; row format supports random access and partial serialization -- **Variable-Length Encoding**: Optimized compression for integers, longs -- **Meta Sharing**: Cached class metadata reduces redundant type information -- **SIMD Acceleration**: Java Vector API support for array operations (Java 16+) +### Features -### Drop-in Replacement +- **Generated Codecs**: JIT-generated serializers reduce virtual dispatch, + branching, and metadata lookups on hot paths. +- **Native and Xlang Modes**: Choose Java-native object semantics or a portable + wire format shared with other Fory implementations. +- **Compact Encoding**: Variable-length integers, metadata sharing, string + compression, and optional numeric-array compression reduce payload size. +- **Object Graph Semantics**: Preserve shared and circular references, + polymorphism, schema evolution, and deep-copy identity. -- **100% JDK Serialization Compatible**: Supports `writeObject`/`readObject`/`writeReplace`/`readResolve`/`readObjectNoData`/`Externalizable` -- **Java 8+ Support**: Works across all modern Java versions including Java 17+ records -- **GraalVM Native Image**: AOT compilation support without reflection configuration -- **Android API 26+ Support**: Core object serialization works on Android without runtime code generation. +### Native Mode Features -### Advanced Features +- **Framework Replacement**: Replace JDK serialization, Kryo, FST, Hessian, or + Java-only Protocol Buffers payloads in Java-only systems. +- **JDK Semantics**: Supports JDK custom serialization behavior and + `Externalizable` in native mode. +- **Security Controls**: Class registration, type checking, depth limits, and + configurable deserialization policies protect decoding boundaries. -- **Reference Tracking**: Automatic handling of shared and circular references -- **Schema Evolution**: Forward/backward compatibility for class schema changes -- **Polymorphism**: Full support for inheritance hierarchies and interfaces -- **Deep Copy**: Efficient deep cloning of complex object graphs with reference preservation -- **Security**: Class registration and configurable deserialization policies +### Installation -## Installation +Add `fory-core` for binary object serialization. Keep all Fory modules in one +application on the same version. -### Maven +#### Maven ```xml + org.apache.fory fory-core @@ -65,15 +71,16 @@ not the native or xlang binary protocol and has its own object-mapping annotatio ``` -### Gradle +#### Gradle ```kotlin +// Binary object serialization implementation("org.apache.fory:fory-core:1.4.0") ``` -### JDK25+ +#### JDK 25 and Later -On JDK25+, open `java.lang.invoke` to Fory. Use `ALL-UNNAMED` when Fory is on +On JDK 25 and later, open `java.lang.invoke` to Fory. Use `ALL-UNNAMED` when Fory is on the classpath: ```bash @@ -86,11 +93,11 @@ Use the Fory core module name when Fory is on the module path: --add-opens=java.base/java.lang.invoke=org.apache.fory.core ``` -## Quick Start +### Quick Start Note that Fory creation is not cheap, the **Fory instances should be reused between serializations** instead of creating it every time. You should keep Fory as a static global variable, or instance variable of some singleton object or limited objects. -### Single-Thread Usage +#### Single-Thread Usage ```java import java.util.List; @@ -118,7 +125,7 @@ public class Example { } ``` -### Multi-Thread Usage +#### Multi-Thread Usage ```java import org.apache.fory.*; @@ -137,7 +144,7 @@ public class Example { } ``` -### Fory Instance Reuse Pattern +#### Fory Instance Reuse Pattern ```java import org.apache.fory.*; @@ -160,7 +167,7 @@ public class Example { } ``` -## Xlang Mode And Native Mode +### Xlang Mode And Native Mode Use xlang mode for cross-language payloads and schemas shared with non-Java implementations. It is the default Java wire mode, and Java examples that use it set `.withXlang(true)` explicitly so the mode choice is visible. @@ -168,11 +175,11 @@ Use native mode for Java-only traffic. Native mode is selected with `.withXlang( See [Native Serialization](native-serialization.md) for Java-only serialization details and [Xlang Serialization](xlang-serialization.md) for Java xlang registration and interoperability rules. -## Thread Safety +### Thread Safety Fory provides two thread-safe Fory instance styles: -### `buildThreadSafeFory` +#### `buildThreadSafeFory` This is the default choice. It uses a fixed-size shared `ThreadPoolFory` sized to `4 * availableProcessors()` and is the preferred instance form for virtual-thread workloads: @@ -187,7 +194,7 @@ ThreadSafeFory fory = Fory.builder() See more details in [Virtual Threads](virtual-threads.md). -### ThreadLocalFory +#### ThreadLocalFory Use `buildThreadLocalFory()` only when you explicitly want one `Fory` instance per long-lived platform thread, or when you want to pin that choice regardless of JDK version: @@ -201,7 +208,7 @@ byte[] bytes = fory.serialize(object); System.out.println(fory.deserialize(bytes)); ``` -### `buildThreadSafeForyPool` +#### `buildThreadSafeForyPool` Use `buildThreadSafeForyPool(poolSize)` when you want to set that fixed shared pool size explicitly. It eagerly creates `poolSize` `Fory` instances, keeps them in shared fixed slots, and @@ -216,7 +223,7 @@ ThreadSafeFory fory = Fory.builder() .buildThreadSafeForyPool(poolSize); ``` -### Builder Methods +#### Builder Methods ```java // Single-thread Fory @@ -239,6 +246,126 @@ ThreadSafeFory threadLocalFory = Fory.builder() .buildThreadLocalFory(); ``` +## Row Format + +Fory row format is a separate cache-friendly binary format for random access, +partial reads, and analytics workloads. + +### Features + +- **Zero-Copy Random Access**: Read fields and nested values without rebuilding + complete objects. +- **Partial Reads**: Decode only the data required by an analytics or query path. +- **Apache Arrow Integration**: Convert between Fory row data and Arrow data for + columnar processing. + +### Installation + +#### Maven + +```xml + + org.apache.fory + fory-format + 1.4.0 + +``` + +#### Gradle + +```kotlin +implementation("org.apache.fory:fory-format:1.4.0") +``` + +See [Row Format](row-format.md) for encoding, typed field access, partial +deserialization, nested values, and Arrow integration. + +## Fory JSON + +Fory JSON is a thread-safe JSON serialization framework for Java, extensively +optimized for maximum performance across JSON encoding, decoding, and Java +object mapping. + +### Features + +- **Maximum Performance**: Optimized readers and writers plus interpreted and + runtime-generated codecs keep JSON encoding and decoding fast. +- **Java Object Mapping**: Supports ordinary objects, Java 17 records, immutable + creator-based classes, common JDK types, generic containers, custom codecs, + and annotation-declared polymorphism. + +### Installation + +`fory-json` includes `fory-core` transitively. Keep both modules on the same +version when another dependency also brings `fory-core` into the application. + +#### Maven + +```xml + + org.apache.fory + fory-json + 1.4.0 + +``` + +#### Gradle + +```kotlin +implementation("org.apache.fory:fory-json:1.4.0") +``` + +On JDK 25 and later, use the same `java.lang.invoke` module open described in +the binary serialization installation section. + +### Quick Start + +`ForyJson` is immutable and thread-safe after construction. Reuse one instance +across threads: + +```java +import org.apache.fory.json.ForyJson; + +public final class JsonExample { + private static final ForyJson JSON = ForyJson.builder().build(); + + public static final class User { + public long id; + public String name; + + public User() {} + + public User(long id, String name) { + this.id = id; + this.name = name; + } + } + + public static void main(String[] args) { + User input = new User(7, "Alice"); + + String text = JSON.toJson(input); + User fromText = JSON.fromJson(text, User.class); + + byte[] utf8 = JSON.toJsonBytes(input); + User fromUtf8 = JSON.fromJson(utf8, User.class); + + System.out.println(fromText.name + " / " + fromUtf8.name); + } +} +``` + +See [JSON Support](json-support.md) for supported types, annotations, custom +codecs, security controls, and platform setup. + +## Platform Support + +- `fory-core` and `fory-json` support Java 8 and later; Java records require + Java 17 or later. +- `fory-format` targets Java 11 and later and is not supported on Android. +- `fory-core` and `fory-json` run on standard JDKs, GraalVM native images, and + Android API level 26 and later. + ## Next Steps - [Configuration](configuration.md) - Learn about ForyBuilder options diff --git a/java/README.md b/java/README.md index e7c41de149..67e97fa675 100644 --- a/java/README.md +++ b/java/README.md @@ -4,89 +4,133 @@ [![Java Version](https://img.shields.io/badge/Java-8%2B-blue?style=for-the-badge)](https://www.oracle.com/java/) [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg?style=for-the-badge)](https://opensource.org/licenses/Apache-2.0) -Apache Fory™ Java provides blazingly-fast serialization for the Java ecosystem, delivering up to **170x performance improvement** over traditional frameworks through JIT compilation and zero-copy techniques. +Apache Fory™ Java provides high-performance binary object serialization, a +cross-language random-access row format, and JSON serialization for the Java +ecosystem. + +## Choose a Format + +| Format | Use it when | Module | Guide | +| ------------------------------- | -------------------------------------------------------------------------------- | ------------- | ----------------------------------------------------- | +| **Binary Object Serialization** | You need compact object graphs in Java native mode or across supported languages | `fory-core` | [Java guide](../docs/guide/java/) | +| **Row Format** | You need zero-copy random access, partial reads, or Arrow integration | `fory-format` | [Row-format guide](../docs/guide/java/row-format.md) | +| **Fory JSON** | You need high-throughput standard JSON for Java applications | `fory-json` | [Fory JSON guide](../docs/guide/java/json-support.md) | + +Keep all Fory modules in one application on the same version. ## Features -### High Performance +### Binary Object Serialization + +- **Generated Codecs**: JIT-generated serializers inline data access and reduce + virtual dispatch, branching, and metadata lookups on hot paths. +- **Native and Cross-Language Modes**: Use Java-native object semantics for + Java-only traffic or a portable wire format across supported languages. +- **Object Graph Semantics**: Preserve shared and circular references, + polymorphism, and schema evolution. +- **Compact Encoding**: Variable-length integer encoding, metadata sharing, + string compression, and optional numeric-array compression reduce payload size. +- **Java Object Model**: Native mode supports ordinary Java classes, records, + JDK custom serialization semantics, `Externalizable`, and deep copy. +- **Security Controls**: Class registration, type checking, depth limits, and + configurable deserialization policies protect decoding boundaries. + +### Row Format -- **JIT Code Generation**: Highly-extensible JIT framework generates serializer code at runtime using async multi-threaded compilation, delivering 20-170x speedup through: - - Inlining variables to reduce memory access - - Inlining method calls to eliminate virtual dispatch overhead - - Minimizing conditional branching - - Eliminating hash lookups -- **Zero-Copy**: Direct memory access without intermediate buffer copies; row format supports random access and partial serialization -- **Variable-Length Encoding**: Optimized compression for integers, longs -- **Meta Sharing**: Cached class metadata reduces redundant type information -- **SIMD Acceleration**: Java Vector API support for array operations (Java 16+) +- **Zero-Copy Random Access**: Read fields and nested values without rebuilding + complete objects. +- **Partial Reads**: Decode only the data required by an analytics or query path. +- **Apache Arrow Integration**: Convert between Fory row data and Arrow data for + columnar processing. -### Drop-in Replacement +### Fory JSON -- **100% JDK Serialization Compatible**: Supports `writeObject`/`readObject`/`writeReplace`/`readResolve`/`readObjectNoData`/`Externalizable` -- **Java 8+ Support**: Works across all modern Java versions including Java 17+ records -- **GraalVM Native Image**: AOT compilation support without reflection configuration +- **Maximum Performance**: Optimized readers and writers plus interpreted + and runtime-generated codecs keep JSON encoding and decoding fast. +- **Java Object Mapping**: Supports ordinary objects, Java 17 records, immutable + creator-based classes, common JDK types, generic containers, custom codecs, + and annotation-declared polymorphism. +- **Thread-Safe Runtime**: Build one immutable `ForyJson` instance and reuse it + across threads. -### Advanced Features +### Platforms -- **Reference Tracking**: Automatic handling of shared and circular references -- **Schema Evolution**: Forward/backward compatibility for class schema changes -- **Polymorphism**: Full support for inheritance hierarchies and interfaces -- **Deep Copy**: Efficient deep cloning of complex object graphs with reference preservation -- **Security**: Class registration and configurable deserialization policies +- `fory-core` and `fory-json` support Java 8 and later; Java records require + Java 17 or later. +- `fory-format` targets Java 11 and later. +- `fory-core` and `fory-json` run on standard JDKs, GraalVM native images, and + Android. Optional SIMD array acceleration in `fory-core` uses the Java Vector + API on Java 16 and later. ## Documentation -| Topic | Description | Source Doc Link | Website Doc Link | -| --------------------------- | -------------------------------- | ------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------- | -| **Java Guide** | Java xlang and native mode usage | [docs/guide/java](../docs/guide/java) | [Java Guide](https://fory.apache.org/docs/guide/java/) | -| **GraalVM Native Image** | Native image support | [graalvm-support.md](../docs/guide/java/graalvm-support.md) | [GraalVM Support](https://fory.apache.org/docs/guide/java/graalvm_support) | -| **Java Serialization Spec** | Protocol specification | [java_serialization_spec.md](../docs/specification/java_serialization_spec.md) | [Java Serialization Spec](https://fory.apache.org/docs/specification/java_serialization_spec) | -| **Java Benchmarks** | Performance data and plots | [java/README.md](../docs/benchmarks/java/README.md) | [Java Benchmarks](https://fory.apache.org/docs/benchmarks/java) | +| Topic | Description | Source Doc Link | Website Doc Link | +| --------------------------- | --------------------------------------------- | ------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------- | +| **Java Guide** | Binary xlang and native mode usage | [docs/guide/java](../docs/guide/java) | [Java Guide](https://fory.apache.org/docs/guide/java/) | +| **Row Format** | Random access and Arrow integration | [row-format.md](../docs/guide/java/row-format.md) | [Row Format](https://fory.apache.org/docs/guide/java/row_format) | +| **Fory JSON** | JSON usage, object mapping, and configuration | [json-support.md](../docs/guide/java/json-support.md) | [Fory JSON](https://fory.apache.org/docs/guide/java/json_support) | +| **GraalVM Native Image** | Native image support | [graalvm-support.md](../docs/guide/java/graalvm-support.md) | [GraalVM Support](https://fory.apache.org/docs/guide/java/graalvm_support) | +| **Java Serialization Spec** | Binary protocol specification | [java_serialization_spec.md](../docs/specification/java_serialization_spec.md) | [Java Serialization Spec](https://fory.apache.org/docs/specification/java_serialization_spec) | +| **Java Benchmarks** | Performance data and plots | [java/README.md](../docs/benchmarks/java/README.md) | [Java Benchmarks](https://fory.apache.org/docs/benchmarks/java) | ## Modules -| Module | Description | Maven Artifact | -| ------------------------------------ | ------------------------------------- | --------------------------------- | -| **fory-core** | Core serialization engine | `org.apache.fory:fory-core` | -| **fory-format** | Row format and Apache Arrow support | `org.apache.fory:fory-format` | -| [**fory-json**](fory-json/README.md) | JSON serialization and parsing | `org.apache.fory:fory-json` | -| **fory-extensions** | Protobuf support and meta compression | `org.apache.fory:fory-extensions` | -| **fory-test-core** | Testing utilities and data generators | `org.apache.fory:fory-test-core` | +| Module | Description | Maven Artifact | +| ------------------------------------ | --------------------------------------------- | --------------------------------- | +| **fory-core** | Binary native and xlang serialization | `org.apache.fory:fory-core` | +| **fory-format** | Row format and Apache Arrow support | `org.apache.fory:fory-format` | +| [**fory-json**](fory-json/README.md) | High-performance JSON serialization framework | `org.apache.fory:fory-json` | +| **fory-extensions** | Protobuf support and metadata compression | `org.apache.fory:fory-extensions` | +| **fory-test-core** | Testing utilities and data generators | `org.apache.fory:fory-test-core` | ## Installation +Add only the artifacts required by your chosen formats and keep their versions +aligned. `fory-json` includes `fory-core` transitively. + ### Maven ```xml + org.apache.fory fory-core 1.4.0 - + org.apache.fory fory-format 1.4.0 + + + org.apache.fory + fory-json + 1.4.0 + + org.apache.fory fory-extensions 1.4.0 - ``` ### Gradle ```gradle dependencies { + // Binary object serialization implementation 'org.apache.fory:fory-core:1.4.0' - // Optional modules + // Row format implementation 'org.apache.fory:fory-format:1.4.0' + // JSON serialization + implementation 'org.apache.fory:fory-json:1.4.0' + // Optional: Protobuf serializers and metadata compression implementation 'org.apache.fory:fory-extensions:1.4.0' } ``` @@ -106,15 +150,14 @@ Use the Fory core module name when Fory is on the module path: --add-opens=java.base/java.lang.invoke=org.apache.fory.core ``` -## Quick Start +## Binary Object Serialization -### Basic Usage +### Quick Start Create a Fory instance, register your classes, and start serializing objects. Remember to reuse the Fory instance for optimal performance: ```java -import org.apache.fory.*; -import org.apache.fory.config.*; +import org.apache.fory.Fory; // Create Fory instance (should be reused). Java defaults to xlang mode with // compatible schema evolution. @@ -123,15 +166,15 @@ Fory fory = Fory.builder() .requireClassRegistration(true) .build(); -// Register your classes -fory.register(MyClass.class); +// Register the same type identity on every xlang peer +fory.register(MyClass.class, "example.MyClass"); // Serialize MyClass object = new MyClass(); byte[] bytes = fory.serialize(object); // Deserialize -MyClass result = (MyClass) fory.deserialize(bytes); +MyClass result = fory.deserialize(bytes, MyClass.class); ``` ### Thread-Safe Usage @@ -139,14 +182,17 @@ MyClass result = (MyClass) fory.deserialize(bytes); For multi-threaded environments, use `ThreadSafeFory` which maintains a pool of Fory instances: ```java -import org.apache.fory.*; -import org.apache.fory.config.*; +import org.apache.fory.Fory; +import org.apache.fory.ThreadSafeFory; // Create thread-safe xlang Fory instance -private static final ThreadSafeFory fory = Fory.builder().withXlang(true).buildThreadSafeFory(); +private static final ThreadSafeFory fory = Fory.builder() + .withXlang(true) + .requireClassRegistration(true) + .buildThreadSafeFory(); static { - fory.register(MyClass.class); + fory.register(MyClass.class, "example.MyClass"); } // Use in multiple threads @@ -215,9 +261,7 @@ fory.register(MyClass.class, 1); byte[] bytes = fory.serialize(object); ``` -## Configuration Options - -### ForyBuilder Options +### Configuration Configure Fory with various options to suit your specific use case: @@ -245,14 +289,17 @@ Fory fory = Fory.builder() See the [Java guide](../docs/guide/java/) for detailed configuration options. -## Advanced Features - -### JDK Serialization Compatibility +### JDK Custom Serialization Semantics In native mode, Fory supports JDK serialization APIs with much better performance. Use native mode when replacing Java-only JDK serialization, Kryo, FST, Hessian, or Protocol Buffers payloads: ```java +import java.io.IOException; +import java.io.ObjectInputStream; +import java.io.ObjectOutputStream; +import java.io.Serializable; + public class MyClass implements Serializable { private void writeObject(ObjectOutputStream out) throws IOException { // Custom serialization logic @@ -286,58 +333,6 @@ MyClass original = new MyClass(); MyClass copy = fory.copy(original); ``` -### Row Format - -Fory provides a cache-friendly binary row format optimized for random access and analytics: - -- **Zero-Copy Random Access**: Read individual fields without deserializing entire objects -- **Partial Serialization**: Skip unnecessary fields during serialization -- **Cross-Language Compatible**: Row format data can be read by Python, C++ -- **Apache Arrow Integration**: Convert row format to/from Arrow RecordBatch for analytics - -```java -import org.apache.fory.format.encoder.*; -import org.apache.fory.format.row.*; - -public class Bar { - String f1; - List f2; -} - -public class Foo { - int f1; - List f2; - Map f3; - List f4; -} - -// Create row encoder -RowEncoder encoder = Encoders.bean(Foo.class); - -// Serialize to row format -Foo foo = new Foo(); -foo.f1 = 10; -foo.f2 = IntStream.range(0, 1000).boxed().collect(Collectors.toList()); -BinaryRow binaryRow = encoder.toRow(foo); - -// Zero-copy random access to fields -BinaryArray f2Array = binaryRow.getArray(1); // Access f2 without deserializing entire object -BinaryArray f4Array = binaryRow.getArray(3); // Access f4 - -// Zero-copy access nested fields -BinaryRow barStruct = f4Array.getStruct(10); // Get 11th element -long value = barStruct.getArray(1).getInt64(5); // Access nested field - -// Partial deserialization -RowEncoder barEncoder = Encoders.bean(Bar.class); -Bar partialBar = barEncoder.fromRow(barStruct); // Deserialize only one Bar object - -// Full deserialization -Foo deserializedFoo = encoder.fromRow(binaryRow); -``` - -See the [Java row-format guide](../docs/guide/java/row-format.md) for more details. - ### Array Compression Use width compression for integer and long arrays to reduce serialized size when array elements have small values. JDK 8 through 15 use scalar range analysis; JDK 16 and later automatically select the Vector API implementation from the multi-release JAR. @@ -364,7 +359,117 @@ On JDK 16 or later, resolve the incubator Vector API module when starting the ap java --add-modules=jdk.incubator.vector ... ``` -### GraalVM Native Image +### Performance Guidelines + +1. Reuse `Fory` or `ThreadSafeFory` instances instead of rebuilding runtime + state for each operation. +2. Register application classes to avoid repeated type metadata and keep type + identity explicit. +3. Use native mode for Java-only payloads; use xlang mode only when payloads + must cross language boundaries. +4. Keep compatible mode enabled when schemas may differ. Disable it only when + every reader and writer always uses the same schema. +5. Disable reference tracking only when shared identity and circular references + are not part of the data model. +6. Enable string, integer, long, or numeric-array compression only after + measuring the payload distribution and throughput tradeoff. +7. Warm up generated serializers before measuring steady-state performance. + +## Row Format + +Fory row format is a cache-friendly binary format for random access and +analytics. It can read fields, arrays, and nested values without rebuilding the +complete object. + +```java +import org.apache.fory.format.encoder.Encoders; +import org.apache.fory.format.encoder.RowEncoder; +import org.apache.fory.format.row.ArrayData; +import org.apache.fory.format.row.binary.BinaryRow; +import org.apache.fory.format.type.Schema; + +public final class RowExample { + public static final class User { + public int id; + public String name; + public int[] scores; + } + + public static void main(String[] args) { + RowEncoder encoder = Encoders.bean(User.class); + + User user = new User(); + user.id = 1; + user.name = "Alice"; + user.scores = new int[] {98, 100, 95}; + + BinaryRow row = encoder.toRow(user); + + Schema schema = encoder.schema(); + Schema.StringField nameField = schema.stringField("name"); + Schema.ArrayField scoresField = schema.arrayField("scores"); + + String name = nameField.get(row); + ArrayData scores = scoresField.get(row); + int secondScore = scores.getInt32(1); + + System.out.println(name + ": " + secondScore); + } +} +``` + +See the [Java row-format guide](../docs/guide/java/row-format.md) for nested +structs, arrays, maps, partial deserialization, and Arrow integration. + +## Fory JSON + +Fory JSON is a thread-safe JSON serialization framework for Java, extensively +optimized for maximum performance across JSON encoding, decoding, and Java +object mapping. + +Build one `ForyJson` instance and reuse it across threads: + +```java +import java.nio.charset.StandardCharsets; +import org.apache.fory.json.ForyJson; + +public final class JsonExample { + private static final ForyJson JSON = ForyJson.builder().build(); + + public static final class User { + public long id; + public String name; + + public User() {} + + public User(long id, String name) { + this.id = id; + this.name = name; + } + } + + public static void main(String[] args) { + User input = new User(7, "Alice"); + + String text = JSON.toJson(input); + User fromText = JSON.fromJson(text, User.class); + + byte[] utf8 = JSON.toJsonBytes(input); + User fromUtf8 = JSON.fromJson(utf8, User.class); + + System.out.println(text); + System.out.println(new String(utf8, StandardCharsets.UTF_8)); + System.out.println(fromText.name + " / " + fromUtf8.name); + } +} +``` + +Fory JSON supports Java 8 and later on standard JDKs, GraalVM native images, +and Android. Java records are supported on Java 17 and later. See the +[Fory JSON guide](../docs/guide/java/json-support.md) for supported types, +annotations, custom codecs, security controls, and platform setup. + +## GraalVM Native Image Fory supports GraalVM Native Image without application reflection configuration. Binary serialization generates serializers while the image is built; the Fory annotation processor @@ -427,16 +532,6 @@ mvn -T16 spotless:apply mvn -T16 checkstyle:check ``` -## Performance Tips - -1. **Reuse Fory Instances**: Creating Fory is expensive; reuse instances across serializations -2. **Enable Compression**: For numeric-heavy data, enable int/long compression -3. **Disable Reference Tracking**: If no circular references exist, disable tracking for better performance -4. **Use native mode**: For Java-only payloads, use `withXlang(false)`. Native mode reduces type metadata overhead and supports more Java-native types not available in xlang mode -5. **Warm Up**: Allow JIT compilation to complete before benchmarking -6. **Register Classes**: Class registration reduces metadata overhead -7. **Compress numeric arrays**: Enable array compression when numeric array values usually fit in narrower primitive types - ## Contributing See [CONTRIBUTING.md](../CONTRIBUTING.md) for development guidelines.