From ab654bb02c114a62cb54326ff27e82e1273f9e20 Mon Sep 17 00:00:00 2001 From: hg-ms <53219833+hg-ms@users.noreply.github.com> Date: Tue, 22 Sep 2026 07:19:40 +0200 Subject: [PATCH 1/3] Support for JEP 401 value classes A value class has no identity, and its instances cannot be created empty and populated afterwards - which is exactly what the reflective path does: create blank, write each field, register the instance under an object id. Every step of that fails for a value, so value classes were unpersistable. This adds a path of their own, from the type handler down to the binary layout, and re-composes the JDK types that the language change turns into value classes. Handlers. BinaryHandlerGenericValueClass and BinaryLegacyTypeHandlerValueClass construct the instance from its persisted state instead of populating one, and BinaryTypeHandlerCreator routes value classes away from the reflective branch. A field whose type is a value class may be laid out inside its owner, so it is reached through a handle rather than by its memory offset - on the current, legacy and enum paths alike. Constructor contract. Constructing from stored state is the inverse of storing only if the constructor assigns each argument to the matching field unmodified. ValueClassConstructorContract verifies that by observing what the constructor does: at handler creation, or against the first instance stored where that cannot answer. Two same-typed fields taken in the wrong order, or a field computed from its argument, are refused instead of drifting silently on every round trip. Value inlining. A field whose type has no identity costs an entity, and an object id, per owner on every store. PersistenceValueInliningResolver decides per field whether the value is written into the owner's own binary form instead: a null marker followed by the field type's layout, described in the type dictionary by the new member kind PersistenceTypeDescriptionMemberFieldValueStruct. The default inlines what the application itself declares; setValueInliningResolver and Disabled() select otherwise. An inlined slot carries no type information of its own, so an abstract declared type, a reference inside the layout and a constructor that cannot be invoked are refused rather than inlined. Evolution. An inlined layout evolves like any other: members added, removed, renamed or translated, nested value classes included, with the shapes that cannot be told apart refused rather than guessed. Identity consequences. PersistenceStoreHandler#applyKnown routes a reference by an object id an earlier store established, so storing a reference list no longer re-assigns ids to unchanged value referents. A lazy reference keeps its link to a value referent. A root refuses a value instance, loading a root meaning that the persisted state is applied to the very instance that was set. JDK types. The java.time types and Optional are now composed from their persisted state through AbstractBinaryHandlerCustomComposedValueType, and the wrapper constants registered at startup are gated on whether the JDK has migrated them. --- .../eclipse/serializer/reflect/XReflect.java | 130 ++ .../eclipse/serializer/typing/ValueType.java | 13 + .../org/eclipse/serializer/typing/XTypes.java | 29 + .../AbstractBinaryHandlerSecondsNanos.java | 174 ++ .../java/time/BinaryHandlerDuration.java | 58 + .../java/time/BinaryHandlerInstant.java | 58 + .../java/time/BinaryHandlerLocalDate.java | 30 +- .../java/time/BinaryHandlerLocalDateTime.java | 69 + .../java/time/BinaryHandlerLocalTime.java | 168 ++ .../java/time/BinaryHandlerMonthDay.java | 26 +- .../time/BinaryHandlerOffsetDateTime.java | 70 + .../java/time/BinaryHandlerOffsetTime.java | 69 + .../binary/java/time/BinaryHandlerPeriod.java | 26 +- .../binary/java/time/BinaryHandlerYear.java | 132 ++ .../java/time/BinaryHandlerYearMonth.java | 26 +- .../java/time/BinaryHandlerZoneRegion.java | 101 + .../java/time/BinaryHandlerZonedDateTime.java | 76 + .../java/util/BinaryHandlerOptional.java | 73 + .../entity/BinaryHandlerEntityLoading.java | 18 + .../BinaryHandlerPersistenceRootsDefault.java | 9 +- .../BinaryHandlerRootReferenceDefault.java | 15 +- .../reference/BinaryHandlerLazyDefault.java | 54 +- ...tBinaryHandlerCustomComposedValueType.java | 325 +++ .../AbstractBinaryHandlerReflective.java | 180 +- ...actBinaryLegacyTypeHandlerTranslating.java | 12 + .../persistence/binary/types/Binary.java | 164 +- .../types/BinaryHandlerGenericEnum.java | 20 +- .../types/BinaryHandlerGenericType.java | 6 +- .../types/BinaryHandlerGenericValueClass.java | 507 +++++ .../types/BinaryLegacyTypeHandlerCreator.java | 46 +- .../BinaryLegacyTypeHandlerRerouting.java | 79 +- .../BinaryLegacyTypeHandlerValueClass.java | 450 ++++ .../binary/types/BinaryLoadItem.java | 6 + .../binary/types/BinaryLoader.java | 84 +- .../binary/types/BinaryPersistence.java | 35 +- .../types/BinaryPersistenceFoundation.java | 2 + .../types/BinaryReferenceTraverser.java | 9 +- .../binary/types/BinaryStorer.java | 35 + .../types/BinaryTypeHandlerCreator.java | 48 +- .../types/BinaryValueHandleFunctions.java | 535 +++++ .../binary/types/BinaryValueReader.java | 97 + .../types/BinaryValueStructFunctions.java | 1872 +++++++++++++++++ .../types/BinaryValueTranslatorProvider.java | 298 ++- .../types/ValueClassConstructorContract.java | 493 +++++ .../persistence/types/Persistence.java | 152 ++ .../types/PersistenceFoundation.java | 39 +- .../PersistenceLegacyTypeHandlerCreator.java | 11 +- .../PersistenceLegacyTypeHandlerWrapper.java | 21 + .../persistence/types/PersistenceManager.java | 28 +- .../types/PersistenceObjectManager.java | 79 +- .../types/PersistenceRootReference.java | 35 + .../types/PersistenceStoreHandler.java | 44 + ...ersistenceTypeDefinitionMemberCreator.java | 58 + ...eTypeDefinitionMemberFieldValueStruct.java | 182 ++ .../PersistenceTypeDescriptionMember.java | 11 +- ...sistenceTypeDescriptionMemberAppender.java | 23 + ...TypeDescriptionMemberFieldValueStruct.java | 296 +++ .../types/PersistenceTypeDictionary.java | 2 + .../PersistenceTypeDictionaryParser.java | 208 +- .../types/PersistenceTypeHandler.java | 81 +- .../PersistenceValueInliningResolver.java | 297 +++ .../org/eclipse/serializer/Serializer.java | 56 +- 62 files changed, 8219 insertions(+), 131 deletions(-) create mode 100644 persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/AbstractBinaryHandlerSecondsNanos.java create mode 100644 persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerDuration.java create mode 100644 persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerInstant.java create mode 100644 persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerLocalDateTime.java create mode 100644 persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerLocalTime.java create mode 100644 persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerOffsetDateTime.java create mode 100644 persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerOffsetTime.java create mode 100644 persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerYear.java create mode 100644 persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerZoneRegion.java create mode 100644 persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerZonedDateTime.java create mode 100644 persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/util/BinaryHandlerOptional.java create mode 100644 persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/AbstractBinaryHandlerCustomComposedValueType.java create mode 100644 persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryHandlerGenericValueClass.java create mode 100644 persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryLegacyTypeHandlerValueClass.java create mode 100644 persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryValueHandleFunctions.java create mode 100644 persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryValueReader.java create mode 100644 persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryValueStructFunctions.java create mode 100644 persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/ValueClassConstructorContract.java create mode 100644 persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceTypeDefinitionMemberFieldValueStruct.java create mode 100644 persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceTypeDescriptionMemberFieldValueStruct.java create mode 100644 persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceValueInliningResolver.java diff --git a/base/src/main/java/org/eclipse/serializer/reflect/XReflect.java b/base/src/main/java/org/eclipse/serializer/reflect/XReflect.java index 3498f09c..efea1e4e 100644 --- a/base/src/main/java/org/eclipse/serializer/reflect/XReflect.java +++ b/base/src/main/java/org/eclipse/serializer/reflect/XReflect.java @@ -17,6 +17,9 @@ import static org.eclipse.serializer.util.X.notEmpty; import static org.eclipse.serializer.util.X.notNull; +import java.lang.invoke.MethodHandle; +import java.lang.invoke.MethodHandles; +import java.lang.invoke.MethodType; import java.lang.reflect.Constructor; import java.lang.reflect.Field; import java.lang.reflect.InvocationTargetException; @@ -56,6 +59,32 @@ */ public final class XReflect { + /////////////////////////////////////////////////////////////////////////// + // constants // + ////////////// + + // null on a JVM without value class support (JEP 401). See #isValueClass. + private static final MethodHandle CLASS_IS_VALUE = resolveClassIsValue(); + + /* Any one of the JDK's cached-constant wrapper types being a value class is enough, and + * deliberately so - see #isValueClassEnabledRuntime. Must be declared after CLASS_IS_VALUE, + * static initializers running in declaration order. + */ + private static final boolean VALUE_CLASSES_ENABLED = + isValueClass(Boolean.class) + || isValueClass(Byte.class) + || isValueClass(Short.class) + || isValueClass(Character.class) + || isValueClass(Integer.class) + || isValueClass(Long.class) + ; + + + + /////////////////////////////////////////////////////////////////////////// + // static methods // + /////////////////// + public static T defaultInstantiate(final Class type) throws NoSuchMethodRuntimeException, InstantiationRuntimeException { @@ -273,6 +302,107 @@ public static boolean isDeclaredEnum(final Class c) { return c != null && c.isEnum(); } + + /** + * Tests whether the passed {@link Class} is a value class (JEP 401), i.e. a class whose instances + * have no identity. + *

+ * Value instances must never be used where identity is required: they cannot be referenced by a + * {@link java.lang.ref.WeakReference} and cannot be synchronized on (both throw + * {@code java.lang.IdentityException}), while {@code ==} and + * {@link System#identityHashCode(Object)} compare their content instead of their identity. + *

+ * The check is done reflectively via {@code Class#isValue}, since the compile baseline does not + * have that method. On a JVM without value class support, this method returns {@literal false}. + * + * @param c the {@link Class} to be tested, may be {@literal null}. + * + * @return whether the passed {@link Class} is a value class. + */ + public static boolean isValueClass(final Class c) + { + /* A primitive and an interface both report themselves as value types, which they are in the type + * system's terms but not in the sense the callers ask about: neither declares fields or a + * constructor, and an interface-typed field holds a reference, never a value laid out in its owner. + */ + if(c == null || c.isPrimitive() || c.isInterface() || CLASS_IS_VALUE == null) + { + return false; + } + + try + { + return (boolean)CLASS_IS_VALUE.invoke(c); + } + catch(final Throwable t) + { + // the handle is resolved from and invoked on java.lang.Class, so a failure is unrecoverable. + throw new Error("Failed to determine value class state of " + c.getName(), t); + } + } + + /** + * Tests whether the passed instance is a value instance. + * + * @param instance the instance to be tested, may be {@literal null}. + * + * @return whether the passed instance is an instance of a value class. + * + * @see #isValueClass(Class) + */ + public static boolean isValueInstance(final Object instance) + { + return instance != null && isValueClass(instance.getClass()); + } + + /** + * Whether value classes are enabled in this runtime. Useful to skip logic that can only ever + * apply where an identity-less instance can exist at all. + *

+ * Deliberately not the mere presence of {@code Class#isValue}: that method exists on every JDK + * since JEP 401 became a preview feature and reports {@literal false} for everything unless the + * JVM was started with {@code --enable-preview}. The wrapper types answer the actual state, since + * a user-defined value class is a preview class file that cannot be loaded without that flag. + *

+ * Any of them suffices, not all, and the asymmetry is the point. The reader with something + * at stake is the constant registry: it holds its entries weakly, so registering one wrapper + * constant that is a value instance throws {@code IdentityException} and the storage does not + * open. Answering {@literal true} while they are all still identity classes costs nothing - their + * constant ids are then resolved arithmetically instead of from the registry, and over the cached + * range that yields the very same instances. So the expensive mistake is answering + * {@literal false} too readily, and one migrated wrapper is enough to rule it out. + *

+ * For that reader this is exact. For the other kind - "can an identity-less instance exist at + * all", which is what a {@code GigaMap} segment asks before keeping a stored-state record - it + * stays a proxy: a JDK enabling value classes without migrating the wrappers would answer + * {@literal false} while a user's value class worked. That costs such an entity a fresh object id + * per store, nothing more. + * + * @return whether an instance without identity can exist in this JVM. + * + * @see #isValueClass(Class) + */ + public static boolean isValueClassEnabledRuntime() + { + return VALUE_CLASSES_ENABLED; + } + + private static MethodHandle resolveClassIsValue() + { + try + { + return MethodHandles.publicLookup().findVirtual( + Class.class, + "isValue", + MethodType.methodType(boolean.class) + ); + } + catch(final ReflectiveOperationException e) + { + // no value class support in this JVM. + return null; + } + } public static boolean isSubEnum(final Class c) { diff --git a/base/src/main/java/org/eclipse/serializer/typing/ValueType.java b/base/src/main/java/org/eclipse/serializer/typing/ValueType.java index f7ec0369..aacf3a7f 100644 --- a/base/src/main/java/org/eclipse/serializer/typing/ValueType.java +++ b/base/src/main/java/org/eclipse/serializer/typing/ValueType.java @@ -26,9 +26,22 @@ * Value types are the only types where inherently implemented equals() and hashCode() are properly applicable. * As Java is sadly missing a SELF typing, the untyped equals(Object obj) can't be defined more specific * (like for example public boolean equals(SELF obj) or such). + *

+ * Since JEP 401 the language says this directly: a {@code value class} has no identity, so {@code ==} is + * substitutability and the inherited {@code equals}/{@code hashCode} are state-based by construction. Such a + * type needs no marker, and marking it changes nothing. + *

+ * What this interface is for is everything that cannot be one: a type that must be handled as a value but + * has to keep its identity - because it is not final, because something synchronizes on it or holds it + * weakly, or simply because it is compiled for a JDK without value classes. Implementing it is a statement + * about intended handling, not about the JVM's notion of a value. + *

+ * Note that {@link XTypes#isValueType(Class)}, which is what reads this marker, does not recognise a + * JEP 401 value class - the two notions are separate. See that method for what the answer is used for. * *

* Also see: + * @see XTypes#isValueType(Class) * @see HashImmutable * @see Immutable * @see Stateless diff --git a/base/src/main/java/org/eclipse/serializer/typing/XTypes.java b/base/src/main/java/org/eclipse/serializer/typing/XTypes.java index cf38c56c..4434754c 100644 --- a/base/src/main/java/org/eclipse/serializer/typing/XTypes.java +++ b/base/src/main/java/org/eclipse/serializer/typing/XTypes.java @@ -180,6 +180,29 @@ public static boolean isLiteralType(final Class c) return c == String.class || c == char.class || c == Character.class; } + /** + * Whether instances of the passed type are to be handled as values: by their state, rather than by + * which instance they are. The set is closed apart from the {@link ValueType} marker, which is the only + * way to add to it from outside this module. + *

+ * This is a handling policy, not the JVM's notion of a value: a JEP 401 {@code value class} is + * deliberately not recognised here (see {@link org.eclipse.serializer.reflect.XReflect#isValueClass} + * for that question). For a value class whose fields are all primitives the distinction is invisible, + * because {@code ==} is then substitutability and {@code System.identityHashCode} is derived from state - + * so identity handling already behaves as value handling. It becomes visible for a value class holding a + * field of an identity type, where identity handling compares that field by reference while value + * handling would compare it by {@code equals}. + *

+ * The reader with persistence-visible consequences is {@code BinaryHandlerGenericEnum}: a {@code final} + * enum field of a value type is treated as a value constant and is therefore not written back + * when the enum is loaded. + * + * @param c the type to be checked. + * + * @return whether instances of that type are to be handled as values. + * + * @see ValueType + */ public static boolean isValueType(final Class c) { // all value types, ordered in common use probability @@ -228,9 +251,15 @@ public static boolean isBoolean(final Object o) /** * Checks if the type of the passed instance is an immutable special value type of the java language. * This includes all primitive wrappers and {@link String}. + *

+ * As with {@link #isValueType(Class)}, a JEP 401 value instance is not recognised here; see there for + * why that is invisible in most cases and where it is not. Note also that this variant does not cover + * {@link java.lang.reflect.Field}, which the {@link Class} variant does. * * @param o the instance to be checked * @return {@code true} if the type of the passed instance is an immutable special value type. + * + * @see #isValueType(Class) */ public static boolean isValueType(final Object o) { diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/AbstractBinaryHandlerSecondsNanos.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/AbstractBinaryHandlerSecondsNanos.java new file mode 100644 index 00000000..bd71a8cf --- /dev/null +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/AbstractBinaryHandlerSecondsNanos.java @@ -0,0 +1,174 @@ +package org.eclipse.serializer.persistence.binary.java.time; + +/*- + * #%L + * Eclipse Serializer Persistence Binary + * %% + * Copyright (C) 2023 - 2026 MicroStream Software + * %% + * This program and the accompanying materials are made + * available under the terms of the Eclipse Public License 2.0 + * which is available at https://www.eclipse.org/legal/epl-2.0/ + * + * SPDX-License-Identifier: EPL-2.0 + * #L% + */ + +import static org.eclipse.serializer.util.X.notNull; + +import java.util.function.ToIntFunction; +import java.util.function.ToLongFunction; + +import org.eclipse.serializer.memory.XMemory; +import org.eclipse.serializer.persistence.binary.types.AbstractBinaryHandlerCustomNonReferentialFixedLength; +import org.eclipse.serializer.persistence.binary.types.Binary; +import org.eclipse.serializer.persistence.types.PersistenceLoadHandler; +import org.eclipse.serializer.persistence.types.PersistenceStoreHandler; +import org.eclipse.serializer.reflect.XReflect; + +/** + * Base for handlers of a {@code java.time} type that holds its state in a {@code long seconds} and an + * {@code int nanos} field, i.e. {@link java.time.Duration} and {@link java.time.Instant}. Both are + * value classes on a suitably recent JVM. + *

+ * Where the handled type is a value class, its instances cannot be created empty and populated + * afterwards: the population write is not reliably visible on an identity-less instance. The instance + * is therefore built from its persisted values by {@link #createFromValues(long, int)}. + *

+ * The instance is created complete either way, since a plugin reusing the value-type handlers (e.g. + * the REST viewer) relies on {@link #create} alone. Where the type is an ordinary class, an already + * registered instance is still populated in {@link #updateState}, preserving the update behavior the + * reflective handling had. + *

+ * The persisted form is the two values under their field names qualified with the declaring type, so + * it is byte-identical to the one the reflective handling produced and existing data is unaffected. + * + * @param the handled type. + */ +public abstract class AbstractBinaryHandlerSecondsNanos +extends AbstractBinaryHandlerCustomNonReferentialFixedLength +{ + /////////////////////////////////////////////////////////////////////////// + // constants // + ////////////// + + private static final long + BINARY_OFFSET_SECONDS = 0 , + BINARY_OFFSET_NANOS = BINARY_OFFSET_SECONDS + Long.BYTES , + BINARY_LENGTH = BINARY_OFFSET_NANOS + Integer.BYTES + ; + + private static final String + FIELD_NAME_SECONDS = "seconds", + FIELD_NAME_NANOS = "nanos" + ; + + + + /////////////////////////////////////////////////////////////////////////// + // instance fields // + //////////////////// + + private final ToLongFunction seconds; + private final ToIntFunction nanos ; + + // only needed where the type is an ordinary class and its instances are populated after creation. + private final long memoryOffsetSeconds; + private final long memoryOffsetNanos ; + + + + /////////////////////////////////////////////////////////////////////////// + // constructors // + ///////////////// + + protected AbstractBinaryHandlerSecondsNanos( + final Class type , + final ToLongFunction seconds, + final ToIntFunction nanos + ) + { + super( + type, + CustomFields( + /* Qualified with the declaring type, so this description stays the one the reflective + * handling produced and data written before this handler existed still matches. + */ + CustomField(long.class, type.getName(), FIELD_NAME_SECONDS), + CustomField(int.class , type.getName(), FIELD_NAME_NANOS ) + ) + ); + + this.seconds = notNull(seconds); + this.nanos = notNull(nanos) ; + + final boolean populated = !this.isValueClassType(); + this.memoryOffsetSeconds = populated + ? XMemory.objectFieldOffset(XReflect.getAnyField(type, FIELD_NAME_SECONDS)) + : -1 + ; + this.memoryOffsetNanos = populated + ? XMemory.objectFieldOffset(XReflect.getAnyField(type, FIELD_NAME_NANOS)) + : -1 + ; + } + + + + /////////////////////////////////////////////////////////////////////////// + // declared methods // + ///////////////////// + + /** + * Builds an instance from its persisted values. The persisted nano value is always within + * {@code [0, 999_999_999]}, so a factory normalizing its arguments reproduces the state exactly. + * + * @param seconds the persisted second value. + * @param nanos the persisted nano-of-second value. + * + * @return the built instance. + */ + protected abstract T createFromValues(long seconds, int nanos); + + + + /////////////////////////////////////////////////////////////////////////// + // override methods // + ///////////////////// + + @Override + public void store( + final Binary data , + final T instance, + final long objectId, + final PersistenceStoreHandler handler + ) + { + data.storeEntityHeader(BINARY_LENGTH, this.typeId(), objectId); + data.store_long(BINARY_OFFSET_SECONDS, this.seconds.applyAsLong(instance)); + data.store_int (BINARY_OFFSET_NANOS , this.nanos.applyAsInt(instance) ); + } + + @Override + public T create(final Binary data, final PersistenceLoadHandler handler) + { + return this.createFromValues( + data.read_long(BINARY_OFFSET_SECONDS), + data.read_int (BINARY_OFFSET_NANOS) + ); + } + + @Override + public void updateState(final Binary data, final T instance, final PersistenceLoadHandler handler) + { + if(this.isValueClassType()) + { + // already complete: it was built from its content rather than populated + return; + } + + XMemory.set_long(instance, this.memoryOffsetSeconds, data.read_long(BINARY_OFFSET_SECONDS)); + XMemory.set_int (instance, this.memoryOffsetNanos , data.read_int (BINARY_OFFSET_NANOS) ); + } + +} diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerDuration.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerDuration.java new file mode 100644 index 00000000..b1bec829 --- /dev/null +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerDuration.java @@ -0,0 +1,58 @@ +package org.eclipse.serializer.persistence.binary.java.time; + +/*- + * #%L + * Eclipse Serializer Persistence Binary + * %% + * Copyright (C) 2023 - 2026 MicroStream Software + * %% + * This program and the accompanying materials are made + * available under the terms of the Eclipse Public License 2.0 + * which is available at https://www.eclipse.org/legal/epl-2.0/ + * + * SPDX-License-Identifier: EPL-2.0 + * #L% + */ + +import java.time.Duration; + +/** + * Handler for {@link Duration}, which holds its state in a second and a nano-of-second value. Where + * it is a value class, the instance is built through {@link Duration#ofSeconds(long, long)}; see + * {@link AbstractBinaryHandlerSecondsNanos} for the mechanism. + */ +public final class BinaryHandlerDuration extends AbstractBinaryHandlerSecondsNanos +{ + /////////////////////////////////////////////////////////////////////////// + // static methods // + /////////////////// + + public static BinaryHandlerDuration New() + { + return new BinaryHandlerDuration(); + } + + + + /////////////////////////////////////////////////////////////////////////// + // constructors // + ///////////////// + + BinaryHandlerDuration() + { + super(Duration.class, Duration::getSeconds, Duration::getNano); + } + + + + /////////////////////////////////////////////////////////////////////////// + // override methods // + ///////////////////// + + @Override + protected Duration createFromValues(final long seconds, final int nanos) + { + return Duration.ofSeconds(seconds, nanos); + } + +} diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerInstant.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerInstant.java new file mode 100644 index 00000000..4c0b6b90 --- /dev/null +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerInstant.java @@ -0,0 +1,58 @@ +package org.eclipse.serializer.persistence.binary.java.time; + +/*- + * #%L + * Eclipse Serializer Persistence Binary + * %% + * Copyright (C) 2023 - 2026 MicroStream Software + * %% + * This program and the accompanying materials are made + * available under the terms of the Eclipse Public License 2.0 + * which is available at https://www.eclipse.org/legal/epl-2.0/ + * + * SPDX-License-Identifier: EPL-2.0 + * #L% + */ + +import java.time.Instant; + +/** + * Handler for {@link Instant}, which holds its state in a second and a nano-of-second value. Where it + * is a value class, the instance is built through {@link Instant#ofEpochSecond(long, long)}; see + * {@link AbstractBinaryHandlerSecondsNanos} for the mechanism. + */ +public final class BinaryHandlerInstant extends AbstractBinaryHandlerSecondsNanos +{ + /////////////////////////////////////////////////////////////////////////// + // static methods // + /////////////////// + + public static BinaryHandlerInstant New() + { + return new BinaryHandlerInstant(); + } + + + + /////////////////////////////////////////////////////////////////////////// + // constructors // + ///////////////// + + BinaryHandlerInstant() + { + super(Instant.class, Instant::getEpochSecond, Instant::getNano); + } + + + + /////////////////////////////////////////////////////////////////////////// + // override methods // + ///////////////////// + + @Override + protected Instant createFromValues(final long seconds, final int nanos) + { + return Instant.ofEpochSecond(seconds, nanos); + } + +} diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerLocalDate.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerLocalDate.java index c80da4c5..17f0e2ea 100644 --- a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerLocalDate.java +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerLocalDate.java @@ -18,6 +18,7 @@ import org.eclipse.serializer.persistence.binary.types.AbstractBinaryHandlerCustomNonReferentialFixedLength; import org.eclipse.serializer.persistence.binary.types.Binary; +import org.eclipse.serializer.persistence.binary.types.ValidatingBinaryHandler; import org.eclipse.serializer.persistence.types.PersistenceLoadHandler; import org.eclipse.serializer.persistence.types.PersistenceStoreHandler; @@ -25,11 +26,12 @@ * Binary Handler for java.time.LocalDate * Required for java ≥ 26 because of transient fields and * changed binary format of YearMonth implementation. - * + * * Binary Format must be compatibe with java < 26 versions! - * + * */ public class BinaryHandlerLocalDate extends AbstractBinaryHandlerCustomNonReferentialFixedLength +implements ValidatingBinaryHandler { static final long BINARY_OFFSET_YEAR = 0L; static final long BINARY_OFFSET_MONTH = BINARY_OFFSET_YEAR + Integer.BYTES; @@ -66,10 +68,27 @@ protected BinaryHandlerLocalDate() // methods // //////////// /// + /** + * This handler transfers state in {@link #create} alone, so an instance it is handed here was + * created elsewhere and cannot be updated. Validating instead of ignoring keeps a divergence from + * dropping the persisted state silently, e.g. for an explicitly set root. + */ @Override public void updateState(final Binary data, final LocalDate instance, final PersistenceLoadHandler handler) { - // no-op + this.validateState(data, instance, handler); + } + + @Override + public LocalDate getValidationStateFromInstance(final LocalDate instance) + { + return instance; + } + + @Override + public LocalDate getValidationStateFromBinary(final Binary data) + { + return binaryState(data); } @Override @@ -84,6 +103,11 @@ public void store(final Binary data, final LocalDate instance, final long object @Override public LocalDate create(final Binary data, final PersistenceLoadHandler handler) + { + return binaryState(data); + } + + private static LocalDate binaryState(final Binary data) { return LocalDate.of( data.read_int (BINARY_OFFSET_YEAR), diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerLocalDateTime.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerLocalDateTime.java new file mode 100644 index 00000000..41a5a365 --- /dev/null +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerLocalDateTime.java @@ -0,0 +1,69 @@ +package org.eclipse.serializer.persistence.binary.java.time; + +/*- + * #%L + * Eclipse Serializer Persistence Binary + * %% + * Copyright (C) 2023 - 2026 MicroStream Software + * %% + * This program and the accompanying materials are made + * available under the terms of the Eclipse Public License 2.0 + * which is available at https://www.eclipse.org/legal/epl-2.0/ + * + * SPDX-License-Identifier: EPL-2.0 + * #L% + */ + +import java.time.LocalDate; +import java.time.LocalDateTime; +import java.time.LocalTime; + +import org.eclipse.serializer.persistence.binary.types.AbstractBinaryHandlerCustomComposedValueType; + +/** + * Handler for {@link LocalDateTime}, which holds its date and time parts in two references. Where it + * is a value class, the instance is built through {@link LocalDateTime#of(LocalDate, LocalTime)}; + * see {@link AbstractBinaryHandlerCustomComposedValueType} for the mechanism. + */ +public final class BinaryHandlerLocalDateTime extends AbstractBinaryHandlerCustomComposedValueType +{ + /////////////////////////////////////////////////////////////////////////// + // static methods // + /////////////////// + + public static BinaryHandlerLocalDateTime New() + { + return new BinaryHandlerLocalDateTime(); + } + + + + /////////////////////////////////////////////////////////////////////////// + // constructors // + ///////////////// + + BinaryHandlerLocalDateTime() + { + super( + LocalDateTime.class, + Part.New(LocalDate.class, "date", LocalDateTime::toLocalDate), + Part.New(LocalTime.class, "time", LocalDateTime::toLocalTime) + ); + } + + + + /////////////////////////////////////////////////////////////////////////// + // override methods // + ///////////////////// + + @Override + protected LocalDateTime createFromParts(final Object[] parts) + { + return LocalDateTime.of( + (LocalDate)parts[0], + (LocalTime)parts[1] + ); + } + +} diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerLocalTime.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerLocalTime.java new file mode 100644 index 00000000..fcf2f05b --- /dev/null +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerLocalTime.java @@ -0,0 +1,168 @@ +package org.eclipse.serializer.persistence.binary.java.time; + +/*- + * #%L + * Eclipse Serializer Persistence Binary + * %% + * Copyright (C) 2023 - 2026 MicroStream Software + * %% + * This program and the accompanying materials are made + * available under the terms of the Eclipse Public License 2.0 + * which is available at https://www.eclipse.org/legal/epl-2.0/ + * + * SPDX-License-Identifier: EPL-2.0 + * #L% + */ + +import java.time.LocalTime; + +import org.eclipse.serializer.memory.XMemory; +import org.eclipse.serializer.persistence.binary.types.AbstractBinaryHandlerCustomNonReferentialFixedLength; +import org.eclipse.serializer.persistence.binary.types.Binary; +import org.eclipse.serializer.persistence.types.PersistenceLoadHandler; +import org.eclipse.serializer.persistence.types.PersistenceStoreHandler; +import org.eclipse.serializer.reflect.XReflect; + +/** + * Handler for {@link LocalTime}. + *

+ * Where {@link LocalTime} is a value class, its instances cannot be created empty and populated + * afterwards, and its constructor takes {@code int} parameters where the fields are {@code byte}, + * so it cannot be constructed generically from its fields either. It is therefore built from its + * persisted values through {@link LocalTime#of(int, int, int, int)}. + *

+ * The instance is created complete either way, since a plugin reusing the value-type handlers (e.g. + * the REST viewer) relies on {@link #create} alone. Where the type is an ordinary class, an already + * registered instance is still populated in {@link #updateState}, preserving the update behavior the + * reflective handling had. + *

+ * The persisted form is byte-identical to the one the reflective handling produced, under the same + * type and member names, so existing data is unaffected. + */ +public final class BinaryHandlerLocalTime extends AbstractBinaryHandlerCustomNonReferentialFixedLength +{ + /////////////////////////////////////////////////////////////////////////// + // constants // + ////////////// + + private static final long + BINARY_OFFSET_HOUR = 0 , + BINARY_OFFSET_MINUTE = BINARY_OFFSET_HOUR + Byte.BYTES , + BINARY_OFFSET_SECOND = BINARY_OFFSET_MINUTE + Byte.BYTES , + BINARY_OFFSET_NANO = BINARY_OFFSET_SECOND + Byte.BYTES , + BINARY_LENGTH = BINARY_OFFSET_NANO + Integer.BYTES + ; + + private static final String TYPE_NAME = LocalTime.class.getName(); + + + + /////////////////////////////////////////////////////////////////////////// + // static methods // + /////////////////// + + public static BinaryHandlerLocalTime New() + { + return new BinaryHandlerLocalTime(); + } + + + + /////////////////////////////////////////////////////////////////////////// + // instance fields // + //////////////////// + + // only needed where the type is an ordinary class and its instances are populated after creation. + private final long memoryOffsetHour ; + private final long memoryOffsetMinute; + private final long memoryOffsetSecond; + private final long memoryOffsetNano ; + + + + /////////////////////////////////////////////////////////////////////////// + // constructors // + ///////////////// + + BinaryHandlerLocalTime() + { + super( + LocalTime.class, + CustomFields( + /* Qualified with the declaring type, so this description stays the one the reflective + * handling produced and data written before this handler existed still matches. + */ + CustomField(byte.class, TYPE_NAME, "hour" ), + CustomField(byte.class, TYPE_NAME, "minute"), + CustomField(byte.class, TYPE_NAME, "second"), + CustomField(int.class , TYPE_NAME, "nano" ) + ) + ); + + final boolean populated = !this.isValueClassType(); + this.memoryOffsetHour = populated + ? XMemory.objectFieldOffset(XReflect.getAnyField(LocalTime.class, "hour")) + : -1 + ; + this.memoryOffsetMinute = populated + ? XMemory.objectFieldOffset(XReflect.getAnyField(LocalTime.class, "minute")) + : -1 + ; + this.memoryOffsetSecond = populated + ? XMemory.objectFieldOffset(XReflect.getAnyField(LocalTime.class, "second")) + : -1 + ; + this.memoryOffsetNano = populated + ? XMemory.objectFieldOffset(XReflect.getAnyField(LocalTime.class, "nano")) + : -1 + ; + } + + + + /////////////////////////////////////////////////////////////////////////// + // override methods // + ///////////////////// + + @Override + public void store( + final Binary data , + final LocalTime instance, + final long objectId, + final PersistenceStoreHandler handler + ) + { + data.storeEntityHeader(BINARY_LENGTH, this.typeId(), objectId); + data.store_byte(BINARY_OFFSET_HOUR , (byte)instance.getHour() ); + data.store_byte(BINARY_OFFSET_MINUTE, (byte)instance.getMinute()); + data.store_byte(BINARY_OFFSET_SECOND, (byte)instance.getSecond()); + data.store_int (BINARY_OFFSET_NANO , instance.getNano() ); + } + + @Override + public LocalTime create(final Binary data, final PersistenceLoadHandler handler) + { + return LocalTime.of( + data.read_byte(BINARY_OFFSET_HOUR) , + data.read_byte(BINARY_OFFSET_MINUTE), + data.read_byte(BINARY_OFFSET_SECOND), + data.read_int (BINARY_OFFSET_NANO) + ); + } + + @Override + public void updateState(final Binary data, final LocalTime instance, final PersistenceLoadHandler handler) + { + if(this.isValueClassType()) + { + // already complete: it was built from its content rather than populated + return; + } + + XMemory.set_byte(instance, this.memoryOffsetHour , data.read_byte(BINARY_OFFSET_HOUR) ); + XMemory.set_byte(instance, this.memoryOffsetMinute, data.read_byte(BINARY_OFFSET_MINUTE)); + XMemory.set_byte(instance, this.memoryOffsetSecond, data.read_byte(BINARY_OFFSET_SECOND)); + XMemory.set_int (instance, this.memoryOffsetNano , data.read_int (BINARY_OFFSET_NANO) ); + } + +} diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerMonthDay.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerMonthDay.java index d1448b08..fb8b7cc0 100644 --- a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerMonthDay.java +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerMonthDay.java @@ -18,6 +18,7 @@ import org.eclipse.serializer.persistence.binary.types.AbstractBinaryHandlerCustomNonReferentialFixedLength; import org.eclipse.serializer.persistence.binary.types.Binary; +import org.eclipse.serializer.persistence.binary.types.ValidatingBinaryHandler; import org.eclipse.serializer.persistence.types.PersistenceLoadHandler; import org.eclipse.serializer.persistence.types.PersistenceStoreHandler; @@ -30,6 +31,7 @@ * */ public class BinaryHandlerMonthDay extends AbstractBinaryHandlerCustomNonReferentialFixedLength +implements ValidatingBinaryHandler { static final long BINARY_OFFSET_MONTH = 0L; static final long BINARY_OFFSET_DAY = BINARY_OFFSET_MONTH + Integer.BYTES; @@ -64,10 +66,27 @@ protected BinaryHandlerMonthDay() // methods // //////////// /// + /** + * This handler transfers state in {@link #create} alone, so an instance it is handed here was + * created elsewhere and cannot be updated. Validating instead of ignoring keeps a divergence from + * dropping the persisted state silently, e.g. for an explicitly set root. + */ @Override public void updateState(final Binary data, final MonthDay instance, final PersistenceLoadHandler handler) { - // no-op + this.validateState(data, instance, handler); + } + + @Override + public MonthDay getValidationStateFromInstance(final MonthDay instance) + { + return instance; + } + + @Override + public MonthDay getValidationStateFromBinary(final Binary data) + { + return binaryState(data); } @Override @@ -81,6 +100,11 @@ public void store(final Binary data, final MonthDay instance, final long objectI @Override public MonthDay create(final Binary data, final PersistenceLoadHandler handler) + { + return binaryState(data); + } + + private static MonthDay binaryState(final Binary data) { return MonthDay.of( data.read_int (BINARY_OFFSET_MONTH), diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerOffsetDateTime.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerOffsetDateTime.java new file mode 100644 index 00000000..0bb068e9 --- /dev/null +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerOffsetDateTime.java @@ -0,0 +1,70 @@ +package org.eclipse.serializer.persistence.binary.java.time; + +/*- + * #%L + * Eclipse Serializer Persistence Binary + * %% + * Copyright (C) 2023 - 2026 MicroStream Software + * %% + * This program and the accompanying materials are made + * available under the terms of the Eclipse Public License 2.0 + * which is available at https://www.eclipse.org/legal/epl-2.0/ + * + * SPDX-License-Identifier: EPL-2.0 + * #L% + */ + +import java.time.LocalDateTime; +import java.time.OffsetDateTime; +import java.time.ZoneOffset; + +import org.eclipse.serializer.persistence.binary.types.AbstractBinaryHandlerCustomComposedValueType; + +/** + * Handler for {@link OffsetDateTime}, which holds its date-time and offset in two references. Where + * it is a value class, the instance is built through + * {@link OffsetDateTime#of(LocalDateTime, ZoneOffset)}; see + * {@link AbstractBinaryHandlerCustomComposedValueType} for the mechanism. + */ +public final class BinaryHandlerOffsetDateTime extends AbstractBinaryHandlerCustomComposedValueType +{ + /////////////////////////////////////////////////////////////////////////// + // static methods // + /////////////////// + + public static BinaryHandlerOffsetDateTime New() + { + return new BinaryHandlerOffsetDateTime(); + } + + + + /////////////////////////////////////////////////////////////////////////// + // constructors // + ///////////////// + + BinaryHandlerOffsetDateTime() + { + super( + OffsetDateTime.class, + Part.New(LocalDateTime.class, "dateTime", OffsetDateTime::toLocalDateTime), + Part.New(ZoneOffset.class , "offset" , OffsetDateTime::getOffset ) + ); + } + + + + /////////////////////////////////////////////////////////////////////////// + // override methods // + ///////////////////// + + @Override + protected OffsetDateTime createFromParts(final Object[] parts) + { + return OffsetDateTime.of( + (LocalDateTime)parts[0], + (ZoneOffset) parts[1] + ); + } + +} diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerOffsetTime.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerOffsetTime.java new file mode 100644 index 00000000..4453df9b --- /dev/null +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerOffsetTime.java @@ -0,0 +1,69 @@ +package org.eclipse.serializer.persistence.binary.java.time; + +/*- + * #%L + * Eclipse Serializer Persistence Binary + * %% + * Copyright (C) 2023 - 2026 MicroStream Software + * %% + * This program and the accompanying materials are made + * available under the terms of the Eclipse Public License 2.0 + * which is available at https://www.eclipse.org/legal/epl-2.0/ + * + * SPDX-License-Identifier: EPL-2.0 + * #L% + */ + +import java.time.LocalTime; +import java.time.OffsetTime; +import java.time.ZoneOffset; + +import org.eclipse.serializer.persistence.binary.types.AbstractBinaryHandlerCustomComposedValueType; + +/** + * Handler for {@link OffsetTime}, which holds its time and offset in two references. Where it is a + * value class, the instance is built through {@link OffsetTime#of(LocalTime, ZoneOffset)}; see + * {@link AbstractBinaryHandlerCustomComposedValueType} for the mechanism. + */ +public final class BinaryHandlerOffsetTime extends AbstractBinaryHandlerCustomComposedValueType +{ + /////////////////////////////////////////////////////////////////////////// + // static methods // + /////////////////// + + public static BinaryHandlerOffsetTime New() + { + return new BinaryHandlerOffsetTime(); + } + + + + /////////////////////////////////////////////////////////////////////////// + // constructors // + ///////////////// + + BinaryHandlerOffsetTime() + { + super( + OffsetTime.class, + Part.New(LocalTime.class , "time" , OffsetTime::toLocalTime), + Part.New(ZoneOffset.class, "offset", OffsetTime::getOffset ) + ); + } + + + + /////////////////////////////////////////////////////////////////////////// + // override methods // + ///////////////////// + + @Override + protected OffsetTime createFromParts(final Object[] parts) + { + return OffsetTime.of( + (LocalTime) parts[0], + (ZoneOffset)parts[1] + ); + } + +} diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerPeriod.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerPeriod.java index 573b013c..085e2ce4 100644 --- a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerPeriod.java +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerPeriod.java @@ -18,10 +18,12 @@ import org.eclipse.serializer.persistence.binary.types.AbstractBinaryHandlerCustomNonReferentialFixedLength; import org.eclipse.serializer.persistence.binary.types.Binary; +import org.eclipse.serializer.persistence.binary.types.ValidatingBinaryHandler; import org.eclipse.serializer.persistence.types.PersistenceLoadHandler; import org.eclipse.serializer.persistence.types.PersistenceStoreHandler; public final class BinaryHandlerPeriod extends AbstractBinaryHandlerCustomNonReferentialFixedLength +implements ValidatingBinaryHandler { static final long BINARY_OFFSET_YEARS = 0L; static final long BINARY_OFFSET_MONTHS = BINARY_OFFSET_YEARS + Integer.BYTES; @@ -79,6 +81,11 @@ public final void store( @Override public final Period create(final Binary data, final PersistenceLoadHandler handler) + { + return binaryState(data); + } + + private static Period binaryState(final Binary data) { return Period.of( data.read_int(BINARY_OFFSET_YEARS), @@ -87,10 +94,27 @@ public final Period create(final Binary data, final PersistenceLoadHandler handl ); } + /** + * This handler transfers state in {@link #create} alone, so an instance it is handed here was + * created elsewhere and cannot be updated. Validating instead of ignoring keeps a divergence from + * dropping the persisted state silently, e.g. for an explicitly set root. + */ @Override public final void updateState(final Binary data, final Period instance, final PersistenceLoadHandler handler) { - // no-op + this.validateState(data, instance, handler); + } + + @Override + public final Period getValidationStateFromInstance(final Period instance) + { + return instance; + } + + @Override + public final Period getValidationStateFromBinary(final Binary data) + { + return binaryState(data); } } diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerYear.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerYear.java new file mode 100644 index 00000000..f7e24e9e --- /dev/null +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerYear.java @@ -0,0 +1,132 @@ +package org.eclipse.serializer.persistence.binary.java.time; + +/*- + * #%L + * Eclipse Serializer Persistence Binary + * %% + * Copyright (C) 2023 - 2026 MicroStream Software + * %% + * This program and the accompanying materials are made + * available under the terms of the Eclipse Public License 2.0 + * which is available at https://www.eclipse.org/legal/epl-2.0/ + * + * SPDX-License-Identifier: EPL-2.0 + * #L% + */ + +import java.time.Year; + +import org.eclipse.serializer.memory.XMemory; +import org.eclipse.serializer.persistence.binary.types.AbstractBinaryHandlerCustomNonReferentialFixedLength; +import org.eclipse.serializer.persistence.binary.types.Binary; +import org.eclipse.serializer.persistence.types.PersistenceLoadHandler; +import org.eclipse.serializer.persistence.types.PersistenceStoreHandler; +import org.eclipse.serializer.reflect.XReflect; + +/** + * Handler for {@link Year}. + *

+ * Where {@link Year} is a value class, its instances cannot be created empty and populated + * afterwards: the population write is not reliably visible on an identity-less instance. It is + * therefore built from its persisted value through {@link Year#of(int)}. + *

+ * The instance is created complete either way, since a plugin reusing the value-type handlers (e.g. + * the REST viewer) relies on {@link #create} alone. Where the type is an ordinary class, an already + * registered instance is still populated in {@link #updateState}, preserving the update behavior the + * reflective handling had. + *

+ * The persisted form is byte-identical to the one the reflective handling produced, under the same + * type and member name, so existing data is unaffected. + */ +public final class BinaryHandlerYear extends AbstractBinaryHandlerCustomNonReferentialFixedLength +{ + /////////////////////////////////////////////////////////////////////////// + // constants // + ////////////// + + private static final long + BINARY_OFFSET_YEAR = 0 , + BINARY_LENGTH = BINARY_OFFSET_YEAR + Integer.BYTES + ; + + + + /////////////////////////////////////////////////////////////////////////// + // static methods // + /////////////////// + + public static BinaryHandlerYear New() + { + return new BinaryHandlerYear(); + } + + + + /////////////////////////////////////////////////////////////////////////// + // instance fields // + //////////////////// + + // only needed where the type is an ordinary class and its instances are populated after creation. + private final long memoryOffsetYear; + + + + /////////////////////////////////////////////////////////////////////////// + // constructors // + ///////////////// + + BinaryHandlerYear() + { + super( + Year.class, + CustomFields( + /* Qualified with the declaring type, so this description stays the one the reflective + * handling produced and data written before this handler existed still matches. + */ + CustomField(int.class, Year.class.getName(), "year") + ) + ); + + this.memoryOffsetYear = this.isValueClassType() + ? -1 + : XMemory.objectFieldOffset(XReflect.getAnyField(Year.class, "year")) + ; + } + + + + /////////////////////////////////////////////////////////////////////////// + // override methods // + ///////////////////// + + @Override + public void store( + final Binary data , + final Year instance, + final long objectId, + final PersistenceStoreHandler handler + ) + { + data.storeEntityHeader(BINARY_LENGTH, this.typeId(), objectId); + data.store_int(BINARY_OFFSET_YEAR, instance.getValue()); + } + + @Override + public Year create(final Binary data, final PersistenceLoadHandler handler) + { + return Year.of(data.read_int(BINARY_OFFSET_YEAR)); + } + + @Override + public void updateState(final Binary data, final Year instance, final PersistenceLoadHandler handler) + { + if(this.isValueClassType()) + { + // already complete: it was built from its content rather than populated + return; + } + + XMemory.set_int(instance, this.memoryOffsetYear, data.read_int(BINARY_OFFSET_YEAR)); + } + +} diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerYearMonth.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerYearMonth.java index c5e587a6..c6ca32a7 100644 --- a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerYearMonth.java +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerYearMonth.java @@ -18,6 +18,7 @@ import org.eclipse.serializer.persistence.binary.types.AbstractBinaryHandlerCustomNonReferentialFixedLength; import org.eclipse.serializer.persistence.binary.types.Binary; +import org.eclipse.serializer.persistence.binary.types.ValidatingBinaryHandler; import org.eclipse.serializer.persistence.types.PersistenceLoadHandler; import org.eclipse.serializer.persistence.types.PersistenceStoreHandler; @@ -30,6 +31,7 @@ * */ public class BinaryHandlerYearMonth extends AbstractBinaryHandlerCustomNonReferentialFixedLength +implements ValidatingBinaryHandler { static final long BINARY_OFFSET_YEAR = 0L; static final long BINARY_OFFSET_MONTH = BINARY_OFFSET_YEAR + Integer.BYTES; @@ -64,10 +66,27 @@ protected BinaryHandlerYearMonth() // methods // //////////// /// + /** + * This handler transfers state in {@link #create} alone, so an instance it is handed here was + * created elsewhere and cannot be updated. Validating instead of ignoring keeps a divergence from + * dropping the persisted state silently, e.g. for an explicitly set root. + */ @Override public void updateState(final Binary data, final YearMonth instance, final PersistenceLoadHandler handler) { - // no-op + this.validateState(data, instance, handler); + } + + @Override + public YearMonth getValidationStateFromInstance(final YearMonth instance) + { + return instance; + } + + @Override + public YearMonth getValidationStateFromBinary(final Binary data) + { + return binaryState(data); } @Override @@ -81,6 +100,11 @@ public void store(final Binary data, final YearMonth instance, final long object @Override public YearMonth create(final Binary data, final PersistenceLoadHandler handler) + { + return binaryState(data); + } + + private static YearMonth binaryState(final Binary data) { return YearMonth.of( data.read_int (BINARY_OFFSET_YEAR), diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerZoneRegion.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerZoneRegion.java new file mode 100644 index 00000000..1cf0af0c --- /dev/null +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerZoneRegion.java @@ -0,0 +1,101 @@ +package org.eclipse.serializer.persistence.binary.java.time; + +/*- + * #%L + * Eclipse Serializer Persistence Binary + * %% + * Copyright (C) 2023 - 2026 MicroStream Software + * %% + * This program and the accompanying materials are made + * available under the terms of the Eclipse Public License 2.0 + * which is available at https://www.eclipse.org/legal/epl-2.0/ + * + * SPDX-License-Identifier: EPL-2.0 + * #L% + */ + +import java.time.ZoneId; + +import org.eclipse.serializer.persistence.binary.types.AbstractBinaryHandlerCustomComposedValueType; + +/** + * Handler for {@code java.time.ZoneRegion}, the package-private {@link ZoneId} implementation for + * region-based zone ids. It holds nothing but its id; the zone rules are transient and re-resolved + * from the id. + *

+ * The instance is built completely through {@link ZoneId#of(String)} on every JVM, with its creation + * deferred until the id can be resolved. Populating instead would leave the rules unresolved, and a + * complete instance is what handlers building their own instances from a resolved zone need, e.g. + * for {@code ZonedDateTime}: a blank zone cannot answer for its rules. See + * {@link AbstractBinaryHandlerCustomComposedValueType} for the mechanism. + *

+ * The persisted form is byte-identical to the one the reflective handling produced, under the same + * type and member name, so existing data is unaffected. + */ +public final class BinaryHandlerZoneRegion extends AbstractBinaryHandlerCustomComposedValueType +{ + /////////////////////////////////////////////////////////////////////////// + // constants // + ////////////// + + private static final String TYPE_NAME = "java.time.ZoneRegion"; + + + + /////////////////////////////////////////////////////////////////////////// + // static methods // + /////////////////// + + @SuppressWarnings("unchecked") + private static Class handledType() + { + try + { + // the class is package-private, so it can only be resolved by name. Every instance is a ZoneId. + return (Class)Class.forName(TYPE_NAME); + } + catch(final ClassNotFoundException e) + { + throw new Error("JDK class " + TYPE_NAME + " not found.", e); + } + } + + public static BinaryHandlerZoneRegion New() + { + return new BinaryHandlerZoneRegion(); + } + + + + /////////////////////////////////////////////////////////////////////////// + // constructors // + ///////////////// + + BinaryHandlerZoneRegion() + { + super( + handledType(), + Part.New(String.class, "id", ZoneId::getId) + ); + } + + + + /////////////////////////////////////////////////////////////////////////// + // override methods // + ///////////////////// + + @Override + public boolean isCreationDeferred() + { + // built from its resolved id instead of being populated, see the type comment. + return true; + } + + @Override + protected ZoneId createFromParts(final Object[] parts) + { + return ZoneId.of((String)parts[0]); + } + +} diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerZonedDateTime.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerZonedDateTime.java new file mode 100644 index 00000000..333806bc --- /dev/null +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/time/BinaryHandlerZonedDateTime.java @@ -0,0 +1,76 @@ +package org.eclipse.serializer.persistence.binary.java.time; + +/*- + * #%L + * Eclipse Serializer Persistence Binary + * %% + * Copyright (C) 2023 - 2026 MicroStream Software + * %% + * This program and the accompanying materials are made + * available under the terms of the Eclipse Public License 2.0 + * which is available at https://www.eclipse.org/legal/epl-2.0/ + * + * SPDX-License-Identifier: EPL-2.0 + * #L% + */ + +import java.time.LocalDateTime; +import java.time.ZoneId; +import java.time.ZoneOffset; +import java.time.ZonedDateTime; + +import org.eclipse.serializer.persistence.binary.types.AbstractBinaryHandlerCustomComposedValueType; + +/** + * Handler for {@link ZonedDateTime}, which holds its date-time, offset and zone in three references. + * Where it is a value class, the instance is built through + * {@link ZonedDateTime#ofInstant(LocalDateTime, ZoneOffset, ZoneId)}: for a stored state whose + * offset is valid for its zone that reproduces the state exactly, and should the zone's rules have + * changed since storing, it preserves the instant instead of failing. Asking the zone for its rules + * requires a complete zone instance, which its handler's deferred creation provides. See + * {@link AbstractBinaryHandlerCustomComposedValueType} for the mechanism. + */ +public final class BinaryHandlerZonedDateTime extends AbstractBinaryHandlerCustomComposedValueType +{ + /////////////////////////////////////////////////////////////////////////// + // static methods // + /////////////////// + + public static BinaryHandlerZonedDateTime New() + { + return new BinaryHandlerZonedDateTime(); + } + + + + /////////////////////////////////////////////////////////////////////////// + // constructors // + ///////////////// + + BinaryHandlerZonedDateTime() + { + super( + ZonedDateTime.class, + Part.New(LocalDateTime.class, "dateTime", ZonedDateTime::toLocalDateTime), + Part.New(ZoneOffset.class , "offset" , ZonedDateTime::getOffset ), + Part.New(ZoneId.class , "zone" , ZonedDateTime::getZone ) + ); + } + + + + /////////////////////////////////////////////////////////////////////////// + // override methods // + ///////////////////// + + @Override + protected ZonedDateTime createFromParts(final Object[] parts) + { + return ZonedDateTime.ofInstant( + (LocalDateTime)parts[0], + (ZoneOffset) parts[1], + (ZoneId) parts[2] + ); + } + +} diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/util/BinaryHandlerOptional.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/util/BinaryHandlerOptional.java new file mode 100644 index 00000000..345d6178 --- /dev/null +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/java/util/BinaryHandlerOptional.java @@ -0,0 +1,73 @@ +package org.eclipse.serializer.persistence.binary.java.util; + +/*- + * #%L + * Eclipse Serializer Persistence Binary + * %% + * Copyright (C) 2023 - 2026 MicroStream Software + * %% + * This program and the accompanying materials are made + * available under the terms of the Eclipse Public License 2.0 + * which is available at https://www.eclipse.org/legal/epl-2.0/ + * + * SPDX-License-Identifier: EPL-2.0 + * #L% + */ + +import java.util.Optional; + +import org.eclipse.serializer.persistence.binary.types.AbstractBinaryHandlerCustomComposedValueType; + +/** + * Handler for {@link Optional}, which holds its content in a single reference. Where it is a value + * class, the instance is built through {@link Optional#ofNullable(Object)}; see + * {@link AbstractBinaryHandlerCustomComposedValueType} for the mechanism. + *

+ * An absent value needs no marker of its own: a null reference reads back as + * {@link Optional#empty()}, which is what {@link Optional#ofNullable(Object)} answers for it. + */ +public final class BinaryHandlerOptional extends AbstractBinaryHandlerCustomComposedValueType> +{ + /////////////////////////////////////////////////////////////////////////// + // static methods // + /////////////////// + + @SuppressWarnings({"unchecked", "rawtypes"}) + private static Class> handledType() + { + // no way to get ".class" of a parameterized type otherwise + return (Class)Optional.class; + } + + public static BinaryHandlerOptional New() + { + return new BinaryHandlerOptional(); + } + + + + /////////////////////////////////////////////////////////////////////////// + // constructors // + ///////////////// + + BinaryHandlerOptional() + { + super( + handledType(), + Part.New(Object.class, "value", optional -> optional.orElse(null)) + ); + } + + + + /////////////////////////////////////////////////////////////////////////// + // override methods // + ///////////////////// + + @Override + protected Optional createFromParts(final Object[] parts) + { + return Optional.ofNullable(parts[0]); + } + +} diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/org/eclipse/serializer/entity/BinaryHandlerEntityLoading.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/org/eclipse/serializer/entity/BinaryHandlerEntityLoading.java index 1b57c1fa..22a29858 100644 --- a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/org/eclipse/serializer/entity/BinaryHandlerEntityLoading.java +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/org/eclipse/serializer/entity/BinaryHandlerEntityLoading.java @@ -111,6 +111,24 @@ public Class type() return this.delegate.type(); } + @Override + public boolean isValueClassType() + { + return this.delegate.isValueClassType(); + } + + @Override + public boolean isCreationDeferred() + { + return this.delegate.isCreationDeferred(); + } + + @Override + public void prepareLoadItem(final Binary data) + { + this.delegate.prepareLoadItem(data); + } + @Override public boolean isValidEntityType(final Class type) { diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/org/eclipse/serializer/persistence/types/BinaryHandlerPersistenceRootsDefault.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/org/eclipse/serializer/persistence/types/BinaryHandlerPersistenceRootsDefault.java index c241e0e0..373448a2 100644 --- a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/org/eclipse/serializer/persistence/types/BinaryHandlerPersistenceRootsDefault.java +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/org/eclipse/serializer/persistence/types/BinaryHandlerPersistenceRootsDefault.java @@ -32,6 +32,7 @@ import org.eclipse.serializer.persistence.types.PersistenceRoots; import org.eclipse.serializer.persistence.types.PersistenceStoreHandler; import org.eclipse.serializer.reference.Referencing; +import org.eclipse.serializer.reflect.XReflect; import org.eclipse.serializer.typing.KeyValue; @@ -303,11 +304,15 @@ private void registerInstancesPerObjectId( final Object rootInstance = rootEntry.value().instance(); // instances can be null when either explicitly registered to be null in the refactoring or legacy enum - if(rootInstance != null) + if(rootInstance != null && !XReflect.isValueInstance(rootInstance)) { + /* Value instances cannot be registered (see PersistenceTypeHandler#isValueClassType). + * Root resolving is unaffected, it goes by identifier. + */ + // must be the original identifier, not the potentially re-mapped identifier of the entry! final Long rootObjectId = rootIdMapping.get(rootEntry.key()); - + // all live instances are registered for their OID. registry.registerConstant(rootObjectId.longValue(), rootInstance); } diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/org/eclipse/serializer/persistence/types/BinaryHandlerRootReferenceDefault.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/org/eclipse/serializer/persistence/types/BinaryHandlerRootReferenceDefault.java index 391d3b0b..a01ebdb0 100644 --- a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/org/eclipse/serializer/persistence/types/BinaryHandlerRootReferenceDefault.java +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/org/eclipse/serializer/persistence/types/BinaryHandlerRootReferenceDefault.java @@ -115,6 +115,13 @@ public final PersistenceRootReference.Default create( final Object rootInstance = this.rootReference.get(); if(rootInstance != null) { + /* Reached with a value instance only when the root was installed through a supplier, which + * cannot resolve it eagerly and therefore cannot check it. Failing here still fails at + * startup, and with the reason, instead of leaving the JVM to refuse the registration below + * with a bare IdentityException. + */ + PersistenceRootReference.validateRootInstance(rootInstance); + /* * If the singleton instance references a defined root object, it must be registered for the persisted * objectId, even if the id references a record of different, incompatible, type. This conflict @@ -161,8 +168,14 @@ public final void updateState( /* * If the instance has no explicit root instance set, a * generically loaded and instantiated root instance is set. + * + * Deliberately not #setRoot: that rejects a value instance because an explicitly set root + * could never receive its persisted state. A loaded root has nothing to receive - it *is* + * the persisted state - so a persisted value root is resolved here like any other. */ - this.rootReference.setRoot(loadedRoot); + this.rootReference.setRootSupplier(() -> + loadedRoot + ); return; } diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/org/eclipse/serializer/reference/BinaryHandlerLazyDefault.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/org/eclipse/serializer/reference/BinaryHandlerLazyDefault.java index 2f4d301d..ae12aeac 100644 --- a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/org/eclipse/serializer/reference/BinaryHandlerLazyDefault.java +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/org/eclipse/serializer/reference/BinaryHandlerLazyDefault.java @@ -94,6 +94,7 @@ public final void store( final Object referent = instance.peek(); final long referenceOid; + boolean relinkValue = false; if(referent == null) { @@ -114,6 +115,28 @@ public final void store( handler.noteTrustedReference(referenceOid); } } + else if(isLinkedValueReferent(instance, referent, handler)) + { + if(handler.isEagerStoring()) + { + /* An eager store must reach everything the referent references, so the referent is applied + * despite being immutable. That assigns a new objectId, hence the link has to move to it - + * which is sound precisely because the two objectIds address indistinguishable state. + */ + referenceOid = handler.apply(referent); + relinkValue = true; + } + else + { + /* A value instance has no identity, so applying it would assign a new objectId on every + * store and invalidate the existing link. That is unnecessary: the referent is immutable, + * so the entity the link points to can never become stale. The established link is kept + * and the referent is referenced instead of stored again, exactly like an unloaded one - + * but with the referent at hand, so a target missing that entity can have it re-stored. + */ + referenceOid = handler.applyKnown(referent, instance.objectId()); + } + } else { // OID validation or updating is done by linking logic @@ -150,15 +173,42 @@ public final void store( * previous state (unstored and thus not clearable, or linked to its prior valid objectId). */ final ObjectSwizzling objectRetriever = handler.getObjectRetriever(); + final boolean moveLink = relinkValue; handler.registerCommitListener(() -> - instance.$link(referenceOid, objectRetriever) - ); + { + if(moveLink) + { + // the previous link is valid until here, so it may only be released after the commit succeeded. + instance.$unlink(); + } + instance.$link(referenceOid, objectRetriever); + }); // lazy reference instance must be stored in any case data.storeEntityHeader(Binary.referenceBinaryLength(1), this.typeId(), objectId); data.store_long(referenceOid); } + /** + * Whether the referent is a value instance behind an established link, whose store must either + * keep the link (the referent is immutable, so the entity behind it cannot become stale) or, on + * an eager store, apply the referent and move the link to the newly assigned objectId. + */ + private static boolean isLinkedValueReferent( + final Lazy.Default instance, + final Object referent, + final PersistenceStoreHandler handler + ) + { + /* A lazy reference of another storage keeps taking the ordinary path, where a differing + * objectId is a genuine error and is reported as such. + */ + return Swizzling.isFoundId(instance.objectId()) + && XReflect.isValueInstance(referent) + && instance.$getLoader() == handler.getObjectRetriever() + ; + } + @SuppressWarnings("unchecked") @Override public final Lazy.Default create(final Binary data, final PersistenceLoadHandler handler) diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/AbstractBinaryHandlerCustomComposedValueType.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/AbstractBinaryHandlerCustomComposedValueType.java new file mode 100644 index 00000000..53006ad1 --- /dev/null +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/AbstractBinaryHandlerCustomComposedValueType.java @@ -0,0 +1,325 @@ +package org.eclipse.serializer.persistence.binary.types; + +/*- + * #%L + * Eclipse Serializer Persistence Binary + * %% + * Copyright (C) 2023 - 2026 MicroStream Software + * %% + * This program and the accompanying materials are made + * available under the terms of the Eclipse Public License 2.0 + * which is available at https://www.eclipse.org/legal/epl-2.0/ + * + * SPDX-License-Identifier: EPL-2.0 + * #L% + */ + +import static org.eclipse.serializer.util.X.notNull; + +import java.util.function.Function; + +import org.eclipse.serializer.collections.BulkList; +import org.eclipse.serializer.collections.types.XGettingSequence; +import org.eclipse.serializer.memory.XMemory; +import org.eclipse.serializer.persistence.binary.exceptions.BinaryPersistenceException; +import org.eclipse.serializer.persistence.types.PersistenceFunction; +import org.eclipse.serializer.persistence.types.PersistenceLoadHandler; +import org.eclipse.serializer.persistence.types.PersistenceReferenceLoader; +import org.eclipse.serializer.persistence.types.PersistenceStoreHandler; +import org.eclipse.serializer.persistence.types.PersistenceTypeDefinitionMemberFieldGeneric; +import org.eclipse.serializer.reflect.XReflect; + +/** + * Base for handlers of a type that holds its state in references to parts, e.g. a + * {@code java.time.LocalDateTime} holding a date and a time, and that is a value class on a suitably + * recent JVM. + *

+ * Where the handled type is a value class, its instances cannot be created empty and populated + * afterwards, and its constructor cannot be reached because its module does not open the package. The + * instance is therefore built from its resolved parts by {@link #createFromParts(Object[])}. That + * resolves references in {@link #create}, which only a handler reporting + * {@link #isCreationDeferred()} may do; the loader defers such a creation until they can be resolved. + * Note that a resolved part is guaranteed to exist but not to be populated, so + * {@link #createFromParts(Object[])} may only use a part's state if that part's own handler creates + * complete instances. + *

+ * Where it is an ordinary class, it keeps being created empty and populated, so that its instances + * stay registered by identity as before. A subclass whose instances must be built from their + * content on every JVM — because populating them would leave them incomplete — opts out + * of that by overriding {@link #isCreationDeferred()} to a constant {@literal true}, which this base + * reads in its constructor. + *

+ * The persisted form is one reference per part, under the field's name qualified with the declaring + * type, so it is byte-identical to the one the reflective handling produced and existing data is + * unaffected. + * + * @param the handled type. + */ +public abstract class AbstractBinaryHandlerCustomComposedValueType extends AbstractBinaryHandlerCustom +implements ValidatingBinaryHandler +{ + /////////////////////////////////////////////////////////////////////////// + // static methods // + /////////////////// + + private static XGettingSequence buildFields( + final Class type , + final Part[] parts + ) + { + final BulkList fields = BulkList.New(); + for(final Part part : parts) + { + /* Qualified with the declaring type, so this description stays the one the reflective + * handling produced and data written before the handler existed still matches. + */ + fields.add(CustomField(part.type, type.getName(), part.name)); + } + + return CustomFields(fields.toArray(PersistenceTypeDefinitionMemberFieldGeneric.class)); + } + + + + /////////////////////////////////////////////////////////////////////////// + // instance fields // + //////////////////// + + private final Part[] parts ; + private final long binaryLength; + + // only needed where the type is an ordinary class and its instances are populated after creation. + private final long[] memoryOffsets; + + + + /////////////////////////////////////////////////////////////////////////// + // constructors // + ///////////////// + + @SafeVarargs + protected AbstractBinaryHandlerCustomComposedValueType(final Class type, final Part... parts) + { + super(type, buildFields(type, parts)); + + this.parts = parts.clone(); + this.binaryLength = parts.length * Binary.referenceBinaryLength(1); + + final boolean populated = !this.isCreationDeferred(); + this.memoryOffsets = new long[parts.length]; + for(int i = 0; i < parts.length; i++) + { + this.memoryOffsets[i] = populated + ? XMemory.objectFieldOffset(XReflect.getAnyField(type, parts[i].name)) + : -1 + ; + } + } + + + + /////////////////////////////////////////////////////////////////////////// + // declared methods // + ///////////////////// + + /** + * Builds an instance from its resolved parts, in the declared part order. Only called where this + * handler's creation is deferred; see the type comment for what a part guarantees at this point. + * + * @param parts the resolved parts, in the order the handler declares them. + * + * @return the built instance. + */ + protected abstract T createFromParts(Object[] parts); + + private long binaryOffset(final int index) + { + return index * Binary.referenceBinaryLength(1); + } + + private Object[] resolveParts(final Binary data, final PersistenceLoadHandler handler) + { + final Object[] parts = new Object[this.parts.length]; + for(int i = 0; i < parts.length; i++) + { + parts[i] = handler.lookupObject(data.read_long(this.binaryOffset(i))); + } + + return parts; + } + + + + /////////////////////////////////////////////////////////////////////////// + // override methods // + ///////////////////// + + @Override + public void store( + final Binary data , + final T instance, + final long objectId, + final PersistenceStoreHandler handler + ) + { + data.storeEntityHeader(this.binaryLength, this.typeId(), objectId); + for(int i = 0; i < this.parts.length; i++) + { + data.store_long(this.binaryOffset(i), handler.apply(this.parts[i].getter.apply(instance))); + } + } + + @Override + public T create(final Binary data, final PersistenceLoadHandler handler) + { + if(!this.isCreationDeferred()) + { + // created blank; the parts are set in #updateState + return XMemory.instantiateBlank(this.type()); + } + + return this.createFromParts(this.resolveParts(data, handler)); + } + + @Override + public void initializeState(final Binary data, final T instance, final PersistenceLoadHandler handler) + { + if(this.isCreationDeferred()) + { + // built complete from its parts, so there is nothing to initialize afterwards + return; + } + + // created blank, so initializing it is populating it + this.updateState(data, instance, handler); + } + + @Override + public void updateState(final Binary data, final T instance, final PersistenceLoadHandler handler) + { + if(this.isCreationDeferred()) + { + /* Reached for an instance this handler did not build - the ones it builds are complete and + * go through #initializeState. Such an instance cannot be updated, so validating is what + * keeps a divergence from dropping the persisted state silently, e.g. for an explicitly set + * root, whose instance is registered for the persisted objectId before this handler ever + * sees the data. + */ + this.validateState(data, instance, handler); + + return; + } + + for(int i = 0; i < this.parts.length; i++) + { + XMemory.setObject( + instance, + this.memoryOffsets[i], + handler.lookupObject(data.read_long(this.binaryOffset(i))) + ); + } + } + + @Override + public void iterateInstanceReferences(final T instance, final PersistenceFunction iterator) + { + for(final Part part : this.parts) + { + iterator.apply(part.getter.apply(instance)); + } + } + + @Override + public void iterateLoadableReferences(final Binary data, final PersistenceReferenceLoader iterator) + { + for(int i = 0; i < this.parts.length; i++) + { + iterator.acceptObjectId(data.read_long(this.binaryOffset(i))); + } + } + + @Override + public boolean hasPersistedReferences() + { + return true; + } + + @Override + public boolean hasVaryingPersistedLengthInstances() + { + return false; + } + + @Override + public void validateState(final Binary data, final T instance, final PersistenceLoadHandler handler) + { + this.validateStates( + instance, + instance, + this.createFromParts(this.resolveParts(data, handler)) + ); + } + + @Override + public T getValidationStateFromInstance(final T instance) + { + return instance; + } + + /** + * Never called: the state a part carries can only be read through the load handler, so + * {@link #validateState} is overridden instead of composed from this method. + */ + @Override + public T getValidationStateFromBinary(final Binary data) + { + throw new BinaryPersistenceException( + "The persisted state of " + this.type().getName() + " can only be read with a load handler." + ); + } + + + + /////////////////////////////////////////////////////////////////////////// + // member types // + ///////////////// + + /** + * One referenced part of the handled type: the field it is persisted as, and how to read it from + * an instance. The field name must be the runtime field's name, since populating on an ordinary + * JVM resolves the field by it. + * + * @param the handled type. + */ + public static final class Part + { + final Class type ; + final String name ; + final Function getter; + + Part(final Class type, final String name, final Function getter) + { + super(); + this.type = type ; + this.name = name ; + this.getter = getter; + } + + /** + * Creates a part description. + * + * @param type the part's declared type, as persisted in the type dictionary. + * @param name the field's name. + * @param getter reads the part from an instance. + * + * @param the handled type. + * + * @return a new part description. + */ + public static Part New(final Class type, final String name, final Function getter) + { + return new Part<>(notNull(type), notNull(name), notNull(getter)); + } + } + +} diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/AbstractBinaryHandlerReflective.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/AbstractBinaryHandlerReflective.java index 9069fc66..ea54abc8 100644 --- a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/AbstractBinaryHandlerReflective.java +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/AbstractBinaryHandlerReflective.java @@ -34,10 +34,13 @@ import org.eclipse.serializer.persistence.types.PersistenceReferenceLoader; import org.eclipse.serializer.persistence.types.PersistenceStoreHandler; import org.eclipse.serializer.persistence.types.PersistenceTypeDefinitionMember; +import org.eclipse.serializer.persistence.types.PersistenceTypeDefinitionMemberField; import org.eclipse.serializer.persistence.types.PersistenceTypeDefinitionMemberFieldReflective; +import org.eclipse.serializer.persistence.types.PersistenceTypeDefinitionMemberFieldValueStruct; import org.eclipse.serializer.persistence.types.PersistenceTypeDescriptionMember; import org.eclipse.serializer.persistence.types.PersistenceTypeHandlerReflective; import org.eclipse.serializer.persistence.types.Persister; +import org.eclipse.serializer.persistence.types.PersistenceValueInliningResolver; import org.eclipse.serializer.reflect.XReflect; import org.eclipse.serializer.util.UtilStackTrace; @@ -82,12 +85,14 @@ protected static EqHashEnum Membe } protected static EqHashEnum deriveMembers( - final XGettingEnum fields , - final PersistenceFieldLengthResolver lengthResolver + final Class entityType , + final XGettingEnum fields , + final PersistenceFieldLengthResolver lengthResolver , + final PersistenceValueInliningResolver inliningResolver ) { final EqHashEnum members = MemberEnum(); - + for(final Field field : fields) { // just a precaution @@ -95,17 +100,62 @@ protected static EqHashEnum deri { throw new PersistenceExceptionTypeConsistency("static fields are not persistable."); } - - final PersistenceTypeDefinitionMemberFieldReflective member = declaredField(field, lengthResolver); - + + final PersistenceTypeDefinitionMemberFieldReflective member = deriveMember( + entityType, field, lengthResolver, inliningResolver + ); + if(!members.add(member)) { throw new PersistenceExceptionTypeConsistency("Duplicate member descriptions."); } } - + return members; } + + private static PersistenceTypeDefinitionMemberFieldReflective deriveMember( + final Class entityType , + final Field field , + final PersistenceFieldLengthResolver lengthResolver , + final PersistenceValueInliningResolver inliningResolver + ) + { + final XGettingEnum inlinedFields = inliningResolver.resolveInlinedFields(entityType, field); + if(inlinedFields == null) + { + return declaredField(field, lengthResolver); + } + + final BulkList inlinedMembers = BulkList.New(); + for(final Field inlinedField : inlinedFields) + { + /* A member of the inlined layout may be inlined itself, and then carries no object id either, so + * the slot stays free of them at every depth. The resolver has already established that the whole + * tree is eligible and acyclic, so asking it again per level only derives what it decided. + */ + final PersistenceTypeDefinitionMemberField inlinedMember = deriveMember( + field.getType(), inlinedField, lengthResolver, inliningResolver + ); + if(inlinedMember.hasReferences()) + { + /* An inlined slot is a fixed-length non-reference member, which is what lets the storage engine + * skip it as it skips a primitive. An object id inside one would be skipped along with it, so + * the garbage collector would never see the entity it points to and would collect it while it is + * still referenced. The member creation below refuses that as well; this here adds the field + * context to point at the resolver that wrongly deemed the field inlinable. + */ + throw new PersistenceExceptionTypeConsistency( + "Field " + field.getDeclaringClass().getName() + "#" + field.getName() + + " cannot be inlined: " + field.getType().getName() + + " holds references, and an inlined slot must not." + ); + } + inlinedMembers.add(inlinedMember); + } + + return PersistenceTypeDefinitionMemberFieldValueStruct.New(field, inlinedMembers); + } protected static final EqConstHashEnum filter( final XGettingCollection fields , @@ -146,12 +196,40 @@ protected static void createStorers( for(final PersistenceTypeDefinitionMemberFieldReflective member : storingMembers) { final boolean isEager = eagerEvaluator.isEagerStoring(entityType, member.field()); - + BinaryValueStorer customFieldStorer = fieldHandlerProvider.lookupFieldStorer(member.field(), isEager, switchByteOrder); - if (customFieldStorer != null) + if(member instanceof PersistenceTypeDefinitionMemberFieldValueStruct) + { + final PersistenceTypeDefinitionMemberFieldValueStruct struct = + (PersistenceTypeDefinitionMemberFieldValueStruct)member + ; + + if(customFieldStorer != null) + { + /* The two describe the same field differently: the custom storer writes an object id where + * the type description states an inlined layout. Reading that back would misinterpret the + * owner's whole binary form, so the conflict must be resolved by configuration. + */ + throw new PersistenceExceptionTypeConsistency( + "Field " + member.identifier() + " is both inlined and handled by a custom field storer." + ); + } + + storers[i++] = BinaryValueStructFunctions.provideStorer( + struct.field(), struct.members(), struct.persistentMinimumLength(), switchByteOrder + ); + } + else if (customFieldStorer != null) { storers[i++] = customFieldStorer; } + else if(XReflect.isValueClass(member.type())) + { + // a value may be laid out inside its owner, where its offset holds no object reference + storers[i++] = BinaryValueHandleFunctions.provideReferenceStorer( + member.field(), isEager, switchByteOrder + ); + } else { storers[i++] = BinaryValueFunctions.getObjectValueStorer(member.type(), isEager, switchByteOrder); @@ -221,6 +299,7 @@ protected static final long[] objectFieldOffsets( private final EqConstHashEnum referenceMembers, primitiveMembers, + structMembers , storingMembers , settingMembers ; @@ -251,6 +330,8 @@ protected static final long[] objectFieldOffsets( private final boolean switchByteOrder; private final BinaryFieldHandlerProvider fieldHandlerProvider; + + private final PersistenceValueInliningResolver inliningResolver; /* (28.10.2019 TM)TODO: encapsulate / abstract BinaryValue~ handling types. * While the per-field handling via the BinaryValue~ handling types is perfectly fine for JDK @@ -275,13 +356,15 @@ protected AbstractBinaryHandlerReflective( final PersistenceFieldLengthResolver lengthResolver , final PersistenceEagerStoringFieldEvaluator eagerEvaluator , final BinaryFieldHandlerProvider fieldHandlerProvider, + final PersistenceValueInliningResolver inliningResolver , final boolean switchByteOrder ) { super(type, typeName); - + this.switchByteOrder = switchByteOrder; this.fieldHandlerProvider = fieldHandlerProvider; + this.inliningResolver = inliningResolver; /* * Unsafe JavaDoc says ensureClassInitialized is "often needed" for getting the field base, so better do it. @@ -291,17 +374,36 @@ protected AbstractBinaryHandlerReflective( XMemory.ensureClassInitialized(type, persistableFields); final EqHashEnum instMembersInDeclOrdr = - deriveMembers(persistableFields, lengthResolver) + deriveMembers(type, persistableFields, lengthResolver, inliningResolver) ; - + this.membersInDeclaredOrder = this.deriveAllMembers(instMembersInDeclOrdr); - + // member instances are created from the persistable fields and split into references and primitives this.referenceMembers = this.filterReferenceMembers(instMembersInDeclOrdr, MemberEnum()).immure(); this.primitiveMembers = this.filterPrimitiveMembers(instMembersInDeclOrdr, MemberEnum()).immure(); - - // persistent order is all reference fields in declared order, then all primitive fields in declared order. - this.storingMembers = MemberEnum(this.referenceMembers).addAll(this.primitiveMembers).immure(); + this.structMembers = this.filterStructMembers(instMembersInDeclOrdr, MemberEnum()).immure(); + + /* Persistent order is all reference fields in declared order, then all inlined fields, then all + * primitive fields. Inlined fields sit behind the references so that the leading run of object ids + * stays exactly the references, which is what the reference iteration relies on. + */ + this.storingMembers = MemberEnum(this.referenceMembers) + .addAll(this.structMembers) + .addAll(this.primitiveMembers) + .immure() + ; + + /* Every member must land in exactly one group. Were one to fall through, it would silently vanish + * from the persistent form while still being part of the instance. + */ + if(this.storingMembers.size() != instMembersInDeclOrdr.size()) + { + throw new PersistenceExceptionTypeConsistency( + "Persistent member count " + this.storingMembers.size() + " of " + type.getName() + + " does not match its " + instMembersInDeclOrdr.size() + " persistable fields." + ); + } this.settingMembers = this.filterSettingMembers(this.storingMembers); // storing/setting memory offsets initialization must be overridable for enum special casing @@ -370,16 +472,52 @@ protected BinaryValueSetter deriveSetter( final PersistenceTypeDefinitionMemberFieldReflective member ) { + if(member instanceof PersistenceTypeDefinitionMemberFieldValueStruct) + { + return this.deriveStructSetter((PersistenceTypeDefinitionMemberFieldValueStruct)member); + } + BinaryValueSetter customFieldSetter = this.fieldHandlerProvider.lookupFieldSetter(member.field(), this.isSwitchedByteOrder()); if (customFieldSetter != null) { return customFieldSetter; } + else if(XReflect.isValueClass(member.type())) + { + // a value may be laid out inside its owner, where its offset holds no object reference + return BinaryValueHandleFunctions.provideReferenceSetter(member.field(), this.isSwitchedByteOrder()); + } else { return BinaryValueFunctions.getObjectValueSetter(member.type(), this.isSwitchedByteOrder()); } } + + private BinaryValueSetter deriveStructSetter(final PersistenceTypeDefinitionMemberFieldValueStruct member) + { + /* The inlined instance is constructed rather than populated, so the constructor's parameter order is + * needed alongside the persistent order. Re-resolving yields it, since the resolver reports the + * inlined type's fields in declaration order. + */ + final XGettingEnum declarationOrder = this.inliningResolver.resolveInlinedFields( + this.type(), member.field() + ); + if(declarationOrder == null) + { + throw new PersistenceExceptionTypeConsistency( + "Field " + member.identifier() + " is described as inlined but is no longer eligible for inlining." + ); + } + + return BinaryValueStructFunctions.provideSetter( + member.field() , + member.type() , + member.members() , + declarationOrder , + member.persistentMinimumLength(), + this.isSwitchedByteOrder() + ); + } protected long[] initializeStoringRefMemOffsets() { @@ -404,6 +542,16 @@ protected EqHashEnum filterRefer ); } + protected EqHashEnum filterStructMembers( + final XGettingCollection members, + final EqHashEnum target + ) + { + return members.filterTo(target, m -> + m instanceof PersistenceTypeDefinitionMemberFieldValueStruct + ); + } + protected EqHashEnum filterPrimitiveMembers( final XGettingCollection members, final EqHashEnum target diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/AbstractBinaryLegacyTypeHandlerTranslating.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/AbstractBinaryLegacyTypeHandlerTranslating.java index 75ee7309..b302c8b7 100644 --- a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/AbstractBinaryLegacyTypeHandlerTranslating.java +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/AbstractBinaryLegacyTypeHandlerTranslating.java @@ -292,6 +292,18 @@ public Class type() return this.typeHandler.type(); } + @Override + public boolean isValueClassType() + { + return this.typeHandler.isValueClassType(); + } + + @Override + public boolean isCreationDeferred() + { + return this.typeHandler.isCreationDeferred(); + } + @Override public void iterateInstanceReferences(final T instance, final PersistenceFunction iterator) { diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/Binary.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/Binary.java index dba35e85..5d343d55 100644 --- a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/Binary.java +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/Binary.java @@ -1433,6 +1433,55 @@ public final void storeReferences( this.storeReferencesAsList(binaryOffset, persister, array, arrayOffset, arrayLength); } + + /** + * Variant of + * {@link #storeReferences(long, long, long, PersistenceFunction, Object[], int, int)} that can + * reference an element by the object id a previous store assigned to it. + * + * @param typeId the entity's type id. + * @param objectId the entity's object id. + * @param binaryOffset the offset to write the reference list at. + * @param handler the store handler to hand the elements to. + * @param array the elements to be referenced. + * @param arrayOffset the first element's index. + * @param arrayLength the number of elements. + * @param storedReferents the element a previous store wrote per slot, overwritten with the one + * written now. Indexed absolutely, see + * {@link #storeReferencesAsList(long, PersistenceStoreHandler, Object[], int, int, Object[], long[])}. + * @param storedObjectIds the object id it was written under, overwritten with the one written + * now. Either array being {@literal null} applies every element. + * + * @see #storeReferencesAsList(long, PersistenceStoreHandler, Object[], int, int, Object[], long[]) + */ + public final void storeReferences( + final long typeId , + final long objectId , + final long binaryOffset , + final PersistenceStoreHandler handler , + final Object[] array , + final int arrayOffset , + final int arrayLength , + final Object[] storedReferents, + final long[] storedObjectIds + ) + { + this.storeEntityHeader( + binaryOffset + calculateReferenceListTotalBinaryLength(arrayLength), + typeId, + objectId + ); + + this.storeReferencesAsList( + binaryOffset, + handler, + array, + arrayOffset, + arrayLength, + storedReferents, + storedObjectIds + ); + } public final void storeFixedSize( final PersistenceStoreHandler handler , @@ -1834,7 +1883,7 @@ public final long collectElementsIntoArray( ) { long nonNullCount = 0L; - + final long binaryElementsStartAddress = this.binaryListElementsAddress(binaryOffset); for(int i = 0; i < target.length; i++) { @@ -1845,10 +1894,31 @@ public final long collectElementsIntoArray( nonNullCount++; } } - + return nonNullCount; } + /** + * Reads the object ids of a reference list without resolving them, so a caller that collected the + * elements themselves can record which id each of them came from. + * + * @param binaryOffset the offset of the list to read. + * @param target receives one id per element, and defines how many are read. + * + * @see #storeReferencesAsList(long, PersistenceStoreHandler, Object[], int, int, Object[], long[]) + */ + public final void collectElementObjectIds( + final long binaryOffset, + final long[] target + ) + { + final long binaryElementsStartAddress = this.binaryListElementsAddress(binaryOffset); + for(int i = 0; i < target.length; i++) + { + target[i] = this.get_longFromAddress(binaryElementsStartAddress + referenceBinaryLength(i)); + } + } + public final int collectListObjectReferences( final long binaryOffset, final PersistenceLoadHandler handler , @@ -2096,9 +2166,97 @@ public final void storeReferencesAsList( final int bound = offset + length; for(int i = offset; i < bound; i++) { - this.set_longToAddress(elementsDataAddress + referenceBinaryLength(i), persister.apply(array[i])); + this.set_longToAddress( + elementsDataAddress + referenceBinaryLength(i), + persister.apply(array[i]) + ); } } + + /** + * Variant of {@link #storeReferencesAsList(long, PersistenceFunction, Object[], int, int)} that can + * reference an element by the object id a previous store assigned to it instead of applying it + * again, which is what spares an identity-less element a superseded copy per store. + *

+ * The two passed arrays describe what a previous store wrote into this very list: element and + * object id per slot. A slot whose element is still the same one is referenced by its recorded id, + * every other slot is applied. Both arrays are overwritten with what this store writes, so they + * describe the previous store again once it is committed - and must therefore not be the arrays a + * previous store is still described by: a store that is not committed may not change what the + * persisted form of this entity references. + *

+ * Both are indexed like {@code array} itself, by the absolute element index rather than relative + * to {@code offset}, so each has to be at least {@code offset + length} long. + *

+ * Only applicable while the entity holding this list still references the recorded ids, which is + * what keeps their entities reachable. + * + * @param memoryOffset the offset to write the list at. + * @param handler the store handler to hand the elements to. + * @param array the elements to be referenced. + * @param offset the first element's index. + * @param length the number of elements. + * @param storedReferents the element a previous store wrote per slot, overwritten with the one + * written now. + * @param storedObjectIds the object id it was written under, overwritten with the one written + * now. Either array being {@literal null} applies every element. + */ + public final void storeReferencesAsList( + final long memoryOffset , + final PersistenceStoreHandler handler , + final Object[] array , + final int offset , + final int length , + final Object[] storedReferents, + final long[] storedObjectIds + ) + { + if(storedReferents == null || storedObjectIds == null) + { + this.storeReferencesAsList(memoryOffset, handler, array, offset, length); + return; + } + + this.storeListHeader( + memoryOffset, + referenceBinaryLength(length), + length + ); + + final long elementsDataAddress = this.address + memoryOffset + LIST_OFFSET_ELEMENTS; + + final int bound = offset + length; + for(int i = offset; i < bound; i++) + { + this.set_longToAddress( + elementsDataAddress + referenceBinaryLength(i), + storeReference(handler, array[i], storedReferents, storedObjectIds, i) + ); + } + } + + private static long storeReference( + final PersistenceStoreHandler handler , + final Object element , + final Object[] storedReferents, + final long[] storedObjectIds, + final int index + ) + { + /* The id describes the element that was written under it, not the slot, so it may only be + * reused for that very element. Comparing the two is what makes the record self-validating: + * a slot written at any point since - including while this store was in flight - simply + * fails to match and is applied. + */ + final long knownObjectId = storedReferents[index] == element + ? storedObjectIds[index] + : Swizzling.notFoundId() + ; + + storedReferents[index] = element; + + return storedObjectIds[index] = handler.applyKnown(element, knownObjectId); + } diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryHandlerGenericEnum.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryHandlerGenericEnum.java index 4018b993..3e7bb76b 100644 --- a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryHandlerGenericEnum.java +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryHandlerGenericEnum.java @@ -33,6 +33,7 @@ import org.eclipse.serializer.persistence.types.PersistenceTypeDefinitionMember; import org.eclipse.serializer.persistence.types.PersistenceTypeDefinitionMemberEnumConstant; import org.eclipse.serializer.persistence.types.PersistenceTypeDefinitionMemberFieldReflective; +import org.eclipse.serializer.persistence.types.PersistenceValueInliningResolver; import org.eclipse.serializer.reflect.XReflect; import org.eclipse.serializer.typing.XTypes; @@ -186,7 +187,8 @@ protected BinaryHandlerGenericEnum( final boolean switchByteOrder ) { - super(type, typeName, persistableFields, persisterFields, lengthResolver, eagerStoringFieldEvaluator, fieldHandlerProvider, switchByteOrder); + // an enum constant is an identity object, so its fields are never inlined into it + super(type, typeName, persistableFields, persisterFields, lengthResolver, eagerStoringFieldEvaluator, fieldHandlerProvider, PersistenceValueInliningResolver.Disabled(), switchByteOrder); // these are instance members in persistent order. Not to be mixed up with members in declared order this.allMembers = this.deriveAllMembers(this.instanceMembers()); @@ -204,10 +206,18 @@ protected BinaryHandlerGenericEnum( @Override protected BinaryValueSetter deriveSetter(final PersistenceTypeDefinitionMemberFieldReflective member) { - return this.isUnsettableField(member) - ? BinaryValueFunctions.getObjectValueSettingSkipper(member.type()) - : BinaryValueFunctions.getObjectValueSetter(member.type(), this.isSwitchedByteOrder()) - ; + if(this.isUnsettableField(member)) + { + return BinaryValueFunctions.getObjectValueSettingSkipper(member.type()); + } + + if(XReflect.isValueClass(member.type())) + { + // a value may be laid out inside its owner, where its offset holds no object reference + return BinaryValueHandleFunctions.provideReferenceSetter(member.field(), this.isSwitchedByteOrder()); + } + + return BinaryValueFunctions.getObjectValueSetter(member.type(), this.isSwitchedByteOrder()); } @Override diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryHandlerGenericType.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryHandlerGenericType.java index beed0972..47992cf0 100644 --- a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryHandlerGenericType.java +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryHandlerGenericType.java @@ -23,6 +23,7 @@ import org.eclipse.serializer.persistence.types.PersistenceFieldLengthResolver; import org.eclipse.serializer.persistence.types.PersistenceLoadHandler; import org.eclipse.serializer.persistence.types.PersistenceTypeInstantiator; +import org.eclipse.serializer.persistence.types.PersistenceValueInliningResolver; public final class BinaryHandlerGenericType extends AbstractBinaryHandlerReflective { @@ -35,6 +36,7 @@ public static BinaryHandlerGenericType New( final PersistenceEagerStoringFieldEvaluator eagerStoringFieldEvaluator , final PersistenceTypeInstantiator persistenceTypeInstantiator, final BinaryFieldHandlerProvider fieldHandlerProvider , + final PersistenceValueInliningResolver inliningResolver , final boolean switchByteOrder ) { @@ -47,6 +49,7 @@ public static BinaryHandlerGenericType New( eagerStoringFieldEvaluator , persistenceTypeInstantiator, fieldHandlerProvider , + inliningResolver , switchByteOrder ); } @@ -74,10 +77,11 @@ protected BinaryHandlerGenericType( final PersistenceEagerStoringFieldEvaluator eagerStoringFieldEvaluator, final PersistenceTypeInstantiator instantiator , final BinaryFieldHandlerProvider fieldHandlerProvider , + final PersistenceValueInliningResolver inliningResolver , final boolean switchByteOrder ) { - super(type, typeName, persistableFields, persisterFields, lengthResolver, eagerStoringFieldEvaluator, fieldHandlerProvider, switchByteOrder); + super(type, typeName, persistableFields, persisterFields, lengthResolver, eagerStoringFieldEvaluator, fieldHandlerProvider, inliningResolver, switchByteOrder); this.instantiator = notNull(instantiator); } diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryHandlerGenericValueClass.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryHandlerGenericValueClass.java new file mode 100644 index 00000000..127f3de8 --- /dev/null +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryHandlerGenericValueClass.java @@ -0,0 +1,507 @@ +package org.eclipse.serializer.persistence.binary.types; + +/*- + * #%L + * Eclipse Serializer Persistence Binary + * %% + * Copyright (C) 2023 - 2026 MicroStream Software + * %% + * This program and the accompanying materials are made + * available under the terms of the Eclipse Public License 2.0 + * which is available at https://www.eclipse.org/legal/epl-2.0/ + * + * SPDX-License-Identifier: EPL-2.0 + * #L% + */ + +import java.lang.invoke.MethodHandle; +import java.lang.invoke.MethodHandles; +import java.lang.invoke.MethodType; +import java.lang.reflect.Constructor; +import java.lang.reflect.Field; + +import org.eclipse.serializer.collections.types.XGettingEnum; +import org.eclipse.serializer.persistence.binary.exceptions.BinaryPersistenceException; +import org.eclipse.serializer.persistence.exceptions.PersistenceExceptionTypeNotPersistable; +import org.eclipse.serializer.persistence.types.PersistenceEagerStoringFieldEvaluator; +import org.eclipse.serializer.persistence.types.PersistenceFieldLengthResolver; +import org.eclipse.serializer.persistence.types.PersistenceLoadHandler; +import org.eclipse.serializer.persistence.types.PersistenceStoreHandler; +import org.eclipse.serializer.persistence.types.PersistenceTypeDefinitionMemberFieldReflective; +import org.eclipse.serializer.persistence.types.PersistenceTypeDefinitionMemberFieldValueStruct; +import org.eclipse.serializer.persistence.types.PersistenceValueInliningResolver; +import org.eclipse.serializer.reflect.XReflect; + +/** + * Reflective type handler for value classes (JEP 401). + *

+ * Value instances have no identity and no mutable state: they cannot be allocated blank and filled + * afterwards like identity instances are. This handler therefore reads all field values from the + * persisted data first and passes them to a constructor, so the instance is complete the moment it + * exists. Storing is inherited unchanged, so the persisted form is identical to the one an identity + * class of the same structure would produce. + *

+ * The constructor is required to accept the persistable instance fields in declaration order, which + * is exactly what a record's canonical constructor does. If no such constructor exists, the type + * cannot be handled generically and a custom type handler must be registered for it. This is + * established once, at handler creation. + *

+ * It is required to do more than exist: loading hands it the stored field values, so it is the inverse + * of storing only if it assigns each argument to the corresponding field unmodified. That cannot be + * read off the type, so it is verified by constructing from chosen values at handler creation and, where + * that cannot answer, against the first instance stored - see {@link ValueClassConstructorContract}, + * which also states why the identity case needs no such check. + *

+ * Note that {@link #create(Binary, PersistenceLoadHandler)} resolves the instance's references, + * unlike the identity case where they are resolved in {@code initializeState}. The loader + * accommodates this by deferring the creation of value instances until their references can be + * resolved. + * + * @param the handled value class. + */ +public final class BinaryHandlerGenericValueClass extends AbstractBinaryHandlerReflective +{ + /////////////////////////////////////////////////////////////////////////// + // static methods // + /////////////////// + + /** + * Creates a new {@link BinaryHandlerGenericValueClass} for the passed value class. + * + * @param the handled value class. + * @param type the value class to be handled. + * @param typeName the type name to be used in the type dictionary. + * @param persistableFields the fields to be persisted. + * @param persisterFields the fields to be set to the persister. + * @param lengthResolver the field length resolver. + * @param eagerEvaluator the eager storing evaluator. + * @param fieldHandlerProvider the custom field handler provider. + * @param inliningResolver the resolver deciding which of the type's own fields are inlined. + * @param switchByteOrder whether persisted values use a non-native byte order. + * + * @return the newly created handler. + * + * @throws PersistenceExceptionTypeNotPersistable if the type has no suitable constructor. + */ + public static BinaryHandlerGenericValueClass New( + final Class type , + final String typeName , + final XGettingEnum persistableFields , + final XGettingEnum persisterFields , + final PersistenceFieldLengthResolver lengthResolver , + final PersistenceEagerStoringFieldEvaluator eagerEvaluator , + final BinaryFieldHandlerProvider fieldHandlerProvider, + final PersistenceValueInliningResolver inliningResolver , + final boolean switchByteOrder + ) + { + return new BinaryHandlerGenericValueClass<>( + type , + typeName , + persistableFields , + persisterFields , + lengthResolver , + eagerEvaluator , + fieldHandlerProvider, + inliningResolver , + switchByteOrder + ); + } + + /** + * Whether the type has a matching constructor that cannot be made accessible because its module + * does not open the package. Such types (e.g. JDK value types) must keep being handled + * reflectively, since there is no legal way to invoke their constructor. + *

+ * A type without any matching constructor is not reported here: that case is a genuine + * persistability problem and is reported by {@link #New} with a fitting explanation. + * + * @param type the value class to be tested. + * @param persistableFields the fields the constructor would have to accept. + * + * @return whether the type's constructor is inaccessible. + */ + public static boolean isConstructorModuleProtected( + final Class type , + final XGettingEnum persistableFields + ) + { + try + { + return !type.getDeclaredConstructor(toParameterTypes(persistableFields)).trySetAccessible(); + } + catch(final NoSuchMethodException e) + { + return false; + } + } + + static MethodHandle resolveConstructor( + final Class type , + final Class[] parameterTypes , + final XGettingEnum persistableFields + ) + { + final Constructor constructor; + try + { + constructor = type.getDeclaredConstructor(parameterTypes); + } + catch(final NoSuchMethodException e) + { + throw new PersistenceExceptionTypeNotPersistable(type, + new BinaryPersistenceException( + "Value class " + type.getName() + " cannot be handled generically: it has no constructor" + + " accepting its persistable fields in declaration order" + + toParameterListString(parameterTypes) + + ". Register a custom type handler for it.", + e + ) + ); + } + + try + { + /* Pre-adapted to a fixed (Object[])Object shape so creation can use invokeExact instead of + * invokeWithArguments, which would redo the argument conversion on every instance. + */ + return MethodHandles.lookup().unreflectConstructor(XReflect.setAccessible(constructor)) + .asSpreader(Object[].class, parameterTypes.length) + .asType(MethodType.methodType(Object.class, Object[].class)) + ; + } + catch(final IllegalAccessException e) + { + throw new PersistenceExceptionTypeNotPersistable(type, + new BinaryPersistenceException( + "Constructor of value class " + type.getName() + " is not accessible.", e + ) + ); + } + } + + private static String toParameterListString(final Class[] parameterTypes) + { + final StringBuilder sb = new StringBuilder(" ("); + for(int i = 0; i < parameterTypes.length; i++) + { + if(i != 0) + { + sb.append(", "); + } + sb.append(parameterTypes[i].getName()); + } + + return sb.append(')').toString(); + } + + + + /////////////////////////////////////////////////////////////////////////// + // instance fields // + //////////////////// + + private final MethodHandle constructor ; + + // settled at handler creation where it can be, otherwise on the first instance stored. See #store. + private final ValueClassConstructorContract constructorContract; + + /* Kept so a legacy handler can build its own readers over the persisted layout and still place the + * values where this type's constructor expects them. See BinaryLegacyTypeHandlerValueClass. + */ + private final XGettingEnum persistableFields; + + // all three arrays are parallel and indexed in persisted (= storing) member order. + private final BinaryValueReader[] readers ; + private final long[] readerOffsets ; + private final int[] argumentIndices; + + + + /////////////////////////////////////////////////////////////////////////// + // constructors // + ///////////////// + + BinaryHandlerGenericValueClass( + final Class type , + final String typeName , + final XGettingEnum persistableFields , + final XGettingEnum persisterFields , + final PersistenceFieldLengthResolver lengthResolver , + final PersistenceEagerStoringFieldEvaluator eagerEvaluator , + final BinaryFieldHandlerProvider fieldHandlerProvider, + final PersistenceValueInliningResolver inliningResolver , + final boolean switchByteOrder + ) + { + super( + type , + typeName , + persistableFields , + persisterFields , + lengthResolver , + eagerEvaluator , + fieldHandlerProvider, + inliningResolver , + switchByteOrder + ); + + final XGettingEnum storingMembers = + this.storingMembers() + ; + final int memberCount = storingMembers.intSize(); + + this.readers = new BinaryValueReader[memberCount]; + this.readerOffsets = new long[memberCount] ; + this.argumentIndices = new int[memberCount] ; + + // members are persisted in storing order (references first), but constructed in declaration order. + long offset = 0; + int i = 0; + for(final PersistenceTypeDefinitionMemberFieldReflective member : storingMembers) + { + /* An inlined member is read as the slot it is and constructed from it, which is the reader the + * construction-based path needs and the reason the fields of a value class can be inlined at all. + */ + this.readers[i] = member instanceof PersistenceTypeDefinitionMemberFieldValueStruct + ? BinaryValueStructFunctions.provideValueReader( + (PersistenceTypeDefinitionMemberFieldValueStruct)member + ) + : BinaryValueReader.provideReader(member.type()) + ; + this.readerOffsets[i] = offset; + this.argumentIndices[i] = indexOfField(persistableFields, member.field()); + offset += member.persistentMinimumLength(); + i++; + } + + validateNoCustomFieldHandlers(type, persistableFields, fieldHandlerProvider, switchByteOrder); + + this.persistableFields = persistableFields; + this.constructor = resolveConstructor(type, toParameterTypes(persistableFields), persistableFields); + + this.constructorContract = ValueClassConstructorContract.New(type, persistableFields, this.constructor); + } + + /** + * The number of arguments the constructor accepts, which is the length every argument array must have. + */ + final int argumentCount() + { + return this.persistableFields.intSize(); + } + + /** + * The constructor argument the passed field is, so a handler reading another layout can place a value + * where this type expects it. + * + * @return the argument index, or {@literal -1} if the field is none of the persistable ones. + */ + final int argumentIndex(final Field field) + { + int i = 0; + for(final Field persistableField : this.persistableFields) + { + if(persistableField.equals(field)) + { + return i; + } + i++; + } + + return -1; + } + + /** + * Constructs an instance from a complete argument array, which is the only way an identity-less + * instance comes into existence. Shared with the legacy handler, which fills the array from the + * persisted layout instead of the current one. + * + * @param arguments the constructor arguments, in declaration order. + * @param objectId the entity's object id, for the failure message. + * + * @return the newly created instance. + */ + final T createInstance(final Object[] arguments, final long objectId) + { + return this.createInstance(arguments, objectId, ""); + } + + /** + * @param defaultedMembers the current members the persisted layout did not carry, comma-separated; + * empty where all of them were carried. Naming them is the actionable half of + * a rejection: the constructor's own reason says what it refused, not that a + * defaulted value is why it was there to refuse. + */ + final T createInstance(final Object[] arguments, final long objectId, final String defaultedMembers) + { + try + { + // cast safety guaranteed by the constructor being the handled type's own. + @SuppressWarnings("unchecked") + final T instance = (T)this.constructor.invokeExact(arguments); + + return instance; + } + catch(final Throwable t) + { + // a failure of the JVM itself is none of this handler's business. + if(t instanceof Error) + { + throw (Error)t; + } + + /* A value class constructor validating its arguments can legitimately reject persisted + * state, e.g. when a field was added and the missing value is defaulted by a mapping. + */ + throw new BinaryPersistenceException( + "Failed to construct instance of value class " + this.type().getName() + + " for objectId " + objectId + + (defaultedMembers.isEmpty() + ? "" + : ", whose persisted layout did not carry " + defaultedMembers + + " so it was constructed with the default") + + ".", + t + ); + } + } + + /** + * Rejects a value class with a custom field handler registered for one of its fields. + *

+ * A custom field handler is a pair: a storer writing the field's own representation and a setter + * writing the value back into an instance. The second half is not applicable here, since + * a value instance is constructed from its field values rather than populated, so honoring the + * custom representation on the storing side while reading it back generically would silently + * produce wrong values. + */ + private static void validateNoCustomFieldHandlers( + final Class type , + final XGettingEnum persistableFields , + final BinaryFieldHandlerProvider fieldHandlerProvider, + final boolean switchByteOrder + ) + { + for(final Field field : persistableFields) + { + if(fieldHandlerProvider.lookupFieldStorer(field, false, switchByteOrder) == null + && fieldHandlerProvider.lookupFieldStorer(field, true, switchByteOrder) == null + && fieldHandlerProvider.lookupFieldSetter(field, switchByteOrder) == null + ) + { + continue; + } + + throw new PersistenceExceptionTypeNotPersistable(type, + new BinaryPersistenceException( + "Field " + field.getName() + " of value class " + type.getName() + " has a custom field" + + " handler registered, which cannot be applied to a type whose instances are created" + + " by their constructor. Register a custom type handler for " + type.getName() + + " instead." + ) + ); + } + } + + static Class[] toParameterTypes(final XGettingEnum persistableFields) + { + final Class[] parameterTypes = new Class[persistableFields.intSize()]; + + int i = 0; + for(final Field field : persistableFields) + { + parameterTypes[i++] = field.getType(); + } + + return parameterTypes; + } + + private static int indexOfField(final XGettingEnum persistableFields, final Field field) + { + int i = 0; + for(final Field persistableField : persistableFields) + { + if(persistableField.equals(field)) + { + return i; + } + i++; + } + + // cannot happen: members are derived from exactly these fields. + throw new BinaryPersistenceException("Unknown field " + field + " for value class handler."); + } + + + + /////////////////////////////////////////////////////////////////////////// + // methods // + //////////// + + /** + * Verifies the constructor contract before storing the first instance, then stores as the reflective + * base does. + *

+ * A no-op where the contract was already settled at handler creation, which is the usual case. It + * matters for what a probe cannot answer - a constructor that rejects the probing values, or a layout + * holding a reference - and then this is the first point an instance exists, still before any byte of + * it is written. See {@link ValueClassConstructorContract}. + */ + @Override + public void store( + final Binary data , + final T instance, + final long objectId, + final PersistenceStoreHandler handler + ) + { + this.constructorContract.validate(instance); + + super.store(data, instance, objectId, handler); + } + + @Override + public T create(final Binary data, final PersistenceLoadHandler handler) + { + final Object[] arguments = new Object[this.readers.length]; + for(int i = 0; i < this.readers.length; i++) + { + arguments[this.argumentIndices[i]] = this.readers[i].readValue(data, this.readerOffsets[i], handler); + } + + return this.createInstance(arguments, data.getBuildItemObjectId()); + } + + /** + * None, since {@link #updateState} has nothing to do: a value instance is constructed from the + * persisted data rather than populated afterwards, so no setter is ever invoked. + *

+ * Deriving them is not merely waste. A member whose type is itself a value class is set through a + * handle on the field rather than at its memory offset, because a value may be laid out inside its + * owner - and resolving that handle on a field declared in a value class is refused by the + * JVM outright, since such a field can never be written. The handler could then not even be + * created, so a value class holding another value class would be unpersistable. + */ + @Override + protected BinaryValueSetter[] deriveSetters() + { + return new BinaryValueSetter[0]; + } + + @Override + public void initializeState(final Binary data, final T instance, final PersistenceLoadHandler handler) + { + // value instances are complete when created, there is no state to initialize afterwards. + } + + @Override + public void updateState(final Binary data, final T instance, final PersistenceLoadHandler handler) + { + /* Updating means replacing the state of an existing instance, which is impossible for an + * immutable one. Since a value instance is always created from the very data that would be + * applied here, there is also nothing to do. + */ + } + +} diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryLegacyTypeHandlerCreator.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryLegacyTypeHandlerCreator.java index 8c4ae421..e28be117 100644 --- a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryLegacyTypeHandlerCreator.java +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryLegacyTypeHandlerCreator.java @@ -19,6 +19,7 @@ import java.lang.reflect.Field; +import org.eclipse.serializer.chars.VarString; import org.eclipse.serializer.util.X; import org.eclipse.serializer.collections.BulkList; import org.eclipse.serializer.collections.HashEnum; @@ -115,6 +116,20 @@ public final class Default // methods // //////////// + /** The members' simple names, so a failed construction can say which ones were defaulted. */ + private static String memberNames( + final XGettingEnum members + ) + { + final VarString vs = VarString.New(); + for(final PersistenceTypeDefinitionMember member : members) + { + vs.add(vs.isEmpty() ? "" : ", ").add(member.name()); + } + + return vs.toString(); + } + private static HashTable createBinaryOffsetMap( final XGettingEnum members ) @@ -199,6 +214,28 @@ protected PersistenceLegacyTypeHandler deriveCustomWrappingHandle ); } + if(currentTypeHandler instanceof BinaryHandlerGenericValueClass) + { + /* A value class is constructed from its members rather than populated, so it can read the + * persisted layout directly instead of having the bytes rewritten into the current one + * first - which is also the only way a member that was referenced and is now inlined can be + * read at all, since resolving it needs the load handler a rewrite does not have. + */ + @SuppressWarnings("unchecked") + final BinaryHandlerGenericValueClass valueClassHandler = + (BinaryHandlerGenericValueClass)currentTypeHandler + ; + + return BinaryLegacyTypeHandlerValueClass.New( + mappingResult.legacyTypeDefinition() , + valueClassHandler , + mappingResult.legacyToCurrentMembers() , + this.valueTranslatorProvider.provideRefactoringResolver(), + this.legacyTypeHandlingListener , + this.switchByteOrder + ); + } + final HashTable targetMemberOffsets = createBinaryOffsetMap( mappingResult.currentTypeHandler().instanceMembers() ); @@ -212,10 +249,11 @@ protected PersistenceLegacyTypeHandler deriveCustomWrappingHandle ); final BinaryLegacyTypeHandlerRerouting reroutingTypeHandler = BinaryLegacyTypeHandlerRerouting.New( - mappingResult.legacyTypeDefinition(), - currentTypeHandler , - translatorsWithTargetOffsets , - this.legacyTypeHandlingListener , + mappingResult.legacyTypeDefinition() , + currentTypeHandler , + translatorsWithTargetOffsets , + this.legacyTypeHandlingListener , + memberNames(mappingResult.newCurrentMembers()), this.switchByteOrder ); diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryLegacyTypeHandlerRerouting.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryLegacyTypeHandlerRerouting.java index 6466c83b..f7806d81 100644 --- a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryLegacyTypeHandlerRerouting.java +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryLegacyTypeHandlerRerouting.java @@ -21,11 +21,13 @@ import org.eclipse.serializer.collections.types.XGettingEnum; import org.eclipse.serializer.memory.XMemory; +import org.eclipse.serializer.persistence.binary.exceptions.BinaryPersistenceException; import org.eclipse.serializer.persistence.types.PersistenceLegacyTypeHandlingListener; import org.eclipse.serializer.persistence.types.PersistenceLoadHandler; import org.eclipse.serializer.persistence.types.PersistenceReferenceLoader; import org.eclipse.serializer.persistence.types.PersistenceTypeDefinition; import org.eclipse.serializer.persistence.types.PersistenceTypeHandler; +import org.eclipse.serializer.reflect.XReflect; import org.eclipse.serializer.typing.KeyValue; /** @@ -71,6 +73,37 @@ public static BinaryLegacyTypeHandlerRerouting New( final PersistenceLegacyTypeHandlingListener listener , final boolean switchByteOrder ) + { + return New( + typeDefinition , + typeHandler , + translatorsWithTargetOffsets, + listener , + "" , + switchByteOrder + ); + } + + /** + * Like {@link #New(PersistenceTypeDefinition, PersistenceTypeHandler, XGettingEnum, PersistenceLegacyTypeHandlingListener, boolean)}, + * additionally naming the current members the persisted layout does not carry. + *

+ * A handler that builds its instances from the persisted values instead of populating them - a value + * class handler - can legitimately reject what a defaulted member holds, and then that member is what + * the failure is about. It is named where it is known, which is here and not in the handler itself. + * + * @param defaultedMembers the current members that take their type's default, comma-separated; empty if none. + * + * @return the newly created legacy handler. + */ + public static BinaryLegacyTypeHandlerRerouting New( + final PersistenceTypeDefinition typeDefinition , + final PersistenceTypeHandler typeHandler , + final XGettingEnum> translatorsWithTargetOffsets, + final PersistenceLegacyTypeHandlingListener listener , + final String defaultedMembers , + final boolean switchByteOrder + ) { return new BinaryLegacyTypeHandlerRerouting<>( notNull(typeDefinition) , @@ -78,6 +111,7 @@ public static BinaryLegacyTypeHandlerRerouting New( toTranslators(translatorsWithTargetOffsets) , toTargetOffsets(translatorsWithTargetOffsets), mayNull(listener) , + notNull(defaultedMembers) , switchByteOrder ); } @@ -88,6 +122,7 @@ public static BinaryLegacyTypeHandlerRerouting New( //////////////////// private final BinaryReferenceTraverser[] newBinaryLayoutReferenceTraversers; + private final String defaultedMembers ; @@ -101,11 +136,14 @@ public static BinaryLegacyTypeHandlerRerouting New( final BinaryValueSetter[] valueTranslators, final long[] targetOffsets , final PersistenceLegacyTypeHandlingListener listener , + final String defaultedMembers, final boolean switchByteOrder ) { super(typeDefinition, typeHandler, valueTranslators, targetOffsets, listener, switchByteOrder); + this.defaultedMembers = defaultedMembers; + /* (01.01.2020 TM)NOTE: Bugfix: * Moved from AbstractBinaryLegacyTypeHandlerTranslating here as this is only correct for ~Rerouting * but incorrect for ~Reflective LegacyHandler @@ -127,8 +165,12 @@ public final void iterateLoadableReferences(final Binary rawData, final Persiste } @Override - protected T internalCreate(final Binary rawData, final PersistenceLoadHandler handler) + public void prepareLoadItem(final Binary rawData) { + /* The rewritten data is what #iterateLoadableReferences describes, so the rewrite has to + * happen before any reference is read out of the load item - which rules out doing it in + * #create, whose execution the loader may defer. + */ final long entityContentLength = this.typeHandler().membersPersistedLengthMaximum(); // kept and new header values @@ -153,11 +195,38 @@ protected T internalCreate(final Binary rawData, final PersistenceLoadHandler ha // registered here to ensure deallocating raw memory at the end of the building process. Neither sooner nor later. rawData.registerHelper(directByteBuffer, directByteBuffer); + } - // the current type handler can now create a new instance with correctly rearranged raw values - final T instance = this.typeHandler().create(rawData, handler); - - return instance; + @Override + protected T internalCreate(final Binary rawData, final PersistenceLoadHandler handler) + { + // the data has been rearranged by #prepareLoadItem, so the current type handler can read it directly. + if(this.defaultedMembers.isEmpty() || !XReflect.isValueClass(this.typeHandler().type())) + { + /* Every custom-wrapped handler reaches this class, enums included, and only one of them builds + * its instance from the persisted values. For all the others a failure of #create says nothing + * about a defaulted member, so naming one would assert a cause rather than report it - and the + * wrapping would hide the exception type a caller matches on. + */ + return this.typeHandler().create(rawData, handler); + } + + try + { + return this.typeHandler().create(rawData, handler); + } + catch(final RuntimeException e) + { + /* Naming them is the point: a handler constructing its instance from the persisted values can + * reject what a defaulted member holds, and the persisted layout not carrying it is the reason. + */ + throw new BinaryPersistenceException( + "Reading " + this.legacyTypeDefinition().toTypeIdentifier() + " as " + + this.typeHandler().toRuntimeTypeIdentifier() + " failed, whose persisted layout did not" + + " carry " + this.defaultedMembers + " so it was read with the default.", + e + ); + } } @Override diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryLegacyTypeHandlerValueClass.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryLegacyTypeHandlerValueClass.java new file mode 100644 index 00000000..d1a0ca53 --- /dev/null +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryLegacyTypeHandlerValueClass.java @@ -0,0 +1,450 @@ +package org.eclipse.serializer.persistence.binary.types; + +/*- + * #%L + * Eclipse Serializer Persistence Binary + * %% + * Copyright (C) 2023 - 2026 MicroStream Software + * %% + * This program and the accompanying materials are made + * available under the terms of the Eclipse Public License 2.0 + * which is available at https://www.eclipse.org/legal/epl-2.0/ + * + * SPDX-License-Identifier: EPL-2.0 + * #L% + */ + +import static org.eclipse.serializer.util.X.mayNull; +import static org.eclipse.serializer.util.X.notNull; + +import java.lang.reflect.Field; + +import org.eclipse.serializer.chars.VarString; +import org.eclipse.serializer.collections.BulkList; +import org.eclipse.serializer.collections.types.XGettingTable; +import org.eclipse.serializer.persistence.binary.exceptions.BinaryPersistenceException; +import org.eclipse.serializer.persistence.types.PersistenceLegacyTypeHandlingListener; +import org.eclipse.serializer.persistence.types.PersistenceLoadHandler; +import org.eclipse.serializer.persistence.types.PersistenceReferenceLoader; +import org.eclipse.serializer.persistence.types.PersistenceTypeDefinition; +import org.eclipse.serializer.persistence.types.PersistenceTypeDescriptionResolver; +import org.eclipse.serializer.persistence.types.PersistenceTypeDefinitionMember; +import org.eclipse.serializer.persistence.types.PersistenceTypeDefinitionMemberFieldReflective; +import org.eclipse.serializer.persistence.types.PersistenceTypeDefinitionMemberFieldValueStruct; +import org.eclipse.serializer.typing.KeyValue; +import org.eclipse.serializer.util.similarity.Similarity; + +/** + * Legacy type handler for a value class: it reads the persisted layout directly and passes the + * values to the current type's constructor, instead of rewriting the bytes into the current layout first. + *

+ * An identity-less instance is constructed rather than populated, so it never needed a rewritten buffer - + * the buffer only existed because {@link BinaryHandlerGenericValueClass#create} reads the current layout. + * Dropping it removes what the rewriting path structurally cannot do: a member that was referenced and is + * now inlined has to be resolved, and a rewrite runs in + * {@link #prepareLoadItem}, before any reference is loaded and with no load handler in reach. Reading the + * persisted layout at creation time has both. + *

+ * What this changes per member, compared to {@link BinaryLegacyTypeHandlerRerouting}: + *

+ * Reference traversal therefore has to describe the persisted layout, as it does for the reflective + * legacy path and unlike the rerouting one - which is also what gets a now-inlined referent loaded in time + * to be resolved. + * + * @param the value class produced by this handler. + * + * @see BinaryHandlerGenericValueClass + * @see BinaryLegacyTypeHandlerRerouting + */ +public final class BinaryLegacyTypeHandlerValueClass +extends AbstractBinaryLegacyTypeHandlerTranslating +{ + /////////////////////////////////////////////////////////////////////////// + // static methods // + /////////////////// + + /** + * Creates a legacy handler reading the persisted layout of a value class directly. + * + * @param typeDefinition the legacy type definition describing the persisted layout. + * @param typeHandler the current value class handler. + * @param legacyToCurrent the member mapping, persisted member to current member. + * @param listener optional listener invoked on each legacy creation, may be {@code null}. + * @param switchByteOrder whether persisted values use a non-native byte order. + * + * @param the value class produced by the handler. + * + * @return the newly created legacy handler. + */ + public static BinaryLegacyTypeHandlerValueClass New( + final PersistenceTypeDefinition typeDefinition , + final BinaryHandlerGenericValueClass typeHandler , + final XGettingTable> + legacyToCurrent, + final PersistenceTypeDescriptionResolver refactoringResolver, + final PersistenceLegacyTypeHandlingListener listener , + final boolean switchByteOrder + ) + { + return new BinaryLegacyTypeHandlerValueClass<>( + notNull(typeDefinition) , + notNull(typeHandler) , + notNull(legacyToCurrent) , + mayNull(refactoringResolver), + mayNull(listener) , + switchByteOrder + ); + } + + /** + * Derives one argument source per persisted member: the reader for its own type, the offset it sits at + * in the persisted layout, and the constructor argument it belongs to. + */ + private static BulkList deriveSources( + final PersistenceTypeDefinition typeDefinition , + final BinaryHandlerGenericValueClass typeHandler , + final XGettingTable> + legacyToCurrent, + final PersistenceTypeDescriptionResolver refactoringResolver + ) + { + final BulkList sources = BulkList.New(); + + long offset = 0; + for(final PersistenceTypeDefinitionMember source : typeDefinition.instanceMembers()) + { + if(!source.isFixedLength()) + { + throw new BinaryPersistenceException( + "Value class " + typeHandler.type().getName() + " cannot be read from the persisted" + + " layout: member " + source.identifier() + " has no fixed length." + ); + } + + final PersistenceTypeDefinitionMember target = resolveTarget(legacyToCurrent, source); + if(target != null) + { + sources.add(new ArgumentSource( + deriveReader(source, target, typeHandler, refactoringResolver), + offset, + argumentIndex(typeHandler, target) + )); + } + + // a member the current type no longer has is simply not read + offset += source.persistentMinimumLength(); + } + + return sources; + } + + private static PersistenceTypeDefinitionMember resolveTarget( + final XGettingTable> + legacyToCurrent, + final PersistenceTypeDefinitionMember source + ) + { + final Similarity match = legacyToCurrent.get(source); + + return match == null ? null : match.targetElement(); + } + + private static int argumentIndex( + final BinaryHandlerGenericValueClass typeHandler, + final PersistenceTypeDefinitionMember target + ) + { + if(!(target instanceof PersistenceTypeDefinitionMemberFieldReflective)) + { + throw new BinaryPersistenceException( + "Value class " + typeHandler.type().getName() + " cannot take " + target.identifier() + + ": only a field of the type itself is a constructor argument." + ); + } + + final Field field = ((PersistenceTypeDefinitionMemberFieldReflective)target).field(); + final int index = field == null ? -1 : typeHandler.argumentIndex(field); + if(index < 0) + { + throw new BinaryPersistenceException( + "Member " + target.identifier() + " is none of the persistable fields of value class " + + typeHandler.type().getName() + "." + ); + } + + return index; + } + + /** + * The reader for one persisted member, chosen by what the member was and what it is. + */ + private static BinaryValueReader deriveReader( + final PersistenceTypeDefinitionMember source , + final PersistenceTypeDefinitionMember target , + final BinaryHandlerGenericValueClass typeHandler, + final PersistenceTypeDescriptionResolver refactoringResolver + ) + { + final boolean sourceIsStruct = source instanceof PersistenceTypeDefinitionMemberFieldValueStruct; + final boolean targetIsStruct = target instanceof PersistenceTypeDefinitionMemberFieldValueStruct; + + if(sourceIsStruct) + { + if(!targetIsStruct) + { + /* The slot holds a whole layout and the current member takes one value. Reading it would + * have to decide which of the layout's values that is, which nothing states. + */ + throw new BinaryPersistenceException( + "Inlined member " + source.identifier() + " of value class " + + typeHandler.type().getName() + " is no longer inlined. Reading an inlined layout as a" + + " single value is not supported." + ); + } + + final PersistenceTypeDefinitionMemberFieldValueStruct sourceStruct = + (PersistenceTypeDefinitionMemberFieldValueStruct)source + ; + final PersistenceTypeDefinitionMemberFieldValueStruct targetStruct = + (PersistenceTypeDefinitionMemberFieldValueStruct)target + ; + + /* A changed layout is paired member for member the same way the owner's path pairs it, so the + * same renames are carried and the same shapes refused. An unchanged one skips that and is read + * by its offsets directly. + */ + return sourceStruct.equalsLayout(targetStruct) + ? BinaryValueStructFunctions.provideValueReader(targetStruct) + : BinaryValueStructFunctions.provideEvolvingValueReader( + sourceStruct, targetStruct, refactoringResolver + ) + ; + } + + if(targetIsStruct) + { + /* The case the rewriting path cannot do: the persisted form is an object id and the current + * member is a slot. Constructed from its values, the instance is simply the resolved referent - + * no slot has to be written for it at all. + */ + if(!source.isReference()) + { + throw new BinaryPersistenceException( + "Member " + source.identifier() + " of value class " + typeHandler.type().getName() + + " was persisted as a single value and is now an inlined layout, which cannot be read" + + " from it." + ); + } + + return BinaryValueReader.provideReader(Object.class); + } + + // read as what was persisted, so a widening to the current type is the constructor's own conversion + return BinaryValueReader.provideReader(persistedType(source)); + } + + /** + * The type the persisted member is read as. A reference is read as an object id and resolved, so any + * non-primitive stands for it - including one whose class no longer exists. + */ + private static Class persistedType(final PersistenceTypeDefinitionMember source) + { + if(source.isReference()) + { + return Object.class; + } + + final Class type = source.type(); + if(type == null || !type.isPrimitive()) + { + throw new BinaryPersistenceException( + "Member " + source.identifier() + " is neither a reference nor a primitive, so it cannot be" + + " read out of the persisted layout of a value class." + ); + } + + return type; + } + + /** + * The argument array a construction starts from, so a member the persisted layout does not carry takes + * its type's default. + */ + private static Object[] deriveDefaults(final BinaryHandlerGenericValueClass typeHandler) + { + final Object[] defaults = new Object[typeHandler.argumentCount()]; + + for(final PersistenceTypeDefinitionMember member : typeHandler.instanceMembers()) + { + if(!(member instanceof PersistenceTypeDefinitionMemberFieldReflective)) + { + continue; + } + + final Field field = ((PersistenceTypeDefinitionMemberFieldReflective)member).field(); + if(field == null) + { + continue; + } + + final int index = typeHandler.argumentIndex(field); + if(index >= 0) + { + defaults[index] = BinaryValueStructFunctions.defaultValue(field.getType()); + } + } + + return defaults; + } + + + + /////////////////////////////////////////////////////////////////////////// + // member types // + ///////////////// + + /** + * The current members no persisted member is read into, comma-separated. They take their type's + * default, which is what a constructor rejecting one of them is actually rejecting. + */ + private static String deriveDefaultedMembers( + final BinaryHandlerGenericValueClass typeHandler, + final BulkList sources + ) + { + final VarString vs = VarString.New(); + + for(final PersistenceTypeDefinitionMember member : typeHandler.instanceMembers()) + { + if(!(member instanceof PersistenceTypeDefinitionMemberFieldReflective)) + { + continue; + } + + final Field field = ((PersistenceTypeDefinitionMemberFieldReflective)member).field(); + final int index = field == null ? -1 : typeHandler.argumentIndex(field); + if(index < 0 || sources.containsSearched(source -> source.index == index)) + { + continue; + } + + vs.add(vs.isEmpty() ? "" : ", ").add(member.name()); + } + + return vs.toString(); + } + + /** One persisted member as a constructor argument: how to read it, where it sits, where it goes. */ + private static final class ArgumentSource + { + final BinaryValueReader reader; + final long offset; + final int index ; + + ArgumentSource(final BinaryValueReader reader, final long offset, final int index) + { + super(); + this.reader = reader; + this.offset = offset; + this.index = index ; + } + } + + + + /////////////////////////////////////////////////////////////////////////// + // instance fields // + //////////////////// + + private final BinaryHandlerGenericValueClass valueClassHandler ; + private final BinaryReferenceTraverser[] oldBinaryLayoutReferenceTraversers; + private final ArgumentSource[] sources ; + private final String defaultedMembers ; + private final Object[] defaults ; + + + + /////////////////////////////////////////////////////////////////////////// + // constructors // + ///////////////// + + BinaryLegacyTypeHandlerValueClass( + final PersistenceTypeDefinition typeDefinition , + final BinaryHandlerGenericValueClass typeHandler , + final XGettingTable> + legacyToCurrent, + final PersistenceTypeDescriptionResolver refactoringResolver, + final PersistenceLegacyTypeHandlingListener listener , + final boolean switchByteOrder + ) + { + /* No value translators: nothing is rewritten, so there is no target layout to write into. The base + * class is used for the wrapper delegation it provides and for #internalCreate, which is the only + * place this handler does any work. + */ + super(typeDefinition, typeHandler, new BinaryValueSetter[0], new long[0], listener, switchByteOrder); + + final BulkList sources = deriveSources(typeDefinition, typeHandler, legacyToCurrent, refactoringResolver); + + this.valueClassHandler = typeHandler; + this.defaults = deriveDefaults(typeHandler); + this.defaultedMembers = deriveDefaultedMembers(typeHandler, sources); + this.sources = sources.toArray(ArgumentSource.class); + + // the persisted layout, not the current one: nothing rewrote the data this handler reads + this.oldBinaryLayoutReferenceTraversers = deriveReferenceTraversers(typeDefinition, switchByteOrder); + } + + + + /////////////////////////////////////////////////////////////////////////// + // methods // + //////////// + + @Override + public void iterateLoadableReferences(final Binary rawData, final PersistenceReferenceLoader iterator) + { + rawData.iterateReferences(this.oldBinaryLayoutReferenceTraversers, iterator); + } + + /** + * None. The instance is complete the moment it exists, and a value class has no setters to apply - + * which is why it is constructed rather than populated in the first place. Delegating here would hand + * the current handler data in the persisted layout, which is not the layout it reads. + */ + @Override + public void updateState(final Binary rawData, final T instance, final PersistenceLoadHandler handler) + { + // no-op, see above + } + + @Override + public void complete(final Binary rawData, final T instance, final PersistenceLoadHandler handler) + { + // no-op for a reflective handler, and the same reasoning as #updateState applies to the data + } + + @Override + protected T internalCreate(final Binary rawData, final PersistenceLoadHandler handler) + { + final Object[] arguments = this.defaults.clone(); + + for(final ArgumentSource source : this.sources) + { + arguments[source.index] = source.reader.readValue(rawData, source.offset, handler); + } + + return this.valueClassHandler.createInstance( + arguments, rawData.getBuildItemObjectId(), this.defaultedMembers + ); + } + +} diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryLoadItem.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryLoadItem.java index 58d596f9..97abf382 100644 --- a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryLoadItem.java +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryLoadItem.java @@ -42,6 +42,12 @@ public class BinaryLoadItem extends Binary PersistenceTypeHandler handler; Object existingInstance, createdInstance; + + /* Some instances are created from their fully resolved state instead of being created blank and + * populated afterwards (value instances foremost), so their creation must be deferred until their + * references are resolvable. See BinaryLoader.Default#ensureDeferredInstance. + */ + boolean deferredCreationPending, deferredCreationActive; BinaryLoadItem next, link; diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryLoader.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryLoader.java index 6c6fc40f..96ff6a41 100644 --- a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryLoader.java +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryLoader.java @@ -29,6 +29,7 @@ import org.eclipse.serializer.persistence.exceptions.PersistenceExceptionConsistencyObject; import org.eclipse.serializer.persistence.exceptions.PersistenceExceptionTypeHandlerConsistencyUnhandledTypeId; import org.eclipse.serializer.persistence.types.*; +import org.eclipse.serializer.reflect.XReflect; import org.eclipse.serializer.util.logging.Logging; import org.slf4j.Logger; @@ -257,9 +258,23 @@ protected PersistenceTypeHandler lookupTypeHandler(final long ti private void createBuildItem(final BinaryLoadItem loadItem) { loadItem.handler = this.lookupTypeHandler(loadItem.getBuildItemTypeId()); + + // must happen before the item's references are read, see PersistenceTypeHandler#prepareLoadItem. + loadItem.handler.prepareLoadItem(loadItem); + if((loadItem.existingInstance = this.objectRegistry.lookupObject(loadItem.getBuildItemObjectId())) == null) { - loadItem.createdInstance = loadItem.handler.create(loadItem, this); + if(loadItem.handler.isCreationDeferred()) + { + /* Such an instance is created from its resolved references, which cannot be + * resolved yet at this point. See #ensureDeferredInstance. + */ + loadItem.deferredCreationPending = true; + } + else + { + loadItem.createdInstance = loadItem.handler.create(loadItem, this); + } } // register build item @@ -306,6 +321,10 @@ private Object getEffectiveInstance(final BinaryLoadItem entry) { return entry.existingInstance; } + if(entry.deferredCreationPending) + { + this.ensureDeferredInstance(entry); + } if(entry.createdInstance == null) { return null; @@ -329,6 +348,42 @@ private Object getEffectiveInstance(final BinaryLoadItem entry) return entry.existingInstance = entry.createdInstance; } + /** + * Creates a pending deferred instance, on demand and hence in dependency order: creating it + * resolves its references, which creates every deferred instance it references first. + *

+ * Non-deferred instances referenced by it are already created at this point, but only a + * handler that creates complete instances guarantees usable state; a blank one suffices when + * only the reference itself is needed. + *

+ * A deferred instance can only ever reference deferred instances that existed before it, so + * the recursion is finite for consistent data. An inconsistent cycle is detected and reported + * instead of overflowing the stack. + * + * @param entry the build item whose deferred instance is to be created. + */ + private void ensureDeferredInstance(final BinaryLoadItem entry) + { + if(entry.deferredCreationActive) + { + throw new BinaryPersistenceException( + "Cyclic reference detected while deferring creation for objectId " + + entry.getBuildItemObjectId() + " of type " + entry.handler.typeName() + "." + ); + } + + entry.deferredCreationActive = true; + try + { + entry.createdInstance = entry.handler.create(entry, this); + } + finally + { + entry.deferredCreationActive = false; + entry.deferredCreationPending = false; + } + } + protected void loadReferences(final BinaryLoadItem entry) { /* @@ -373,6 +428,15 @@ public final void registerDefaultRootRefactoring(final Object rootInstance, fina private void registerRoot(final Object rootInstance, final long rootObjectId) { + // value instances may not be published to the registry, see #isValueClassType. + if(XReflect.isValueInstance(rootInstance)) + { + /* A root is reached by its identifier, not by a registry lookup, so skipping this costs + * nothing. The registration would throw IdentityException and leave the storage unopenable. + */ + return; + } + // root instances are global, so it is appropriate and required to register it globally right away this.objectRegistry.registerObject(rootObjectId, rootInstance); } @@ -424,6 +488,12 @@ private void registerBuiltInstances() continue; } + // value items may not be published to the registry, see #isValueClassType. + if(entry.handler.isValueClassType()) + { + continue; + } + final Object registered = this.objectRegistry.optionalRegisterObject( entry.getBuildItemObjectId(), entry.createdInstance @@ -655,7 +725,17 @@ private boolean isUnrequiredReferenceLazy(final long objectId) this.putSkipItem(objectId, instance); return true; } - + + /* JDK constants are not registry-resident if they are value instances, but their ids are + * still reserved and never resolvable as entities, so they must be resolved arithmetically. + */ + final Object javaConstant; + if((javaConstant = Persistence.resolveJavaConstantInstance(objectId)) != null) + { + this.putSkipItem(objectId, javaConstant); + return true; + } + // reaching here means the reference is really required to be resolved (loaded) return false; } diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryPersistence.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryPersistence.java index 1a9bc2a1..bb357a21 100644 --- a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryPersistence.java +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryPersistence.java @@ -64,11 +64,20 @@ import org.eclipse.serializer.persistence.binary.java.sql.BinaryHandlerSqlTime; import org.eclipse.serializer.persistence.binary.java.sql.BinaryHandlerSqlTimestamp; import org.eclipse.serializer.persistence.binary.java.sql.BinaryLegacyTypeHandlerSqlTimestamp; +import org.eclipse.serializer.persistence.binary.java.time.BinaryHandlerDuration; +import org.eclipse.serializer.persistence.binary.java.time.BinaryHandlerInstant; import org.eclipse.serializer.persistence.binary.java.time.BinaryHandlerLocalDate; +import org.eclipse.serializer.persistence.binary.java.time.BinaryHandlerLocalDateTime; +import org.eclipse.serializer.persistence.binary.java.time.BinaryHandlerLocalTime; import org.eclipse.serializer.persistence.binary.java.time.BinaryHandlerMonthDay; +import org.eclipse.serializer.persistence.binary.java.time.BinaryHandlerOffsetDateTime; +import org.eclipse.serializer.persistence.binary.java.time.BinaryHandlerOffsetTime; import org.eclipse.serializer.persistence.binary.java.time.BinaryHandlerPeriod; +import org.eclipse.serializer.persistence.binary.java.time.BinaryHandlerYear; import org.eclipse.serializer.persistence.binary.java.time.BinaryHandlerYearMonth; +import org.eclipse.serializer.persistence.binary.java.time.BinaryHandlerZonedDateTime; import org.eclipse.serializer.persistence.binary.java.time.BinaryHandlerZoneOffset; +import org.eclipse.serializer.persistence.binary.java.time.BinaryHandlerZoneRegion; import org.eclipse.serializer.persistence.binary.java.util.BinaryHandlerArrayDeque; import org.eclipse.serializer.persistence.binary.java.util.BinaryHandlerArrayList; import org.eclipse.serializer.persistence.binary.java.util.BinaryHandlerBitSet; @@ -86,6 +95,7 @@ import org.eclipse.serializer.persistence.binary.java.util.BinaryHandlerLinkedHashSet; import org.eclipse.serializer.persistence.binary.java.util.BinaryHandlerLinkedList; import org.eclipse.serializer.persistence.binary.java.util.BinaryHandlerLocale; +import org.eclipse.serializer.persistence.binary.java.util.BinaryHandlerOptional; import org.eclipse.serializer.persistence.binary.java.util.BinaryHandlerOptionalDouble; import org.eclipse.serializer.persistence.binary.java.util.BinaryHandlerOptionalInt; import org.eclipse.serializer.persistence.binary.java.util.BinaryHandlerOptionalLong; @@ -330,6 +340,15 @@ static final void initializeNativeTypeId( BinaryHandlerYearMonth.New(), BinaryHandlerMonthDay.New(), + /* Self-contained java.time types whose instances, as value classes, cannot be populated + * after creation. The reference-holding java.time types are registered with the + * referencing-type handlers below. + */ + BinaryHandlerLocalTime.New(), + BinaryHandlerYear.New() , + BinaryHandlerInstant.New() , + BinaryHandlerDuration.New() , + /* (12.11.2019 TM)NOTE: * One might think that "empty" implementations of a collection interface would have no fields, anyway. * But no, those classes extends 5 other classes, some of which bring along several times @@ -420,8 +439,20 @@ static final void initializeNativeTypeId( BinaryHandlerLazyDefault.New(), - // the way Optional is implemented, only a generically (low-level) working handler can handle it correctly - typeHandlerCreator.createTypeHandlerGeneric(Optional.class) + /* Optional holds its content in a single reference. A generic handler would create it empty + * and populate it afterwards, which does not take on a value class. + */ + BinaryHandlerOptional.New(), + + /* The java.time types holding their parts in references. Not value-type handlers: their + * creation resolves references, which a plugin reusing the value-type handlers (e.g. the + * REST viewer) cannot provide. + */ + BinaryHandlerLocalDateTime.New() , + BinaryHandlerOffsetTime.New() , + BinaryHandlerOffsetDateTime.New(), + BinaryHandlerZonedDateTime.New() , + BinaryHandlerZoneRegion.New() ); return nativeHandlers; diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryPersistenceFoundation.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryPersistenceFoundation.java index b29054f2..7244250f 100644 --- a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryPersistenceFoundation.java +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryPersistenceFoundation.java @@ -314,6 +314,7 @@ protected PersistenceTypeHandlerCreator ensureTypeHandlerCreator() this.getInstantiatorProvider(), this.referenceTypeHandlerManager(), this.getFieldHandlerProvider(), + this.getValueInliningResolver(), this.isByteOrderMismatch() ); } @@ -395,6 +396,7 @@ protected BinaryValueTranslatorProvider ensureValueTranslatorProvider() this.getCustomTranslatorLookup(), this.getTranslatorKeyBuilders(), this.getValueTranslatorMappingProvider(), + this.getTypeDescriptionResolverProvider(), this.isByteOrderMismatch() ); } diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryReferenceTraverser.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryReferenceTraverser.java index fd831157..f28f9d46 100644 --- a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryReferenceTraverser.java +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryReferenceTraverser.java @@ -1163,9 +1163,14 @@ else if(member.isReference()) } else { - // only remaining option is fixed length primitive + /* Everything left is fixed length and holds no reference to traverse: a primitive, or an + * inlined value slot, which is a composite of primitives plus a null marker and reports + * its whole length here (references inside one are refused where such a member is + * created, precisely so that this skip stays correct). Fixed length means minimum and + * maximum are the same, so the minimum is the length to skip. + */ this.finishReferenceRange(); - this.skipLength += member.persistentMinimumLength(); // fixed length primitive ensured above + this.skipLength += member.persistentMinimumLength(); } } diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryStorer.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryStorer.java index 8cbf472c..add03780 100644 --- a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryStorer.java +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryStorer.java @@ -552,6 +552,28 @@ public final long applyEager(final T instance) return this.registerGuaranteed(instance); } + @Override + public long applyKnown(final T instance, final long knownObjectId) + { + /* + * An eager store must reach every instance, so what the caller knows about a previous store + * is irrelevant to it - and the ids it assigns are what the caller has to remember instead. + */ + if(instance == null || Swizzling.isNotFoundId(knownObjectId) || this.isEagerStoring()) + { + return this.apply(instance); + } + + /* + * Referenced without being stored in this commit, which is exactly the state ensureObjectId + * reports for an instance the object registry already knows - so it is registered the same + * way, and gets the same pin, trusted-id record and target-side validation. + */ + this.registerSkippedOptional(knownObjectId, instance, null); + + return knownObjectId; + } + @Override public long applyEager(final T instance, final PersistenceTypeHandler localTypeHandler) { @@ -705,6 +727,13 @@ public void iterateMergeableEntries(final PersistenceAcceptor iterator) continue; } + // value items may not be merged into the registry, see #isValueClassType. + // the handler is non-null here, a null one being the skip criterion checked above. + if(e.typeHandler.isValueClassType()) + { + continue; + } + // mergeable entry iterator.accept(e.oid, e.instance); } @@ -1614,6 +1643,12 @@ public class Eager extends Default // methods // //////////// + @Override + public final boolean isEagerStoring() + { + return true; + } + @Override public final long apply(final T instance) { diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryTypeHandlerCreator.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryTypeHandlerCreator.java index 51b5e80e..44fcabe8 100644 --- a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryTypeHandlerCreator.java +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryTypeHandlerCreator.java @@ -48,7 +48,11 @@ import org.eclipse.serializer.persistence.types.PersistenceTypeHandlerManager; import org.eclipse.serializer.persistence.types.PersistenceTypeInstantiatorProvider; import org.eclipse.serializer.persistence.types.PersistenceTypeResolver; +import org.eclipse.serializer.persistence.types.PersistenceValueInliningResolver; import org.eclipse.serializer.reference.Referencing; +import org.eclipse.serializer.reflect.XReflect; +import org.eclipse.serializer.util.logging.Logging; +import org.slf4j.Logger; public interface BinaryTypeHandlerCreator extends PersistenceTypeHandlerCreator @@ -90,6 +94,7 @@ public static BinaryTypeHandlerCreator New( final PersistenceTypeInstantiatorProvider instantiatorProvider , final Referencing> typeHandlerManager , final BinaryFieldHandlerProvider fieldHandlerProvider , + final PersistenceValueInliningResolver inliningResolver , final boolean switchByteOrder ) { @@ -101,6 +106,7 @@ public static BinaryTypeHandlerCreator New( notNull(instantiatorProvider) , notNull(typeHandlerManager) , notNull(fieldHandlerProvider) , + notNull(inliningResolver) , switchByteOrder ); } @@ -113,11 +119,14 @@ public class Default // instance fields // //////////////////// + private final static Logger logger = Logging.getLogger(BinaryTypeHandlerCreator.class); + final PersistenceTypeInstantiatorProvider instantiatorProvider ; final Referencing> typeHandlerManager ; final boolean switchByteOrder ; EntityTypeHandlerManager entityTypeHandlerManager; final private BinaryFieldHandlerProvider fieldHandlerProvider ; + final private PersistenceValueInliningResolver inliningResolver ; /////////////////////////////////////////////////////////////////////////// @@ -132,6 +141,7 @@ public class Default final PersistenceTypeInstantiatorProvider instantiatorProvider , final Referencing> typeHandlerManager , final BinaryFieldHandlerProvider fieldHandlerProvider , + final PersistenceValueInliningResolver inliningResolver , final boolean switchByteOrder ) { @@ -139,6 +149,7 @@ public class Default this.instantiatorProvider = instantiatorProvider; this.typeHandlerManager = typeHandlerManager ; this.fieldHandlerProvider = fieldHandlerProvider; + this.inliningResolver = inliningResolver ; this.switchByteOrder = switchByteOrder ; } @@ -242,6 +253,40 @@ protected PersistenceTypeHandler internalCreateTypeHandlerGeneric * */ + /* Value instances cannot be allocated blank and populated afterwards, so they need a + * handler constructing them from their persisted state. That includes stateless ones, + * hence the check before the emptiness shortcut below. + */ + if(XReflect.isValueClass(type)) + { + if(!BinaryHandlerGenericValueClass.isConstructorModuleProtected(type, persistableFields)) + { + return BinaryHandlerGenericValueClass.New( + type, + this.deriveTypeName(type), + persistableFields, + persisterFields, + this.lengthResolver(), + this.eagerStoringFieldEvaluator(), + this.fieldHandlerProvider , + this.inliningResolver , + this.switchByteOrder + ); + } + + /* Reported because such a type keeps being allocated blank and populated, which an + * immutable instance does not support and only works as long as the memory accessor + * tolerates it. The remedy is a custom handler using the type's public API. + */ + logger.warn( + "Value class {} is handled reflectively: its constructor cannot be made accessible" + + " because its module does not open the package. Its instances are created blank and" + + " populated, which an immutable instance does not support.", + type.getName() + ); + } + + if(persistableFields.isEmpty()) { return this.internalCreateTypeHandlerGenericStateless(type); @@ -257,6 +302,7 @@ protected PersistenceTypeHandler internalCreateTypeHandlerGeneric this.eagerStoringFieldEvaluator(), this.instantiatorProvider.provideTypeInstantiator(type), this.fieldHandlerProvider , + this.inliningResolver , this.switchByteOrder ); } @@ -340,7 +386,7 @@ final > PersistenceTypeHandler createEnumHandler this.switchByteOrder ); } - + @Override protected PersistenceTypeHandler internalCreateTypeHandlerEntity( final Class type diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryValueHandleFunctions.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryValueHandleFunctions.java new file mode 100644 index 00000000..1732c17e --- /dev/null +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryValueHandleFunctions.java @@ -0,0 +1,535 @@ +package org.eclipse.serializer.persistence.binary.types; + +/*- + * #%L + * Eclipse Serializer Persistence Binary + * %% + * Copyright (C) 2023 - 2026 MicroStream Software + * %% + * This program and the accompanying materials are made + * available under the terms of the Eclipse Public License 2.0 + * which is available at https://www.eclipse.org/legal/epl-2.0/ + * + * SPDX-License-Identifier: EPL-2.0 + * #L% + */ + +import java.lang.invoke.MethodHandle; +import java.lang.invoke.MethodHandles; +import java.lang.invoke.MethodType; +import java.lang.reflect.Field; +import java.lang.reflect.InvocationTargetException; +import java.lang.reflect.Method; + +import org.eclipse.serializer.memory.XMemory; +import org.eclipse.serializer.persistence.binary.exceptions.BinaryPersistenceException; +import org.eclipse.serializer.reflect.XReflect; + +/** + * Storers and setters that reach a field through a handle rather than through its memory offset. + *

+ * A field whose type is a value class may be laid out inside its owner rather than as a reference to a + * separate instance. There is then no object reference at the field's offset, and reading one from there is + * not merely inaccurate: depending on what the embedded content happens to be, it either yields the wrong + * object or dereferences a value as if it were a pointer, which ends the process. Nothing in the public API + * reports which fields are laid out that way, so every field of a value class is reached this way, whether + * or not this JVM embedded that particular one. + *

+ * Both directions go through the accessible field's {@link MethodHandle}, which the JVM resolves against + * the field's actual layout, which writes final fields (unlike a {@link java.lang.invoke.VarHandle}), and + * whose signature can be pre-adapted so every invocation is exact. The one exception is a record's fields, + * which accept no handle write at all and are written through the same internal accessor that populates + * their other fields, made layout-aware. Offsets stay in use for every field that is not value-typed. + * + * @see BinaryValueFunctions + */ +public final class BinaryValueHandleFunctions +{ + /////////////////////////////////////////////////////////////////////////// + // static methods // + /////////////////// + + /** + * Resolves the reader for the passed field, resolved by the JVM against the field's actual layout. + * + * @param field the field to read; must not be {@code null}. + * + * @return the reader for that field. + * + * @throws BinaryPersistenceException if the declaring class does not grant the required access. + */ + public static FieldReader provideFieldReader(final Field field) + { + final MethodHandle getter; + try + { + // pre-adapted so every read is an exact invocation instead of an adapted one. + getter = MethodHandles.lookup() + .unreflectGetter(XReflect.setAccessible(field)) + .asType(MethodType.methodType(Object.class, Object.class)) + ; + } + catch(final IllegalAccessException e) + { + throw new BinaryPersistenceException( + "Cannot read field " + field + ". Its module must open the declaring package.", e + ); + } + + return owner -> + { + try + { + return getter.invokeExact(owner); + } + catch(final RuntimeException | Error e) + { + throw e; + } + catch(final Throwable t) + { + throw new BinaryPersistenceException("Failed to read field " + field + ".", t); + } + }; + } + + /** + * Resolves the writer for the passed field, capable of writing a final field and resolved by the + * JVM against the field's actual layout. + * + * @param field the field to write; must not be {@code null}. + * + * @return the writer for that field. + * + * @throws BinaryPersistenceException if the declaring class does not grant the required access. + */ + public static FieldWriter provideFieldWriter(final Field field) + { + if(field.getDeclaringClass().isRecord()) + { + /* A record's fields refuse every handle or reflective write, however accessible. They are + * populated through the internal accessor like every other of the record's fields, with the + * one difference that a value field may be laid out flat inside the record, where a plain + * reference write would destroy the owner's memory instead of setting the field. + */ + return InternalUnsafeFlatAccess.provideFieldWriter(field); + } + + final MethodHandle setter; + try + { + // the accessible flag is what permits writing a final field, unlike for a VarHandle. + setter = MethodHandles.lookup() + .unreflectSetter(XReflect.setAccessible(field)) + .asType(MethodType.methodType(void.class, Object.class, Object.class)) + ; + } + catch(final IllegalAccessException e) + { + throw new BinaryPersistenceException( + "Cannot write field " + field + ". Its module must open the declaring package.", e + ); + } + + return (owner, value) -> + { + try + { + setter.invokeExact(owner, value); + } + catch(final RuntimeException | Error e) + { + throw e; + } + catch(final Throwable t) + { + throw new BinaryPersistenceException("Failed to write field " + field + ".", t); + } + }; + } + + /** + * Creates the storer writing the object id of a value class field's value. + * + * @param field the field to read; must not be {@code null}. + * @param isEager whether the referenced value is stored eagerly. + * @param switchByteOrder whether the persistent form uses the reversed byte order. + * + * @return the storer for the field. + */ + public static BinaryValueStorer provideReferenceStorer( + final Field field , + final boolean isEager , + final boolean switchByteOrder + ) + { + final FieldReader reader = provideFieldReader(field); + + return (source, sourceOffset, targetAddress, persister) -> + { + final Object value = reader.readValue(source); + final long objectId = isEager + ? persister.applyEager(value) + : persister.apply(value) + ; + + XMemory.set_long(targetAddress, switchByteOrder ? Long.reverseBytes(objectId) : objectId); + + return targetAddress + Binary.objectIdByteLength(); + }; + } + + /** + * Creates the setter resolving an object id into a value class field. + * + * @param field the field to write; must not be {@code null}. + * @param switchByteOrder whether the persistent form uses the reversed byte order. + * + * @return the setter for the field. + */ + public static BinaryValueSetter provideReferenceSetter( + final Field field , + final boolean switchByteOrder + ) + { + final FieldWriter writer = provideFieldWriter(field); + + return (srcAddress, target, trgOffset, handler) -> + { + final long rawObjectId = XMemory.get_long(srcAddress); + final long objectId = switchByteOrder ? Long.reverseBytes(rawObjectId) : rawObjectId; + + writer.writeValue(target, handler.lookupObject(objectId)); + + return srcAddress + Binary.objectIdByteLength(); + }; + } + + + + /** + * @param primitiveType the type to look up. + * + * @return the wrapper type for the passed primitive type, or {@code null} if it is not one. + */ + private static Class wrapperTypeOf(final Class primitiveType) + { + return primitiveType == byte.class + ? Byte.class + : primitiveType == boolean.class + ? Boolean.class + : primitiveType == short.class + ? Short.class + : primitiveType == char.class + ? Character.class + : primitiveType == int.class + ? Integer.class + : primitiveType == float.class + ? Float.class + : primitiveType == long.class + ? Long.class + : primitiveType == double.class + ? Double.class + : null + ; + } + + /** + * Creates the setter reading a persisted primitive and writing it into a value class field of the + * corresponding wrapper type, boxing it on the way. + *

+ * This is the layout-aware counterpart of the {@code primitive -> wrapper} translators in + * {@link BinaryValueTranslators}, which box and write in one step at the field's memory offset and + * therefore cannot reach a field the JVM lays out inside its owner. + *

+ * Only the pairs those translators register are served, and only in this JVM's own byte order: + * their reversed-byte-order counterparts do not exist either, so a widening under a switched byte + * order stays the loud refusal it already was. + * + * @param field the field to write; must not be {@code null}. + * @param sourceType the persisted member's primitive type. + * @param switchByteOrder whether the persistent form uses the reversed byte order. + * + * @return the boxing setter, or {@code null} if the pair is not covered. + */ + public static BinaryValueSetter provideBoxingSetter( + final Field field , + final Class sourceType , + final boolean switchByteOrder + ) + { + /* Only the exact primitive-to-its-own-wrapper pairs are defined, as in the offset-based table, + * so a widening to a different wrapper is not served here either. + */ + if(switchByteOrder || wrapperTypeOf(sourceType) != field.getType()) + { + return null; + } + + final FieldWriter writer = provideFieldWriter(field); + + if(sourceType == byte.class) + { + return (srcAddress, target, trgOffset, handler) -> + { + writer.writeValue(target, Byte.valueOf(XMemory.get_byte(srcAddress))); + return srcAddress + XMemory.byteSize_byte(); + }; + } + if(sourceType == boolean.class) + { + return (srcAddress, target, trgOffset, handler) -> + { + writer.writeValue(target, Boolean.valueOf(XMemory.get_boolean(srcAddress))); + return srcAddress + XMemory.byteSize_boolean(); + }; + } + if(sourceType == short.class) + { + return (srcAddress, target, trgOffset, handler) -> + { + writer.writeValue(target, Short.valueOf(XMemory.get_short(srcAddress))); + return srcAddress + XMemory.byteSize_short(); + }; + } + if(sourceType == char.class) + { + return (srcAddress, target, trgOffset, handler) -> + { + writer.writeValue(target, Character.valueOf(XMemory.get_char(srcAddress))); + return srcAddress + XMemory.byteSize_char(); + }; + } + if(sourceType == int.class) + { + return (srcAddress, target, trgOffset, handler) -> + { + writer.writeValue(target, Integer.valueOf(XMemory.get_int(srcAddress))); + return srcAddress + XMemory.byteSize_int(); + }; + } + if(sourceType == float.class) + { + return (srcAddress, target, trgOffset, handler) -> + { + writer.writeValue(target, Float.valueOf(XMemory.get_float(srcAddress))); + return srcAddress + XMemory.byteSize_float(); + }; + } + if(sourceType == long.class) + { + return (srcAddress, target, trgOffset, handler) -> + { + writer.writeValue(target, Long.valueOf(XMemory.get_long(srcAddress))); + return srcAddress + XMemory.byteSize_long(); + }; + } + if(sourceType == double.class) + { + return (srcAddress, target, trgOffset, handler) -> + { + writer.writeValue(target, Double.valueOf(XMemory.get_double(srcAddress))); + return srcAddress + XMemory.byteSize_double(); + }; + } + + return null; + } + + + + /////////////////////////////////////////////////////////////////////////// + // member types // + ///////////////// + + /** + * Reads one field's value from an owner instance. + * + * @see BinaryValueHandleFunctions#provideFieldReader(Field) + */ + @FunctionalInterface + public interface FieldReader + { + /** + * Reads this reader's field from the passed owner. + * + * @param owner the instance to read from. + * + * @return the field's value, may be {@code null}. + */ + public Object readValue(Object owner); + } + + /** + * Writes one field's value into an owner instance. + * + * @see BinaryValueHandleFunctions#provideFieldWriter(Field) + */ + @FunctionalInterface + public interface FieldWriter + { + /** + * Writes the passed value into this writer's field of the passed owner. + * + * @param owner the instance to write into. + * @param value the value to write, may be {@code null}. + */ + public void writeValue(Object owner, Object value); + } + + /** + * Access to the internal accessor's layout-aware field writing, resolved reflectively since it is + * compiled against a JDK baseline that does not have those methods. Only ever needed for a record's + * value class fields, so being unresolvable is not reported before such a field is actually written. + */ + private static final class InternalUnsafeFlatAccess + { + // all null if the internal accessor is unavailable, see #resolve and #provideFieldWriter. + private static final Object UNSAFE ; + private static final Method OBJECT_FIELD_OFFSET ; + private static final Method IS_FLAT_FIELD ; + private static final Method FIELD_LAYOUT ; + private static final Method PUT_FLAT_VALUE ; + private static final Method PUT_REFERENCE ; + + static + { + Object unsafe = null; + Method objectFieldOffset = null; + Method isFlatField = null; + Method fieldLayout = null; + Method putFlatValue = null; + Method putReference = null; + try + { + final Class unsafeClass = Class.forName("jdk.internal.misc.Unsafe"); + unsafe = unsafeClass.getMethod("getUnsafe").invoke(null); + objectFieldOffset = unsafeClass.getMethod("objectFieldOffset", Field.class); + isFlatField = unsafeClass.getMethod("isFlatField", Field.class); + fieldLayout = unsafeClass.getMethod("fieldLayout", Field.class); + putFlatValue = unsafeClass.getMethod( + "putFlatValue", Object.class, long.class, int.class, Class.class, Object.class + ); + putReference = unsafeClass.getMethod("putReference", Object.class, long.class, Object.class); + } + catch(final ReflectiveOperationException e) + { + // left unresolved; reported when a field actually needs it. + } + UNSAFE = unsafe ; + OBJECT_FIELD_OFFSET = objectFieldOffset; + IS_FLAT_FIELD = isFlatField ; + FIELD_LAYOUT = fieldLayout ; + PUT_FLAT_VALUE = putFlatValue ; + PUT_REFERENCE = putReference ; + } + + static FieldWriter provideFieldWriter(final Field field) + { + if(UNSAFE == null) + { + throw new BinaryPersistenceException( + "Cannot write field " + field + ": no access to \"jdk.internal.misc.Unsafe\"." + + " Please start the VM with --add-exports java.base/jdk.internal.misc=ALL-UNNAMED" + ); + } + + final long offset = (Long)invoke(OBJECT_FIELD_OFFSET, field, field); + + if(!(Boolean)invoke(IS_FLAT_FIELD, field, field)) + { + // not laid out inside its owner, so the field holds a plain reference. + return bindWriter(field, PUT_REFERENCE, Long.valueOf(offset)); + } + + final int layout = (Integer)invoke(FIELD_LAYOUT, field, field); + + return bindWriter(field, PUT_FLAT_VALUE, Long.valueOf(offset), Integer.valueOf(layout), field.getType()); + } + + /** + * Binds everything but the owner and the value into a {@link MethodHandle}, so a write is an exact + * invocation rather than a reflective call. + *

+ * The introspection above happens once per field and can stay reflective. This does not: it is on + * the load path, once per field of every record instance built, and {@link Method#invoke} would + * allocate a varargs array and box the offset and the layout on each of them. + * + * @param field the field being written, for the failure message. + * @param method the accessor to bind. + * @param bound the arguments after the owner and before the value. + * + * @return the writer for that field. + */ + private static FieldWriter bindWriter(final Field field, final Method method, final Object... bound) + { + final MethodHandle writer; + try + { + writer = MethodHandles.insertArguments( + MethodHandles.lookup().unreflect(method).bindTo(UNSAFE), 1, bound + ) + .asType(MethodType.methodType(void.class, Object.class, Object.class)) + ; + } + catch(final IllegalAccessException e) + { + throw new BinaryPersistenceException("Cannot write field " + field + ".", e); + } + + return (owner, value) -> + { + try + { + writer.invokeExact(owner, value); + } + catch(final RuntimeException | Error e) + { + throw e; + } + catch(final Throwable t) + { + throw new BinaryPersistenceException("Failed to write field " + field + ".", t); + } + }; + } + + private static Object invoke(final Method method, final Field field, final Object... arguments) + { + try + { + return method.invoke(UNSAFE, arguments); + } + catch(final InvocationTargetException e) + { + final Throwable cause = e.getCause(); + if(cause instanceof RuntimeException) + { + throw (RuntimeException)cause; + } + if(cause instanceof Error) + { + throw (Error)cause; + } + throw new BinaryPersistenceException("Failed to write field " + field + ".", cause); + } + catch(final IllegalAccessException e) + { + throw new BinaryPersistenceException("Failed to write field " + field + ".", e); + } + } + } + + + + /////////////////////////////////////////////////////////////////////////// + // constructors // + ///////////////// + + private BinaryValueHandleFunctions() + { + // static only + throw new UnsupportedOperationException(); + } + +} diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryValueReader.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryValueReader.java new file mode 100644 index 00000000..9da06747 --- /dev/null +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryValueReader.java @@ -0,0 +1,97 @@ +package org.eclipse.serializer.persistence.binary.types; + +/*- + * #%L + * Eclipse Serializer Persistence Binary + * %% + * Copyright (C) 2023 - 2026 MicroStream Software + * %% + * This program and the accompanying materials are made + * available under the terms of the Eclipse Public License 2.0 + * which is available at https://www.eclipse.org/legal/epl-2.0/ + * + * SPDX-License-Identifier: EPL-2.0 + * #L% + */ + +import org.eclipse.serializer.persistence.binary.exceptions.BinaryPersistenceException; +import org.eclipse.serializer.persistence.types.PersistenceLoadHandler; + +/** + * Reads a single persisted field value and returns it, as opposed to {@link BinaryValueSetter}, + * which writes it directly into an instance's memory. + *

+ * This indirection is required wherever the values must exist before the instance does, i.e. + * for constructor-based instantiation. + * + * @see BinaryHandlerGenericValueClass + */ +@FunctionalInterface +public interface BinaryValueReader +{ + /** + * Reads the value located at {@code offset} in the passed entity data. + * + * @param data the entity data to read from. + * @param offset the value's offset in the entity's content. + * @param handler the load handler used to resolve references. + * + * @return the read value, boxed if primitive. + */ + public Object readValue(Binary data, long offset, PersistenceLoadHandler handler); + + + /** + * Provides the reader matching the passed field type. + *

+ * Byte order needs no handling here: {@link Binary} reads through virtual accessors, and a load + * item for persisted data in a non-native byte order already reverses every value it reads. + * + * @param type the field type whose value is to be read. + * + * @return the matching reader. + */ + public static BinaryValueReader provideReader(final Class type) + { + if(!type.isPrimitive()) + { + return (data, offset, handler) -> handler.lookupObject(data.read_long(offset)); + } + + if(type == int.class) + { + return (data, offset, handler) -> Integer.valueOf(data.read_int(offset)); + } + if(type == long.class) + { + return (data, offset, handler) -> Long.valueOf(data.read_long(offset)); + } + if(type == boolean.class) + { + return (data, offset, handler) -> Boolean.valueOf(data.read_boolean(offset)); + } + if(type == byte.class) + { + return (data, offset, handler) -> Byte.valueOf(data.read_byte(offset)); + } + if(type == double.class) + { + return (data, offset, handler) -> Double.valueOf(data.read_double(offset)); + } + if(type == float.class) + { + return (data, offset, handler) -> Float.valueOf(data.read_float(offset)); + } + if(type == char.class) + { + return (data, offset, handler) -> Character.valueOf(data.read_char(offset)); + } + if(type == short.class) + { + return (data, offset, handler) -> Short.valueOf(data.read_short(offset)); + } + + throw new BinaryPersistenceException("Unhandled primitive type: " + type.getName()); + } + +} diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryValueStructFunctions.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryValueStructFunctions.java new file mode 100644 index 00000000..b1437094 --- /dev/null +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryValueStructFunctions.java @@ -0,0 +1,1872 @@ +package org.eclipse.serializer.persistence.binary.types; + +/*- + * #%L + * Eclipse Serializer Persistence Binary + * %% + * Copyright (C) 2023 - 2026 MicroStream Software + * %% + * This program and the accompanying materials are made + * available under the terms of the Eclipse Public License 2.0 + * which is available at https://www.eclipse.org/legal/epl-2.0/ + * + * SPDX-License-Identifier: EPL-2.0 + * #L% + */ + +import static org.eclipse.serializer.persistence.types.PersistenceTypeDescriptionMemberFieldValueStruct.NULL_MARKER_ABSENT; +import static org.eclipse.serializer.persistence.types.PersistenceTypeDescriptionMemberFieldValueStruct.NULL_MARKER_LENGTH; +import static org.eclipse.serializer.persistence.types.PersistenceTypeDescriptionMemberFieldValueStruct.NULL_MARKER_PRESENT; + +import java.lang.invoke.MethodHandle; +import java.lang.reflect.Field; + +import org.eclipse.serializer.chars.VarString; +import org.eclipse.serializer.collections.BulkList; +import org.eclipse.serializer.collections.HashEnum; +import org.eclipse.serializer.collections.types.XGettingEnum; +import org.eclipse.serializer.collections.types.XGettingSequence; +import org.eclipse.serializer.memory.XMemory; +import org.eclipse.serializer.persistence.binary.exceptions.BinaryPersistenceException; +import org.eclipse.serializer.persistence.binary.types.BinaryValueHandleFunctions.FieldReader; +import org.eclipse.serializer.persistence.binary.types.BinaryValueHandleFunctions.FieldWriter; +import org.eclipse.serializer.persistence.types.PersistenceLoadHandler; +import org.eclipse.serializer.persistence.types.PersistenceStoreHandler; +import org.eclipse.serializer.persistence.types.PersistenceTypeDefinitionMemberField; +import org.eclipse.serializer.persistence.types.PersistenceTypeDefinitionMemberFieldValueStruct; +import org.eclipse.serializer.persistence.types.PersistenceTypeDescriptionMemberFieldValueStruct; +import org.eclipse.serializer.persistence.types.PersistenceTypeDescriptionResolver; +import org.eclipse.serializer.reflect.XReflect; +import org.eclipse.serializer.typing.KeyValue; +import org.eclipse.serializer.util.logging.Logging; +import org.slf4j.Logger; + +/** + * Storer and setter for a field that is written into its owner's own binary form rather than referenced by + * an object id. + *

+ * The slot is a null marker byte followed by the inlined type's own persistent layout, so it is fixed-length + * and its content is byte-identical to what that type's entity form would contain. A {@code null} field + * writes the marker and zeroes the rest, which keeps the slot's length independent of its content. + *

+ * Storing walks the inlined type's fields with the same per-field storers an entity of that type would use. + * Loading cannot mirror that, because the fields of an identity-less instance cannot be written after + * construction: the values are read into an argument array first and the instance is then constructed from + * them, the same way its own entity form is reconstructed. + */ +public final class BinaryValueStructFunctions +{ + /////////////////////////////////////////////////////////////////////////// + // static fields // + ////////////////// + + private final static Logger logger = Logging.getLogger(BinaryValueStructFunctions.class); + + + + /////////////////////////////////////////////////////////////////////////// + // static methods // + /////////////////// + + /** + * Creates the storer writing an inlined field into its owner's binary form. + * + * @param ownerField the owner field holding the inlined value; must not be {@code null}. + * @param members the inlined layout's members, in persistent order. + * @param structLength the slot's fixed length, including the null marker. + * @param switchByteOrder whether the persistent form uses the reversed byte order. + * + * @return the storer for the inlined field. + */ + public static BinaryValueStorer provideStorer( + final Field ownerField , + final XGettingSequence members , + final long structLength , + final boolean switchByteOrder + ) + { + final int count = members.intSize(); + final BinaryValueStorer[] storers = new BinaryValueStorer[count]; + final long[] offsets = new long[count]; + + int i = 0; + for(final PersistenceTypeDefinitionMemberField member : members) + { + final Field field = validateField(member); + if(member instanceof PersistenceTypeDefinitionMemberFieldValueStruct) + { + /* A nested slot is written by a storer of its own, which reads its value through a handle on + * the field and so needs no offset - a value laid out inside its owner has none to take. + */ + final PersistenceTypeDefinitionMemberFieldValueStruct nested = + (PersistenceTypeDefinitionMemberFieldValueStruct)member + ; + storers[i] = provideStorer( + field, nested.members(), nested.persistentMinimumLength(), switchByteOrder + ); + offsets[i] = 0; + } + else + { + storers[i] = BinaryValueFunctions.getObjectValueStorer(field.getType(), false, switchByteOrder); + offsets[i] = XMemory.objectFieldOffset(field); + } + i++; + } + + /* The slot is read back by constructing the inlined type from it, so the same constructor contract + * the type's own entity handler verifies applies here. Resolved rather than passed in: the setter + * resolves it too, and a storer that never wrote a value would otherwise never have it checked. + */ + final Class valueType = ownerField.getType(); + final HashEnum declarationOrder = declarationOrder(members, valueType); + + return new StructStorer( + BinaryValueHandleFunctions.provideFieldReader(ownerField), + storers, + offsets, + structLength, + ValueClassConstructorContract.New( + valueType, + declarationOrder, + BinaryHandlerGenericValueClass.resolveConstructor( + valueType, + BinaryHandlerGenericValueClass.toParameterTypes(declarationOrder), + declarationOrder + ) + ) + ); + } + + /** + * Creates the setter reading an inlined field out of its owner's binary form. + * + * @param ownerField the owner field holding the inlined value; must not be {@code null}. + * @param valueType the inlined type; must not be {@code null}. + * @param members the inlined layout's members, in persistent order. + * @param declarationOrder the inlined type's persistable fields in declaration order, which is the order + * its constructor accepts them in. + * @param structLength the slot's fixed length, including the null marker. + * @param switchByteOrder whether the persistent form uses the reversed byte order. + * + * @return the setter for the inlined field. + */ + public static BinaryValueSetter provideSetter( + final Field ownerField , + final Class valueType , + final XGettingSequence members , + final XGettingEnum declarationOrder, + final long structLength , + final boolean switchByteOrder + ) + { + final int count = members.intSize(); + final StructReader[] readers = new StructReader[count]; + final int[] targets = new int[count]; + + int i = 0; + for(final PersistenceTypeDefinitionMemberField member : members) + { + final Field field = validateField(member); + readers[i] = provideMemberReader(member, switchByteOrder); + + /* The persistent order need not be the declaration order the constructor accepts, so every slot + * carries the argument index it belongs to rather than relying on the two coinciding. + */ + targets[i] = indexOf(declarationOrder, field); + if(targets[i] < 0) + { + throw new BinaryPersistenceException( + "Inlined field " + field + " is not among the persistable fields of " + valueType.getName() + ); + } + i++; + } + + final MethodHandle constructor = BinaryHandlerGenericValueClass.resolveConstructor( + valueType, + BinaryHandlerGenericValueClass.toParameterTypes(declarationOrder), + declarationOrder + ); + + return new StructSetter( + BinaryValueHandleFunctions.provideFieldWriter(ownerField), + valueType, + readers, + targets, + constructor, + declarationOrder.intSize(), + structLength + ); + } + + /** + * Creates the setter for an inlined field described by a legacy type definition, deriving the inlined + * type's declaration order from the type itself. + *

+ * Only applicable while the described layout still matches the type's current one, which the caller has + * to establish: the constructor takes every field, so a described layout missing one could not be + * invoked, and one carrying an extra field would leave bytes unread. + * + * @param member the inlined member as described by the legacy definition. + * @param switchByteOrder whether the persistent form uses the reversed byte order. + * + * @return the setter for the inlined field. + */ + public static BinaryValueSetter provideSetter( + final PersistenceTypeDefinitionMemberFieldValueStruct member , + final boolean switchByteOrder + ) + { + final Class valueType = member.type(); + if(valueType == null) + { + throw new BinaryPersistenceException( + "Inlined field " + member.identifier() + " has no runtime type." + ); + } + + final HashEnum declarationOrder = HashEnum.New(); + for(final Field field : valueType.getDeclaredFields()) + { + if(!XReflect.isStatic(field) && isDescribed(member, field)) + { + declarationOrder.add(field); + } + } + + if(declarationOrder.intSize() != member.members().intSize()) + { + throw new BinaryPersistenceException( + "Inlined layout of " + member.identifier() + " describes " + member.members().intSize() + + " fields, but " + valueType.getName() + " has " + declarationOrder.intSize() + " of them." + ); + } + + return provideSetter( + member.field() , + valueType , + member.members() , + declarationOrder , + member.persistentMinimumLength(), + switchByteOrder + ); + } + + /** + * Creates the setter for an inlined field whose described layout has since gained or lost a member, + * which is what an inlined type evolving under existing data produces. + *

+ * The described members are matched to the current ones by name. A member the type no longer has is + * stepped over by its persisted length, and a member it has gained takes its type's default - the same + * answer legacy mapping gives for a referenced field. What is not translated is a member whose type + * changed: its bytes would have to be converted rather than placed, and placing them would misread + * silently, so that is refused. + *

+ * The two members describe the same field of the same owner, paired by the legacy mapping, so a renamed + * inlined type is already accounted for by the time this is reached. A renamed member of that type is + * matched by the refactoring mapping, keyed by the member's own identifier ({@code #}) + * exactly as it would be while the field is still referenced. A rule mapping it to {@code null} states + * that it was removed deliberately, which discards its persisted value without further complaint. + *

+ * A member lost and another of the same type gained, neither of them mapped, is refused: on the wire a + * rename is those same two facts, so carrying the value would guess and dropping it would lose it + * silently. A mapping entry - to the gained member, or to {@code null} - states which it is. + * + * @param sourceMember the inlined member as the legacy definition describes it. + * @param targetMember the inlined member as the current type describes it. + * @param refactoringResolver the resolver consulted for member mappings; may be {@code null}. + * @param switchByteOrder whether the persistent form uses the reversed byte order. + * + * @return the setter for the evolved inlined field. + */ + public static BinaryValueSetter provideEvolvingSetter( + final PersistenceTypeDefinitionMemberFieldValueStruct sourceMember , + final PersistenceTypeDefinitionMemberFieldValueStruct targetMember , + final PersistenceTypeDescriptionResolver refactoringResolver, + final boolean switchByteOrder + ) + { + final EvolvingLayout layout = evolvingLayout( + sourceMember, targetMember, refactoringResolver, switchByteOrder + ); + + return new EvolvingStructSetter( + BinaryValueHandleFunctions.provideFieldWriter(targetMember.field()), + layout.valueType , + layout.readers , + layout.targets , + layout.constructor , + layout.defaults , + layout.defaultedNames , + layout.structLength + ); + } + + /** + * The reader for a nested inlined member whose described layout differs from the current one, which is + * what an inlined type nested in another evolving under existing data produces. It answers one level + * down exactly as {@link #provideEvolvingSetter} does for the owner's field. + */ + private static StructReader provideEvolvingNestedReader( + final PersistenceTypeDefinitionMemberFieldValueStruct sourceMember , + final PersistenceTypeDefinitionMemberFieldValueStruct targetMember , + final PersistenceTypeDescriptionResolver refactoringResolver, + final boolean switchByteOrder + ) + { + final EvolvingLayout layout = evolvingLayout( + sourceMember, targetMember, refactoringResolver, switchByteOrder + ); + + return new EvolvingNestedStructReader( + layout.valueType , + layout.readers , + layout.targets , + layout.constructor , + layout.defaults , + layout.defaultedNames, + layout.structLength + ); + } + + /** + * Matches a persisted inlined layout to the current one and derives everything needed to read the one + * into the other, whether the result is written to a field or handed to an enclosing constructor. + */ + private static EvolvingLayout evolvingLayout( + final PersistenceTypeDefinitionMemberFieldValueStruct sourceMember , + final PersistenceTypeDefinitionMemberFieldValueStruct targetMember , + final PersistenceTypeDescriptionResolver refactoringResolver, + final boolean switchByteOrder + ) + { + final Class valueType = targetMember.type(); + if(valueType == null) + { + throw new BinaryPersistenceException( + "Inlined field " + targetMember.identifier() + " has no runtime type." + ); + } + + /* The order the constructor accepts, derived from the type itself the way the unchanged path does, + * and restricted to what the current layout describes. + */ + final HashEnum declarationOrder = declarationOrder(targetMember, valueType); + + final BulkList matches = matchEvolvingMembers( + sourceMember, targetMember, refactoringResolver, declarationOrder + ); + + final int count = matches.intSize(); + final StructReader[] readers = new StructReader[count]; + final int[] targets = new int[count]; + + int i = 0; + for(final MemberMatch match : matches) + { + if(match.isDropped()) + { + // the type no longer has this member: step over what was written for it + final long skipped = match.source.persistentMinimumLength(); + readers[i] = (address, args, index) -> skipped; + targets[i] = 0; + } + else + { + readers[i] = match.isNested() + ? provideEvolvingNestedReader( + match.sourceStruct(), match.targetStruct(), refactoringResolver, switchByteOrder + ) + : provideReader(match.field.getType(), switchByteOrder) + ; + targets[i] = indexOf(declarationOrder, match.field); + } + i++; + } + + final HashEnum defaulted = defaultedFields(declarationOrder, matches); + + final MethodHandle constructor = BinaryHandlerGenericValueClass.resolveConstructor( + valueType, + BinaryHandlerGenericValueClass.toParameterTypes(declarationOrder), + declarationOrder + ); + + return new EvolvingLayout( + valueType , + readers , + targets , + constructor , + defaultArguments(declarationOrder), + fieldNames(defaulted) , + sourceMember.persistentMinimumLength() + ); + } + + /** + * Pairs every persisted member of an inlined layout with the current member it is read into, which is + * the half of evolving a layout that does not depend on where it is read from. Both readers build on + * it: the one reading a slot inside an owner's entity and the one reading it inside a value class's. + *

+ * Everything that can refuse the evolution happens here - a member changing between inlined and plain, + * a changed type, two rules pointing at one member, and the ambiguous loss-plus-gain - as does the + * report of what was carried, defaulted and dropped. + * + * @return one match per persisted member, in persisted order. + */ + private static BulkList matchEvolvingMembers( + final PersistenceTypeDefinitionMemberFieldValueStruct sourceMember , + final PersistenceTypeDefinitionMemberFieldValueStruct targetMember , + final PersistenceTypeDescriptionResolver refactoringResolver, + final HashEnum declarationOrder + ) + { + final BulkList matches = BulkList.New(); + final HashEnum carried = HashEnum.New(); + + // persisted members whose value is discarded without a rule saying so, which is worth reporting + final HashEnum dropped = HashEnum.New(); + + for(final PersistenceTypeDefinitionMemberField source : sourceMember.members()) + { + // null means a mapping rule states the member was removed deliberately + final String targetName = resolveTargetName(refactoringResolver, targetMember, source); + final Field field = targetName == null + ? null + : findField(declarationOrder, targetName) + ; + if(field == null) + { + matches.add(new MemberMatch(source, null, null)); + if(targetName != null) + { + dropped.add(source); + } + + continue; + } + + final PersistenceTypeDefinitionMemberField target = findMember(targetMember, targetName); + + final boolean sourceIsStruct = source instanceof PersistenceTypeDefinitionMemberFieldValueStruct; + final boolean targetIsStruct = target instanceof PersistenceTypeDefinitionMemberFieldValueStruct; + if(sourceIsStruct != targetIsStruct) + { + /* The same member, inlined on one side and referenced on the other. The persisted bytes are + * a whole layout where the current form expects one value, or the other way round, and + * neither can be read as the other. + */ + throw new BinaryPersistenceException( + "Inlined layout member " + source.identifier() + " was persisted " + + (sourceIsStruct ? "as an inlined layout and is now a single value" + : "as a single value and is now an inlined layout") + + ". Changing whether a member of an inlined layout is itself inlined is not supported." + ); + } + + if(!isSameType(refactoringResolver, source.typeName(), field.getType())) + { + throw new BinaryPersistenceException( + "Inlined layout member " + source.identifier() + " was persisted as " + + source.typeName() + " and is now " + field.getType().getName() + + ". Converting the type of an inlined member is not supported." + ); + } + + matches.add(new MemberMatch(source, target, field)); + + if(!carried.add(field.getName())) + { + /* Two persisted members reading into one, which only a mapping rule can produce: the + * construction would take whichever is read last and discard the other. + */ + throw new BinaryPersistenceException( + "Inlined layout of " + targetMember.identifier() + " reads more than one persisted" + + " member into " + field.getName() + ", the last of them " + source.identifier() + + ". Every persisted member needs a target of its own, or none." + ); + } + } + + final HashEnum defaulted = HashEnum.New(); + for(final Field field : declarationOrder) + { + if(!carried.contains(field.getName())) + { + defaulted.add(field); + } + } + + validateUnambiguousEvolution(targetMember, dropped, defaulted); + reportEvolution(targetMember, carried, dropped, defaulted); + + return matches; + } + + /** The current members no persisted member is read into, which take their type's default. */ + private static HashEnum defaultedFields( + final HashEnum declarationOrder, + final BulkList matches + ) + { + final HashEnum defaulted = HashEnum.New(); + + for(final Field field : declarationOrder) + { + if(!matches.containsSearched(match -> field.equals(match.field))) + { + defaulted.add(field); + } + } + + return defaulted; + } + + /** One persisted member of an inlined layout and the current member it is read into, if any. */ + private static final class MemberMatch + { + final PersistenceTypeDefinitionMemberField source; + final PersistenceTypeDefinitionMemberField target; + final Field field ; + + MemberMatch( + final PersistenceTypeDefinitionMemberField source, + final PersistenceTypeDefinitionMemberField target, + final Field field + ) + { + super(); + this.source = source; + this.target = target; + this.field = field ; + } + + boolean isDropped() + { + return this.field == null; + } + + boolean isNested() + { + return this.target instanceof PersistenceTypeDefinitionMemberFieldValueStruct; + } + + PersistenceTypeDefinitionMemberFieldValueStruct sourceStruct() + { + return (PersistenceTypeDefinitionMemberFieldValueStruct)this.source; + } + + PersistenceTypeDefinitionMemberFieldValueStruct targetStruct() + { + return (PersistenceTypeDefinitionMemberFieldValueStruct)this.target; + } + } + + /** + * Whether the persisted member's type is the current field's, either by name or because a refactoring + * rule renamed it - the same rule that carries such a rename while the type is referenced rather than + * nested, so the two forms answer alike. + */ + private static boolean isSameType( + final PersistenceTypeDescriptionResolver resolver , + final String persistedName, + final Class currentType + ) + { + if(currentType.getName().equals(persistedName)) + { + return true; + } + + return resolver != null && currentType.getName().equals(resolver.resolveRuntimeTypeName(persistedName)); + } + + /** The current member of the passed name, or {@code null} where the current layout has none. */ + private static PersistenceTypeDefinitionMemberField findMember( + final PersistenceTypeDefinitionMemberFieldValueStruct targetMember, + final String name + ) + { + for(final PersistenceTypeDefinitionMemberField member : targetMember.members()) + { + if(member.name().equals(name)) + { + return member; + } + } + + return null; + } + + /** Everything needed to read a persisted inlined layout into the current one. */ + private static final class EvolvingLayout + { + final Class valueType ; + final StructReader[] readers ; + final int[] targets ; + final MethodHandle constructor ; + final Object[] defaults ; + final String defaultedNames; + final long structLength ; + + EvolvingLayout( + final Class valueType , + final StructReader[] readers , + final int[] targets , + final MethodHandle constructor , + final Object[] defaults , + final String defaultedNames, + final long structLength + ) + { + super(); + this.valueType = valueType ; + this.readers = readers ; + this.targets = targets ; + this.constructor = constructor ; + this.defaults = defaults ; + this.defaultedNames = defaultedNames; + this.structLength = structLength ; + } + + } + + /** + * The name of the current member the persisted one is to be read into: the one a refactoring rule names, + * or the persisted member's own name where no rule applies. + *

+ * The rule is keyed by the member's own identifier, which is the inlined type's name and the field's, + * the same key a rename of that field carries while it is still referenced rather than inlined. + * + * @return the current member's name, or {@code null} where a rule states the member was removed. + */ + private static String resolveTargetName( + final PersistenceTypeDescriptionResolver resolver , + final PersistenceTypeDefinitionMemberFieldValueStruct targetMember, + final PersistenceTypeDefinitionMemberField source + ) + { + if(resolver == null) + { + return source.name(); + } + + final KeyValue entry = resolver.lookup(source.identifier()); + if(entry == null) + { + return source.name(); + } + if(entry.value() == null) + { + // can be null for members explicitly marked as deleted + return null; + } + + for(final PersistenceTypeDefinitionMemberField member : targetMember.members()) + { + if(entry.value().equals(member.identifier()) + || entry.value().equals(XReflect.fieldIdentifierDelimiter() + member.name()) + ) + { + return member.name(); + } + } + + throw new BinaryPersistenceException( + "Unresolvable inlined member refactoring mapping: " + source.identifier() + " -> \"" + + entry.value() + "\", which is no member of the inlined layout of " + targetMember.identifier() + + "." + ); + } + + /** + * Refuses a layout that lost one member and gained another of the same type without either of them being + * mapped. + *

+ * A rename and a removal plus an addition are the same two facts on the wire, so there is nothing to tell + * them apart by: carrying the value over would guess, and the alternative loses it without saying so. A + * mapping entry states which of the two it is, and the refusal names the entry to write. + */ + private static void validateUnambiguousEvolution( + final PersistenceTypeDefinitionMemberFieldValueStruct targetMember, + final XGettingEnum dropped , + final XGettingEnum defaulted + ) + { + for(final PersistenceTypeDefinitionMemberField drop : dropped) + { + for(final Field gain : defaulted) + { + if(!gain.getType().getName().equals(drop.typeName())) + { + continue; + } + + throw new BinaryPersistenceException( + "Inlined layout of " + targetMember.identifier() + " lost member " + drop.identifier() + + " and gained member " + gain.getName() + ", both of type " + drop.typeName() + + ". A renamed member and a removed one plus an added one cannot be told apart here," + + " so the persisted value would either be placed by guess or discarded silently." + + " State which it is by mapping \"" + drop.identifier() + "\" to \"" + + gain.getDeclaringClass().getName() + XReflect.fieldIdentifierDelimiter() + + gain.getName() + "\" to carry the value over, or to null to discard it." + ); + } + } + } + + /** + * States what became of the persisted layout's members, which the mapping report for the owner cannot + * show: there the inlined field is one member mapped to one member, and this is what happened inside it. + */ + private static void reportEvolution( + final PersistenceTypeDefinitionMemberFieldValueStruct targetMember, + final XGettingEnum carried , + final XGettingEnum dropped , + final XGettingEnum defaulted + ) + { + final VarString vs = VarString.New() + .add("Inlined layout of ").add(targetMember.identifier()) + .add(" differs from the persisted one. Carried: ").list(", ", carried) + .add(". Defaulted: ").add(fieldNames(defaulted)) + .add(". Dropped: ").add(memberNames(dropped)) + .add('.') + ; + + if(dropped.isEmpty()) + { + logger.info(vs.toString()); + } + else + { + logger.warn(vs.add(" A dropped member's persisted value is discarded.").toString()); + } + } + + private static String fieldNames(final XGettingEnum fields) + { + final VarString vs = VarString.New(); + for(final Field field : fields) + { + vs.add(vs.isEmpty() ? "" : ", ").add(field.getName()); + } + + return vs.toString(); + } + + private static String memberNames( + final XGettingEnum members + ) + { + final VarString vs = VarString.New(); + for(final PersistenceTypeDefinitionMemberField member : members) + { + vs.add(vs.isEmpty() ? "" : ", ").add(member.name()); + } + + return vs.toString(); + } + + /** + * The argument array a construction starts from, so a member the persisted layout does not carry takes + * its type's default. + */ + private static Object[] defaultArguments(final XGettingEnum declarationOrder) + { + final Object[] defaults = new Object[declarationOrder.intSize()]; + + /* Deliberately not a conditional-operator chain: mixing the box types in one would have binary + * numeric promotion widen every branch to the same type, so every default would come back as the + * widest of them and the constructor would reject it. + */ + int i = 0; + for(final Field field : declarationOrder) + { + defaults[i++] = defaultValue(field.getType()); + } + + return defaults; + } + + /** + * The value a member takes where the persisted layout does not carry one: the type's own default. + * Shared with {@link BinaryLegacyTypeHandlerValueClass}, which starts its argument array from these. + * + * @param type the member's type. + * + * @return the boxed default, or {@code null} for a non-primitive. + */ + static Object defaultValue(final Class type) + { + if(type == byte.class) + { + return Byte.valueOf((byte)0); + } + if(type == boolean.class) + { + return Boolean.FALSE; + } + if(type == short.class) + { + return Short.valueOf((short)0); + } + if(type == char.class) + { + return Character.valueOf('\0'); + } + if(type == int.class) + { + return Integer.valueOf(0); + } + if(type == float.class) + { + return Float.valueOf(0f); + } + if(type == long.class) + { + return Long.valueOf(0L); + } + if(type == double.class) + { + return Double.valueOf(0d); + } + + /* A member of an inlined layout is either a primitive or inlined itself, and the default of the + * latter is the absence of a value - which is what its own slot's marker states for it. + */ + return null; + } + + private static Field findField(final XGettingEnum fields, final String name) + { + for(final Field field : fields) + { + if(field.getName().equals(name)) + { + return field; + } + } + + return null; + } + + private static boolean isDescribed( + final PersistenceTypeDefinitionMemberFieldValueStruct member, + final Field field + ) + { + return isDescribed(member.members(), field); + } + + private static boolean isDescribed( + final XGettingSequence members, + final Field field + ) + { + for(final PersistenceTypeDefinitionMemberField describedMember : members) + { + if(field.getName().equals(describedMember.name())) + { + return true; + } + } + + return false; + } + + /** + * Creates the translator copying an inlined slot from a persisted layout into the current one, for the + * rerouting path that rewrites the binary form rather than writing into an instance. + *

+ * Only applicable while the two layouts are equal, which the caller has to establish: the bytes are + * copied as they are, so a layout that changed would be reinterpreted rather than converted. Byte order + * needs no handling, since source and target are the same persisted form. + * + * @param structLength the slot's fixed length, including the null marker. + * + * @return the translator copying the slot. + */ + public static BinaryValueSetter provideRewriter(final long structLength) + { + return (srcAddress, target, trgOffset, handler) -> + { + XMemory.copyRange(srcAddress, trgOffset, structLength); + + return srcAddress + structLength; + }; + } + + /** + * Creates the setter skipping an inlined field whose owner no longer has it. + * + * @param member the inlined member as described by the legacy definition. + * + * @return a setter advancing past the slot without reading it. + */ + public static BinaryValueSetter provideSkipper( + final PersistenceTypeDescriptionMemberFieldValueStruct member + ) + { + final long structLength = member.persistentMinimumLength(); + + return (srcAddress, target, trgOffset, handler) -> srcAddress + structLength; + } + + private static Field validateField(final PersistenceTypeDefinitionMemberField member) + { + final Field field = member.field(); + if(field == null) + { + throw new BinaryPersistenceException( + "Inlined layout member " + member.identifier() + " has no runtime field." + ); + } + + return field; + } + + private static int indexOf(final XGettingEnum fields, final Field field) + { + int i = 0; + for(final Field f : fields) + { + if(f.equals(field)) + { + return i; + } + i++; + } + + return -1; + } + + /** + * Creates the reader for an inlined field of a value class, which is read out of the entity's own data + * rather than out of an enclosing slot. + *

+ * A value instance is constructed rather than populated, so its members are read through + * {@link BinaryValueReader}s into an argument array instead of being written to fields. An inlined + * member is read the same way, one level down: its slot's marker decides whether a value is + * constructed at all. + *

+ * Byte order needs no handling here, for the reason {@link BinaryValueReader#provideReader} states: a + * load item for data in a non-native byte order reverses every value it reads. + * + * @param member the inlined member as the current type describes it. + * + * @return the reader for the inlined member. + */ + public static BinaryValueReader provideValueReader( + final PersistenceTypeDefinitionMemberFieldValueStruct member + ) + { + final Class valueType = member.type(); + if(valueType == null) + { + throw new BinaryPersistenceException( + "Inlined layout member " + member.identifier() + " has no runtime type." + ); + } + + final HashEnum declarationOrder = declarationOrder(member, valueType); + + final int count = member.members().intSize(); + final BinaryValueReader[] readers = new BinaryValueReader[count]; + final long[] offsets = new long[count]; + final int[] targets = new int[count]; + + // the marker precedes the content, so the first member sits behind it + long offset = NULL_MARKER_LENGTH; + int i = 0; + for(final PersistenceTypeDefinitionMemberField nested : member.members()) + { + final Field field = validateField(nested); + readers[i] = nested instanceof PersistenceTypeDefinitionMemberFieldValueStruct + ? provideValueReader((PersistenceTypeDefinitionMemberFieldValueStruct)nested) + : BinaryValueReader.provideReader(field.getType()) + ; + offsets[i] = offset; + targets[i] = indexOf(declarationOrder, field); + if(targets[i] < 0) + { + throw new BinaryPersistenceException( + "Inlined field " + field + " is not among the persistable fields of " + valueType.getName() + ); + } + offset += nested.persistentMinimumLength(); + i++; + } + + final MethodHandle constructor = BinaryHandlerGenericValueClass.resolveConstructor( + valueType, + BinaryHandlerGenericValueClass.toParameterTypes(declarationOrder), + declarationOrder + ); + + return new StructValueReader(valueType, readers, offsets, targets, constructor, declarationOrder.intSize()); + } + + /** + * Creates the reader for an inlined field of a value class whose described layout differs from the + * current one - the counterpart of {@link #provideEvolvingSetter} for a type that is constructed rather + * than populated, and the reason a value class stored as an entity can evolve an inlined member of its + * own. + *

+ * The members are paired exactly as they are for an owner's field, by + * {@link #matchEvolvingMembers}, so the same renames are carried, the same mapping rules apply and the + * same shapes are refused. Only the reading differs: each carried member is read at its own offset in + * the persisted slot, so a dropped one is simply never read rather than stepped over. + * + * @param sourceMember the inlined member as the legacy definition describes it. + * @param targetMember the inlined member as the current type describes it. + * @param refactoringResolver the resolver consulted for member mappings; may be {@code null}. + * + * @return the reader for the evolved inlined member. + */ + public static BinaryValueReader provideEvolvingValueReader( + final PersistenceTypeDefinitionMemberFieldValueStruct sourceMember , + final PersistenceTypeDefinitionMemberFieldValueStruct targetMember , + final PersistenceTypeDescriptionResolver refactoringResolver + ) + { + final Class valueType = targetMember.type(); + if(valueType == null) + { + throw new BinaryPersistenceException( + "Inlined field " + targetMember.identifier() + " has no runtime type." + ); + } + + final HashEnum declarationOrder = declarationOrder(targetMember, valueType); + final BulkList matches = matchEvolvingMembers( + sourceMember, targetMember, refactoringResolver, declarationOrder + ); + + final BulkList readers = BulkList.New(); + final BulkList offsets = BulkList.New(); + final BulkList targets = BulkList.New(); + + // the marker precedes the content, so the first member sits behind it + long offset = NULL_MARKER_LENGTH; + for(final MemberMatch match : matches) + { + if(!match.isDropped()) + { + readers.add(match.isNested() + ? provideEvolvingValueReader(match.sourceStruct(), match.targetStruct(), refactoringResolver) + : BinaryValueReader.provideReader(match.field.getType()) + ); + offsets.add(offset); + targets.add(indexOf(declarationOrder, match.field)); + } + + offset += match.source.persistentMinimumLength(); + } + + final HashEnum defaulted = defaultedFields(declarationOrder, matches); + + final MethodHandle constructor = BinaryHandlerGenericValueClass.resolveConstructor( + valueType, + BinaryHandlerGenericValueClass.toParameterTypes(declarationOrder), + declarationOrder + ); + + return new EvolvingStructValueReader( + valueType , + readers.toArray(BinaryValueReader.class) , + toLongArray(offsets) , + toIntArray(targets) , + constructor , + defaultArguments(declarationOrder) , + fieldNames(defaulted) + ); + } + + private static long[] toLongArray(final BulkList values) + { + final long[] array = new long[values.intSize()]; + + int i = 0; + for(final Long value : values) + { + array[i++] = value.longValue(); + } + + return array; + } + + private static int[] toIntArray(final BulkList values) + { + final int[] array = new int[values.intSize()]; + + int i = 0; + for(final Integer value : values) + { + array[i++] = value.intValue(); + } + + return array; + } + + /** + * The reader for one member of an inlined layout: a nested inlined member is read as a slot of its own + * and constructed, anything else is read as the primitive it is. + */ + private static StructReader provideMemberReader( + final PersistenceTypeDefinitionMemberField member , + final boolean switchByteOrder + ) + { + if(member instanceof PersistenceTypeDefinitionMemberFieldValueStruct) + { + return provideNestedReader( + (PersistenceTypeDefinitionMemberFieldValueStruct)member, switchByteOrder + ); + } + + return provideReader(validateField(member).getType(), switchByteOrder); + } + + /** + * The reader for an inlined member that is itself inlined: it reads the nested slot's own marker and + * members, constructs the instance and places it in the enclosing layout's argument array, the same way + * {@link StructSetter} does for the owner - only returning the value instead of writing it to a field. + */ + private static StructReader provideNestedReader( + final PersistenceTypeDefinitionMemberFieldValueStruct member , + final boolean switchByteOrder + ) + { + final Class valueType = member.type(); + if(valueType == null) + { + throw new BinaryPersistenceException( + "Inlined layout member " + member.identifier() + " has no runtime type." + ); + } + + final HashEnum declarationOrder = declarationOrder(member, valueType); + + final int count = member.members().intSize(); + final StructReader[] readers = new StructReader[count]; + final int[] targets = new int[count]; + + int i = 0; + for(final PersistenceTypeDefinitionMemberField nested : member.members()) + { + final Field field = validateField(nested); + readers[i] = provideMemberReader(nested, switchByteOrder); + targets[i] = indexOf(declarationOrder, field); + if(targets[i] < 0) + { + throw new BinaryPersistenceException( + "Inlined field " + field + " is not among the persistable fields of " + valueType.getName() + ); + } + i++; + } + + final MethodHandle constructor = BinaryHandlerGenericValueClass.resolveConstructor( + valueType, + BinaryHandlerGenericValueClass.toParameterTypes(declarationOrder), + declarationOrder + ); + + return new NestedStructReader( + valueType , + readers , + targets , + constructor , + declarationOrder.intSize() , + member.persistentMinimumLength() + ); + } + + /** + * The inlined type's described fields in declaration order, which is the order its constructor accepts + * them in - derived from the type itself, as the persistent order need not match it. + */ + private static HashEnum declarationOrder( + final PersistenceTypeDefinitionMemberFieldValueStruct member , + final Class valueType + ) + { + return declarationOrder(member.members(), valueType); + } + + private static HashEnum declarationOrder( + final XGettingSequence members , + final Class valueType + ) + { + final HashEnum declarationOrder = HashEnum.New(); + for(final Field field : valueType.getDeclaredFields()) + { + if(!XReflect.isStatic(field) && isDescribed(members, field)) + { + declarationOrder.add(field); + } + } + + return declarationOrder; + } + + private static StructReader provideReader(final Class type, final boolean switchByteOrder) + { + if(switchByteOrder) + { + return provideReaderReversed(type); + } + + if(type == byte.class) + { + return (address, args, index) -> { args[index] = XMemory.get_byte(address); return Byte.BYTES; }; + } + if(type == boolean.class) + { + return (address, args, index) -> { args[index] = XMemory.get_boolean(address); return Byte.BYTES; }; + } + if(type == short.class) + { + return (address, args, index) -> { args[index] = XMemory.get_short(address); return Short.BYTES; }; + } + if(type == char.class) + { + return (address, args, index) -> { args[index] = XMemory.get_char(address); return Character.BYTES; }; + } + if(type == int.class) + { + return (address, args, index) -> { args[index] = XMemory.get_int(address); return Integer.BYTES; }; + } + if(type == float.class) + { + return (address, args, index) -> { args[index] = XMemory.get_float(address); return Float.BYTES; }; + } + if(type == long.class) + { + return (address, args, index) -> { args[index] = XMemory.get_long(address); return Long.BYTES; }; + } + if(type == double.class) + { + return (address, args, index) -> { args[index] = XMemory.get_double(address); return Double.BYTES; }; + } + + throw new BinaryPersistenceException("Type cannot be inlined: " + type.getName()); + } + + private static StructReader provideReaderReversed(final Class type) + { + if(type == byte.class) + { + return (address, args, index) -> { args[index] = XMemory.get_byte(address); return Byte.BYTES; }; + } + if(type == boolean.class) + { + return (address, args, index) -> { args[index] = XMemory.get_boolean(address); return Byte.BYTES; }; + } + if(type == short.class) + { + return (address, args, index) -> + { + args[index] = Short.reverseBytes(XMemory.get_short(address)); + return Short.BYTES; + }; + } + if(type == char.class) + { + return (address, args, index) -> + { + args[index] = Character.reverseBytes(XMemory.get_char(address)); + return Character.BYTES; + }; + } + if(type == int.class) + { + return (address, args, index) -> + { + args[index] = Integer.reverseBytes(XMemory.get_int(address)); + return Integer.BYTES; + }; + } + if(type == float.class) + { + return (address, args, index) -> + { + args[index] = Float.intBitsToFloat(Integer.reverseBytes(XMemory.get_int(address))); + return Float.BYTES; + }; + } + if(type == long.class) + { + return (address, args, index) -> + { + args[index] = Long.reverseBytes(XMemory.get_long(address)); + return Long.BYTES; + }; + } + if(type == double.class) + { + return (address, args, index) -> + { + args[index] = Double.longBitsToDouble(Long.reverseBytes(XMemory.get_long(address))); + return Double.BYTES; + }; + } + + throw new BinaryPersistenceException("Type cannot be inlined: " + type.getName()); + } + + + + /////////////////////////////////////////////////////////////////////////// + // member types // + ///////////////// + + @FunctionalInterface + private interface StructReader + { + /** + * Reads one inlined value into the constructor argument it belongs to. + * + * @return the number of bytes read. + */ + long readValue(long address, Object[] args, int index); + } + + /** + * Reads an inlined slot out of a value class's own entity data and returns the instance it describes, + * so the enclosing value class can pass it to its constructor. An absent marker yields {@code null}. + */ + private static final class StructValueReader implements BinaryValueReader + { + private final Class valueType ; + private final BinaryValueReader[] readers ; + private final long[] offsets ; + private final int[] targets ; + private final MethodHandle constructor ; + private final int argumentCount; + + StructValueReader( + final Class valueType , + final BinaryValueReader[] readers , + final long[] offsets , + final int[] targets , + final MethodHandle constructor , + final int argumentCount + ) + { + super(); + this.valueType = valueType ; + this.readers = readers ; + this.offsets = offsets ; + this.targets = targets ; + this.constructor = constructor ; + this.argumentCount = argumentCount; + } + + @Override + public Object readValue(final Binary data, final long offset, final PersistenceLoadHandler handler) + { + if(data.read_byte(offset) == NULL_MARKER_ABSENT) + { + return null; + } + + final Object[] args = new Object[this.argumentCount]; + for(int i = 0; i < this.readers.length; i++) + { + args[this.targets[i]] = this.readers[i].readValue(data, offset + this.offsets[i], handler); + } + + try + { + return (Object)this.constructor.invokeExact(args); + } + catch(final Error e) + { + throw e; + } + catch(final Throwable t) + { + throw new BinaryPersistenceException( + "Could not construct inlined instance of " + this.valueType.getName(), t + ); + } + } + + } + + /** + * Reads an inlined slot whose described layout differs from the current one out of a value class's own + * entity data. The argument array starts from the current type's defaults, so a member the persisted + * layout does not carry keeps its default, and one the type no longer has is simply never read. + */ + private static final class EvolvingStructValueReader implements BinaryValueReader + { + private final Class valueType ; + private final BinaryValueReader[] readers ; + private final long[] offsets ; + private final int[] targets ; + private final MethodHandle constructor ; + private final Object[] defaults ; + private final String defaultedNames; + + EvolvingStructValueReader( + final Class valueType , + final BinaryValueReader[] readers , + final long[] offsets , + final int[] targets , + final MethodHandle constructor , + final Object[] defaults , + final String defaultedNames + ) + { + super(); + this.valueType = valueType ; + this.readers = readers ; + this.offsets = offsets ; + this.targets = targets ; + this.constructor = constructor ; + this.defaults = defaults ; + this.defaultedNames = defaultedNames; + } + + @Override + public Object readValue(final Binary data, final long offset, final PersistenceLoadHandler handler) + { + if(data.read_byte(offset) == NULL_MARKER_ABSENT) + { + return null; + } + + final Object[] args = this.defaults.clone(); + for(int i = 0; i < this.readers.length; i++) + { + args[this.targets[i]] = this.readers[i].readValue(data, offset + this.offsets[i], handler); + } + + try + { + return (Object)this.constructor.invokeExact(args); + } + catch(final Error e) + { + throw e; + } + catch(final Throwable t) + { + throw new BinaryPersistenceException( + "Could not construct inlined instance of " + this.valueType.getName() + + (this.defaultedNames.isEmpty() + ? "" + : ", whose persisted layout did not carry " + this.defaultedNames + + " so it was constructed with the default") + , t + ); + } + } + + } + + /** + * Reads a nested inlined slot whose described layout differs from the current one: the argument array + * starts from the current type's defaults, so a member the persisted layout does not carry keeps its + * default, and a reader for a member the type no longer has advances past it without writing. + */ + private static final class EvolvingNestedStructReader implements StructReader + { + private final Class valueType ; + private final StructReader[] readers ; + private final int[] targets ; + private final MethodHandle constructor ; + private final Object[] defaults ; + private final String defaultedNames; + private final long structLength ; + + EvolvingNestedStructReader( + final Class valueType , + final StructReader[] readers , + final int[] targets , + final MethodHandle constructor , + final Object[] defaults , + final String defaultedNames, + final long structLength + ) + { + super(); + this.valueType = valueType ; + this.readers = readers ; + this.targets = targets ; + this.constructor = constructor ; + this.defaults = defaults ; + this.defaultedNames = defaultedNames; + this.structLength = structLength ; + } + + @Override + public long readValue(final long address, final Object[] args, final int index) + { + if(XMemory.get_byte(address) == NULL_MARKER_ABSENT) + { + args[index] = null; + + return this.structLength; + } + + final Object[] nested = this.defaults.clone(); + + long a = address + NULL_MARKER_LENGTH; + for(int i = 0; i < this.readers.length; i++) + { + a += this.readers[i].readValue(a, nested, this.targets[i]); + } + + args[index] = this.createValue(nested); + + return this.structLength; + } + + private Object createValue(final Object[] args) + { + try + { + return (Object)this.constructor.invokeExact(args); + } + catch(final Error e) + { + throw e; + } + catch(final Throwable t) + { + throw new BinaryPersistenceException( + "Could not construct nested inlined instance of " + this.valueType.getName() + + (this.defaultedNames.isEmpty() + ? "" + : ", whose persisted layout did not carry " + this.defaultedNames + + " so it was constructed with the default") + , t + ); + } + } + + } + + /** + * Reads a nested inlined slot and returns the instance it describes, so the layout enclosing it can pass + * it to its own constructor. An absent marker yields {@code null}, and the slot's fixed length is + * returned either way - that length, not the bytes actually read, is what the enclosing layout advances + * by. + */ + private static final class NestedStructReader implements StructReader + { + private final Class valueType ; + private final StructReader[] readers ; + private final int[] targets ; + private final MethodHandle constructor ; + private final int argumentCount; + private final long structLength ; + + NestedStructReader( + final Class valueType , + final StructReader[] readers , + final int[] targets , + final MethodHandle constructor , + final int argumentCount, + final long structLength + ) + { + super(); + this.valueType = valueType ; + this.readers = readers ; + this.targets = targets ; + this.constructor = constructor ; + this.argumentCount = argumentCount; + this.structLength = structLength ; + } + + @Override + public long readValue(final long address, final Object[] args, final int index) + { + if(XMemory.get_byte(address) == NULL_MARKER_ABSENT) + { + args[index] = null; + + return this.structLength; + } + + final Object[] nested = new Object[this.argumentCount]; + + long a = address + NULL_MARKER_LENGTH; + for(int i = 0; i < this.readers.length; i++) + { + a += this.readers[i].readValue(a, nested, this.targets[i]); + } + + args[index] = this.createValue(nested); + + return this.structLength; + } + + private Object createValue(final Object[] args) + { + try + { + return (Object)this.constructor.invokeExact(args); + } + catch(final Error e) + { + throw e; + } + catch(final Throwable t) + { + throw new BinaryPersistenceException( + "Could not construct nested inlined instance of " + this.valueType.getName(), t + ); + } + } + + } + + private static final class StructStorer implements BinaryValueStorer + { + private final FieldReader ownerReader ; + private final BinaryValueStorer[] storers ; + private final long[] offsets ; + private final long structLength; + private final ValueClassConstructorContract contract ; + + StructStorer( + final FieldReader ownerReader , + final BinaryValueStorer[] storers , + final long[] offsets , + final long structLength, + final ValueClassConstructorContract contract + ) + { + super(); + this.ownerReader = ownerReader ; + this.storers = storers ; + this.offsets = offsets ; + this.structLength = structLength; + this.contract = contract ; + } + + @Override + public long storeValueFromMemory( + final Object source , + final long sourceOffset , + final long targetAddress, + final PersistenceStoreHandler persister + ) + { + // reached through a handle: a value may be laid out inside its owner, with no reference at its offset + final Object value = this.ownerReader.readValue(source); + if(value == null) + { + // zeroed rather than skipped, so the slot's content never depends on what was there before + XMemory.fillMemory(targetAddress, this.structLength, (byte)0); + return targetAddress + this.structLength; + } + + // before the first slot is written, so a violation is reported instead of persisted + this.contract.validate(value); + + XMemory.set_byte(targetAddress, NULL_MARKER_PRESENT); + + long address = targetAddress + NULL_MARKER_LENGTH; + for(int i = 0; i < this.storers.length; i++) + { + address = this.storers[i].storeValueFromMemory(value, this.offsets[i], address, persister); + } + + return address; + } + + } + + /** + * Reads an inlined slot whose described layout differs from the current one: the argument array starts + * from the current type's defaults, so a member the persisted layout does not carry keeps its default, + * and a reader for a member the type no longer has advances past it without writing. + */ + private static final class EvolvingStructSetter implements BinaryValueSetter + { + private final FieldWriter ownerWriter ; + private final Class valueType ; + private final StructReader[] readers ; + private final int[] targets ; + private final MethodHandle constructor ; + private final Object[] defaults ; + private final String defaultedNames ; + private final long structLength ; + + EvolvingStructSetter( + final FieldWriter ownerWriter , + final Class valueType , + final StructReader[] readers , + final int[] targets , + final MethodHandle constructor , + final Object[] defaults , + final String defaultedNames, + final long structLength + ) + { + super(); + this.ownerWriter = ownerWriter ; + this.valueType = valueType ; + this.readers = readers ; + this.targets = targets ; + this.constructor = constructor ; + this.defaults = defaults ; + this.defaultedNames = defaultedNames; + this.structLength = structLength ; + } + + @Override + public long setValueToMemory( + final long srcAddress, + final Object target , + final long trgOffset , + final PersistenceLoadHandler handler + ) + { + if(XMemory.get_byte(srcAddress) == NULL_MARKER_ABSENT) + { + this.ownerWriter.writeValue(target, null); + return srcAddress + this.structLength; + } + + final Object[] args = this.defaults.clone(); + + long address = srcAddress + NULL_MARKER_LENGTH; + for(int i = 0; i < this.readers.length; i++) + { + address += this.readers[i].readValue(address, args, this.targets[i]); + } + + this.ownerWriter.writeValue(target, this.createValue(args)); + + return address; + } + + private Object createValue(final Object[] args) + { + try + { + return (Object)this.constructor.invokeExact(args); + } + catch(final Error e) + { + throw e; + } + catch(final Throwable t) + { + /* Naming the defaulted members is the point: a constructor validating its arguments is the + * likely reason a construction that used to work now fails, and the defaulted value is what + * it rejected. + */ + throw new BinaryPersistenceException( + "Could not construct inlined instance of " + this.valueType.getName() + + (this.defaultedNames.isEmpty() + ? "" + : ", whose persisted layout did not carry " + this.defaultedNames + + " so it was constructed with the default") + , t + ); + } + } + + } + + private static final class StructSetter implements BinaryValueSetter + { + private final FieldWriter ownerWriter ; + private final Class valueType ; + private final StructReader[] readers ; + private final int[] targets ; + private final MethodHandle constructor ; + private final int argumentCount; + private final long structLength ; + + StructSetter( + final FieldWriter ownerWriter , + final Class valueType , + final StructReader[] readers , + final int[] targets , + final MethodHandle constructor , + final int argumentCount, + final long structLength + ) + { + super(); + this.ownerWriter = ownerWriter ; + this.valueType = valueType ; + this.readers = readers ; + this.targets = targets ; + this.constructor = constructor ; + this.argumentCount = argumentCount; + this.structLength = structLength ; + } + + @Override + public long setValueToMemory( + final long srcAddress, + final Object target , + final long trgOffset , + final PersistenceLoadHandler handler + ) + { + if(XMemory.get_byte(srcAddress) == NULL_MARKER_ABSENT) + { + this.ownerWriter.writeValue(target, null); + return srcAddress + this.structLength; + } + + final Object[] args = new Object[this.argumentCount]; + + long address = srcAddress + NULL_MARKER_LENGTH; + for(int i = 0; i < this.readers.length; i++) + { + address += this.readers[i].readValue(address, args, this.targets[i]); + } + + this.ownerWriter.writeValue(target, this.createValue(args)); + + return address; + } + + private Object createValue(final Object[] args) + { + try + { + return (Object)this.constructor.invokeExact(args); + } + catch(final Error e) + { + throw e; + } + catch(final Throwable t) + { + throw new BinaryPersistenceException( + "Could not construct inlined instance of " + this.valueType.getName(), t + ); + } + } + + } + + + + /////////////////////////////////////////////////////////////////////////// + // constructors // + ///////////////// + + private BinaryValueStructFunctions() + { + // static only + throw new UnsupportedOperationException(); + } + +} diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryValueTranslatorProvider.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryValueTranslatorProvider.java index a7bc2b3f..2a364454 100644 --- a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryValueTranslatorProvider.java +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryValueTranslatorProvider.java @@ -23,8 +23,14 @@ import org.eclipse.serializer.persistence.binary.exceptions.BinaryPersistenceException; import org.eclipse.serializer.persistence.types.PersistenceTypeDefinition; import org.eclipse.serializer.persistence.types.PersistenceTypeDefinitionMember; +import org.eclipse.serializer.persistence.types.PersistenceTypeDefinitionMemberFieldReflective; +import org.eclipse.serializer.persistence.types.PersistenceTypeDefinitionMemberFieldValueStruct; +import org.eclipse.serializer.persistence.types.PersistenceTypeDescriptionMemberFieldValueStruct; import org.eclipse.serializer.persistence.types.PersistenceTypeDescriptionMember; +import org.eclipse.serializer.persistence.types.PersistenceTypeDescriptionResolver; +import org.eclipse.serializer.persistence.types.PersistenceTypeDescriptionResolverProvider; import org.eclipse.serializer.persistence.types.PersistenceTypeHandler; +import org.eclipse.serializer.reflect.XReflect; import org.eclipse.serializer.typing.TypeMappingLookup; @@ -72,9 +78,21 @@ public BinaryValueSetter provideBinaryValueTranslator( PersistenceTypeHandler targetCurrentType, PersistenceTypeDefinitionMember targetMember ); - - - + + /** + * The resolver the refactoring rules come from, for a handler that reads a persisted layout itself + * instead of having a translator do it - a value class, whose instances are constructed rather than + * populated. + * + * @return the refactoring resolver, or {@code null} where none is configured. + */ + public default PersistenceTypeDescriptionResolver provideRefactoringResolver() + { + return null; + } + + + /** * Creates a new default {@link BinaryValueTranslatorProvider}. * @@ -91,11 +109,42 @@ public static BinaryValueTranslatorProvider New( final BinaryValueTranslatorLookupProvider translatorLookupProvider, final boolean switchByteOrder ) + { + return New( + customTranslatorLookup , + translatorKeyBuilders , + translatorLookupProvider, + null , + switchByteOrder + ); + } + + /** + * Creates a new default {@link BinaryValueTranslatorProvider} consulting the passed resolver for the + * members of an inlined layout, which the legacy type mapping does not reach: it pairs the inlined field + * as a whole, while the value type's own fields are members inside that pair. + * + * @param customTranslatorLookup optional map of registered custom translators keyed by lookup key, may be {@code null}. + * @param translatorKeyBuilders optional sequence of key builders to consult, may be {@code null} or empty. + * @param translatorLookupProvider the generic primitive-to-primitive translator table provider. + * @param typeDescriptionResolverProvider the refactoring resolver provider, may be {@code null}. + * @param switchByteOrder whether the persisted form uses a non-native byte order. + * + * @return the newly created provider. + */ + public static BinaryValueTranslatorProvider New( + final XGettingMap customTranslatorLookup , + final XGettingSequence translatorKeyBuilders , + final BinaryValueTranslatorLookupProvider translatorLookupProvider , + final PersistenceTypeDescriptionResolverProvider typeDescriptionResolverProvider, + final boolean switchByteOrder + ) { return new BinaryValueTranslatorProvider.Default( mayNull(customTranslatorLookup), unwrapKeyBuilders(translatorKeyBuilders), notNull(translatorLookupProvider), + mayNull(typeDescriptionResolverProvider), switchByteOrder ); } @@ -121,31 +170,34 @@ public final class Default implements BinaryValueTranslatorProvider // instance fields // //////////////////// - private final XGettingMap customTranslatorLookup ; - private final BinaryValueTranslatorKeyBuilder[] translatorKeyBuilders ; - private final BinaryValueTranslatorLookupProvider translatorLookupProvider; - private final boolean switchByteOrder ; - + private final XGettingMap customTranslatorLookup ; + private final BinaryValueTranslatorKeyBuilder[] translatorKeyBuilders ; + private final BinaryValueTranslatorLookupProvider translatorLookupProvider ; + private final PersistenceTypeDescriptionResolverProvider typeDescriptionResolverProvider; + private final boolean switchByteOrder ; + private transient TypeMappingLookup translatorLookup; - - - + + + /////////////////////////////////////////////////////////////////////////// // constructors // ///////////////// - + Default( - final XGettingMap customTranslatorLookup , - final BinaryValueTranslatorKeyBuilder[] translatorKeyBuilders , - final BinaryValueTranslatorLookupProvider translatorLookupProvider, - final boolean switchByteOrder + final XGettingMap customTranslatorLookup , + final BinaryValueTranslatorKeyBuilder[] translatorKeyBuilders , + final BinaryValueTranslatorLookupProvider translatorLookupProvider , + final PersistenceTypeDescriptionResolverProvider typeDescriptionResolverProvider, + final boolean switchByteOrder ) { super(); - this.customTranslatorLookup = customTranslatorLookup ; - this.translatorKeyBuilders = translatorKeyBuilders ; - this.translatorLookupProvider = translatorLookupProvider; - this.switchByteOrder = switchByteOrder ; + this.customTranslatorLookup = customTranslatorLookup ; + this.translatorKeyBuilders = translatorKeyBuilders ; + this.translatorLookupProvider = translatorLookupProvider ; + this.typeDescriptionResolverProvider = typeDescriptionResolverProvider; + this.switchByteOrder = switchByteOrder ; } @@ -166,6 +218,14 @@ private TypeMappingLookup translatorLookup() private BinaryValueSetter provideValueSkipper(final PersistenceTypeDefinitionMember sourceMember) { + if(sourceMember instanceof PersistenceTypeDescriptionMemberFieldValueStruct) + { + // an inlined slot is longer than the object id a reference would occupy here + return BinaryValueStructFunctions.provideSkipper( + (PersistenceTypeDescriptionMemberFieldValueStruct)sourceMember + ); + } + if(sourceMember.isReference()) { // skip the long-typed OID value @@ -184,7 +244,7 @@ private static void validateIsPrimitiveType(final PersistenceTypeDefinitionMembe final Class memberType = member.type(); if(memberType == null || !memberType.isPrimitive()) { - throw new BinaryPersistenceException("Unhandled type \"" + toTypedIdentifier(member) + "."); + throw new BinaryPersistenceException("Unhandled type " + toTypedIdentifier(member) + "."); } } @@ -220,7 +280,7 @@ private static BinaryValueSetter throwUnhandledPrimitiveException( ) { throw new BinaryPersistenceException( - "Unhandled primitive type \"" + toTypedIdentifier(sourceMember) + "." + "Unhandled primitive type " + toTypedIdentifier(sourceMember) + "." ); } @@ -275,7 +335,127 @@ private BinaryValueSetter provideReferenceResolver() { return BinaryValueFunctions.getObjectValueSetter(Object.class, this.switchByteOrder); } + + @Override + public PersistenceTypeDescriptionResolver provideRefactoringResolver() + { + return this.typeDescriptionResolverProvider == null + ? null + : this.typeDescriptionResolverProvider.provideTypeDescriptionResolver() + ; + } + + /** + * An inlined slot carries the field's content rather than an object id, so reading it means + * constructing the instance from that content. That is only possible while the described layout still + * matches the type's current one, member for member and name for name: the constructor takes every + * field, so a layout that has since gained or lost one cannot be invoked from what was written, and + * one whose members merely swapped places would have each of them take the other's value. + *

+ * A layout that has since gained or lost a member is translated member by member, matched by name: + * a gained one takes its type's default and a lost one is stepped over, which is the answer legacy + * mapping already gives for a referenced field. Defaulting the field as a whole is what would drop + * what was stored without saying so, and is not what happens here. + *

+ * Two changes are still refused, because placing the bytes would misread them rather than convert + * them: a member whose type changed, and a current form that is not inlined at all. + */ + private BinaryValueSetter provideInlinedValueTranslator( + final PersistenceTypeDefinitionMember sourceMember, + final PersistenceTypeDefinitionMember targetMember + ) + { + /* Derived from the target, not the source: the two describe the same layout by the same names + * here, but only the target is bound to the runtime field it has to be written through. A legacy + * member deliberately binds no field once its declaring type was renamed, and the value would + * then be written through nothing at all. + */ + if(sourceMember.equalsLayout(targetMember) + && targetMember instanceof PersistenceTypeDefinitionMemberFieldValueStruct + ) + { + return BinaryValueStructFunctions.provideSetter( + (PersistenceTypeDefinitionMemberFieldValueStruct)targetMember, + this.switchByteOrder + ); + } + + /* The layout differs, but both describe the same field of the same owner, paired by the legacy + * mapping, so their members can be matched by name, or by the refactoring rule naming one where + * the name changed: one the type has gained takes its default, one it has lost is stepped over. + * A member whose type changed is refused there, as is an unmapped loss plus gain. + */ + if(sourceMember instanceof PersistenceTypeDefinitionMemberFieldValueStruct + && targetMember instanceof PersistenceTypeDefinitionMemberFieldValueStruct + ) + { + return BinaryValueStructFunctions.provideEvolvingSetter( + (PersistenceTypeDefinitionMemberFieldValueStruct)sourceMember, + (PersistenceTypeDefinitionMemberFieldValueStruct)targetMember, + this.provideRefactoringResolver() , + this.switchByteOrder + ); + } + + throw new BinaryPersistenceException( + "The inlined layout of " + toTypedIdentifier(sourceMember) + " cannot be read into " + + toTypedIdentifier(targetMember) + ": the current form is not an inlined one, and an" + + " inlined slot carries no object id to reference the value by instead." + ); + } + /** + * A field whose type is a value class may be laid out inside its owner, where the field's memory + * offset does not address it: writing there misses the field and corrupts whatever is embedded at + * that place instead. The value is written through a handle on the field, the way the current + * handler's own setters do it. + *

+ * A reference source is resolved into the field. A persisted primitive widening into its own + * wrapper is boxed into it, because the offset-based translator for that pair boxes and writes in + * one step and so cannot be redirected by swapping the setter. Any other source form is refused + * rather than written blindly - writing it is precisely what this guards against. + * + * Only applicable where the translator writes into an instance, which is the reflective + * legacy path. The rewriting path hands its setters a {@literal null} target and an address to + * write to, and a handle writes into neither - it would fail on the null. That path cannot reach + * here today, its target members being generic rather than reflective, so this is stated rather + * than guarded: a guard would suggest the case is expected and leave the reader wondering what + * produces it. + * + * @param sourceMember the legacy member being read. + * @param targetMember the current member being written, whose type is a value class. + * @param switchByteOrder whether the persisted form has the opposite byte order. + * + * @return the setter writing the value into the target field. + */ + private static BinaryValueSetter provideValueTypeFieldSetter( + final PersistenceTypeDefinitionMember sourceMember , + final PersistenceTypeDefinitionMemberFieldReflective targetMember , + final boolean switchByteOrder + ) + { + if(sourceMember.isReference()) + { + return BinaryValueHandleFunctions.provideReferenceSetter(targetMember.field(), switchByteOrder); + } + + final BinaryValueSetter boxingSetter = BinaryValueHandleFunctions.provideBoxingSetter( + targetMember.field(), + sourceMember.type() , + switchByteOrder + ); + if(boxingSetter != null) + { + return boxingSetter; + } + + throw new BinaryPersistenceException( + "Cannot read " + toTypedIdentifier(sourceMember) + " into " + toTypedIdentifier(targetMember) + + ": a field whose type is a value class may be laid out inside its owner, so only a" + + " reference or a primitive widening into its own wrapper can be written into it." + ); + } + private static void validateIsReferenceType(final PersistenceTypeDescriptionMember member) { if(member.isReference()) @@ -284,7 +464,7 @@ private static void validateIsReferenceType(final PersistenceTypeDescriptionMemb } throw new BinaryPersistenceException( - "Non-reference type \"" + toTypedIdentifier(member) + "\" cannot be handled generically." + "Non-reference type " + toTypedIdentifier(member) + " cannot be handled generically." ); } @@ -295,12 +475,12 @@ private static void validateIsReferenceType(final Class type) return; } - throw new BinaryPersistenceException("Unhandled primitive type: \"" + type.getName() + "."); + throw new BinaryPersistenceException("Unhandled primitive type: \"" + type.getName() + "\"."); } private static String toTypedIdentifier(final PersistenceTypeDescriptionMember member) { - return member.typeName() + "\" of " + return "\"" + member.typeName() + "\" of " + PersistenceTypeDescriptionMember.class.getSimpleName() + " " + member.identifier() ; } @@ -364,8 +544,24 @@ public BinaryValueSetter provideTargetValueTranslator( return customValueSetter; } + if(sourceMember instanceof PersistenceTypeDescriptionMemberFieldValueStruct) + { + return this.provideInlinedValueTranslator(sourceMember, targetMember); + } + + if(targetMember instanceof PersistenceTypeDefinitionMemberFieldReflective + && XReflect.isValueClass(targetMember.type()) + ) + { + return provideValueTypeFieldSetter( + sourceMember, + (PersistenceTypeDefinitionMemberFieldReflective)targetMember, + this.switchByteOrder + ); + } + // note: see #validateCompatibleTargetType for target field type compatability validation. - + // check for generically handleable types on both sides final Class sourceType = sourceMember.type(); final Class targetType = targetMember.type(); @@ -386,11 +582,59 @@ public final BinaryValueSetter provideBinaryValueTranslator( final PersistenceTypeDefinitionMember targetMember ) { + /* Before the reference test, not after it: an inlined slot on either side is what this path + * cannot rewrite, and a reference source reaching the generic translator below would be + * reported as a primitive-versus-reference mismatch, which names neither the slot nor the + * reason. + */ + if(sourceMember instanceof PersistenceTypeDescriptionMemberFieldValueStruct) + { + if(targetMember == null) + { + // the current type no longer has the field: step over the slot, writing nothing + return this.provideValueSkipper(sourceMember); + } + + /* This path rewrites one binary form into another, so an unchanged slot is copied as it is. + * A changed one would have to be laid out anew, which is refused rather than reinterpreted: + * the bytes of one layout read as another are wrong values, not a conversion. + */ + if(sourceMember.equalsLayout(targetMember)) + { + return BinaryValueStructFunctions.provideRewriter( + sourceMember.persistentMinimumLength() + ); + } + + throw new BinaryPersistenceException( + "The inlined layout of " + toTypedIdentifier(sourceMember) + " changed and cannot be" + + " rewritten into " + toTypedIdentifier(targetMember) + ": a type whose instances are" + + " constructed rather than populated is read through its current layout, which the" + + " persisted bytes no longer describe." + ); + } + + if(targetMember instanceof PersistenceTypeDescriptionMemberFieldValueStruct) + { + /* Writing a slot means having the value it describes, and this path runs before any + * reference is resolved - it has no load handler and the referent does not exist yet. + * A type whose instances are constructed reads its members instead of being rewritten + * into, which is what BinaryLegacyTypeHandlerValueClass does; anything else has to be + * refused here rather than guessed. + */ + throw new BinaryPersistenceException( + "Field " + toTypedIdentifier(sourceMember) + " cannot be rewritten into the inlined slot " + + toTypedIdentifier(targetMember) + ": an inlined slot carries the content of a whole" + + " type, which a single persisted value does not, and the value it would need cannot" + + " be resolved before the referenced entities are loaded." + ); + } + if(sourceMember.isReference()) { return BinaryValueTranslators.provideReferenceValueBinaryTranslator(sourceMember, targetMember); } - + validateIsPrimitiveType(sourceMember); // target may be null (meaning the source member/field value shall be skipped) diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/ValueClassConstructorContract.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/ValueClassConstructorContract.java new file mode 100644 index 00000000..509f7203 --- /dev/null +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/ValueClassConstructorContract.java @@ -0,0 +1,493 @@ +package org.eclipse.serializer.persistence.binary.types; + +/*- + * #%L + * Eclipse Serializer Persistence Binary + * %% + * Copyright (C) 2023 - 2026 MicroStream Software + * %% + * This program and the accompanying materials are made + * available under the terms of the Eclipse Public License 2.0 + * which is available at https://www.eclipse.org/legal/epl-2.0/ + * + * SPDX-License-Identifier: EPL-2.0 + * #L% + */ + +import static org.eclipse.serializer.util.X.notNull; + +import java.lang.invoke.MethodHandle; +import java.lang.reflect.Field; + +import org.eclipse.serializer.collections.types.XGettingEnum; +import org.eclipse.serializer.persistence.binary.exceptions.BinaryPersistenceException; +import org.eclipse.serializer.persistence.binary.types.BinaryValueHandleFunctions.FieldReader; +import org.eclipse.serializer.persistence.exceptions.PersistenceExceptionTypeNotPersistable; +import org.eclipse.serializer.util.logging.Logging; +import org.slf4j.Logger; + +/** + * Verifies that a value class's constructor reproduces the field values it is handed, which is what the + * generic value class handling assumes and nothing else can establish. + *

+ * An identity instance is created blank and populated at its fields' memory offsets, so storing and + * loading address the same fields the same way and the two are symmetric by construction. A value + * instance cannot be populated - the JVM refuses to write its fields - so it is constructed instead, and + * the persisted form is read back by handing the stored field values to the constructor. That is the + * inverse of storing only if the constructor assigns each argument to the corresponding field unmodified. + *

+ * Nothing in the language states that it does. A constructor that swaps two same-typed parameters swaps + * the two values on every load, and one that computes a field from its argument drifts the value further + * on every store-and-load cycle. Both are silent, and neither can be recovered from afterwards, so the + * assumption is verified rather than trusted. + *

+ * It is settled in two stages, because neither alone covers everything: + *

    + *
  • {@link #probe()}, at handler creation, constructs throwaway instances from values of this class's + * own choosing. Handlers are built for every type the storage contains, so a violation surfaces when + * the storage is opened - before the application can store anything, and even if it only ever loads. + * Chosen values also discriminate where an application's own may not: all-default field values are a + * fixed point of many a wrong constructor.
    • + *
  • {@link #validate(Object)}, on the first instance actually stored, for what the probe could not + * answer - a constructor that rejects the probing values, or a layout holding a reference, whose type + * cannot be fabricated. It reports before any byte of that instance is written.
    • + *
+ * The two are complementary: the probe chooses good inputs but can be refused, the validation is always + * accepted but takes what it is given. Once either has answered, the check is a single field read. + *

+ * One contract per handler and per inlined slot, so a type reachable through several owners is verified + * once for each of them, which is cheaper than sharing a verdict through a static registry keyed by class + * would be. + * + * @see BinaryHandlerGenericValueClass + */ +final class ValueClassConstructorContract +{ + /////////////////////////////////////////////////////////////////////////// + // constants // + ////////////// + + private final static Logger logger = Logging.getLogger(ValueClassConstructorContract.class); + + + + /////////////////////////////////////////////////////////////////////////// + // static methods // + /////////////////// + + /** + * @param type the value class whose constructor is to be verified; must not be {@code null}. + * @param fields the persistable fields, in the order the constructor accepts them; + * must not be {@code null}. + * @param constructor the constructor as a {@code (Object[])Object} handle; must not be {@code null}. + * + * @return the contract for that type. + */ + static ValueClassConstructorContract New( + final Class type , + final XGettingEnum fields , + final MethodHandle constructor + ) + { + final ValueClassConstructorContract contract = new ValueClassConstructorContract( + notNull(type) , + notNull(fields).toArray(Field.class), + notNull(constructor) + ); + + contract.probe(); + + return contract; + } + + /** + * The value a parameter is given in one probing round. Two per type is all the scheme needs, since a + * round encodes one bit of the parameter's index - which is what lets {@code boolean}, with only two + * values to its name, be probed at all. + */ + private static Object sentinel(final Class type, final boolean high) + { + if(type == boolean.class) + { + return Boolean.valueOf(high); + } + + /* Deliberately 1 and 2 rather than the extremes: a constructor that range-checks its arguments + * accepts small positive values far more often, and a probe that is rejected answers nothing. + */ + if(type == byte.class) + { + return Byte.valueOf((byte)(high ? 2 : 1)); + } + if(type == short.class) + { + return Short.valueOf((short)(high ? 2 : 1)); + } + if(type == char.class) + { + return Character.valueOf((char)(high ? 2 : 1)); + } + if(type == int.class) + { + return Integer.valueOf(high ? 2 : 1); + } + if(type == long.class) + { + return Long.valueOf(high ? 2L : 1L); + } + if(type == float.class) + { + return Float.valueOf(high ? 2f : 1f); + } + if(type == double.class) + { + return Double.valueOf(high ? 2d : 1d); + } + + throw new BinaryPersistenceException("No sentinel for " + type.getName()); + } + + /** + * Whether parameter {@code index} takes the high sentinel in round {@code round}. + *

+ * The first {@code bits} rounds encode the parameter's index, counted from one so that no parameter + * keeps the same value throughout: two parameters differ in at least one bit, so a constructor that + * swaps them hands one of them the other's value in that round. The two rounds after those are all-low + * and all-high, which gives every parameter both values and is what catches a constructor that computes + * its field rather than storing it. + */ + private static boolean isHigh(final int index, final int round, final int bits) + { + if(round < bits) + { + return ((index + 1) >>> round & 1) == 1; + } + + return round != bits; + } + + private static boolean areAllPrimitive(final Field[] fields) + { + for(final Field field : fields) + { + if(!field.getType().isPrimitive()) + { + return false; + } + } + + return true; + } + + /** + * Whether the constructor placed the passed argument where it belongs. + *

+ * A primitive is compared by value: the constructor either stored what it was given or it did not, + * and a mismatch is a computed field, which drifts further on every cycle. + *

+ * A reference is compared by {@code ==}, which for a value-typed field is substitutability and for an + * identity-typed one is identity. Where that does not hold, the two cases are told apart rather than + * lumped together: the argument turning up in another field is a permutation and always wrong, + * as is a value that was dropped or conjured. Anything else means the constructor derived the field + * from its argument - a defensive copy being the common case - which cannot be distinguished from a + * faithful one by inspection and is therefore left to the author. + */ + private static boolean isFaithful( + final Class fieldType, + final Object[] arguments, + final Object rebuilt , + final int index + ) + { + final Object argument = arguments[index]; + + if(fieldType.isPrimitive()) + { + // a primitive argument is never null, the reader having boxed it + return argument.equals(rebuilt); + } + + if(argument == rebuilt) + { + return true; + } + + if(argument == null || rebuilt == null) + { + // dropped or conjured, which no derivation can excuse + return false; + } + + for(int i = 0; i < arguments.length; i++) + { + if(i != index && arguments[i] == rebuilt) + { + // this field received another field's argument: a permutation + return false; + } + } + + // derived from its argument, which cannot be judged from here + return true; + } + + + + /////////////////////////////////////////////////////////////////////////// + // instance fields // + //////////////////// + + private final Class type ; + private final Field[] fields ; + private final MethodHandle constructor; + + // resolved on first use and kept, the probe and the store-time fallback needing the same ones. + private FieldReader[] readers; + + /* Set once the contract is established, which it cannot stop being - and also once it has been + * established that it cannot be checked at all, since retrying would answer the same. See #validate. + */ + private boolean verified; + + + + /////////////////////////////////////////////////////////////////////////// + // constructors // + ///////////////// + + private ValueClassConstructorContract( + final Class type , + final Field[] fields , + final MethodHandle constructor + ) + { + super(); + this.type = type ; + this.fields = fields ; + this.constructor = constructor; + } + + + + /////////////////////////////////////////////////////////////////////////// + // methods // + //////////// + + /** + * Settles the contract at handler creation where it can be, by constructing throwaway instances from + * chosen values instead of waiting for one the application produces. + *

+ * Preferred over {@link #validate(Object)} for two reasons. It fails at startup - handlers are built + * for every type the storage contains, so a violation surfaces when the storage is opened rather than + * on the first store of that type, and it is caught even in an application that only ever loads. And + * its values are chosen to be discriminating, where an application's first instance may happen not to + * be: all-default field values are a fixed point of many a wrong constructor, so a real specimen can + * pass while proving nothing. + *

+ * Silent where it cannot answer - a constructor that rejects the values, or a field whose type has no + * sentinel - and {@link #validate(Object)} then remains as the net. The two are complementary: this one + * chooses good inputs but may be refused, that one is always accepted but takes what it is given. + * + * @throws PersistenceExceptionTypeNotPersistable if the constructor does not reproduce the values. + */ + private void probe() + { + /* Restricted to all-primitive layouts: a reference field would need two distinguishable instances + * of its own type, which cannot be fabricated for an arbitrary one. Every inlined layout qualifies + * by construction, its members being primitives or inlined themselves. + */ + if(!areAllPrimitive(this.fields)) + { + return; + } + + if(this.fields.length == 0) + { + // a layout with no members has nothing that could be misplaced + this.verified = true; + return; + } + + final FieldReader[] readers = this.resolveReaders(); + if(readers == null) + { + // reported once by #resolveReaders; nothing here can be checked + return; + } + + final int count = this.fields.length; + final int bits = Math.max(1, Integer.SIZE - Integer.numberOfLeadingZeros(count)); + + // the index-encoding rounds, then the all-low and all-high ones + for(int round = 0; round < bits + 2; round++) + { + final Object[] arguments = new Object[count]; + for(int i = 0; i < count; i++) + { + arguments[i] = sentinel(this.fields[i].getType(), isHigh(i, round, bits)); + } + + final Object rebuilt; + try + { + rebuilt = (Object)this.constructor.invokeExact(arguments); + } + catch(final Error e) + { + throw e; + } + catch(final Throwable t) + { + // the constructor refused the probing values, which says nothing about the contract + return; + } + + this.compare(readers, arguments, rebuilt, true); + } + + this.verified = true; + } + + /** + * Verifies the contract against the passed instance, unless it already has been - the fallback for + * what {@link #probe()} could not settle at handler creation. + *

+ * Deliberately unsynchronized: concurrent first stores verify the same immutable state and reach the + * same verdict, so the worst a race costs is a second verification. + * + * @param specimen an instance of the handled type, may be {@literal null}. + * + * @throws PersistenceExceptionTypeNotPersistable if the constructor does not reproduce the instance's + * field values. + */ + void validate(final Object specimen) + { + if(this.verified || specimen == null) + { + return; + } + + final FieldReader[] readers = this.resolveReaders(); + if(readers == null) + { + // reported once by #resolveReaders; nothing here can be checked + return; + } + + final Object[] arguments = new Object[readers.length]; + for(int i = 0; i < arguments.length; i++) + { + arguments[i] = readers[i].readValue(specimen); + } + + this.compare(readers, arguments, this.construct(arguments), false); + + this.verified = true; + } + + /** + * @param probing whether the values are this class's own rather than an instance the application + * produced, which the message has to say or it describes an instance nobody wrote. + */ + private void compare( + final FieldReader[] readers , + final Object[] arguments, + final Object rebuilt , + final boolean probing + ) + { + for(int i = 0; i < arguments.length; i++) + { + final Object rebuiltValue = readers[i].readValue(rebuilt); + if(!isFaithful(this.fields[i].getType(), arguments, rebuiltValue, i)) + { + throw this.violation( + (probing ? "constructed with test values, " : "") + + "field " + this.fields[i].getName() + " was given " + arguments[i] + + " but came back as " + rebuiltValue + + ". Its constructor must assign each argument to the corresponding field unmodified," + + " since loading hands it the stored field values" + , null + ); + } + } + } + + /** + * The readers for this type's fields, or {@literal null} where the type's module does not grant the + * access to read them. + *

+ * Resolved here rather than at handler creation, and answered with {@literal null} rather than a + * throw, because a type can be perfectly persistable while its fields cannot be read reflectively: a + * JDK value type opted into inlining is stored through memory offsets and needs no such access. Making + * the check a new reason for such a configuration to stop working would trade a silent defect for a + * loud regression. It is reported once, since an unverifiable contract is still worth knowing about. + */ + private FieldReader[] resolveReaders() + { + if(this.readers != null) + { + return this.readers; + } + + final FieldReader[] readers = new FieldReader[this.fields.length]; + + for(int i = 0; i < readers.length; i++) + { + try + { + readers[i] = BinaryValueHandleFunctions.provideFieldReader(this.fields[i]); + } + catch(final RuntimeException e) + { + logger.warn( + "Cannot verify that the constructor of value class {} reproduces the field values it is" + + " given, field {} not being readable: {}. A constructor that assigns an argument to" + + " another field, or computes a field from it, would change the stored values on every" + + " load without any indication.", + this.type.getName(), + this.fields[i].getName(), + e.getMessage() + ); + + // asking again would answer the same, so the contract stops trying + this.verified = true; + + return null; + } + } + + return this.readers = readers; + } + + private Object construct(final Object[] arguments) + { + try + { + return (Object)this.constructor.invokeExact(arguments); + } + catch(final Error e) + { + throw e; + } + catch(final Throwable t) + { + /* Rejecting the very values one of its own instances holds means no instance of this type + * could be loaded back, so it is the same contract violation reported one step earlier. + */ + throw this.violation("its constructor rejected the field values of one of its own instances", t); + } + } + + private PersistenceExceptionTypeNotPersistable violation(final String reason, final Throwable cause) + { + return new PersistenceExceptionTypeNotPersistable(this.type, + new BinaryPersistenceException( + "Value class " + this.type.getName() + " cannot be handled generically: " + reason + + ". Register a custom type handler for it.", + cause + ) + ); + } + +} diff --git a/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/Persistence.java b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/Persistence.java index 38876bb0..6ab8265a 100644 --- a/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/Persistence.java +++ b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/Persistence.java @@ -137,6 +137,14 @@ public class Persistence static final long START_CID_REAL = START_CID_BASE + 10_000L; // first 10K reserved for JLS constants static final long START_TID_REAL = START_TID_BASE + 1_000_000L; // first new type gets 1M1 assigned. + /* Read from XReflect rather than probed here, so the two cannot drift apart: this decides whether + * #registerJavaConstants registers the wrapper constants at all, and registering a single value + * instance throws IdentityException out of the registry's weak entry. That probe covers every + * wrapper type registered below, any one of which is enough - see #isValueClassEnabledRuntime for + * why "any" and not "all". + */ + static final boolean JAVA_CONSTANTS_ARE_VALUE_INSTANCES = XReflect.isValueClassEnabledRuntime(); + // CHECKSTYLE.OFF: ConstantName: type names are intentionally unchanged // java.lang and basic types. @@ -577,6 +585,15 @@ public static final , Long>> C iterateJavaBasicTyp */ public static final R registerJavaConstants(final R registry) { + /* Value class wrappers have no identity, so they can neither be weakly referenced by the + * registry nor looked up in it. Their constant ids are resolved arithmetically instead, + * see #resolveJavaConstantInstance and #lookupJavaConstantId. + */ + if(areJavaConstantsValueInstances()) + { + return registry; + } + long oidByte = START_CID_BYTE , oidBoolean = START_CID_BOOLEAN , @@ -610,6 +627,141 @@ public static final R registerJavaConstant return registry; } + /** + * Whether the JDK constants covered by {@link #registerJavaConstants(PersistenceObjectRegistry)} + * are value instances on the current JVM and therefore cannot be held by an object registry. + *

+ * The wrapper types are migrated as a group, so {@link Boolean} is representative for all of them. + * + * @return whether the JDK constants must be resolved arithmetically instead of via the registry. + * + * @see #resolveJavaConstantInstance(long) + * @see #lookupJavaConstantId(Object) + */ + public static final boolean areJavaConstantsValueInstances() + { + return JAVA_CONSTANTS_ARE_VALUE_INSTANCES; + } + + /** + * Resolves the constant id of a JDK constant covered by + * {@link #registerJavaConstants(PersistenceObjectRegistry)} arithmetically, i.e. without an object + * registry lookup. Constant ids are a deterministic function of type and value, so the mapping is + * identical to the registered one and stays stable across JVMs. + * + * @param instance the instance to look up, may be {@literal null}. + * + * @return the reserved constant id, or {@link Swizzling#notFoundId()} if the passed instance is + * not such a constant. + * + * @see #resolveJavaConstantInstance(long) + */ + public static final long lookupJavaConstantId(final Object instance) + { + if(instance instanceof Boolean) + { + return START_CID_BOOLEAN + (((Boolean)instance).booleanValue() ? 1L : 0L); + } + if(instance instanceof Byte) + { + // the whole byte value range is covered by the JLS cache range. + return START_CID_BYTE + ((Byte)instance).byteValue() - JSL_CACHE_INTEGER_START; + } + if(instance instanceof Short) + { + return cachedNumberConstantId(START_CID_SHORT, ((Short)instance).shortValue()); + } + if(instance instanceof Integer) + { + return cachedNumberConstantId(START_CID_INTEGER, ((Integer)instance).intValue()); + } + if(instance instanceof Long) + { + final long value = ((Long)instance).longValue(); + + return value >= JSL_CACHE_INTEGER_START && value < JSL_CACHE_INTEGER_BOUND + ? START_CID_LONG + value - JSL_CACHE_INTEGER_START + : Swizzling.notFoundId() + ; + } + if(instance instanceof Character) + { + final char value = ((Character)instance).charValue(); + + return value < JSL_CACHE_CHARACTER_BOUND + ? START_CID_CHARACTER + value - JSL_CACHE_CHARACTER_START + : Swizzling.notFoundId() + ; + } + + return Swizzling.notFoundId(); + } + + private static long cachedNumberConstantId(final long startCid, final long value) + { + return value >= JSL_CACHE_INTEGER_START && value < JSL_CACHE_INTEGER_BOUND + ? startCid + value - JSL_CACHE_INTEGER_START + : Swizzling.notFoundId() + ; + } + + /** + * Reverse of {@link #lookupJavaConstantId(Object)}: resolves the instance a reserved JDK constant + * id stands for. Note that the returned instance is the canonical cached one, so for value + * instances it is substitutable with any equal instance by definition. + * + * @param objectId the constant id to resolve. + * + * @return the constant instance, or {@literal null} if the passed id is not a reserved JDK + * constant id. + * + * @see #lookupJavaConstantId(Object) + */ + public static final Object resolveJavaConstantInstance(final long objectId) + { + // fast exit for the common case: all reserved JDK constant ids are inside this range. + if(objectId < START_CID_BYTE || objectId >= START_CID_REAL) + { + return null; + } + + long offset; + + if((offset = objectId - START_CID_BOOLEAN) >= 0 && offset < 2) + { + return offset == 0 ? Boolean.FALSE : Boolean.TRUE; + } + if(isCachedNumberOffset(offset = objectId - START_CID_BYTE)) + { + return Byte.valueOf((byte)(JSL_CACHE_INTEGER_START + offset)); + } + if(isCachedNumberOffset(offset = objectId - START_CID_SHORT)) + { + return Short.valueOf((short)(JSL_CACHE_INTEGER_START + offset)); + } + if(isCachedNumberOffset(offset = objectId - START_CID_INTEGER)) + { + return Integer.valueOf((int)(JSL_CACHE_INTEGER_START + offset)); + } + if(isCachedNumberOffset(offset = objectId - START_CID_LONG)) + { + return Long.valueOf(JSL_CACHE_INTEGER_START + offset); + } + if((offset = objectId - START_CID_CHARACTER) >= 0 + && offset < JSL_CACHE_CHARACTER_BOUND - JSL_CACHE_CHARACTER_START + ) + { + return Character.valueOf((char)(JSL_CACHE_CHARACTER_START + offset)); + } + + return null; + } + + private static boolean isCachedNumberOffset(final long offset) + { + return offset >= 0 && offset < JSL_CACHE_INTEGER_BOUND - JSL_CACHE_INTEGER_START; + } + /** diff --git a/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceFoundation.java b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceFoundation.java index d21fc813..41ea0a76 100644 --- a/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceFoundation.java +++ b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceFoundation.java @@ -253,6 +253,8 @@ public interface PersistenceFoundation> public PersistenceEagerStoringFieldEvaluator getReferenceFieldEagerEvaluator(); + public PersistenceValueInliningResolver getValueInliningResolver(); + public BufferSizeProviderIncremental getBufferSizeProvider(); public PersistenceFieldEvaluator getFieldEvaluatorPersistable(); @@ -434,6 +436,17 @@ public default F setTypeDictionaryIoHandler( public F setReferenceFieldEagerEvaluator(PersistenceEagerStoringFieldEvaluator evaluator); + /** + * Sets the resolver deciding which fields are written into their owner instead of being referenced by an + * object id. The default inlines every eligible field of an application-declared value type; pass + * {@link PersistenceValueInliningResolver#Disabled()} to keep every value field referenced. + * + * @param resolver the resolver to use. + * + * @return this foundation. + */ + public F setValueInliningResolver(PersistenceValueInliningResolver resolver); + public F setRootResolverProvider(PersistenceRootResolverProvider rootResolverProvider); public F setRootReferenceProvider(PersistenceRootReferenceProvider rootReferenceProvider); @@ -594,6 +607,7 @@ public class Default> private PersistenceFieldEvaluator fieldEvaluatorEnum ; private PersistenceFieldEvaluator fieldEvaluatorCollection ; private PersistenceEagerStoringFieldEvaluator eagerStoringFieldEvaluator ; + private PersistenceValueInliningResolver valueInliningResolver ; // (14.09.2018 TM)NOTE: that type handling stuff grows to a size where it could use its own foundation. private PersistenceTypeManager typeManager ; @@ -1291,7 +1305,18 @@ public PersistenceEagerStoringFieldEvaluator getReferenceFieldEagerEvaluator() return this.eagerStoringFieldEvaluator; } - + + @Override + public PersistenceValueInliningResolver getValueInliningResolver() + { + if(this.valueInliningResolver == null) + { + this.valueInliningResolver = this.dispatch(this.ensureValueInliningResolver()); + } + + return this.valueInliningResolver; + } + @Override public PersistenceRootResolverProvider getRootResolverProvider() { @@ -1966,6 +1991,13 @@ public F setReferenceFieldEagerEvaluator( this.eagerStoringFieldEvaluator = evaluator; return this.$(); } + + @Override + public F setValueInliningResolver(final PersistenceValueInliningResolver resolver) + { + this.valueInliningResolver = resolver; + return this.$(); + } @Override public F setRootResolverProvider(final PersistenceRootResolverProvider rootResolverProvider) @@ -2422,6 +2454,11 @@ protected PersistenceFieldEvaluator ensureFieldEvaluatorCollection() return Persistence.defaultFieldEvaluatorCollection(); } + protected PersistenceValueInliningResolver ensureValueInliningResolver() + { + return PersistenceValueInliningResolver.New(this.getTypeAnalyzer()); + } + protected PersistenceEagerStoringFieldEvaluator ensureReferenceFieldEagerEvaluator() { return Persistence.defaultReferenceFieldEagerEvaluator(); diff --git a/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceLegacyTypeHandlerCreator.java b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceLegacyTypeHandlerCreator.java index ba4ed20d..28ed2992 100644 --- a/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceLegacyTypeHandlerCreator.java +++ b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceLegacyTypeHandlerCreator.java @@ -194,7 +194,16 @@ public PersistenceLegacyTypeHandler createLegacyTypeHandler( return this.createTypeHandlerUnchangedInstanceStructure(result); } - if(result.currentTypeHandler() instanceof PersistenceTypeHandlerReflective) + /* A value class handler is reflective in how it derives its members, but its instances + * cannot be created empty and populated afterwards: it builds them from the persisted + * values in #create. The reflective legacy branch would hand it the LEGACY layout there + * and then populate fields, so such a handler must not take it. What it takes instead is + * decided one layer down: the binary creator intercepts it into a handler that reads the + * persisted layout directly and constructs from it, rewriting nothing. + */ + if(result.currentTypeHandler() instanceof PersistenceTypeHandlerReflective + && !result.currentTypeHandler().isValueClassType() + ) { final PersistenceLegacyTypeHandler reflectiveHandler = this.deriveReflectiveHandler( result, diff --git a/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceLegacyTypeHandlerWrapper.java b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceLegacyTypeHandlerWrapper.java index ad06716d..80efe220 100644 --- a/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceLegacyTypeHandlerWrapper.java +++ b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceLegacyTypeHandlerWrapper.java @@ -115,6 +115,13 @@ public void iterateLoadableReferences(final D data, final PersistenceReferenceLo this.typeHandler.iterateLoadableReferences(data, iterator); } + @Override + public void prepareLoadItem(final D data) + { + // the wrapped handler's layout is the persisted one in this case, so preparation applies as-is. + this.typeHandler.prepareLoadItem(data); + } + @Override public T create(final D data, final PersistenceLoadHandler handler) { @@ -144,6 +151,20 @@ public final Class type() { return this.typeHandler.type(); } + + @Override + public boolean isValueClassType() + { + // Must pass through all default methods to be a correct wrapper. + return this.typeHandler.isValueClassType(); + } + + @Override + public boolean isCreationDeferred() + { + // Must pass through all default methods to be a correct wrapper. + return this.typeHandler.isCreationDeferred(); + } diff --git a/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceManager.java b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceManager.java index e781bd6e..57d1a691 100644 --- a/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceManager.java +++ b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceManager.java @@ -15,6 +15,7 @@ */ import org.eclipse.serializer.collections.Set_long; +import org.eclipse.serializer.reference.Swizzling; import org.eclipse.serializer.util.BufferSizeProviderIncremental; import org.eclipse.serializer.util.X; @@ -507,13 +508,36 @@ public long currentObjectId() @Override public final long lookupObjectId(final Object object) { - return this.objectRegistry.lookupObjectId(object); + final long objectId = this.objectRegistry.lookupObjectId(object); + if(Swizzling.isProperId(objectId) || !Persistence.areJavaConstantsValueInstances()) + { + return objectId; + } + + /* A JDK constant that is a value instance is never registry-resident, but its id is + * deterministic and reserved, so the lookup can and must still answer it: it is the id + * every store references the constant by. Purely arithmetic, so the lock-free contract + * of this lookup is unaffected. + */ + final long constantId = Persistence.lookupJavaConstantId(object); + + return Swizzling.isFoundId(constantId) + ? constantId + : objectId + ; } @Override public final Object lookupObject(final long objectId) { - return this.objectRegistry.lookupObject(objectId); + final Object object = this.objectRegistry.lookupObject(objectId); + if(object != null || !Persistence.areJavaConstantsValueInstances()) + { + return object; + } + + // see #lookupObjectId: the reverse direction of the same reserved constant ids. + return Persistence.resolveJavaConstantInstance(objectId); } @Override diff --git a/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceObjectManager.java b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceObjectManager.java index 65597f2f..e4c82bcf 100644 --- a/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceObjectManager.java +++ b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceObjectManager.java @@ -264,7 +264,22 @@ public long lookupObjectId(final Object object) { synchronized(this.objectRegistry) { - return this.objectRegistry.lookupObjectId(object); + final long objectId = this.objectRegistry.lookupObjectId(object); + if(Swizzling.isProperId(objectId) || !Persistence.areJavaConstantsValueInstances()) + { + return objectId; + } + + /* A JDK constant that is a value instance is never registry-resident, but its id is + * deterministic and reserved, so the lookup can and must still answer it: it is the id + * every store references the constant by. + */ + final long constantId = Persistence.lookupJavaConstantId(object); + + return Swizzling.isFoundId(constantId) + ? constantId + : objectId + ; } } @@ -276,7 +291,14 @@ public Object lookupObject(final long objectId) // ); synchronized(this.objectRegistry) { - return this.objectRegistry.lookupObject(objectId); + final Object object = this.objectRegistry.lookupObject(objectId); + if(object != null || !Persistence.areJavaConstantsValueInstances()) + { + return object; + } + + // see #lookupObjectId: the reverse direction of the same reserved constant ids. + return Persistence.resolveJavaConstantInstance(objectId); } } @@ -306,6 +328,15 @@ public long ensureObjectId( long objectId; if(Swizzling.isNotProperId(objectId = this.objectRegistry.lookupObjectId(object))) { + /* Probed only after the registry misses: on a JVM where the JDK constants are value + * instances, they are never registry-resident, so the probe is needed - but only here, + * off the dominant already-registered path. + */ + if(Swizzling.isFoundId(objectId = this.synchRegisterJavaConstant(object, objectIdRequestor, optionalHandler))) + { + return objectId; + } + if(Swizzling.isNotProperId(objectId = this.synchCheckLocalRegistries(objectIdRequestor, object, optionalHandler))) { // see below about not globally registering the newly assigned objectId @@ -368,6 +399,11 @@ public long ensureObjectIdGuaranteedRegister( long objectId; if(Swizzling.isNotProperId(objectId = this.objectRegistry.lookupObjectId(object))) { + if(Swizzling.isFoundId(objectId = this.synchRegisterJavaConstant(object, objectIdRequestor, optionalHandler))) + { + return objectId; + } + if(Swizzling.isNotProperId(objectId = this.synchCheckLocalRegistries(objectIdRequestor, object, optionalHandler))) { objectId = this.oidProvider.provideNextObjectId(); @@ -381,6 +417,45 @@ public long ensureObjectIdGuaranteedRegister( } } + /** + * Assigns the reserved constant id of a JDK constant that cannot be registered because it is a + * value instance, and has the requestor skip it. + *

+ * Skipping is essential, not an optimization: constant ids are deliberately not resolvable as + * storage entities, so an entity must never be written for one. On a JVM that registers those + * constants, the registry lookup yields the same id and the same skipping, so the persisted + * form is identical either way. + * + * @param object the instance to check. + * @param objectIdRequestor the requestor to notify. + * @param optionalHandler the type handler responsible for {@code object}, may be {@literal null}. + * + * @param the instance type. + * + * @return the reserved constant id or {@link Swizzling#notFoundId()} if not applicable. + */ + private long synchRegisterJavaConstant( + final T object , + final PersistenceObjectIdRequestor objectIdRequestor, + final PersistenceTypeHandler optionalHandler + ) + { + if(!Persistence.areJavaConstantsValueInstances()) + { + return Swizzling.notFoundId(); + } + + final long constantId = Persistence.lookupJavaConstantId(object); + if(Swizzling.isNotFoundId(constantId)) + { + return Swizzling.notFoundId(); + } + + objectIdRequestor.registerSkippedOptional(constantId, object, optionalHandler); + + return constantId; + } + private long synchCheckLocalRegistries( final PersistenceObjectIdRequestor objectIdRequestor, final T instance , diff --git a/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceRootReference.java b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceRootReference.java index 4c200d73..34ad27a7 100644 --- a/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceRootReference.java +++ b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceRootReference.java @@ -18,7 +18,9 @@ import java.util.function.Supplier; +import org.eclipse.serializer.persistence.exceptions.PersistenceException; import org.eclipse.serializer.reference.Reference; +import org.eclipse.serializer.reflect.XReflect; /** * Mutable reference to the application's user-defined root instance. The reference itself is what the @@ -52,14 +54,47 @@ public default void set(final Object newRoot) * @param newRoot the new root instance; may be {@code null}. * * @return the previously referenced root (resolved via the previous supplier). + * + * @throws PersistenceException if {@code newRoot} is a value instance. */ public default Object setRoot(final Object newRoot) { + validateRootInstance(newRoot); + return this.setRootSupplier(() -> newRoot ); } + /** + * Guarantees that {@code rootInstance} can serve as an explicitly set root. + *

+ * A value instance cannot: setting a root means the persisted state is applied to the very + * instance the application holds, which is impossible without identity, so the persisted state + * would be dropped without any indication. Rejecting the root is the only way to not lose it + * silently. + *

+ * This applies to an explicitly set root only. A root that is merely loaded needs no state + * applied to it, so a persisted value root resolves like any other instance. + * + * @param rootInstance the instance to be validated; may be {@code null}. + * + * @throws PersistenceException if {@code rootInstance} is a value instance. + */ + public static void validateRootInstance(final Object rootInstance) + { + if(!XReflect.isValueInstance(rootInstance)) + { + return; + } + + throw new PersistenceException( + "A value instance cannot be used as a root: " + rootInstance.getClass().getName() + + " has no identity, so its persisted state could not be applied to it on loading." + + " Use an identity type as root and hold the value instance in one of its fields." + ); + } + /** * Replaces the supplier that resolves the root instance. Use this overload when the root cannot be * eagerly created (e.g. because doing so would trigger class-loading order issues). diff --git a/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceStoreHandler.java b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceStoreHandler.java index 23c5e711..dd114e0c 100644 --- a/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceStoreHandler.java +++ b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceStoreHandler.java @@ -15,6 +15,7 @@ */ import org.eclipse.serializer.reference.ObjectSwizzling; +import org.eclipse.serializer.reference.Swizzling; /** * Per-instance callback handed to {@link PersistenceTypeHandler}s during storing. Combines the storer-side @@ -79,6 +80,49 @@ public interface PersistenceStoreHandler extends PersistenceFunction, Storer */ public long applyEager(T instance, PersistenceTypeHandler localTypeHandler); + /** + * Variant of {@link #apply(Object)} for an instance the caller already knows an object id for: the + * instance is unchanged since it was stored under {@code knownObjectId}, and the entity of that id + * still exists in the target. Referencing it is then equivalent to storing it again, and avoids the + * superseded copy an identity-less instance would leave behind - it has no object registry entry, + * so {@link #apply(Object)} assigns it a fresh object id every single time. + *

+ * Passing an id is a statement about the caller's own state, so the implementation treats it like + * any other reference it writes without storing the referent: it reports the id for the target to + * validate, together with the instance, so a target that repairs a missing entity can re-store it. + *

+ * The id is ignored - and the instance applied - when it is {@link Swizzling#notFoundId()}, when the + * instance is {@literal null}, and whenever {@link #isEagerStoring()} holds, an eager store having + * to reach every instance regardless of what the caller knows about it. + * + * @param the instance type. + * @param instance the instance to reference, may be {@literal null}. + * @param knownObjectId the object id the instance was last stored under, + * {@link Swizzling#notFoundId()} if none is known. + * + * @return the object id to reference the instance by. + */ + public default long applyKnown(final T instance, final long knownObjectId) + { + return this.apply(instance); + } + + /** + * Whether this handler stores every encountered instance instead of skipping already known ones. + *

+ * Relevant to handlers that may skip storing a referent: skipping is only valid while the storing + * logic is lazy, since an eager store's purpose is to reach everything the referent references, + * whether the referent itself needs storing or not. + *

+ * The default implementation returns {@literal false}. + * + * @return whether every encountered instance is stored. + */ + public default boolean isEagerStoring() + { + return false; + } + @Override public void registerCommitListener(PersistenceCommitListener listener); diff --git a/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceTypeDefinitionMemberCreator.java b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceTypeDefinitionMemberCreator.java index 04a06141..05e1333c 100644 --- a/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceTypeDefinitionMemberCreator.java +++ b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceTypeDefinitionMemberCreator.java @@ -18,8 +18,10 @@ import java.lang.reflect.Field; +import org.eclipse.serializer.collections.BulkList; import org.eclipse.serializer.collections.XUtilsCollection; import org.eclipse.serializer.collections.types.XGettingSequence; +import org.eclipse.serializer.persistence.exceptions.PersistenceException; /** * Visitor that lifts a {@link PersistenceTypeDescriptionMember} into its runtime-bound @@ -84,6 +86,18 @@ public PersistenceTypeDefinitionMemberFieldReflective createDefinitionMember( PersistenceTypeDescriptionMemberFieldReflective description ); + /** + * Lifts an {@linkplain PersistenceTypeDescriptionMemberFieldValueStruct inlined field} into its + * definition counterpart, lifting the members describing the inlined layout along with it. + * + * @param description the description to lift. + * + * @return the runtime-bound definition member. + */ + public PersistenceTypeDefinitionMemberFieldValueStruct createDefinitionMember( + PersistenceTypeDescriptionMemberFieldValueStruct description + ); + /** * Lifts a {@linkplain PersistenceTypeDescriptionMemberFieldGenericSimple simple generic field} into * its definition counterpart, attempting to resolve the runtime type for diagnostic purposes. @@ -293,6 +307,50 @@ public PersistenceTypeDefinitionMemberFieldReflective createDefinitionMember( ); } + @Override + public PersistenceTypeDefinitionMemberFieldValueStruct createDefinitionMember( + final PersistenceTypeDescriptionMemberFieldValueStruct description + ) + { + final Class currentType = this.tryResolveCurrentType(description.typeName()); + + final String runtimeDeclaringType = this.resolveRuntimeTypeName(description.declaringTypeName()); + + // same rationale as for a plain reflective field: a renamed declaring type must not bind a field + final Field field = description.declaringTypeName().equals(runtimeDeclaringType) + ? this.resolveField(runtimeDeclaringType, description.name()) + : null + ; + + final BulkList members = BulkList.New(); + for(final PersistenceTypeDescriptionMemberField member : description.members()) + { + // double dispatch, so a nested member that is itself inlined lifts as one + final PersistenceTypeDefinitionMember lifted = member.createDefinitionMember(this); + if(!(lifted instanceof PersistenceTypeDefinitionMemberField)) + { + throw new PersistenceException( + "Inlined layout of " + description.identifier() + " contains a non-field member: " + + lifted.identifier() + ); + } + members.add((PersistenceTypeDefinitionMemberField)lifted); + } + + return PersistenceTypeDefinitionMemberFieldValueStruct.New( + runtimeDeclaringType , + field == null + ? null + : field.getDeclaringClass(), + field , + currentType , + description.typeName() , + description.name() , + description.declaringTypeName(), + members + ); + } + @Override public PersistenceTypeDefinitionMemberFieldGenericSimple createDefinitionMember( final PersistenceTypeDescriptionMemberFieldGenericSimple description diff --git a/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceTypeDefinitionMemberFieldValueStruct.java b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceTypeDefinitionMemberFieldValueStruct.java new file mode 100644 index 00000000..075b98f2 --- /dev/null +++ b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceTypeDefinitionMemberFieldValueStruct.java @@ -0,0 +1,182 @@ +package org.eclipse.serializer.persistence.types; + +/*- + * #%L + * Eclipse Serializer Persistence + * %% + * Copyright (C) 2023 - 2026 MicroStream Software + * %% + * This program and the accompanying materials are made + * available under the terms of the Eclipse Public License 2.0 + * which is available at https://www.eclipse.org/legal/epl-2.0/ + * + * SPDX-License-Identifier: EPL-2.0 + * #L% + */ + +import static org.eclipse.serializer.util.X.mayNull; +import static org.eclipse.serializer.util.X.notNull; + +import java.lang.reflect.Field; + +import org.eclipse.serializer.collections.types.XGettingSequence; + +/** + * Runtime-bound counterpart of {@link PersistenceTypeDescriptionMemberFieldValueStruct}: it additionally + * carries the owner's {@link Field} and the inlined type, and its {@link #members()} carry the inlined + * type's own fields, which is what the binary handler needs to read the inlined content out of an instance + * and to reconstruct one from it. + *

+ * The runtime bindings are optional, exactly as for a plain reflective field: a definition derived from a + * dictionary describes a layout that may no longer have a matching class or field. + * + * @see PersistenceTypeDescriptionMemberFieldValueStruct + */ +public interface PersistenceTypeDefinitionMemberFieldValueStruct +extends PersistenceTypeDescriptionMemberFieldValueStruct, PersistenceTypeDefinitionMemberFieldReflective +{ + @Override + public XGettingSequence members(); + + /** + * Full-control factory for an inlined field definition. Use {@link #New(Field, XGettingSequence)} for the + * common case where the field is known at runtime. + * + * @param runtimeDeclaringClass the runtime declaring-class name; may be {@code null}. + * @param declaringClass the runtime declaring class; may be {@code null}. + * @param field the runtime {@link Field}; may be {@code null}. + * @param type the runtime field type; may be {@code null}. + * @param typeName the dictionary type name; must not be {@code null}. + * @param name the simple field name; must not be {@code null}. + * @param declaringTypeName the dictionary declaring-type name; must not be {@code null}. + * @param members the inlined layout's members, in persistent order; must not be {@code null}. + * + * @return a new inlined field definition. + */ + public static PersistenceTypeDefinitionMemberFieldValueStruct New( + final String runtimeDeclaringClass, + final Class declaringClass , + final Field field , + final Class type , + final String typeName , + final String name , + final String declaringTypeName , + final XGettingSequence members + ) + { + return new PersistenceTypeDefinitionMemberFieldValueStruct.Default( + mayNull(runtimeDeclaringClass), + mayNull(declaringClass) , + mayNull(field) , + mayNull(type) , + notNull(typeName) , + notNull(name) , + notNull(declaringTypeName) , + notNull(members) + ); + } + + /** + * Convenience factory deriving every name and class binding from the passed {@link Field}. + * + * @param field the owner's field holding the inlined value; must not be {@code null}. + * @param members the inlined type's fields, in persistent order; must not be {@code null}. + * + * @return a new inlined field definition. + */ + public static PersistenceTypeDefinitionMemberFieldValueStruct New( + final Field field , + final XGettingSequence members + ) + { + return New( + field.getDeclaringClass().getName(), + field.getDeclaringClass() , + field , + field.getType() , + field.getType().getName() , + field.getName() , + field.getDeclaringClass().getName(), + members + ); + } + + public final class Default + extends PersistenceTypeDescriptionMemberFieldValueStruct.Default + implements PersistenceTypeDefinitionMemberFieldValueStruct + { + /////////////////////////////////////////////////////////////////////////// + // instance fields // + //////////////////// + + private final String runtimeDeclaringClassName; + private final Class declaringClass ; + private final Field field ; + private final Class type ; + + + + /////////////////////////////////////////////////////////////////////////// + // constructors // + ///////////////// + + Default( + final String runtimeDeclClassName, + final Class declaringClass , + final Field field , + final Class type , + final String typeName , + final String name , + final String declaringTypeName , + final XGettingSequence members + ) + { + super(typeName, declaringTypeName, name, members); + + this.runtimeDeclaringClassName = runtimeDeclClassName; + this.declaringClass = declaringClass ; + this.field = field ; + this.type = type ; + } + + + + /////////////////////////////////////////////////////////////////////////// + // override methods // + ///////////////////// + + @SuppressWarnings("unchecked") + @Override + public XGettingSequence members() + { + // the constructor accepts nothing but definition members, so the narrowing always holds + return (XGettingSequence)super.members(); + } + + @Override + public String runtimeDeclaringClassName() + { + return this.runtimeDeclaringClassName; + } + + @Override + public Class declaringClass() + { + return this.declaringClass; + } + + @Override + public Field field() + { + return this.field; + } + + @Override + public Class type() + { + return this.type; + } + + } + +} diff --git a/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceTypeDescriptionMember.java b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceTypeDescriptionMember.java index b2cf0eb4..732515cf 100644 --- a/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceTypeDescriptionMember.java +++ b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceTypeDescriptionMember.java @@ -158,7 +158,16 @@ public default boolean equalsStructure(final PersistenceTypeDescriptionMember ot */ public default boolean equalsLayout(final PersistenceTypeDescriptionMember other) { - return other != null && Objects.equals(this.typeName(), other.typeName()); + /* The lengths are part of the layout, not a consequence of the type name: the same type can be + * written either as an object id or as its own content embedded in the owner. Comparing only the + * name would let the two pass as layout-equal, and the legacy data would then be read with the + * wrong layout instead of being translated. + */ + return other != null + && Objects.equals(this.typeName(), other.typeName()) + && this.persistentMinimumLength() == other.persistentMinimumLength() + && this.persistentMaximumLength() == other.persistentMaximumLength() + ; } /** diff --git a/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceTypeDescriptionMemberAppender.java b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceTypeDescriptionMemberAppender.java index c091c268..ec9a02a8 100644 --- a/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceTypeDescriptionMemberAppender.java +++ b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceTypeDescriptionMemberAppender.java @@ -75,6 +75,13 @@ public interface PersistenceTypeDescriptionMemberAppender extends Consumer members = typeMember.members(); + final PersistenceTypeDescriptionMemberAppender appender = members.iterate( + new TypeDictionaryAppenderBuilder(this.vs, this.level + 1) + ).yield(); + members.iterate(appender); + this.indentMember(); + this.vs.add(MEMBER_COMPLEX_DEF_END); + } + @Override public void appendTypeMemberDescription(final PersistenceTypeDescriptionMemberPrimitiveDefinition typeMember) { diff --git a/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceTypeDescriptionMemberFieldValueStruct.java b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceTypeDescriptionMemberFieldValueStruct.java new file mode 100644 index 00000000..50a0cfd6 --- /dev/null +++ b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceTypeDescriptionMemberFieldValueStruct.java @@ -0,0 +1,296 @@ +package org.eclipse.serializer.persistence.types; + +/*- + * #%L + * Eclipse Serializer Persistence + * %% + * Copyright (C) 2023 - 2026 MicroStream Software + * %% + * This program and the accompanying materials are made + * available under the terms of the Eclipse Public License 2.0 + * which is available at https://www.eclipse.org/legal/epl-2.0/ + * + * SPDX-License-Identifier: EPL-2.0 + * #L% + */ + +import static org.eclipse.serializer.util.X.notNull; + +import org.eclipse.serializer.collections.types.XGettingSequence; +import org.eclipse.serializer.collections.types.XImmutableSequence; +import org.eclipse.serializer.persistence.exceptions.PersistenceException; + +/** + * A field whose value is not referenced by an object id but written into the owner's own binary form, + * described by a nested sequence of members that mirrors the field type's own persistent layout. + *

+ * The persistent form is a null marker byte followed by that layout, so the slot is fixed-length and the + * content after the marker is byte-identical to what the field type's own entity form would contain. A + * marker of {@link #NULL_MARKER_ABSENT} states that the field is {@code null}; the remaining bytes of the + * slot are then zero and carry no meaning. + *

+ * Inlining a field this way removes one entity, and with it one object id, per owner. It is only applicable + * to a field whose declared type is statically known to be exactly the type described here, since the + * persistent form carries no type information of its own. + * + * @see PersistenceTypeDescriptionMemberFieldReflective + */ +public interface PersistenceTypeDescriptionMemberFieldValueStruct +extends PersistenceTypeDescriptionMemberFieldReflective +{ + /////////////////////////////////////////////////////////////////////////// + // constants // + ////////////// + + /** + * The marker value stating that the inlined field is {@code null}. + */ + public static final byte NULL_MARKER_ABSENT = 0; + + /** + * The marker value stating that the inlined field holds a value. + */ + public static final byte NULL_MARKER_PRESENT = 1; + + /** + * The length of the null marker preceding the inlined content. + */ + public static final long NULL_MARKER_LENGTH = Byte.BYTES; + + + + /////////////////////////////////////////////////////////////////////////// + // static methods // + /////////////////// + + /** + * Calculates the fixed persistent length of an inlined slot holding the passed members: the null marker + * plus every member's own fixed length. + * + * @param members the members describing the inlined layout. + * + * @return the fixed persistent length of the slot. + * + * @throws PersistenceException if a member is not fixed-length, which would make the slot unskippable. + */ + public static long calculateStructLength( + final XGettingSequence members + ) + { + long length = NULL_MARKER_LENGTH; + + for(final PersistenceTypeDescriptionMember member : members) + { + if(!member.isFixedLength()) + { + throw new PersistenceException( + "Variable length member " + member.identifier() + " cannot be inlined." + ); + } + length += member.persistentMinimumLength(); + } + + return length; + } + + /** + * Validates that no member of an inlined layout holds a reference and returns the resulting + * {@code hasReferences} state, which is consequently always {@literal false}. + *

+ * An inlined slot is skipped as one fixed-length block wherever references are traversed, most + * critically by the storage garbage collector: an object id inside one would be invisible to it, + * so the entity it points to would be collected while still referenced. Nothing downstream can + * detect that, so a reference-bearing layout is refused wherever such a member is created, + * dictionary parsing included. + * + * @param name the described field's name, used for reporting. + * @param members the members describing the inlined layout. + * + * @return always {@literal false}. + * + * @throws PersistenceException if a member holds references. + */ + public static boolean validateNoReferences( + final String name , + final XGettingSequence members + ) + { + for(final PersistenceTypeDescriptionMember member : members) + { + if(member.hasReferences()) + { + throw new PersistenceException( + "Member " + member.identifier() + " of the inlined field " + name + " holds references," + + " which an inlined slot must not: they would be invisible to reference traversal," + + " the storage garbage collector included." + ); + } + } + + return false; + } + + + + /////////////////////////////////////////////////////////////////////////// + // methods // + //////////// + + /** + * The ordered sequence of members describing the inlined layout, in the same order the field type's own + * entity form uses. + * + * @return the nested members. + */ + public XGettingSequence members(); + + @Override + public default boolean equalsDescription(final PersistenceTypeDescriptionMember other) + { + // does NOT call #equalsStructure to avoid redundant member iteration + return PersistenceTypeDescriptionMember.equalTypeAndNameAndQualifier(this, other) + && other instanceof PersistenceTypeDescriptionMemberFieldValueStruct + && PersistenceTypeDescriptionMember.equalDescriptions( + this.members(), + ((PersistenceTypeDescriptionMemberFieldValueStruct)other).members() + ) + ; + } + + @Override + public default boolean equalsStructure(final PersistenceTypeDescriptionMember other) + { + return PersistenceTypeDescriptionMemberFieldReflective.super.equalsStructure(other) + && other instanceof PersistenceTypeDescriptionMemberFieldValueStruct + && PersistenceTypeDescriptionMember.equalStructures( + this.members(), + ((PersistenceTypeDescriptionMemberFieldValueStruct)other).members() + ) + ; + } + + /** + * Unlike every other member kind, the inlined layout's own members are compared by name as well as by + * type, which is what {@link PersistenceTypeDescriptionMember#equalsStructure} does. + *

+ * Ignoring the name is safe where the legacy mapping pairs the members by name separately and only the + * bytes still have to be confirmed, which is the case for the member this one is - the field of its + * owner. Nothing pairs the members inside the layout, though: they would be paired by position + * alone, so two same-typed fields that swapped places would read layout-equal and each take the other's + * value with nothing said. The member's own name stays out of it, as it does everywhere else. + */ + @Override + public default boolean equalsLayout(final PersistenceTypeDescriptionMember other) + { + return PersistenceTypeDescriptionMemberFieldReflective.super.equalsLayout(other) + && other instanceof PersistenceTypeDescriptionMemberFieldValueStruct + && PersistenceTypeDescriptionMember.equalStructures( + this.members(), + ((PersistenceTypeDescriptionMemberFieldValueStruct)other).members() + ) + ; + } + + @Override + public default PersistenceTypeDefinitionMemberFieldValueStruct createDefinitionMember( + final PersistenceTypeDefinitionMemberCreator creator + ) + { + return creator.createDefinitionMember(this); + } + + /** + * Creates an inlined field description. + * + * @param typeName the inlined field type's name; must not be {@code null}. + * @param declaringTypeName the fully qualified name of the declaring class; must not be {@code null}. + * @param name the field's simple name; must not be {@code null}. + * @param members the members describing the inlined layout; must not be {@code null}. + * + * @return a new inlined field description. + */ + public static PersistenceTypeDescriptionMemberFieldValueStruct New( + final String typeName , + final String declaringTypeName, + final String name , + final XGettingSequence members + ) + { + return new PersistenceTypeDescriptionMemberFieldValueStruct.Default( + notNull(typeName) , + notNull(declaringTypeName), + notNull(name) , + notNull(members) + ); + } + + public class Default + extends PersistenceTypeDescriptionMemberField.Abstract + implements PersistenceTypeDescriptionMemberFieldValueStruct + { + /////////////////////////////////////////////////////////////////////////// + // instance fields // + //////////////////// + + final XImmutableSequence members; + + + + /////////////////////////////////////////////////////////////////////////// + // constructors // + ///////////////// + + protected Default( + final String typeName , + final String declaringTypeName, + final String name , + final XGettingSequence members + ) + { + // the slot is fixed length, so the same value is both bounds; walking the members twice for it + // would be pointless, and a super() call cannot be preceded by a local at this language level. + this(typeName, declaringTypeName, name, members, calculateStructLength(members)); + } + + private Default( + final String typeName , + final String declaringTypeName, + final String name , + final XGettingSequence members, + final long structLength + ) + { + super( + typeName , + declaringTypeName, + name , + false , // not a reference: the content is inlined, not pointed to + false , // not a primitive: it is a composite of its own members + validateNoReferences(name, members), + structLength , + structLength + ); + this.members = members.immure(); + } + + + + /////////////////////////////////////////////////////////////////////////// + // methods // + //////////// + + @Override + public XGettingSequence members() + { + return this.members; + } + + @Override + public void assembleTypeDescription(final PersistenceTypeDescriptionMemberAppender assembler) + { + assembler.appendTypeMemberDescription(this); + } + + } + +} diff --git a/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceTypeDictionary.java b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceTypeDictionary.java index 2d66864e..c32a045b 100644 --- a/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceTypeDictionary.java +++ b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceTypeDictionary.java @@ -869,12 +869,14 @@ public class Symbols protected static final transient String KEYWORD_PRIMITIVE = "primitive"; protected static final transient String KEYWORD_ENUM = XReflect.typename_enum(); + protected static final transient String KEYWORD_VALUE = "value" ; protected static final transient String TYPE_CHARS = "[char]" ; protected static final transient String TYPE_BYTES = "[byte]" ; protected static final transient String TYPE_COMPLEX = "[list]" ; protected static final transient char[] ARRAY_KEYWORD_PRIMITIVE = KEYWORD_PRIMITIVE.toCharArray(); protected static final transient char[] ARRAY_KEYWORD_ENUM = KEYWORD_ENUM .toCharArray(); + protected static final transient char[] ARRAY_KEYWORD_VALUE = KEYWORD_VALUE .toCharArray(); protected static final transient char[] ARRAY_TYPE_CHARS = TYPE_CHARS .toCharArray(); protected static final transient char[] ARRAY_TYPE_BYTES = TYPE_BYTES .toCharArray(); protected static final transient char[] ARRAY_TYPE_COMPLEX = TYPE_COMPLEX .toCharArray(); diff --git a/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceTypeDictionaryParser.java b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceTypeDictionaryParser.java index 5d4e0696..46bf6cec 100644 --- a/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceTypeDictionaryParser.java +++ b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceTypeDictionaryParser.java @@ -20,6 +20,7 @@ import org.eclipse.serializer.collections.BulkList; import org.eclipse.serializer.collections.types.XGettingSequence; import org.eclipse.serializer.math.XMath; +import org.eclipse.serializer.persistence.exceptions.PersistenceException; import org.eclipse.serializer.persistence.exceptions.PersistenceExceptionParser; import org.eclipse.serializer.persistence.exceptions.PersistenceExceptionParserIncompleteInput; import org.eclipse.serializer.persistence.exceptions.PersistenceExceptionParserMissingComplexTypeDefinition; @@ -373,10 +374,122 @@ private static int parseTypeMember( { return i; } - + + i = parseValueStructDefinition(input, iStart, iBound, member); + if(i != iStart) + { + return i; + } + return parseFieldDefinition(input, iStart, iBound, member); } + private static int parseValueStructDefinition( + final char[] input , + final int iStart, + final int iBound, + final AbstractMemberBuilder member + ) + { + if(!equalsCharSequence(input, iStart, iBound, ARRAY_KEYWORD_VALUE)) + { + return iStart; + } + + /* The keyword must be followed by a separating whitespace. Without that check, a field whose type + * name merely starts with the keyword (a package named "value", say) would be misparsed. + */ + final int afterKeyword = iStart + ARRAY_KEYWORD_VALUE.length; + if(afterKeyword >= iBound || input[afterKeyword] > ' ') + { + return iStart; + } + + member.isValueStruct = true; + + int i = skipWhitespaces(input, afterKeyword, iBound); + i = parseFieldTypeName(input, i, iBound, member); + i = parseValueStructMemberName(input, i, iBound, member); + + final NestedMemberBuilder nestedMemberBuilder = member.createNestedMemberBuilder(); + while(i < iBound) + { + if(input[i] == MEMBER_COMPLEX_DEF_END) + { + break; + } + + /* A member of the inlined layout may be inlined itself, which the appender writes by the same + * dispatch it writes the outer one by. Reading it needs the same recursion, or a dictionary + * this parser's own assembler produced could not be read back. + */ + final int afterNested = parseValueStructDefinition(input, i, iBound, nestedMemberBuilder); + if(afterNested != i) + { + i = afterNested; + member.structMembers.add(nestedMemberBuilder.buildMemberValueStruct()); + } + else + { + i = parseFieldSimple(input, i, iBound, nestedMemberBuilder); + member.structMembers.add(nestedMemberBuilder.buildMemberField()); + } + nestedMemberBuilder.reset(); + + // skip trailing whitespaces before bound is checked again + i = skipWhitespaces(input, i, iBound); + } + checkForIncompleteInput(i, iBound); + + // +1 to skip the inlined layout's end + return parseMemberTermination(input, i + 1, iBound); + } + + private static int parseValueStructMemberName( + final char[] input , + final int iStart, + final int iBound, + final AbstractMemberBuilder member + ) + { + int i = skipTerminated(input, iStart, iBound, MEMBER_FIELD_QUALIFIER_SEPERATOR, MEMBER_COMPLEX_DEF_START, TYPE_END); + if(i == iStart) + { + throw new PersistenceExceptionParserMissingMemberName(i); + } + + int a = skipWhitespaces(input, i, iBound); + checkForIncompleteInput(a, iBound); + + // the owner's field may be qualified by its declaring type, exactly as a plain field is + if(input[a] == MEMBER_FIELD_QUALIFIER_SEPERATOR) + { + member.setQualifier(new String(input, iStart, i - iStart)); + + final int nameStart = skipWhitespaces(input, a + 1, iBound); + i = skipTerminated(input, nameStart, iBound, MEMBER_COMPLEX_DEF_START, TYPE_END, MEMBER_TERMINATOR); + if(i == nameStart) + { + throw new PersistenceExceptionParserMissingMemberName(i); + } + setFieldName(input, nameStart, i, member); + + a = skipWhitespaces(input, i, iBound); + checkForIncompleteInput(a, iBound); + } + else + { + setFieldName(input, iStart, i, member); + } + + if(input[a] != MEMBER_COMPLEX_DEF_START) + { + throw new PersistenceExceptionParserMissingComplexTypeDefinition(a); + } + + return skipWhitespaces(input, a + 1, iBound); + } + private static int parseFieldTypeName( final char[] input , final int iStart, @@ -959,8 +1072,11 @@ abstract class AbstractMemberBuilder final PersistenceTypeNameMapper typeNameMapper ; final BulkList nestedMembers = new BulkList<>(); - - boolean isVariableLength, isComplex; + + // members describing the layout of an inlined field, filled only while such a member is parsed + final BulkList structMembers = new BulkList<>(); + + boolean isVariableLength, isComplex, isValueStruct; String qualifier, originalTypeName, typeName, fieldName; @@ -1047,10 +1163,12 @@ AbstractMemberBuilder reset() this.qualifier = null ; this.isVariableLength = false; this.isComplex = false; + this.isValueStruct = false; this.originalTypeName = null ; this.typeName = null ; this.fieldName = null ; this.nestedMembers.clear(); + this.structMembers.clear(); return this; } @@ -1101,6 +1219,54 @@ final PersistenceTypeDescriptionMemberFieldGeneric buildGenericFieldMember() ); } + final PersistenceTypeDescriptionMemberField buildMemberField() + { + final String qualifier = this.qualifier(); + final Class resolvableDeclaringType = qualifier == null + ? null + : this.typeResolver.tryResolveType(qualifier) + ; + + /* If the qualifier is resolvable to a class, the field is reflectivly derivable. + * If not, it must be a generic simple field. Even if that field was reflectively derived in the past. + * It is not (or no longer), it cannot be handled reflectively, but only generically. + * And, of course, should a type formerly being a class have changed to being an interface, + * the field cannot be handled reflectively, either. Whatever future improvements might come to interfaces + * (like protected methods, since they already can have private ones, now), it should be pretty much + * out of the question that they could possibly ever have instance fields. That would make them identical + * to classes and defeat their purpose of implementing multiple inheritance. + */ + + final String tName = this.typeName(); + final boolean isPrim = XReflect.isPrimitiveTypeName(tName); + final String fName = this.fieldName(); + final long minLen = this.resolveMemberMinimumLength(); + final long maxLen = this.resolveMemberMaximumLength(); + + return resolvableDeclaringType != null && !resolvableDeclaringType.isInterface() + ? PersistenceTypeDescriptionMemberFieldReflective.New(tName, qualifier, fName, !isPrim, minLen, maxLen) + : PersistenceTypeDescriptionMemberFieldGenericSimple.New(tName, qualifier, fName, !isPrim, minLen, maxLen) + ; + } + + final PersistenceTypeDescriptionMemberFieldValueStruct buildMemberValueStruct() + { + if(this.qualifier() == null) + { + // an inlined field is a field of a class, so its declaring type is part of its identity + throw new PersistenceException( + "Inlined field " + this.fieldName() + " lacks its declaring type qualifier." + ); + } + + return PersistenceTypeDescriptionMemberFieldValueStruct.New( + this.typeName() , + this.qualifier(), + this.fieldName(), + this.structMembers + ); + } + abstract long resolveMemberMinimumLength(); abstract long resolveMemberMaximumLength(); @@ -1173,6 +1339,11 @@ final PersistenceTypeDescriptionMember buildTypeMember() return this.buildMemberEnumConstant(); } + if(this.isValueStruct) + { + return this.buildMemberValueStruct(); + } + if(!this.isVariableLength) { return this.buildMemberField(); @@ -1200,37 +1371,6 @@ final PersistenceTypeDescriptionMemberEnumConstant buildMemberEnumConstant() this.fieldName() ); } - - final PersistenceTypeDescriptionMemberField buildMemberField() - { - final String qualifier = this.qualifier(); - final Class resolvableDeclaringType = qualifier == null - ? null - : this.typeResolver.tryResolveType(qualifier) - ; - - /* If the qualifier is resolvable to a class, the field is reflectivly derivable. - * If not, it must be a generic simple field. Even if that field was reflectively derived in the past. - * It is not (or no longer), it cannot be handled reflectively, but only generically. - * And, of course, should a type formerly being a class have changed to being an interface, - * the field cannot be handled reflectively, either. Whatever future improvements might come to interfaces - * (like protected methods, since they already can have private ones, now), it should be pretty much - * out of the question that they could possibly ever have instance fields. That would make them identical - * to classes and defeat their purpose of implementing multiple inheritance. - */ - - final String tName = this.typeName(); - final boolean isPrim = XReflect.isPrimitiveTypeName(tName); - final String fName = this.fieldName(); - final long minLen = this.resolveMemberMinimumLength(); - final long maxLen = this.resolveMemberMaximumLength(); - - return resolvableDeclaringType != null && !resolvableDeclaringType.isInterface() - ? PersistenceTypeDescriptionMemberFieldReflective.New(tName, qualifier, fName, !isPrim, minLen, maxLen) - : PersistenceTypeDescriptionMemberFieldGenericSimple.New(tName, qualifier, fName, !isPrim, minLen, maxLen) - ; - } - /////////////////////////////////////////////////////////////////////////// diff --git a/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceTypeHandler.java b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceTypeHandler.java index ff2ffada..987a6ed9 100644 --- a/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceTypeHandler.java +++ b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceTypeHandler.java @@ -85,6 +85,43 @@ public interface PersistenceTypeHandler extends PersistenceTypeDefinition, @Override public Class type(); + /** + * Whether this handler's instances are value instances (JEP 401), i.e. have no identity. + *

+ * Such instances must never be registered in a {@link PersistenceObjectRegistry}: it holds its + * entries by {@link java.lang.ref.WeakReference}, which is not applicable to them. They are + * consequently assigned a new objectId per store and reconstructed per load, which is invisible + * to the application because equal value instances are indistinguishable by definition. + *

+ * The default implementation returns {@literal false}, so handlers that cannot or need not + * determine their type are treated as identity handlers. + * + * @return whether instances handled by this handler have no identity. + */ + public default boolean isValueClassType() + { + return false; + } + + /** + * Whether {@link #create} may only be called once every reference it resolves is resolvable, i.e. + * after all of a load's build items exist. + *

+ * Required by a handler that builds its instances from resolved references instead of creating + * them blank and populating them afterwards: a value instance cannot be populated at all, and an + * identity instance whose creation needs another instance's state (not merely the + * reference) must wait until that instance can be provided completely. + *

+ * The default implementation returns {@link #isValueClassType()}, since a value class handler is + * exactly such a handler. + * + * @return whether the loader must defer this handler's instance creation. + */ + public default boolean isCreationDeferred() + { + return this.isValueClassType(); + } + /** * Tests whether the passed entity type is valid input for this handler. Default implementation * accepts {@code type} when it is assignable to {@link #type()} — i.e. a sub-type of the @@ -165,10 +202,38 @@ public default void validateEntityType(final Class type) */ public void store(D data, T instance, long objectId, PersistenceStoreHandler handler); + /** + * Prepares the passed persisted form before anything else reads it, in particular before + * {@link #iterateLoadableReferences} determines which references have to be loaded. + *

+ * Required by handlers that rewrite the persisted data, e.g. a legacy handler translating an + * outdated layout into the current one: the reference offsets those handlers report belong to + * the rewritten data, so the rewrite may not be deferred to {@link #create}. + *

+ * The default implementation does nothing. + * + * @param data the persisted form of an instance of the handled type. + */ + public default void prepareLoadItem(final D data) + { + // no preparation needed by default. + } + /** * Allocates and returns a fresh, uninitialized {@code T} from the passed persisted form. The * returned instance must already be of the right runtime type but does not yet need its fields * populated — field population happens in {@link #initializeState} or {@link #updateState}. + *

+ * References must not be resolved here. This method is called for every entity of a load + * before any of them is populated, so {@link PersistenceLoadHandler#lookupObject} would return + * {@literal null} for anything not yet created. Resolving a reference is only valid from + * {@link #initializeState} / {@link #updateState} onwards. + *

+ * The one exception is a handler reporting {@link #isCreationDeferred()}: it builds its instances + * from resolved references, so the loader defers its creation until they can be resolved. Note + * that a resolved reference is guaranteed to exist but not to be populated yet, unless its own + * handler reports {@link #isCreationDeferred()} as well or creates complete instances to begin + * with. * * @param data the persisted form to read identity-information from. * @param handler receives nested-reference resolution requests. @@ -505,9 +570,12 @@ protected static final String deriveTypeName(final Class type) // basic type swizzling // private final Class type; - + // differs from Class#getName to properly identify synthetic classes instead using of those "$1,2,3..." names. private final String typeName; + + // determined once since it is queried per stored and per loaded instance. See #isValueClassType. + private final boolean isValueClassType; // effectively final / immutable: gets only initialized once later on and is never mutated again. initially 0. private long typeId = Swizzling.notFoundId(); @@ -526,8 +594,9 @@ protected Abstract(final Class type) protected Abstract(final Class type, final String typeName) { super(); - this.type = notNull(type) ; - this.typeName = notNull(typeName); + this.type = notNull(type) ; + this.typeName = notNull(typeName) ; + this.isValueClassType = XReflect.isValueClass(type); } @@ -551,6 +620,12 @@ public final Class type() return this.type; } + @Override + public final boolean isValueClassType() + { + return this.isValueClassType; + } + @Override public final long typeId() { diff --git a/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceValueInliningResolver.java b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceValueInliningResolver.java new file mode 100644 index 00000000..713a52ff --- /dev/null +++ b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceValueInliningResolver.java @@ -0,0 +1,297 @@ +package org.eclipse.serializer.persistence.types; + +/*- + * #%L + * Eclipse Serializer Persistence + * %% + * Copyright (C) 2023 - 2026 MicroStream Software + * %% + * This program and the accompanying materials are made + * available under the terms of the Eclipse Public License 2.0 + * which is available at https://www.eclipse.org/legal/epl-2.0/ + * + * SPDX-License-Identifier: EPL-2.0 + * #L% + */ + +import static org.eclipse.serializer.util.X.notNull; + +import java.lang.reflect.Field; +import java.util.function.BiPredicate; + +import org.eclipse.serializer.collections.HashEnum; +import org.eclipse.serializer.collections.types.XGettingEnum; +import org.eclipse.serializer.reflect.XReflect; +import org.eclipse.serializer.util.logging.Logging; +import org.slf4j.Logger; + +/** + * Decides whether an owner's field is written into the owner's own binary form instead of being referenced + * by an object id, and supplies the inlined type's fields when it is. + *

+ * An object id is a persistent identity, and a value has none, so referencing one says something about it + * that is not true. The cost follows from that: with no object registry entry the field is assigned a new + * object id and stored again on every store of its owner, leaving a superseded copy behind each time. + * Writing it into the owner instead is the representation that matches what it is, which is why + * {@link #New(PersistenceTypeAnalyzer)} is the default. + *

+ * It cannot be the only representation. A field declared as {@code Object} or as an interface, and every + * collection element, has no static type to inline against, and an inlined slot carries no type of its own. + * Those keep the referenced form, so the two coexist within one owner. + *

+ * What it costs is evolution: the owner's layout embeds the inlined type's layout, so changing that type + * changes every owner that inlines it. + * + * @see PersistenceTypeDescriptionMemberFieldValueStruct + */ +@FunctionalInterface +public interface PersistenceValueInliningResolver +{ + /////////////////////////////////////////////////////////////////////////// + // constants // + ////////////// + + static final Logger logger = Logging.getLogger(PersistenceValueInliningResolver.class); + + + + /** + * Decides whether the passed field is inlined into its owner. + * + * @param ownerType the class declaring the field. + * @param field the field to decide about. + * + * @return the inlined type's persistable fields, or {@code null} if the field is not inlined. + */ + public XGettingEnum resolveInlinedFields(Class ownerType, Field field); + + /** + * @return a resolver that inlines nothing. + */ + public static PersistenceValueInliningResolver Disabled() + { + return (ownerType, field) -> null; + } + + /** + * Creates the default resolver: it inlines every eligible field whose type the application itself + * declares, and leaves the types the JDK declares referenced. + *

+ * A JDK value type is excluded because its persistent form is not the application's to decide: several + * have custom type handlers, and the cached instances among them are persisted under reserved constant + * ids ({@link Persistence}), both of which inlining would bypass. Opting one in is possible through + * {@link #New(PersistenceTypeAnalyzer, BiPredicate)}, but it changes the layout of every owner that + * has such a field, so it is a decision to take deliberately. + * + * @param typeAnalyzer the analyzer determining a type's persistable fields; must not be {@code null}. + * + * @return the new resolver. + */ + public static PersistenceValueInliningResolver New(final PersistenceTypeAnalyzer typeAnalyzer) + { + return New(typeAnalyzer, (ownerType, field) -> isApplicationDeclared(field.getType())); + } + + /** + * @param type the type to test. + * + * @return whether the type is declared by the application rather than by the JDK. + */ + public static boolean isApplicationDeclared(final Class type) + { + final ClassLoader loader = type.getClassLoader(); + + return loader != null && loader != ClassLoader.getPlatformClassLoader(); + } + + /** + * @return whether the type has a constructor accepting the passed fields that can actually be invoked. + */ + static boolean isConstructorInvocable(final Class type, final XGettingEnum fields) + { + final Class[] parameterTypes = new Class[fields.intSize()]; + + int i = 0; + for(final Field field : fields) + { + parameterTypes[i++] = field.getType(); + } + + try + { + return type.getDeclaredConstructor(parameterTypes).trySetAccessible(); + } + catch(final NoSuchMethodException e) + { + return false; + } + } + + static XGettingEnum decline(final Field field, final String reason) + { + logger.debug("Field {} is not inlined: {}", field, reason); + + return null; + } + + /** + * Creates a resolver that inlines every field the passed selector accepts and that is eligible: the field's + * declared type must be a concrete value class whose own persistable fields are each either a primitive or + * inlinable by the same rules, so the eligibility test is recursive. + *

+ * The type must be a value class because only an identity-less instance can be reconstructed from its + * content alone without changing what the owner refers to. It must be the field's declared type because + * the inlined form carries no type information, so a field typed as a supertype could not be read back. + * Requiring every member to be a primitive or itself inlined keeps the inlined slot free of object ids at + * every depth, which is what lets the storage engine skip it as it skips any other fixed-length + * non-reference member. A type that would contain itself is refused rather than expanded: such a class + * compiles, but its layout has no fixed length. + * + * @param typeAnalyzer the analyzer determining a type's persistable fields; must not be {@code null}. + * @param selector decides which eligible fields to actually inline; must not be {@code null}. + * + * @return the new resolver. + */ + public static PersistenceValueInliningResolver New( + final PersistenceTypeAnalyzer typeAnalyzer, + final BiPredicate, Field> selector + ) + { + return new PersistenceValueInliningResolver.Default( + notNull(typeAnalyzer), + notNull(selector) + ); + } + + public final class Default implements PersistenceValueInliningResolver + { + /////////////////////////////////////////////////////////////////////////// + // instance fields // + //////////////////// + + private final PersistenceTypeAnalyzer typeAnalyzer; + private final BiPredicate, Field> selector ; + + + + /////////////////////////////////////////////////////////////////////////// + // constructors // + ///////////////// + + Default( + final PersistenceTypeAnalyzer typeAnalyzer, + final BiPredicate, Field> selector + ) + { + super(); + this.typeAnalyzer = typeAnalyzer; + this.selector = selector ; + } + + + + /////////////////////////////////////////////////////////////////////////// + // override methods // + ///////////////////// + + @Override + public XGettingEnum resolveInlinedFields(final Class ownerType, final Field field) + { + return this.resolveInlinedFields(ownerType, field, HashEnum.New()); + } + + /** + * @param enclosing the value types the current layout is already nested in, so a type that would + * contain itself is refused instead of expanded forever. + */ + private XGettingEnum resolveInlinedFields( + final Class ownerType, + final Field field , + final HashEnum> enclosing + ) + { + final Class valueType = field.getType(); + + if(!XReflect.isValueClass(valueType)) + { + // not a value class, so there is nothing to decide and nothing worth reporting + return null; + } + + if(enclosing.contains(valueType)) + { + /* A value class may hold itself, directly or through another, and such a type compiles. Its + * inlined layout would have no fixed length and the computation of it would not terminate, + * so the cycle has to be refused rather than merely not reached. + */ + return decline(field, valueType.getName() + " already encloses this field, so the layout would" + + " contain itself" + ); + } + + if(XReflect.isAbstract(valueType)) + { + /* An abstract type is not exact: the field may hold subtype instances, whose state the + * declared type's layout cannot represent. Only a concrete value class, which is final by + * definition, guarantees that the layout written is the layout of every instance held. + */ + return decline(field, valueType.getName() + " is abstract, so the field may hold subtype instances"); + } + + if(!this.selector.test(ownerType, field)) + { + return decline(field, "not selected for inlining"); + } + + final HashEnum persistable = HashEnum.New(); + final HashEnum persister = HashEnum.New(); + final HashEnum problematic = HashEnum.New(); + this.typeAnalyzer.collectPersistableFieldsEntity(valueType, persistable, persister, problematic); + + if(!persister.isEmpty() || !problematic.isEmpty()) + { + return decline(field, valueType.getName() + " has persister or problematic fields"); + } + + enclosing.add(valueType); + try + { + for(final Field valueField : persistable) + { + if(valueField.getType().isPrimitive()) + { + continue; + } + + /* A member that is inlined itself carries no object id either, so the slot stays free of + * them at every depth. One that is not leaves the slot holding a reference the storage + * engine cannot see, which is what makes the whole layout ineligible rather than only + * that member. + */ + if(this.resolveInlinedFields(valueType, valueField, enclosing) == null) + { + return decline(field, XReflect.deriveFieldIdentifier(valueField) + + " is neither a primitive nor inlinable itself" + ); + } + } + } + finally + { + enclosing.removeOne(valueType); + } + + if(!isConstructorInvocable(valueType, persistable)) + { + /* The inlined instance is constructed rather than populated, so a constructor that cannot be + * reached would only fail later, when the handler is built. + */ + return decline(field, valueType.getName() + " has no reachable constructor for its fields"); + } + + return persistable; + } + + } + +} diff --git a/serializer/src/main/java/org/eclipse/serializer/Serializer.java b/serializer/src/main/java/org/eclipse/serializer/Serializer.java index b6a83fe6..41f4a8df 100644 --- a/serializer/src/main/java/org/eclipse/serializer/Serializer.java +++ b/serializer/src/main/java/org/eclipse/serializer/Serializer.java @@ -14,27 +14,45 @@ * #L% */ +import static org.eclipse.serializer.util.X.mayNull; +import static org.eclipse.serializer.util.X.notNull; + +import java.nio.ByteBuffer; +import java.util.function.Function; + import org.eclipse.serializer.collections.BulkList; import org.eclipse.serializer.collections.HashTable; import org.eclipse.serializer.collections.types.XGettingCollection; import org.eclipse.serializer.hashing.XHashing; import org.eclipse.serializer.memory.XMemory; import org.eclipse.serializer.meta.NotImplementedYetError; -import org.eclipse.serializer.persistence.binary.types.*; +import org.eclipse.serializer.persistence.binary.types.Binary; +import org.eclipse.serializer.persistence.binary.types.BinaryStorer; +import org.eclipse.serializer.persistence.binary.types.ChunksBuffer; +import org.eclipse.serializer.persistence.binary.types.ChunksBufferByteReversing; +import org.eclipse.serializer.persistence.binary.types.ChunksWrapper; import org.eclipse.serializer.persistence.exceptions.PersistenceExceptionTransfer; -import org.eclipse.serializer.persistence.types.*; +import org.eclipse.serializer.persistence.types.PersistenceCommitListener; +import org.eclipse.serializer.persistence.types.PersistenceIdSet; +import org.eclipse.serializer.persistence.types.PersistenceManager; +import org.eclipse.serializer.persistence.types.PersistenceObjectIdRequestor; +import org.eclipse.serializer.persistence.types.PersistenceObjectManager; +import org.eclipse.serializer.persistence.types.PersistenceObjectRegistrationListener; +import org.eclipse.serializer.persistence.types.PersistenceRoots; +import org.eclipse.serializer.persistence.types.PersistenceSource; +import org.eclipse.serializer.persistence.types.PersistenceStoreHandler; +import org.eclipse.serializer.persistence.types.PersistenceStorer; +import org.eclipse.serializer.persistence.types.PersistenceTarget; +import org.eclipse.serializer.persistence.types.PersistenceTypeHandler; +import org.eclipse.serializer.persistence.types.PersistenceTypeHandlerManager; +import org.eclipse.serializer.persistence.types.Persister; +import org.eclipse.serializer.persistence.types.Storer; import org.eclipse.serializer.reference.Lazy; import org.eclipse.serializer.reference.ObjectSwizzling; import org.eclipse.serializer.reference.Swizzling; import org.eclipse.serializer.util.BufferSizeProviderIncremental; import org.eclipse.serializer.util.X; -import java.nio.ByteBuffer; -import java.util.function.Function; - -import static org.eclipse.serializer.util.X.mayNull; -import static org.eclipse.serializer.util.X.notNull; - /** * Convenient API layer to use the binary persistence functionality for a simple serializer. *

@@ -451,6 +469,12 @@ public PersistenceStorer ensureCapacity(final long desiredCapacity) return this; } + @Override + public boolean isEagerStoring() + { + return true; + } + @Override public long apply(final T instance) { @@ -662,6 +686,22 @@ public void registerEagerOptional( // default is eager logic. this.registerGuaranteed(objectId, instance, optionalHandler); } + + @Override + public void registerSkippedOptional( + final long objectId , + final T instance , + final PersistenceTypeHandler optionalHandler + ) + { + /* Skipping is a referencing optimization for persistent storage, where a JDK constant + * must never be written as an entity under its reserved id. This serializer's medium is + * self-contained, though: a constant that is not registry-resident has to be part of the + * medium itself, or a graph consisting of nothing but the constant could not be + * expressed at all. Its reserved id is kept, so the form stays deterministic. + */ + this.registerGuaranteed(objectId, instance, optionalHandler); + } protected final long registerGuaranteed(final Object instance) { From cd5df6925180b63197a24a786f05a59af497b99c Mon Sep 17 00:00:00 2001 From: hg-ms <53219833+hg-ms@users.noreply.github.com> Date: Thu, 24 Sep 2026 12:33:55 +0200 Subject: [PATCH 2/3] Register the canonicalizing java.time handlers only for value classes LocalTime.of answers one shared instance per whole hour, Instant.ofEpochSecond one for the epoch and Duration.ofSeconds one for zero. Building an instance from the persisted state is what a value class needs and what an identity class must not have: a storage written before these handlers existed can hold two separately obtained objects of such a value, and one instance built for both of them cannot be registered under both their object ids, so the storage does not open at all. The three handlers therefore move to #platformDependentHandlers and are registered per type, only where the JDK has migrated that type. Where it keeps its identity, the generic handler populates one instance per entity, as before. --- .../binary/types/BinaryPersistence.java | 38 +++++++++++++++---- 1 file changed, 30 insertions(+), 8 deletions(-) diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryPersistence.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryPersistence.java index bb357a21..4f4e29e9 100644 --- a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryPersistence.java +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/types/BinaryPersistence.java @@ -1,6 +1,9 @@ package org.eclipse.serializer.persistence.binary.types; import java.lang.reflect.Field; +import java.time.Duration; +import java.time.Instant; +import java.time.LocalTime; import java.util.Collections; import java.util.Comparator; import java.util.Optional; @@ -146,6 +149,7 @@ import org.eclipse.serializer.persistence.types.PersistenceTypeIdLookup; import org.eclipse.serializer.reference.Referencing; import org.eclipse.serializer.reference.Swizzling; +import org.eclipse.serializer.reflect.XReflect; import org.eclipse.serializer.typing.XTypes; import org.eclipse.serializer.util.VMInfo; @@ -340,14 +344,12 @@ static final void initializeNativeTypeId( BinaryHandlerYearMonth.New(), BinaryHandlerMonthDay.New(), - /* Self-contained java.time types whose instances, as value classes, cannot be populated - * after creation. The reference-holding java.time types are registered with the - * referencing-type handlers below. + /* A self-contained java.time type whose instances, as value classes, cannot be populated + * after creation. The reference-holding ones are registered with the referencing-type + * handlers below, those whose factory canonicalizes its result by + * #platformDependentHandlers. */ - BinaryHandlerLocalTime.New(), - BinaryHandlerYear.New() , - BinaryHandlerInstant.New() , - BinaryHandlerDuration.New() , + BinaryHandlerYear.New(), /* (12.11.2019 TM)NOTE: * One might think that "empty" implementations of a collection interface would have no fields, anyway. @@ -515,7 +517,8 @@ static final void initializeNativeTypeId( /** * @return the catalogue of handlers whose registration depends on the current runtime platform (for - * instance, handlers skipped on Android due to absent JDK classes). + * instance, handlers skipped on Android due to absent JDK classes, or handlers that may only + * be used where the JDK type they handle is a value class). */ @SuppressWarnings("unchecked") public static final XGettingSequence> platformDependentHandlers() @@ -529,6 +532,25 @@ static final void initializeNativeTypeId( platformDependentHandlers.add(BinaryHandlerSetFromMap.New()); } + /* These three types are constructed from their persisted state, which a value class requires and + * an identity class must not have: their factories answer a shared instance for certain values - + * LocalTime.of for a whole hour, Instant.EPOCH, Duration.ZERO - so two entities holding such a + * value would build one instance, which cannot be registered under both their object ids. Where + * the type has identity, the generic handler populates one instance per entity, as before. + */ + if(XReflect.isValueClass(LocalTime.class)) + { + platformDependentHandlers.add(BinaryHandlerLocalTime.New()); + } + if(XReflect.isValueClass(Instant.class)) + { + platformDependentHandlers.add(BinaryHandlerInstant.New()); + } + if(XReflect.isValueClass(Duration.class)) + { + platformDependentHandlers.add(BinaryHandlerDuration.New()); + } + return platformDependentHandlers; } From 9c66863def8de1e6a7a9e607b9caffa488821e64 Mon Sep 17 00:00:00 2001 From: hg-ms <53219833+hg-ms@users.noreply.github.com> Date: Thu, 24 Sep 2026 13:32:26 +0200 Subject: [PATCH 3/3] Install a loaded custom root without the explicit-root validation The legacy root refactoring installs a root it has just loaded through #set, which delegates to #setRoot and therefore rejects a value instance - the very case the validation's own documentation exempts, a loaded root having no state applied to it. Such a storage does not open at all, and advises a change to data that is already written. It now uses #setRootSupplier, like the loaded default root in the branch below it and like the root reference handler. The supplier's javadoc states that this is the installation path for a root resolved from persisted state, and why that one does not validate. --- .../types/BinaryHandlerPersistenceRootsDefault.java | 9 ++++++++- .../persistence/types/PersistenceRootReference.java | 5 +++++ 2 files changed, 13 insertions(+), 1 deletion(-) diff --git a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/org/eclipse/serializer/persistence/types/BinaryHandlerPersistenceRootsDefault.java b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/org/eclipse/serializer/persistence/types/BinaryHandlerPersistenceRootsDefault.java index 373448a2..91255900 100644 --- a/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/org/eclipse/serializer/persistence/types/BinaryHandlerPersistenceRootsDefault.java +++ b/persistence/binary/src/main/java/org/eclipse/serializer/persistence/binary/org/eclipse/serializer/persistence/types/BinaryHandlerPersistenceRootsDefault.java @@ -222,7 +222,14 @@ private boolean ensureRefactoredOldRoots( ); } - this.rootResolverProvider.rootReference().set(customRoot); + /* + * Deliberately not #set: that rejects a value instance because an explicitly set root + * could never receive its persisted state. This root is that state already, so it is + * installed as it stands, like the loaded default root below. + */ + this.rootResolverProvider.rootReference().setRootSupplier(() -> + customRoot + ); resolvedRoots.add(Persistence.customRootIdentifier(), null); return true; diff --git a/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceRootReference.java b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceRootReference.java index 34ad27a7..4c316332 100644 --- a/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceRootReference.java +++ b/persistence/persistence/src/main/java/org/eclipse/serializer/persistence/types/PersistenceRootReference.java @@ -98,6 +98,11 @@ public static void validateRootInstance(final Object rootInstance) /** * Replaces the supplier that resolves the root instance. Use this overload when the root cannot be * eagerly created (e.g. because doing so would trigger class-loading order issues). + *

+ * This is also how a root resolved from persisted state is installed, and the only way to do so: + * unlike {@link #setRoot(Object)} it applies no {@linkplain #validateRootInstance(Object) + * validation}, which is correct for a loaded root - it receives no state, it is the state, + * so it may be a value instance. * * @param rootSupplier the new supplier; may be {@code null}. *