Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/sync.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,13 @@ apache/fory-site@main:
dest: docs/community/DEVELOPMENT.md
- source: docs/guide/
dest: docs/guide/
deleteOrphaned: true
- source: docs/specification/
dest: docs/specification/
deleteOrphaned: true
- source: docs/compiler/
dest: docs/compiler/
deleteOrphaned: true
- source: docs/benchmarks/
dest: docs/benchmarks/
deleteOrphaned: true
21 changes: 14 additions & 7 deletions ci/release.py
Original file line number Diff line number Diff line change
Expand Up @@ -165,12 +165,7 @@ def bump_version(**kwargs):
_update_scala_version,
)
elif lang == "kotlin":
_bump_version(
"kotlin",
"pom.xml",
_normalize_java_version(new_version),
_update_kotlin_version,
)
bump_kotlin_version(_normalize_java_version(new_version))
elif lang == "rust":
bump_rust_version(new_version)
elif lang == "python":
Expand Down Expand Up @@ -269,6 +264,16 @@ def bump_rust_version(new_version):
)


def bump_kotlin_version(new_version):
_bump_version("kotlin", "pom.xml", new_version, _update_kotlin_version)
for p in [
"kotlin/fory-kotlin",
"kotlin/fory-kotlin-ksp",
"kotlin/fory-kotlin-tests",
]:
_bump_version(p, "pom.xml", new_version, _update_pom_parent_version)


def bump_cpp_version(new_version):
for p in [
"cpp",
Expand Down Expand Up @@ -371,7 +376,7 @@ def _update_scala_version(lines, v):

def _update_kotlin_version(lines, v):
v = _normalize_java_version(v)
return _update_pom_version(lines, v, "<artifactId>fory-kotlin</artifactId>")
return _update_pom_version(lines, v, "<artifactId>fory-kotlin-parent</artifactId>")


def _update_parent_pom_version(lines, v):
Expand All @@ -384,6 +389,8 @@ def _update_pom_version(lines, v, prev):
if prev in line:
target_index = index + 1
break
if target_index == -1:
raise ValueError(f"Could not find POM version marker: {prev}")
current_version_line = lines[target_index]
# Find the start and end of the version number
start = current_version_line.index("<version>") + len("<version>")
Expand Down
14 changes: 12 additions & 2 deletions docs/guide/cpp/configuration.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Configuration
sidebar_position: 2
sidebar_position: 4
id: configuration
license: |
Licensed to the Apache Software Foundation (ASF) under one or more
Expand Down Expand Up @@ -170,8 +170,18 @@ auto fory = Fory::builder().xlang(true).build_thread_safe(); // Returns ThreadS
| `max_dyn_depth(uint32_t)` | Maximum nesting depth for dynamic types | `5` |
| `check_struct_version(bool)` | Enable struct version checking | `false` |

## Security

Security-related configuration:

- Register all structs and polymorphic implementations before deserializing untrusted payloads.
- Use `check_struct_version(true)` with `compatible(false)` when exact schema matching is required.
- Keep `max_dyn_depth(...)` as low as your model permits to reject unexpectedly deep polymorphic
graphs.
- Prefer concrete fields over broad polymorphic fields for untrusted input.

## Related Topics

- [Basic Serialization](basic-serialization.md) - Using configured Fory
- [Cross-Language](cross-language.md) - xlang mode details
- [Xlang Serialization](xlang-serialization.md) - xlang mode details
- [Type Registration](type-registration.md) - Registering types
2 changes: 1 addition & 1 deletion docs/guide/cpp/custom-serializers.md
Original file line number Diff line number Diff line change
Expand Up @@ -368,4 +368,4 @@ static MyType read_data(ReadContext &ctx) {
- [Type Registration](type-registration.md) - Registering serializers
- [Basic Serialization](basic-serialization.md) - Using FORY_STRUCT macro
- [Schema Evolution](schema-evolution.md) - Compatible mode
- [Cross-Language](cross-language.md) - Cross-language serialization
- [Xlang Serialization](xlang-serialization.md) - Cross-language serialization
8 changes: 5 additions & 3 deletions docs/guide/cpp/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ The C++ implementation provides high-performance serialization with compile-time
## Why Apache Fory™ C++?

- **Fast binary encoding**: Fast serialization and optimized binary protocols
- **Cross-language**: Seamlessly serialize/deserialize data across Java, Python, C++, Go, JavaScript, and Rust
- **Xlang**: Seamlessly serialize/deserialize data across Java, Python, C++, Go, JavaScript, and Rust
- **Type-safe**: Compile-time type checking with macro-based struct registration
- **Reference tracking**: Automatic tracking of shared and circular references
- **Schema evolution**: Compatible mode for independent schema changes
Expand Down Expand Up @@ -212,7 +212,7 @@ Use xlang mode for cross-language payloads and schemas shared with other Fory ru

Use native mode for C++-only traffic. Native mode is selected with `.xlang(false)`, uses schema-consistent payloads unless compatible mode is enabled, and keeps C++ object serialization on the C++ runtime path. It is optimized for C++ types and avoids portable xlang type-mapping constraints when the payload never leaves C++.

See [Cross-Language Serialization](cross-language.md) for C++ xlang registration and interoperability rules, and [Configuration](configuration.md) for native-mode builder options.
See [Xlang Serialization](xlang-serialization.md) for C++ xlang registration and interoperability rules, and [Native Serialization](native-serialization.md) for C++-only payloads.

## Thread Safety

Expand Down Expand Up @@ -262,9 +262,11 @@ std::thread t2([&]() {

- [Configuration](configuration.md) - Builder options and modes
- [Basic Serialization](basic-serialization.md) - Object graph serialization
- [Cross-Language](cross-language.md) - xlang mode and interoperability
- [Xlang Serialization](xlang-serialization.md) - xlang mode and interoperability
- [Native Serialization](native-serialization.md) - C++-only serialization
- [Schema Metadata](schema-metadata.md) - Field-level metadata (nullable, ref tracking)
- [Schema Evolution](schema-evolution.md) - Compatible mode and schema changes
- [Type Registration](type-registration.md) - Registering types
- [Supported Types](supported-types.md) - All supported types
- [Custom Serializers](custom-serializers.md) - Extend serialization behavior
- [Row Format](row-format.md) - Zero-copy row-based format
211 changes: 211 additions & 0 deletions docs/guide/cpp/native-serialization.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,211 @@
---
title: Native Serialization
sidebar_position: 3
id: native_serialization
license: |
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
---

C++ native serialization is the C++-only wire mode selected with `.xlang(false)`. Use it when every
writer and reader is C++ and the payload should follow C++ type behavior instead of the portable
xlang type system.

Use [Xlang Serialization](xlang-serialization.md), the default C++ mode, when bytes must be read by
Java, Python, Go, Rust, JavaScript, or another non-C++ Fory runtime.

## When To Use Native Serialization

Use native serialization when:

- A payload is produced and consumed only by C++ applications.
- The data model uses C++-specific types such as character types, unsigned-native type IDs,
`std::tuple`, smart pointers, or C++ polymorphic models.
- You want schema-consistent C++ payloads for lockstep services.
- You need compatible schema evolution for C++-only rolling deployments.
- You want to avoid portable xlang type-mapping constraints for a C++ boundary.

## Create a Native Runtime

```cpp
#include "fory/serialization/fory.h"
#include <cassert>
#include <cstdint>
#include <string>

using namespace fory::serialization;

struct Order {
int64_t id;
double amount;

bool operator==(const Order &other) const {
return id == other.id && amount == other.amount;
}
};
FORY_STRUCT(Order, id, amount);

int main() {
auto fory = Fory::builder()
.xlang(false)
.build();
fory.register_struct<Order>(100);

Order order{1, 42.5};
auto bytes = fory.serialize(order).value();
auto decoded = fory.deserialize<Order>(bytes).value();
assert(order == decoded);
}
```

Use one configured `Fory` instance per thread, or build a thread-safe runtime when the same runtime
is shared by multiple threads:

```cpp
auto fory = Fory::builder()
.xlang(false)
.track_ref(true)
.build_thread_safe();
```

Register types before concurrent serialization starts.

## Schema Evolution

Native serialization defaults to schema-consistent mode. Enable compatible mode when C++-only
writer and reader schemas can differ:

```cpp
auto fory = Fory::builder()
.xlang(false)
.compatible(true)
.build();
```

Compatible mode writes schema metadata so readers can tolerate added, removed, or reordered fields
when field identity remains compatible. See [Schema Evolution](schema-evolution.md).

## Registration

Register structs with stable IDs or names before serialization:

```cpp
fory.register_struct<Order>(100);
fory.register_struct<Order>("example", "Order");
```

Use numeric IDs for compact payloads. Use namespace/type-name registration when independent teams
coordinate type identity by names.

## C++ Object Surface

Native serialization owns the C++-specific object surface:

- Structs and classes described by `FORY_STRUCT`.
- Standard containers such as `std::vector`, `std::map`, `std::unordered_map`, `std::set`, and
`std::unordered_set`.
- `std::optional`, `std::variant`, and tuple-like values.
- `std::shared_ptr` and `std::unique_ptr`.
- Character types such as `char`, `char16_t`, and `char32_t`.
- Unsigned integer types with native-mode type IDs.
- Polymorphic serialization registered through the C++ runtime.

Use [Supported Types](supported-types.md) for the full type surface and xlang mapping notes.

## References And Smart Pointers

Native serialization supports smart pointers and reference tracking:

```cpp
auto fory = Fory::builder()
.xlang(false)
.track_ref(true)
.build();
```

When reference tracking is enabled, shared pointer identity can be preserved and cyclic object
graphs can be represented through supported pointer patterns. Disable reference tracking for
value-shaped data when identity is not part of the model.

## Native-Only Scalar Shapes

Some C++ scalar shapes are not portable xlang payloads. Use native serialization when these shapes
must round-trip as C++ values:

```cpp
auto fory = Fory::builder().xlang(false).build();

auto char_bytes = fory.serialize(char32_t{U'A'}).value();
auto value = fory.deserialize<char32_t>(char_bytes).value();

auto unsigned_bytes = fory.serialize(uint64_t{42}).value();
auto unsigned_value = fory.deserialize<uint64_t>(unsigned_bytes).value();
```

For xlang payloads, use schema metadata and the shared xlang type mapping instead of relying on
C++ native-only type IDs.

## Performance Guidelines

- Reuse configured `Fory` instances.
- Use single-threaded `Fory` per thread for the fastest path; use `build_thread_safe()` for shared
concurrent use.
- Keep native schema-consistent mode for lockstep C++ services.
- Enable `.compatible(true)` only when C++-only schema evolution is required.
- Register structs with explicit numeric IDs for compact payloads.
- Disable reference tracking for value-shaped graphs.
- Prefer concrete types over polymorphic/dynamic fields on hot paths.

## Native And Xlang Comparison

| Requirement | Use native serialization | Use xlang serialization |
| ---------------------------------------- | ------------------------ | ----------------------- |
| C++-only payloads | Yes | Optional |
| Non-C++ readers or writers | No | Yes |
| C++ native character and unsigned shapes | Yes | Limited |
| Smart pointers and C++ object graphs | Yes | Limited |
| Schema-consistent same-language payloads | Yes | No |
| Compatible schema evolution by default | No | Yes |
| Portable type mapping across runtimes | No | Yes |

## Troubleshooting

### A non-C++ runtime cannot read the payload

The writer is using native serialization. Rebuild it with `.xlang(true)` and align type
registration with every peer runtime.

### A rolling deployment fails after a field change

Native serialization defaults to schema-consistent mode. Use `.compatible(true)` on both writer and
reader when schemas can differ.

### A native-only scalar does not map to another language

Use xlang serialization with explicit schema metadata for portable payloads. Native C++ type IDs
are only for C++ readers.

### A shared pointer graph loses identity

Enable `.track_ref(true)` and verify the graph uses supported pointer patterns.

## Related Topics

- [Xlang Serialization](xlang-serialization.md) - Cross-runtime C++ payloads
- [Configuration](configuration.md) - Builder options
- [Basic Serialization](basic-serialization.md) - Object graph serialization
- [Supported Types](supported-types.md) - C++ type support
- [Polymorphic Serialization](polymorphism.md) - Polymorphic object models
- [Schema Evolution](schema-evolution.md) - Compatible mode
2 changes: 1 addition & 1 deletion docs/guide/cpp/polymorphism.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Polymorphic Serialization
sidebar_position: 7
sidebar_position: 8
id: polymorphism
license: |
Licensed to the Apache Software Foundation (ASF) under one or more
Expand Down
2 changes: 1 addition & 1 deletion docs/guide/cpp/row-format.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Row Format
sidebar_position: 20
sidebar_position: 11
id: row_format
license: |
Licensed to the Apache Software Foundation (ASF) under one or more
Expand Down
6 changes: 3 additions & 3 deletions docs/guide/cpp/schema-evolution.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Schema Evolution
sidebar_position: 5
sidebar_position: 6
id: schema_evolution
license: |
Licensed to the Apache Software Foundation (ASF) under one or more
Expand Down Expand Up @@ -387,7 +387,7 @@ Test both upgrade and downgrade scenarios:
// Test V3 -> V1
```

## Cross-Language Schema Evolution
## Xlang Schema Evolution

Schema evolution works across languages when using xlang mode:

Expand All @@ -413,4 +413,4 @@ Both instances can exchange data even with different schema versions.

- [Configuration](configuration.md) - Enabling compatible mode
- [Type Registration](type-registration.md) - Type ID management
- [Cross-Language](cross-language.md) - Cross-language considerations
- [Xlang Serialization](xlang-serialization.md) - Cross-language considerations
2 changes: 1 addition & 1 deletion docs/guide/cpp/schema-metadata.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Schema Metadata
sidebar_position: 4
sidebar_position: 5
id: schema_metadata
license: |
Licensed to the Apache Software Foundation (ASF) under one or more
Expand Down
Loading
Loading