Skip to content

docs(isthmus): javadoc for io.substrait.isthmus type system classes #762

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
bestbeforetoday merged 2 commits into from
Mar 25, 2026
Merged

docs(isthmus): javadoc for io.substrait.isthmus type system classes #762

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a9e32b3
fix(javadoc): partial isthmus
mbwhite Mar 18, 2026
b59a7df
review comments
mbwhite Mar 25, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 39 additions & 2 deletions isthmus/src/main/java/io/substrait/isthmus/SubstraitTypeSystem.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,28 +2,45 @@

import org.apache.calcite.avatica.util.TimeUnit;
import org.apache.calcite.jdbc.JavaTypeFactoryImpl;
import org.apache.calcite.rel.type.RelDataType;
import org.apache.calcite.rel.type.RelDataTypeFactory;
import org.apache.calcite.rel.type.RelDataTypeSystem;
import org.apache.calcite.rel.type.RelDataTypeSystemImpl;
import org.apache.calcite.sql.SqlIntervalQualifier;
import org.apache.calcite.sql.parser.SqlParserPos;
import org.apache.calcite.sql.type.SqlTypeName;

/**
* Custom {@link RelDataTypeSystem} implementation for Substrait.
*
* <p>Defines type system rules such as precision, scale, and interval qualifiers for Substrait
* integration with Calcite.
*/
public class SubstraitTypeSystem extends RelDataTypeSystemImpl {

/** Singleton instance of Substrait type system. */
public static final RelDataTypeSystem TYPE_SYSTEM = new SubstraitTypeSystem();

/** Default type factory using the Substrait type system. */
public static final RelDataTypeFactory TYPE_FACTORY = new JavaTypeFactoryImpl(TYPE_SYSTEM);

// Interval qualifier from year to month
/** Interval qualifier from year to month. */
public static final SqlIntervalQualifier YEAR_MONTH_INTERVAL =
new SqlIntervalQualifier(TimeUnit.YEAR, TimeUnit.MONTH, SqlParserPos.ZERO);

// Interval qualifier from day to fractional second at microsecond precision
/** Interval qualifier from day to fractional second at microsecond precision. */
public static final SqlIntervalQualifier DAY_SECOND_INTERVAL =
new SqlIntervalQualifier(TimeUnit.DAY, -1, TimeUnit.SECOND, 6, SqlParserPos.ZERO);

/** Private constructor to enforce singleton usage. */
private SubstraitTypeSystem() {}

/**
* Returns the maximum precision for the given SQL type.
*
* @param typeName The {@link SqlTypeName} for which precision is requested.
* @return Maximum precision for the type.
*/
@Override
public int getMaxPrecision(final SqlTypeName typeName) {
switch (typeName) {
Expand All @@ -41,6 +58,13 @@ public int getMaxPrecision(final SqlTypeName typeName) {
return super.getMaxPrecision(typeName);
}

/**
* Returns default precision for this type if supported, otherwise {@link
* RelDataType#PRECISION_NOT_SPECIFIED} if precision is either unsupported or must be specified
* explicitly.
*
* @return Default precision
*/
Comment on lines +61 to +67
Copy link
Member

@bestbeforetoday bestbeforetoday Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This description is incorrect. The parent interface Javadoc says:

/**
   * Returns default precision for this type if supported, otherwise
   * {@link RelDataType#PRECISION_NOT_SPECIFIED}
   * if precision is either unsupported or must be specified explicitly.
   *
   * @return Default precision
   */

Also, the return value (in the current implementation) is 38 for the decimal type, but not for any other type.

@Override
public int getDefaultPrecision(final SqlTypeName typeName) {
switch (typeName) {
Expand All @@ -51,6 +75,14 @@ public int getDefaultPrecision(final SqlTypeName typeName) {
}
}

/**
* Returns the maximum scale allowed for this type, or {@link RelDataType#SCALE_NOT_SPECIFIED} if
* scale is not applicable for this type.
*
* <p>The maximum scale for the decimal type is 38.
*
* @return Maximum allowed scale
*/
Comment on lines +78 to +85
Copy link
Member

@bestbeforetoday bestbeforetoday Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This description is incorrect. The parent interface Javadoc says:

/**
   * Returns the maximum scale allowed for this type, or
   * {@link RelDataType#SCALE_NOT_SPECIFIED}
   * if scale is not applicable for this type.
   *
   * @return Maximum allowed scale
   */

The maximum scale for the decimal type (in the current implementation) is 38. For all other types it is different.

@Override
public int getMaxScale(final SqlTypeName typeName) {
switch (typeName) {
Expand All @@ -60,6 +92,11 @@ public int getMaxScale(final SqlTypeName typeName) {
return super.getMaxScale(typeName);
}

/**
* Indicates whether ragged union types should be converted to varying types.
*
* @return {@code true}, as Substrait requires conversion to varying types.
*/
@Override
public boolean shouldConvertRaggedUnionTypesToVarying() {
return true;
Expand Down
62 changes: 61 additions & 1 deletion isthmus/src/main/java/io/substrait/isthmus/TypeConverter.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,26 @@
import org.apache.calcite.sql.type.SqlTypeName;
import org.jspecify.annotations.Nullable;

/**
* Utility for converting between Calcite {@link org.apache.calcite.rel.type.RelDataType} and
* Substrait {@link io.substrait.type.Type}.
*
* <p>Supports primitive, complex, and user-defined types in both directions.
*
* @see UserTypeMapper
* @see io.substrait.type.Type
* @see org.apache.calcite.rel.type.RelDataType
*/
public class TypeConverter {

private final UserTypeMapper userTypeMapper;

// DEFAULT TypeConverter which does not handle user-defined types
/**
* Default {@link TypeConverter} instance that does not handle user-defined types.
*
* <p>Both {@link UserTypeMapper#toSubstrait(RelDataType)} and {@link
* UserTypeMapper#toCalcite(Type.UserDefined)} return {@code null} in this default configuration.
*/
public static TypeConverter DEFAULT =
new TypeConverter(
new UserTypeMapper() {
Expand All @@ -41,14 +56,39 @@ public RelDataType toCalcite(Type.UserDefined type) {
}
});

/**
* Creates a {@link TypeConverter} with a provided user type mapper.
*
* @param userTypeMapper Mapper for converting user-defined types between Calcite and Substrait.
*/
public TypeConverter(UserTypeMapper userTypeMapper) {
this.userTypeMapper = userTypeMapper;
}

/**
* Converts a Calcite {@link RelDataType} to a Substrait {@link Type}.
*
* @param type Calcite type to convert.
* @return Corresponding Substrait type.
* @throws UnsupportedOperationException if the type cannot be converted or has unsupported
* properties.
*/
public Type toSubstrait(RelDataType type) {
return toSubstrait(type, new ArrayList<>());
}

/**
* Converts a Calcite {@link RelDataType} of SQL type {@link SqlTypeName#ROW} to a Substrait
* {@link NamedStruct}.
*
* <p>Field names are extracted from the Calcite struct type and paired with the converted
* Substrait struct.
*
* @param type Calcite struct type ({@link SqlTypeName#ROW}).
* @return Substrait {@link NamedStruct} containing field names and struct type.
* @throws IllegalArgumentException if {@code type} is not a struct ({@code ROW}).
* @throws UnsupportedOperationException if any child field type cannot be converted.
*/
public NamedStruct toNamedStruct(RelDataType type) {
if (type.getSqlTypeName() != SqlTypeName.ROW) {
throw new IllegalArgumentException("Expected type of struct.");
Expand Down Expand Up @@ -153,11 +193,31 @@ private Type toSubstrait(RelDataType type, List<String> names) {
}
}

/**
* Converts a Substrait {@link TypeExpression} to a Calcite {@link RelDataType}.
*
* @param relDataTypeFactory Calcite type factory.
* @param typeExpression Substrait type expression to convert.
* @return Calcite relational type.
* @throws UnsupportedOperationException if the expression contains unsupported precision or
* user-defined types cannot be mapped.
*/
public RelDataType toCalcite(
RelDataTypeFactory relDataTypeFactory, TypeExpression typeExpression) {
return toCalcite(relDataTypeFactory, typeExpression, null);
}

/**
* Converts a Substrait {@link TypeExpression} to a Calcite {@link RelDataType}, with optional
* field names for DFS/nested structs.
*
* @param relDataTypeFactory Calcite type factory.
* @param typeExpression Substrait type expression to convert.
* @param dfsFieldNames Optional list of field names to apply to struct fields, in DFS order.
* @return Calcite relational type.
* @throws UnsupportedOperationException if the expression contains unsupported precision or
* user-defined types cannot be mapped.
*/
public RelDataType toCalcite(
RelDataTypeFactory relDataTypeFactory,
TypeExpression typeExpression,
Expand Down
11 changes: 11 additions & 0 deletions isthmus/src/main/java/io/substrait/isthmus/Utils.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,24 @@
import org.apache.calcite.jdbc.CalciteSchema;
import org.jspecify.annotations.NonNull;

/**
* Utility helpers for Substrait conversions and Calcite schema management.
*
* <p>Includes helpers for computing cartesian products and building hierarchical Calcite schemas.
*/
public class Utils {
/**
* Compute the cartesian product for n lists.
*
* <p>Based on <a
* href="https://thomas.preissler.me/blog/2020/12/29/permutations-using-java-streams">Soln by
* Thomas Preissler</a>
*
* @param <T> element type contained within each list.
* @param lists A list of lists whose cross product is computed. Null or empty inner lists are
* skipped.
* @return A stream of lists representing the cartesian product (each output list has one element
* from each input list), or an empty stream if {@code lists} is empty.
*/
public static <T> Stream<List<T>> crossProduct(List<List<T>> lists) {

Expand Down
Loading