[SPARK-59273][SQL] Complete CHAR/VARCHAR support at core execution boundaries

[srielau]

What changes were proposed in this pull request?

When spark.sql.charVarchar.standardSemantics.enabled is true, CharType and VarcharType are first-class StringType subtypes. Several execution-boundary matchers still used exact StringType (or an explicit CHAR/VARCHAR reject), so those paths failed or skipped constrained string columns.

This patch treats CHAR/VARCHAR as the string family at:

• JDBC getters/setters and JDBC array element conversion
• JDBC schema inference (CHAR/VARCHAR keep first-class types when standard semantics is on, even if charVarcharAsString is also set)
• File partition value decoding
• RowToColumnConverter
• DataFrame.na.fill for string replacement values
• ANALYZE TABLE ... FOR COLUMNS (string-family stats)

Read-side CHAR padding and VARCHAR overflow still come from existing CAST / ApplyCharTypePadding paths rather than being reimplemented in each converter.

Why are the changes needed?

With first-class CHAR/VARCHAR, JDBC scans/writes, file-only partition discovery, columnar conversion, na.fill("..."), and column stats currently throw or silently ignore those columns. That blocks enabling standard semantics.

JIRA: https://issues.apache.org/jira/browse/SPARK-59273 (subtask of SPARK-58794)

Does this PR introduce any user-facing change?

Yes, when spark.sql.charVarchar.standardSemantics.enabled is true (still default false):

• JDBC read/write of CHAR/VARCHAR (including arrays) no longer fails with an unsupported JDBC type.
• File partition columns declared as CHAR/VARCHAR can be decoded; CHAR is padded on scan and oversize VARCHAR fails with EXCEED_LIMIT_LENGTH.
• Columnar row-to-column conversion accepts CHAR/VARCHAR.
• df.na.fill("x") fills null CHAR/VARCHAR columns (CHAR values are padded by CAST).
• ANALYZE TABLE ... FOR COLUMNS collects string-family stats on CHAR/VARCHAR instead of rejecting them.

How was this patch tested?

Added/extended unit tests:

• JDBCSuite: read CHAR/VARCHAR and arrays; write CHAR/VARCHAR; standard semantics wins over charVarcharAsString in schema inference
• ParquetV1PartitionDiscoverySuite / ParquetV2PartitionDiscoverySuite: CHAR/VARCHAR partition values and oversize VARCHAR
• RowToColumnConverterSuite: CHAR/VARCHAR and nested CHAR arrays
• DataFrameNaFunctionsSuite: na.fill on CHAR/VARCHAR
• StatisticsCollectionSuite: ANALYZE TABLE ... FOR COLUMNS on CHAR/VARCHAR

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Cursor Grok 4.6

[srielau]

Should this test !conf.charVarcharFirstClassTypes instead? preserveCharVarcharTypeInfo is also a first-class mode. With preserve=true and charVarcharAsString=true, this branch currently degrades JDBC CHAR/VARCHAR to STRING, contrary to the shared first-class-type contract. The VARCHAR branch below has the same issue. Could we also extend the precedence test to cover this combination?