Skip to content

TEZ-4688 TEZ-4648: Tez upgrade to Hadoop 3.5.0 and jersey 2.x #474

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
maheshrajus wants to merge 11 commits into
base: master
Choose a base branch
from
Open

TEZ-4688 TEZ-4648: Tez upgrade to Hadoop 3.5.0 and jersey 2.x #474

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
d5cc9ef
TEZ-4688: Tez upgrade to Hadoop 3.5.0
maheshrajus Apr 8, 2026
eab1b7d
TEZ-4648: Upgrade jersey to 2.x
abstractdog Sep 15, 2025
af2f4db
jersey2 migration
abstractdog Sep 15, 2025
f9f1cad
jupiter fixes
abstractdog Sep 16, 2025
e548565
spotbugs fix
maheshrajus Apr 8, 2026
044f792
test failures fix
maheshrajus Apr 6, 2026
d37d8cd
review comments fix
maheshrajus Apr 22, 2026
1a01f07
Review comments fix_2
maheshrajus May 20, 2026
e325ba9
Review comments fix 3
maheshrajus Jun 8, 2026
bcdcf4b
Fix Tez UI timeline parsing after Hadoop 3.5.0 upgrade
maheshrajus Jun 11, 2026
d97fa47
Fix javax servlet & jackson exceptions
maheshrajus Jun 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
184 changes: 160 additions & 24 deletions pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,11 +81,17 @@
<plexus-velocity.version>2.3.0</plexus-velocity.version>
<frontend-maven-plugin.version>1.15.0</frontend-maven-plugin.version>
<guava.version>32.0.1-jre</guava.version>
<hadoop.version>3.4.2</hadoop.version>
<jersey.version>1.19.4</jersey.version>
<hadoop.version>3.5.0</hadoop.version>
Comment thread
maheshrajus marked this conversation as resolved.
<!-- Align all Jackson modules with the version shipped by Hadoop 3.5.0 (2.18.6).
Importing jackson-bom in dependencyManagement ensures any transitive dependency
(e.g. avro) that pulls an older jackson-core is automatically overridden.
This is the Jackson project's recommended approach for version alignment. -->
<jackson.version>2.18.6</jackson.version>
<jersey.version>2.46</jersey.version>
<jettison.version>1.5.4</jettison.version>
<jsr305.version>3.0.0</jsr305.version>
<junit.version>4.13.2</junit.version>
<junit.jupiter.version>5.9.3</junit.jupiter.version>
Comment thread
maheshrajus marked this conversation as resolved.
<leveldbjni-all.version>1.8</leveldbjni-all.version>
<lifecycle-mapping.version>1.0.0</lifecycle-mapping.version>
<metrics-core.version>3.1.0</metrics-core.version>
Expand All @@ -98,7 +104,7 @@
<restrict-imports.enforcer.version>3.0.0</restrict-imports.enforcer.version>
<roaringbitmap.version>1.2.1</roaringbitmap.version>
<scm.url>scm:git:https://gitbox.apache.org/repos/asf/tez.git</scm.url>
<servlet-api.version>3.1.0</servlet-api.version>
<servlet-api.version>4.0.4</servlet-api.version>
<slf4j.version>1.7.36</slf4j.version>
<snappy-java.version>1.1.10.4</snappy-java.version>
<spotless-maven-plugin.version>3.1.0</spotless-maven-plugin.version>
Expand Down Expand Up @@ -153,6 +159,18 @@

<dependencyManagement>
<dependencies>
<!-- Import jackson-bom to enforce a consistent Jackson version across all
transitive sources. avro 1.11.5 (via hadoop-common/mapreduce-client-common)
pulls in jackson-core 2.14.3 which lacks BufferRecycler.releaseToPool()
required by Jersey 2.46 in Hadoop 3.5.0+. The BOM overrides ALL transitive
jackson versions without needing per-library exclusions -->
<dependency>
<groupId>com.fasterxml.jackson</groupId>
<artifactId>jackson-bom</artifactId>
<version>${jackson.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>org.apache.tez</groupId>
<artifactId>hadoop-shim</artifactId>
Expand Down Expand Up @@ -257,6 +275,16 @@
<groupId>org.apache.pig</groupId>
<artifactId>pig</artifactId>
<version>${pig.version}</version>
<exclusions>
<exclusion>
<groupId>org.mortbay.jetty</groupId>
<artifactId>*</artifactId>
</exclusion>
<exclusion>
<groupId>org.codehaus.jackson</groupId>
<artifactId>*</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>io.dropwizard.metrics</groupId>
Expand Down Expand Up @@ -306,8 +334,8 @@
<version>${netty.version}</version>
</dependency>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
<groupId>jakarta.servlet</groupId>
<artifactId>jakarta.servlet-api</artifactId>
<version>${servlet-api.version}</version>
</dependency>
<dependency>
Expand All @@ -325,16 +353,12 @@
<artifactId>servlet-api</artifactId>
</exclusion>
<exclusion>
<groupId>commons-logging</groupId>
<artifactId>commons-logging-api</artifactId>
</exclusion>
<exclusion>
<groupId>com.sun.jersey</groupId>
<artifactId>jersey-core</artifactId>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
</exclusion>
<exclusion>
<groupId>com.sun.jersey</groupId>
<artifactId>jersey-server</artifactId>
<groupId>commons-logging</groupId>
<artifactId>commons-logging-api</artifactId>
</exclusion>
<exclusion>
<groupId>org.eclipse.jdt</groupId>
Expand Down Expand Up @@ -385,13 +409,25 @@
<groupId>org.xerial.snappy</groupId>
<artifactId>snappy-java</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.avro</groupId>

@Aggarwal-Raghav Aggarwal-Raghav Jun 12, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@maheshrajus , please revert this avro exclusion. UT's are working after that. As we have jackson-bom defined now, we don't need this. All jackson jar have 2.18.6 now

jackson-annotations-2.18.6.jar
jackson-core-2.18.6.jar
jackson-databind-2.18.6.jar
jackson-jaxrs-base-2.18.6.jar
jackson-jaxrs-json-provider-2.18.6.jar
jackson-module-jaxb-annotations-2.18.6.jar

<artifactId>avro</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-auth</artifactId>
<version>${hadoop.version}</version>
<exclusions>
<exclusion>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
</exclusion>
<exclusion>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
</exclusion>
<exclusion>
<groupId>io.netty</groupId>
<artifactId>netty</artifactId>
Expand All @@ -410,16 +446,12 @@
<artifactId>servlet-api</artifactId>
</exclusion>
<exclusion>
<groupId>commons-logging</groupId>
<artifactId>commons-logging-api</artifactId>
</exclusion>
<exclusion>
<groupId>com.sun.jersey</groupId>
<artifactId>jersey-core</artifactId>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
</exclusion>
<exclusion>
<groupId>com.sun.jersey</groupId>
<artifactId>jersey-server</artifactId>
<groupId>commons-logging</groupId>
<artifactId>commons-logging-api</artifactId>
</exclusion>
<exclusion>
<groupId>io.netty</groupId>
Expand Down Expand Up @@ -499,6 +531,14 @@
<groupId>com.sun.jersey</groupId>
<artifactId>jersey-core</artifactId>
</exclusion>
<exclusion>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
</exclusion>
<exclusion>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
</exclusion>
<exclusion>
<groupId>io.netty</groupId>
<artifactId>netty</artifactId>
Expand All @@ -514,6 +554,14 @@
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-util</artifactId>
</exclusion>
<exclusion>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
</exclusion>
<exclusion>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
Expand All @@ -533,6 +581,14 @@
<groupId>org.eclipse.jetty</groupId>
<artifactId>*</artifactId>
</exclusion>
<exclusion>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
</exclusion>
<exclusion>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
Expand All @@ -551,6 +607,14 @@
<groupId>org.eclipse.jetty</groupId>
<artifactId>*</artifactId>
</exclusion>
<exclusion>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
</exclusion>
<exclusion>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
Expand Down Expand Up @@ -602,6 +666,14 @@
<groupId>com.google.inject.extensions</groupId>
<artifactId>guice-servlet</artifactId>
</exclusion>
<exclusion>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
</exclusion>
<exclusion>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
</exclusion>
<exclusion>
<groupId>io.netty</groupId>
<artifactId>netty</artifactId>
Expand All @@ -617,6 +689,14 @@
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-yarn-server-common</artifactId>
</exclusion>
<exclusion>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
</exclusion>
<exclusion>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
</exclusion>
<exclusion>
<groupId>io.netty</groupId>
<artifactId>netty</artifactId>
Expand All @@ -637,6 +717,14 @@
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-mapreduce-client-shuffle</artifactId>
</exclusion>
<exclusion>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
</exclusion>
<exclusion>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
</exclusion>
<exclusion>
<groupId>io.netty</groupId>
<artifactId>netty</artifactId>
Expand Down Expand Up @@ -666,6 +754,14 @@
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-mapreduce-client-common</artifactId>
</exclusion>
<exclusion>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
</exclusion>
<exclusion>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
</exclusion>
<exclusion>
<groupId>io.netty</groupId>
<artifactId>netty</artifactId>
Expand All @@ -679,6 +775,14 @@
<type>test-jar</type>
<version>${hadoop.version}</version>
<exclusions>
<exclusion>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
</exclusion>
<exclusion>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
</exclusion>
<exclusion>
<groupId>io.netty</groupId>
<artifactId>netty</artifactId>
Expand All @@ -702,6 +806,14 @@
<groupId>org.eclipse.jetty</groupId>
<artifactId>*</artifactId>
</exclusion>
<exclusion>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
</exclusion>
<exclusion>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
</exclusion>
<exclusion>
<groupId>com.sun.jersey</groupId>
<artifactId>jersey-core</artifactId>
Expand All @@ -714,6 +826,10 @@
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
</exclusion>
<exclusion>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
</exclusion>
<exclusion>
<groupId>javax.servlet.jsp</groupId>
<artifactId>jsp-api</artifactId>
Expand All @@ -722,6 +838,14 @@
<groupId>tomcat</groupId>
<artifactId>jasper-runtime</artifactId>
</exclusion>
<exclusion>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
</exclusion>
<exclusion>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
</exclusion>
<exclusion>
<groupId>io.netty</groupId>
<artifactId>netty</artifactId>
Expand Down Expand Up @@ -773,6 +897,18 @@
<version>${junit.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<version>${junit.jupiter.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
<version>${junit.jupiter.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
Expand All @@ -789,13 +925,13 @@
<version>${jettison.version}</version>
</dependency>
<dependency>
<groupId>com.sun.jersey</groupId>
<groupId>org.glassfish.jersey.core</groupId>
<artifactId>jersey-client</artifactId>
<version>${jersey.version}</version>
</dependency>
<dependency>
<groupId>com.sun.jersey</groupId>
<artifactId>jersey-json</artifactId>
<groupId>org.glassfish.jersey.media</groupId>
<artifactId>jersey-media-json-jackson</artifactId>
<version>${jersey.version}</version>
</dependency>
<!-- BouncyCastle should be in default scope to make their way to tez.tar.gz since Hadoop 3.4.1 -->
Expand Down
6 changes: 3 additions & 3 deletions tez-api/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,12 +94,12 @@
<artifactId>junit</artifactId>
</dependency>
<dependency>
<groupId>com.sun.jersey</groupId>
<groupId>org.glassfish.jersey.core</groupId>
<artifactId>jersey-client</artifactId>
</dependency>
<dependency>
<groupId>com.sun.jersey</groupId>
<artifactId>jersey-json</artifactId>
<groupId>org.glassfish.jersey.media</groupId>
<artifactId>jersey-media-json-jackson</artifactId>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
Expand Down
13 changes: 7 additions & 6 deletions tez-api/src/main/java/org/apache/tez/common/ATSConstants.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,15 +27,16 @@ public class ATSConstants {

/* Top level keys */
public static final String ENTITIES = "entities";
public static final String ENTITY = "entity";
public static final String ENTITY_TYPE = "entitytype";
public static final String ENTITY_ID = "entityId";
Comment thread
maheshrajus marked this conversation as resolved.
public static final String ENTITY_TYPE = "entityType";
public static final String EVENTS = "events";
public static final String EVENT_TYPE = "eventtype";
public static final String EVENT_TYPE = "eventType";
public static final String TIMESTAMP = "ts";
public static final String EVENT_INFO = "eventinfo";
public static final String EVENT_INFO = "eventInfo";
public static final String RELATED_ENTITIES = "relatedEntities";
public static final String PRIMARY_FILTERS = "primaryfilters";
public static final String OTHER_INFO = "otherinfo";
public static final String PRIMARY_FILTERS = "primaryFilters";
public static final String OTHER_INFO = "otherInfo";
public static final String DOMAIN_ID = "domainId";

/* Section for related entities */
public static final String APPLICATION_ID = "applicationId";
Expand Down
Loading