Added persistent HTTP session support with connection pooling and improved retry handling

[Argu333]

Added persistent HTTP session support with connection pooling and improved retry handling

Description

First-time contributor here, so please let me know if there are any changes needed to better align with project standards or contribution guidelines.

This PR optimizes the existing session management implementation by introducing persistent HTTP sessions with connection pooling and improving retry/backoff behavior.

Changes include:

• Added a shared requests.Session() instance to reuse connections across API calls.
• Added an HTTPAdapter with connection pooling (pool_connections=200, pool_maxsize=200).
• Updated request execution to utilize the shared session instead of creating a new connection for every request.
• Improved retry handling by resetting the exponential backoff timer once a retry cycle succeeds, preventing previous backoff delays from persisting indefinitely across future retries.
• Maintained existing authentication, token refresh, and retry functionality while improving performance and scalability for high-volume workloads.

While this is marked as a new feature, the intent is primarily to optimize the existing implementation rather than introduce new functionality or alter the public API.

Motivation and Context

The existing implementation established a new HTTP connection for every API request. While functional, this became a significant bottleneck when executing large numbers of API calls, especially when using parallel processing.

By introducing persistent sessions and connection pooling:

• TCP/TLS connections can be reused across requests.
• Connection establishment overhead is significantly reduced.
• Parallel processing scales more efficiently.
• Higher worker counts can be utilized without connection creation becoming the bottleneck.

Additionally, the existing retry logic maintained a global exponential backoff timer that did not reset after a retried request eventually succeeded. In long-running jobs, this would cause future transient failures to immediately inherit the maximum backoff delay, even after the system had recovered and requests were succeeding normally.

This change resets the backoff timer when a retried request succeeds, allowing future retries to start from the initial delay while still preserving exponential backoff protection during periods of sustained rate limiting or service instability.

How Has This Been Tested?

Test Environment

• PCPI version: 1.0.1
• OS: Windows
• IDE: VS Code
• Python virtual environment created specifically for testing

Functional Testing

I tested both:

1. Standard sequential execution (normal for-loop processing)
2. Parallel execution using ThreadPoolExecutor

The most significant improvements were observed during parallel execution, though sequential execution also showed noticeable performance gains.

Test Scenario 1

Workload:

• Approximately 35,000 API calls to the same endpoint.

Results before changes:

• Sequential execution: 20+ hours
• Threaded execution: ~1.5 hours
• Optimal worker count: ~35 workers

Results after changes:

• Threaded execution: ~3 minutes
• Optimal worker count: ~150 workers

I did not re-test the sequential version of this workload after implementing the changes.

Test Scenario 2

Workload:

• Approximately 1,000 account-detail API calls.
• Multiple additional service-detail API calls per account.
• Accounts processed in parallel while services for those accounts were also processed in parallel.

Results before changes:

• Sequential execution: ~3 hours

Results after changes:

• Sequential execution: ~30 minutes
• Threaded execution: ~50 seconds
• Worker count: ~150 workers at both processing levels

Additional Notes

I also tested large threaded workloads using ThreadPoolExecutor by submitting approximately:

• 1,000 API call futures
• 5,000 API call futures
• 35,000 API call futures

For all three test sizes, the overhead of creating and submitting futures to the executor remained relatively consistent at approximately 10-30 seconds. Once the API calls began executing, the impact of connection reuse and pooling became apparent.

Observed end-to-end runtimes were approximately:

• ~1,000 API calls: ~35 seconds
• ~5,000 API calls: ~55 seconds
• ~35,000 API calls: ~3 minutes

These results were achieved using approximately 150 worker threads. The relatively small increase in runtime despite substantial increases in API volume highlights the scalability improvements gained from persistent sessions and connection pooling.

I verified:

• Authentication/login functionality
• Token refresh behavior
• Retry handling for 401, 429, and 5xx responses
• Retry timer reset behavior after successful requests

Types of changes

• New feature (non-breaking change which adds functionality)

Checklist

• I have updated the documentation accordingly.

  • I have not updated the documentation because I am unsure whether these changes require documentation updates. Guidance is appreciated.

• I have read the CONTRIBUTING document.

• I have added tests to cover my changes if appropriate.

  • I would appreciate guidance on what tests would be expected for this type of change and where they should be added.

• All new and existing tests passed.

  • I am unsure of the project's expected test execution process and would appreciate guidance on validating this correctly before marking it complete.