DAOS-19396 pool: mind pool_discard() dss_rpc_send() errors - #18894
Conversation
Experimental product code change on release/2.6 branch to *not* proceed to pool map update step of ds_pool_update_handler() for reintegration operations when the previous pool_discard() step got an error in dss_rpc_send() (e.g., a -DER_TIMEDOUT failure). This gives retries either by the MS host daos_engine dsc_pool_client or by the test program itself (re-issuing the dmg pool reintegrate command) a chance to wait out and possibly recover from whatever communication failures are occurring among some engines and the reintegrating engine. Signed-off-by: Kenneth Cain <kenneth.cain@hpe.com>
|
Test stage Build RPM on Leap 15.5 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-18894/1/execution/node/308/log |
|
Test stage Build RPM on EL 9 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-18894/1/execution/node/394/log |
|
Test stage Build RPM on EL 8 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-18894/1/execution/node/429/log |
|
Test stage Unit Test bdev on EL 8.8 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos/job/PR-18894/1/display/redirect |
|
Test stage Unit Test bdev with memcheck on EL 8.8 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos/job/PR-18894/1/display/redirect |
|
Test stage Unit Test on EL 8.8 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos/job/PR-18894/1/display/redirect |
|
Test stage Unit Test with memcheck on EL 8.8 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos/job/PR-18894/1/display/redirect |
|
Test stage NLT on EL 8.8 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos/job/PR-18894/1/display/redirect |
|
Ticket title is 'daos_test/suite.py:DaosCoreTest.test_daos_rebuild_ec - pool reintegrate timeout' |
Experimental product code change on release/2.6 branch to not proceed to pool map update step of ds_pool_update_handler() for reintegration operations when the previous pool_discard() step got an error in dss_rpc_send() (e.g., a -DER_TIMEDOUT failure).
This gives retries either by the MS host daos_engine dsc_pool_client or by the test program itself (re-issuing the dmg pool reintegrate command) a chance to wait out and possibly recover from whatever communication failures are occurring among some engines and the reintegrating engine.
Steps for the author:
After all prior steps are complete: