Fix brittle layer count

[jloftin-nv]

Description

Fixes test_get_layer_info flakiness across GPU

Type of change

Please delete options that are not relevant and/or add your own.

• Bug fix (non-breaking change which fixes an issue)

Checklist:

• [X ] My code follows the style guidelines of this project (You can use the linters)
• [ X] I have performed a self-review of my own code
• [X ] I have commented my code, particularly in hard-to-understand areas and hacks
• [X ] I have made corresponding changes to the documentation
• [ X] I have added tests to verify my fix or my feature
• [X ] New and existing unit tests pass locally with my changes
• [ X] I have added the relevant labels to my PR in so that relevant reviewers are notified

[narendasan]

LGTM