Add more resource plotting to the reports - #357
Open
harriscr wants to merge 3 commits into
Open
Conversation
added 3 commits
July 23, 2026 15:51
When generating a report CBT can pull CPU and memory usage from the fio output files. This is not very useful, however as it only reports the resource usage for the fio process itself. CBT also supports running other resource monitoring tools such as top and collectl. This PR deals with specifically plotting CPU and memory usage for collectl. The basic infrastructure already existed, so all that was needed was the pieces to parse the collectl output files and store the resource usage data in the intermediate format. The code changes were: - a factory method to instantiate the correct resource_result class to process resouce usage data - the collectl resource_result class - updates to the run_result to use the factory method - plot multiple CPU lines - unit tests fpr the new code Signed-off-by: Chris Harris <harriscr@uk.ibm.com> Assisted-by: IBM Bob 1.03 and 2.0
Resource monitoring in CBT is contained within the monitoring class in CBT. This file contains multiple classes, one for each potential tool to use. To make this easier to maintain in the future and to meet current Python coding guidelines this should be split up into once class per file. These should have a common abstract base class which they all ingerit from, and a factory method to make sure the expected monitoring classes are run Signed-off-by: Chris Harris <harriscr@uk.ibm.com> Assisted-by: IBM Bob 2.0
Add plotting of top and collectl resource monitoring to CBT. This required a slight re-factoring of both top and perf monitoring as they were specifically targetted twards OSDs, requiring an OSD PID file and only monitoring OSD PIDs, no matter what arguments were passed. There is now a generic TopMonitoring and PerfMonitoring class, which will run the arguments passed in the YAML, an an Osd*Monitoring class that will require the OSD PID file as the current behaviour. Signed-off-by: Chris Harris: <harriscr@uk.ibm.com> Assisted-by: IBM Bob v 2.0 and 2.0.1
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The post processing modules in CBT can produce a plot with CPU usage during a run. Currently only the CPU usage as reported by the FIO process can be plotted.
CBT supports additional tools for collecting resource usage statistics such as top, collectl and perf.
This PR adds support for resource usage as reported by the top and collectl tools.
As part of this work a minor refactor of the monitoring classes was done to bring them up to modern python coding standards. Additionally the top monitoring relied on an OSD pid file being present, which is a very specific use case.
This is still supported, but as a sub-class of a more generic top monitoring.
IBM Bob 2.0 and 2.0.1 was used to assist with this code
Testing
This was tested by:
Creating a report containing top and collectl CPU usage performance_report_260724_125457.pdf
A regression test for creating a report without and CPU uasage statistics.