Repository navigation
4 | GitHub Status Badge Pathfinding - #39
Draft
ReedClanton wants to merge 122 commits into
Draft
ReedClanton wants to merge 122 commits into
ReedClanton wants to merge 122 commits into
Conversation
Splitting runs of unit tests that include profiling and code coverage calculation out into a separate workflow that's not run as often. This speeds up the workflow that's run more often.
Probably not the problem.
Testing out fix. Also removed debugging code.
The shell is not packaged.
Changed how shfmt (style checking tool) is installed. Rather than using a snap, I just download the binaries directly. This is a lot faster and allows me to pin the version so updates to shfmt in the future don't break my pipeline. I created actions for installing shells to both Ubuntu and MacOS. I liked this more than including the install of everything in one action. It makes the workflow files easier to read, while still allowing for a lot of re-use. In addition to the re-use, it allows for error checking so missing shell installs don't show up as failing tests. I updated all workflows to utilize the new actions and removed the old workflows and actions. Also made minor updates to the style checking workflows so they are less wordy.
The path to the CUT should be reflected by the path to the tests. This will also fix the pipeline running of those tests.
ShellSpec is now profiling tests, kcov is in use, and zsh is found, however, code coverage is reporting at 0%. Gotta figure out why that's happening.
I think it's just failing because of #20. Need to work that.
Also renamed badge to reflect what it really represents.
Now that I've tested that the failure badge works, I've removed the failing test.
The shell does some strange stuff were it doesn't have POSIX stuff builtin any more. Instead, it outsources them to programs that it adds to the path. This is an issue because ShellSpec clears out `PATH` when testing if a shell is POSIX compliant. Because of this, newer versions of yash no longer work with ShellSpec. There's an issue relating to this on ShellSpec's project, and there are things that I could do to bypass this, but I've never heard of yash, so I don't really care to support it. Especially given how much work it would be. I did test out yash version 2.53. That was before the yash change that broke compatibility with ShellSpec. There were test failures, so I would have to *also* update my code... just not worth it.
Rather than each file controlling when it's run, I'm attempting to switch to a single workflow that controls when every other workflow is run. I'm doing this to centralize this logic so it's not spread out across dozens of files. Also, given what I want, it's kinda complicated. GitHub doesn't allow me to do exactly what I want out of the box. Note that I've documented what I want and how it *should* work in the `README.md` files.
Badge creation is now handled by `ci.yml`, so it shouldn't be called from each file.
Added more stuff I noticed that must be done to PR template.
When a file or directory is renamed, it will ultimately still break the pipeline if the pipeline is not updated, but this is, arguably an improvement.
Code coverage will now be triggered in in the same way as other tests are. Code coverage takes longer to run, and runs the same tests that are already run, however, I think it's important to know when code coverage drops. I intend on marking the code coverage portions as depending on the non code coverage workflows. This way we don't bother running test that are known to fail.
After thinking about this yesterday, I think that only changes to the CUT should trigger unit tests rather than also including changes to the dependencies of the CUT. Long term, I'd like to create a basic set of functional tests that verify the interfaces between..
There's no sense in calculating code coverage when tests are failing.
The shell I'm primarily targeting is zsh. So if tests fail on dash, I still want to know what level of code coverage I have. If tests fail on zsh, the sky is falling, so I don't give a shit about code coverage.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of Changes
Split up runs of unit tests that include profiling and code coverage calculation out into a separate workflow that's not run as often. This speeds up the workflow that's run more often.
Reduced what gets run when so the pipeline takes less time. For example, if you touch some code, the the pipeline for that portion of code will run, but not the entire pipeline.
Fixed some style guide issues after adding style guide checks to more of the code base. Note that the
backUp()function doesn't pass the style guide check. I may end up removing the check if I can't get it to pass and writing a GitHub issue to cover the work.Finally, I added badges to tell a user looking at the documentation if the code runs on the shell they are using.
The Basics
These should be checked immediately upon PR (Peer Review) creation:
<feature/bug>/<issueId>-<issue-title-with-dashes-between-and-no-capitalization><issueId> | <Issue Title>main).Draftwhen not ready for review.Check Before Moving Out of
DraftREADME.mdat the root of the project.README.mdat the root of the unit testing directory.README.mdof all code touched, including unit tests:main.shfiles).<repoRoot>/.github/badge-dependencies..ymlfiles that run the unit tests in the pipeline to increase the minimum coverage level required for pipeline success.--fail-low-coverageShellSpecOption *Always* Causes Pipeline Failure #20 is worked, pipeline doesn't enforce failure due to lack of code coverage!spec_helper_precheck()function in<repoRoot>/spec/spec_helper.sh.Check Before Each Push
README.mdof all code touched, including unit tests:main.shfiles)..ymlfiles that run the unit tests in the pipeline to increase the minimum coverage level required for pipeline success.spec_helper_precheck()function in<repoRoot>/spec/spec_helper.sh.Check After Final Push
README.mdat the root of the repository.README.mdat the root of the unit testing directory.README.mdof all code touched, including unit tests:main.shfiles)..ymlfiles that run the unit tests in the pipeline to increase the minimum coverage level required for pipeline success.spec_helper_precheck()function in<repoRoot>/spec/spec_helper.sh.Draft.