Improve error messages when render target discovery encounters permissions issues

[xtimbeau]

I have:

• searched the issue tracker for similar issues
• installed the latest version of Quarto CLI
• formatted my issue following the Bug Reports guide

Bug description

A _quarto.yml file placed somehow high in the hierarchy of folder (/users/me/_quarto.yml) would crach a lot of quarto commands whith a difficult to interpret error message.

Steps to reproduce

On MacOS at least, put a _quarto.yml in /users/{you}/_quarto.yml with the following content

project:
  type: website
format:
  html: default

then run quarto check from the terminal anywhere under /users/{you}/. You should get the error message: (user folder has been changed)

me ~ % quarto check
ERROR: PermissionDenied: Operation not permitted (os error 1): readdir '/Users/me/Pictures/Photos Library.photoslibrary'

Stack trace:
    at async Object.[Symbol.asyncIterator] (ext:deno_fs/30_fs.js:222:19)
    at async walk (file:///Applications/quarto/bin/quarto.js:2884:20)
    at async walk (file:///Applications/quarto/bin/quarto.js:2917:7)
    at async walk (file:///Applications/quarto/bin/quarto.js:2917:7)
    at async addDir (file:///Applications/quarto/bin/quarto.js:92493:22)
    at async projectInputFilesInternal (file:///Applications/quarto/bin/quarto.js:92536:18)
    at async projectContext (file:///Applications/quarto/bin/quarto.js:92106:36)
    at async initializeProjectContextAndEngines (file:///Applications/quarto/bin/quarto.js:144407:19)
    at async _Command.actionHandler (file:///Applications/quarto/bin/quarto.js:164506:3)
    at async _Command.execute (file:///Applications/quarto/bin/quarto.js:102189:7)
    at async _Command.parseCommand (file:///Applications/quarto/bin/quarto.js:102066:14)
    at async quarto4 (file:///Applications/quarto/bin/quarto.js:188195:5)
    at async file:///Applications/quarto/bin/quarto.js:188224:5
    at async file:///Applications/quarto/bin/quarto.js:188078:14
    at async mainRunner (file:///Applications/quarto/bin/quarto.js:188080:5)
    at async file:///Applications/quarto/bin/quarto.js:188217:3
me ~ % 

Actual behavior

the error message

Expected behavior

a normal quarto diagnosis

Your environment

macos 26.6.2

Quarto check output

See above

[cderv]

Thanks for the report.

quarto commands will use which _quarto.yml found in the folder hierarchy. This is the case so you can call quarto render on a file inside a project and still have the correct quarto project be used.
And this applies to quarto check too.

I am curious what a normal quarto diagnosis is to you in this case ?

The error you encounter is because for some quarto project, there will be a file listing operation to find all .qmd files in the project to be able to know what to render. As /users/{you}/_quarto.yml is the file found, project starts at /users/{you}/ and then search goes into all subfolders - /Users/me/Pictures/Photos Library.photoslibrary is one of them.

I agress this is not ideal error. Do you expect PermissionDenied: Operation not permitted to be skipped ? or just a better frame error that tells you the file can't be read as part as render target discovery step ?

Just curious about the expected results that would seem better to you.

Thank you

[xtimbeau]

Hi @cderv, thanks for the quick answer (and the great job on quarto) !

By a normal quarto diagnosis I mean the informative result of quarto check instead of a failure.

I guess the most logical thing would be to ignore folder you can't scan is skipped so you can put your _quarto.yml where ever you want and it still works.
Let me give you some context: I did this manipulation unintendedly and I had some difficulties discovering why it was failing. Funny though, quarto inspect was called programmatically before rendering in a folder with a _quarto.yml so the extra-high-in-the-hierarchy-forgotten yml would not have been such a problem.

For my use case, the best would have been not to fail on this one and still return the probably not very reasonnable result of quarto inspect as all .qmd in my user folder would have been scanned -- but it would have helped in finding that lost yml.

[cderv]

so you can put your _quarto.yml where ever you want and it still works.

Just to be clear on that point: Where your _quarto.yml as an impact, it defines what will be considered project root. And _quarto.yml found or not found will define if Quarto considers this is a Quarto project, or we are in case of what we called Single File render. This changes the context.

Let me give you some context: I did this manipulation unintendedly and I had some difficulties discovering why it was failing.

I am thinking we should add in quarto check if and where a _quarto.yml has been found. This would probably help.

Thanks for the feedback. If you have any more thoughts please do share.

[cderv]

I looked more closely at this and I can reproduce with any unreadable folder under the project root (tested on Windows with a folder denied for listing). quarto check, quarto inspect, and even quarto render doc.qmd on a file inside the project all fail with the same raw PermissionDenied and stack trace.

quarto check does not actually need the list of project files. It loads the project only to find engines declared in _quarto.yml, but it goes through the same function that also lists every input file of the project:

quarto-cli/src/command/check/cmd.ts

Line 32 in 9b307f7

await initializeProjectContextAndEngines();

quarto-cli/src/project/project-context.ts

Lines 947 to 962 in 9b307f7

	const addDir = async (dir: string): Promise<FileInclusion[]> => {
	const promises: Promise<FileInclusion[]>[] = [];
	for await (
	const walkEntry of walk(dir, {
	includeDirs: false,
	// this was done b/c some directories e.g. renv/packrat and potentially python
	// virtualenvs include symblinks to R or Python libraries that are in turn
	// circular. much safer to not follow symlinks!
	followSymlinks: false,
	skip: [kSkipHidden].concat(
	engineIgnoreDirs().map((ignore) =>
	globToRegExp(join(dir, ignore) + SEP)
	),
	),
	})
	) {

So in your case quarto check was walking your whole home folder for nothing, and it would also create a .quarto folder there.

What we could do:

• quarto check only reads the project configuration, without listing the files. It then can't fail on an unreadable folder.
• quarto check reports which _quarto.yml was found and where the project root is, or that no project was found. And a warning when the project root is the home folder or filesystem root, as this is very likely a mistake.
• For quarto render and quarto preview, which do need the file list, keep the error but make it explicit: which folder can't be read, and which _quarto.yml defines the project root this folder belongs to. No stack trace.

About skipping unreadable folders: I don't think we should. quarto inspect is meant to be read by tools and needs to be complete, so we will keep it as is. A result silently missing files would be wrong in a way that is hard to notice. For quarto render it is the same, a website rendered with missing pages without failing is not better. With the error message pointing to the _quarto.yml, you would have found the forgotten file directly I think.

Does that match what would have helped you ?