Skip to content

fix(audio): keep built-in mic capture alive on Apple Silicon - #912

Open
Defake wants to merge 5 commits into
altic-dev:mainfrom
Defake:852-built-in-mic-fix
Open

fix(audio): keep built-in mic capture alive on Apple Silicon#912
Defake wants to merge 5 commits into
altic-dev:mainfrom
Defake:852-built-in-mic-fix

Conversation

@Defake

@Defake Defake commented Aug 28, 2026

Copy link
Copy Markdown

Description

Direct Core Audio capture from the built-in microphone can become silent a few seconds after dictation starts on Apple Silicon, which truncates the transcription.

This change uses AVAudioEngine for the built-in microphone on Apple Silicon.

Type of Change

  • 🐞 Bug fix
  • ✨ New feature
  • 💥 Breaking change
  • 🧹 Chore
  • 📝 Documentation update

Related Issue or Discussion

Closes #852

Testing

  • Tested on Intel Mac
  • Tested on Apple Silicon Mac
  • Tested on macOS version: 26.6.2
  • Ran linter locally: swiftlint --strict --config .swiftlint.yml Sources
  • Ran formatter locally: swiftformat --config .swiftformat Sources
  • Ran tests locally: 334/335 passed on Apple Silicon; The Whisper Tiny E2E test failed because its Hugging Face model download timed out after three attempts (no access to internet during testing – was intended).

Screenshots / Video

  • No UI/visual changes; screenshots/video are not applicable.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b79d729a95

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread Sources/Fluid/Services/ASRService.swift Outdated
@greptile-apps

greptile-apps Bot commented Aug 28, 2026

Copy link
Copy Markdown

Greptile Summary

The PR adds a silent built-in-output keep-alive while direct Core Audio captures from the internal microphone on Apple Silicon.

  • Selects the single live built-in output for eligible internal-microphone sessions.
  • Starts and synchronizes the output keep-alive before input capture, then tears both down together.
  • Adds output-liveness querying and policy-focused reliability tests.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Reviews (3): Last reviewed commit: "fix(audio): keep built-in output active ..." | Re-trigger Greptile

Comment thread Sources/Fluid/Services/ASRService.swift Outdated
@Defake
Defake marked this pull request as draft August 28, 2026 15:28
@Defake
Defake marked this pull request as ready for review August 28, 2026 15:42
Comment thread Sources/Fluid/Services/ASRService.swift Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 02aa44437f

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread Sources/Fluid/Services/ASRService.swift Outdated
Comment on lines +5318 to +5320
isInternalMicrophone &&
selectedInputUID != nil &&
selectedInputUID == defaultInputUID

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep non-default built-in microphones off direct capture

When the user prioritizes the built-in microphone in FluidVoice while macOS has an external input as its default, this equality makes the compatibility policy return false, so startConfiguredAudioCapture falls through to DirectCoreAudioLifecycleController for the built-in device. That is precisely the backend this change identifies as becoming silent after a few seconds on Apple Silicon, so dictation can still be truncated in this supported independent-routing configuration. The new UID-equality guard avoids the previously reported misrouting, but the fresh behavior is that it routes the selected built-in microphone back through the known-silent backend; the compatibility solution needs to preserve the selected endpoint rather than opting out.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's intended. Otherwise the app would ignore the selected user choice.

@altic-dev

Copy link
Copy Markdown
Owner

Hi, what is this PR supposed to do again? You have an issue?

Microphone is a very sensitive area and the issues that come up are very large. So I'd be very careful before merging this. But if you want to have a discussion on this, we are happy to take a look at it.

@grohith327 for viz

@Defake

Defake commented Aug 29, 2026

Copy link
Copy Markdown
Author

@altic-dev yes, FluidVoice is not usable for me since 1.6.7.

The issue I had created: #852

This PR fixes that built-in microphone stops capturing input after 3 seconds of dictation on Apple M1.

It switches built-in microphone capture to AVAudioEngine instead of using direct Core Audio. Only switches if both selected and default microphone are built-in and it's Apple Silicon.

@altic-dev

Copy link
Copy Markdown
Owner

@altic-dev yes, FluidVoice is not usable for me since 1.6.7.

The issue I had created: #852

This PR fixes that built-in microphone stops capturing input after 3 seconds of dictation on Apple M1.

It switches built-in microphone capture to AVAudioEngine instead of using direct Core Audio. Only switches if both selected and default microphone are built-in and it's Apple Silicon.

We're trying to move away from AvAudio and w e need to find a bette way for this. @grohith327 wdyt?

Not sure how to fix this in a bette way

@grohith327

Copy link
Copy Markdown
Collaborator

@Defake The AVAudioEngine flow that exists right now will simply be removed in future commits as we have fully committed to the CoreAudio flow.

The equivalent fix for this in CoreAudio flow is there in this commit - 6fe6143

Can you try building this and let us know if it works. Thank you for trying really hard to get FV work!

@Defake

Defake commented Aug 30, 2026

Copy link
Copy Markdown
Author

@grohith327 thanks, I tried it 2 weeks ago, it didn't fix the issue: #852 (comment)

The problem is not zero-PCM. Capture starts normally and works for 3.5 seconds. Then corespeechd mutes the built-in microphone.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 95b08c30a4

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +645 to +648
status = fv_start_output_keep_alive(capture);
if (status != noErr) {
(void) fv_stop_and_destroy_output_keep_alive(capture);
return status;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Allow capture when the output keep-alive cannot start

When the built-in output is alive but cannot start this IOProc—for example, while held in exclusive/hog mode or when its first callback exceeds the fixed one-second timeout—this path aborts startup even though the selected microphone remains independently usable. startConfiguredAudioCapture propagates direct-capture failures, and the direct backend is mandatory, so after retrying or switching inputs the recording ultimately fails; treat a keep-alive failure as a degraded capture mode or fall back to another backend instead of making output availability a prerequisite for dictation.

Useful? React with 👍 / 👎.

@Defake

Defake commented Aug 30, 2026

Copy link
Copy Markdown
Author

I reworked this PR so it works with Core Audio now.
On my Mac, direct Core Audio capture starts normally, but corespeechd mutes the built-in microphone after 3.5 seconds when no output stream is active. The PR now runs a silent Core Audio output callback on the built-in output while dictation is active.
@altic-dev @grohith327 please take a look.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Dictation stops detecting microphone input after 3 seconds

3 participants