cocoaKinect

Archive / fork. This is not original work by the repository owner. cocoaKinect was written in November 2010 by Robert Pointon (fernlightning) — rpointon@fernlightning.com, IRC nick <baby-rabbit> — as a demo for the OpenKinect community. This repository preserves that source, cleaned up and documented for archival purposes. All credit for the application code goes to the original author. See notes.txt for the author's own notes, reproduced verbatim.

A native macOS (Cocoa + OpenGL) viewer for the Microsoft Kinect, built directly on libfreenect. It renders the depth and video streams in real time and exports the captured geometry to mesh files.

Mesh mode: the depth stream triangulated in real time, shaded by the built-in depth colormap — near surfaces warm, far ones cool. The wireframe density is the detail setting, which decimates the grid before triangulating it.

Features

• Live depth, RGB and IR streams from the Kinect
• Three draw modes: 2D image, point cloud, and triangle mesh
• Adjustable depth clipping (minDepth / maxDepth), detail/decimation, mirror, surface normals, and background removal
• Motor tilt control and LED control
• Per-stream FPS counters (depth, video, view)
• Export to PLY, ASCII STL, and binary STL, with a delayed snapshot timer

Repository layout

.
├── README.md                     This file
├── THIRD-PARTY-LICENSES.md       Licensing of every vendored dependency — read before redistributing
├── docs/screenshot.jpg           Mesh mode, captured with the original 2010 build
├── scripts/syntax-check.sh       Parses the sources against the current SDK (see below)
├── notes.txt                     Original author's notes (2010), verbatim
└── src/
    ├── cocoaKinect.xcodeproj/    Project file (objectVersion 46, SDKROOT macosx10.6)
    ├── AppDelegate.{h,m}         Kinect device handling, stream callbacks, mesh export
    ├── GLView.{h,m}              OpenGL view — rendering of points/mesh/2D modes
    ├── GLProgram.{h,m}           GLSL shader loading and linking helper
    ├── main.m                    Entry point
    ├── English.lproj/            MainMenu.xib and localized strings
    ├── kinect/                   Vendored libfreenect snapshot (upstream rev 2402fcd)
    │   ├── include/ src/         Library sources actually compiled into the app
    │   ├── APACHE20 GPL2 CONTRIB License texts and author list
    │   └── OpenKinect-libfreenect-2402fcd
    │                             Empty marker file recording the upstream revision
    └── libusb-1.0/               Vendored libusb 1.0 header + prebuilt binaries
        ├── libusb.h
        ├── lib/                  libusb-1.0.a, libusb-1.0.0.dylib (+ symlinks)
        └── COPYING AUTHORS       LGPL-2.1 text and copyright holders

The vendored dependencies stay inside src/ because the project references them relative to SOURCE_ROOT (HEADER_SEARCH_PATHS, LIBRARY_SEARCH_PATHS). Moving them would break the build.

Building

Open src/cocoaKinect.xcodeproj and build the cocoaKinect target.

It builds and runs on an Intel Mac. That is the maintainer's report from an actual machine — this repository's CI only parses the sources, so nothing here independently confirms it.

On Apple Silicon it will not link as-is: the vendored libusb-1.0 binaries are x86_64 only — lipo -info reports a non-fat x86_64 .dylib and .a — so they have to be rebuilt or replaced with a universal build first. The project also declares SDKROOT = macosx10.6 and uses the legacy fixed-function OpenGL pipeline, deprecated but still functional on current systems.

A Copy Frameworks build phase stages libusb into Contents/Frameworks/libusb-1.0, and a run script rewrites the install name:

install_name_tool -change /usr/local/lib/libusb-1.0.0.dylib \
  @executable_path/../Frameworks/libusb-1.0/libusb-1.0.0.dylib \
  ${TARGET_BUILD_DIR}/cocoaKinect.app/Contents/MacOS/cocoaKinect

Refreshing the vendored libraries

From the original author's notes:

• libfreenect — copy the include and lib folders from the c folder of the upstream git project into src/kinect/. Files may need to be added to or removed from the project as upstream changes.
• libusb — after the usual configure / make / install, copy the files from /usr/local/lib and /usr/local/include into src/libusb-1.0/.

Changes from the original

The first commit in this repository is the 2010 source as received, only reorganised. Everything below was applied on top of it, so git diff against that commit shows the full divergence from the original.

Performance, in the real-time render path (GLView.m, drawScene):

• The per-pixel depth clamp called [ctrl getDetail], [ctrl getMin] and [ctrl getMax] inside the loop — up to 7 objc_msgSend per pixel — plus two fmod() calls per pixel, across 307200 pixels every frame. The accessors are now hoisted and the modulo replaced by the loop's own column index. Measured on an M-series Mac: 4.69 ms → 0.71 ms per frame (6.6x).
• The mesh index builder walked x outer / y inner, striding FREENECT_FRAME_W * 2 bytes per step through a row-major buffer. Loop order is swapped so the walk is sequential: 0.087 ms → 0.019 ms per frame at full detail.
• Each frame released its depth and video buffers with free() and the producer immediately malloc'd replacements — 600 KB and 900 KB blocks, above the allocator's mmap threshold, so every frame paid two mmap/munmap round trips plus page faults. -recycleDepthData: / -recycleVideoData: now hand buffers back for reuse. Calling free() on them instead remains correct, so no existing caller was invalidated.

Both rewritten loops were checked against the originals over 960 randomised parameter combinations (depth buffers, detail 0-40, min/max sweeps): byte-identical output, and identical index sets and counts for the mesh.

Correctness and leaks:

• saveSTLB declared its facet counter as NSUInteger *faceCount — a pointer — so faceCount + 2 was pointer arithmetic advancing by 16. The binary STL header announced 8x more facets than were written. Now a uint32_t, matching the 4-byte field the format expects.
• binaryVector: did [[NSMutableData alloc] autorelease] with no init.
• All three export paths leaked their 600 KB depth buffer on every invocation, and savePly / saveSTL leaked their accumulator strings. Fixed.
• PLY and STL text generation built one NSString per coordinate — 12 temporary objects per facet. Replaced by a single format pass producing byte-identical output.

Robustness:

• All three export actions waited for a frame with while (!depth) { depth = [self createDepthData]; }, which has no exit. With the device stopped or absent — and the export menu items stay enabled in that state — _depthUpdate never turns YES, so this pinned a core on the main thread and froze the UI permanently. Replaced by -waitForDepthDataUntil:, a bounded wait that gives up after two seconds and reports it in the status field.
• GLProgram.m logged shader compile failures with NSLog(@"...%s", desc, code) where desc is an NSString *. %s read the object pointer as a C string: garbage output or a crash, on the single path whose job is to report shader errors. code was passed but never printed.
• -[GLView dealloc] released the display link without stopping it first, while its callback still draws into the view, then ran closeScene's glDelete* calls without making the view's context current — deleting whatever context happened to be bound.
• _device and _halt are each written on one thread and polled on the other with no barrier. The polls only work because they call opaque functions that force a reload; both are now volatile so that is explicit rather than accidental. The underlying race is unchanged — a proper fix would use a condition variable.
• Mesh indices now stream through an element buffer object instead of a client-side array the driver must copy inline at draw time.

Interface:

• The export actions were never validated, so they stayed clickable with no device running — the exact state in which they hung. -validateMenuItem: and -validateUserInterfaceItem: now gray them out. AppKit validates menu and toolbar items only; plain buttons in the nib still need their enabled state bound.
• Deprecated AppKit calls replaced: runModalForDirectory:file: → setDirectoryURL: + runModal, NSOKButton → NSModalResponseOK, NSShiftKeyMask → NSEventModifierFlagShift, and writeToFile:atomically: → the encoding:error: form. This raises the deployment target well above the 10.6 the project declares — it is a step toward building against a current SDK. It will not compile against the 10.6 SDK the project still declares.
• setAllowedFileTypes: is deprecated in favour of allowedContentTypes, which needs the UniformTypeIdentifiers framework added to the project. Left as is rather than editing the project file blind.

Left alone deliberately:

• saveSTLB has a face-detection condition mixing && and || without parentheses (-Wlogical-op-parentheses). The precedence the compiler applies may not be what the author meant, but that cannot be settled without hardware, and parenthesising it would freeze in one reading. Flagged, not touched.
• Point mode always draws all 307200 points regardless of the detail setting. Changing it would alter what the app renders.
• prepareOpenGL and reshape do not call super, and the video offset properties pair a synthesized setter with a hand-written getter under atomic. Both are warnings on a rendering path that cannot be re-tested here.

None of this is verified against real hardware. The changes compile clean under clang -fsyntax-only with the current macOS SDK, and the loop rewrites are covered by the equivalence tests described above, but the app was never built or run against a Kinect — see the build caveats above.

Licensing

There is no formal license on the application code. main.m carries Copyright 2010 fernlightning. All rights reserved., while notes.txt grants permission informally:

The project is for demo purposes. It's the same as https://github.com/OpenKinect/libfreenect/ — frankly I don't care what you do with this code, hopefully you'll be nice and add me to your credits

That grant is the original author's stated intent, not a license text. No LICENSE file has been added here, because the repository owner is not the copyright holder and cannot license someone else's work. If you intend to reuse this code beyond archival or study, contact the original author.

The vendored dependencies do carry formal licenses (libfreenect: Apache-2.0 or GPL-2.0 at your option; libusb: LGPL-2.1). Their terms, obligations and copyright notices are detailed in THIRD-PARTY-LICENSES.md.

Credits

• Robert Pointon (fernlightning) — cocoaKinect application, 2010
• The OpenKinect project — libfreenect; original code and engineering by Hector Martin (marcan), community lead Josh Blake, integration by Kyle Machulis
• The libusb project — libusb-1.0