A modular framework for precise multisensory stimulus generation in systems neuroscience.
stimpack presents multisensory stimuli to a subject, in open or closed loop, and records what was presented with timing precise enough to line it up with whatever the rig acquires alongside. It drives perspective-corrected visual displays, sound playback, movement trackers and analog output hardware from one protocol, and keeps everything specific to a particular lab — rig geometry, hardware drivers, protocols — outside the package.
📖 Documentation · 🚀 Your first stimulus · 🤝 Contributing
Requires Python ≥ 3.10.
python3 -m venv .stimpack
source .stimpack/bin/activate # Windows: .stimpack\Scripts\activate
pip install stimpackBoth data backends (HDF5 and NWB) are installed; pick one per config, or in the startup
dialog. To work on stimpack itself, clone the
repository and pip install -e .[test].
Running stimpack opens the experiment GUI. See the
installation guide if it doesn't.
From protocol code to a running experiment. Left: the built-in AudiovisualPairing protocol,
abridged, and the module calls stimpack makes from its descriptors on each trial — one visual,
one audio, one shared start. Right: the Main tab mid-run, its parameter fields built from the
class's own declarations, so a new protocol is drivable without writing interface code; the
list-valued freq sweeps across trials, and "This trial" shows the current draw.
from stimpack.visual_stim.stim_server import launch_stim_server
from stimpack.visual_stim.screen import Screen
from time import sleep
manager = launch_stim_server(Screen(fullscreen=False, vsync=True))
sleep(2)
manager.load_stim(name='Checkerboard')
manager.start_stim()
sleep(2)
manager.stop_stim(print_profile=True)More, all runnable, in examples/.
An experiment runs as several processes:
ExperimentGUI ── BaseClient ──socket── BaseServer ──┬── visual ── one subprocess per screen (GL)
├── locomotion ── tracker
├── voltage_out ── DAQ
└── audio ── sound card
The client runs the protocol, decides what each trial contains, and writes the data file. The server owns the hardware and usually runs on the rig machine while the client runs wherever the experimenter is sitting. Each screen is its own subprocess with its own GL context, so one display stalling cannot stall another.
They talk over a small JSON protocol, addressed to a module:
manager.target('visual').load_stim(name='MovingPatch', width=10, height=30)
manager.target('voltage_out').output_step(output_channels='DAC0', pre_time=0, step_time=1)
A protocol names the module each call is for, and the server routes it there. Inputs update a subject state that outputs follow, so the closed loop does not pass through the client. The four modules shown all ship with stimpack; a lab adds a further capability as a new module rather than a change to the core.
Calls are one-way. There is no return value to branch on, and attribute access alone never
fails — a mistyped name still produces a callable. The failure isn't silent, though: the server
pushes messages back over the same link, so a name it doesn't have is reported — an error that
aborts the run when the call can only be a mistake, a warning when it's a legitimate difference
between rigs. Use has_server_function() to check before calling, and stimpack --check-labpack
to catch the rest before a run.
Stimuli are rendered for a subject at a known position relative to a display of known size and placement, so an object subtends the angle it should. There are two paths, chosen by the type of screen, and one rig may use both.
Flat screens are described by the three physical corners of each region, in meters — pa
lower-left, pb lower-right, pc upper-left. Each stimulus is drawn once per region through a
generalized off-axis perspective (Kooima 2009),
which is what corrects for a screen that is neither square to the subject nor equidistant from it.
Curved screens — a bowl, a cylinder — cannot be described by a flat frustum, so they render through a cube map instead. The scene is drawn into the faces of a cube from the subject's position, then the screen is drawn once in projector coordinates, each fragment sampling the cube along its own direction. The screen is described by a surface and a projector:
# a 7.7 cm bowl in front of the subject, lit by a projector on the bowl's own axis
CurvedScreen(
surface=SphericalSurface(radius=0.0775, elevation_range=(25, 90), pole=(0, 1, 0)),
projector=PinholeProjector(position=(0, 0.35, 0), look_at=(0, 0, 0), up=(0, 0, 1),
throw_ratio=1.58),
) # 94% of the bowl lit, ±65° azimuth and ±53° elevation
One stimulus specification, three display geometries, one visual experience. A single monitor, two monitors meeting ahead of the subject, and a hemisphere lit by a projector (a) each receive their own frames (b) — the same scene, warped per display — yet the subject's visual field, reconstructed from those very frames (c), matches wherever coverages overlap. Red: the photodiode watching each display's synchronization square; its trace (d) catches two dropped frames.
Because the screen is one draw call however finely it is tessellated, the cost scales with the scene
and the number of cube faces — not with the screen's complexity. draw_curved_screen() plots the
geometry, and the mesh reports its own coverage and pixels-per-degree, so a rig can be checked before
anything is projected onto it.
stimpack contains no hardware-specific code. A labpack is a lab's own directory of protocols,
rig configs, custom stimuli and device drivers, kept in its own repository and pointed at by a
config file. See labpack-template to start one,
and --check-labpack to verify it.
Experiments write HDF5 by default, or NWB with data_format: nwb in the config. One GUI handles
both. See the config reference.
pip install -e .[test]
pytest -m unit # fast; no GL, GUI or hardware
pytest # everything the machine can runTiers and what each needs are described in CONTRIBUTING.md.
If stimpack contributes to work you publish, please cite it — see CITATION.cff.
GNU General Public License v3.0 — see LICENSE.