agent with skill

agent/deeptrack2-reference-to-code/SKILL.md

@@ -0,0 +1,207 @@
+---
+name: deeptrack2-reference-to-code
+description: Generate standard, runnable DeepTrack2 Python code from user descriptions or microscopy reference images, possible for training the machine learning model further.
+---
+
+# DeepTrack2 Reference-to-Code Skill
+
+## Goal
+
+Generate standard, runnable DeepTrack2 Python code from either:
+
+1. A user task description, or
+2. A microscopy reference image plus a desired synthetic-data goal.
+
+The output should approximate the visual style or imaging modality of the reference image while following DeepTrack2 code patterns, ready for machine learning application.
+
+## When to use this skill
+
+Use this skill when the user asks to:
+
+- generate DeepTrack2 code
+- simulate microscopy images
+- create synthetic microscopy datasets
+- match a microscopy modality from a reference image
+- produce DeepTrack2 examples from provided documentation or code examples
+- refactor or standardize DeepTrack2 code
+- prepare the simulation data for machine learning model
+
+## Source priority
+
+Follow this order:
+
+1. Local examples in `examples/`
+2. Docstring inside function/class
+3. Official DeepTrack2 documentation/example
+4. User-provided code or images
+
+Do not invent unsupported DeepTrack2 APIs.
+
+## Important library rules
+
+- Use `import deeptrack as dt`
+- Prefer complete runnable Python scripts
+- Use type hints where practical
+- Use `if __name__ == "__main__":`
+- Do not use private or undocumented APIs
+- Must wrapper new operators use `dt.Lambda`, read `/examples/Lambda.py` everytime creat scripts
+- Chaining as a pipeline using `>>`, `^`, and `&` as much as possible
+- DeepTrack2 2.0+ does not support TensorFlow
+- Do not generate TensorFlow/Keras-based DeepTrack code unless the user explicitly asks for legacy DeepTrack 1.7 code
+
+## General workflow
+
+1. Identify the user’s goal:
+   - particle simulation
+   - cell simulation
+   - brightfield image generation
+   - fluorescence image generation
+   - holography-like image generation
+   - dataset generation
+   - augmentation / noise modeling
+   - training-data pipeline
+
+2. Search local examples for the closest pattern.
+
+3. Extract:
+   - imports
+   - initialization style
+   - scatterer/sample construction
+   - optics/microscope construction
+   - noise and augmentation steps
+   - `.resolve()` or documented execution pattern
+
+4. Generate code in the same style as examples.
+
+5. Validate:
+   - every `dt.<name>` appears in local examples or docs
+   - no fake APIs
+   - no TensorFlow unless legacy requested
+   - code is complete and runnable
+
+## Reference-image workflow
+
+When the user provides a microscopy example image:
+
+1. Inspect the image visually.
+
+2. Infer likely modality:
+   - brightfield
+   - fluorescence
+   - holography
+   - phase contrast / DIC-like
+   - unknown
+
+3. Extract visual traits:
+   - background brightness
+   - object shape
+   - object density
+   - contrast polarity
+   - blur / PSF size
+   - noise type
+   - illumination gradient
+   - field of view
+   - artifacts
+
+4. Map traits to DeepTrack2 components:
+   - particles/cells -> scatterers
+   - optics -> Brightfield / Fluorescence / holography-related examples
+   - blur -> optics or postprocessing
+   - camera noise -> Poisson or Gaussian noise examples
+   - variability -> random parameter distributions
+   - normalization -> documented preprocessing or augmentation examples
+
+5. Generate a runnable DeepTrack2 script that creates visually similar synthetic images.
+
+6. Clearly mark inferred parameters as estimates.
+
+## Modality mapping
+
+### Brightfield-like
+
+Visual clues:
+
+- gray or bright background
+- darker objects
+- halos or soft shadows
+- uneven illumination possible
+
+Prefer examples involving:
+
+- particles or cells
+- brightfield optics
+- background variation
+- Gaussian noise
+- blur
+
+### Fluorescence-like
+
+Visual clues:
+
+- dark background
+- bright objects
+- glowing cells / spots
+- shot-noise appearance
+
+Prefer examples involving:
+
+- fluorescence optics
+- bright scatterers
+- Poisson noise
+- Gaussian blur
+- intensity variation
+- normalization
+
+### Holography-like
+
+Visual clues:
+
+- diffraction rings
+- interference fringes
+- phase-like contrast
+- central particle with halo
+
+Prefer examples involving:
+
+- holography examples
+- particle scatterers
+- coherent-style optical setup
+- ring-like artifacts
+- background correction
+
+## Output format
+
+Return:
+
+1. Inferred goal or modality
+2. Key visual/code assumptions
+3. Complete Python code
+4. Notes on parameters to calibrate
+5. Caveats if the API or modality is uncertain
+
+## Standard simulation code style
+
+Generated simualtion code should follow this structure:
+
+```python
+from __future__ import annotations
+
+import deeptrack as dt
+import matplotlib.pyplot as plt
+
+
+def build_pipeline():
+    ...
+
+
+def main() -> None:
+    pipeline = build_pipeline()
+    image = pipeline.resolve()
+
+    plt.imshow(image.squeeze(), cmap="gray")
+    plt.axis("off")
+    plt.show()
+
+
+if __name__ == "__main__":
+    main()

agent/deeptrack2-reference-to-code/examples/Brightfield.py

@@ -0,0 +1,56 @@
+import numpy as np
+import deeptrack as dt
+from matplotlib import pyplot as plt
+
+# ## Brightfield
+# 
+# In `DeepTrack2`, the brightfield microscope model simulates illumination using a single wavelength of light. This is equivalent to a brightfield microscope operating with a monochromatic light source.
+# 
+# In experimental setups however, a brightfield microscope typically uses a broad spectrum of wavelengths (white light).
+# 
+# To achieve a more realistic simulation of white light, we will need to simulate multiple brightfield images (sampling different parts of the visible light spectrum) and then averaging the contributions of these images. 
+# 
+# We start by generating a spectrum of wavelengths and a particle scatterer object. 
+
+IMAGE_SIZE = 150
+
+# Generate a spectrum of wavelengths to sample from, 5 will be enough.
+wavelengths = np.linspace(450e-9, 700e-9, 5)
+
+particle = dt.Sphere(
+    position=(IMAGE_SIZE//2, IMAGE_SIZE//4, 2) * dt.units.pixel, #put particles in the middle
+    radius= 0.3e-6,
+    position_unit="pixel",
+    refractive_index=1.42,
+)
+
+
+# Image the particle by sampling the spectrum.
+
+# Sample the wavelengths.
+imaged_particle_list = []
+for wavelength in wavelengths:
+    # Create a brightfield microscope for a given wavelength.
+    single_wavelength_optics = dt.Brightfield(
+        resolution=1e-6,
+        magnification=10,
+        wavelength=wavelength,
+        padding=(32, 32, 32, 32),
+        output_region=(0, 0, IMAGE_SIZE, IMAGE_SIZE//2),
+    )
+
+    # Image the particle.
+    imaged_particle = single_wavelength_optics(particle)
+
+    # Add background noise.
+    imaged_particle = imaged_particle >> dt.Gaussian(0, 0.01)
+
+    # Append to list.
+    imaged_particle_list.append(imaged_particle)
+
+# Take the average of the images in the list.
+brightfield_image = (
+    sum(imaged_particle_list) / len(imaged_particle_list)
+).resolve()
+plt.imshow(brightfield_image, cmap="gray")
+plt.show()

agent/deeptrack2-reference-to-code/examples/Darkfield.py

@@ -0,0 +1,63 @@
+import numpy as np
+import deeptrack as dt
+from matplotlib import pyplot as plt
+
+# ## Darkfield
+# We will image a particle in a darkfield modality. This requires the scatterer to be defined as a Mie scatterer object.
+# 
+
+IMAGE_SIZE = 150
+
+# Generate a spectrum of wavelengths to sample from, 5 will be enough.
+wavelengths = np.linspace(450e-9, 700e-9, 5)
+
+mie_scatterer = dt.MieSphere(
+    position=(IMAGE_SIZE//4, IMAGE_SIZE//2, 2) * dt.units.pixel, #put particles in the middle
+    radius= 2e-6,
+    position_unit="pixel",
+    refractive_index=1.42,
+    L=10,
+)
+
+
+# Then, we define our darkfield microscope and image the particle through this.
+
+
+optics = dt.Darkfield(
+    resolution=1e-6,
+    magnification=10,
+    wavelength=600e-9,
+    padding=(32, 32, 32, 32),
+    output_region=(0, 0, IMAGE_SIZE//2, IMAGE_SIZE),
+)
+
+image = optics(mie_scatterer)
+
+# Sample the wavelengths.
+imaged_particle_list = []
+for wavelength in wavelengths:
+
+    # Create a darkfield microscope for a given wavelength.
+    single_wavelength_optics = dt.Darkfield(
+        resolution=1e-6,
+        magnification=10,
+        wavelength=wavelength,
+        padding=(32, 32, 32, 32),
+        output_region=(0, 0, IMAGE_SIZE, IMAGE_SIZE),
+    )
+
+    # Image the particle.
+    imaged_particle = single_wavelength_optics(mie_scatterer)
+
+    # Add background noise.
+    imaged_particle = imaged_particle >> dt.Gaussian(0, 0.00015)
+
+    # Append to list.
+    imaged_particle_list.append(imaged_particle)
+
+# Take the average of the images in the list.
+darkfield_image = (
+    sum(imaged_particle_list) / len(imaged_particle_list)
+).resolve()
+plt.imshow(darkfield_image, cmap="gray")
+plt.show()

agent/deeptrack2-reference-to-code/examples/Fluorescence.py

@@ -0,0 +1,36 @@
+import numpy as np
+import deeptrack as dt
+from matplotlib import pyplot as plt
+
+# ## Fluorescence
+# Fluorescence images in `DeepTrack2` are simulated with discrete volumes to act as light sources (fluorophores) which emit light from a scatterer.
+# 
+# Simulating fluorescence images are easy and straightforward to implement, we start by defining our microscope.
+# Create a fluorescence microscope for a given wavelength.
+
+IMAGE_SIZE = 150
+
+particle = dt.Sphere(
+    position=(IMAGE_SIZE//8, IMAGE_SIZE//4, 2) * dt.units.pixel, #put particles in the middle
+    radius= 0.3e-6,
+    position_unit="pixel",
+    refractive_index=1.42,
+)
+
+fluorescence_optics = dt.Fluorescence(
+    NA=1.4,
+    resolution=1e-6,
+    magnification=10,
+    wavelength=600e-9,
+    padding=(32, 32, 32, 32),
+    output_region=(0, 0, IMAGE_SIZE//4, IMAGE_SIZE//2),
+)
+
+# Image the particle.
+imaged_particle = fluorescence_optics(particle)
+
+# Add background noise.
+fluorescence_image = (imaged_particle >> dt.Gaussian(0, 0.001)).resolve()
+
+plt.imshow(fluorescence_image, cmap="gray")
+plt.show()