Tiling Acquisition

This example demonstrates how to perform a 2D stage tiling acquisition centered on the current stage position and reconstruct a stitched panorama.

In the Inscoper API, a tiling acquisition is defined as a sequence of Status objects. Each Status represents a single tile of the 2D scan grid, mapping the active channel settings to specific stage X and Y coordinates. To align stage movements with the camera sensor, the stage calibration parameters (mirroring, scale, and angle) are applied to transform the grid offsets. When the sequence runs, the device controller drives the stage axes to each coordinate and triggers the camera to acquire the corresponding tile.

Prerequisites

This example builds on the device exploration pattern introduced in Explore Devices and the channel definition concepts from Define Channels. The device map is used to locate the stage controllers (X_AXIS, Y_AXIS, MOVE_AXIS) and active optical components.

Overview

The stage tiling acquisition sequence relies on the following sub-device types:

Sub-device type	Role
X_AXIS	Controls the horizontal axis (X position) of the microscope stage
Y_AXIS	Controls the vertical axis (Y position) of the microscope stage
MOVE_AXIS	Moves the microscope stage to a new position

The workflow executes the following steps in order:

1. Resolve the stage sub-devices: Identify the stage axis controller handles from the device map.
2. Start the camera: Prepare the camera to stream images using the SOFTWARE_TRIGGER mode.
3. Establish the reference channel: Load the desired channel settings.
4. Establish the reference position: Prompt the user to manually adjust the microscope stage to the desired center of the tiling area.
5. Define tiling grid slices:
   • Retrieve the camera dimensions and calculate the field of view (FOV) based on pixel size calibration.
   • Query stage step constraints using the API (getStepValue) to prevent axis step errors.
   • Compute target coordinate offsets in camera space, then apply the stage calibration matrix (mirroring, scale, and rotation) to map them to physical stage coordinate movements.
   • Generate a list of Status objects following a snake pattern grid to minimize stage travel time and distance.
6. Execute acquisition sequence: Load and run the tiling sequence to capture all tiles sequentially.
7. Retrieve and display panorama: Gather all captured images from the listener, stitch them together based on their calculated grid coordinates and overlap, and display the final reconstructed panorama.

Note

The stage calibration parameters (angle, mirror, and scale) used in step 5 can be determined automatically using the Tiling Calibration example.

This will result in a panorama like this:

Tiling Acquisition

Python

# ── Step 1: Resolve the sub-device IDs for the stage
x_axis_sub_device_id = None
y_axis_sub_device_id = None
move_xy_sub_device_id = None

for device_id, device in device_map.items():
    for sub_device_tag, sub_device in device.sub_device_map.items():
        if sub_device.inscoper_type == "X_AXIS":
            x_axis_sub_device_id = inscoper_api.SubDeviceId(device.name, sub_device.name)
        elif sub_device.inscoper_type == "Y_AXIS":
            y_axis_sub_device_id = inscoper_api.SubDeviceId(device.name, sub_device.name)
        elif sub_device.inscoper_type == "MOVE_AXIS":
            move_xy_sub_device_id = inscoper_api.SubDeviceId(device.name, sub_device.name)

print(f"X Axis: {x_axis_sub_device_id}")
print(f"Y Axis: {y_axis_sub_device_id}")
print(f"Move XY: {move_xy_sub_device_id}")


# ── Step 2: Start the camera in the correct mode
start_cam_status = inscoper_api.Status()
start_cam_status.addTag("MyCamera")
start_cam_status.addTag("SOFTWARE_TRIGGER")

my_bridge.loadSequence(0, "StartCamera", [start_cam_status])
my_bridge.runSequence(0, False)

# ── Step 3: Set the blue channel
my_bridge.loadSequence(1, "DeviceUpdate", [blue_status])
my_bridge.runSequence(1, False)


# ── Step 4: Wait for the user to adjust the X and Y position on the microscope
input("Please manually adjust the X and Y position on the microscope. Once done, press Enter to continue...")


# ── Step 5: Get current stage positions and define the tiling grid
assert x_axis_sub_device_id is not None, "No X axis sub-device was found. Check your configuration."
assert y_axis_sub_device_id is not None, "No Y axis sub-device was found. Check your configuration."

current_x = my_bridge.getLongValue(x_axis_sub_device_id)
current_y = my_bridge.getLongValue(y_axis_sub_device_id)
print(f"Current X position: {current_x} nm")
print(f"Current Y position: {current_y} nm")

# Control parameters for tiling acquisition
tiling_size = 3          # Tiling grid dimension (e.g. 3 for a 3x3 grid)
overlap = 0.10           # Overlap fraction between adjacent tiles (10%)
pixel_size_um = 0.2166     # Calibration: Pixel size in microns (microns per pixel)

# Stage calibration parameters (aligns stage movements with camera sensor)
stage_calib_mirror = True
stage_calib_scale = 0.99942
stage_calib_angle = 4.711666431527656  # in radians (~270 degrees)

# Get camera dimensions to compute the field of view (FOV)
my_camera = my_bridge.getCamera("MyCamera")
camera_width = my_camera.getImageWidth()
camera_height = my_camera.getImageHeight()
print(f"Camera frame dimensions: {camera_width}x{camera_height} pixels")

# Calculate FOV in nanometers
fov_x_nm = camera_width * pixel_size_um * 1000
fov_y_nm = camera_height * pixel_size_um * 1000
print(f"FOV size: {fov_x_nm/1000:.2f} um x {fov_y_nm/1000:.2f} um")

# Calculate step sizes in nanometers, taking overlap into account
step_x = fov_x_nm * (1.0 - overlap)
step_y = fov_y_nm * (1.0 - overlap)

# Read the step constraints from the API
try:
    x_step = my_bridge.getStepValue(x_axis_sub_device_id)
    if x_step <= 0:
        x_step = 1
except Exception:
    x_step = 1

try:
    y_step = my_bridge.getStepValue(y_axis_sub_device_id)
    if y_step <= 0:
        y_step = 1
except Exception:
    y_step = 1

print(f"X Axis Step Constraint: {x_step} nm")
print(f"Y Axis Step Constraint: {y_step} nm")

tiling_status = []
tile_grid_coords = []


# Generate a grid centered on (current_x, current_y)
for j in range(tiling_size):
    # Use a snake pattern to minimize stage travel distance
    cols = range(tiling_size - 1, -1, -1) if j % 2 != 0 else range(tiling_size)
    for i in cols:
        # Compute tile offset in camera coordinate system (in nanometers)
        offset_cam_x = (i - (tiling_size - 1) / 2.0) * step_x
        offset_cam_y = (j - (tiling_size - 1) / 2.0) * step_y

        # Apply mirroring to camera Y coordinate if calibrated
        offset_cam_y_prime = -offset_cam_y if stage_calib_mirror else offset_cam_y

        # Rotate and scale offsets to get stage coordinate offsets (nanometers)
        cos_a = np.cos(stage_calib_angle)
        sin_a = np.sin(stage_calib_angle)
        offset_stage_x = stage_calib_scale * (offset_cam_x * cos_a - offset_cam_y_prime * sin_a)
        offset_stage_y = stage_calib_scale * (offset_cam_x * sin_a + offset_cam_y_prime * cos_a)

        # Round to the nearest multiple of the step constraint
        target_x = int(round((current_x + offset_stage_x) / x_step)) * int(x_step)
        target_y = int(round((current_y + offset_stage_y) / y_step)) * int(y_step)

        tile_status = inscoper_api.Status()
        for sub_device_id, value in blue_channel_description.items():
            tile_status.addParam(sub_device_id, value)

        tile_status.addParam(x_axis_sub_device_id, target_x)
        tile_status.addParam(y_axis_sub_device_id, target_y)

        tile_status.addTag("MyCamera")
        tile_status.addTag("SOFTWARE_TRIGGER")

        tiling_status.append(tile_status)

        # Map loop indices (j, i) to display coordinates (row, col)
        tile_grid_coords.append((j, i))

# ── Step 6: Load and run the acquisition
my_bridge.loadSequence(2, "Acquisition", tiling_status)
my_bridge.runSequence(2, False)

# ── Step 7: Retrieve and display the images
received_images = my_listener.getReceivedImages()

if received_images:
    N = len(received_images)
    print(f"Received {N} images.")

    # Get image properties from the first received image
    first_image = received_images[0]
    W = first_image.getWidth()
    H = first_image.getHeight()
    pixel_size_bytes = first_image.getPixelSize()
    bit_depth = first_image.getBitDepth()
    bytes_per_value = bit_depth // 8
    values_per_pixel = pixel_size_bytes // bytes_per_value

    # Calculate step size in pixels for the panorama mapping
    step_x_px = int(round(W * (1.0 - overlap)))
    step_y_px = int(round(H * (1.0 - overlap)))

    # Total size of the panorama
    pano_width = (tiling_size - 1) * step_x_px + W
    pano_height = (tiling_size - 1) * step_y_px + H

    print(f"Stitching {N} tiles into a {pano_width}x{pano_height} pixel panorama...")

    # Initialize panorama and weight matrices (assuming single-channel / grayscale)
    pano = np.zeros((pano_height, pano_width), dtype=np.float32)
    weights = np.zeros((pano_height, pano_width), dtype=np.float32)

    for idx, image in enumerate(received_images):
        if idx >= len(tile_grid_coords):
            break
        j, i = tile_grid_coords[idx]
        print(f"Processing tile {idx + 1}/{N} at grid position ({j}, {i})...")

        image_data = image.getImageData()
        img = np.reshape(image_data, (H, W, values_per_pixel))
        img_channel = img[:, :, 0].astype(np.float32)

        # Place in panorama
        start_x = i * step_x_px
        start_y = j * step_y_px
        end_x = start_x + W
        end_y = start_y + H

        pano[start_y:end_y, start_x:end_x] += img_channel
        weights[start_y:end_y, start_x:end_x] += 1.0

    # Normalize and display the panorama
    print("Normalizing overlap regions...")
    # Avoid division by zero in case of unmapped pixels
    mask = weights > 0
    pano[mask] /= weights[mask]

    # Calculate the total field of view (FOV) in microns
    total_fov_x_um = pano_width * pixel_size_um
    total_fov_y_um = pano_height * pixel_size_um

    print("Displaying final stitched panorama...")
    fig_pano, ax_pano = plt.subplots(figsize=(8, 8))
    ax_pano.imshow(pano, cmap="gray")
    ax_pano.set_title(f"Convalaria: {total_fov_x_um:.1f}x{total_fov_y_um:.1f} um, Overlap: {overlap*100:.0f}%")
    ax_pano.axis("off")
    plt.tight_layout()
    #plt.savefig("tiling.png", bbox_inches="tight", dpi=300, transparent=True)
    plt.show()
else:
    print("No images received to plot.")