Project setup ============= This tutorial describes how to setup a SNUB project with video, neural activity recordings and behavioral annotations. The neural activity is used to generate a low-dimensional UMAP embedding, and we describe some of the user-interface tools that help link the embedding to single-neuron traces and to behavior. There are two example pipelines for calcium imaging and electrophysiology respectively. The input data and processed outputs from these tutorials are `available on Zenodo `_. Calcium Imaging --------------- Download the `example behavior and calcium imaging data `_. The final SNUB project generated in this tutorial is available `here `_. This tutorial is based on head-mounted 1-photon calcium imaging and video recordings from a mouse engaged in social interaction. The camera and microscope were synchronized and have associated timestamps in seconds. There are also behavior annotations for each frame generated by MoSeq. Load data ~~~~~~~~~ The ephys data consists of sorted spikes, including their timestamps ``spike_times`` and unit assignments ``spike_labels``. The units were filtered for quality, yielding a subset ``good_units`` for analysis in SNUB. .. code-block:: python import numpy as np # Load Z-scored calcium traces for each cell calcium_data = np.load('calcium_data.npy') # Load video timestamps video_timestamps = np.load('video_timestamps.npy') # Load mouse velocity behavior_annotations = np.load('behavior_annotations.npy') behavior_labels = open('behavior_labels.txt','r').read().split('\n') Create a SNUB project ~~~~~~~~~~~~~~~~~~~~~ .. code-block:: python import snub.io project_directory = 'ca2_imaging_project' snub.io.create_project( project_directory, start_time=video_timestamps.min(), end_time=video_timestamps.max()) Add IR video ~~~~~~~~~~~~ This experiment was originally filmed in 16bit monochrome. The ``.mp4`` file below was generated using :py:meth:`snub.io.transform_azure_ir_stream` .. code-block:: python video_path = 'ir_video.mp4' snub.io.add_video( project_directory, video_path, timestamps=video_timestamps, name='IR_camera') Add calcium traces ~~~~~~~~~~~~~~~~~~ .. code-block:: python snub.io.add_heatmap( project_directory, 'my_ca2_data', calcium_data, sort_method='rastermap', height_ratio=10, vmin=0, vmax=4, start_time=6.666, # the calcium recordings began after the IR video started binsize=1/15) # the head mounted miniscope recordes at 15 fps Add behavior annotations ~~~~~~~~~~~~~~~~~~~~~~~~ .. code-block:: python snub.io.add_heatmap( project_directory, 'behavior annotations', behavior_annotations, labels=behavior_labels, height_ratio=3, start_time=0.1, binsize=1/30) Add a UMAP plot of neural activity states ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. code-block:: python # bin the calcium data into 400ms bins prior to UMAP binned_calcium_data = snub.io.bin_data(calcium_data, 6) # bin the behavior annotations so we can plot them over the UMAP # also truncate so that they are aligned with the neural data start time behavior_truncated = behavior_annotations[:,200:-200] binned_behavior_annotations = snub.io.bin_data(behavior_truncated, 12) # check that truncation was correct - array sizes must have same # of columns print(binned_calcium_data.shape, binned_behavior_annotations.shape) coordinates = snub.io.umap_embedding( binned_calcium_data, n_pcs=10, n_neighbors=100) snub.io.add_scatter( project_directory, 'umap embedding', coordinates, binsize=0.5, start_time=6.666, pointsize=5, variables=binned_behavior_annotations.T, variable_labels=behavior_labels) Electrophysiology ----------------- Download the `example ephys and video data `_. The final SNUB project directory generated in this tutorial is available `here `_. This tutorial is based on electrophysiology and video recordings from a mouse behaving in an open field. The camera and ephys probe were synchronized and have associated timestamps in seconds. Load data ~~~~~~~~~ The ephys data consists of sorted spikes, including their timestamps ``spike_times`` and unit assignments ``spike_labels``. The units were filtered for quality, yielding a subset ``good_units`` for analysis in SNUB. .. code-block:: python import numpy as np # Load ephys data spike_times = np.load('spike_times.npy') spike_labels = np.load('spike_labels.npy') good_units = np.load('good_units.npy') # Load video timestamps video_timestamps = np.load('behavior_video_timestamps.npy') # Load mouse velocity mouse_velocity = np.load('mouse_velocity.npy') velocity_timestamps = np.load('mouse_velocity_timestamps.npy') Create a SNUB project ~~~~~~~~~~~~~~~~~~~~~ .. code-block:: python import snub.io project_directory = 'ephys_project' snub.io.create_project( project_directory, start_time=spike_times.min(), end_time=spike_times.max()) Add IR video ~~~~~~~~~~~~ This experiment was originally filmed in 16bit monochrome. The ``.mp4`` file below was generated using :py:meth:`snub.io.transform_azure_ir_stream` .. code-block:: python video_path = 'behavior_video.mp4' snub.io.add_video( project_directory, video_path, timestamps=video_timestamps, name='IR_camera') Add spike-sorted ephys data ~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. code-block:: python # remove all spikes with a label not in good_units good_spikes = np.in1d(spike_labels, good_units) spike_times = spike_times[good_spikes] spike_labels = spike_labels[good_spikes] # rename spike labels as consecutive integers renaming_dict = {old:new for new,old in enumerate(good_units)} spike_labels = np.array([renaming_dict[i] for i in spike_labels]) # combine spike times and labels into a single array spike_data = np.vstack((spike_times,spike_labels)).T snub.io.add_spikeplot( project_directory, 'my_ephys_data', spike_data, labels=[str(i) for i in good_units], sort_method='rastermap', height_ratio=10) Add a UMAP plot of neural activity states ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. code-block:: python # Generate UMAP coordinates using ephys firing rates # calculated from non-overlapping 100ms windows firing_rates, start_time = snub.io.firing_rates( spike_times, spike_labels, window_size=0.1, window_step=0.1) coordinates = snub.io.umap_embedding( firing_rates, min_dist=.01) snub.io.add_scatter( project_directory, 'umap embedding', coordinates, binsize=0.1, start_time=start_time) Add a plot of mouse velocity ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. code-block:: python traces = {'velocity': np.vstack((velocity_timestamps,mouse_velocity)).T} snub.io.add_traceplot( project_directory, 'velocity', traces, linewidth=2) Video Annotation ---------------- The code below shows how to set up a SNUB project for video annotation, e.g., for marking the intervals when one or more behaviors are occuring. .. code-block:: python import snub.io video_path = "path/to/my/video.mp4" labels = ["run", "rear", "groom"] project_directory = 'annotation_project' # create project directory video_duration = snub.io.generate_video_timestamps(video_path).max() snub.io.create_project( project_directory, duration=video_duration, layout_mode="rows" ) # add video snub.io.add_video(project_directory, video_path) # add annotation widget snub.io.add_annotator(project_directory, "my_annotator", labels=labels)